Specification

WIP-Level: 2

1. Overview

Iconet is about presenting content from foreign sources in situation where a client is not able to present this content with native means.

This document and subdocuments linked here will guide you through the process of supporting interconnectivity. The walkthrough comprises the following steps:

  1. To begin with, a message is sent to a client that does not have the capabilities to display / present it to the user.

  2. The message however has iconet-(meta)data attached which the client can use as a fallback. The data required is described in section 2. Required Iconet Data for a Packet. The iconet data may provide the message in different format as well.

  3. The iconet data includes information that allows the client to render a presentation from the message (using iconet iframes) and optionally translate the message to different formats (TODO: Link translators). See section 3. Interpreter Manifests.

  4. The presentation is rendered (or translated) with an iconet iframe. Iconet iframes and the communication flow between the iconet iframe and the embedding application, as well as sandboxing, is described in section 4. Iconet Iframes.

  5. Last but not least - some messages (e.g. polls) need support for interactions, to communicate back to sender of a message (or even to third parties). Receiving updates to certain messages (e.g. read receipts or comments) are valid use cases as well. How this can be done is discussed in sections 4.2.4. Sending Interaction Packets and 4.2.5. Receiving Updates / Interactions <iframe-receiving-interactions>.

1.1 Introductory Notes

The format and schema of packets vary among protocols. To support an iconet fallback presentation (iconet iframe), applications will however have to support a common set of standardized metadata.

What is standardized?

  • Iconet-specific field names

  • Communication between iconet iframe and embedding application

  • Interpreter Manifests (document about the interpreters used to display or convert a packet)

  • (Communicating interaction made with iconet iframes)

What is not standardized?

  • What packets look like that are to be presented to the user

  • What is communicated

  • How packets are transported from network A to B / What transport protocol is used.

  • How the iconet metadata is formatted in a given protocol

The data objects described here are JSON-LD formatted. If you are not familiar with JSON-LD, think of it as plain JSON with some fancy @context and @type fields. They allow the JSON keys to be globally uniquely identifiable.
Iconet’s JSON-LD @context namespace is: https://iconet-foundation.org/ns#

2. Required Iconet Data for a Packet

WIP-Level: 2

A client received a foreign packet and cannot provide a presentation natively. However, the packet contains iconet data that the client knows how to deal with. This section describes the required data iconet data.

The following example shows a representation of how the iconet metadata could be contained in a JSON-LD object. The following packet format is not the same across protocols. It is up to a given protocol, how this metadata is actually contained in a packet. An XML-based protocol may wish to use an XML-based representation over a JSON-based one. The re-structuring of the metadata is thus the responsibility of the bridge when a packet crosses protocol borders.

{
    "@context": "https://iconet-foundation.org/ns#",
    "@type": "Packet",
    "interpreterManifests": [
        {
            "manifestUri": "<e.g. https://app.example/interpreter/manifest>",
            "inputTypes": ["<accepting packet input type, e.g. application/mastodon+json>"],
            "targetTypes":["<available output types, e.g. application/iconet+html or application/matrix+json>"],
            "sha-512": "<sha-512 hash of the manifest document linked>"
        }
    ],
    "content": [
        {
            "packetType": "<the type of the packet, e.g. Mastodon/ActivityPub>",
            "payload": "<data of the native packet>"
        },
        {
            "packetType": "<e.g. text/plain>",
            "payload": "<e.g. This message is a poll which your client does not support>"
        }
    ]
}

2.1 Field Descriptions

Field Name

Type

Description

@context

string|object|array

The JSON-LD context namespace. This should be set to https://iconet-foundation.org/ns#. For the beginning, this is not super relevant. You can find more details here.

@type

string

This will either be Packet, Interaction, TranslatedPacket, (Update, UpdateInquiry). Detailed descriptions can be found in the corresponding sections. TODO.

interpreterManifests

array of interpreter manifest descriptions

An interpreter manifest contains a list of interpreter descriptions. Interpreters take a foreign protocol’s packet and either show a presentation or translate the packet to a different format.

interpreterManifests[i] .manifestUri

string

The location of the manifest or a data: URI containing the interpreter manifest.

interpreterManifests[i] .inputTypes

array of mime type string

A list of mime types or custom, application-specific types.
For each input type in the list, the manifest must provide an interpreter accepting the given input type.

interpreterManifests[i] .targetTypes

array of mime type string

A list of mime types or custom, application-specific types.
For each target type in the list, the manifest must provide an interpreter producing the given target type.
Every packet must be able to find an interpreter with target application/iconet+html (mime type of iconet iframe).

interpreterManifests[i] ['sha-512']

array of mime type string

A sha-512 signature of the interpreter manifest document. Tip: can be computed with crypto.subtle.digest('sha-512', data) in javascript.

content

array of content records.

Each content record consists of fields that describe and hold the same data but in a different format. If a client does not support the first listed content record, it can go down the list. It is advisable to provide a plain text fallback.

content[i].packetType

an extended mime type string

This field describes the type of the payload field content. This may be a general type like text/plain or image/jpeg, or it can be a non-standard application-specific mime type (e.g. application/Matrix).

content[i].payload

any

This field contains the data of the packet sent. The type and format is not specified and needs to be interpreted by the application or an interpreter that is linked in the interpreter manifest.
The field may be undefined, if the wrapping packet (i.e. the packet that contains this iconet packet) is to be used as payload.

3. Interpreter Manifests

WIP-Level: 2

The interpreter manifests are JSON-documents that contain the metadata for iconet iframes and translators. The manifests are linked from iconet-supporting packet and should be cached by the clients. When a client does not know how to present a packet to a user, it will fetch the manifest for a given source packet type and use an iconet iframe or translator referenced in the interpreter manifest to present or translate the foreign packet.

The manifest format is standardized, in comparison to the data described in the section above (Required Iconet Data for a Packet). You can see an example below:

{
  "@context": "https://iconet-foundation.org/ns#",
  "@type": "InterpreterManifest",
  "@id": "<URI of this document>",
  "interpreters": [
    {
      "@id": "<URI to iconet iframe>",
      "sourceType": "<(custom) mime type, e.g. application/activity+json>",
      "targetType": "<(custom) mime type application/matrix+json or application/iconet+html>",
      "sha-512": "<sha-512 hash of the linked document>",
      "permissions": {
        "<some permission>": "<value>"
      }
    }
  ]
}

4. Iconet Iframes

Iconet iframes are the core of presenting packets to the user or translating them.
To maintain security and safety for users, it is important to sandbox them and restrict permissions. For this, see the section linked here:

4.2 Communication between Embedding Application and Iconet Iframe

The communication flow to set up the communication looks as follows

  1. The iconet iframe initiates the communication, once loaded. It transfers a message port to the parent.

  2. The parent receives the initiation request and responds with the iconet packet to render to the user.

  3. For Translators: The iconet translator iframe responds with the translated packet. Communication has finished.

  4. Optional: The iconet presenter iframe requests to send an interaction packet away.

  5. Optional & at Discussion: The parent receives an additional packet that is passed on to the iconet iframe.

4.2.1 Iconet Iframe Ready

WIP-Level: 1

When the iconet iframe is ready to receive messages, it calls parent.postMessage() (see reference). If the iconet iframe knows the parent origin, it will pass it as targetOrigin parameter to postMessage(). If not, it will use target origin *. The message will look as follows:

{
  "@context": "https://iconet-foundation.org/ns#",
  "@type": "IframeReady"
}

The iconet iframe MUST create a message channel and transfer a message port with the parent.postMessage() function. The message channel must be used for all future communication.

The initiation in the iconet iframe may look something like:

const messageChannel = new MessageChannel();

messageChannel.port1.onmessage = handleMessageFromParent;

// Send initial message to parent, transferring the message port.
parent.postMessage(
  {
    "@context": "https://iconet-foundation.org/ns#",
    "@type": "IframeReady"
  }
,
  [messageChannel.port2]
);

4.2.2 Parent Response

The parent, listening to its window’s message events, receives the message event and the transferred message port. Using the message port event.ports[0].postMessage() (or event.source.postMessage if not using message ports), the parent responds with the iconet packet. TODO: reference Iconet packet schema.

When listening to events sent to the parent window using window.postMessage(), the parent has to validate the origin of the event, if possible, and the source.

The listener may look something like this:

window.addEventListener("message", (event) => {
  // Validate event source.
  if (
    event.source !== targetIframe.contentWindow ||
    event.origin !== targetIframeOrigin
  ) {
    // Handle illegal message.
    return;
  }

  iframeMessagePort = event.ports[0];
  iframeMessagePort.onMessage = handleIframeMessage;

  // Pass the received foreign iconet packet to the iframe.
  iframeMessagePort.postMessage(
    {
      "@context": "https://iconet-foundation.org/ns#",
      "@type": "Packet",
      ".. more fields": "<more data>"
    }
  );
});

4.2.3. Packet Translation Response

WIP-Level: 2

If the iconet iframe is a translator, and receiveed the foreign packet, it responds a translated packet as following:

{
  "@context": "https://iconet-foundation.org/ns#",
  "@type": "TranslatedPacket",
  "originalPacket": "<id of original packet>",
  "mimetype": "<translated packet's mimetype",
  "payload": "<translated packet>"
}

4.2.4. Sending Interaction Packets

The iconet presenter iframe has two possibilities to send interactions. Either, the iconet iframe is allowed to establish connections itself, or the the interactions are communicated through a controlled message channel provided by the embedding application:

4.2.4.1 Iconet Iframe Establishes Connections Itself

The iconet iframe is allowed to communicate to certain whitelisted sources. Everything by the iconet iframe. This leverages flexible capabilities for the iconet iframe creator but may lead to a larger amount of (meta) data leakage to the whitelisted sources. In addition, the whitelisted sources need to be approved by client and/or user which might turn out to be challenging from a UX perspective. Iconet iframe permissions are discussed in SECTION XYZ.

4.2.4.2 Iconet Iframe Communicates Over its Embedding Application

The iconet iframe can communicate over a controlled channel provided by its embedding application. Packets sent over the controlled channel are sent back to the sender of the original packet. Since transport is handled by the embedding application, this reduces overhead for the iconet iframe creators and moves questions of signing and authenticating as well as encrypting packets there. Therefore, for end-to-end encrypted communication, this has the advantage of minimizing possible data-leakage to third-parties. On the other hand, it is less flexible.

The interaction packet is a JSON-LD-formatted object, the format is standardized:

{
  "@context": "https://iconet-foundation.org/ns#",
  "@type": "Interaction",
  "packetId": "<id of packet, preferably a UUID>",
  "timestamp": "<ISO 8601 timestamp>",
  "originalPacket": "<id of original packet>",
  "payload": "<custom data>"
}

4.2.5. Receiving Updates / Interactions

WIP-Level: 4

In some cases, it can be helpful to receive updates to a packet, e.g. to show read receipts or reactions. There are multiple methods in discussion.

For the pull-like method, the client would just ask the sender’s inbox for updates. The reply would be a new packet, potentially with new interactions attached. Details still need to be figured out, especially for encrypted contexts. This would require additional specification.

For the push-like method, the client would receive updates that are passed on to the iconet iframe. When the client receives the packet from the inbox, it will pass it on to the iconet iframe. Open Question: Does the iconet iframe maintain state or do all packets need to be replayed when the client re-opens the iconet iframe?