Specification

1. Overview

Iconet is about presenting content from foreign sources in situation where a client is not able to present this content with native means.

This document and subdocuments linked here will guide you through the process of supporting interconnectivity. The walkthrough comprises the following steps:

To begin with, a packet is sent to a client that does not have the capabilities to display / present it to the user.
The packet however has iconet-(meta)data attached which the client can use as a fallback. The data required is described in section 2. Required Iconet Data for a Packet. The iconet data may also provide the packet in additional formats.
The iconet data includes information that allows the client to render a presentation from the packet (using fallback-iframes) and optionally translate the packet to different formats. Note, that packet content (dynamic & private) is intentionally separated from the fallback iframe data (static & public). See section 3. Interpreter Manifests.
The presentation is rendered (or translated) with a fallback-iframe. Fallback-iframes and the communication flow between the fallback-iframe and the embedding application, as well as sandboxing, is described in section 4. Fallback-iframes.
Last but not least - some packet types (e.g. polls) need support for interactions, to communicate back to sender of a packet (or even to third parties). Receiving updates to certain packets (e.g. read receipts or comments) are valid use cases as well. How this can be done is discussed in sections 4.2.4. Sending Interactions and 4.2.5. Receiving Updates / Interactions.

1.1 Introductory Notes

The format and schema of packets varies among protocols. To support an iconet fallback presentation (fallback-iframe), embedding applications and fallback iframe developers will however have to support a common set of standardized procedures.

What is standardized?

Communication between fallback-iframe and embedding application
Iconet-specific metadata
Interpreter Manifests (documents about the fallback iframes (i.e. interpreters) used to display or convert a packet)
Communicating interactions between embedding application with fallback-iframes

What is not standardized?

What packets look like that are to be presented to the user
What is communicated
How packets are transported from network A to B / What transport protocol is used.
How the iconet metadata is formatted in a given protocol packet

The data objects described here are JSON-LD formatted. If you are not familiar with JSON-LD, think of it as plain JSON with some fancy @context and @type fields. They allow the JSON keys to be globally uniquely identifiable.
Iconet’s JSON-LD @context namespace is: https://ns.iconet-foundation.org#

2. Required Iconet Data for a Packet

Say, a client received a foreign packet and cannot provide a presentation natively. However, the packet contains iconet data that the client knows how to deal with. This section describes the required iconet data. Note, that iconet metadata that describes how to interpret a packet is static. This makes it cacheable and reduces the privacy and security attack surface for the actual user-content.

The following example shows a representation of how the iconet metadata could be contained in a JSON-LD object. It holds minimal amount of data, since only the mandatory interpreter from native to html is given.

{
  "@context": "https://ns.iconet-foundation.org#",
  "@type": "Packet",
  "actor":"iconet:alice@alicenet.net",
  "to": "iconet:bob@bobnet.net",
  "interpreterManifests": [
    {
      "@id": "<e.g. https://app.example/interpreter/manifest>",
      "sourceTypes": ["<accepting packet source type, e.g. application/mastodon+json>"],
      "targetTypes":["<available output types, e.g. application/iconet+html or application/matrix+json>"],
      "sha512": "<sha512 hash of the manifest document linked>"
    }
  ],
  "content": [
    {
      "packetType": "<the type of the packet, e.g. application/mastodon+json>",
      "payload": "<data of the native packet>"
    },
    {
      "packetType": "<e.g. text/plain>",
      "payload": "<e.g. This message is a poll which your client does not support>"
    }
  ]
}

Note

It is up to a given protocol, how this metadata is actually contained in a packet. XML-based protocol designers may wish to use an XML-based representation over a JSON-based one, for example. When the packet crosses protocol borders however, it needs to be ensured to be formatted in JSON-LD.

2.1 Field Descriptions

Field Name	Type	Description
`@context`	`string\|object\|array`	The JSON-LD context namespace. This should be set to `https://ns.iconet-foundation.org#`. You can find more details here.
`@type`	`string`	The type of data, the packet contains. For regular payloads, this would be `Packet`. Depending on the context, the types `Interaction`, `TranslatedPacket`, (`Update`, `UpdateInquiry`) may be applicable.\|br\| The semantics for those packets are discussed in later sections.
`interpreterManifests`	`array` of interpreter manifest descriptions	An interpreter manifest contains a list of interpreter descriptions. Interpreters take a foreign protocol’s packet and either show a presentation or translate the packet to a different format.
`interpreterManifests[i] .@id`	`string`	The location of the manifest or a `data:` URI containing the interpreter manifest.
`interpreterManifests[i] .sourceTypes`	`array` of mime type `string`	A list of mime types or custom, application-specific types. For each input type in the list, the manifest must provide an interpreter accepting the given input type.
`interpreterManifests[i] .targetTypes`	`array` of mime type `string`	A list of mime types or custom, application-specific types. For each target type in the list, the manifest must provide an interpreter producing the given target type. Every packet must be able to find an interpreter with target `application/iconet+html` (mime type of fallback presentation-iframe).
`interpreterManifests[i] .sha512`	hex `string`	A sha512 hex signature of the interpreter manifest document. Tip: can be computed with `crypto.subtle.digest('sha512', data)` in javascript.
`content`	`array` of content records.	Each content record consists of fields that describe and hold the same data but in a different format. If a client does not support the first listed content record, it can go down the list. It is advisable to provide a plain text fallback as last item.
`content[i].packetType`	an extended mime type `string`	This field describes the type of the `payload` field content. This may be a general type like `text/plain` or `image/jpeg`, or it can be a non-standard application-specific mime type (e.g. application/matrix).
`content[i].payload`	`string`	This field contains the data of the packet sent. The type and format is not specified and needs to be interpreted by the application or an interpreter that is linked in the interpreter manifest. An implementing protocol will likely want to allow this field to be be undefined, if the wrapping packet (i.e. the packet that contains this iconet packet) is to be used as payload.

3. Interpreter Manifests

WIP-Level: 2

The interpreter manifests are JSON-documents that contain the metadata for fallback-iframes and translators. The manifests are linked by iconet-supporting packets and should be cached by the clients. When a client does not know how to present a packet to a user, it will fetch the manifest for a given source packet type and use a fallback presenter- or translator-iframe referenced in the interpreter manifest to present or translate the foreign packet.

The manifest format is standardized, in comparison to the metadata described in the section above (Required Iconet Data for a Packet). You can see an example below:

{
  "@context": "https://ns.iconet-foundation.org#",
  "@type": "InterpreterManifest",
  "@id": "<URI of this document>",
  "interpreters": [
    {
      "@id": "<URI to fallback-iframe>",
      "sourceType": "<(custom) mime type, e.g. application/activity+json>",
      "targetType": "<(custom) mime type application/matrix+json or application/iconet+html>",
      "sha512": "<sha512 hash of the linked document>",
      "permissions": {
        "<some permission>": "<value>"
      }
    }
  ]
}

If the targetType of a here given interpreter is application/iconet+html, the @id field must reference a HTML document that is a presenter iframe.
If the targetType of a here given interpreter is anything else, the @id field must reference a HTML document that is a translator iframe for that target type.
The @id field may have a data: URI value containing the HTML payload.
Permissions are described in 4.1.1 Fallback-iframe Permissions.

4. Fallback-iframes

Fallback-iframes build the foundation of presenting packets to the user or translating them. The interpreter manifest’s targetType must reference a fallback iframe HTML document. The HTML document must be embedded and sandboxed using the HTML iframe tag.
There are multiple methods described for embedding and sandboxing in section 4.1.3 Sandboxing Iframes.

This section describes the communication and encapsulation between embedding application and fallback-iframe. The embedding application passes a packet to the embedded fallback-iframe to display it, see 4.2 Communication between Embedding Application and Fallback-iframe for the walkthrough.

There are two types of fallback iframes: Translator Iframes and Presenter Iframes. Translator iframes simply return a packet translated to a different format when they are given a packet, as described in 4.2.3. Packet Translation Response. Presenter iframes instead render a HTML presentation. Users may interact with them and the iframes may trigger interaction packets back to the sender of the packet, as described in 4.2.4. Sending Interactions.

4.2 Communication between Embedding Application and Fallback-iframe

The communication flow to set up the communication looks as follows

The fallback-iframe initiates the communication, once loaded. It transfers a message port to the parent.
The parent receives the initiation request and responds with the iconet packet to render to the user.
For Translators: The iconet translator iframe responds with the translated packet. Communication has finished.
Optional: The iconet presenter iframe requests to send away an interaction packet.
Optional & at Discussion: The parent receives an additional packet that is passed on to the fallback-iframe.

4.2.1 Fallback-Iframe Ready

When the fallback-iframe is ready to receive packets, it calls parent.postMessage() (see reference) with targetOrigin of *.

The message parameter must look as follows:

{
  "@context": "https://ns.iconet-foundation.org#",
  "@type": "IframeReady"
}

The fallback-iframe MUST create a message channel and pass its message port using the transfer parameter of parent.postMessage().
The message channel must be used for all future communication.

The initiation in the fallback-iframe may look something like:

document.addEventListener("DOMContentLoaded", async() => {
  // Create a message channel for the future communication with the parent.
  const messageChannel = new MessageChannel();
  messageChannel.port1.onmessage = (messageEvent) => {
    console.info("Message received from parent!", messageEvent);
    // Handle incoming data and render a presentation.
  };

  // Send initial message to parent, transferring the message port.
  parent.postMessage({
      "@context": "https://ns.iconet-foundation.org#",
      "@type": "IframeReady"
    },
    "*",
    [messageChannel.port2]
  );
});

4.2.2 Parent Response

The parent, listening to the iframe’s message events, receives the message event and the transferred message port. Using the message port, the parent responds with the packet payload (event.ports[0].postMessage(packetPayload)).

The listener must validate that the iframe’s initial message comes from the expected source iframe.

See the code example for an embedding application listening to an iframe:

window.addEventListener("message", (event) => {
  // Validate event source.
  if (event.source !== targetIframe.contentWindow) {
    // Handle illegal message.
    return;
  }

  // The iframe passed a message port for further and secure communication.
  iframeMessagePort = event.ports[0];
  iframeMessagePort.onMessage =  (message) => {
    // Handle message from iframe here.
  };

  // Pass the received foreign iconet packet payload to the iframe.
  iframeMessagePort.postMessage(payloadForIframesMimeType);
});

The fallback iframe can now use the received payload (which was formatted in the sourceType specified by the interpreter manifest), to either render a presentation (if the targetType was application/iconet+html) or translate the packet to the according targetType otherwise.

4.2.3. Packet Translation Response

WIP-Level: 2

If the fallback-iframe is a translator iframe and has received the foreign packet, it responds with the translated packet as following:

{
  "@context": "https://ns.iconet-foundation.org#",
  "@type": "TranslatedPacket",
  "originalPacket": "<id of original packet>",
  "mimetype": "<translated packet's mimetype",
  "payload": "<translated packet>"
}

4.2.4. Receiving Updates / Sending Interactions

It is necessary to receive updates to a packet, e.g. to load the newest version of a post. Since creation of the post and the actual viewing by the recipient interactions and updates may have happened.

The updates are requested and interactions are sent by the fallback iframe through native APIs provided by the sending server.

In the manifest such native API’s need to be listed in allowedSources according to the Fallback-iframe Permissions.

Example:

"allowedSources": [
  "http://localhost:8001/api/getPost.php",
  "http://localhost:8001/api/addComment.php"
]

This leverages flexible capabilities for the fallback-iframe developer but may lead to a larger amount of (meta) data leakage to the whitelisted sources. In addition, the whitelisted sources should be approved by the embedding application itself and/or the user which might turn out to be challenging from a UX perspective.

5. Transport

So how do the packets reach their supposed destination? There are three ways:

5.1 Native Transport

The core use-case of iconet is, to provide a fallback for within networks, where connectivity is already established. In this case decentral applications with already common understandings of actors, relationships do share means of communication and security.

It is trivial for them, to also include the required fallback fields into their exchanged packets.

Here you see an example of a activitystream packet including the iconet information:

{
  "type":"Create",
  "@context":"https://www.w3.org/ns/activitystreams",
  "id":"https://bridge.localhost/b3f764ba-6343-419a-be42-a91580b7454b",
  "actor":"https://bridge.localhost/user/alice__neta.localhost",
  "object":{
    "type":"Note",
    "https://ns.iconet-foundation.org#iconet":{
      "@context":"https://ns.iconet-foundation.org#",
      "@type":"Packet",
      "@id":"https://bridge.localhost/9c94ca30-60e0-4951-aef2-e406309b0390",
      "actor":"alice__neta.localhost@bridge.localhost",
      "to":[
        "admin@localhost:3000"
      ],
      "interpreterManifests":[
        {
          "manifestUri":"http://neta.localhost/iconet/formats/markdown/manifest.json",
          "sourceTypes":[
            "text/markdown"
          ],
          "targetTypes":[
            "application/iconet+html"
          ],
          "sha-512":"empty"
        }
      ],
      "content":[
        {
          "packetType":"text/markdown",
          "payload":"## Title\nText `code`"
        }
      ],
      "XDEBUG_SESSION_START":"13228"
    },
    "@context":"https://www.w3.org/ns/activitystreams",
    "id":"https://bridge.localhost/4fb30562-6f2e-48f2-9e98-bb59b7f9e049",
    "published":"2023-02-26T14:21:36+00:00",
    "content":"This status contains iconet data.",
    "to":[
      "http://localhost:3000/users/admin",
      "https://www.w3.org/ns/activitystreams#Public"
    ]
  }
}

5.2 Fallback Transport

For communication between parties on different protocols, the information format, schema, authentication methods, API endpoints, transmission protocol, etc. might not align.

Different networks which have the same fallback mechanism established and manage to exchange the fallback information, can provide interconnectivity to their users.

That’s why it makes also sense, to have a fallback way for transport established.

5.2.1 Adressing & Routing

To send a message, the addressed receiver needs to be uniquely identifiable and locatable.

Users need to exchange addresses in URI format. If they do not have a common addressing protocol, they shall use the iconet schema:

iconet:<domain-specific identifier>@<dereferenceable address>

The dereferenceable address is the endpoint the message is sent to.
The domain-specific identifier is only required to be processed by the receiving server.

The iconet: URI scheme specifies the protocol to be used, i.e. makes clear that this is an iconet address.

As transport protocol HTTPS is used. As endpoint address we define <dereferenceable address>/iconet/, while in the future a port on the main address should be reserved for fallback transportation.

5.2.2 Encryption & Security

If the sending and receiving party do not share means for encryption and authentication, we recommend a fallback encryption method.

We use a hybrid cryptosystem similiar to PGP for our encryption porpuses.

The payload of each packet gets encrypted symmetrically with an AES-Secret. The secret get encrypted with each contacts public key. How each contacts public key is obtained and trusted, needs to be done according to the personal trust model.

Here is our example for an encrypted Packet:

{
  "@context": "https://ns.iconet-foundation.org#",
  "@type": "EncryptedPacket",
  "actor": "bob@netb.localhost",
  "to": "alice@neta.localhost",
  "EncryptedSecret": <AES-Secret, Encrypted with recipients public key>
  "EncryptedContent": <Manifestinfo & Packetcontent, Encrypted with AES-Secret>
}

5.3 Bridged Transport

Bridges are the hybrid solution between each native transport and a universal fallback transport.

While there are a couple of downside with bridges, they can be useful in the early adoption phase, when not many servers have adopted the fallback transport yet. This way more users can be reached with interconnectivity.

Where are bridges hosted?
- Case 1: Every individual user hosts its own bridge.
  Bridges are a risk for privacy, since traffic has to be decrypted before it can be bridged (and encrypted again). This issue can be avoided, when users host their own bridges on a trusted device for example on the client. Also, bridging remains easier since the bridge only needs to act as the user on the remote network (puppet bridge). Downside: Every user will have to host an own bridge.
- Case 2: Hosting a bridge per room or server.
  In this case, users don’t need individual bridges. However, all users will have to trust the bridge not to abuse its power to read and manipulate users messages. Additionally, many platforms do not support bridges that operate this way. In the worst-case a bridge bot that joined a room will post a message that describes the sender by appending its name to the message body.
(How) can we discover and reach out to users of different protocols?
Ideally, a user is able to find contacts on different platforms (e.g. by phone number or email address). Therefore, not only a cross-platform user index needs to be present but also an endpoint must be clear which the protocol can target to transfer a message across to a different protocol. The bridging endpoint needs to be added to each users address, so it can be forwarded from here. E.g.: alice@netA.org@bridge.net

There are different types of bridges to transfer packets across platforms. This matrix post discusses different types of bridges for the interested reader.