Home Original page

FBP Network Protocol

The Flow-Based Programming network protocol (FBP protocol) has been designed primarily for flow-based programming interfaces like the Flowhub to communicate with various FBP runtimes. However, it can also be utilized for communication between different runtimes, for example server-to-server or server-to-microcontroller.

Implementations

Clients

  • noflo-ui is an open source visual IDE, which powers Flowhub
  • fbp-spec is a data-driven testing tool for FBP components/graphs.
  • fbp-protocol-client is a low-level JavaScript client library supporting all the common FBP protocol transports
  • fbp-client is a higher-level JavaScript client library supporting all the common FBP protocol transports

Runtimes

  • noflo-runtime-base is a transport-independent implementation of the protocol for NoFlo, client-side and server-side JavaScript programming. WebSocket, WebRTC and iFrame implementations exists.
  • microflo is a WebSocket implementation of the protocol in JS/C++, for microcontrollers and embedded systems
  • Elixir FBP is a WebSocket implementation in Elixir, for programming with systems in Elixir runnin on the Erlang VM.
  • imgflo is a WebSocket implementation in C, for image processing
  • javafbp-runtime is a Websocket implementation in Java for JavaFBP, for JRE and Android development
  • sndflo is a WebSocket implementation for the SuperCollider audio programming environment
  • MsgFlo is a Websocket implementation in Node.js for heterogenous distributed systems communicating via message queues
  • rill is a Python FBP runtime implementation, with support for the protocol over WebSocket.

Some examples have also been created, to help implementors.

Test suite

The fbp-protocol tool provides a set of tests for FBP protocol implementations.

Changes

  • 2018-03-27:
    • Added schema for network:edges output message
    • Modified subgraph key of network:data and other network packet events to be an array as specified in the text
  • 2018-03-26:
    • Fixed documentation for component:setsource to use component:source input, and component:component output
    • Added schema for trace:error message
    • Added :error output to all capabilities where user can perform actions that may fail
  • 2018-03-23:
    • Added optional graph key to network:error payloads
  • 2018-03-22: Version 0.7
    • Added network:debug and network:getstatus to the network:control permission
  • 2018-03-21:
    • Fixed signature of runtime:packet.payload, runtime:packetsent.payload, and port definition default to accept any payload type
    • Added values and default keys for port definitions
    • Added schema for component:componentsready output message
    • Added schema for graph:clear output message
    • Added packetsent response for runtime:packet input message
  • 2017-09-17:
    • Added schema support for ports and packets
    • Documented known metadata keys for various graph entities
  • 2017-04-09: Version 0.6
    • Version 0.6. No breaking changes over 0.5.
    • Added additional capabilities graph:readonly, network:control, network:data, network:status. Especially useful for read-only access.
    • Deprecated the protocol:network capability in favor of the new fine-gained network:* capabilities.
    • Each capability now defines the set of messages contained in it. Available as inputs and outputs in the schema /shared/capabilities.
  • 2017-05-04:
    • Fixed protocol errors (graph:error, component:error and runtime:error) to have mandatory message string payload.
    • Fixed missing required markers in some JSON schemas for graph protocol. Affected messages: graph:renamegroup, graph:renameinport, graph:removeinport, graph:addinitial, graph:changeedge
    • More readable HTML output, including property value types and examples
  • 2017-05-03:
    • Added more optional metadata to runtime:runtime message: repository, repositoryVersion and namespace
  • 2017-02-20:
    • Fixed payload definition of network:edges missing mandatory graph key
  • 2016-07-01:
    • network:error payload may now contain an optional stacktrace
  • 2016-06-23:
    • Trace subprotocol also available in machine-readable format
  • 2016-06-17:
    • Protocol definition available as machine-readable JSON schemas.
    • The human-readable HTML documentation is generated from this defintion.
    • The npm package fbp-protocol contains the schemas as YAML, JSON and .js modules.
  • 2015-11-20:
    • Initial trace subprotocol, for Flowtrace support
  • 2015-03-27:
    • Documented network persist and component componentsready messages
  • 2015-03-26: Version 0.5
    • All messages sent to runtime should include the secret in payload
    • Runtime description message includes an allCapabilities array describing capabilities of the runtime, including ones not available to current user
  • 2014-10-23
    • added clarifications to network running state in status, started, and stopped messages
  • 2014-09-26
  • 2014-08-05:
    • Add get, list, graph, graphsdone commands to the graph protocol
    • Add list, network commands to the network protocol
  • 2014-07-15:
    • Add changenode, changeedge, addgroup, removegroup, renamegroup, changegroup commands to the graph protocol
  • 2014-03-13: Version 0.4
    • Capability discovery support
    • Network exported port messaging for remote subgraphs
  • 2014-02-18: Version 0.3
    • Support for exported graph ports
  • 2014-01-09: Version 0.2
    • Multi-graph support via the graph key in payload
    • Harmonization with JSON format by renaming from/to in edges to src/tgt
    • Network edges message

Basics

The FBP protocol is a message-based protocol that can be handled using various different transport mechanisms. The messages are designed to be independent, and not to form a request-response cycle in order to allow highly asynchronous operations and situations where multiple protocol clients talk with the same runtime.

There are currently three transports utilized commonly:

  • Web Messaging (postMessage) for communication between different web pages or WebWorkers running inside the same browser instance
  • WebSocket for communicating between a browser and a server, or between two server instances
  • WebRTC for peer-to-peer communications between a runtime and a client

Different transports can be utilized as needed. It could be interesting to implement the FBP protocol using MQTT, for instance.

Sub-protocols

The FBP protocol is divided into sub-protocols for each of the major resources that can be manipulated:

  • runtime: communications about runtime capabilities and its exported ports
  • graph: communications about graph changes
  • component: communications about available components and changes to them
  • network: communications related to running a FBP graph
  • trace: communications related to tracing a FBP network

Capabilities

Not all runtimes implementation supports all features of the protocol. Also, a runtime may restrict access to features, either to all clients based on configuration, or based on the secret provided in the messages. To support this a set of capabilities are defined, which are reported by the runtime in the runtime:runtime message.

When receiving a message, the runtime should check for the associated capability. If the capability is not supported, or the client does not have access to the capability, the runtime should respond with an error reply on the relevant protocol.

A few commands do not require any capabilities: the runtime info request/response (runtime:getruntime and runtime:runtime), and the error responses (runtime:error, graph:error, network:error, component:error).

protocol:network

can control and introspect its running networks using the Network protocol. DEPRECATED. Instead the individual network:... capabilities should be used. Implies capabilities network:status, network:data, network:control. Does not imply capability network:persist.

network:persist

can *flash* a running graph setup to the runtime, making it persistent across restarts

network:status

can get and follow changes to network status

network:data

can listen to data flowing through the network and processes

network:control

can start and stop networks

protocol:component

can list components of the runtime using the component:list message.

component:getsource

read and send component source code back to client

component:setsource

runtime is able to compile and run custom components sent as source code strings

protocol:runtime

can expose ports of main graph and transmit packet information to/from them

graph:readonly

read and follow changes to runtime graphs (but not modify)

protocol:graph

read & modify runtime graphs using the Graph protocol.

protocol:trace

runtime is able to record and send over flowtraces, used for retroactive debugging.

Message structure

This document describes all messages as the data structures that are passed. The way these are encoded depends on the transport being used. For example, with WebSockets all messages are encoded as stringified JSON.

All messages consist of three parts:

  • Sub-protocol identifier (graph, component, or network)
  • Topic (for example, addnode)
  • Message payload (typically a data structure specific to the sub-protocol and topic)

The keys listed in specific messages are for the message payload.

An example message

"protocol": "graph",
"command": "addnode",
"payload": {
	"component": "canvas/Draw",
	"graph": "hello-canvas-example",
	"id": "draw",
	"metadata": {
		"label": "Draw onto canvas element"
	}
}

Runtime protocol

When a client connects to a FBP procotol it may choose to discover the capabilities and other information about the runtime.

getruntime

Request the information about the runtime. When receiving this message the runtime should response with a runtime message. If the runtime is currently running a graph and it is able to speak the full Runtime protocol, it should follow up with a ports message.

packet

Runtimes that can be used as remote subgraphs (i.e. ones that have reported supporting the protocol:runtime capability) need to be able to receive and transmit information packets at their exposed ports. These packets can be send from the client to the runtimes input ports, or from runtimes output ports to the client.

  • "array"
  • "https://example.net/schemas/person.json"

error

Error response to a command on runtime protocol

ports

Message sent by the runtime to signal its available ports. The runtime is responsible for sending the up-to-date list of available ports back to client whenever it changes.

    • Each item contains:
    • "boolean"
    • "https://example.net/schemas/person.json"
    • true
    • false
    • Each item contains:
    • "boolean"
    • "https://example.net/schemas/person.json"
    • true
    • false

runtime

Response from the runtime to the getruntime request.

  • "f18a4924-9d4f-414d-a37c-cd24b39bba10"
  • "0.6"
  • "service-main"
  • "microflo"
  • "my-project-foo"
  • "https://github.com/flowbased/fbp-protocol.git"
  • "0.6.3-8-g90edcfc"

packetsent

Confirmation that a packet has been sent

  • "array"
  • "https://example.net/schemas/person.json"

Graph protocol

This protocol is utilized for communicating about graph changes in both directions.

clear

Initialize an empty graph.

addnode

Add node to a graph.

removenode

Remove a node from a graph.

renamenode

Change the ID of a node in the graph

changenode

Change the metadata associated to a node in the graph

addedge

Add an edge to the graph

removeedge

Remove an edge from the graph

changeedge

Change an edge's metadata

addinitial

Add an IIP to the graph

removeinitial

Remove an IIP from the graph

addinport

Add an exported inport to the graph.

removeinport

Remove an exported port from the graph

renameinport

Rename an exported port in the graph

addoutport

Add an exported outport to the graph.

removeoutport

Remove an exported port from the graph

renameoutport

Rename an exported port in the graph

addgroup

Add a group to the graph

removegroup

Remove a group from the graph

renamegroup

Rename a group in the graph.

changegroup

Change a group's metadata

Component protocol

Protocol for handling the component registry.

list

Request a list of currently available components. Will be responded with a set of `component` messages.

getsource

Request for the source code of a given component. Will be responded with a `source` message.

  • "my-project/SomeComponent"

source

Source code for a component. In cases where a runtime receives a `source` message, it should do whatever operations are needed for making that component available for graphs, including possible compilation.

  • "MyComponent"
  • "c++"
  • "components-common"

error

Error response to a command on component protocol

component

Transmit the metadata about a component instance.

  • "my-project/MyComponent"
    • Each item contains:
    • "boolean"
    • "https://example.net/schemas/person.json"
    • true
    • false
    • Each item contains:
    • "boolean"
    • "https://example.net/schemas/person.json"
    • true
    • false

componentsready

Indication that a component listing has finished

Network protocol

Protocol for starting and stopping FBP networks, and finding out about their state.

start

Start execution of a FBP network based on a given graph.

getstatus

Get the current status of the runtime. The runtime should respond with a status message.

stop

Stop execution of a FBP network based on a given graph.

persist

Tells the runtime to persist the current state of graphs and components so that they are available between restarts. Requires the network:persist capability.

debug

Set a network into debug mode

edges

List of edges user has selected for inspection in a user interface or debugger, sent from UI to a runtime.

    • Each item contains:

stopped

Inform that a given network has stopped.

started

Inform that a given network has been started.

status

Response to a getstatus message.

output

An output message from a running network, roughly similar to STDOUT output of a Unix process, or a line of console.log in JavaScript. Output can also be used for passing images from the runtime to the UI.'

error

An error from a running network, roughly similar to STDERR output of a Unix process, or a line of console.error in JavaScript.'

processerror

When in debug mode, a network can signal an error happening inside a process.

icon

Icon of a component instance has changed.

connect

Beginning of transmission on an edge.

begingroup

Beginning of a group (bracket IP) on an edge.

data

Data transmission on an edge.

endgroup

Ending of a group (bracket IP) on an edge.

disconnect

End of transmission on an edge.

Trace protocol

This protocol is utilized for triggering and transmitting [Flowtrace](https://github.com/flowbased/flowtrace)s

start

Enable/start tracing of a network.

stop

Stop/disable tracing of a network.

dump

undefined

clear

Clear current tracing buffer.

error

Error response to a command on trace protocol