Generic Backend Server
Version 2023-10-08, dd42e2e9f0ed9642
The latest version is available on viereck.ch.
Code and documentation may be used free of charge under the CC-BY-SA license.
UDP can be used to submit sensor data, access small documents, and get asynchronous notifications when a document changes.
The protocol targets low-bandwidth applications, e.g. a sensor submitting 1 kB of sensor data per minute. For such applications, it is much more efficient than HTTP/REST requests.
To use the UDP server, specify a UDP port in the configuration file:
The port number may be the same as for HTTP, which uses TCP as underlying protocol.
UDP packets have the following structure:
Every device accessing via UDP should use a unique 8-byte device ID. Every device ID is configured as account with the prefix UDP/.... The corresponding account key is used to encrypt the payload and produce the MAC.
Since the same key is used to encrypt all packets, every packet must use a different slice of the Ctr range, e.g. by picking a random Ctr value for each packet.
The same message format is used for the request and the reply packets. The server sets the very first bit of the Ctr value to 1, and the sender should set it to 0.
Malformed messages, or messages with a wrong MAC are silently ignored.
UDP packets are sent and forgotten, but may get lost or delayed in the network. The sender should therefore retransmit a request if it does not receive any response within a certain amount of time.
Note that either the request or the response packet may have gotten lost. The protocol is stateless, and can handle duplicates.
Requests a documents. The server sends the following response:
If the document does not exist, the revision is 0, and the value is empty.
Using offset and length, large documents can be retrieved using multiple requests.
Writes a document. The server sends the following response:
where type is either "W" (write succeeded), "C" (conflict, i.e. wrong derived from revision), "F" (forbidden, wrong write key), or "I" (invalid value).
Note that the maximum document size is limited by the maximum UDP packet size of about 65000 bytes (with packet fragmentation). Larger documents must be written via HTTP.
Listens for changes for about 2 minutes. Whenever the document revision changes during this time, the following response is sent:
If the document revision does not change during this time, no response is sent at all. The request expires after 2 minutes, and must be renewed.
Adds or updates a message. The server sends the following response:
where type is either "M" (message set), "N" (not an inbox).
Requests a blob. The server sends the following response:
If the blob does not exist, the slice is empty.
Adds or updates a message. The server sends the following response:
The token is the SHA256 sum of the bytes.
Note that the maximum blob size is limited by the maximum UDP packet size of about 65000 bytes (with packet fragmentation). Larger blobs must be written via HTTP.
Writes a chunk of (sensor) data to the data stream. The server confirms the chunk with the following response:
Every UDP device may send a data stream. To enable the data stream, add a data stream entry to the account configuration:
The data stream document may be empty.
Conceptually, a data stream is a virtual strip of 264 bytes, to which the UDP device may write arbintrary (binary) data. The UDP device may use any parts of the strip, although contiguous ranges yield better performance than random chunks here and there.
When receiving a chunk of data, the server writes a new blob, and links it to the data stream. The data stream document has the following structure:
As more chunks are coming in, the server condenses contiguous ranges to blobs of up to 10 MB:
Later chunks overwrite earlier chunks. The application processing the data may freely modify the data stream document.