TCP bridge protocol
The TCP bridge is how a Mission Control System (MCS) exchanges live data with
Umoja during an access window: telemetry, telecommands, antenna pointing,
Doppler and SLE, all on one TLS connection, cut into small frames.
portal-link speaks it for you. This page is the contract for writing your
own client instead. The protocol is REMOS-BRIDGE/2.
When to use it
| You want | Use |
|---|---|
| An MCS (Yamcs, COSMOS/OpenC3, Gpredict) on plain local TCP ports, with nothing to write | portal-link |
| Live data in a browser or a web service | The realtime WebSocket |
| The live session inside your own software, without portal-link | This protocol |
Only the bridge carries external antenna control, external Doppler and SLE, whether through portal-link or your own client. The WebSocket has none of them.
Before you start
- An access token. Exchange your organisation's API client id
(
api-<company>) and secret for one with aclient_credentialsgrant athttps://umoja.remosspace.com/login/realms/umoja/protocol/openid-connect/token, as Getting a token shows. The answer'sexpires_inis the token's lifetime in seconds. - A User-Agent that names your client. The token endpoint and the API
refuse Python's default (
Python-urllib) with403anderror code: 1010, so set your own, as the reference client below does. - The station's UUID. The bridge does not take a station's short name.
Each access window carries the UUID as
station.id. - The access window's
id.GET /api/v1/access-windows?station_id=<uuid>lists your windows on the station with theirstarts_at,ends_atandtransmit_from(see Access windows). A booking names its window asaccess_window_id. - The right for each uplink you ask for.
tcandsleneed the transmit right on the station,ext_antennaneedsantenna.controlandext_dopplerneedsdoppler.control. The window'smay_transmit,may_control_antennaandmay_control_dopplersay whether you hold them. Downlink streams need nothing beyond the window. - Outbound TCP to port 10100. The bridge is raw TCP with TLS, not HTTP, so your firewall has to let it through directly rather than via an HTTP proxy.
Connecting
Open a TLS connection to bridge.umoja.remosspace.com, port 10100.
- Verify the certificate against your system's trusted CAs and check that it matches the host name, as any HTTPS client does. Never turn verification off: the first thing you send is your access token.
- Never pin the certificate or its key, and never connect by IP address. The certificate is short-lived and replaced every few days, so a pinned one stops working within days.
- Finish the TLS handshake within 10 seconds of connecting, or the hub closes the connection.
Signing in
Every line in the sign-in is UTF-8 and ends in \n. Read lines up to the
\n, not a fixed number of bytes, because the first line is not always the
banner.
- The hub sends the banner
REMOS-BRIDGE/2\n. It may sendAUTH_FAIL:server_full\norAUTH_FAIL:rate_limited\nin its place and close. - Within 10 seconds, send one line: a JSON object of at most 4096 bytes, newline included.
json
{"token": "<access token>", "station_id": "<station UUID>", "access_window_id": "<window id>", "streams": ["tm", "command", "tc"]}
| Field | Value |
|---|---|
token |
The access token, as a string, without Bearer. |
station_id |
The station's UUID. |
access_window_id |
The window's id. The session is bound to this window and no other. |
streams |
Stream names from Streams, at least one. Ask only for what you need: one you may not use refuses the whole session. |
- The hub answers
AUTH_OK\n, orAUTH_FAIL:<reason>\nand closes. It checks the envelope, then the token, then your rights and the window, and names the first problem it finds (see How to react to each refusal).
If you send nothing for 10 seconds, or close before the newline, the hub closes without an answer and counts a failed sign-in against your address.
Frames follow AUTH_OK at once, sometimes in the same TCP segment. Keep one
buffered reader for the whole connection, from the banner to the last frame.
A line reader thrown away after the sign-in can take the start of the first
frame with it.
Frames
After AUTH_OK both directions carry frames:
offset size field
0 4 length u32, little-endian: the stream id byte plus the payload
4 1 stream id
5 length - 1 payload
lengthis at most 65536, so a payload is at most 65535 bytes.- A length of
0is a heartbeat: four zero bytes, with no stream id and no payload. Both sides send them. - A length of
0xFFFFFFFF(FF FF FF FF) is REAUTH: you send it to hand over a new token (see Refreshing the token). The hub never sends it. - Any other length over 65536 from you ends the session with
protocol_error.
The connection is a byte stream. One read can return part of a frame, or
several frames, and TLS records do not line up with frames either. Read the
4-byte length, then exactly length more bytes, and repeat.
Each downlink frame holds one whole unit: one control event, one telecommand
ack, or one Expedite packet on tm, command, antenna, ephemeris,
doppler, ext_antenna and ext_doppler. sle is the exception: its
frames carry a byte stream (see SLE). portal-link's local ports are
different too: there, packets arrive back to back with no boundaries.
Streams
| Id | Name | Direction | Payload |
|---|---|---|---|
0x00 |
control | hub → you | One JSON event (see Control events). Always on; do not name it in streams. |
0x04 |
doppler |
hub → you | One predicted-Doppler packet (Expedite type 0x06). |
0x05 |
ephemeris |
hub → you | One predicted antenna-angle packet (type 0x03). |
0x06 |
antenna |
hub → you | One antenna-angle packet (type 0x03): where the rotator reports it is pointing. |
0x07 |
tm |
hub → you | One telemetry packet (type 0x00). |
0x08 |
tc |
both | Up: one telecommand. Down: its JSON ack. See Telecommands. |
0x09 |
command |
hub → you | One packet: a telecommand as the station handed it to the modem (type 0x01), or the modem's acknowledgement (type 0x02). |
0x0A |
sle |
both | [connection id: u16 big-endian][SLE bytes]. See SLE. |
0x0B |
ext_antenna |
both | One Expedite packet: type 0x04 up, the station's ACK (type 0x05) down. |
0x0C |
ext_doppler |
both | One Expedite packet: type 0x07 up, the station's ACK (type 0x08) down. |
Expedite packets shows how to unwrap
each packet type. The ext_* packets you send are laid out in
External antenna control and external Doppler.
You may send only on tc, sle, ext_antenna and ext_doppler, and only on
those you named in streams. The hub ignores any other frame you send, and
the tenth in one session ends it with protocol_error. Ignore frames on
stream ids you do not know.
Heartbeats and timeouts
| What | Limit |
|---|---|
| TLS handshake | Within 10 seconds of connecting. |
| Hub heartbeats | One every 30 seconds, whatever else it sends. |
| Your heartbeats | Send one every 30 seconds. The hub closes a connection it has read nothing from for 90 seconds, without an event. More than 100 heartbeats within 10 seconds ends the session with protocol_error. |
| Your read timeout | 90 seconds without a frame or heartbeat: treat the connection as dead and reconnect. |
| Envelope | Within 10 seconds of the banner. |
| A frame's body | Within 10 seconds of its length, or the hub closes without an event. |
| REAUTH line | Within 5 seconds of its length. Send both in one write. |
| Reading | Keep reading, and do slow work outside the read loop. The hub keeps a short backlog per stream and drops frames beyond it, and a frame it cannot hand to you for 30 seconds makes it reset the connection. |
Refreshing the token
An access token usually expires before the window ends. Fetch a new one at
about 80 % of its expires_in, and hand it to the hub on the open connection:
the REAUTH length followed by one JSON line, in one write.
FF FF FF FF {"token": "<new access token>"}\n
- Success: no reply. The session goes on under the new token.
- Failure: the hub sends
{"event": "session_closed", "reason": "reauth_failed"}and closes. The whole session ends, downlink included. A REAUTH fails when the line is late, is not a JSON object, or carries a token that is invalid, expired, or for another API client or organisation. Fetch a new token before you reconnect: the refused one would be refused again, asinvalid_token. - Not checked: if the hub cannot check the new token at that moment, it says nothing and the session goes on under the old token, as if no REAUTH had come.
Without a REAUTH the downlink keeps flowing, but from 60 seconds after the
token expires every uplink frame is refused with token_expired. If you see
token_expired although you sent a REAUTH, send one again with a fresh
token.
Control events
A control frame (stream 0x00) holds one JSON object with an event field.
Ignore events and fields you do not know.
access_ended
{"event": "access_ended", "reason": "window_ended"}
Your access to the window is over. It is the last frame: the hub closes the connection after it.
reason |
Meaning | What to do |
|---|---|---|
window_ended |
The window reached its ends_at, or was released. |
Move on to your next window. |
window_suspended |
The window was suspended, for example because a cancel was requested. | Watch the window with GET /api/v1/access-windows/{access_window_id}, and connect again if it resumes before its end. |
no_such_window |
The window was deleted, or is no longer your organisation's. | Move on to your next window. |
unknown_station |
Your organisation can no longer reach the station. | Stop, and contact Umoja support. |
uplink_refused
{"event": "uplink_refused", "stream": "ext_antenna", "reason": "window_not_open"}
The hub dropped ext_antenna or ext_doppler packets. The session stays
open. The event comes at most once every 5 seconds per stream, so it does not
say how many packets were dropped. reason is one of these, or one added
after this page was written:
invalid_packet: not one well-formed packet of the stream's type: its size, type or framing. The CRC and the values are checked only at the station (see External antenna and external Doppler).window_not_open: the station has not taken the window yet, or has put it back. Normal for about a second at the window's start.station_offline: the station's agent is not connected to the hub.token_expired: no fresh token reached the hub after the old one expired.window_ended,window_suspended,no_such_window,unknown_station: the window is over or no longer yours, for the reason access_ended gives. That event follows.unsupported_stream: the station's software no longer offers the stream.antenna_control_not_permitted,doppler_control_not_permitted: the right was withdrawn.
When an uplink right is withdrawn mid-session, the hub sends one
uplink_refused with the right's *_not_permitted reason for each stream
that needs it, tc and sle included, then session_closed with
uplink_revoked.
session_closed
{"event": "session_closed", "reason": "server_shutdown"}
The hub ended the session for a reason of its own. It is the last frame before the close.
reason |
Meaning | What to do |
|---|---|---|
reauth_failed |
The hub refused your REAUTH. | Fetch a new token, then reconnect. |
protocol_error |
The hub could not read what you sent: a length over 65536, ten frames on streams you may not send on, or more than 100 heartbeats within 10 seconds. | Fix your client. Do not reconnect in a loop. |
uplink_revoked |
A right one of your uplink streams needs was withdrawn. | Reconnect without that stream. With it, you get AUTH_FAIL:*_not_permitted. |
server_shutdown |
The hub is restarting. | Reconnect after about 10 seconds, and back off while connections fail. |
internal_error |
A fault in the hub ended the session. | Reconnect after about 10 seconds. |
| Any other | A reason added after this page was written. | Reconnect after about 10 seconds. |
There is no session_closed when the window ends (access_ended says so),
when you close, or when the hub drops a connection that went silent or
stopped reading. Treat a close with no event as a network drop: reconnect if
the window is still running.
Sending uplink
Telecommands
- One
tcframe is one telecommand, payload only. The station wraps it in an Expedite packet, so never send a packet you have wrapped yourself. The modem takes at most 16368 bytes. - Telecommands go up from the window's
transmit_from, which for a booked pass is AOS. Before then they are refused withuplink_not_open. - Each telecommand is answered on
tcwith one JSON ack:
json
{"ack": "ok", "cmd_id": "3f9a6c21d0b4"}
{"ack": "error", "cmd_id": "7c01e5aa9b32", "reason": "uplink_not_open"}
- The hub assigns
cmd_id; you cannot set it. Acks come back one per telecommand, in the order you sent them, so match them by order. - The hub handles one telecommand at a time. It reads nothing more from your connection until the station has answered, normally well within a second and at most about 10 seconds. Send the next telecommand once the previous ack has arrived.
okmeans the station handed the telecommand to its modem. The modem's own verdict follows oncommandas a type0x02packet.errorcarries areason. These codes mean the telecommand did not go up:uplink_not_open,window_not_open(the station has not taken the window yet, normally for about a second at its start: send again),station_offline,token_expired,window_ended,window_suspended,no_such_windowandunknown_station. Any other reason is a sentence, such asthe station did not answer in timeorthe station agent disconnected, and means the telecommand may or may not have gone up: check your spacecraft before you send it again. Show sentences as they are, and act only on the codes above.
External antenna and external Doppler
- One frame is exactly one complete Expedite packet: type
0x04(36 bytes) onext_antenna, type0x07(24 bytes) onext_doppler, with Expedite's own CRC. Any other size, type or framing is refused withinvalid_packet. - Both are taken from the window's start, once the station has taken the
window, not from
transmit_from. - The station's ACKs come back on the same stream, one packet per frame, from your first packet in the session on. They can repeat, so do not pair them one to one with what you sent.
- A packet the hub drops is reported with
uplink_refused. The hub does not check the CRC or the values: a packet with a bad CRC, or a value out of range (azimuth 0–360°, elevation −5–90° or the station's tighter limits, Doppler within ±5000 Hz), passes the hub and is dropped by the station with no ACK and nouplink_refused.
SLE
- Each payload is
[connection id: u16 big-endian][bytes]. The id is big-endian, unlike the frame length. - The bytes after the id are a byte stream, cut wherever the station's side read it: one frame can hold part of an SLE message, or several. Feed them to your SLE stack as one stream per id, in order, not as one message per frame.
- Give each connection your SLE user opens (one per service instance: RAF, RCF, FCLTU) its own id. The first bytes on a new id open a fresh connection to the station's SLE provider, and the bytes pass through untouched both ways, so SLE authentication and BIND parameters are what the provider expects.
- Send the id alone (a 2-byte payload) to close that connection. The hub sends the id alone when the station's side has closed: the provider closed it or could not be reached, or the hub refused your bytes, as it does for about a second at the window's start. Then BIND again on a new id.
- You receive the station's bytes, and its closes, only for ids you have sent on in this session.
- Never reuse an id for a new association, even after a close or a reconnect: bytes or a close from the old one may still be on their way and would land on the new one. Count ids up for the life of your program.
- SLE is taken from the window's start and needs the transmit right. No association outlives the bridge session.
Session lifecycle
- Connect at
starts_at. The hub goes by its own clock. Earlier, it answersAUTH_FAIL:window_not_open: wait untilstarts_at, then try again every 5 seconds. If that goes on paststarts_at, check your clock (NTP). - The lead-in. A booked pass's window opens a turnaround (five minutes by
default) before AOS.
sleandext_*are taken from the window's start once the station has taken the window, normally within a second.tcwaits fortransmit_from. - Only the window's data.
tmandcommandpackets whose time tag falls outside the window are not passed on, such as a station's leftover buffer from the pass before. The tag is judged on the hub's clock, allowing for the station clock's offset. Onsleandext_*you get only what answers your own traffic in this session. Other streams flow while the session is open. - An exact end. At
ends_atby the hub's clock you getaccess_endedwithwindow_ended, and the connection closes. Nothing on any stream that reaches the hub afterends_atis passed on. A window that is shortened, suspended or released ends the session at once. A window that is moved or extended is followed to its new end. - One client per exclusive stream. On each station, one session at a time
may hold
sle,ext_antennaandext_doppler. A second session for the same window that asks for one getsAUTH_FAIL:stream_busy:<name>. Up to your organisation's session limit (10, seeserver_full), sessions may sharetcand the downlink streams. A session left over from an earlier window never blocks the next window's. - Reconnecting mid-pass. After a drop, connect again with the same window
and streams. If you held an exclusive stream, your previous session keeps
it until the hub knows it is gone: at once if you closed the connection,
within 90 seconds if the connection died. Until then you get
stream_busy. You receive what arrives from then on. If the window records, what you missed is in its archive (see Replaying a session). - Every session needs a window, including one that asks only for
antenna. To watch the dish outside your windows, use the WebSocket'santennastream.
How to react to each refusal
"Counted" says which rate limit the refusal spends.
AUTH_FAIL: reason |
Why | Counted | What to do |
|---|---|---|---|
server_full |
In place of the banner: the bridge has as many sessions open as it takes. After the envelope: your organisation has as many sessions open, or signing in, as it may hold at once (10). | No | Try again after 60 seconds. If it is your organisation's limit, close sessions you no longer use: a connection that dropped without a close holds its place for up to 90 seconds. |
rate_limited |
In place of the banner: your address spent its limit. After the envelope: your organisation spent its limit. | No | Wait at least 60 seconds before each try. Find and fix the refusals that caused it. |
envelope_too_large |
The envelope line is over 4096 bytes. | Address | Stop and fix the client. |
invalid_json |
The envelope is not a JSON object in UTF-8. | Address | Stop and fix the client. |
no_streams, invalid_streams |
streams is missing, empty, or not a list of names. |
Address | Stop and fix the client. |
unknown_stream:<name> |
The hub knows no stream by that name. | Address | Stop and fix the name. |
missing_fields |
token, station_id or access_window_id is missing or empty. |
Address | Stop and fix the client. |
invalid_token |
The token is invalid or expired, or your API client is not set up with an organisation. | Address | Fetch a new token and try once more. If a freshly fetched token is refused, stop: check the client id, secret and token URL, and if they are right, contact Umoja support. |
auth_unavailable |
The hub could not check your token just now. The token may be fine. | No, unless the token is signed with a key the hub has not seen yet: Address | Try again after 60 seconds. |
invalid_station_id |
station_id is not a UUID, such as a station's short name. |
Organisation | Stop and send the UUID. |
transmit_not_permitted |
You asked for tc or sle without the transmit right on the station. |
Organisation | Stop. Drop the stream, or ask Umoja support for the right. |
antenna_control_not_permitted |
You asked for ext_antenna without antenna.control. |
Organisation | Stop. Drop the stream, or ask for the right. |
doppler_control_not_permitted |
You asked for ext_doppler without doppler.control. |
Organisation | Stop. Drop the stream, or ask for the right. |
unknown_station |
The station does not exist, or your organisation cannot reach it. | Organisation | Stop and check the station's UUID. |
no_such_window |
No window of yours with that id is on that station. | Organisation | Move on to the next window from the list. If a window the list shows is refused this way, check that station_id is that window's station.id. |
window_not_open |
The window starts later, by the hub's clock. | No | Try again after 5 seconds, or wait until starts_at if it is further off. |
window_ended |
The window has ended, or was released after its start. | Organisation | Move on to the next window. |
window_suspended |
The window is suspended (for example, a cancel was requested), or was released before its start. | Organisation | Do not retry the bridge in a loop. Read the window with GET /api/v1/access-windows/{access_window_id} every minute or so, connect when it is running again, and move on if it is released or ends. |
unsupported_stream:<name> |
The station's software does not offer that uplink stream. | Organisation | Stop and drop the stream. |
stream_busy:<name> |
Another session for this window holds sle, ext_antenna or ext_doppler on this station. |
No | Try again after 90 seconds, then every 2 minutes. After a drop it is usually your own previous session. If it does not clear, stop the other client. |
| Any other | A reason added after this page was written. | Maybe | Log it, and try again after 60 seconds. |
Rate limits
The hub counts refused sign-ins, so that a broken or looping client does not crowd out everyone else.
| Limit | What counts | Allowed | Once spent |
|---|---|---|---|
| Per source address | Sign-ins refused before your token is verified (the "Address" rows above), and connections closed or left silent before the envelope, during the TLS handshake included. Every client behind the address counts. | 5 in any 5 minutes | Up to 5 connections from the address in any 5 minutes get AUTH_FAIL:rate_limited in place of the banner. The hub closes any more at once, before TLS and without an answer. Both last until the oldest counted failure is 5 minutes old. |
| Per organisation | Sign-ins refused after your token is verified (the "Organisation" rows above). Other organisations behind your address are not affected. | 30 in any 5 minutes | Every sign-in by your organisation gets AUTH_FAIL:rate_limited right after its envelope. |
window_not_open, stream_busy, server_full, rate_limited itself and,
with the one exception its row gives, auth_unavailable are not counted.
Nor is a connection the hub closes because your address's limit is spent.
So waiting out a limit does not extend it.
Apart from these, at most 20 connections from one address may be signing in
at once, from the TCP connect to the hub's answer to the envelope. The hub
closes any more at once, before TLS and without an answer. And while the
bridge has as many sessions open as it takes, up to 5 connections from one
address in any 5 minutes get AUTH_FAIL:server_full in place of the banner,
and the hub closes any more at once in the same way. Neither is counted. To
stay under all of them:
- Stop on a refusal that retrying cannot fix. The table above says which.
- Fetch a new token before the old one expires, so
invalid_tokendoes not happen in normal running. - Back off between attempts, and connect when the window opens rather than in a tight loop before it.
- Do not check the port by opening and closing connections. Each one that closes before signing in counts as a failure, whether or not it finished the TLS handshake.
- Stop when the hub's certificate does not verify, rather than retry. Each failed TLS handshake counts against your address, so a client that retries one every minute keeps the address spent for every client behind it.
Forward compatibility
- Sign-in will change; framing will not. A future version will change how a connection signs in: the banner, the envelope and its answer. The frame format, the stream ids, heartbeats and control events stay as they are. Keep your sign-in code apart from your frame handling, so that only the sign-in has to change.
- Check the banner. Go on only after exactly
REMOS-BRIDGE/2\n, or handle theAUTH_FAILline that came in its place. Anything else is another protocol version: stop and report it rather than guess. - Expect new values.
AUTH_FAILreasons, control events, their reasons, ack reasons and stream ids are all open sets. Ignore events, fields and stream ids you do not know, and treat anAUTH_FAILreason you do not know as "try again after 60 seconds".
Reference client (Python)
A complete client in Python 3.10 or later, standard library only. It looks
the window up and waits for it to open, signs in, prints the size of each
telemetry packet, sends one telecommand from the window's transmit_from if
you give it one and prints its ack, hands the hub a fresh token before the old
one expires, reconnects after a drop with a wait that grows while connections
fail, and stops when the window ends. The telecommand is sent again only when
its ack says it did not go up. A reconnect never repeats one that may have.
export UMOJA_CLIENT_ID=api-<company> UMOJA_CLIENT_SECRET=<secret>
export UMOJA_STATION_ID=<station UUID> UMOJA_ACCESS_WINDOW_ID=<window id>
python3 bridge_client.py
Leave UMOJA_TC_HEX unset unless you mean to transmit: whatever it holds
goes up to your spacecraft.
#!/usr/bin/env python3
"""A minimal REMOS-BRIDGE/2 client. Python 3.10 or later, standard library only.
Set UMOJA_CLIENT_ID and UMOJA_CLIENT_SECRET (your API client), UMOJA_STATION_ID
(the station's UUID) and UMOJA_ACCESS_WINDOW_ID (the window's id). Set
UMOJA_TC_HEX to send that one telecommand from the window's transmit_from: it
asks for the tc stream, which needs the transmit right. Unset, the client only
listens.
"""
import asyncio
import json
import os
import random
import ssl
import struct
import urllib.error
import urllib.parse
import urllib.request
from datetime import datetime, timezone
BRIDGE_HOST = "bridge.umoja.remosspace.com"
BRIDGE_PORT = 10100
API_URL = "https://umoja.remosspace.com/api/v1"
TOKEN_URL = "https://umoja.remosspace.com/login/realms/umoja/protocol/openid-connect/token"
# The token endpoint and the API refuse Python's default User-Agent
# (Python-urllib) with 403, so the client names itself.
USER_AGENT = "umoja-bridge-client/1.0"
BANNER = b"REMOS-BRIDGE/2\n"
HEARTBEAT, REAUTH, MAX_LENGTH = 0, 0xFFFFFFFF, 65536
CONTROL, TM, TC, COMMAND = 0x00, 0x07, 0x08, 0x09
HEARTBEAT_SECONDS = 30 # the hub drops a client it has heard nothing from for 90 s
READ_TIMEOUT = 90 # the hub sends a heartbeat every 30 s
# AUTH_FAIL reasons that trying again cannot fix: the window is over or called
# off, or the configuration is wrong.
FINAL = {
"window_ended", "window_suspended", "no_such_window", "invalid_token",
"invalid_station_id", "unknown_station", "missing_fields", "invalid_json",
"envelope_too_large", "no_streams", "invalid_streams",
}
# Telecommand ack reasons that say it did not go up. Any other reason is a
# sentence, and may mean it did.
NOT_SENT = {
"uplink_not_open", "window_not_open", "station_offline", "token_expired",
"window_ended", "window_suspended", "no_such_window", "unknown_station",
}
class Refused(Exception):
"""The hub answered AUTH_FAIL:<reason>. The reason is the message."""
def retry_after(reason):
"""Seconds to wait before trying again after AUTH_FAIL:<reason>, or None to stop."""
if reason == "window_not_open":
return 5 # starts_at has passed by our clock, not yet by the hub's
if reason.startswith("stream_busy:"):
return 90 # usually our own previous session, not released yet
if (reason in FINAL or reason.endswith("_not_permitted")
or reason.startswith(("unknown_stream:", "unsupported_stream:"))):
return None
return 60 # server_full, rate_limited, and any reason this client does not know
class Backoff:
"""The wait before reconnecting: 10 s, doubled for each try in a row that
does not sign in, up to 120 s. Up to 5 s more at random, so that clients
cut off together do not all come back at the same moment."""
def __init__(self):
self.failures = 0
def delay(self):
wait = min(10 * 2 ** self.failures, 120)
self.failures += 1
return wait + random.uniform(0, 5)
def fetch_token():
"""A client_credentials access token, and its lifetime in seconds."""
form = urllib.parse.urlencode({
"grant_type": "client_credentials",
"client_id": os.environ["UMOJA_CLIENT_ID"],
"client_secret": os.environ["UMOJA_CLIENT_SECRET"],
}).encode()
request = urllib.request.Request(TOKEN_URL, data=form,
headers={"User-Agent": USER_AGENT})
with urllib.request.urlopen(request, timeout=10) as answer:
body = json.load(answer)
return body["access_token"], body["expires_in"]
def fetch_window(token):
"""The access window: the `data` of GET /api/v1/access-windows/{id}."""
request = urllib.request.Request(
f"{API_URL}/access-windows/{os.environ['UMOJA_ACCESS_WINDOW_ID']}",
headers={"Authorization": f"Bearer {token}", "User-Agent": USER_AGENT})
with urllib.request.urlopen(request, timeout=10) as answer:
return json.load(answer)["data"]
def seconds_until(timestamp):
"""Seconds from now until an API timestamp, 0 once it has passed or for none."""
if timestamp is None:
return 0
moment = datetime.fromisoformat(timestamp.replace("Z", "+00:00"))
return max((moment - datetime.now(timezone.utc)).total_seconds(), 0)
def frame(stream_id, payload):
return struct.pack("<IB", 1 + len(payload), stream_id) + payload
def unexpected(line):
"""The error for a sign-in line that is not the one we waited for."""
if line.startswith(b"AUTH_FAIL:"):
return Refused(line[len(b"AUTH_FAIL:"):].decode().strip())
if not line:
return ConnectionError("the hub closed the connection")
return RuntimeError(f"not a REMOS-BRIDGE/2 hub: {line!r}")
class Bridge:
"""One connection: one buffered reader for its whole life, one write at a time."""
def __init__(self, reader, writer):
self.reader, self.writer = reader, writer
self.lock = asyncio.Lock()
async def send(self, data):
async with self.lock: # whole frames only, never interleaved
self.writer.write(data)
await self.writer.drain()
# --- sign-in: this part changes in a future protocol version ---
@classmethod
async def sign_in(cls, token, streams):
# The default context checks the certificate and the host name.
reader, writer = await asyncio.wait_for(asyncio.open_connection(
BRIDGE_HOST, BRIDGE_PORT, ssl=ssl.create_default_context()), 15)
bridge = cls(reader, writer)
try:
line = await asyncio.wait_for(reader.readline(), 10)
if line != BANNER: # AUTH_FAIL:server_full or rate_limited come in its place
raise unexpected(line)
envelope = {"token": token, "station_id": os.environ["UMOJA_STATION_ID"],
"access_window_id": os.environ["UMOJA_ACCESS_WINDOW_ID"],
"streams": streams}
await bridge.send(json.dumps(envelope).encode() + b"\n")
line = await asyncio.wait_for(reader.readline(), 10)
if line != b"AUTH_OK\n":
raise unexpected(line)
except BaseException:
writer.close()
raise
# The same reader from here on: the first frame may already be in its buffer.
return bridge
# --- framing: this part stays ---
async def read_frame(self):
"""The next (stream id, payload). Heartbeats are skipped."""
while True:
header = await asyncio.wait_for(self.reader.readexactly(4), READ_TIMEOUT)
(length,) = struct.unpack("<I", header)
if length == HEARTBEAT:
continue
if length > MAX_LENGTH:
raise ConnectionError(f"bad frame length {length}")
body = await asyncio.wait_for(self.reader.readexactly(length), READ_TIMEOUT)
return body[0], body[1:]
async def heartbeats(self):
while True:
await asyncio.sleep(HEARTBEAT_SECONDS)
await self.send(struct.pack("<I", HEARTBEAT))
async def refresh_token(self, expires_in):
"""Hand the hub a fresh token (REAUTH) at 80 % of the current one's lifetime."""
delay = expires_in * 0.8
while True:
await asyncio.sleep(delay)
try:
token, expires_in = await asyncio.to_thread(fetch_token)
except OSError as error: # the token endpoint is unreachable: try again soon
print("token refresh failed:", error)
delay = 30
continue
line = json.dumps({"token": token}).encode() + b"\n"
# One write: the line has to follow its length within 5 s.
await self.send(struct.pack("<I", REAUTH) + line)
print("token refreshed")
delay = expires_in * 0.8
class Telecommand:
"""The telecommand from UMOJA_TC_HEX, sent from the window's transmit_from.
Sent again only after an ack that says it did not go up (NOT_SENT). Any
other ack, or none, may mean it did, so it is never sent again, after a
reconnect either: sent twice, it may run twice on the spacecraft.
"""
def __init__(self):
tc_hex = os.environ.get("UMOJA_TC_HEX")
self.payload = bytes.fromhex(tc_hex) if tc_hex else None
async def send(self, bridge, acks, transmit_from):
"""Send it, then print its ack. Send the next one only after the ack."""
while self.payload:
await asyncio.sleep(seconds_until(transmit_from)) # refused before then
payload, self.payload = self.payload, None # gone, unless the ack says otherwise
try:
await bridge.send(frame(TC, payload))
ack = await asyncio.wait_for(acks.get(), 30)
except BaseException: # the session ended, or no ack came
print("tc: no ack, so it may or may not have gone up. "
"Check the spacecraft before you send it again.")
raise
print("tc ack:", ack)
if ack.get("ack") == "error" and ack.get("reason") in NOT_SENT:
self.payload = payload # it did not go up: send it again shortly
await asyncio.sleep(5 if ack["reason"].endswith("_not_open") else 30)
async def session(token, expires_in, window, telecommand, backoff):
"""One session, from sign-in to its end. Returns the control event that ended it."""
streams = ["tm", "command"] + (["tc"] if telecommand.payload else [])
bridge = await Bridge.sign_in(token, streams)
backoff.failures = 0
print("signed in:", ", ".join(streams))
acks = asyncio.Queue()
tasks = [asyncio.create_task(bridge.heartbeats()),
asyncio.create_task(bridge.refresh_token(expires_in))]
if telecommand.payload:
transmit_from = window.get("transmit_from")
tasks.append(asyncio.create_task(telecommand.send(bridge, acks, transmit_from)))
try:
while True:
stream_id, payload = await bridge.read_frame()
if stream_id == TM:
print(f"tm: {len(payload)} bytes") # one Expedite packet
elif stream_id == COMMAND:
print(f"command: {len(payload)} bytes")
elif stream_id == TC:
acks.put_nowait(json.loads(payload))
elif stream_id == CONTROL:
event = json.loads(payload)
print("control:", event)
if event.get("event") in ("access_ended", "session_closed"):
return event # the last frame: the hub closes the connection next
# Any other stream id is one this client does not use: ignore it.
finally:
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
bridge.writer.close()
async def main():
telecommand, backoff = Telecommand(), Backoff()
while True:
try:
token, expires_in = await asyncio.to_thread(fetch_token)
window = await asyncio.to_thread(fetch_window, token)
wait = seconds_until(window["starts_at"])
if wait > 0: # the hub refuses a sign-in before starts_at
print(f"waiting {wait:.0f} s for the window to open")
await asyncio.sleep(min(wait, 300))
continue # then look again, with a new token: the window may have moved
event = await session(token, expires_in, window, telecommand, backoff)
except Refused as refused:
reason = str(refused)
wait = retry_after(reason)
print(f"AUTH_FAIL:{reason}" + ("" if wait is None else f", retrying in {wait} s"))
if wait is None:
return
await asyncio.sleep(wait)
continue
except (OSError, asyncio.IncompleteReadError, asyncio.TimeoutError) as error:
if isinstance(error, urllib.error.HTTPError) and error.code in (400, 401, 403, 404, 422):
raise # credentials refused, or no window of yours by that id
cause = error.reason if isinstance(error, urllib.error.URLError) else error
if isinstance(cause, ssl.SSLCertVerificationError):
raise # a certificate that does not verify, the hub's or the token endpoint's
wait = backoff.delay() # the hub or the token endpoint is down, busy or unreachable
print(f"connection lost: {error!r}, retrying in {wait:.0f} s")
await asyncio.sleep(wait)
continue
if event["event"] == "access_ended":
return # the window is over; a long-running client looks up its next one here
if event.get("reason") in ("protocol_error", "uplink_revoked"):
return # fix the client, or drop the uplink stream, before trying again
await asyncio.sleep(backoff.delay()) # server_shutdown, internal_error, reauth_failed, ...
if __name__ == "__main__":
asyncio.run(main())
A session with a telecommand on a booked pass prints something like this:
waiting 41 s for the window to open
signed in: tm, command, tc
tc ack: {'ack': 'ok', 'cmd_id': 'd8740fa92e2a'}
tm: 64 bytes
tm: 64 bytes
command: 29 bytes
token refreshed
control: {'event': 'access_ended', 'reason': 'window_ended'}
A long-running client adds what this one leaves out: finding the next window from the window list, waiting for a suspended window to resume, and taking the packets apart as Expedite packets shows.