Federation v2

Federation v2 lets one Chorus instance deliver signals and operate the tasks it created on another instance. It is disabled by default, has no v1 compatibility mode, and uses a separate authority module rather than Bearer identity authentication.

Enable the authority

FEDERATION_ENABLED=true
CHORUS_FEDERATION_PUBLIC_ORIGIN=https://chorus.example
CHORUS_FEDERATION_KEY_ENCRYPTION_KEY_FILE=/run/secrets/chorus_federation_key
CHORUS_FEDERATION_PROOF_TTL_SECONDS=60
CHORUS_FEDERATION_PREVIOUS_KEY_OVERLAP_SECONDS=300

CHORUS_FEDERATION_PUBLIC_ORIGIN is the exact public HTTP(S) origin whose authority is signed. It cannot contain credentials, a path, query, fragment, uppercase host text, or a redundant default port. HTTPS is required outside loopback.

The encryption-key file contains 32–4096 raw bytes; trailing CR/LF is trimmed. Chorus reads it once before opening the database and derives an AES-256-GCM key. Missing, unreadable, weak, or oversized files abort startup. Every replica must mount the same owner-only file.

Establish reciprocal trust

Both operators first create an active peer-org record for the other instance. Record ids are local, so exchange the two ids explicitly; never assume they match.

1. Instance A calls POST /org/company/peers/:id/establish with explicit actions, signal types, ring/identity scopes, maximum urgency, and lifetime. A returns key_id, generation 1, and a 32-byte base64url key_secret exactly once.
2. Transfer that response over an authenticated operator channel.
3. Instance B calls POST /org/company/peers/:id/install with the same policy plus key_id, key_generation: 1, and key_secret. B encrypts the key immediately and does not echo it.

Only AES-GCM ciphertext, IV, and a SHA-256 fingerprint are stored. The old peer_org.shared_secret field is not consulted by v2.

Rotation uses the same paired flow:

1. POST /org/company/peers/:id/rotate with expected_current_generation, lifetime, and optional overlap. Generation compare-and-swap prevents two administrators from silently winning.
2. Transfer the returned successor key once.
3. Call POST /org/company/peers/:id/rotate/install on the peer. The installed generation must be exactly the expected generation plus one.

During overlap, only the current and immediately previous generation verify. The previous key stops at the earlier of its expiry or verify_until; older generations are revoked. Revoked, future, expired, unknown, and overlap-expired keys fail closed. Revoking the current generation revokes the trust; rotation cannot resurrect it.

Signed request contract

Every request carries:

Header	Value
X-Federation-Version	exactly 2
X-Federation-Peer	the peer id assigned by the receiver
Content-Digest	RFC 9530 sha-256 digest of the exact transmitted bytes
X-Federation-Proof	strict JSON v2 proof envelope

The HMAC-SHA256 signature follows a fixed RFC-9421-inspired canonical form and binds all of:

• version, peer id, key id, and key generation;
• uppercase method and canonical public authority;
• raw encoded path and the exact query (? means absent);
• Content-Digest of the exact guarded byte sequence;
• issued-at, expiry, nonce, algorithm, and protocol tag.

The receiver compares the digest with the bytes admitted by the aggregate body guard before JSON parsing. Missing version, 1, 02, comma-joined versions, legacy proof shapes, unknown fields, or any bound-field change is rejected; there is no downgrade path.

Atomic replay protection and receipts

After proof verification and the post-proof peer rate limit, SurrealDB accepts the request in one transaction. A unique (peer_id, SHA-256(nonce)) claim, the requested mutation or durable rejection, provenance, and signed receipt commit together using database time. Raw nonces are not stored.

An exact retry after a lost acknowledgement returns the persisted receipt and does not repeat the mutation. Reusing the nonce with a different request id or digest returns FED_REPLAY_DETECTED. This invariant is shared across replicas and survives restarts.

Federated claims have one additional crash boundary. The lease and a provisional request row commit first without an accepted receipt. The acquisition and reconciliation transactions each write-fence the deterministic stripe for that signal; lease-rollout transitions mutate every stripe atomically. Reconciliation verifies the exact peer, signal, and lease generation are still current and atomically finalizes one signed accepted or rejected receipt. If the process stops in between, an exact retry resumes reconciliation; it never treats a provisional row as accepted. If quiescence won or the provisional lease expired, Chorus compensates only that exact generation back to open/pending work and persists a signed rejection (lease_acquisition_quiesced or resource_state_conflict). Concurrent exact retries compare-and-set the provisional outcome, so all callers replay the same finalized receipt.

The response receipt binds the request id/digest, peer, authority, key id/generation, outcome, result id, status, and issue time. A successful claim additionally binds the exact lease_generation into the signed receipt. The outbound client treats HTTP success as failure unless that receipt verifies and agrees with the HTTP status and response body, including the top-level claim generation.

Scope, ownership, and provenance

Trust policy is explicit. deliver checks action, signal type, target ring, optional target identity, and maximum urgency. claim, cancel, and ack work only on a signal whose durable federation_peer equals the authenticated peer. A different peer cannot operate it even if both peers share the same ring scope.

claim returns { result_id, lease_generation, receipt }. The peer must retain that generation and send it back on ack; cancellation of actively leased work must send it as well. A stale, expired, substituted, or unsigned generation cannot mutate a replacement lease. Exact claim retries return the same durable generation and receipt rather than acquiring again.

Inbound signals use fixed local role role:federation_v2. The remote principal remains an opaque string in dedicated federation provenance; it is never written into a local identity or role reference. The signal also records the peer, request, and key generation.

Outbound delivery

The admin JSON-RPC method federation/deliver requires:

• peer_id: your local peer-org record for the receiver;
• remote_peer_id: the id the receiver assigned to your instance;
• signal_type, content, and to_ring;
• optional to_identity, urgency, tags, and metadata;
• optional idempotency_key: 8–128 visible ASCII characters owned by the caller.

Use the same idempotency_key only when retrying the same delivery payload as the same authenticated identity and receiver peer. Chorus derives a domain-separated proof nonce scoped to that peer and principal. If the first response is lost after commit, a later call signs a fresh time-valid proof, returns the original signal with replayed: true, and receives a fresh signed receipt bound to the new proof. Reusing the key with changed content fails with FED_REPLAY_DETECTED; using a different key creates a separate delivery. Omitting the key preserves the legacy random-nonce behavior and therefore does not make caller retries idempotent.

The receiver stores proof-independent delivery content/intent digests and bounded replay counters. Pre-migration rows without those digests accept only an exact proof replay. claim, cancel, and ack also remain exact-proof-only. A currently valid rotated key can refresh a delivery receipt; revoked or expired proof authority is rejected before replay recovery.

The v2 service loads and decrypts only the current valid key, signs the exact serialized bytes, and sends through the shared safe outbound client. Peer origins are canonical origins; DNS rebinding, redirects, proxy inheritance, private/special-purpose destinations, oversized responses, and caller-controlled framing headers are blocked by the common egress policy.

Legacy migration

Migration 046-federation-v2.surql does not import v1 authority. It stores only the legacy secret hash/scopes/expiry in metadata.federation_v1_quarantine, erases the raw secret and v1 trust fields, and creates the v2 key/trust/request/receipt tables. Operators must establish v2 trust explicitly after upgrade. Legacy routes and JSON-RPC sending never fall back to the v1 secret.