Ghost to-device receive path / Olm wedge self-heal #52

Closed
opened 2026-07-03 00:30:50 +00:00 by robocub · 4 comments
Collaborator

Summary

Give ghosts a to-device receive path so their Olm sessions with listeners can
self-heal when they wedge. Today ghosts are strictly send-only (#44), which means a
wedged ghost→listener Olm session can never recover on its own → that Discord speaker is
permanently silent for that listener until the ghost's device changes (respawn/restart).

Deferred by design (2026-07-02): the immediate root cause of observed wedging is stale
device accumulation (#34) + device-id churn (#35); hygiene removes the dominant trigger
while keeping ghosts light. This issue tracks the residual recovery mechanism — build it
only if wedging still occurs after hygiene lands.

Background: how a ghost session wedges

A 1:1 Olm session starts when the sender claims one of the receiver's single-use one-time
keys (OTKs) and sends a pre-key message. A send-only ghost never receives a reply, so
it keeps re-sending pre-key messages reusing the same OTK forever. If the listener ever
fails to build its inbound side (OTK already consumed/discarded, e.g. under OTK exhaustion
or a stale-device signature failure), it hits MissingOneTimeKey and can never rebuild
— every retry references the now-gone OTK. Element Call then marks the frame key invalid and
drops that publisher's audio.

Live-proven from a listener's Element Desktop crypto console (2026-07-02): repeated
Failed to create a new Olm session from a pre-key message: MissingOneTimeKey(...),
chain_index climbing 14→15→16, for one specific ghost while others were fine. Bridge logs
showed delivered=1 (the Olm PUT succeeded) — misleading; the listener couldn't decrypt it.

Why send-only can't self-heal

Matrix's standard recovery ("m.dummy" unwedge) is receiver-initiated: the receiver
builds a new session to the sender and sends an m.dummy; the sender adopts it (newest
session wins for outbound) — but only if the sender receives it. A send-only ghost never
does. Two hard limits (matrix-sdk-crypto 0.18 + spec):

  • No public send-side session-discard / force-new-session (store mutators are pub(crate)),
    so we can't rebuild a wedged session in place.
  • The peer won't even attempt an unwedge for a first-contact MissingOneTimeKey
    (gated on an existing >1h-old session — matrix-rust-sdk#3110). So a receive path repairs
    the later (>1h existing-session) wedge class, not the first-contact one (that's what
    hygiene #34 prevents).

Proposed architecture (light, for the hosted multi-tenant goal)

The receiver is OlmMachine::receive_sync_changes (public; feed it to-device events +
device-list/OTK counts). It auto-adopts a peer's new session for outbound. The question is
how to deliver to-device events to N in-memory ghost machines cheaply.

Preferred: single shared appservice to-device push (MSC2409). One HS→bridge stream for
ALL ghosts; demux each to-device event by target device_id → owning GhostCrypto
receive_sync_changes. Ghost cost stays O(1) in connections. Not buildable as-is — the
bridge has no inbound HTTP server, registration url is null, no hs_token handling,
matrix-sdk is built without appservice support, and Tuwunel's MSC2409 support is
unconfirmed. Work: add an inbound HTTP server (new dep), wire url+hs_token+the
de.sorunome.msc2409.push_ephemeral registration flag, and the device→ghost demux. First
step: confirm Tuwunel pushes to-device to appservices.

Fallback: per-ghost filtered /sync (to-device only, timeline disabled) on each ghost's
existing token. Trivial to add (reuses the outbound reqwest pattern) but re-imposes the
O(ghosts) /sync fan-out #44 deleted
— wrong for a light, multi-tenant service. Only
consider for small/self-hosted deployments.

Also on the table: send-only self-heal by periodic crypto-device rotation (fresh
device = fresh session; #34 prunes the old one) — keeps ghosts strictly send-only but is
blunt (re-establishes everything) and carries the #44 rejoin-churn risk.

Gate

Land device hygiene (#34) + stable device (#35) first, then measure whether wedging
still occurs. Build the appservice-push receive path only if it does. Related: #34, #35, #44.

## Summary Give ghosts a **to-device receive path** so their Olm sessions with listeners can **self-heal** when they wedge. Today ghosts are strictly send-only (#44), which means a wedged ghost→listener Olm session can never recover on its own → that Discord speaker is **permanently silent for that listener** until the ghost's device changes (respawn/restart). **Deferred by design** (2026-07-02): the immediate root cause of observed wedging is stale **device accumulation** (#34) + device-id churn (#35); hygiene removes the dominant trigger while keeping ghosts light. This issue tracks the *residual* recovery mechanism — build it only if wedging still occurs after hygiene lands. ## Background: how a ghost session wedges A 1:1 Olm session starts when the sender claims one of the receiver's single-use one-time keys (OTKs) and sends a *pre-key* message. A **send-only** ghost never receives a reply, so it keeps re-sending pre-key messages reusing the **same** OTK forever. If the listener ever fails to build its inbound side (OTK already consumed/discarded, e.g. under OTK exhaustion or a stale-device signature failure), it hits `MissingOneTimeKey` and can **never** rebuild — every retry references the now-gone OTK. Element Call then marks the frame key invalid and drops that publisher's audio. Live-proven from a listener's Element Desktop crypto console (2026-07-02): repeated `Failed to create a new Olm session from a pre-key message: MissingOneTimeKey(...)`, `chain_index` climbing 14→15→16, for one specific ghost while others were fine. Bridge logs showed `delivered=1` (the Olm PUT succeeded) — misleading; the listener couldn't decrypt it. ## Why send-only can't self-heal Matrix's standard recovery ("m.dummy" unwedge) is **receiver-initiated**: the receiver builds a new session to the sender and sends an `m.dummy`; the sender adopts it (newest session wins for outbound) — but only if the sender **receives** it. A send-only ghost never does. Two hard limits (matrix-sdk-crypto 0.18 + spec): - No public send-side session-discard / force-new-session (store mutators are `pub(crate)`), so we can't rebuild a wedged session in place. - The peer won't even *attempt* an unwedge for a **first-contact** `MissingOneTimeKey` (gated on an existing >1h-old session — matrix-rust-sdk#3110). So a receive path repairs the *later* (>1h existing-session) wedge class, **not** the first-contact one (that's what hygiene #34 prevents). ## Proposed architecture (light, for the hosted multi-tenant goal) The receiver is `OlmMachine::receive_sync_changes` (public; feed it to-device events + device-list/OTK counts). It auto-adopts a peer's new session for outbound. The question is how to *deliver* to-device events to N in-memory ghost machines cheaply. **Preferred: single shared appservice to-device push (MSC2409).** One HS→bridge stream for ALL ghosts; demux each to-device event by target `device_id` → owning `GhostCrypto` → `receive_sync_changes`. Ghost cost stays O(1) in connections. **Not buildable as-is** — the bridge has no inbound HTTP server, registration `url` is `null`, no `hs_token` handling, `matrix-sdk` is built without appservice support, and Tuwunel's MSC2409 support is unconfirmed. Work: add an inbound HTTP server (new dep), wire `url`+`hs_token`+the `de.sorunome.msc2409.push_ephemeral` registration flag, and the device→ghost demux. First step: **confirm Tuwunel pushes to-device to appservices.** **Fallback: per-ghost filtered `/sync`** (to-device only, timeline disabled) on each ghost's existing token. Trivial to add (reuses the outbound reqwest pattern) but **re-imposes the O(ghosts) `/sync` fan-out #44 deleted** — wrong for a light, multi-tenant service. Only consider for small/self-hosted deployments. **Also on the table:** send-only self-heal by periodic **crypto-device rotation** (fresh device = fresh session; #34 prunes the old one) — keeps ghosts strictly send-only but is blunt (re-establishes everything) and carries the #44 rejoin-churn risk. ## Gate Land device hygiene (#34) + stable device (#35) first, then **measure** whether wedging still occurs. Build the appservice-push receive path only if it does. Related: #34, #35, #44.
Author
Collaborator

Clean live reproduction tonight (2026-07-04), severity upgrade candidate.

During #53 live-testing the bridge process was restarted several times (each restart drops all ghosts' in-memory Olm sessions and establishes fresh ones on the next key send). Afterward, on the prod alpha.2 service, the listener (@dark, device ihjNlBVX90) could hear five of six Discord speakers — everyone except Ruffle's ghost (@nether-voicebridge_941211140774563870, crypto device Cwbc3t2wx, stable identity minted that evening).

Bridge side was proven clean end-to-end via the #51 fingerprint methodology:

  • capture healthy (speech RMS −12…−25 dB, zero capture/jitter/opus errors),
  • generated == frame-encrypt == sent key fp (b20e23b0) for every heartbeat,
  • every to-device send transport=olm delivered=1, zero warnings, no unreachable devices.

⇒ the listener's client held a dead inbound Olm session for exactly this one ghost and could not decrypt its key messages; delivered=1 cannot see that. There is no recovery: the ghost's send-only OlmMachine reuses its established outbound session indefinitely (a dead peer session is undetectable to it), and the client has no key-request path for to-device Olm.

Remediation that confirmed the diagnosis: restarting the bridge (fresh MemoryStore ⇒ fresh outbound session ⇒ new one-time-key claim) made Ruffle audible immediately (new key fp c2a7158c).

Design note for the fix: any periodic/opportunistic outbound-session rotation (e.g. rotate the Olm session on every media-key ratchet, or after N sends without any observable client activity) would bound the wedge duration without needing a receive path.

**Clean live reproduction tonight (2026-07-04), severity upgrade candidate.** During #53 live-testing the bridge process was restarted several times (each restart drops all ghosts' in-memory Olm sessions and establishes fresh ones on the next key send). Afterward, on the prod alpha.2 service, the listener (@dark, device `ihjNlBVX90`) could hear **five of six** Discord speakers — everyone except Ruffle's ghost (`@nether-voicebridge_941211140774563870`, crypto device `Cwbc3t2wx`, stable identity minted that evening). Bridge side was proven clean end-to-end via the #51 fingerprint methodology: - capture healthy (speech RMS −12…−25 dB, zero capture/jitter/opus errors), - `generated == frame-encrypt == sent` key fp (`b20e23b0`) for every heartbeat, - every to-device send `transport=olm delivered=1`, zero warnings, no unreachable devices. ⇒ the listener's client held a **dead inbound Olm session** for exactly this one ghost and could not decrypt its key messages; `delivered=1` cannot see that. There is no recovery: the ghost's send-only OlmMachine reuses its established outbound session indefinitely (a dead peer session is undetectable to it), and the client has no key-request path for to-device Olm. **Remediation that confirmed the diagnosis:** restarting the bridge (fresh MemoryStore ⇒ fresh outbound session ⇒ new one-time-key claim) made Ruffle audible immediately (new key fp `c2a7158c`). Design note for the fix: any periodic/opportunistic outbound-session rotation (e.g. rotate the Olm session on every media-key ratchet, or after N sends without any observable client activity) would bound the wedge duration without needing a receive path.
Author
Collaborator

Implemented on branch feat/issue-52-session-rotation (tip f26fa2e; unit-tested, NOT yet live-verified or merged): opportunistic outbound-session rotation.

Since matrix-sdk-crypto 0.18 has no public session-discard (store mutators are pub(crate), and the machine's internal session cache masks store-level deletion), rotation = rebuild the ghost's OlmMachine from its persisted account pickle with a fresh MemoryStore — exactly what a bridge restart does, which is the remediation that fixed the 2026-07-04 Ruffle wedge instantly. Same device id + identity keys (no SigningKeyChanged; the re-uploaded device keys are byte-identical), all sessions dropped, next send claims fresh OTKs.

Triggers: each membership change (rate-limited to 1/90s against EC membership flaps) + a 15-min max-age checked on the key heartbeat. Expected worst-case one-speaker silence: ~90s in an active call (a listener's rejoin now actually heals them), ~16 min in a fully static call — vs unbounded today. Config: rotate_ghost_sessions (bridge/server profile, default on, Olm transport only). Ghosts stay send-only, zero FDs (#44 preserved).


Research corrections to this issue's framing (verified 2026-07-04 from matrix-sdk-crypto 0.18 source, upstream trackers, and Tuwunel source):

  1. Wrong issue cited. The unwedge gating is matrix-rust-sdk #3427 (no unwedge when the receiver has NO stored session for the device — still open) and #3356 (session selection uses creation_time, ignores MSC1719), not #3110 (an unrelated multi-process cache bug, fixed 2024).
  2. The 1h UNWEDGING_INTERVAL is a throttle, not a wait. A listener unwedges immediately on failed decrypt if its newest stored session with the device is >1h old; it's blocked only for young pairs (<1h), and never fires with no stored session (#3427 — plausibly the exact Ruffle case: that ghost's crypto identity was freshly minted the same evening). Element Web/Desktop runs exactly this code (Rust crypto WASM; legacy removed matrix-js-sdk v37, 2025-02).
  3. The listener's client likely unwedges correctly and sends m.dummy — into the void. Our wedge persists because a send-only ghost discards the recovery message, not because of the gates. Sender-side never re-sessions on its own (element-meta#2356).
  4. The receive path is homeserver-blocked, more specifically than "MSC2409 unconfirmed": to-device push to appservices was split out of MSC2409 into MSC4203 (still open; only Synapse + hungryserv implement). Tuwunel's AS transaction push hardcodes to_device: Vec::new(), // TODO (src/service/sending/sender.rs:873, current git 2026-07-02); its spec-compliance campaign targets the published spec, so MSC4203 won't arrive as a side effect. MSC3202's transaction extensions (OTK counts / device-list changes) are also missing. The MSC2409 "70/70 partial" on Tuwunel's compliance page is the EDU half only (typing+receipts; not presence, not to-device).
  5. Conclusion: rotation covers precisely the recovery holes the protocol machinery leaves open (no-session + young-pair) and is the only self-heal available while ghosts are send-only on this homeserver. A future receive path (if Tuwunel grows MSC4203) would be complementary — event-driven healing in seconds — not a replacement; rotation would then relax to a lazy belt-and-braces bound.
**Implemented on branch `feat/issue-52-session-rotation` (tip `f26fa2e`; unit-tested, NOT yet live-verified or merged): opportunistic outbound-session rotation.** Since matrix-sdk-crypto 0.18 has no public session-discard (store mutators are `pub(crate)`, and the machine's internal session cache masks store-level deletion), rotation = rebuild the ghost's `OlmMachine` from its persisted account pickle with a fresh `MemoryStore` — exactly what a bridge restart does, which is the remediation that fixed the 2026-07-04 Ruffle wedge instantly. Same device id + identity keys (no `SigningKeyChanged`; the re-uploaded device keys are byte-identical), all sessions dropped, next send claims fresh OTKs. Triggers: each membership change (rate-limited to 1/90s against EC membership flaps) + a 15-min max-age checked on the key heartbeat. Expected worst-case one-speaker silence: **~90s in an active call (a listener's rejoin now actually heals them), ~16 min in a fully static call — vs unbounded today.** Config: `rotate_ghost_sessions` (bridge/server profile, default on, Olm transport only). Ghosts stay send-only, zero FDs (#44 preserved). --- **Research corrections to this issue's framing** (verified 2026-07-04 from matrix-sdk-crypto 0.18 source, upstream trackers, and Tuwunel source): 1. **Wrong issue cited.** The unwedge gating is matrix-rust-sdk **#3427** (no unwedge when the receiver has NO stored session for the device — still open) and **#3356** (session selection uses `creation_time`, ignores MSC1719), not #3110 (an unrelated multi-process cache bug, fixed 2024). 2. **The 1h `UNWEDGING_INTERVAL` is a throttle, not a wait.** A listener unwedges *immediately* on failed decrypt if its newest stored session with the device is >1h old; it's blocked only for young pairs (<1h), and *never* fires with no stored session (#3427 — plausibly the exact Ruffle case: that ghost's crypto identity was freshly minted the same evening). Element Web/Desktop runs exactly this code (Rust crypto WASM; legacy removed matrix-js-sdk v37, 2025-02). 3. **The listener's client likely unwedges correctly and sends `m.dummy` — into the void.** Our wedge persists because a send-only ghost discards the recovery message, not because of the gates. Sender-side never re-sessions on its own (element-meta#2356). 4. **The receive path is homeserver-blocked, more specifically than "MSC2409 unconfirmed":** to-device push to appservices was split out of MSC2409 into **MSC4203** (still open; only Synapse + hungryserv implement). Tuwunel's AS transaction push hardcodes `to_device: Vec::new(), // TODO` (`src/service/sending/sender.rs:873`, current git 2026-07-02); its spec-compliance campaign targets the *published* spec, so MSC4203 won't arrive as a side effect. MSC3202's transaction extensions (OTK counts / device-list changes) are also missing. The MSC2409 "70/70 partial" on Tuwunel's compliance page is the EDU half only (typing+receipts; not presence, not to-device). 5. **Conclusion:** rotation covers precisely the recovery holes the protocol machinery leaves open (no-session + young-pair) and is the only self-heal available while ghosts are send-only on this homeserver. A future receive path (if Tuwunel grows MSC4203) would be *complementary* — event-driven healing in seconds — not a replacement; rotation would then relax to a lazy belt-and-braces bound.
robocub referenced this issue from a commit 2026-07-04 02:18:53 +00:00
robocub referenced this issue from a commit 2026-07-05 01:24:09 +00:00
Author
Collaborator

Live verification on prod (2026-07-05, alpha.6 running): PASSED — closing.

Rotation fired during real encrypted calls on the Purple Voice bridge, exactly as designed:

  • 10 rotation events across two calls (02:00:57 and 03:31:24 UTC) — every ghost in the call, reason="membership", logged as rotated Olm outbound sessions (fresh machine, same device identity). Same-device-identity means no SigningKeyChanged exposure to peers.
  • Non-disruptive: each rotation was immediately followed by successful key delivery on the fresh sessions (delivered=1 to the listener's live device, burst 6→1 counting down normally, then the 60s heartbeat). Zero WARN/ERROR in either window; the only WARN anywhere is the normal ClientInitiated LiveKit disconnect at call end.
  • No failures: no "rotation failed" / fail-open lines anywhere in the journal since the alpha.4 deploy.
  • FDs flat: 338 open descriptors after all rotations — no per-rotation leakage from the OlmMachine rebuilds.

Not yet observed in the wild: the max-age (15 min) trigger — all encrypted calls since deploy were shorter than 15 minutes, so it legitimately never armed. The path shares the same rotate mechanism as the membership trigger (only the scheduling differs) and is covered by the rotation-policy unit tests; it will self-evidence in the journal during the next long encrypted call.

The membership trigger is the fast path that heals an active-call wedge (the Ruffle incident class) within ~90s of any join/leave, and it is now proven live end-to-end. The full receive-path self-heal remains blocked on Tuwunel gaining MSC4203 (to-device appservice push) — tracked in the issue body's research notes.

**Live verification on prod (2026-07-05, alpha.6 running): PASSED — closing.** Rotation fired during real encrypted calls on the Purple Voice bridge, exactly as designed: - **10 rotation events across two calls** (02:00:57 and 03:31:24 UTC) — every ghost in the call, `reason="membership"`, logged as `rotated Olm outbound sessions (fresh machine, same device identity)`. Same-device-identity means no `SigningKeyChanged` exposure to peers. - **Non-disruptive:** each rotation was immediately followed by successful key delivery on the fresh sessions (`delivered=1` to the listener's live device, burst 6→1 counting down normally, then the 60s heartbeat). Zero WARN/ERROR in either window; the only WARN anywhere is the normal `ClientInitiated` LiveKit disconnect at call end. - **No failures:** no "rotation failed" / fail-open lines anywhere in the journal since the alpha.4 deploy. - **FDs flat:** 338 open descriptors after all rotations — no per-rotation leakage from the OlmMachine rebuilds. Not yet observed in the wild: the **max-age (15 min) trigger** — all encrypted calls since deploy were shorter than 15 minutes, so it legitimately never armed. The path shares the same rotate mechanism as the membership trigger (only the scheduling differs) and is covered by the rotation-policy unit tests; it will self-evidence in the journal during the next long encrypted call. The membership trigger is the fast path that heals an active-call wedge (the Ruffle incident class) within ~90s of any join/leave, and it is now proven live end-to-end. The full receive-path self-heal remains blocked on Tuwunel gaining MSC4203 (to-device appservice push) — tracked in the issue body's research notes.
Author
Collaborator

Definitive live confirmation, ~1 hour after closing — a real wedge, healed by max-age rotation, witnessed end-to-end.

During a static encrypted call on Purple Voice (2026-07-05 ~03:37–03:53 UTC), the listener reported one Discord speaker (Spooki) inaudible while every other participant was fine — the classic single-ghost wedge:

  • Bridge side was provably clean: Spooki's ghost publishing, frames flowing, capture_errs=0, live RMS levels, and his ghost's key delivered=1 to the listener's device on every heartbeat. delivered=1 ≠ decryptable — dead inbound Olm session on the client, exactly the Ruffle incident class.
  • No membership churn occurred (static call), so the membership trigger never armed — this was the max-age path's case to prove.
  • At 03:53:03 (15 min after the ghosts' key loops started) all ghosts rotated reason="max-age", keys re-delivered on the fresh sessions ~200ms later.
  • Listener confirmed Spooki audible immediately after. No restart, no rejoin, no user action.

Also observed in the same call: a new Discord speaker joined at ~03:50 and its ghost's membership-triggered rotation fired at 03:52:38 — so both trigger paths are now proven against live traffic, and the max-age path is proven against a real wedge, not just a clean rotation.

This closes the last open verification item. The static-call worst-case heal time (~15–16 min) behaved exactly as designed; anyone hitting it sooner can leave+rejoin the call (membership rotation heals in seconds). Receive-path reactive healing remains blocked on Tuwunel MSC4203.

**Definitive live confirmation, ~1 hour after closing — a real wedge, healed by max-age rotation, witnessed end-to-end.** During a static encrypted call on Purple Voice (2026-07-05 ~03:37–03:53 UTC), the listener reported one Discord speaker (Spooki) inaudible while every other participant was fine — the classic single-ghost wedge: - Bridge side was provably clean: Spooki's ghost publishing, frames flowing, `capture_errs=0`, live RMS levels, and his ghost's key `delivered=1` to the listener's device on every heartbeat. `delivered=1 ≠ decryptable` — dead inbound Olm session on the client, exactly the Ruffle incident class. - No membership churn occurred (static call), so the membership trigger never armed — this was the max-age path's case to prove. - At **03:53:03** (15 min after the ghosts' key loops started) all ghosts rotated `reason="max-age"`, keys re-delivered on the fresh sessions ~200ms later. - **Listener confirmed Spooki audible immediately after.** No restart, no rejoin, no user action. Also observed in the same call: a new Discord speaker joined at ~03:50 and its ghost's membership-triggered rotation fired at 03:52:38 — so both trigger paths are now proven against live traffic, and the max-age path is proven against a *real* wedge, not just a clean rotation. This closes the last open verification item. The static-call worst-case heal time (~15–16 min) behaved exactly as designed; anyone hitting it sooner can leave+rejoin the call (membership rotation heals in seconds). Receive-path reactive healing remains blocked on Tuwunel MSC4203.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
dark/nether-voicebridge#52
No description provided.