perf(matrix-rtc): replace per-ghost full crypto Client with a lean send-only OlmMachine (MemoryStore) to retire FD exhaustion #44

Closed
opened 2026-06-24 23:00:25 +00:00 by robocub · 1 comment
Collaborator

Problem

In an encrypted call, each Discord speaker's ghost spins up a full matrix_sdk::Client purely to deliver its media key — own SQLite crypto store, a bounded initial sync_once, and a perpetual /sync loop (crates/matrix-rtc/src/rtc/ghost.rs:1260, build_ghost_crypto_client; sync spawned at :1322). A 20-speaker encrypted call therefore runs 20 crypto stores, 20 homeserver /sync long-polls, and ~40 long-lived tasks.

This is the real cause of the file-descriptor exhaustion seen under load (currently worked around with LimitNOFILE=65536). SqliteCryptoStore opens a deadpool connection pool sized max(2, physical_cores × 4) — ~32 connections per ghost store on an 8-core box, not the ~3 one might assume — so a couple dozen ghosts blow past the 1024 soft nofile limit, crypto sync dies, and those speakers go inaudible. The perpetual per-ghost sync is also an O(ghosts) long-poll fan-out on the homeserver.

Key observation: the ghost is send-only

The ghost crypto client never receives or decrypts anything — its /sync exists only to keep recipient device-lists fresh, and the sync results are discarded (ghost.rs:1314). Its entire job is to Olm-encrypt one media key to the in-call recipient devices (OlmTransport::send_key, key_transport.rs ~211–280; recipients enumerated from m.call.member by enumerate_key_recipients, ghost.rs:1480).

Constraint that bounds the fix

We cannot collapse all ghosts onto one shared crypto identity. Element Call attributes a received media key to the authenticated to-device sender (event.getSender()) and ignores the member.id in the payload — verified in matrix-js-sdk (ToDeviceKeyTransport/RTCEncryptionManager, released tag v41.8.0). So N distinct attributable E2EE participants require N distinct sending identities. We can make each identity cheap; we can't make them one.

Proposed change

Replace the per-ghost full Client with a standalone matrix-sdk-crypto::OlmMachine backed by an in-memory MemoryStore, driven by a small hand-rolled HTTP loop:

  • Upload device keys via outgoing_requests() → raw /keys/upload PUT → mark_request_as_sent(). (OTK upload is likely skippable — the ghost is a pure sender, never an Olm recipient; confirm.)
  • Track recipients on demand with update_tracked_users() + a /keys/query before a key send, instead of a perpetual /sync.
  • Establish Olm sessions via get_missing_sessions()/keys/claim.
  • Encrypt + send via encrypt_content_for_devices() (behind the experimental-send-custom-to-device feature the bridge already enables) → raw /sendToDevice PUT. This is exactly what encrypt_and_send_raw_to_device wraps today, so it's a lower-level rewiring of the same operation, not new crypto.

Touch points: build_ghost_crypto_client (ghost.rs:1260) and the Olm path in key_transport.rs. The distribute_key_loop burst+heartbeat policy (ghost.rs:1354) and recipient enumeration are unchanged.

What this removes

  • Perpetual per-ghost /sync task → replaced by on-demand /keys/query (removes the O(ghosts) homeserver fan-out).
  • All crypto file descriptors — MemoryStore uses zero, making FD cost independent of ghost count. This is the proper fix; LimitNOFILE=65536 is a stopgap.

What stays (inherent)

O(ghosts × in-call recipient devices) pairwise Olm sessions per call — bounded by call size, now in-memory and cheap. Same reason EC's own key distribution is O(n²).

Tradeoffs / decisions

  • Ghosts are ephemeral (alive only during a live call; a restart tears down all calls), so MemoryStore's "keys lost on drop" fits — nothing worth persisting. But a fresh in-memory account = a new device per ghost lifecycle → re-key on rejoin. Peer-safe per the #30 device-identity notes, but a deliberate change from today's persistent-store-reused-across-rejoins behaviour. (Decision: in-memory vs. a leaner persistent store.)
  • Going raw-OlmMachine means hand-rolling the HTTP/retry/lock logic Client gave for free (the SDK tutorial warns to lock around outgoing_requests to avoid duplicate sends). Pin the experimental feature flag.

Optional follow-up (separate)

A per-room shared device-tracker: every ghost currently queries the same recipient device-list independently; one shared /keys/query per room feeding all ghosts collapses device discovery from O(ghosts) to O(1) per room (Olm sessions stay per-ghost). Track separately if pursued.

Acceptance

  • Encrypted multi-speaker call still audible to all participants (correctness gate — everyone hears every ghost).
  • Open FD count stays flat as the number of concurrent encrypted ghosts grows (no per-ghost SQLite pool).
  • No perpetual per-ghost /sync; device-lists refreshed on demand.

Context and full analysis: docs/nether-voicebridge-public-instance-feasibility.md ("Scaling encrypted callers"), and the scaling discussion on #43.

## Problem In an **encrypted** call, each Discord speaker's ghost spins up a full `matrix_sdk::Client` purely to deliver its media key — own SQLite crypto store, a bounded initial `sync_once`, and a **perpetual `/sync` loop** (`crates/matrix-rtc/src/rtc/ghost.rs:1260`, `build_ghost_crypto_client`; sync spawned at `:1322`). A 20-speaker encrypted call therefore runs 20 crypto stores, 20 homeserver `/sync` long-polls, and ~40 long-lived tasks. This is the real cause of the file-descriptor exhaustion seen under load (currently worked around with `LimitNOFILE=65536`). `SqliteCryptoStore` opens a **deadpool connection pool sized `max(2, physical_cores × 4)`** — ~32 connections per ghost store on an 8-core box, not the ~3 one might assume — so a couple dozen ghosts blow past the 1024 soft `nofile` limit, crypto sync dies, and those speakers go inaudible. The perpetual per-ghost sync is also an O(ghosts) long-poll fan-out on the homeserver. ## Key observation: the ghost is send-only The ghost crypto client never receives or decrypts anything — its `/sync` exists *only* to keep recipient device-lists fresh, and the sync results are discarded (`ghost.rs:1314`). Its entire job is to Olm-encrypt one media key to the in-call recipient devices (`OlmTransport::send_key`, `key_transport.rs` ~211–280; recipients enumerated from `m.call.member` by `enumerate_key_recipients`, `ghost.rs:1480`). ## Constraint that bounds the fix We **cannot** collapse all ghosts onto one shared crypto identity. Element Call attributes a received media key to the authenticated to-device **sender** (`event.getSender()`) and ignores the `member.id` in the payload — verified in matrix-js-sdk (`ToDeviceKeyTransport`/`RTCEncryptionManager`, released tag v41.8.0). So N distinct attributable E2EE participants require N distinct sending identities. We can make each identity cheap; we can't make them one. ## Proposed change Replace the per-ghost full `Client` with a standalone `matrix-sdk-crypto::OlmMachine` backed by an in-memory `MemoryStore`, driven by a small hand-rolled HTTP loop: - **Upload device keys** via `outgoing_requests()` → raw `/keys/upload` PUT → `mark_request_as_sent()`. (OTK upload is likely skippable — the ghost is a pure sender, never an Olm recipient; confirm.) - **Track recipients on demand** with `update_tracked_users()` + a `/keys/query` *before a key send*, instead of a perpetual `/sync`. - **Establish Olm sessions** via `get_missing_sessions()` → `/keys/claim`. - **Encrypt + send** via `encrypt_content_for_devices()` (behind the `experimental-send-custom-to-device` feature the bridge **already** enables) → raw `/sendToDevice` PUT. This is exactly what `encrypt_and_send_raw_to_device` wraps today, so it's a lower-level rewiring of the same operation, not new crypto. Touch points: `build_ghost_crypto_client` (`ghost.rs:1260`) and the Olm path in `key_transport.rs`. The `distribute_key_loop` burst+heartbeat policy (`ghost.rs:1354`) and recipient enumeration are unchanged. ## What this removes - Perpetual per-ghost `/sync` task → replaced by on-demand `/keys/query` (removes the O(ghosts) homeserver fan-out). - **All** crypto file descriptors — `MemoryStore` uses zero, making FD cost independent of ghost count. This is the proper fix; `LimitNOFILE=65536` is a stopgap. ## What stays (inherent) O(ghosts × in-call recipient devices) pairwise Olm sessions per call — bounded by call size, now in-memory and cheap. Same reason EC's own key distribution is O(n²). ## Tradeoffs / decisions - Ghosts are **ephemeral** (alive only during a live call; a restart tears down all calls), so `MemoryStore`'s "keys lost on drop" fits — nothing worth persisting. But a fresh in-memory account = a **new device per ghost lifecycle** → re-key on rejoin. Peer-safe per the #30 device-identity notes, but a deliberate change from today's persistent-store-reused-across-rejoins behaviour. (Decision: in-memory vs. a leaner persistent store.) - Going raw-`OlmMachine` means hand-rolling the HTTP/retry/lock logic `Client` gave for free (the SDK tutorial warns to lock around `outgoing_requests` to avoid duplicate sends). Pin the experimental feature flag. ## Optional follow-up (separate) A per-room shared **device-tracker**: every ghost currently queries the *same* recipient device-list independently; one shared `/keys/query` per room feeding all ghosts collapses device *discovery* from O(ghosts) to O(1) per room (Olm sessions stay per-ghost). Track separately if pursued. ## Acceptance - Encrypted multi-speaker call still audible to all participants (correctness gate — everyone hears every ghost). - Open FD count stays flat as the number of concurrent encrypted ghosts grows (no per-ghost SQLite pool). - No perpetual per-ghost `/sync`; device-lists refreshed on demand. Context and full analysis: `docs/nether-voicebridge-public-instance-feasibility.md` ("Scaling encrypted callers"), and the scaling discussion on #43.
dark added this to the Roadmap project 2026-06-27 16:28:08 +00:00
robocub referenced this issue from a commit 2026-06-28 18:40:15 +00:00
Author
Collaborator

Shipped in v0.3.0-alpha.1 (master 00eff03). Each ghost's full matrix_sdk::Client is replaced by a lean send-only OlmMachine on a MemoryStore — no per-ghost SQLite crypto store, no per-ghost /sync loop. FD usage is now flat with speaker count (the alpha.1 verification runs showed no per-ghost FD growth), retiring the exhaustion this issue was filed for (the LimitNOFILE=65536 unit setting stays as belt-and-braces).

One regression was caught during verification and fixed before release: ghosts stopped re-keying a rejoining listener; fixed via the GhostCryptoCache (respawned ghosts reuse cached crypto + re-send keys).

Follow-ups spun out of this rework:

  • #50 — recipient enumeration still reads room state through the shared bot Client (deferred, low priority)
  • #51 — rejoin re-key latency (fixed in alpha.2, closing separately)
  • #55 — receive path / reactive re-key on client unwedge (blocked on Tuwunel MSC4203)

Closing.

**Shipped in v0.3.0-alpha.1** (master `00eff03`). Each ghost's full `matrix_sdk::Client` is replaced by a lean send-only `OlmMachine` on a `MemoryStore` — no per-ghost SQLite crypto store, no per-ghost `/sync` loop. FD usage is now flat with speaker count (the alpha.1 verification runs showed no per-ghost FD growth), retiring the exhaustion this issue was filed for (the `LimitNOFILE=65536` unit setting stays as belt-and-braces). One regression was caught during verification and fixed before release: ghosts stopped re-keying a *rejoining* listener; fixed via the `GhostCryptoCache` (respawned ghosts reuse cached crypto + re-send keys). Follow-ups spun out of this rework: - #50 — recipient enumeration still reads room state through the shared bot `Client` (deferred, low priority) - #51 — rejoin re-key latency (fixed in alpha.2, closing separately) - #55 — receive path / reactive re-key on client unwedge (blocked on Tuwunel MSC4203) Closing.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
dark/nether-voicebridge#44
No description provided.