nether/nether-voicebridge

Watch

Fork 1

rtc: Commet callers in encrypted rooms are silent toward Discord — their `io.element.call.encryption_keys` lacks `member.id`, the typed to-device handler drops it without a log #135

New issue

Closed

opened 2026-09-27 23:04:54 +00:00 by robocub · 1 comment

robocub commented

2026-09-27 23:04:54 +00:00

Member

Observed 2026-09-27 22:44–22:47 UTC, prod, Dark Voice (!kKwK98PeeZHpAGx8tx). @dark joined from "Phone - Commet" (device mZ3kATp88n, Commet's experimental encrypted-call support). They heard Discord; Discord heard nothing from them.

Evidence (prod journal)

Bridge subscribed to the track on each of three joins, then LiveKit: E2EE state changed identity=@dark:nether.im:mZ3kATp88n state=MissingKey — and it never left that state.
track subscribed before its peer media key — will re-subscribe when the key lands (#73) fired, but no received io.element.call.encryption_keys to-device and no installed peer media key line for the whole session (both are info!).
Ghost→phone keys: ghost media key sent … transport="olm" delivered=1 for all 7 ghosts — hence they could hear us.
Room state: bridge subscriber membership _@voicebridge_as:nether.im_dark-voice-9w6i5s4q_m.call and @dark's membership both well-formed; no key material posted as room events.
The receive path itself is healthy on this build: last EC key receipt on prod 2026-08-29 (bsch, @eezra:matrix.org, with member_id=<uuid>).

Root cause

Commet's key sender (commet/lib/client/matrix/components/voip_room/matrix_livekit_encryption_key_provider.dart, sendKeyToParticipants) emits

"member": {"claimed_device_id": "<device>"}

with no member.id. Our EncKeyMember (crates/matrix-rtc/src/rtc/keys.rs) declared id: String (required). matrix-sdk's typed add_event_handler skips the handler when content deserialization fails and logs only under matrix_sdk::event_handler=warn, which prod's RUST_LOG doesn't enable → no install, no log, one-way audio. Element Call / matrix-js-sdk do send member.id (a UUID), which is why EC users work. member.id is only logged by us — the peer's LiveKit identity is sender:claimed_device_id — so requiring it bought nothing.

Fix (branch `fix/commet-encryption-keys-member-id`)

member.id → Option<String> (#[serde(default)]); sent_ts gets #[serde(default)] too (js-sdk marks it sent_ts?). Wire shape we send is unchanged (id: Some("{device_id}_m.call")).
Interop tripwire: a catch-all AnyToDeviceEvent + RawEvent handler re-parses any io.element.call.encryption_keys event and WARNs with the content's field names (never values) when it doesn't fit our type — so the next client-shape gap is loud.
Unit tests: Commet-shaped content parses (id None), EC-shaped still parses, shape describer leaks no key bytes.

Residual / verification

Tuwunel prod logs record nothing for to-device sends at the current level, so "Commet actually emitted the event to the bridge device" is inferred from its source, not observed. After deploy, a Commet join should show received io.element.call.encryption_keys to-device … member_id=- followed by installed peer media key and the #73 heal line; if instead the new WARN fires, the shape diverges further; if neither fires, Commet didn't target our device (it only sends to devices in its userDeviceKeys cache).

Upstream note for Commet tracked in the companion issue.

**Observed 2026-09-27 22:44–22:47 UTC, prod, Dark Voice (`!kKwK98PeeZHpAGx8tx`).** @dark joined from "Phone - Commet" (device `mZ3kATp88n`, Commet's experimental encrypted-call support). They heard Discord; Discord heard nothing from them. ## Evidence (prod journal) - Bridge subscribed to the track on each of three joins, then `LiveKit: E2EE state changed identity=@dark:nether.im:mZ3kATp88n state=MissingKey` — and it never left that state. - `track subscribed before its peer media key — will re-subscribe when the key lands (#73)` fired, but no `received io.element.call.encryption_keys to-device` and no `installed peer media key` line for the whole session (both are `info!`). - Ghost→phone keys: `ghost media key sent … transport="olm" delivered=1` for all 7 ghosts — hence they could hear us. - Room state: bridge subscriber membership `_@voicebridge_as:nether.im_dark-voice-9w6i5s4q_m.call` and @dark's membership both well-formed; no key material posted as room events. - The receive path itself is healthy on this build: last EC key receipt on prod 2026-08-29 (bsch, @eezra:matrix.org, with `member_id=<uuid>`). ## Root cause Commet's key sender (`commet/lib/client/matrix/components/voip_room/matrix_livekit_encryption_key_provider.dart`, `sendKeyToParticipants`) emits ```json "member": {"claimed_device_id": "<device>"} ``` with **no `member.id`**. Our `EncKeyMember` (`crates/matrix-rtc/src/rtc/keys.rs`) declared `id: String` (required). matrix-sdk's typed `add_event_handler` skips the handler when content deserialization fails and logs only under `matrix_sdk::event_handler=warn`, which prod's `RUST_LOG` doesn't enable → **no install, no log, one-way audio.** Element Call / matrix-js-sdk *do* send `member.id` (a UUID), which is why EC users work. `member.id` is only logged by us — the peer's LiveKit identity is `sender:claimed_device_id` — so requiring it bought nothing. ## Fix (branch `fix/commet-encryption-keys-member-id`) 1. `member.id` → `Option<String>` (`#[serde(default)]`); `sent_ts` gets `#[serde(default)]` too (js-sdk marks it `sent_ts?`). Wire shape we *send* is unchanged (`id: Some("{device_id}_m.call")`). 2. Interop tripwire: a catch-all `AnyToDeviceEvent + RawEvent` handler re-parses any `io.element.call.encryption_keys` event and WARNs with the content's field *names* (never values) when it doesn't fit our type — so the next client-shape gap is loud. 3. Unit tests: Commet-shaped content parses (id None), EC-shaped still parses, shape describer leaks no key bytes. ## Residual / verification Tuwunel prod logs record nothing for to-device sends at the current level, so "Commet actually emitted the event to the bridge device" is inferred from its source, not observed. After deploy, a Commet join should show `received io.element.call.encryption_keys to-device … member_id=-` followed by `installed peer media key` and the #73 heal line; if instead the new WARN fires, the shape diverges further; if neither fires, Commet didn't target our device (it only sends to devices in its `userDeviceKeys` cache). Upstream note for Commet tracked in the companion issue.

robocub added the

bug

label

2026-09-27 23:04:54 +00:00

robocub referenced this issue

2026-09-27 23:04:54 +00:00

chore(upstream): report to commetchat/commet — encrypted-call io.element.call.encryption_keys omits member.id (and other interop notes vs matrix-js-sdk) #136

robocub referenced this issue from a commit

2026-09-27 23:09:19 +00:00

fix(rtc): accept encryption_keys to-device without member.id, warn on unparseable keys (#135)

robocub referenced this issue from a pull request that will close it,

2026-09-27 23:09:51 +00:00

fix(rtc): accept encryption_keys to-device without member.id; warn on unparseable keys (#135) #137

robocub closed this issue

2026-09-28 00:03:16 +00:00

robocub referenced this issue from a commit

2026-09-28 00:03:17 +00:00

Merge fix/commet-encryption-keys-member-id: accept encryption_keys to-device without member.id (Commet one-way audio, #135) + unparseable-key tripwire

robocub referenced this issue from a commit

2026-09-28 00:05:17 +00:00

release: 0.3.7 — Commet encrypted-call interop (member.id optional) + unparseable-key tripwire

robocub commented

2026-09-28 01:27:06 +00:00

Author

Member

Live-verified on prod 2026-09-28 01:25 UTC, v0.3.7 — operator joined Dark Voice from Commet (desktop device inMftF8sdw) and was heard on Discord.

Journal, Dark Voice bridge (new pid, no WARN/ERROR):

01:25:17.177  received io.element.call.encryption_keys to-device sender=@dark:nether.im claimed_device_id=inMftF8sdw member_id=- … key_index=1
01:25:17.177  installed peer media key peer_identity=@dark:nether.im:inMftF8sdw key_index=1 installed=true
01:25:17.928  received io.element.call.encryption_keys to-device … member_id=- … key_index=2
01:25:17.928  installed peer media key … key_index=2 installed=true
01:25:18.439  LiveKit: E2EE state changed identity=@dark:nether.im:inMftF8sdw state=Ok

member_id=- = the Commet event with no member.id, now accepted. A ~3 s MissingKey→Ok blip at 01:25:19–21 matches Commet's membership-triggered key rotation (new index used after its fixed 5 s delay) — self-healed, noted in #136. The tripwire WARN did not fire, so Commet's shape is otherwise within our type.

Shipped in v0.3.7 (prod-deployed 01:02 UTC). Closing.

**Live-verified on prod 2026-09-28 01:25 UTC, v0.3.7** — operator joined Dark Voice from Commet (desktop device `inMftF8sdw`) and was heard on Discord. Journal, Dark Voice bridge (new pid, no WARN/ERROR): ``` 01:25:17.177 received io.element.call.encryption_keys to-device sender=@dark:nether.im claimed_device_id=inMftF8sdw member_id=- … key_index=1 01:25:17.177 installed peer media key peer_identity=@dark:nether.im:inMftF8sdw key_index=1 installed=true 01:25:17.928 received io.element.call.encryption_keys to-device … member_id=- … key_index=2 01:25:17.928 installed peer media key … key_index=2 installed=true 01:25:18.439 LiveKit: E2EE state changed identity=@dark:nether.im:inMftF8sdw state=Ok ``` `member_id=-` = the Commet event with no `member.id`, now accepted. A ~3 s `MissingKey`→`Ok` blip at 01:25:19–21 matches Commet's membership-triggered key rotation (new index used after its fixed 5 s delay) — self-healed, noted in #136. The tripwire WARN did not fire, so Commet's shape is otherwise within our type. Shipped in v0.3.7 (prod-deployed 01:02 UTC). Closing.

robocub referenced this issue

2026-10-01 23:47:34 +00:00

rtc: Commet joiners in encrypted rooms hear Discord ghosts only after seconds — 3 of 4 ghosts re-keyed the new device ~17 s after its membership (bridge-side recipient lag) #144

robocub referenced this issue

2026-10-02 16:53:22 +00:00

Evaluate element-hq/matrix-rust-rtc as the MatrixRTC layer for crates/matrix-rtc (#41) #146