e2e: DM/invite lifecycle coverage — E2EE DMs, restart races, cooldown semantics (gaps exposed by #96/#97) #99

Closed
opened 2026-07-12 04:17:47 +00:00 by robocub · 2 comments
Collaborator

The first real self-serve user surfaced two DM-path failures the 20-scenario suite never exercises (#96 wrong-address stranding, #97 restart-window double-processing → auto-decline). The suite covers the happy DM path (create/greet/commands, invite-gate rejects) but only with LOCAL, PLAINTEXT, steady-state DMs. Gaps, in priority order:

  1. E2EE DM actor (biggest blind spot). Real clients (Element) often create DMs with encryption ON — dau's first DM was E2EE. Whether the mgmt bot can DECRYPT commands in an encrypted DM has never been tested anywhere (harness DMs are raw-CS-API plaintext; the bot's reply-side encryption is believed to work via the SDK, receive-side is unknown). Needs a crypto-capable test actor — a small matrix-sdk-based client in the harness (heavier than the raw-CS masquerade actors, but reusable). Scenario: encrypted DM → greeting decryptable → help answered → link flow end-to-end.
  2. Offline/restart invite handling. Scenario: send the DM invite while the bridge process is DOWN, then start it — assert exactly ONE accept, greeting delivered, NO decline (the #97 regression test at the scenario level; the dedupe/cooldown-after-join logic itself should also get in-crate tests with the #97 fix).
  3. Invite-cooldown semantics (unit tier). #89a's cooldown must distinguish "N distinct invites from one sender" (rate-limit) from "the same invite seen twice" (dedupe, never decline) — pin with in-crate tests once #97 lands.
  4. Federated-inviter caveat (document, don't build). dau's failures involved federation (tinier.town); the harness can't mint remote users. Note the limitation in the testing design doc rather than pretending local invites cover it.

Suggested order: #97 fix + its unit tests first, then the offline-invite scenario (cheap — one process-start reordering), then the E2EE actor (the real investment).

The first real self-serve user surfaced two DM-path failures the 20-scenario suite never exercises (#96 wrong-address stranding, #97 restart-window double-processing → auto-decline). The suite covers the happy DM path (create/greet/commands, invite-gate rejects) but only with LOCAL, PLAINTEXT, steady-state DMs. Gaps, in priority order: 1. **E2EE DM actor (biggest blind spot).** Real clients (Element) often create DMs with encryption ON — dau's first DM was E2EE. Whether the mgmt bot can DECRYPT commands in an encrypted DM has never been tested anywhere (harness DMs are raw-CS-API plaintext; the bot's reply-side encryption is believed to work via the SDK, receive-side is unknown). Needs a crypto-capable test actor — a small matrix-sdk-based client in the harness (heavier than the raw-CS masquerade actors, but reusable). Scenario: encrypted DM → greeting decryptable → `help` answered → `link` flow end-to-end. 2. **Offline/restart invite handling.** Scenario: send the DM invite while the bridge process is DOWN, then start it — assert exactly ONE accept, greeting delivered, NO decline (the #97 regression test at the scenario level; the dedupe/cooldown-after-join logic itself should also get in-crate tests with the #97 fix). 3. **Invite-cooldown semantics (unit tier).** #89a's cooldown must distinguish "N distinct invites from one sender" (rate-limit) from "the same invite seen twice" (dedupe, never decline) — pin with in-crate tests once #97 lands. 4. **Federated-inviter caveat (document, don't build).** dau's failures involved federation (tinier.town); the harness can't mint remote users. Note the limitation in the testing design doc rather than pretending local invites cover it. Suggested order: #97 fix + its unit tests first, then the offline-invite scenario (cheap — one process-start reordering), then the E2EE actor (the real investment).
Author
Collaborator

#99 implemented on feat/99-invite-dm-scenarios — live evidence (2026-07-12, staging, compressed profile)

Suite 20 → 23. Runner: clippy -D warnings clean, cargo test --workspace --locked green (331 tests).

1. mgmt_dm_e2eePASS 11.5 s (first try, pre-alpha.25 bridge; 10.5 s re-run post-alpha.25)

New crypto actor MatrixCryptoDmClient (full matrix-sdk client, ephemeral server-minted device, in-memory stores, logout-on-teardown). Encrypted DM → bot auto-joins → greeting decryptablehelp round-trips through encryption both ways → transport proof: every m.room.message crossed the wire as m.room.encrypted (TimelineEventKind::Decrypted vs PlainText doubles as the wire check), traffic asserted both directions.

2. mgmt_invite_offline — shipped Pass, not KnownFail (deviation from the brief, evidence-driven)

Pre-fix bridge, mgmt_cooldown_secs = 60 ARMED via the new start_with_management_opts extra-limits seam (the standing harness block leaves it unset — exactly why 20 scenarios never reproduced #97a): unexpected_pass 21.4 s — exactly ONE accept, ZERO declines, bot stayed joined (verified in artifacts). The #97a trigger needs the seconds-long federated-join redelivery window; a same-server inviter (all the harness can mint) doesn't produce it. Re-classified per the runner's own unexpected_pass recommendation; the scenario stays as the post-fix-contract regression guard. Federation limitation documented in the testing-design doc. Post-alpha.25: pass 19.7 s.

3. mgmt_dm_pending_reapKnownFail{#97}, still failing post-alpha.25 (see #100 comment)

Pre-fix: reap reproduced deterministically — the compressed #89b sweep reaped the fresh operator-msg DM ~100 ms after creation, before the message could even be sent (403 sender's membership \leave` is not `join``); room-id-correlated in artifacts. Post-alpha.25 (3 runs): still known_fail — two residual modes, detail + mechanism posted on #100.

Bonus finding while building it: the SDK's create_dm mints operator-msg DMs encrypted, so the reap-scenario target is a crypto client — coverage of the msg outreach path is E2EE end-to-end.

4. Regression (bridge_proc seam changes)

mgmt_self_serve_link pass 61.4 s · room_misconfig_notice pass 29.3 s.

Branch is rebased onto master 1a769f9 (alpha.25), runner-green. Awaiting the operator's merge word.

## #99 implemented on `feat/99-invite-dm-scenarios` — live evidence (2026-07-12, staging, compressed profile) **Suite 20 → 23.** Runner: clippy `-D warnings` clean, `cargo test --workspace --locked` green (331 tests). ### 1. `mgmt_dm_e2ee` — **PASS 11.5 s** (first try, pre-alpha.25 bridge; 10.5 s re-run post-alpha.25) New crypto actor `MatrixCryptoDmClient` (full matrix-sdk client, ephemeral server-minted device, in-memory stores, logout-on-teardown). Encrypted DM → bot auto-joins → greeting **decryptable** → `help` round-trips through encryption both ways → transport proof: every `m.room.message` crossed the wire as `m.room.encrypted` (`TimelineEventKind::Decrypted` vs `PlainText` doubles as the wire check), traffic asserted both directions. ### 2. `mgmt_invite_offline` — shipped **Pass**, not KnownFail (deviation from the brief, evidence-driven) Pre-fix bridge, `mgmt_cooldown_secs = 60` ARMED via the new `start_with_management_opts` extra-limits seam (the standing harness block leaves it unset — exactly why 20 scenarios never reproduced #97a): **unexpected_pass 21.4 s** — exactly ONE accept, ZERO declines, bot stayed joined (verified in artifacts). The #97a trigger needs the seconds-long federated-join redelivery window; a same-server inviter (all the harness can mint) doesn't produce it. Re-classified per the runner's own `unexpected_pass` recommendation; the scenario stays as the post-fix-contract regression guard. Federation limitation documented in the testing-design doc. Post-alpha.25: pass 19.7 s. ### 3. `mgmt_dm_pending_reap` — **KnownFail{#97}, still failing post-alpha.25** (see #100 comment) Pre-fix: reap reproduced deterministically — the compressed #89b sweep reaped the fresh operator-`msg` DM **~100 ms after creation**, before the message could even be sent (403 `sender's membership \`leave\` is not \`join\``); room-id-correlated in artifacts. Post-alpha.25 (3 runs): still known_fail — two residual modes, detail + mechanism posted on #100. Bonus finding while building it: the SDK's `create_dm` mints operator-`msg` DMs **encrypted**, so the reap-scenario target is a crypto client — coverage of the `msg` outreach path is E2EE end-to-end. ### 4. Regression (bridge_proc seam changes) `mgmt_self_serve_link` pass 61.4 s · `room_misconfig_notice` pass 29.3 s. Branch is rebased onto master `1a769f9` (alpha.25), runner-green. **Awaiting the operator's merge word.**
Author
Collaborator

Merged to master with the operator's word: d2503b6 (#99 squash — suite 20 → 23) + 26605c9 (nightly e2e identity fix: the workflow still masqueraded the deactivated pre-#96 @nether-voicebridge_bot; both env blocks now use nether-voicebridge_e2e, so tomorrow's 06:00 nightly runs with the right identity).

Remaining follow-up lives on #100: mgmt_dm_pending_reap stays KnownFail{#97} until the alpha.25 residuals (store-lag reap race + cold-DM first-message UTD) are fixed in crates/matrix-rtc — then flip to Pass and re-run live. Closing this issue: all four #99 deliverables are on master with live evidence above.

Merged to master with the operator's word: `d2503b6` (#99 squash — suite 20 → 23) + `26605c9` (nightly e2e identity fix: the workflow still masqueraded the deactivated pre-#96 `@nether-voicebridge_bot`; both env blocks now use `nether-voicebridge_e2e`, so tomorrow's 06:00 nightly runs with the right identity). Remaining follow-up lives on #100: `mgmt_dm_pending_reap` stays `KnownFail{#97}` until the alpha.25 residuals (store-lag reap race + cold-DM first-message UTD) are fixed in `crates/matrix-rtc` — then flip to `Pass` and re-run live. Closing this issue: all four #99 deliverables are on master with live evidence above.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
dark/nether-voicebridge#99
No description provided.