Public Instance #43

Open
opened 2026-06-24 21:11:04 +00:00 by dark · 3 comments
Owner

There are several public instances of mautrix-telegram. Instead of installing the software yourself, you just invite the management bot to your matrix channel, and a corresponding bot to your telegram channel, and control the bridge from there.

Offering something similar for Discord users who want to bridge voice channels /could/ be very compelling and might be worth investigating before screen sharing.

There are several [public instances](https://docs.mau.fi/bridges/go/telegram/index.html) of mautrix-telegram. Instead of installing the software yourself, you just invite the management bot to your matrix channel, and a corresponding bot to your telegram channel, and control the bridge from there. Offering something similar for Discord users who want to bridge voice channels /could/ be very compelling and might be worth investigating before screen sharing.
Collaborator

Did a feasibility pass on this. Short version: conceivable, but meaningfully harder than a mautrix text bridge — and the make-or-break question is pool scaling.

Scaling (the good news). A single Discord bot holds one voice connection per guild, independently — so a pool of N bots serves unlimited guilds, each guild showing up to N simultaneous puppets, bounded by CPU/bandwidth, not by a global slot count. That's how public music bots scale. The only real ceiling is Discord's 100-guild verification gate (a per-bot onboarding step). So a small pool genuinely can back many tenants.

The catch. Our current allocator doesn't exploit this — bot availability is a single process-global busy flag (crates/discord/src/puppet.rs:202), so one busy guild artificially starves the others. The voice layer already supports per-guild concurrency (songbird keys Calls by guild, puppet.rs:2260); only the allocator's data model needs reworking — per-(bot, guild) allocation plus multiplexing a bot's capture/playback across guilds. That refactor is the precondition for any multi-tenant offering.

Other structural differences from mautrix.

  • Discord needs bot applications invited to each guild, not a user login; bot avatars are global per app, so puppets share generic identities across tenants.
  • The tenant's homeserver must itself run MatrixRTC / Element Call — the call is anchored to their room and uses their HS's SFU. Unlike a text bridge, "any homeserver" doesn't work.
  • The bridge is static-config / single-process today (no runtime provisioning), and per-ghost crypto stores already hit an FD ceiling under load — both want work before multi-tenant scale.

Suggested sequencing.

  1. Tier 0 — managed hosting: works today with no code (operator adds a bridge + invites the pool bots; onboarding by request).
  2. Multi-guild puppet refactor: the precondition above.
  3. Tier 1 — semi-self-serve: management bot + Discord slash-command binding + runtime dynamic bridges.
  4. Tier 2 — full self-serve: gate on Discord ToS for a public voice relay + per-tenant resource isolation.

Full write-up with code references lives in docs/nether-voicebridge-public-instance-feasibility.md.

Did a feasibility pass on this. Short version: conceivable, but meaningfully harder than a mautrix *text* bridge — and the make-or-break question is pool scaling. **Scaling (the good news).** A single Discord bot holds one voice connection *per guild*, independently — so a pool of N bots serves unlimited guilds, each guild showing up to N simultaneous puppets, bounded by CPU/bandwidth, not by a global slot count. That's how public music bots scale. The only real ceiling is Discord's 100-guild *verification* gate (a per-bot onboarding step). So a small pool genuinely can back many tenants. **The catch.** Our current allocator doesn't exploit this — bot availability is a single process-global `busy` flag (`crates/discord/src/puppet.rs:202`), so one busy guild artificially starves the others. The voice layer already supports per-guild concurrency (songbird keys Calls by guild, `puppet.rs:2260`); only the allocator's data model needs reworking — per-(bot, guild) allocation plus multiplexing a bot's capture/playback across guilds. That refactor is the precondition for any multi-tenant offering. **Other structural differences from mautrix.** - Discord needs bot *applications* invited to each guild, not a user login; bot avatars are global per app, so puppets share generic identities across tenants. - The tenant's *homeserver* must itself run MatrixRTC / Element Call — the call is anchored to their room and uses their HS's SFU. Unlike a text bridge, "any homeserver" doesn't work. - The bridge is static-config / single-process today (no runtime provisioning), and per-ghost crypto stores already hit an FD ceiling under load — both want work before multi-tenant scale. **Suggested sequencing.** 1. **Tier 0 — managed hosting:** works today with no code (operator adds a bridge + invites the pool bots; onboarding by request). 2. **Multi-guild puppet refactor:** the precondition above. 3. **Tier 1 — semi-self-serve:** management bot + Discord slash-command binding + runtime dynamic bridges. 4. **Tier 2 — full self-serve:** gate on Discord ToS for a public voice relay + per-tenant resource isolation. Full write-up with code references lives in `docs/nether-voicebridge-public-instance-feasibility.md`.
Collaborator

Follow-up on the processing/scaling angle, specifically managing encrypted callers — since that's where a public instance gets expensive. Researched it against Element Call's actual key-distribution code; there's a concrete, bounded solution.

The unit cost. In an encrypted call each Discord speaker's ghost spins up a full matrix-sdk crypto client — its own SQLite store, a perpetual /sync, and a key-distribution loop (crates/matrix-rtc/src/rtc/ghost.rs:1260). A 20-speaker encrypted call = 20 crypto stores + 20 homeserver /sync long-polls + 40 long-lived tasks. That per-ghost footprint, not encryption itself, is what hit the FD ceiling.

What can't be collapsed (verified, not guessed). I'd hoped one shared identity could distribute every ghost's key. It can't: matrix-js-sdk attributes a received media key to the authenticated to-device sender (event.getSender()) and ignores the member.id in the payload. So N distinct attributable E2EE participants genuinely need N distinct sending identities — the same reason EC's own key distribution is O(n²).

What can be fixed — and it retires the FD wall. The ghost is send-only (it never receives or decrypts; its /sync only refreshes recipient device-lists, whose results are discarded). So each ghost can drop from a full Client to a standalone matrix-sdk-crypto::OlmMachine backed by a MemoryStore:

  • Perpetual /sync → gone, replaced by on-demand /keys/query before a key send (removes the O(ghosts) sync fan-out on the homeserver).
  • All crypto FDs → gone. (SqliteCryptoStore actually uses a deadpool pool of max(2, cores×4) connections — ~32 per ghost store on an 8-core box, not ~3 — which is the real reason ~20 ghosts blew past 1024. MemoryStore makes FD cost independent of ghost count; the LimitNOFILE=65536 bump is a stopgap.)
  • Ghosts are ephemeral (alive only during a live call), so "keys lost on drop" fits. The send primitive (encrypt_content_for_devices, behind a feature flag the bridge already enables) is exactly what encrypt_and_send_raw_to_device wraps today — a lower-level rewiring of the same operation, not new crypto.

Residual floor: O(ghosts × in-call recipient devices) pairwise Olm sessions per call — bounded by call size, now cheap rather than eliminated. Optional further win: a per-room shared device-tracker collapses redundant /keys/query from O(ghosts) to O(1) per room.

Trusted-HS shortcut: the plaintext to-device path (Path 2, masqueraded as the ghost) needs zero per-ghost crypto and current EC accepts it — fine for a self-hosted/trusted homeserver, but for a public instance bridging third-party encrypted rooms it lets the operator's homeserver read the media keys (an E2EE downgrade), so it's off the table for the multi-tenant case.

Net for Tier 2: cheap ghosts (OlmMachine + MemoryStore, contained to the ghost path in matrix-rtc, no protocol change) plus per-tenant process/runtime isolation. The footprint shrinking from "~32 FDs + a sync loop" per ghost to "some memory + occasional HTTP" is what makes that isolation affordable.

Full analysis with code/source references is in docs/nether-voicebridge-public-instance-feasibility.md under "Scaling encrypted callers".

Follow-up on the processing/scaling angle, specifically **managing encrypted callers** — since that's where a public instance gets expensive. Researched it against Element Call's actual key-distribution code; there's a concrete, bounded solution. **The unit cost.** In an *encrypted* call each Discord speaker's ghost spins up a full `matrix-sdk` crypto client — its own SQLite store, a perpetual `/sync`, and a key-distribution loop (`crates/matrix-rtc/src/rtc/ghost.rs:1260`). A 20-speaker encrypted call = 20 crypto stores + 20 homeserver `/sync` long-polls + 40 long-lived tasks. That per-ghost footprint, not encryption itself, is what hit the FD ceiling. **What can't be collapsed (verified, not guessed).** I'd hoped one shared identity could distribute every ghost's key. It can't: matrix-js-sdk attributes a received media key to the *authenticated to-device sender* (`event.getSender()`) and ignores the `member.id` in the payload. So N distinct attributable E2EE participants genuinely need N distinct sending identities — the same reason EC's own key distribution is O(n²). **What can be fixed — and it retires the FD wall.** The ghost is *send-only* (it never receives or decrypts; its `/sync` only refreshes recipient device-lists, whose results are discarded). So each ghost can drop from a full `Client` to a standalone `matrix-sdk-crypto::OlmMachine` backed by a `MemoryStore`: - Perpetual `/sync` → gone, replaced by on-demand `/keys/query` before a key send (removes the O(ghosts) sync fan-out on the homeserver). - All crypto FDs → gone. (`SqliteCryptoStore` actually uses a deadpool pool of `max(2, cores×4)` connections — ~32 per ghost store on an 8-core box, not ~3 — which is the real reason ~20 ghosts blew past 1024. `MemoryStore` makes FD cost independent of ghost count; the `LimitNOFILE=65536` bump is a stopgap.) - Ghosts are ephemeral (alive only during a live call), so "keys lost on drop" fits. The send primitive (`encrypt_content_for_devices`, behind a feature flag the bridge already enables) is exactly what `encrypt_and_send_raw_to_device` wraps today — a lower-level rewiring of the same operation, not new crypto. **Residual floor:** O(ghosts × in-call recipient devices) pairwise Olm sessions per call — bounded by call size, now cheap rather than eliminated. Optional further win: a per-room shared device-tracker collapses redundant `/keys/query` from O(ghosts) to O(1) per room. **Trusted-HS shortcut:** the plaintext to-device path (Path 2, masqueraded as the ghost) needs *zero* per-ghost crypto and current EC accepts it — fine for a self-hosted/trusted homeserver, but for a public instance bridging *third-party encrypted rooms* it lets the operator's homeserver read the media keys (an E2EE downgrade), so it's off the table for the multi-tenant case. **Net for Tier 2:** cheap ghosts (OlmMachine + `MemoryStore`, contained to the ghost path in `matrix-rtc`, no protocol change) **plus** per-tenant process/runtime isolation. The footprint shrinking from "~32 FDs + a sync loop" per ghost to "some memory + occasional HTTP" is what makes that isolation affordable. Full analysis with code/source references is in `docs/nether-voicebridge-public-instance-feasibility.md` under "Scaling encrypted callers".
dark added this to the Roadmap project 2026-06-27 16:29:34 +00:00
robocub referenced this issue from a commit 2026-06-28 18:40:15 +00:00
Collaborator

Open registration shipped: merged to master 896e116 (unreleased, rides the next release).

The last feature gap between the allowlist-only management bot and a self-serve public instance. New [management] open_registration knob (default off — nothing changes until enabled), which config validation only accepts with BOTH two-sided-auth knobs on (require_room_authority + require_discord_approval, from #47) — that chain is the safety model: a self-serve user can only bridge a Matrix room they moderate, into a Discord guild whose manager approves.

What a non-allowlisted user gets when it's on:

  • DM the bot: direct invites only (auto-join + greeting); non-direct room invites are rejected so nobody can pull the bot into arbitrary rooms; deny-listed owners are refused at the door (fail-closed on an unreadable ban list).
  • Scoped commands: link (full two-sided flow), unlink/list limited to bridges they own (via the #47 owner records — nvb_config::tenant_owners() is the new read-only helper), and a self-serve help. ban/unban/bans reply "operator-only".
  • [limits] max_tenants_per_owner: caps bridges per self-serve owner; pending approval codes count toward it so parked codes can't dodge the cap. Operators are exempt (bounded by max_tenants).

Live-tested on staging 2026-07-06 (bridgebot de-allowlisted as the untrusted driver; prod down ~10 min, restored clean), 8/8 rows exact: non-direct invite rejection, DM auto-join + greeting, operator-verb refusal, self-serve help, full link→pending-code→/bridge approve→up flow (owner recorded, link_pendinglink_ok(approved_by) audit), owner-scoped list (a seeded foreign-owned tenant and all static bridges hidden), owner-cap refusal on a second link, and "not yours" on unlinking a foreign tenant (own unlink fine). Zero unexpected errors.

Runner-verified: 232 workspace tests + clippy --workspace --all-targets -D warnings clean.

Operational note: legacy tenants without an owner record can't be unlinked by self-serve users (operator-only) — expected, they predate owner records. Enabling this on the public instance stays gated on the ToS follow-ups (privacy policy, disclosure/consent UX, 100-server verification plan) per docs/nether-voicebridge-discord-tos.md.

**Open registration shipped: merged to master `896e116` (unreleased, rides the next release).** The last feature gap between the allowlist-only management bot and a self-serve public instance. New `[management] open_registration` knob (default **off** — nothing changes until enabled), which config validation only accepts with BOTH two-sided-auth knobs on (`require_room_authority` + `require_discord_approval`, from #47) — that chain is the safety model: a self-serve user can only bridge a Matrix room they moderate, into a Discord guild whose manager approves. What a non-allowlisted user gets when it's on: - **DM the bot**: direct invites only (auto-join + greeting); non-direct room invites are rejected so nobody can pull the bot into arbitrary rooms; deny-listed owners are refused at the door (fail-closed on an unreadable ban list). - **Scoped commands**: `link` (full two-sided flow), `unlink`/`list` limited to bridges they own (via the #47 owner records — `nvb_config::tenant_owners()` is the new read-only helper), and a self-serve `help`. `ban`/`unban`/`bans` reply "operator-only". - **`[limits] max_tenants_per_owner`**: caps bridges per self-serve owner; pending approval codes count toward it so parked codes can't dodge the cap. Operators are exempt (bounded by `max_tenants`). **Live-tested on staging 2026-07-06** (bridgebot de-allowlisted as the untrusted driver; prod down ~10 min, restored clean), 8/8 rows exact: non-direct invite rejection, DM auto-join + greeting, operator-verb refusal, self-serve help, full link→pending-code→`/bridge approve`→up flow (owner recorded, `link_pending`→`link_ok(approved_by)` audit), owner-scoped `list` (a seeded foreign-owned tenant and all static bridges hidden), owner-cap refusal on a second link, and "not yours" on unlinking a foreign tenant (own unlink fine). Zero unexpected errors. Runner-verified: 232 workspace tests + `clippy --workspace --all-targets -D warnings` clean. Operational note: legacy tenants without an `owner` record can't be unlinked by self-serve users (operator-only) — expected, they predate owner records. Enabling this on the public instance stays gated on the ToS follow-ups (privacy policy, disclosure/consent UX, 100-server verification plan) per `docs/nether-voicebridge-discord-tos.md`.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
dark/nether-voicebridge#43
No description provided.