Screen share with audio on Linux and Windows #4

New issue

Open

opened 2026-10-04 20:18:48 +00:00 by robocub · 1 comment

robocub commented

2026-10-04 20:18:48 +00:00

Member

Why

Streaming a game or a video with its sound is a core Discord habit, and a common reason people say they can't switch (Commet #969 says exactly that).

What exists

Video only. matrix_livekit_voip_session.dart builds ScreenShareCaptureOptions with source, framerate and bitrate, and publishes one video track. No audio is captured or published.

Plan

Windows: WASAPI loopback of the default output device. This is what Chromium does for screen-share audio on Windows.
Linux: PipeWire. The screencast portal provides video only, so audio needs its own capture: either the sink monitor (whole system) or a per-application node (better). Vencord's venmic shows the per-app approach.
Publish as a separate audio track with LiveKit source screenShareAudio so Element Call and other clients treat it as stream audio, not as a second microphone.
Echo: whole-system loopback includes the call itself, so everyone would hear themselves back. Either capture a single application, or exclude the app's own output stream. This must be solved before shipping.
A toggle in the source picker: "Share audio".

Notes

This also touches the forked flutter-webrtc / LiveKit SDK, like voice isolation.
The Nether voice bridge does not forward screen shares to Discord today; stream audio would need its own bridge work.

Acceptance

Share a browser playing a video on Linux (PipeWire) and on Windows. Element Call on another machine hears the video's audio in sync. Nobody hears their own voice echoed back.

Filed with LLM assistance. This is a fork-only issue; never refile it on Commet's tracker.

## Why Streaming a game or a video with its sound is a core Discord habit, and a common reason people say they can't switch (Commet #969 says exactly that). ## What exists Video only. `matrix_livekit_voip_session.dart` builds `ScreenShareCaptureOptions` with source, framerate and bitrate, and publishes one video track. No audio is captured or published. ## Plan - **Windows:** WASAPI loopback of the default output device. This is what Chromium does for screen-share audio on Windows. - **Linux:** PipeWire. The screencast portal provides video only, so audio needs its own capture: either the sink monitor (whole system) or a per-application node (better). Vencord's venmic shows the per-app approach. - Publish as a separate audio track with LiveKit source `screenShareAudio` so Element Call and other clients treat it as stream audio, not as a second microphone. - **Echo:** whole-system loopback includes the call itself, so everyone would hear themselves back. Either capture a single application, or exclude the app's own output stream. This must be solved before shipping. - A toggle in the source picker: "Share audio". ## Notes - This also touches the forked flutter-webrtc / LiveKit SDK, like voice isolation. - The Nether voice bridge does not forward screen shares to Discord today; stream audio would need its own bridge work. ## Acceptance Share a browser playing a video on Linux (PipeWire) and on Windows. Element Call on another machine hears the video's audio in sync. Nobody hears their own voice echoed back. --- _Filed with LLM assistance. This is a fork-only issue; never refile it on Commet's tracker._

robocub added the

feature

voice

effort: large

labels

2026-10-04 20:18:48 +00:00

robocub commented

2026-10-05 22:37:54 +00:00

Author

Member

Upstream is working on this — waiting (2026-10-05)

Upstream in flight: commetchat/commet branch update-webrtc (2 commits, 2026-10-05, "initial support for screenshare with audio"). Upstream issue: commetchat/commet#969 (open, no PR yet).

What their branch does:

bumps flutter_webrtc 1.4.0 → 1.6.2 and livekit_client 2.7.0 → 2.13, and drops the hkdf forks (watch encrypted-call interop when this lands)
Windows only (supportsSystemAudio => PlatformUtils.isWindows): createScreenShareTracksWithAudio, audio published as a track named "screenshare"
receive side: new VoipStreamType.screenshareAudio, hidden from the tile grid

What the library already has:

Windows: WASAPI application/desktop loopback (flutter-webrtc 1.5.0, #2060)
Linux: default-sink monitor capture (flutter-webrtc #2115). That includes the call's own playback, so everyone hears themselves echoed back. It also can't capture one app on its own.

Decision: wait until update-webrtc reaches Commet main and our upstream sync picks it up. Then do the Vommet part as a topic on top of it:

Audio source picker in the share dialog (None / Entire System / one app), independent of the video source, like Vesktop's.
Linux per-app capture in our own flutter-webrtc fork: record the chosen app's Pulse/PipeWire sink-inputs (monitor-stream), following new ones as they appear. "Entire System" = every sink-input except Vommet's own, so no echo. Still to check: does pipewire-pulse honour monitor-stream?
Windows: connect the picker to the existing app-loopback mode.
Later: macOS (check the library's loopback), Android 10+ AudioPlaybackCapture (apps can opt out). iOS stays on hold.

Filed with LLM assistance. Fork-only; never refile on Commet's tracker.

## Upstream is working on this — waiting (2026-10-05) **Upstream in flight:** commetchat/commet branch [`update-webrtc`](https://github.com/commetchat/commet/compare/main...update-webrtc) (2 commits, 2026-10-05, "initial support for screenshare with audio"). Upstream issue: commetchat/commet#969 (open, no PR yet). What their branch does: - bumps `flutter_webrtc` 1.4.0 → 1.6.2 and `livekit_client` 2.7.0 → 2.13, and drops the `hkdf` forks (watch encrypted-call interop when this lands) - **Windows only** (`supportsSystemAudio => PlatformUtils.isWindows`): `createScreenShareTracksWithAudio`, audio published as a track named `"screenshare"` - receive side: new `VoipStreamType.screenshareAudio`, hidden from the tile grid What the library already has: - Windows: WASAPI application/desktop loopback (flutter-webrtc 1.5.0, #2060) - Linux: default-sink **monitor** capture (flutter-webrtc #2115). That includes the call's own playback, so everyone hears themselves echoed back. It also can't capture one app on its own. **Decision: wait** until `update-webrtc` reaches Commet main and our upstream sync picks it up. Then do the Vommet part as a topic on top of it: 1. Audio source picker in the share dialog (None / Entire System / one app), independent of the video source, like Vesktop's. 2. Linux per-app capture in our own flutter-webrtc fork: record the chosen app's Pulse/PipeWire sink-inputs (monitor-stream), following new ones as they appear. "Entire System" = every sink-input except Vommet's own, so no echo. Still to check: does pipewire-pulse honour monitor-stream? 3. Windows: connect the picker to the existing app-loopback mode. 4. Later: macOS (check the library's loopback), Android 10+ AudioPlaybackCapture (apps can opt out). iOS stays on hold. --- _Filed with LLM assistance. Fork-only; never refile on Commet's tracker._

robocub added the

upstream

label

2026-10-05 22:37:57 +00:00