Make Vommet fast on a Pinebook Pro, and publish a fair benchmark against Commet #63

New issue

Open

opened 2026-10-06 19:34:37 +00:00 by robocub · 0 comments

robocub commented

2026-10-06 19:34:37 +00:00

Member

Plan (operator, 2026-10-06)

x86_64 first. Iterate on the existing x64 builds for now; no arm64 build until the operator asks for one. Do the measuring and optimising on x64. To approximate a weak machine, pin Vommet to fewer cores and lower the clock (taskset -c 0,1, cpufreq scaling_max_freq, or a VM with 2 slow vCPUs and 4 GB) and use the same cold-start cache drop. The Pinebook Pro runs and the video come once there's an arm64 build.

Prerequisites

A Linux arm64 build (deferred, see Plan). CI currently builds Linux x86_64 (tarball + Flatpak) and Android arm64 only. Options: an arm64 Flatpak (flatpak-builder on an arm64 host or under QEMU) or a native arm64 tarball. The bridge project already builds arm64 on the shared runner, so check what that runner can do.
The startup timers from perf/startup-postload-queue (Settings › Developer › Performance › General, developer mode): preferences, database server, file cache + config, accounts loaded, first frame, first sync, deferred room-state loads.

What to measure (median of 5 runs each)

Cold start: launch → first frame → room list usable → first sync done. Drop the page cache between runs (sync; echo 3 > /proc/sys/vm/drop_caches) so eMMC reads count.
Warm start: the same, without dropping caches.
Opening a busy room: tap → timeline visible.
Scrolling: frame times scrolling a long timeline and the room list (Flutter performance overlay, or --profile build timeline).
Memory: RSS after startup, and after 10 minutes of normal use.
CPU while idle: average over 5 minutes with the window visible and with it minimised (catches constant repaint, cf. upstream #1024).

Fairness rules (for anything published)

Same device, account, network, settings, and the same upstream base: Vommet's last upstream sync point against the matching Commet release. State both versions.
Publish the method and the raw numbers, not just a headline.
Frame it as what Vommet changed (each speed-up tied to a specific fix, e.g. queued room-state loading or the cached full-size images), with Commet credited as the base.
Give it back, only if Commet's maintainers say they want it: Commet's CONTRIBUTING.md prohibits generative-AI use and LLM-written issues/comments, and it's unclear whether AI-assisted findings reported in a human's own words are welcome. The operator asks them first; that question is parked in #64. If yes, the operator writes each report by hand, discloses that the investigation was AI-assisted, and never pastes LLM text. If no, the fixes stay in Vommet.

Deliverables

A results table (Vommet vs Commet, per metric) and a short side-by-side video of a cold start and of scrolling.
A vommet.app section with the video, specs and a link to the raw data.
Upstream bug reports for each relevant finding, if the maintainers agree (see above).

Likely work items (to confirm with measurements)

Queued room-state loading (done on perf/startup-postload-queue; measure).
Drawing the room list from a snapshot before the database finishes loading (Telegram-style).
Space colour-scheme generation at startup (upstream PR #1057 adds a flag to disable it).
Image decode sizes and caching (partly done: sticker picker sizes, cached full-size images).
Idle repaint / CPU use (upstream #1024 and the pause-animations-unfocused branch).

Goal: Vommet should be comfortable on low-end ARM laptops. The Pinebook Pro is the reference: RK3399 (2× A72 + 4× A53), 4 GB RAM, eMMC, Panfrost GPU. When it is, publish a short video plus numbers showing the difference from Commet, done fairly (see below). ## Plan (operator, 2026-10-06) **x86_64 first.** Iterate on the existing x64 builds for now; no arm64 build until the operator asks for one. Do the measuring and optimising on x64. To approximate a weak machine, pin Vommet to fewer cores and lower the clock (`taskset -c 0,1`, cpufreq `scaling_max_freq`, or a VM with 2 slow vCPUs and 4 GB) and use the same cold-start cache drop. The Pinebook Pro runs and the video come once there's an arm64 build. ## Prerequisites - **A Linux arm64 build** (deferred, see Plan). CI currently builds Linux x86_64 (tarball + Flatpak) and Android arm64 only. Options: an arm64 Flatpak (flatpak-builder on an arm64 host or under QEMU) or a native arm64 tarball. The bridge project already builds arm64 on the shared runner, so check what that runner can do. - The **startup timers** from `perf/startup-postload-queue` (Settings › Developer › Performance › General, developer mode): preferences, database server, file cache + config, accounts loaded, first frame, first sync, deferred room-state loads. ## What to measure (median of 5 runs each) 1. **Cold start:** launch → first frame → room list usable → first sync done. Drop the page cache between runs (`sync; echo 3 > /proc/sys/vm/drop_caches`) so eMMC reads count. 2. **Warm start:** the same, without dropping caches. 3. **Opening a busy room:** tap → timeline visible. 4. **Scrolling:** frame times scrolling a long timeline and the room list (Flutter performance overlay, or `--profile` build timeline). 5. **Memory:** RSS after startup, and after 10 minutes of normal use. 6. **CPU while idle:** average over 5 minutes with the window visible and with it minimised (catches constant repaint, cf. upstream #1024). ## Fairness rules (for anything published) - Same device, account, network, settings, and **the same upstream base**: Vommet's last upstream sync point against the matching Commet release. State both versions. - Publish the method and the raw numbers, not just a headline. - Frame it as *what Vommet changed* (each speed-up tied to a specific fix, e.g. queued room-state loading or the cached full-size images), with Commet credited as the base. - Give it back, **only if Commet's maintainers say they want it**: Commet's CONTRIBUTING.md prohibits generative-AI use and LLM-written issues/comments, and it's unclear whether AI-assisted *findings* reported in a human's own words are welcome. The operator asks them first; that question is parked in #64. If yes, the operator writes each report by hand, discloses that the investigation was AI-assisted, and never pastes LLM text. If no, the fixes stay in Vommet. ## Deliverables - A results table (Vommet vs Commet, per metric) and a short side-by-side video of a cold start and of scrolling. - A vommet.app section with the video, specs and a link to the raw data. - Upstream bug reports for each relevant finding, if the maintainers agree (see above). ## Likely work items (to confirm with measurements) - Queued room-state loading (done on `perf/startup-postload-queue`; measure). - Drawing the room list from a snapshot before the database finishes loading (Telegram-style). - Space colour-scheme generation at startup (upstream PR #1057 adds a flag to disable it). - Image decode sizes and caching (partly done: sticker picker sizes, cached full-size images). - Idle repaint / CPU use (upstream #1024 and the `pause-animations-unfocused` branch).

robocub added the

feature

infra

effort: large

labels

2026-10-06 19:34:37 +00:00

robocub referenced this issue

2026-10-06 19:38:23 +00:00

Parked: can we report our upstream-relevant bug findings to Commet, given their AI policy? #64

robocub referenced this issue

2026-10-06 21:08:58 +00:00

Transcribe voice messages to text (like Telegram) #70