c067a5d069
Rudder was measured slowing after about 1,400 words while a Tauri build ran. The word count is incidental — it is a proxy for how long the build had been going. The speech model runs on the CPU (backend/rudder-engine starts it with --device cpu), and a Rust build saturates every core, so at equal priority it starves the one process whose latency a person can hear. Three changes, and the direction of the first is forced rather than chosen: 1. internal/desktop-build.mjs nices the whole build tree to 10, through the one run() helper every step goes through, so cargo, rustc and the linker inherit it. We nice the COMPETITOR because an unprivileged process cannot raise its own priority on Linux — a negative nice needs CAP_SYS_NICE — so "promote the ASR" would fail with EPERM. If nice(1) is missing the build still runs and says so, because a silent fallback would look exactly like the fix working. 2. cargo is capped at cores-1. nice decides who wins a CONTESTED core; it does not stop the build taking all of them, and a fully subscribed machine still adds scheduling latency to the audio path. An explicit CARGO_BUILD_JOBS is honoured rather than overridden. 3. yieldCPU reports its error instead of discarding it. It was `_ = syscall.Setpriority(...)`, which cannot fail visibly — a call that silently does nothing is indistinguishable from one that worked, and this one guards something audible. Also, one real O(n) in the widget, found while checking studio's answer that the collapsed pill already shows the last words: sendActivity split the ENTIRE draft on every activity tick to take five words. Measured at 3,000 words: 1.61ms per tick against 0.051ms for a bounded tail, 31x, with byte-identical output. Verify the priority work with `ps -o ni=` on the running cargo processes, never by any of these calls returning. |
||
|---|---|---|
| .. | ||
| index.html | ||
| index.js | ||
| README.md | ||
| rudder-voice.meta.json | ||
| rudder-voice.vue | ||
rudder-voice — Rudder, private voice-to-text (desktop target)
Rudder is a plugin that isn't a map. It shows that the same dx-maps pattern — one folder in
plugin-designs/, developed live in the Plugin Studio — can ship a desktop app with its own local
backend, not only embeddable maps. It is also a genuinely useful tool for the people who build here:
talk to your agents instead of typing, with nothing leaving your computer.
-
Streaming. Words appear in the text box while you speak; talk for as long as you like.
-
Type and talk at once. Type, paste links or code anywhere in the box; everything you say is added to the end. While you're typing, Rudder holds its words and adds them when you pause.
-
CPU only. No graphics card. About 1 GB of memory; keeps up with normal speech on a laptop CPU.
-
Offline, no telemetry. NVIDIA Nemotron 3.5 ASR Streaming (0.6B) run by NVIDIA's open-source NeMo-Speech.cpp (ggml). Traced with
stracewhile transcribing: zero sockets, zero DNS lookups. -
Microphones like a video-call app. Pick the input, hot-plug updates, a 3-second mic test, and plain-language errors (muted, disconnected, dead Bluetooth mic link — which it tries to recover).
-
Robot voice (optional). Settings → Robot voice: every sentence you say is spoken back in a synthetic voice, so you can record a video that uses dictation without your real voice in it. Offline neural voices (Piper, run by the engine on the CPU) or the browser's own on-device voices; nothing leaves the computer. While the robot talks through loudspeakers, the engine stops listening, so it never transcribes itself; with headphones it keeps listening.
Run it in the Plugin Studio
bash backend/rudder-engine/setup-runtime.sh # once: runtime + model (verified by SHA-256) + engine build
bun run dev # starts the engine too; open "rudder-voice" in the studio
Linux with PipeWire or PulseAudio (parec, pactl). The engine listens on 127.0.0.1 only.
How it fits together
<rudder-voice> (this folder) ──HTTP + SSE /api/rudder──► backend/rudder-engine (Go)
├─ parec / pw-record (mic, open only while recording)
└─ WebSocket ► nemo-speech serve (127.0.0.1, model stays loaded)
bun run dev starts backend/bin/rudder-engine through internal/backend-dev-plugin.mjs and forwards
/api/rudder to it, so the plugin calls the same API in the browser as it will inside the desktop app,
where Tauri runs the engine as a sidecar and passes its address in profile-config (apiBase).
rudder-voice.meta.json declares "targets": ["desktop"]: on a plain website there is no local engine,
so this plugin is not meant for web embeds.
Settings (environment variables)
| Variable | Default | What it does |
|---|---|---|
RUDDER_PORT |
5193 |
engine HTTP port (loopback) |
RUDDER_NEMO_PORT |
5194 |
speech server port (loopback) |
RUDDER_MODEL / RUDDER_NEMO_BIN |
~/.local/share/rudder/... |
model and runtime paths; the engine refuses a model that isn't a local file |
RUDDER_LANG |
en-US |
default language (the model supports ~30) |
RUDDER_KEEP_MSBC |
unset | keep Bluetooth headsets on mSBC instead of switching them to the more stable CVSD |
RUDDER_BT_RESET |
unset | optional command that fully resets a Bluetooth headset, used when its mic link dies |
Not yet
- Desktop build (
desktop:build) with the engine as a Tauri sidecar. - A saved word-correction table (PocketBase) for technical terms the model mishears.