Doxa Maps a3a7c4b5b6 Rudder: Robot voice — speak each sentence in a synthetic voice, offline, for videos without your real voice

Settings → Robot voice: a checkbox and a voice dropdown. When on, every finished sentence is spoken back
(sentences, not words: TTS needs context for its intonation). speak(text) is the single entry point, so a
future conversational mode can reuse the same voice.

- Providers, both on this computer only: Piper (offline neural TTS run by the engine on the CPU; six
  LibriTTS-R speakers offered as "Offline voice A–F") and the browser's own voices, restricted to
  localService === true (some browsers synthesise "online" voices on a server — never offered).
- backend/rudder-engine/tts.go: POST /api/rudder/tts (text → audio/wav, 1–2000 characters, one synthesis
  at a time), GET /api/rudder/tts/voices, POST /api/rudder/duck. The spoken text is never logged, only
  its length. Piper 2023.11.14-2 (MIT) bundles ONNX Runtime 1.14.1: no telemetry client; a synthesis
  traced with strace made 0 network calls. ~150 MB RAM; a 10.7 s sentence rendered in 2.2 s on CPU.
- While the robot speaks through loudspeakers, the engine feeds the transcriber silence, so it never
  transcribes itself (and that silence is not mistaken for a dead mic). With a Bluetooth headset or
  headphones it keeps listening — pausing would only lose the person's words.
- setup-runtime.sh: Piper and the en_US-libritts_r-medium voice (CC BY 4.0), pinned by SHA-256.
- Verified in Firefox against a dev server: dropdown lists A–F, each finished sentence is synthesised
  (200 audio/wav) between duck on/off, zero requests to any other host.

2026-09-28 14:30:56 +00:00

3.6 KiB

Raw Blame History

rudder-voice — Rudder, private voice-to-text (desktop target)

Rudder is a plugin that isn't a map. It shows that the same dx-maps pattern — one folder in plugin-designs/, developed live in the Plugin Studio — can ship a desktop app with its own local backend, not only embeddable maps. It is also a genuinely useful tool for the people who build here: talk to your agents instead of typing, with nothing leaving your computer.

Streaming. Words appear in the text box while you speak; talk for as long as you like.
Type and talk at once. Type, paste links or code anywhere in the box; everything you say is added to the end. While you're typing, Rudder holds its words and adds them when you pause.
CPU only. No graphics card. About 1 GB of memory; keeps up with normal speech on a laptop CPU.
Offline, no telemetry. NVIDIA Nemotron 3.5 ASR Streaming (0.6B) run by NVIDIA's open-source NeMo-Speech.cpp (ggml). Traced with strace while transcribing: zero sockets, zero DNS lookups.
Microphones like a video-call app. Pick the input, hot-plug updates, a 3-second mic test, and plain-language errors (muted, disconnected, dead Bluetooth mic link — which it tries to recover).
Robot voice (optional). Settings → Robot voice: every sentence you say is spoken back in a synthetic voice, so you can record a video that uses dictation without your real voice in it. Offline neural voices (Piper, run by the engine on the CPU) or the browser's own on-device voices; nothing leaves the computer. While the robot talks through loudspeakers, the engine stops listening, so it never transcribes itself; with headphones it keeps listening.

Run it in the Plugin Studio

bash backend/rudder-engine/setup-runtime.sh   # once: runtime + model (verified by SHA-256) + engine build
bun run dev                                   # starts the engine too; open "rudder-voice" in the studio

Linux with PipeWire or PulseAudio (parec, pactl). The engine listens on 127.0.0.1 only.

How it fits together

<rudder-voice> (this folder)  ──HTTP + SSE /api/rudder──►  backend/rudder-engine (Go)
                                                            ├─ parec / pw-record  (mic, open only while recording)
                                                            └─ WebSocket ► nemo-speech serve (127.0.0.1, model stays loaded)

bun run dev starts backend/bin/rudder-engine through internal/backend-dev-plugin.mjs and forwards /api/rudder to it, so the plugin calls the same API in the browser as it will inside the desktop app, where Tauri runs the engine as a sidecar and passes its address in profile-config (apiBase).

rudder-voice.meta.json declares "targets": ["desktop"]: on a plain website there is no local engine, so this plugin is not meant for web embeds.

Settings (environment variables)

Variable	Default	What it does
`RUDDER_PORT`	`5193`	engine HTTP port (loopback)
`RUDDER_NEMO_PORT`	`5194`	speech server port (loopback)
`RUDDER_MODEL` / `RUDDER_NEMO_BIN`	`~/.local/share/rudder/...`	model and runtime paths; the engine refuses a model that isn't a local file
`RUDDER_LANG`	`en-US`	default language (the model supports ~30)
`RUDDER_KEEP_MSBC`	unset	keep Bluetooth headsets on mSBC instead of switching them to the more stable CVSD
`RUDDER_BT_RESET`	unset	optional command that fully resets a Bluetooth headset, used when its mic link dies

Not yet

Desktop build (desktop:build) with the engine as a Tauri sidecar.
A saved word-correction table (PocketBase) for technical terms the model mishears.

3.6 KiB Raw Blame History