With RUDDER_AUDIO=browser the engine records no local microphone. The page
captures the reader's mic, converts it to 16 kHz mono PCM in an AudioWorklet,
and streams it over a WebSocket at /api/rudder/stream into the same speech
session a local recorder feeds. The words come back on that socket to that
reader only, never on the shared event stream. One session at a time; a second
reader is told the engine is busy. The engine honours RUDDER_HOME, accepts the
public host named in RUDDER_ALLOW_HOST, and refuses a socket from another
origin. The dev proxy now carries WebSockets to backends.
Two gaps found while checking the corner widget (1775f5ac: "Needs eyes: open, HMR, collapse" — all
three verified in Firefox against the running Plugin Studio: opens, keeps the SAME element across a
Studio hot update, collapses).
- LIVE MIC WHEN COLLAPSED: rudder-voice emits `recording` (true/false); the Studio's corner button
turns red with "● " while it listens, so closing the panel never hides that the mic is on.
- DEV BACKENDS SURVIVE VITE RESTARTS: Vite restarts itself when vite.config.js or anything it imports
changes. The backend was stopped on every restart and a new one started at once; the old process
still held the port, the new one died "address already in use", and Rudder showed "engine offline"
until `bun run dev` was restarted by hand (measured twice). Backends now live in a process-wide
registry: reused across restarts, stopped only when the dev process exits, restarted if they crash.
The "engine not running, quietly" answers (89ff2238) are unchanged and read the same registry.
dx-shepherd (R500): with no rudder-engine process the dev proxy returned 500 ECONNREFUSED for
/api/rudder/{devices,events}, which showed as console errors. While a backend is down, backendDevPlugin now
answers its routes itself: /events streams one status event (engine offline, 'The desktop engine is not
running.'), which Rudder renders as its own state, and other routes return an empty JSON list. Running
engines are proxied as before.
Rudder is the first plugin that isn't a map: it shows that one plugin-designs/ folder, developed live
in the Plugin Studio, can ship as a desktop app with its own local backend.
- plugin-designs/rudder-voice: <rudder-voice> element. Words stream into the end of a text area while
you type/paste anywhere (typing holds the stream; nothing duplicated or lost). Mic picker with
hot-plug, 3-second mic test, plain-language errors, info panel, privacy-safe debug report.
meta.json declares "targets": ["desktop"].
- backend/rudder-engine (Go): supervises NVIDIA NeMo-Speech.cpp (ggml, CPU) with the Nemotron 3.5 ASR
Streaming GGUF over a loopback WebSocket; records with parec/pw-record only while recording; recovers
dead Bluetooth mic links (reopen -> CVSD voice profile -> optional RUDDER_BT_RESET) and starts
headsets on CVSD. No telemetry: traced with strace, zero sockets/DNS while transcribing.
- Privacy: log lines redact MAC addresses and the home folder, transcripts are never logged, Host check
against DNS rebinding, CORS only for the desktop webview, random API key for the speech server.
- internal/backend-dev-plugin.mjs + vite.config.js: `bun run dev` starts built plugin backends and
proxies /api/<name> same-origin, so a plugin calls the same API in the studio and on desktop.
- backend/rudder-engine/setup-runtime.sh: fetches the runtime and model pinned by SHA-256.
Not yet: the Tauri desktop build (sidecar) and the PocketBase word-correction table.
