WhimprFlow

WhimprFlow

A local-first, cross-platform voice dictation app — hold a key, speak, and clean text lands wherever your cursor is. Speech is transcribed on-device with Whisper and cleaned up (filler removal, self-corrections, punctuation, lists/newlines) by a local LLM, with an optional cloud path. It re-creates the workflow of a Wispr-Flow-style dictation tool from scratch, with its own name, palette, and code.

⚠️ This is a proof of concept, vibe-coded in a few hours. It works and the core loop is real, but it is rough and needs a lot of polish, testing, and hardening before it's anything like production quality. Treat it as a starting point, not a finished product.

---

Platform status

Platform	Status
macOS 14+	Built and working — developed and tested locally (Apple Silicon).
Windows 10/11	Built and working — compiles and runs on real Windows 11 (MSVC). Push-to-talk (hold Right Ctrl), Whisper ASR, clipboard+SendInput paste, and cloud cleanup (OpenAI or any OpenAI-compatible API, e.g. OpenRouter) are verified end-to-end. Auto-learn dictionary capture is still macOS-only; the local (on-device) LLM cleanup worker builds but is CPU-only for now (no CUDA/Vulkan yet).

Both platforms are build-from-source only for now — there's no signed installer/release pipeline yet, so git clone + the steps below is the way to run it on either OS.

---

What's in it

• On-device ASR — Whisper (via whisper.cpp), running on the GPU. Ships a small English model by default; larger models are auto-preferred if present.
• Local LLM cleanup — Qwen3-4B-Instruct (via llama.cpp) runs as a separate worker process and cleans the transcript: removes fillers, resolves spoken self-corrections ("meet at 2… no wait, 3" → "3"), applies spoken punctuation, and formats lists/paragraphs. Deterministic gates guard against over-editing, with a raw-transcript fallback.
• Optional cloud cleanup — publik API (the built-in cloud option: already set up, priced per use at 50% of the model's published list price, most people spend under $2 a month; a cost + data-path notice is shown the first time you pick it, and nothing is minted or sent before you accept it; right after setup a card shows your balance, why it costs money, and "Link this computer & pick a plan" — a new computer starts at $0.00, linking it to your publik account gives $0.05 of free use, once, and "Later" changes nothing), or your own OpenAI / Anthropic key, all behind one trait. Local stays the default. Keys are stored in the OS keychain (macOS Keychain / Windows Credential Manager), never in a file.
• Floating pill UI — an always-on-top bar that appears only while WhimprFlow is working (recording, cleaning up, the done flash, or an error) and disappears the moment it's idle, so it never sits on your screen at rest.
• Personal dictionary + auto-learn — teach it names and terms; on macOS a post-paste Accessibility observer watches for a one-word correction and learns it automatically (conservative filters to avoid junk). Auto-learn capture is macOS-only so far.
• Usage stats — words dictated, words-per-minute, day streak, time saved, 7-day activity, all stored locally.

Architecture

Tauri v2 (Rust core + React/TypeScript webviews). Platform-agnostic logic lives in crates/whimpr-core (state machine, cleanup prompts/gates, dictionary, stats). ASR, audio, and the LLM worker are separate crates. The Tauri app in src-tauri/ hosts the UI and wires the native hotkey/injection per platform (hotkey.rs on macOS, win.rs on Windows).

crates/
  whimpr-core/       state machine, cleanup (prompts/gates/levels), dictionary, stats
  whimpr-asr/        Whisper ASR
  whimpr-audio/      mic capture + resampling
  whimpr-cleanup/    OpenAI / Anthropic cloud providers
  whimpr-llm-worker/ local llama.cpp cleanup worker (separate process)
src-tauri/           Tauri shell: hotkey/paste/autolearn (macOS), win.rs (Windows)
ui/                  React Hub + overlay pill
docs/                spec, architecture notes, research

Releasing (macOS)

A release build needs two things in the environment: a Developer ID identity (see scripts/build-macos.sh) and PUBLIK_APP_TOKEN — the app token that lets a downloaded copy mint its own publik API key after the user accepts the disclosure. The script refuses a release without either. The token is read at compile time and never printed; scripts/verify-macos.sh checks the module is compiled in without touching the value. CI does the same from the PUBLIK_APP_TOKEN repository secret on a v* tag (.github/workflows/release.yml). A dev build (--skip-notarize, or ./dev.sh) may omit the token — publik API then shows "not available in this build" and Local / your own key keep working.

Troubleshooting: "I hold the key, speak, and nothing gets typed"

WhimprFlow now surfaces the exact reason on the pill and in the Hub instead of failing silently (previously this only ever logged to a terminal, which is why it looked like nothing was happening at all). If you still hit this:

• macOS — Accessibility. This is the #1 cause. Open System Settings → Privacy & Security → Accessibility and confirm WhimprFlow is toggled ON. The Hub's onboarding screen blocks you here on first launch; if you granted it once and it still doesn't work, especially after rebuilding the app, see the next point.
• macOS — "granted but still nothing" after a rebuild. Every local tauri build produces a differently-signed binary, and macOS can leave a stale Accessibility entry for the old signature that looks granted but isn't. Fix: in System Settings → Privacy & Security → Accessibility, remove WhimprFlow with the − button and re-add it (or toggle it off/on), then relaunch. WhimprFlow's pill and Hub will now show "Fn key isn't wired up" when this happens instead of just doing nothing.
• Windows — Right Ctrl does nothing. Another app may be holding a conflicting global keyboard hook (some anti-cheat/security tools do this); close it and relaunch WhimprFlow.
• "Speech model not installed". Click Download speech model in the popup or on the Hub banner (148 MB, no relaunch). To place a model by hand, see docs/MODELS.md.
• Still stuck? Run the app from a terminal (./dev.sh on macOS, or the built .exe from PowerShell on Windows) and hold the key once — every failure path also logs a [whimpr]-prefixed line explaining what happened.

Notes & disclaimers

• Not affiliated with, endorsed by, or connected to Wispr Flow or any other product. WhimprFlow is an independent, from-scratch reimplementation of the dictation workflow, with its own name, branding, colors, strings, and code. No third-party code or assets are included.
• Proof of concept. Rushed, under-tested, and missing plenty (auto-learn is macOS-only and conservative, no installer/notarization/signing pipeline on either OS, error handling is thin). Contributions and fixes welcome.
• Privacy. ASR and default cleanup run on-device. Cloud cleanup is opt-in and only sends the transcript (not audio) to the provider you choose. With publik API your transcript goes through publik's servers to a shared model account; publik never trains on it and does not store it — the same two sentences the app shows before you turn it on. API keys never touch disk in plaintext.

License

MIT — see LICENSE.