Building a Fully Local Russian Dictation System on Mac: GigaAM, Whisper, Qwen, and Apple

Introduction

Voice dictation has become an essential productivity tool for writers, developers, and knowledge workers. For Russian speakers, however, the path has been fraught with compromises: cloud-based solutions like Google Docs Voice Typing or Yandex SpeechKit offer decent accuracy but require an internet connection and send audio to external servers, raising privacy concerns. Local alternatives were either limited to English or required complex engineering. A new open‑source project documented on Habr aims to solve this problem by combining three powerful speech models — GigaAM, Whisper, Qwen — with Apple’s native speech framework to create a fully offline dictation system for Mac. The result is a practical, privacy‑preserving setup that runs entirely on the user’s machine. This article unpacks how it works, what each component contributes, and what trade‑offs remain.

Source

The Problem: Cloud Dictation and Its Downsides

Most mainstream dictation tools rely on cloud inference. Services like Yandex SpeechKit or Google Cloud Speech‑to‑Text provide excellent recognition quality for Russian, but every audio snippet leaves your device. For journalists handling sensitive interviews, developers typing code comments in private repositories, or any user concerned about data sovereignty, that’s a deal‑breaker. Local alternatives have existed — such as CMU Sphinx or older Kaldi models — but their accuracy for Russian was often disappointing, with word error rates (WER) well above 20%. The project described in the Habr article tackles this head‑on by assembling a pipeline of modern, open‑weight neural models that can run on a laptop without GPU acceleration—if carefully optimized.

Core Components of the Local Dictation Pipeline

The developers behind the project selected four key technologies, each serving a distinct role:

The combination is unusual: GigaAM (a CTC‑based acoustic model) is used as the primary recognizer because of its efficiency on CPU. Whisper serves as a backup when GigaAM’s confidence is low, and Qwen is invoked after transcription to restore punctuation and rephrase minor errors — a step often skipped in pure ASR pipelines.

Architecture: How the Pipeline Works

The project implements a multi‑stage pipeline:

1. Audio capture – The Apple Speech Framework taps into the Mac’s microphone (or any Core Audio input) and streams 16‑bit PCM audio at 16 kHz.
2. Voice activity detection (VAD) – A simple energy‑based VAD (or optionally WebRTC VAD) segments the stream into utterances.
3. First‑pass ASR with GigaAM – The audio chunks are fed into a pre‑compiled ONNX version of GigaAM. The model outputs character‑level probabilities, which are decoded with a small language model (n‑gram) for first‑pass text.
4. Confidence check – If the log‑probability score falls below a threshold, the chunk is re‑recognised using a smaller Whisper model (e.g., Whisper tiny) running in Core ML format.
5. Post‑processing with Qwen – The raw text from either ASR engine is sent to a quantized Qwen 1.8B model (via llama.cpp). Qwen restores capitalization, adds punctuation, and corrects common homophone errors (e.g., “на счет” vs. “насчёт”).
6. Output injection – The final text is inserted into the active application — either via a simulated keyboard (CGEvent) or through the Accessibility API.

All models run locally. The authors note that on an M2 MacBook Air (8‑core CPU, no GPU), the pipeline achieves a latency of about 1.2–2.5 seconds per utterance of 5–10 seconds, which is acceptable for real‑time dictation.

Performance and Accuracy

The Habr article provides a small but promising evaluation. The developers tested the pipeline on a set of 200 phrases taken from the Russian Common Voice dataset and from their own recordings (various microphones, quiet and noisy environments). The key metrics:

The biggest jump comes from adding Qwen: punctuation and capitalisation become near‑perfect, which dramatically improves the readability of dictation output. The WER drop is modest but meaningful; the authors attribute it to Qwen’s ability to fix “confident” mistakes such as “мы с братом” being misheard as “мыс братом” (the language model knows “мыс” is unlikely in context).

However, the evaluation is small — 200 phrases — and the article acknowledges that WER may vary significantly with microphone quality, background noise, and speaker accent. For a production‑grade solution, a larger test set is needed.

Practical Setup: What You Need

The project is available as a set of scripts and model conversion guides. The hardware requirements are modest:

• Mac with Apple Silicon (M1 or later) — Intel Macs may work but will be slower due to lack of Core ML optimisations.
• At least 8 GB RAM (16 GB recommended for running Qwen alongside other apps).
• Disk space: about 4 GB for the three models (GigaAM ONNX ~800 MB, Whisper tiny ~300 MB, Qwen 1.8B quantised ~1.5 GB, plus runtime dependencies).

The software stack includes Python 3.10+, ONNX Runtime, Core ML tools, llama.cpp (for Qwen), and a wrapper script that ties everything together. The original article provides step‑by‑step installation instructions, but the key takeaway is that no cloud API keys or internet access are required after initial setup.

Trade‑offs and Limitations

No local solution is perfect. The developers identify several current limitations:

• Language model quality: Qwen 1.8B is a relatively small LLM. For some domain‑specific terms (medical, legal, technical), it may introduce incorrect “corrections”. The authors recommend disabling the Qwen post‑processing step when dictating code or specialised vocabulary.
• Latency for longer utterances: The pipeline is designed for short bursts (up to 15 seconds). Dictating a full paragraph without pausing leads to accumulating delay and higher memory usage. The VAD threshold needs tuning per user.
• Multi‑speaker scenario: The system assumes a single speaker. There is no speaker diarization; overlapping speech will produce garbled output.
• Mac‑only: The current implementation relies on Apple‑specific frameworks (Core Audio, Accessibility API). Porting to Windows or Linux would require rewriting the capture and injection layers.

Real‑World Use Cases

Several early testers reported using the system for:

• Writing draft blog posts and emails without touching the keyboard, achieving about 120 words per minute after a short adaptation period.
• Transcribing personal voice memos — all audio stays on the device, a major plus for note‑taking apps like Obsidian or Bear.
• Coding (with Qwen disabled) by dictating function names and comments; the GigaAM + Whisper combination correctly recognised Python keywords like def, return, and for in Russian‑accented speech.

One user tried to dictate a short Bash command — the ASR transcribed `grep -r