Limitations & responsible AI
What VibeVoice can't do, what it needs, and how to use it responsibly. Search to jump to a question.
Can I use it commercially?
It is intended for research and development. The maintainers do not recommend commercial deployment without further testing of safety and quality.
Which languages are supported?
It depends on the model. For TTS, English and Chinese are the most reliable and other languages are less stable. Offline ASR is natively multilingual across 50+ languages, but the streaming ASR build narrows that to 10 (ZH, EN, FR, DE, IT, JA, KO, PT, RU, ES). Realtime ships multilingual voices (DE, FR, IT, JP, KR, NL, PL, PT, ES) plus 11 English style voices.
Is there a misuse safeguard?
Yes — the model incorporates imperceptible watermarking to flag AI generation and encourages audible disclaimers. You should always disclose AI-generated audio.
What hardware do I need?
For TTS, the 1.5B model runs on an RTX 4070 Ti (12 GB) in BF16; full quality peaks around 14 GB VRAM. A 90-minute synthesis takes roughly 8–12 minutes on an A100 40 GB, with a real-time factor near 0.1×. For recognition you no longer need a GPU at all: the BitNet build is 1.58 GB and reaches a real-time factor under 1 on three CPU threads.
Can it transcribe live, while someone is still talking?
Yes, since September 2026. VibeVoice-ASR-Streaming (1.5B and 7B) emits speaker-attributed text once per audio chunk instead of waiting for the recording to end. Read the cadence honestly though: the shipped checkpoints use a ~2.9 second chunk with ~0.5 seconds of lookahead, so it is a live transcript rather than an instant one.
Are there independent benchmarks for the new models?
Not yet for the streaming build. Microsoft's own technical report puts the 7B streaming model at the lowest average WER/CER across five evaluation sets and best or tied-best on 12 of 13 speaker-attribution settings — but those are the authors' numbers. Treat them as a starting point and test on your own audio.
Can it do overlapping speech or background music?
No. It cannot synthesise simultaneous overlapping speakers, background music or sound effects — it is tuned for clean, sequential dialogue.
How much emotion control is there?
Only basic sentiment. Fine-grained emotion control beyond simple tones is limited.
Why was it briefly pulled in Sep 2025?
Microsoft temporarily removed the TTS code over misuse concerns. The weights stayed on Hugging Face, and access has shifted over time.
What's the license?
MIT.
What are the risks?
High-quality synthetic speech can be misused for impersonation, fraud or disinformation. Disclose AI-generated content, verify transcripts, and avoid misleading deployments.