Post
120
📄 Our paper on synthetic speech augmentation for Estonian and Slovenian ASR is officially out in Machine Learning with Applications.
The Estonian and Slovenian Parakeet and Whisper models have been on the Hub since March. This is the peer-reviewed paper behind them, with the full adaptation ladder, ablations, and a large-scale error analysis.
Quick recap: we fine-tuned Parakeet-TDT-0.6B-v3 and Whisper-large-v3 on identical data, CommonVoice 17 alone and with ~10 hours per language of synthetic speech (LLM-generated text, LLM-as-judge filtering, TTS). Synthetic speech improves in-domain recognition for every model-language pair, cutting CV17 test WER by 1.3 to 3.7 points over real-data-only fine-tuning. Parakeet, at less than half Whisper's size, reaches the lower WER in every fine-tuned condition. Across 95,070 re-aligned sentence pairs, the architecture largely decides which error types persist, while the adaptation regime mainly changes how many.
📦 Resources
- Paper: https://www.sciencedirect.com/science/article/pii/S2666827026002082
- Parakeet ET:
yuriyvnv/parakeet-tdt-0.6b-estonian
- Parakeet SL:
yuriyvnv/parakeet-tdt-0.6b-slovenian
- Whisper ET:
yuriyvnv/whisper-large-v3-estonian
- Whisper SL:
yuriyvnv/whisper-large-v3-slovenian
- Dataset:
yuriyvnv/synthetic_asr_et_sl
- Code: https://github.com/yuriyvnv/TTS-Augmented-ASR
If you train ASR for other morphologically rich, low-resource languages, I'd like to hear whether the same error patterns show up.
@reach-vb @ylacombe @hf-audio
#speech #asr #syntheticdata #lowresource #estonian #slovenian
The Estonian and Slovenian Parakeet and Whisper models have been on the Hub since March. This is the peer-reviewed paper behind them, with the full adaptation ladder, ablations, and a large-scale error analysis.
Quick recap: we fine-tuned Parakeet-TDT-0.6B-v3 and Whisper-large-v3 on identical data, CommonVoice 17 alone and with ~10 hours per language of synthetic speech (LLM-generated text, LLM-as-judge filtering, TTS). Synthetic speech improves in-domain recognition for every model-language pair, cutting CV17 test WER by 1.3 to 3.7 points over real-data-only fine-tuning. Parakeet, at less than half Whisper's size, reaches the lower WER in every fine-tuned condition. Across 95,070 re-aligned sentence pairs, the architecture largely decides which error types persist, while the adaptation regime mainly changes how many.
📦 Resources
- Paper: https://www.sciencedirect.com/science/article/pii/S2666827026002082
- Parakeet ET:
yuriyvnv/parakeet-tdt-0.6b-estonian
- Parakeet SL:
yuriyvnv/parakeet-tdt-0.6b-slovenian
- Whisper ET:
yuriyvnv/whisper-large-v3-estonian
- Whisper SL:
yuriyvnv/whisper-large-v3-slovenian
- Dataset:
yuriyvnv/synthetic_asr_et_sl
- Code: https://github.com/yuriyvnv/TTS-Augmented-ASR
If you train ASR for other morphologically rich, low-resource languages, I'd like to hear whether the same error patterns show up.
@reach-vb @ylacombe @hf-audio
#speech #asr #syntheticdata #lowresource #estonian #slovenian