Improving speech recognition for individuals with dysarthria, a motor speech disorder, remains a significant challenge due to limited data and varying severity levels. A new study by Sapkota, Paban, Kathania, Hemant Kumar, Kadiri, Sudarsana Reddy, and Narayanan, Shrikanth proposes in-domain data augmentation techniques to enhance end-to-end automatic speech recognition (ASR) systems.
Approach and Model
The researchers fine-tuned the pre-trained Wav2Vec2 model, an end-to-end ASR architecture, using individually fine-tuned models for each severity class as baselines. To overcome data scarcity, they investigated four prominent augmentation methods: Speaking-Rate Modification (SRM), Pitch Modification (PM), Formant Modification (FM), and vocal tract Length Perturbation (VTLP). Each technique targets specific acoustic characteristics affected by dysarthria.
Augmentation Techniques
- Speaking-Rate Modification (SRM): Adjusts speech tempo to simulate slower or faster articulation.
- Pitch Modification (PM): Alters the fundamental frequency to mimic variations in vocal pitch.
- Formant Modification (FM): Modifies formant frequencies to reflect changes in vocal tract shape.
- vocal tract Length Perturbation (VTLP): Perturbs the perceived length of the vocal tract, affecting formant structure.
Results Across Severity Levels
The effectiveness of each method varied by severity. The following table presents the best word error rates (WER) achieved and relative improvements over baseline:
| Severity Level | Best Method | Parameter | WER (%) | Relative Improvement (%) |
|---|---|---|---|---|
| Low | SRM | s=0.8 | 9.02 | 30.02 |
| Medium | SRM | s=0.8 | 38.11 | 16.64 |
| High | PM | τ=0.8 | 55.15 | 15.47 |
SRM with a speed factor of 0.8 yielded the best results for low and medium severity, while PM with a pitch factor of 0.8 performed best for high severity. The results confirm that targeted data augmentation can significantly improve dysarthric ASR performance, with relative reductions ranging from 15.47% to 30.02%.
Implications for Accessibility
The study demonstrates that in-domain augmentation tailored to specific speech impairments can bridge the performance gap in ASR systems. For enterprise technology, these advancements can facilitate more inclusive voice interfaces, enabling individuals with dysarthria to use speech-controlled applications in logistics, warehouse management, and other hands-free environments. By reducing WER, such systems become viable for daily use, improving communication and operational efficiency.
