Topic
speech synthesis
FlowEdit: Associative Memory Framework Cuts TTS Pronunciation Errors by 92.7% Without Retraining
FlowEdit, a new lifelong adaptation framework for flow-matching text-to-speech systems, corrects pronunciation errors on out-of-vocabulary proper nouns without retraining. By storing corrections as latent edits in a Modern Hopfield Network, it achieves a 92.7% reduction in Phoneme Error Rate on 312 multilingual proper nouns while maintaining speech quality.
Repurposing a Speech Classifier for Guided Diffusion-Based Speech Generation
Researchers Makarov and Gerkmann propose a method to repurpose a conventionally trained speech classifier as the backbone for diffusion-based speech generation. By attaching a lightweight subnetwork and training only that under a Denoising Score Matching objective, they achieve high-quality speech synthesis with reduced memory footprint and computational cost compared to traditional classifier guidance that requires two separately trained models.
ZeSTA Framework Enhances Zero-Shot TTS Augmentation for Data-Efficient Personalized Speech Synthesis
Researchers propose ZeSTA, a domain-conditioned training framework that distinguishes real and synthetic speech via a lightweight domain embedding, combined with real-data oversampling. The approach improves speaker similarity over naive synthetic augmentation while preserving intelligibility and perceptual quality in low-resource personalized speech synthesis.
How Do Instructions Shape Speech? New Cross-Attribution Method Reveals Style Control in TTS
A research paper introduces cross-attention attribution for style-captioned text-to-speech, adapting the DAAM framework to speech diffusion models. The method extracts per-token heatmaps across layers and steps, analyzing 3,600 combinations to reveal how caption tokens influence waveforms. Key findings include lower temporal variance for style tokens, correlation with F0 and energy, and peak style conditioning in early ODE steps and deep layers.
Pixel-TTS: Image-Based Text Rendering Improves Robustness in Speech Synthesis
Researchers propose Pixel-TTS, the first visually grounded text-to-speech framework that renders text as images and processes them with 2D convolutions. This eliminates embedding matrix expansion during fine-tuning and improves robustness to unseen characters and orthographic variations. Experiments show competitive performance with faster convergence and zero-shot generalization.
New Research Advances Emotional Speech Synthesis with Latent Representations and FastSpeech 2
Researchers have published an empirical study on arXiv detailing a method for emotional speech synthesis by integrating speaker embedding and a prosody bottleneck into the FastSpeech 2 architecture. The approach addresses two sub-tasks: generating emotional speech for a single speaker and transferring speaking styles from another speaker while retaining target speaker identity. The work was submitted to the VLSP 2022 competition.