Topic
text-to-speech
How Do Instructions Shape Speech? New Cross-Attribution Method Reveals Style Control in TTS
A research paper introduces cross-attention attribution for style-captioned text-to-speech, adapting the DAAM framework to speech diffusion models. The method extracts per-token heatmaps across layers and steps, analyzing 3,600 combinations to reveal how caption tokens influence waveforms. Key findings include lower temporal variance for style tokens, correlation with F0 and energy, and peak style conditioning in early ODE steps and deep layers.
Pixel-TTS: Image-Based Text Rendering Improves Robustness in Speech Synthesis
Researchers propose Pixel-TTS, the first visually grounded text-to-speech framework that renders text as images and processes them with 2D convolutions. This eliminates embedding matrix expansion during fine-tuning and improves robustness to unseen characters and orthographic variations. Experiments show competitive performance with faster convergence and zero-shot generalization.