在自回归语音克隆TTS模型中发现可解释的声学特征
Discovering Interpretable Acoustic Features in Autoregressive Voice-Cloning TTS Models
Anuj Diwan, David Harwath, Eunsol Choi
AI 速览：该论文提出一个基于特征的框架，用于解释自回归语音克隆TTS模型内部表征的声学属性，并评估稀疏自编码器、PCA和随机方向在探测与操控上的效果。
Voice-cloning TTS models synthesize an unseen speaker's voice from brief speech prompts, but it remains unclear which acoustic properties they represent and use. We introduce a feature-based interpretability framework for autoregressive voice-cloning TTS models, analyzing features from sparse autoencoders (SAEs), PCA, and random directions via sparse probing, steering, and quantitative auto-interpretability. We focus on Qwen3-TTS and extend probing and steering to $5$ additional models to test generalization across architectures. These models encode concepts like pitch, energy, noise, and accent, but sparse prediction and steering are effective only for some concepts. Strong probing does not always translate to steering or vice-versa. SAEs are useful for sparse probing of human-labelled concepts and novel-concept discovery, while PCA is competitive for dense probing and steering. Auto-interpretability finds $684$ automatically validated SAE features in Qwen3-TTS covering speaker clusters, speaking style, and recording conditions beyond known concepts. Overall, we perform a comprehensive interpretability study of autoregressive voice-cloning TTS models.