arXiv:2610.35432 · v1
Probing the phon* representations of the Mimi codebooks
摘要
Neural audio codecs are increasingly used to encode speech for integration with large language models. This paper investigates the phonetic representations in the first codebook (CB0) of the Mimi codec. ABX style evaluation is poorly suited to Mimi fixed 80 ms tokenization window, which often spans multiple phonemes; we instead use forced alignment, entropy analysis, and Phone-Normalized Mutual Information (PNMI). Using TIMIT speech (58/55 speakers, SA1/SA2), individual tokens are strongly associated with specific phonemes (PNMI = 0.84, above random), yet phonemes are realized by multiple competing tokens rather than a one-to-one mapping.
作者备注：Accepted at the 9th International Conference on Natural Language and Speech Processing (ICNLSP-2026) https://aclanthology.org/2026.icnlsp-1.30/