arXiv:2610.29164 · v2
XingChen-UniFE: A Unified Generative Speech Front-End via Complementary Acoustic-Semantic Modeling
摘要
This paper presents XingChen-UniFE, a unified large audio language model (LALM) for general speech restoration (GSR) and target speaker extraction (TSE). The proposed model employs complementary semantic and acoustic representations through a dual-stream encoder, where a self-supervised learning model provides hierarchical semantic features and a learnable acoustic branch captures fine-grained structures from complex spectra. A bidirectional cross-attention module further facilitates information exchange between the two representations. The fused representation conditions the LALM for discrete speech token generation with task and speaker information. Experiments demonstrate strong performance on both GSR and TSE and robust generalization to unseen degradation conditions.
作者备注：Submitted to ICASSP 2027. Added an appendix with additional evaluation results for speech restoration and target speaker extraction