CAF-Score: Calibrating CLAP with LALMs for Reference-free Audio-Text Alignment Evaluation
Insung Lee, Taeyoung Jeong, HaeJun Yoo, Du-Seong Chang, Myoung-Wan Koo
原文摘要
While Large Audio-Language Models (LALMs) have advanced audio understanding and generation, robustly evaluating audio-text alignment remains difficult: reference-based metrics require costly annotations, while Contrastive Language-Audio Pretraining (CLAP)-based reference-free metrics frequently overlook syntactic errors and fine-grained details. We propose CAF-Score, a reference-free metric that calibrates CLAP's coarse-grained semantic alignment with the fine-grained comprehension and syntactic awareness of LALMs. Because CAF-Score scores an audio-text pair symmetrically, one configuration applies to both alignment directions: audio captioning evaluation (the BRACE benchmark) and text-to-audio (TTA) generation evaluation (the RELATE and PAM benchmarks), with a single fusion weight selected once on BRACE and transferred without re-tuning. Across all three human-annotated benchmarks, CAF-Score improves agreement with human judgments over standalone CLAP in the large majority of CLAP-LALM combinations (and in all combinations with the strongest LALM backbone), and outperforms standalone LALM metrics on the main captioning benchmark. It also remains competitive with reference-based metrics in challenging scenarios.
cs.SD · cs.AI · cs.CL