← 研踪 PaperTrailarXiv:2603.19615 · v2
CAF-Score: Calibrating CLAP with LALMs for Reference-free Audio-Text Alignment Evaluation
首次提交 2026-03-20 11:46
版本更新 2026-11-16 13:35
雷达观测 2026-11-17 17:12
摘要
While Large Audio-Language Models (LALMs) have advanced audio understanding and generation, robustly evaluating audio-text alignment remains difficult: reference-based metrics require costly annotations, while Contrastive Language-Audio Pretraining (CLAP)-based reference-free metrics frequently overlook syntactic errors and fine-grained details. We propose CAF-Score, a reference-free metric that calibrates CLAP's coarse-grained semantic alignment with the fine-grained comprehension and syntactic awareness of LALMs. Because CAF-Score scores an audio-text pair symmetrically, one configuration applies to both alignment directions: audio captioning evaluation (the BRACE benchmark) and text-to-audio (TTA) generation evaluation (the RELATE and PAM benchmarks), with a single fusion weight selected once on BRACE and transferred without re-tuning. Across all three human-annotated benchmarks, CAF-Score improves agreement with human judgments over standalone CLAP in the large majority of CLAP-LALM combinations (and in all combinations with the strongest LALM backbone), and outperforms standalone LALM metrics on the main captioning benchmark. It also remains competitive with reference-based metrics in challenging scenarios.
作者备注:Accepted to EACL 2027 (Main Conference). 16 pages, 9 figures, 4 tables. Code: https://github.com/inseong00/CAF-Score
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-16 13:35 · 最新观测版本
CAF-Score: Calibrating CLAP with LALMs for Reference-free Audio-Text Alignment Evaluation
While Large Audio-Language Models (LALMs) have advanced audio understanding and generation, robustly evaluating audio-text alignment remains difficult: reference-based metrics require costly annotations, while Contrastive Language-Audio Pretraining (CLAP)-based reference-free metrics frequently overlook syntactic errors and fine-grained details. We propose CAF-Score, a reference-free metric that calibrates CLAP's coarse-grained semantic alignment with the fine-grained comprehension and syntactic awareness of LALMs. Because CAF-Score scores an audio-text pair symmetrically, one configuration applies to both alignment directions: audio captioning evaluation (the BRACE benchmark) and text-to-audio (TTA) generation evaluation (the RELATE and PAM benchmarks), with a single fusion weight selected once on BRACE and transferred without re-tuning. Across all three human-annotated benchmarks, CAF-Score improves agreement with human judgments over standalone CLAP in the large majority of CLAP-LALM combinations (and in all combinations with the strongest LALM backbone), and outperforms standalone LALM metrics on the main captioning benchmark. It also remains competitive with reference-based metrics in challenging scenarios.
查看此版本解读 →查看此版本原文 ↗