← 研踪 PaperTrailCVF · CVPR 2026:cvf:299570bd5c5f4672af6083dab0600b07 · 记录 #1
HAVE-Bench: Hierarchical Audio-Visual Evaluation from Perception to Interaction
Zhong, MuyanCui, ErfeiXing, SenWang, WeiyunWu, WenHu, YuchenZhang, YantingHu, XiaoweiWang, WenhaiZhang, ChaoDai, Jifeng 来源日期(仅到年份) 2026
来源记录日期 2026
雷达观测 2026-11-01 23:30
摘要
Multimodal large language models (MLLMs) have expanded from vision-language systems to include audio, unlocking new capabilities in cross-modal reasoning and interaction. To address the limitation that existing benchmarks focus mainly on perception tasks and lack a unified cognitive evaluation framework, we propose Hierarchical Audio-Visual Evaluation Benchmark (HAVE-Bench). It systematically evaluates the audio-related capabilities of MLLMs along a three-level cognitive hierarchy: Perception, Reasoning, and Interaction, utilizing 2,451 curated samples and manually annotated multi-turn interaction-level tasks. Experiments using this unified framework reveal significant gaps in existing models at the reasoning and interaction levels, with speech-driven visual question answering (VQA) performance significantly lagging behind the text-image setting. These findings underscore the urgency of enhancing models' handling of long and complex audio and facilitating the transfer of reasoning capabilities from the vision-text to the audio-visual domain.
论文发现出处
这些页面提供论文线索,不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化,不是作者修订号。来源日期仅到月份或年份时,按该月或该年首日存档。
记录 #1 · 2026 · 最新观测版本
HAVE-Bench: Hierarchical Audio-Visual Evaluation from Perception to Interaction
Multimodal large language models (MLLMs) have expanded from vision-language systems to include audio, unlocking new capabilities in cross-modal reasoning and interaction. To address the limitation that existing benchmarks focus mainly on perception tasks and lack a unified cognitive evaluation framework, we propose Hierarchical Audio-Visual Evaluation Benchmark (HAVE-Bench). It systematically evaluates the audio-related capabilities of MLLMs along a three-level cognitive hierarchy: Perception, Reasoning, and Interaction, utilizing 2,451 curated samples and manually annotated multi-turn interaction-level tasks. Experiments using this unified framework reveal significant gaps in existing models at the reasoning and interaction levels, with speech-driven visual question answering (VQA) performance significantly lagging behind the text-image setting. These findings underscore the urgency of enhancing models' handling of long and complex audio and facilitating the transfer of reasoning capabilities from the vision-text to the audio-visual domain.
查看此版本解读 →查看此版本原文 ↗