HAVE-Bench: Hierarchical Audio-Visual Evaluation from Perception to Interaction
Zhong, Muyan, Cui, Erfei, Xing, Sen, Wang, Weiyun, Wu, Wen, Hu, Yuchen, Zhang, Yanting, Hu, Xiaowei 等 11 位作者
原文摘要
Multimodal large language models (MLLMs) have expanded from vision-language systems to include audio, unlocking new capabilities in cross-modal reasoning and interaction. To address the limitation that existing benchmarks focus mainly on perception tasks and lack a unified cognitive evaluation framework, we propose Hierarchical Audio-Visual Evaluation Benchmark (HAVE-Bench). It systematically evaluates the audio-related capabilities of MLLMs along a three-level cognitive hierarchy: Perception, Reasoning, and Interaction, utilizing 2,451 curated samples and manually annotated multi-turn interaction-level tasks. Experiments using this unified framework reveal significant gaps in existing models at the reasoning and interaction levels, with speech-driven visual question answering (VQA) performance significantly lagging behind the text-image setting. These findings underscore the urgency of enhancing models' handling of long and complex audio and facilitating the transfer of reasoning capabilities from the vision-text to the audio-visual domain.
cs.CV · cs.CL
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表