UAV-DualCog: A Dual-Cognition Benchmark for Grounded UAV Agent-World Reasoning with MLLMs
Like Liu, Zhengzheng Xu, Hongzhe Li, Lin Wang, Dian Shao
Multimodal large language models (MLLMs) perform strongly across vision-language tasks, yet remain insufficiently explored in UAV scenarios. Recent UAV-oriented benchmarks have begun to evaluate MLLMs in aerial scenarios, but typically focus on scene understanding, event recognition, or navigation completion, rather than jointly assessing the dual-cognition capability required for UAV agents: reasoning about the UAV's own position, behavior, and observation state alongside external landmark states under continuously changing viewpoints, supported by explicit spatio-temporal grounding. To address this gap, we present UAV-DualCog, the first benchmark to formulate UAV dual cognition as the coordinated evaluation of Agent-State Reasoning and World-State Reasoning across multi-view images and egocentric videos, with explicit spatial or temporal grounding. The State-aware UAV Simulation Platform (SUSP) and the DualCog-aware Capture System (DCCS) construct 4,096 image and 2,048 video samples from 12 scenes, pairing complementary state questions within shared Agent-Landmark contexts. Extensive evaluations on 45 models show that current MLLMs remain far from reliable in UAV dual cognition: Agent-State Reasoning is generally weaker than World-State Reasoning, semantic answers do not reliably coincide with spatial or temporal grounding, and coordinated abilities remain limited. Thinking-mode comparisons, open-ended analyses, and sim-to-real validation show that these limitations persist across inference settings, formats, and environments. We further construct UAV-DualCog-Train and improve UAV dual-cognition performance through a lightweight optimization probe. Supplementary website, code, and dataset: https://smartdianlab.github.io/UAV-DualCog/
尚无 AI 解读，展示原文摘要