跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+2新增论文
+2其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-12 22:44 · 全站最近入库：2026-11-12 06:37各信源进度 →

统计口径与数据新鲜度

统计 2026-11-11 23:12 — 2026-11-12 23:12（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-12 22:44最近发现新增版本：2026-11-12 06:37库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,675 已收录论文18,323 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 5,654 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-12 22:37（北京时间）
首次观测 · 记录 #1CVF · CVPR 2026:cvf:74d6c0801560feeb8137111d6d939f97CVF · CVPR 2026

Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts

Pan, Hongkun, Wu, Yuwei, Hong, Wanyi, Hu, Shenghui, Yan, Qitong, Yang, Yi, Han, Rufei, Zhou, Changju 等 11 位作者

原文摘要

Multimodal large language models (MLLMs) have shown considerable potential in chart understanding and reasoning tasks. However, they still struggle with high information density (HID) charts characterized by multiple subplots, legends, and dense annotations due to three major challenges: (1) limited fine-grained perception results in the omission of critical visual cues; (2) redundant or noisy visual information undermines the performance of multimodal reasoning; (3) lack of adaptive deep reasoning relative to the amount of visual information. To tackle these challenges, we present a novel focus-driven fine-grained chart reasoning model, Chart-FR1, to improve perception, focusing efficiency, and adaptive deep reasoning on HID charts. Specifically, we propose Focus-CoT, a visual focusing chain-of-thought that enhances fine-grained perception by explicitly linking reasoning steps to key visual cues, such as local image regions and OCR signals. Building on this, we introduce Focus-GRPO, a focus-driven reinforcement learning algorithm with an information-efficiency reward that compresses redundant visual information for efficient focusing, and an adaptive KL penalty mechanism that enables flexible control over reasoning depth as more visual cues are discovered. Furthermore, to fill the gap in benchmarks for HID charts, we build HID-Chart, a challenging benchmark with an information-density metric designed to evaluate fine-grained chart reasoning capabilities. Extensive experiments on multiple chart benchmarks demonstrate that Chart-FR1 outperforms state-of-the-art MLLMs in chart understanding and reasoning. Code is available at https://github.com/phkhub/Chart-FR1.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:757220d9a3a2ff4e1bcd7abafe8c8d44CVF · CVPR 2026

OpenVoxel: Training-Free Grouping and Captioning Voxels for Open-Vocabulary 3D Scene Understanding

Huang, Sheng-Yu, Choe, Jaesung, Wang, Yu-Chiang Frank, Sun, Cheng

原文摘要

We propose OpenVoxel, a training-free algorithm for grouping and captioning sparse voxels for the open-vocabulary 3D scene understanding tasks. Given the sparse voxel rasterization (SVR) model obtained from multi-view images of a 3D scene, our OpenVoxel is able to produce meaningful groups that describe different objects in the scene. Also, by leveraging powerful Vision Language Models (VLMs) and Multi-modal Large Language Models (MLLMs), our OpenVoxel successfully build an informative scene map by captioning each group, enabling further 3D scene understanding tasks such as open-vocabulary segmentation (OVS) or referring expression segmentation (RES). Unlike previous methods, our method is training-free and does not introduce embeddings from a CLIP/BERT text encoder. Instead, we directly proceed with text-to-text search using MLLMs. Through extensive experiments, our method demonstrates superior performance compared to recent studies, particularly in complex referring expression segmentation (RES) tasks.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7596eb77843d57da386eda9a6a8219beCVF · CVPR 2026

Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR

Zhang, Yulong, Liang, Tianyi, Cui, Erfei, Wang, Guoqing, Guo, Xu, Li, Chenhui, Liu, Gongshen

原文摘要

Optical Character Recognition (OCR) is fundamental to Vision-Language Models (VLMs) and high-quality data generation for LLM training. Yet, despite progress in average OCR accuracy, state-of-the-art VLMs still struggle with detecting sample-level errors and lack effective unsupervised quality control.We introduce Consensus Entropy (CE), a training-free, model-agnostic metric that estimates output reliability by measuring inter-model agreement entropy. The core insight is that correct predictions converge in output space, while errors diverge. Based on CE, we develop CE-OCR, a lightweight multi-model framework that verifies outputs by ensemble agreement, selects the best outputs, and further improves efficiency through adaptive routing. Experiments demonstrate that CE is robust for quality verification, improving F1 scores by 42.1% over VLM-as-Judge. CE-OCR achieves consistent OCR gains, outperforming self-consistency and single-model baselines at the same cost. Notably, CE requires no training or supervision, enabling plug-and-play integration.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:75b45278d03d81bacdee82ccdd1109bbCVF · CVPR 2026

TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language Models

Ye, Jinlun, Liao, Jiang, Lai, Runhe, Lu, Xinhua, Zhuang, Jiaxin, Gan, Zhiyong, Wang, Ruixuan

原文摘要

Vision-language models (VLMs) such as CLIP exhibit strong Out-of-distribution (OOD) detection capabilities by aligning visual and textual representations. Recent CLIP-based test-time adaptation methods further improve detection performance by incorporating external OOD labels. However, such labels are finite and fixed, while the real OOD semantic space is inherently open-ended. Consequently, fixed labels fail to represent the diverse and evolving OOD semantics encountered in test streams.To address this limitation, we introduce Test-time Textual Learning (TTL), a framework that dynamically learns OOD textual semantics from unlabeled test streams, without relying on external OOD labels.TTL updates learnable prompts using pseudo-labeled test samples to capture emerging OOD knowledge. To suppress noise introduced by pseudo-labels, we introduce an OOD knowledge purification strategy that selects reliable OOD samples for adaptation while suppressing noise. In addition, TTL maintains an OOD Textual Knowledge Bank that stores high-quality textual features, providing stable score calibration across batches. Extensive experiments on two standard benchmarks with nine OOD datasets demonstrate that TTL consistently achieves state-of-the-art performance, highlighting the value of textual adaptation for robust test-time OOD detection. Our code is available at https://github.com/figec/TTL.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:75f51021057e086d5dd8c4b54d2c1f48CVF · CVPR 2026

MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction

Tang, Zitian, Zhang, Xu, Yuan, Jianbo, Zou, Yang, Gunjal, Varad, Jiang, Songyao, Modolo, Davide

原文摘要

Multimodal Large Language Models (MLLMs) have recently demonstrated promising capabilities in multimodal coding tasks such as chart-to-code generation. However, existing methods primarily rely on supervised fine-tuning (SFT), which requires the model to learn code patterns through chart-code pairs but does not expose the model to a code execution environment. Moreover, while self-correction through execution feedback offers a potential route to improve coding quality, even state-of-the-art MLLMs have been shown to struggle with effective self-correction. In this work, we introduce MM-ReCoder, a chart-to-code generation model trained with reinforcement learning (RL) and equipped with self-correction ability. We propose a two-stage multi-turn self-correction RL strategy based on Group Relative Policy Optimization (GRPO). The first stage enhances the model's self-correction ability via rolling out a shared first turn, while the second stage improves the coding capability with full-trajectory optimization. MM-ReCoder learns to produce more accurate and executable code through the interaction with the environment and by iteratively correcting its own outputs. Our results on three chart-to-code benchmarks demonstrate the state-of-the-art performance of MM-ReCoder.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:761d8577b2add38c4e7f58490566aa7dCVF · CVPR 2026

GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation

Li, Rang, Li, Lei, Ren, Shuhuai, Tian, Hao, Gu, Shuhao, Li, Shicheng, Yue, Zihao, Wang, Yudong 等 13 位作者

原文摘要

Visual grounding, localizing objects from natural language descriptions, represents a critical bridge between language and vision understanding. While multimodal large language models (MLLMs) achieve impressive scores on existing benchmarks, a fundamental question remains: can MLLMs truly visually ground with human-like sophistication, or are they merely pattern-matching on simplified datasets? Current benchmarks fail to capture real-world complexity where humans effortlessly navigate intricate references and recognize when grounding is impossible. To rigorously assess MLLMs' true capabilities, we introduce GroundingME, a benchmark that systematically challenges models across four critical dimensions: (1) Discriminative: distinguishing highly similar objects, (2) Spatial: understanding complex relational descriptions, (3) Limited: handling occlusions or tiny objects, and (4) Rejection: recognizing ungroundable queries. Through careful curation combining automated generation with human verification, we create 1,005 challenging examples mirroring real-world complexity. Evaluating 25 state-of-the-art MLLMs reveals a profound capability gap: the best model achieves only 45.1% accuracy, while most score 0% on rejection tasks. We explore two strategies for improvements: (1) test-time scaling selects optimal response by thinking trajectory to improve overall performance by up to 4.5%, and (2) data-mixture training boosts rejection accuracy from 0% to 27.9%. GroundingME thus serves as both a diagnostic tool revealing current limitations in MLLMs and a roadmap toward human-level visual grounding. Project page: https://groundingme.github.io

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:762b6b291a2b0a8753134192abc69e78CVF · CVPR 2026

MIBURI: Towards Expressive Interactive Gesture Synthesis

Mughal, M. Hamza, Dabral, Rishabh, Demberg, Vera, Theobalt, Christian

原文摘要

Embodied Conversational Agents (ECAs) aim to emulate human face-to-face interaction through speech, gestures, and facial expressions. Current large language model (LLM)-based conversational agents lack embodiment and the expressive gestures essential for natural interaction. Existing solutions for ECAs often produce rigid, low-diversity motions, that are unsuitable for human-like interaction. Alternatively, generative methods for co-speech gesture synthesis yield natural body gestures but depend on future speech context and require long run-times. To bridge this gap, we present MIBURI, the first online, causal framework for generating expressive full-body gestures and facial expressions synchronized with real-time spoken dialogue. We employ body-part aware gesture codecs that encode hierarchical motion details into multi-level discrete tokens. These tokens are then autoregressively generated by a two-dimensional causal framework conditioned on LLM-based speech-text embeddings, modeling both temporal dynamics and part-level motion hierarchy in real time. Further, we introduce auxiliary objectives to encourage expressive and diverse gestures while preventing convergence to static poses. Comparative evaluations demonstrate that our causal and real-time approach produces natural and contextually aligned gestures against recent baselines. We urge the reader to explore demo videos on https://vcai.mpi-inf.mpg.de/projects/MIBURI/

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:764575e93b017a43fa63b1e44ffefdb2CVF · CVPR 2026

Bridging Facial Understanding and Animation via Language Models

Song, Luchuan, Liu, Pinxin, Liu, Haiyang, Jin, Zhenchao, Tang, Yolo Yunlong, Xu, Zichong, Liang, Susan, Bi, Jing 等 10 位作者

原文摘要

Text-guided human body animation has advanced rapidly, yet facial animation lags due to the scarcity of well-annotated, text-paired facial corpora. To close this gap, we leverage foundation generative models to synthesize a large, balanced corpus of facial behavior. We design prompts suite covering emotions and head motions, generate about 80 hours of facial videos with multiple generators, and fit per-frame 3D facial parameters, yielding large-scale (prompt and parameter) pairs for training. Building on this dataset, we probe language models for bidirectional competence over facial motion via two complementary tasks: (1) Motion2Language: given a sequence of 3D facial parameters, the model produces natural-language descriptions capturing content, style, and dynamics; and (2) Language2Motion: given a prompt, the model synthesizes the corresponding sequence of 3D facial parameters via quantized motion tokens for downstream animation. Extensive experiments show that in this setting language models can both interpret and synthesize facial motion with strong generalization. To best of our knowledge, this is the first work to cast facial-parameter modeling as a language problem, establishing a unified path for text-conditioned facial animation and motion understanding.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7652ded2e64c52311910c657bc12f130CVF · CVPR 2026

LAM: Language Articulated Object Modelers

Gao, Yipeng, Ge, Yunhao, Cai, Peilin, Seita, Daniel, Itti, Laurent

原文摘要

We introduce LAM, a system that explores collaboration between large language models and vision-language models to generate articulated objects from text prompts without a visual prior or prebuilt 3D assets. In contrast, we formulate articulated object generation as a unified code-generation task, in which geometry and articulations can be co-designed from scratch. Given an input text, LAM coordinates a team of specialized modules to generate code to represent the desired articulated object procedurally. LAM first reasons about the hierarchical structure of parts (links) with Link Designer, then writes code, compiles it, and debugs it with Geometry and Articulation Coders and self-corrects with Geometry and Articulation Checkers. The code serves as a structured, interpretable bridge between individual links, ensuring the correct relationships among them. Experiments demonstrate the power of leveraging code as a generative medium within a collaboration system, showcasing its effectiveness in automatically constructing complex articulated objects.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7653b28d0fb04400b810cd974d2c2a7fCVF · CVPR 2026

Multimodal Continual Instruction Tuning with Dynamic Gradient Guidance

Li, Songze, Gao, Mingyu, Su, Tonghua, Zhang, Xu-Yao, Wang, Zhongjie

原文摘要

Multimodal continual instruction tuning enables multimodal large language models to sequentially adapt to new tasks while building upon previously acquired knowledge. However, this continual learning paradigm faces the significant challenge of catastrophic forgetting, where learning new tasks leads to performance degradation on previous ones. In this paper, we introduce a novel insight into catastrophic forgetting by conceptualizing it as a problem of missing gradients from old tasks during new task learning. Our approach approximates these missing gradients by leveraging the geometric properties of the parameter space, specifically using the directional vector between current parameters and previously optimal parameters as gradient guidance. This approximated gradient can be further integrated with real gradients from a limited replay buffer and regulated by a Bernoulli sampling strategy that dynamically balances model stability and plasticity. Extensive experiments on multimodal continual instruction tuning datasets demonstrate that our method achieves state-of-the-art performance without model expansion, effectively mitigating catastrophic forgetting while maintaining a compact architecture.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:76b11fc748e5a32cee8cf61558dc44bdCVF · CVPR 2026

ReasonX: MLLM-Guided Intrinsic Image Decomposition

Dirik, Alara, Wang, Tuanfeng Yang, Ceylan, Duygu, Zafeiriou, Stefanos, Frühstück, Anna

原文摘要

Intrinsic image decomposition aims to separate images into physical components such as albedo, depth, normals, and illumination. While recent diffusion- and transformer-based models benefit from paired supervision from synthetic datasets, their generalization to diverse, real-world scenarios remains challenging. We propose ReasonX, a novel framework that leverages a multimodal large language model (MLLM) as a perceptual judge providing relative intrinsic comparisons, and uses these comparisons as GRPO rewards for fine-tuning intrinsic decomposition models on unlabeled, in-the-wild images. Unlike RL methods for generative models, our framework aligns conditional intrinsic predictors by rewarding agreement between the judge's relational assessments and analytically derived relations from the model's outputs. ReasonX is model-agnostic and can be applied to different intrinsic predictors. Across multiple base architectures and modalities, ReasonX yields significant improvements, including 9-25% WHDR reduction on IIW albedo and up to 46% depth accuracy gains on ETH3D, highlighting the promise of MLLM-guided comparative supervision to bridge low- and high-level vision reasoning.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:76ef006389b678168b08c008c815a346CVF · CVPR 2026

RoadSceneBench: A Lightweight Benchmark for Mid-Level Road Scene Understanding

Liu, Xiyan, Wang, Han, Wang, Yuhu, Cai, Junjie, Cao, Zhe, Yang, Jianzhong, Lu, Zhen

原文摘要

Understanding mid-level road semantics, which capture the structural and contextual cues that link low-level perception to high-level planning, is essential for reliable autonomous driving and digital map construction. However, existing benchmarks primarily target perception tasks such as detection or segmentation, overlooking the reasoning capabilities required to infer road topology and dynamic scene structure. To address this gap, we present RoadSceneBench, a lightweight yet information-rich benchmark designed to evaluate and advance visual reasoning in complex road environments. Unlike large-scale perception datasets, RoadSceneBench emphasizes relational understanding and structural consistency, encouraging models to capture the underlying logic of real-world road scenes. Furthermore, to enhance reasoning reliability, we propose Hierarchical Relational Reward Propagation with Temporal Consistency (HRRP-T), a training framework for Vision-Language Models (VLMs) in which reward signals adaptively promote spatial coherence and semantic alignment throughout the reasoning process. This paradigm enables models to move beyond static recognition toward geometry-aware and temporally consistent reasoning. Extensive experiments demonstrate that our method achieves state-of-the-art performance across diverse road configurations. RoadSceneBench thus provides a compact yet powerful foundation for studying mid-level road semantics and fostering structure-aware autonomous perception. Our dataset is available at https://github.com/XiyanLiu/RoadSceneBench.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7746be7bdd58bcc02203a1cd5440d363CVF · CVPR 2026

LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding

Qiu, Jihao, Xie, Lingxi, Huo, Xinyue, Tian, Qi, Ye, Qixiang

原文摘要

This paper addresses the critical and underexplored challenge of long video understanding with low computational budgets.We propose LongVideo-R1, an active, reasoning-equipped multimodal large language model (MLLM) agent designed for efficient video context navigation, avoiding the redundancy of exhaustive search.At the core of LongVideo-R1 lies a reasoning module that leverages high-level visual cues to infer the most informative video clip for subsequent processing.During inference, the agent initiates traversal from top-level visual summaries and iteratively refines its focus, immediately halting the exploration process upon acquiring sufficient knowledge to answer the query.To facilitate training, we first extract hierarchical video captions from CGBench, a video corpus with grounding annotations, and guide GPT-5 to generate 33K high-quality chain-of-thought-with-tool trajectories. The LongVideo-R1 agent is fine-tuned upon the Qwen-3-8B model through a two-stage paradigm: supervised fine-tuning (SFT) followed by reinforcement learning (RL), where RL employs a specifically designed reward function to maximize selective and efficient clip navigation.Experiments on multiple long video benchmarks validate the effectiveness of name, which enjoys superior tradeoff between QA accuracy and efficiency. All curated data and source code are provided in the supplementary material and will be made publicly available.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7871482721fa3dd08c7d1f218b843810CVF · CVPR 2026

Paper2Figure: A Multi-Agent Collaborative System for Figure Generation Towards Academic Research Paper

Han, Siwei, Ji, Haonian, Xin, Siyang, Shi, Juanquan, Qiu, Shi, Ye, Xinyu, Xia, Peng, Liu, Jiaqi 等 13 位作者

原文摘要

Automatically generating clear and accurate figures for research papers remains challenging, as it requires semantic understanding, precise structure, and visual aesthetics. Existing approaches struggle to balance fidelity and quality: large language model (LLM) code-based methods (e.g., SVG, Mermaid) are structured but inflexible, while image-generation models (e.g., GPT-Image-1, Nano Banana) produce hard-to-edit and often inaccurate figures. We present Paper2Figure, a dual multi-agent system with an interactive web platform for paper-to-figure generation. Generation Agents convert text into our designed FigScript language, encoding figure semantics, styles and layout. The web system renders the FigScript into an initial image, which Refinement Agents iteratively analyze to locate issues and revise the FigScript for improved logic, alignment, aesthetics and text accuracy. Crucially, users can further refine results through an intuitive web interface, ensuring full control over the final output. To evaluate Paper2Figure, we introduce Paper2Figure Bench, a benchmark comprising 100 academic figures with paired descriptions. Experiments demonstrate that Paper2Figure markedly improves accuracy by 12%, beauty by 13.5%, and completeness by 17.0% over state-of-the-art baselines in fully automatic generation without human adjustment. By combining automated generation with interactive edit, Paper2Figure bridges the gap between AI assistance and researcher control, offering a practical solution for high-quality academic figure creation.

cs.CV · cs.CL · cs.MA

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:78fb1ca019d70a8a1ac932e37ff5f216CVF · CVPR 2026

ViStoryBench: Comprehensive Benchmark Suite for Story Visualization

Zhuang, Cailin, Huang, Ailin, Hu, Yaoqi, Wu, Jingwei, Cheng, Wei, Liao, Jiaqi, Wang, Hongyuan, Liao, Xinyao 等 15 位作者

原文摘要

Story visualization aims to generate coherent image sequences that faithfully represent a narrative and match given character references. Despite progress in generative models, existing benchmarks remain narrow in scope, often limited to short prompts, lacking character references, or single-image cases, failing to reflect real-world narrative complexity and obscuring true model performance. We introduce ViStoryBench, a comprehensive benchmark designed to evaluate story visualization models across varied narrative structures, visual styles, and character settings. It features richly annotated multi-shot scripts derived from curated stories spanning literature, film, and folklore. Large language models assist in story summarization and script generation, with all outputs verified by humans for coherence and fidelity. Character references are carefully curated to maintain consistency across different artistic styles. ViStoryBench proposes a suite of multi-dimensional automated metrics to evaluate character consistency, style similarity, prompt alignment, aesthetic quality, and artifacts like copy-paste behavior. These metrics are validated through human studies and used to assess a broad range of open-source and commercial models, enabling systematic analysis and encouraging advances in visual storytelling.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7a0739c07d62ceb97527d6c635675572CVF · CVPR 2026

Prompt-Anchored Vision-Text Distillation for Lifelong Person Re-identification

Wen, Wen, Chen, Hao, Zhang, Shiliang

原文摘要

Lifelong person re-identification (LReID) aims to train a generalizable model with sequentially collected data. However, such models often suffer from semantic drift, limited adaptability, and catastrophic forgetting as new domains emerge. Existing exemplar-free approaches largely rely on visual-only distillation or parameter regularization, while overlooking the potential of auxiliary modalities, such as text, to preserve semantic stability and enable incremental plasticity. We observe that the frozen text encoder in pretrained vision-language models can serve as a stable semantic anchor across domains. To decouple the roles of vision and text, we propose Prompt-Anchored vision-text Distillation (PAD), an asymmetric vision-text framework for semantic alignment and cross-domain generalization. On the textual side, we distill prompts to preserve vision-text alignment under a fixed semantic space, acting as a global semantic reference rather than a dominant learning signal. On the visual side, an EMA-based teacher with an adaptive prompt pool enables domain-wise adaptation by allocating new slots while freezing past ones. Extensive experiments show that PAD substantially outperforms state-of-the-art methods across seen and unseen domains, achieving a strong balance between stability and plasticity. Project page: https://github.com/zu-zi/PAD.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7a2c2a6c8e5c87b4e271dc743e6c3d03CVF · CVPR 2026

Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT

Liu, Yesheng, Li, Hao, Xu, Haiyu, Pei, Baoqi, Wang, Jiahao, Zhao, Mingxuan, Zheng, Jing-Shu, He, Zheqi 等 12 位作者

原文摘要

Multiple-choice question answering (MCQA) has been a popular format for evaluating and reinforcement fine-tuning (RFT) of modern multimodal language models. Its constrained output format allows for simplified, deterministic automatic verification.However, we find that the options may leak exploitable signals, which makes the accuracy metrics unreliable for indicating real capabilities and encourages explicit or implicit answer guessing behaviors during RFT. We propose ReVeL (Rewrite and Verify by LLM), a framework that rewrites multiple-choice questions into open-form questions while keeping answers verifiable whenever possible. The framework categorizes questions according to different answer types, apply different rewriting and verification schemes, respectively. When applied for RFT, we converted 20k MCQA examples and use GRPO to finetune Qwen2.5-VL models. Models trained on ReVeL-OpenQA match MCQA accuracy on multiple-choice benchmarks and improve OpenQA accuracy by about six percentage points, indicating better data efficiency and more robust reward signals than MCQA-based training. When used for evaluation, ReVeL also reveals up to 20 percentage points of score inflation in MCQA benchmarks (relative to OpenQA), improves judging accuracy, and reduces both cost and latency. We will release code and data publicly.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7a8f93e392aec610e0d9675be02314afCVF · CVPR 2026

Interactive Episodic Memory with User Feedback

Subedi, Nikesh, Bazzani, Loris, Al-Halah, Ziad

原文摘要

In episodic memory with natural language queries (EM-NLQ), a user may ask a question (e.g., "Where did I place the mug?") that requires searching a long egocentric video, captured from the user's perspective, to find the moment that answers it. However, queries can be ambiguous or incomplete, leading to incorrect responses. Current methods ignore this key aspect and address EM-NLQ in a one-shot setup, limiting their applicability in real-world scenarios. In this work, we address this gap and introduce the Episodic Memory with Questions and Feedback task (EM-QnF). Here, the user can provide feedback on the model's initial prediction or add more information (e.g., "Before this. I'm looking for the big blue mug not the white one"), helping the model refine its predictions interactively. To this end, we collect datasets for feedback-based interaction and propose a lightweight training scheme that avoids expensive sequential optimization. We also introduce a plug-and-play Feedback ALignment Module (FALM) that enables existing EM-NLQ models to incorporate user feedback effectively. Our approach significantly improves over the state of the art on three challenging benchmarks and is better than or competitive with commercial large vision-language models while remaining efficient. Evaluation with human-generated feedback shows that it generalizes well to real-world scenarios.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7acddf2239e9684900704e5328f68a0cCVF · CVPR 2026

Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow

Liu, Chengxin, Choi, Wonseok, Zhang, Chenshuang, Oh, Tae-Hyun

原文摘要

Vision-Language Models (VLMs) have demonstrated strong capability in a wide range of tasks such as visual recognition, document parsing, and visual grounding. Nevertheless, recent work shows that while VLMs often manage to capture the correct image region corresponding to the question, they do not necessarily produce the correct answers. In this work, we demonstrate that this misalignment could be attributed to suboptimal information flow within VLMs, where text tokens distribute too much attention to irrelevant visual tokens, leading to incorrect answers. Based on the observation, we show that modulating the information flow during inference can improve the perception capability of VLMs. The idea is that text tokens should only be associated with important visual tokens during decoding, eliminating the interference of irrelevant regions. To achieve this, we propose a token dynamics-based method to determine the importance of visual tokens, where visual tokens that exhibit distinct activation patterns during different decoding stages are viewed as important. We apply our approach to representative open-source VLMs and evaluate on various datasets, including visual question answering, visual grounding and counting, optical character recognition, and object hallucination. The results show that our approach significantly improves the performance of baselines. Project page: https://cxliu0.github.io/AIF/.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7aecaa7b4c5155a2fd6feebebf38e642CVF · CVPR 2026

Streamlined Open-Vocabulary Human-Object Interaction Detection

Sun, Chang, Liao, Dongliang, Ding, Changxing

原文摘要

Open-vocabulary human-object interaction (HOI) detection aims to localize and recognize all human-object interactions in an image, including those unseen during training. Existing approaches usually rely on the collaboration between a conventional HOI detector and a Vision-Language Model (VLM) to recognize unseen HOI categories. However, feature fusion in this paradigm is challenging due to significant gaps in cross-model representations. To address this issue, we introduce SL-HOI, a StreamLined open vocabulary HOI detection framework based solely on the powerful DINOv3 model. Our design leverages the complementary strengths of DINOv3's components: its backbone for fine-grained localization and its text-aligned vision head for open-vocabulary interaction classification. Moreover, to facilitate smooth cross-attention between the interaction queries and the vision head's output, we propose first feeding both the interaction queries and the backbone image tokens into the vision head, effectively bridging their representation gaps. All DINOv3 parameters in our approach are frozen, with only a small number of learnable parameters added, allowing a fast adaptation to the HOI detection task. Extensive experiments show that SL-HOI achieves state-of-the-art performance on both the SWiG-HOI and HICO-DET benchmarks, demonstrating the effectiveness of our streamlined model architecture. Code is available at https://github.com/MPI-Lab/SL-HOI.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7b24df1b7c203f2221ee42c94bfb8626CVF · CVPR 2026

MM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning

Song, Junha, Jo, Yongsik, Min, So Yeon, Xie, Quanting, Kim, Taehwan, Bisk, Yonatan, Choo, Jaegul

原文摘要

Systems such as video chatbots and navigation robots often depend on streaming image captioning to interpret visual inputs. Existing approaches typically employ large multimodal language models (MLLMs) for this purpose, but their substantial computational cost hinders practical application.This limitation motivates our development of a lightweight captioning model. Our investigation begins by replacing the large-scale language component in MLLMs with a compact 125M-parameter model.Surprisingly, this compact model, despite a 93x reduction in size, achieves comparable performance to MLLMs, suggesting that factual image captioning does not significantly require the complex reasoning abilities of LLMs. Despite this promising result, our lightweight model still lacks reliability. To address this, we draw inspiration from the human visual process: perceiving a global and coarse understanding of the scene before attending to finer details. Accordingly, we propose a multimodal self-refinement framework that guides the model to utilize features from salient regions, identified by referencing the previous coarse caption, and to produce a refined description. Experimental results demonstrate the superiority of our model in both single-sentence and detailed captioning, extending even to long-range video QA tasks.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7b3e2a0b73359857e60a05df6ab1b04eCVF · CVPR 2026

Stable and Efficient Single-Rollout RL for Multimodal Reasoning

Liu, Rui, Yu, Dian, Ke, Lei, Liu, Haolin, Zhou, Yujun, Liang, Zhenwen, Mi, Haitao, Tokekar, Pratap 等 9 位作者

原文摘要

Reinforcement Learning with Verifiable Rewards (RLVR) has become a key paradigm to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs). However, prevalent group-based algorithms such as GRPO require multi-rollout sampling for each prompt. While more efficient single-rollout variants have recently been explored in text-only settings, we find that they suffer from severe instability in multimodal contexts, often leading to training collapse. To address this sample efficiency-stability trade-off, we introduce MSSR (Multimodal Stabilized Single-Rollout), a group-free RLVR framework that achieves both stable optimization and effective multimodal reasoning performance. MSSR achieves this via an entropy-based advantage-shaping mechanism that adaptively regularizes advantage magnitudes, preventing collapse and maintaining training stability. While such mechanisms have been used in group-based RLVR, we show that in the multimodal single-rollout setting they are not merely beneficial but essential for stability. In in-distribution evaluations, MSSR demonstrates superior rollout sample efficiency, achieving similar validation accuracy with half the training steps. When trained for the same number of steps, MSSR's performance surpasses the group-based baseline and shows consistent generalization improvements across five diverse reasoning-intensive benchmarks. Together, these results demonstrate that MSSR enables stable, sample-efficient, and effective RLVR for complex multimodal reasoning tasks.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7b40c8d1419138f3811241a8bc1b2faaCVF · CVPR 2026

Authorize-on-Demand: Dynamic Authorization with Legality-Aware Intellectual Property Protection for VLMs

Wang, Lianyu, Wang, Meng, Fu, Huazhu, Zhang, Daoqiang

原文摘要

The rapid adoption of vision-language models (VLMs) has heightened the demand for robust intellectual property (IP) protection of these high-value pretrained models. Effective IP protection should proactively confine model deployment within authorized domains and prevent unauthorized transfers. However, existing methods rely on static training-time definitions, limiting flexibility in dynamic environments and often producing opaque responses to unauthorized inputs. To address these limitations, we propose a novel dynamic authorization with legality-aware intellectual property protection (AoD-IP) for VLMs, a framework that supports authorize-on-demand and legality-aware assessment. AoD-IP introduces a lightweight dynamic authorization module that enables flexible, user-controlled authorization, allowing users to actively specify or switch authorized domains on demand at deployment time. This enables the model to adapt seamlessly as application scenarios evolve and provides substantially greater extensibility than existing static-domain approaches. In addition, AoD-IP incorporates a dual-path inference mechanism that jointly predicts input legality-aware and task-specific outputs. Comprehensive experimental results on multiple cross-domain benchmarks demonstrate that AoD-IP maintains strong authorized-domain performance and reliable unauthorized detection, while supporting user-controlled authorization for adaptive deployment in dynamic environments.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7b7cec201f0af444bd65ccc56957d8d9CVF · CVPR 2026

Same or Not? Enhancing Visual Perception in Vision-Language Models

Marsili, Damiano, Mehta, Aditya, Lin, Ryan Y., Gkioxari, Georgia

原文摘要

Vision-language models (VLMs) excel at broad visual understanding but remain coarse-grained, exhibit visual biases, and miss subtle visual details. Existing training corpora reinforce this limitation by emphasizing general recognition ("Is it a cat or a dog?") over fine-grained perception. To address this, we introduce a new training corpus and task designed to enhance the perceptual abilities of VLMs. TWIN is a large-scale dataset of 561,000 image-pair queries that task models to determine whether two visually similar images depict the same object, encouraging attention to nuanced visual cues. The dataset spans a diverse range of everyday objects across contexts, viewpoints, and appearances. Fine-tuning VLMs on TWIN yields notable gains in fine-grained recognition, even on unseen domains such as art, animals, plants, and landmarks. To quantify these gains, we introduce FGVQA, a benchmark suite of 12,000 queries that repurposes fine-grained recognition and retrieval datasets from multiple domains. While existing VLMs struggle on FGVQA, when fine-tuned on TWIN they improve by up to 19.3%, without compromising performance on general VQA benchmarks. Finally, our TWIN dataset scales favorably with object annotations, and our analysis shows that scale is key to performance. We envision TWIN as a drop-in addition to open-source VLM training corpora, advancing perceptual precision of future models.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7c2458b4bcbf657951f9bab7a79d23c2CVF · CVPR 2026

Noise-Aware Few-Shot Learning through Bi-directional Multi-View Prompt Alignment

Niu, Lu, Xue, Cheng

原文摘要

Vision-language models offer strong few-shot capability through prompt tuning but remain vulnerable to noisy labels, which can corrupt prompts and degrade cross-modal alignment. Existing approaches struggle because they often lack the ability to model fine-grained semantic cues and to adaptively separate clean from noisy signals. To address these challenges, we propose NA-MVP, a framework for Noise-Aware few-shot learning through bi-directional Multi-View Prompt alignment. NA-MVP is built upon a key conceptual shift: robust prompt learning requires moving from global matching to region-aware alignment that explicitly distinguishes clean cues from noisy ones. To realize this, NA-MVP employs (1) multi-view prompts combined with unbalanced optimal transport to achieve fine-grained patch-to-prompt correspondence while suppressing unreliable regions; (2) a bi-directional prompt design that captures complementary clean-oriented and noise-aware cues, enabling the model to focus on stable semantics; and (3) an alignment-guided selective refinement strategy that uses optimal transport to correct only mislabeled samples while retaining reliable data. Experiments on synthetic and real-world noisy benchmarks demonstrate that NA-MVP consistently outperforms state-of-the-art baselines, confirming its effectiveness in enabling robust few-shot learning under noisy supervision.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7cf1482c692bcb5f6903e6e8147e5183CVF · CVPR 2026

Personalized Image Descriptions from Attention Sequences

Xue, Ruoyu, Le, Hieu, Xu, Jingyi, Mondal, Sounak, Leite, Abe, Zelinsky, Gregory, Hoai, Minh, Samaras, Dimitris

原文摘要

People can view the same image differently: they focus on different regions, objects, and details in varying orders and describe them in distinct linguistic styles. This leads to substantial variability in image descriptions. However, existing models for personalized image description focus on linguistic style alone, with no prior work leveraging individual viewing patterns. We address this gap by explicitly modeling personalized viewing behavior as a core factor in description generation. Our method, DEPER (DEscription-PERception persona encoder), learns a subject embedding that captures both linguistic style and viewing behavior, guided by an auxiliary attention-prediction task. A lightweight adapter aligns these embeddings with a frozen vision-language model, enabling few-shot personalization without retraining. Across four datasets spanning diverse viewing tasks and both short and detailed descriptions, DEPER achieves a 24% average improvement, showing that modeling personalized attention produces more human-aligned and high-quality descriptions. We posit that understanding how people see helps predict what they say; modeling human diversity in perception can improve both performance and human alignment in multi-modal systems. Code is available at: https://github.com/cvlab-stonybrook/Personalized-Image-Description

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7d274b80bd57555b45c914802ee61154CVF · CVPR 2026

ZINA: Multimodal Fine-grained Hallucination Detection and Editing

Wada, Yuiga, Matsuda, Kazuki, Sugiura, Komei, Neubig, Graham

原文摘要

Multimodal Large Language Models (MLLMs) often generate hallucinations, where the output deviates from the visual content. Given that these hallucinations can take diverse forms, detecting hallucinations at a fine-grained level is essential for comprehensive evaluation and analysis. To this end, we propose a novel task of multimodal fine-grained hallucination detection and editing for MLLMs. Moreover, we propose ZINA, a novel method that identifies hallucinated spans at a fine-grained level, classifies their error types into six categories, and suggests appropriate refinements. To train and evaluate models for this task, we constructed VisionHall, a dataset comprising 6.9k outputs from twelve MLLMs manually annotated by 211 annotators, and 20k synthetic samples generated using a graph-based method that captures dependencies among error types. We demonstrated that ZINA outperformed existing methods, including GPT-4o and Llama-3.2, in both detection and editing tasks.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7d61e1d18daf13e6358adf7979938735CVF · CVPR 2026

M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG

Anugraha, David, Irawan, Patrick Amadeus, Singh, Anshul, Lee, En-Shiun Annie, Winata, Genta Indra

原文摘要

Vision-language models (VLMs) have achieved strong performance in visual question answering (VQA), yet they remain constrained by static training data. Retrieval-Augmented Generation (RAG) mitigates this limitation by enabling access to up-to-date, culturally grounded, and multilingual information; however, multilingual multimodal RAG remains largely underexplored. We introduce M4-RAG, a massive-scale benchmark spanning 42 languages, 56 regional dialects and registers, and 189 countries, comprising over 80,000 culturally diverse image-question pairs for evaluating retrieval-augmented VQA across languages and modalities. To balance realism with reproducibility, we build a controlled retrieval environment containing millions of carefully curated multilingual documents relevant to the query domains, approximating real-world retrieval conditions while ensuring consistent experimentation. Our systematic evaluation reveals that although RAG consistently benefits smaller VLMs, it fails to scale to larger models and often even degrades their performance, exposing a critical mismatch between model size and current retrieval effectiveness. Our cross-lingual evaluations also reveal significant performance degradation when prompts or retrieved context are provided in non-English languages. The code, datasets, and evaluation protocols for M4-RAG are available as open-source at https://github.com/davidanugraha/M4-RAG.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7d67cfc3d0012238ef6e1c578fb92de3CVF · CVPR 2026

Semantic-Guided Global-Local Collaborative Prompt Learning for Few-Shot Class Incremental Learning

Yan, Yongxin, Chen, Weisen, Chen, Xingye, Shao, Yuanjie, Zuo, Zhengrong, Tan, Wenming, Ren, Wenqi, Gao, Changxin 等 9 位作者

原文摘要

Few-Shot Class-Incremental Learning (FSCIL) poses a critical challenge in machine learning, requiring models to continuously integrate novel classes with limited samples while preserving knowledge of previously seen classes. While existing FSCIL approaches have demonstrated promising results, they still suffer from catastrophic forgetting and few-shot overfitting due to the challenge of balancing old knowledge retention with new knowledge acquisition. To address these challenges, we propose an innovative Semantic-Guided Global-Local Collaborative Prompt Learning (SGLC) framework. Built upon powerful pre-trained Vision-Language Models (VLMs), the framework first introduces a dual-alignment mechanism: globally aligning visual features with visual-textual prototypes and locally aligning multi-view visual features with local textual attribute features, which facilitates effective knowledge learning while preserving existing knowledge via frozen prototypes of previous classes. Furthermore, to alleviate overfitting, we incorporate Large Language Models (LLMs) to generate semantically rich textual descriptions, which simultaneously guide both global and local prompt learning through knowledge distillation. Extensive experiments on the miniImageNet, CIFAR-100, and CUB200 datasets demonstrate that SGLC performs favorably against the state-of-the-art methods.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:7d9cdeb50138a9e5144dac2f1cc7135fCVF · CVPR 2026

UniChange: Unifying Change Detection with Multimodal Large Language Model

Zhang, Xu, Li, Danyang, Dong, Xiaohang, Wu, Tianhao, Yu, Hualong, Wang, Jianye, Li, Qicheng, Li, Xiang

原文摘要

Change detection (CD) is a fundamental task for monitoring and analysing land cover dynamics. While recent high performance models and high quality datasets have significantly advanced the field, a critical limitation persists. Current models typically acquire limited knowledge from single-type annotated data and cannot concurrently leverage diverse binary change detection (BCD) and semantic change detection (SCD) datasets. This constraint leads to poor generalisation and limited versatility. The recent advancements in Multimodal Large Language Models (MLLMs) introduce new possibilities for a unified CD framework. We leverage the language priors and unification capabilities of MLLMs to develop UniChange, the first MLLM-based unified change detection model. UniChange integrates generative language abilities with specialised CD functionalities. We introduce three special tokens: [T1], [T2], and [CHANGE], utilising their embeddings as the key to query variations. This approach successfully accommodates both BCD and SCD tasks. Furthermore, UniChange utilises text prompts to guide the identification of change categories, eliminating the reliance on predefined classification heads. This design allows UniChange to effectively acquire knowledge from multi-source datasets, even when their class definitions conflict. Experiments on four public benchmarks (WHU-CD, S2Looking, LEVIR-CD+, and SECOND) demonstrate SOTA performance, achieving IoU scores of 90.41, 53.04, 78.87, and 57.62, respectively, surpassing all previous methods. The code is available at https://github.com/NKU-HLT/UniChange.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗