跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+2新增论文
+2其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-13 06:05 · 全站最近入库：2026-11-12 06:37各信源进度 →

统计口径与数据新鲜度

统计 2026-11-12 06:28 — 2026-11-13 06:28（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-13 06:05最近发现新增版本：2026-11-12 06:37库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,675 已收录论文18,323 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 5,654 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-13 06:03（北京时间）
首次观测 · 记录 #1CVF · CVPR 2026:cvf:d97d205ff7d66d79573632b6e6c8aec7CVF · CVPR 2026

Role-SynthCLIP: A Role-Play Driven Diverse Synthetic Data Approach

Huangfu, Yuanxiang, Wang, Chaochao, Wang, Weilei

原文摘要

The effectiveness of Contrastive Language-Image Pre-training (CLIP) models critically depends on the semantic diversity and quality of their training data. However, while existing synthetic data generation methods primarily focus on increasing data volume, such emphasis often leads to limited semantic diversity and redundant or shallow captions. To address this limitation, we propose Role-SynthCLIP, a novel data synthesis framework that leverages multi-perspective role-playing prompts (e.g., a compositional analyst, an interpreter of image context) to guide Multimodal Large Language Models (MLLMs) in generating semantically diverse captions from distinct viewpoints. This mechanism enhances the semantic diversity and fine-grained image-text alignment of synthetic pairs by enriching each image with multiple complementary captions, while keeping the number of training images fixed. Experimental results demonstrate the effectiveness and efficiency of our method. A CLIP-B/16 model trained on only 1 million images achieves a Recall@1 of 64.1% on the MS COCO validation set, surpassing the best existing synthetic data baseline (trained on 5M pairs) by 2.8 percentage points. The code and trained models can be found at https://github.com/huangfu170/Role-SynthCLIP.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:daabb7ccc8f3baa7378ee036b123185eCVF · CVPR 2026

Fast Reasoning Segmentation for Images and Videos

Shen, Yiqing, Unberath, Mathias

原文摘要

Reasoning segmentation enables open-set object segmentation via implicit text queries, therefore serving as a foundation for embodied agents that should operate autonomously in real-world environments. However, existing methods for reasoning segmentation require multimodal large language models with billions of parameters that exceed the computational capabilities of edge devices that typically deploy the embodied AI systems. Distillation offers a pathway to compress these models while preserving their capabilities. Yet, existing distillation approaches fail to transfer the multi-step reasoning capabilities that reasoning segmentation demands, as they focus on matching output predictions and intermediate features rather than preserving reasoning chains. The emerging paradigm of reasoning over digital twin representations presents an opportunity for more effective distillation by re-framing the problem. Consequently, we propose FastReasonSeg, which employs digital twin representations that decouple perception from reasoning to enable more effective distillation. Our distillation scheme first relies on supervised fine-tuning on teacher-generated reasoning chains. Then it is followed by reinforcement fine-tuning with joint rewards evaluating both segmentation accuracy and reasoning quality alignment. Experiments on two video (JiTBench, RVTBench) and two image benchmarks (ReasonSeg, LLM-Seg40K) demonstrate that our FastReasonSeg achieves state-of-the-art reasoning segmentation performance. Moreover, the distilled 0.6B variant outperforms models with 20 times more parameters while achieving 7.79 FPS throughput with only 2.1GB memory consumption. This efficiency enables deployment in resource-constrained environments to enable real-time reasoning segmentation.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:daac897085efc3353833290cea813834CVF · CVPR 2026

Monet: Reasoning in Latent Visual Space Beyond Image and Language

Wang, Qixun, Shi, Yang, Wang, Yifei, Zhang, Yuanxing, Wan, Pengfei, Gai, Kun, Ying, Xianghua, Wang, Yisen

原文摘要

Thinking with images has emerged as an effective paradigm for advancing visual reasoning, extending beyond text-only chains of thought by injecting visual evidence into intermediate reasoning steps. However, existing methods fall short of human-like abstract visual thinking, as their flexibility is fundamentally limited by external tools. In this work, we introduce Monet, a training framework that enables multimodal large language models (MLLMs) to reason directly within the latent visual space by generating continuous embeddings that function as intermediate visual thoughts. We identify two core challenges in training MLLMs for latent visual reasoning--high computational cost in latent-vision alignment and insufficient supervision over latent embeddings, and address them with a three-stage distillation-based supervised fine-tuning (SFT) pipeline. We further reveal a limitation of applying GRPO to latent reasoning: it primarily enhances text-based reasoning rather than latent reasoning. To overcome this, we propose VLPO (Visual-latent Policy Optimization), a reinforcement learning method that explicitly incorporates latent embeddings into policy gradient updates.To support SFT, we construct Monet-SFT-125K, a high-quality text-image interleaved CoT dataset containing 125K real-world, chart, OCR, and geometry CoTs. Our model, Monet-7B, shows consistent gains across real-world perception and reasoning benchmarks and exhibits strong out-of-distribution generalization on challenging abstract visual reasoning tasks. We also empirically analyze the role of each training component and discuss our early unsuccessful attempts, providing insights for future developments in visual latent reasoning.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:dadda51a87f6e6a1d4615964077775f4CVF · CVPR 2026

Phrase-grounded APO for Improving Chest X-ray Report Generation

Mahmood, Raziuddin, Syeda-Mahmood, Tanveer

原文摘要

The deployment of automatic radiology report generator (RRG) models in clinical workflows is being hampered by the lack of factual correctness in the produced reports. Existing methods to improve the report generators use alignment approaches that require pairs of ground truth preferred and dis-preferred responses. As these are not available at inference time in clinical workflows, new alignment methods are needed to improve report quality at inference time. In this paper, we present a new phrase-grounded automatic preference optimization (APO) alignment method which offers such improvement during inference without needing additional ground truth. Specifically, the method generates surrogate ground truth preference data for alignment automatically from the RRG model response itself though fact-checking and LLM-prompted correction. We also develop a novel APO loss function that combines preference response alignment loss with phrasal grounding loss paying attention to both the description of the finding and its image location. We show that this method of alignment, on the average, improves the report quality at inference time by 30-40% across various SOTA report generators as tested on multi-institutional chest X-ray datasets.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:db75557cfb99b205faf9f26b98cd6b51CVF · CVPR 2026

Dynamic Token Reweighting for Robust Vision-Language Models

Jiang, Tanqiu, Liang, Jiacheng, Zhu, Rongyi, Zhou, Jiawei, Ma, Fenglong, Wang, Ting

原文摘要

Large vision-language models (VLMs) are highly vulnerable to multimodal jailbreak attacks that exploit visual-textual interactions to bypass safety guardrails. In this paper, we present DTR, a novel inference-time defense that mitigates multimodal jailbreak attacks through optimizing the model's key-value (KV) caches. Rather than relying on curated safety-specific data or costly image-to-text conversion, we introduce a new formulation of the safety-relevant distributional shift induced by the visual modality. This formulation enables DTR to dynamically adjust visual token weights, minimizing the impact of adversarial visual inputs while preserving the model's general capabilities and inference efficiency. Extensive evaluation across diverse VLMs and attack benchmarks demonstrates that DTR outperforms existing defenses in both attack robustness and benign-task performance, marking the first successful application of KV cache optimization for safety enhancement in multimodal foundation models. The code for replicating DTR is available at: https://github.com/TanqiuJiang/DTR.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:dbde0b34f454766d84b41cf4acca1e92CVF · CVPR 2026

RAISE: Requirement-Adaptive Evolutionary Refinement for Training-Free Text-to-Image Alignment

Jiang, Liyao, Chen, Ruichen, Gao, Chao, Niu, Di

原文摘要

Recent text-to-image (T2I) diffusion models achieve remarkable realism, yet faithful prompt-image alignment remains challenging, particularly for complex prompts with multiple objects, relations, and fine-grained attributes. Existing training-free inference-time scaling methods rely on fixed iteration budgets that cannot adapt to prompt difficulty, while reflection-tuned models require carefully curated reflection datasets and extensive joint fine-tuning of diffusion and vision-language models, often overfitting to reflection paths data and lacking transferability across models. We introduce RAISE (Requirement-Adaptive Self-Improving Evolution), a training-free, requirement-driven evolutionary framework for adaptive T2I generation. RAISE formulates image generation as a requirement-driven adaptive scaling process, evolving a population of candidates at inference time through a diverse set of refinement actions--including prompt rewriting, noise resampling, and instructional editing. Each generation is verified against a structured checklist of requirements, enabling the system to dynamically identify unsatisfied items and allocate further computation only where needed. This achieves adaptive test-time scaling that aligns computational effort with semantic query complexity. On GenEval and DrawBench, RAISE attains state-of-the-art alignment (0.94 overall GenEval) while incurring fewer generated samples (reduced by 30-40%) and VLM calls (reduced by 80%) than prior scaling and reflection-tuned baselines, demonstrating efficient, generalizable, and model-agnostic multi-round self-improvement. Code is available at https://github.com/LiyaoJiang1998/RAISE.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:dc38d90966b0d47f90b819b7dcd38663CVF · CVPR 2026

The Power of Prior: Training-Free Open-Vocabulary Semantic Segmentation with LLaVA

Zhang, Bingfeng, Yu, Siyue, Li, Hui, Lin, Jiahua, Wang, Wenwu, Xiao, Jimin

原文摘要

Multimodal Large Language Models (MLLMs) like LLaVA have demonstrated remarkable capabilities in multi-modal understanding and generation. This success motivates us to investigate whether the inherent prior knowledge embedded within such MLLMs contains sufficient spatial awareness for dense prediction tasks, without requiring any task-specific fine-tuning. Thus, in this paper, we explore the utilization of LLaVA for training-free open-vocabulary semantic segmentation. We discover that certain layers within the LLM part of LLaVA can generate localized features corresponding to given object classes. Building on this intrinsic capability, we design three modules: A question-answer pipeline to identify target classes in the image, a text-visual response module to extract initial reliable pixel-level activations for the target class, and a visual generation module to produce reliable refined prompts, which further serve as guidance for SAM to generate the predictions. Our LLaVA-based approach achieves new state-of-the-art performance on "Thing" category datasets, e.g., PASCAL VOC 2012 and COCO-object. Moreover, our method does not require explicit background class names, demonstrating its exceptional potential for handling open-world scenarios. Code is available at https://github.com/zbf1991/FSeg-LLaVA.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:dc7c6004a0c1113c3b12bd8d7db3e2b1CVF · CVPR 2026

CoT-Edit: Let CoT Guide Instruction Video Editing

Liang, Sen, Guan, Fengbin, Zhang, Youliang, Li, Xin, Chen, Zhibo

原文摘要

Text-driven instruction-based video editing in complex scenes remains challenging: purely textual prompts often fail to capture precise spatial relationships and physical constraints, resulting in target ambiguity and physically implausible outcomes. To address this, we propose a plan-guide-edit framework that explicitly bridges semantic intent and spatial execution. In our framework, a Chain-of-Thought (CoT)-enhanced multimodal large language model (MLLM) serves as a planner, performing structured reasoning over the video and instructions to derive a precise sequence of bounding boxes and attribute-enriched editing directives. These spatial priors then guide a box-conditioned mask generator, transforming ambiguous global retrieval into localized, context-aware refinement and producing masks that more accurately capture object scale, contact relationships, and placement. Building on these spatial and semantic signals, a diffusion-based editor integrates the masks, enriched instructions, and frame features to render high-fidelity edits that remain temporally coherent and spatially well aligned. Trained first in a modular manner and then jointly, our framework achieves superior performance with reduced data requirements, delivering precise localization in scenes with multiple similar objects and physically consistent object additions, and extensive experiments demonstrate state-of-the-art performance over multiple strong baseline methods. More details are available at: https://github.com/flying-sky999/CoT-Edit

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:dcd5de55ff95002bb6d9592c7ccb2d6fCVF · CVPR 2026

PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks

Cui, Cheng, Zhang, Yubo, Sun, Ting, Wang, Xueqing, Liu, Hongen, Lin, Manhui, Zhang, Yue, Gao, Tingquan 等 14 位作者

原文摘要

The advent of "OCR 2.0" and large-scale vision-language models (VLMs) has set new benchmarks in text recogni- tion. However, these unified architectures often come with significant computational demands, challenges in precise text localization within complex layouts, and a propen- sity for textual hallucinations. Revisiting the prevailing notion that model scale is the sole path to high accu- racy, this paper introduces PP-OCRv5, a meticulously op- timized, lightweight OCR system with merely 5 million pa- rameters. We demonstrate that PP-OCRv5 achieves per- formance competitive with many billion-parameter VLMs on standard OCR benchmarks, while offering superior lo- calization precision and reduced hallucinations. The cor- nerstone of our success lies not in architectural expansion but in a data-centric investigation. We systematically dis- sect the role of training data by quantifying three critical dimensions: data difficulty, data accuracy, and data diver- sity. Our extensive experiments reveal that with a sufficient volume of high-quality, accurately labeled, and diverse data, the performance ceiling for traditional, efficient two- stage OCR pipelines is far higher than commonly assumed. This work provides compelling evidence for the viability of lightweight, specialized models in the large-model era and offers practical insights into data curation for OCR. The source code and models are publicly available at https: //github.com/PaddlePaddle/PaddleOCR.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:dcf50588c05e974fa7d19b57e741754eCVF · CVPR 2026

Nonparametric Deep Fine-grained Clustering with Low-Rank Guided Vision-Language Model

Ye, Xulun, Wu, Benyu, Hong, Jie, Zhou, Kun

原文摘要

The scarcity of labeled fine-grained data presents a significant challenge for deep clustering. Vision-Language Models (VLMs) on existing coarse-grained datasets (characterized by high inter-class and low intra-class variance) struggle to capture the subtle distinctions essential for fine-grained categorization, leading to suboptimal clustering performance. To address this, we propose a novel framework that adapts VLMs for fine-grained clustering without requiring fine-grained labels. Our method steers the model to focus on discriminative fine-grained features by integrating a Bayesian nonparametric process with a tailored representation learning objective, which includes low-rank guidance and orthogonal guidance. This allows our model to dynamically discover clusters that reflect fine-grained categories. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on multiple fine-grained benchmarks. Code is available at https://github.com/HenryWells02/VLM-Fine-Clustering.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:dd02bc96b91287327f17e5212619763cCVF · CVPR 2026

Reconstructing CLIP for Open-Vocabulary Dense Perception

Liu, Yajie, Zhang, Jinjin, Liu, Qingjie, Huang, Di

原文摘要

Large-scale vision-language models (VLMs) such as CLIP have excelled in zero-shot image classification, yet they struggle to achieve the dense cross-modal alignment required by open-vocabulary dense perception (OVDP). While recent self-distillation methods address this by aligning dense features with the generalizable global semantics, a key question remains: how should such dense features be constructed to achieve optimal alignment? To address this, we propose DenseRC, a principled Dense Representations Construction framework that reconstructs CLIP for OVDP based on two key insights.First, by analyzing the internal semantics encoded in the global cls token, we identify that multi-layer value embeddings serve as an informative basis for dense features. Second, we reveal that spatial aggregation tends to amplify semantic misalignment. Motivated by this, we design a lightweight Head-Selective Gating (HSG) module that adaptively reweights feature heads according to their intrinsic heterogeneity, enabling discriminative and alignment-friendly dense representations construction. Extensive experiments demonstrate that DenseRC delivers consistent and substantial gains across OVDP tasks including object detection and semantic segmentation, setting new state-of-the-art performance on multiple benchmarks.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:de6353733d13303171502596b58306eaCVF · CVPR 2026

MA-Bench: Towards Fine-grained Micro-Action Understanding

Li, Kun, Gu, Jihao, Wang, Fei, Wu, Zhiliang, Fan, Hehe, Guo, Dan

原文摘要

With the rapid development of Multimodal Large Language Models (MLLMs), their potential in Micro-Action understanding, a vital role in human emotion analysis, remains unexplored due to the absence of specialized benchmarks. To tackle this issue, we present MA-Bench, a benchmark comprising 1,000 videos and a three-tier evaluation architecture that progressively examines micro-action perception, relational comprehension, and interpretive reasoning. MA-Bench contains 12,000 structured question-answer pairs, enabling systematic assessment of both recognition accuracy and action interpretation. The results of 23 representative MLLMs reveal that there are significant challenges in capturing motion granularity and fine-grained body-part dynamics. To address these challenges, we further construct MA-Bench-Train, a large-scale training corpus with 20.5K videos annotated with structured micro-action captions for fine-tuning MLLMs. The results of Qwen3-VL-8B fine-tuned on MA-Bench-Train show clear performance improvements across micro-action reasoning and explanation tasks. Our work aims to establish a foundation benchmark for advancing MLLMs in understanding subtle micro-action and human-related behaviors. Project Page: https://MA-Bench.github.io

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:de906c80d996060dfacf16f40213cb2bCVF · CVPR 2026

Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models

Zhang, Gengwei, Peng, Jie, Tan, Zhen, Qiu, Mufan, Mahjoub, Hossein Nourkhiz, Tadiparthi, Vaishnav, Lee, Kwonjoon, Zhang, Yanyong 等 9 位作者

原文摘要

The recent success of reinforcement learning (RL) in large reasoning models has inspired the growing adoption of RL for post-training Multimodal Large Language Models (MLLMs) to enhance their visual reasoning capabilities. Although many studies have reported improved performance, it remains unclear whether RL training truly enables models to learn from visual information. In this work, we propose the Hallucination-as-Cue Framework, an analytical framework designed to investigate the effects of RL-based post-training on multimodal reasoning models from the perspective of model hallucination. Specifically, we introduce hallucination-inductive, modality-specific corruptions that remove or replace essential information required to derive correct answers, thereby forcing the model to reason by hallucination. By applying these corruptions during both training and evaluation, our framework provides a unique perspective for diagnosing RL training dynamics and understanding the intrinsic properties of datasets. Through extensive experiments and analyses across multiple multimodal reasoning benchmarks, we reveal that the role of model hallucination for RL-training is more significant than previously recognized. For instance, we find that RL post-training under purely hallucination-inductive settings can still significantly improve models' reasoning performance, and in some cases even outperform standard training. These findings challenge prevailing assumptions about MLLM reasoning training and motivate the development of more modality-aware RL-based training designs.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:dee394d7b63266ca6d1602b49c0291a7CVF · CVPR 2026

PPM-CLIP: Probabilistic Prompt Modeling for Generalizable AI-Generated Image Detection

Wang, Xinyuan, Lai, Yingxin, Luo, Zhiming, Liu, Zhihui

原文摘要

The rapid rise of highly realistic AI-generated images necessitates reliable and generalizable detection methods. However, existing methods are constrained by their discriminative nature: by learning a single static decision boundary, they tend to memorize generator-specific artifacts and consequently fail to generalize to the unseen distributions of new generative models. To overcome this limitation, we propose PPM-CLIP, a new framework that shifts from static classification to conditional generative modeling based on the CLIP vision-language model. Instead of learning a fixed decision boundary, a Probabilistic Prompt Modeling (PPM) module is used as a generator that produces an adaptive distribution of prompts according to the input image. This allows the model to flexibly capture novel artifacts, rather than matching them against fixed templates. In addition, to enhance the visual encoder's sensitivity to subtle artifacts, a Patch-Wise Contrastive Learning (PWCL) strategy is introduced. Extensive experiments on Ojha, GenImage, and DRCT benchmarks demonstrate that our generative paradigm significantly outperforms state-of-the-art methods, especially in cross-domain detection.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:df3f68b2f21ca80f80aec73422783267CVF · CVPR 2026

Prototypical Action Reasoning Facilitated by Vision-Language Alignment for Egocentric Action Anticipation

Shao, Jiang, Zhao, Xinbo, Tuo, Wenyin, Zou, Xiaochun

原文摘要

Egocentric Action Anticipation aims to infer future actions from videos, which is crucial for embodied AI systems. However, its advancement is hindered by the inherent stochasticity of the future, which introduces significant prediction uncertainty. Prevailing methods typically adopt an end-to-end approach to model holistic spatiotemporal contexts, yet they often lack explicit semantic reasoning capabilities, making it difficult to handle open-ended future uncertainties. To address these challenges, we propose a Prototypical Action Reasoning Framework Facilitated by Vision-Language Alignment (PAR-VLA), which leverages the semantic alignment capability of vision-language models to learn disentangled visual prototypes for verbs and nouns. These prototypes serve as robust semantic anchors, transforming the unconstrained temporal prediction problem into a conditional forecasting task guided by well-defined semantic concepts. Our multi-stage framework first extracts visually-grounded and text-aligned prototype groups from a VLM, learning multiple prototypes per category to capture intra-class diversity. Subsequently, a novel Prototypical Context Reasoning-guided Verb-Noun Encoding branch dynamically retrieves the most relevant verb and noun concepts based on visual observations and explicitly models their interactions to guide temporal anticipation. Furthermore, we introduce Dual-Stream Symbiotic Predictive Decoders to more finely capture the interdependencies between verbs and nouns during the prediction process. Experiments Results demonstrate that PAR-VLA achieves state-of-the-art performance and exhibits a strong capability in dealing with future uncertainty.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e03f3cbea144b839566711b4b200134cCVF · CVPR 2026

FAPE-IR: Frequency-Aware Planning and Execution Framework for All-in-One Image Restoration

Liu, Jingren, Xu, Shuning, Yang, Qirui, Wang, Yun, Chen, Xiangyu, Ji, Zhong

原文摘要

All-in-One Image Restoration (AIO-IR) aims to develop a unified model that can handle multiple degradations under complex conditions. However, existing methods often rely on task-specific designs or latent routing strategies, making it hard to adapt to real-world scenarios with various degradations. We propose FAPE-IR, a Frequency-Aware Planning and Execution framework for image restoration. It uses a frozen Multimodal Large Language Model (MLLM) as a planner to analyze degraded images and generate concise, frequency-aware restoration plans. These plans guide a LoRA-based Mixture-of-Experts (LoRA-MoE) module within a diffusion-based executor, which dynamically selects high- or low-frequency experts, complemented by frequency features of the input image. To further improve restoration quality and reduce artifacts, we introduce adversarial training and a frequency regularization loss. By coupling semantic planning with frequency-based restoration, FAPE-IR offers a unified and interpretable solution for all-in-one image restoration. Extensive experiments show that FAPE-IR achieves state-of-the-art performance across seven restoration tasks and exhibits strong zero-shot generalization under mixed degradations. Code is available at https://github.com/Programmergg/FAPE-IR.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e07c17aa956bea5a50670d641d23cbdcCVF · CVPR 2026

Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models

Zhang, Huatian, Mao, Zhendong, Zhang, Lei, Zhang, Yongdong

原文摘要

Direct Preference Optimization (DPO) has proven to be an effective solution for mitigating hallucination in Multimodal Large Language Models (MLLMs) by learning from preference pairs. One of its key challenges lies in how to transfer the sequence-level preference into fine-grained supervision on visual fidelity. To safeguard vision-related tokens that are prone to hallucination, existing methods typically allocate training emphasis according to the model's self-assessed visual sensitivity signals. However, such sensitivity, estimated by a model still under training, introduces self-referential bias: reinforcing already well-learned visual cues while neglecting hard-to-perceive but critical details, thereby limiting deeper alignment. In this work, we propose an Uncertainty-aware Exploratory Direct Preference Optimization (UE-DPO) method for MLLMs, which enables the model to uncover its cognitive deficiencies and actively explore for self-correction, guided by token-level epistemic uncertainty. Specifically, we first quantify the uncertainty from the model's failure to ground token predictions in the given image. Then, based on an uncertainty-aware exploration intensity, we encourage more learning pressure on visually deficient tokens in preferred samples, and alleviate the over-penalization of beneficial knowledge in dispreferred samples. Further, we provide a theoretical justification for our method, and extensive experiments demonstrate its effectiveness and robustness.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e0aea2276fbecdd2b14660303fb361a4CVF · CVPR 2026

Towards Streaming Referring Video Segmentation via Large Language Model

Zhang, Wenkang, Yang, Kaicheng, An, Xiang, Li, Qiang, Feng, Ziyong, Yang, Wankou, Deng, Jiankang

原文摘要

Current referring video segmentation methods typically operate in an offline manner, where sparse frames are first selected for image-level referring segmentation, and the resulting masks are then propagated across the video. Although video sampling captures global context, its isolated processing steps not only complicate optimization but also restrict applicability to real-world streaming scenarios. In this paper, we propose a simple but efficient MLLM-based framework StreamingRVOS, which can extend image-level segmentation to video-level via a streaming pipeline without introducing extra parameters. Specifically, we employ a Semantic Embedding Recycling (SER) method to propagate temporal context across frames, enabling the model to perceive semantic representation in the video. Then, we propose an Online Mask Consistency Perception (OMCP) strategy to adaptively invoke the MLLM to re-perceive the current scene and regenerate the semantic embedding. We conduct extensive experiments on multiple downstream datasets to prove the effectiveness of StreamingRVOS. Compared to previous methods, our method achieves excellent performance in referring video segmentation (1B variant improves upon Sa2VA by 19.2 on the MeViS dataset), while operating at an average speed of 7 FPS under streaming inference on 1 xA800 GPU.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e18b690d11222c2e3db213ddba2928f7CVF · CVPR 2026

PureProof: Diffusion-Resistant Black-box Targeted Attack on Large Vision-Language Models

Cao, Yiming, Wang, Dong, Lyu, Xinqi, Xiao, Bin

原文摘要

Large Vision-Language Models (VLMs) are increasingly deployed across diverse applications, such as AI agents, yet remain vulnerable to targeted adversarial attacks. However, the practical robustness of such attacks often remains unclear with limited evaluation under defenses. Diffusion-based purification (DBP), a widely adopted black-box defense, effectively blocks current attacks by removing adversarial perturbations via generative diffusion. Prior DBP evasion attacks target white-box image classifiers and are ill-suited to VLMs, incurring high computational costs and gradient instability when adapted. In this paper, we present PureProof, a black-box targeted attack on VLMs resilient to DBP. It consists of three core components. Stochastic Reverse Alignment (SRA) guides adversarial optimization via single-step reverse prediction, avoiding costly full-trajectory backpropagation. Adaptive Re-noising Augmentation (ARA) mitigates diffusion stochasticity through timestep-adaptive re-noising. Self-Consistency Regularization (SCR) stabilizes optimization by promoting local temporal coherence. Extensive experiments on open-source and commercial VLMs show that PureProof consistently outperforms prior attacks against DBP and achieves strong noise resilience, revealing critical vulnerabilities in VLMs and offering broader safety implications for real-world VLM deployments, including emerging agentic settings.

cs.CV · cs.CL · cs.AI

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e1acfe02bf6f152505bbc951e99cf880CVF · CVPR 2026

ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectors

Kuang, Liming, Velikova, Yordanka, Saleh, Mahdi, Zaech, Jan-Nico, Paudel, Danda Pani, Busam, Benjamin

原文摘要

Object pose estimation is a fundamental task in computer vision and robotics, yet most methods require extensive, dataset-specific training. Concurrently, large-scale vision language models show remarkable zero-shot capabilities. In this work, we bridge these two worlds by introducing ConceptPose, a framework for object pose estimation that is both training-free and model-free. ConceptPose leverages a vision-language-model (VLM) to create open-vocabulary 3D concept maps, where each point is tagged with a concept vector derived from saliency maps. By establishing robust 3D-3D correspondences across concept maps, our approach allows precise estimation of 6DoF relative pose. Without any object or dataset-specific training, our approach achieves state-of-the-art results on common zero shot relative pose estimation benchmarks, outperforming the strongest baseline by a relative 62% in average ADD(-S) score, including methods that utilize extensive dataset-specific training.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e1c7c2b0ba804bf0304d90a863fa720eCVF · CVPR 2026

Pointer-CAD: Unifying B-Rep and Command Sequences via Pointer-based Edges & Faces Selection

Qi, Dacheng, Wang, Chenyu, Xu, Jingwei, Chu, Tianzhe, Zhao, Zibo, Liu, Wen, Ding, Wenrui, Ma, Yi 等 9 位作者

原文摘要

Constructing computer-aided design (CAD) models is labor-intensive but essential for engineering and manufacturing. Recent advances in Large Language Models (LLMs) have inspired the LLM-based CAD generation by representing CAD as command sequences. But these methods struggle in practical scenarios because command sequence representation does not support entity selection (e.g. faces or edges), limiting its ability to support complex editing operations such as chamfer or fillet. Further, the discretization of a continuous variable during sketch and extrude operations may result in topological errors. To address these limitations, we present Pointer-CAD, a novel LLM-based CAD generation framework that leverages a pointer-based command sequence representation to explicitly incorporate the geometric information of B-rep models into sequential modeling. In particular, Pointer-CAD decomposes CAD model generation into steps, conditioning the generation of each subsequent step on both the textual description and the B-rep generated from previous steps. Whenever an operation requires the selection of a specific geometric entity, the LLM predicts a Pointer that selects the most feature-consistent candidate from the available set. Such a selection operation also reduces the quantization error in the command sequence-based representation. To support the training of Pointer-CAD, we develop a data annotation pipeline that produces expert-level natural language descriptions and apply it to build a dataset of approximately 575K CAD models. Extensive experimental results demonstrate that Pointer-CAD effectively supports the generation of complex geometric structures and reduces segmentation error to an extremely low level, achieving a significant improvement over prior command sequence methods, thereby significantly mitigating the topological inaccuracies introduced by quantization error. Our code is available at https://github.com/Snitro/Pointer-CAD.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e1ec3d4be4fb9610e492c2bb72a9159dCVF · CVPR 2026

ORIC: Benchmarking Object Recognition under Contextual Incongruity in Large Vision-Language Models

Li, Zhaoyang, Ling, Zhan, Zhou, Yuchen, Gong, Litian, Biyik, Erdem, Su, Hao

原文摘要

Large Vision-Language Models (LVLMs) excel at captioning, visual question answering, and robotics by combining vision and language, yet they often miss obvious objects or hallucinate nonexistent ones in atypical scenes. We examine these failures through the lens of uncertainty, focusing on contextual incongruity, where objects appear unexpectedly or fail to appear in expected contexts, and show that such cases increase recognition difficulty for state-of-the- art LVLMs. To study this regime, we introduce the Object Recognition in Incongruous Context (ORIC) framework, which constructs incongruous object-context pairs through two complementary strategies: (1) LLM-guided sampling to identify hard-to-recognize objects present in the image and (2) CLIP-guided sampling to mine plausible but absent ones. Applied to MSCOCO, ORIC creates ORIC-Bench and ORIC-style training data. Evaluating 18 LVLMs and 2 open-vocabulary detectors reveals significant degradation and bias under incongruous contexts. Visual Reinforcement Fine-Tuning of Qwen3-VL-8B-Instruct on 600 ORIC sam- ples improves performance on ORIC-Bench, AMBER, and HallusionBench. Overall, we show that contextual incongruity is a key source of uncertainty and provide tools for more reliable LVLMs. The dataset and code are publicly available at https://github.com/ZhaoyangLi-1/ORIC.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e208552d9d542b22a7ac14e62d63b343CVF · CVPR 2026

IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting

Zhang, Tao, Hong, Yuyang, Xia, Yang, Ding, Kun, Zhang, Zeyu, Wang, Ying, Xiang, Shiming, Pan, Chunhong

原文摘要

Recent advances in multimodal large language models (MLLMs) have led to impressive progress across various benchmarks. However, their capability in understanding infrared images remains unexplored. To address this gap, we introduce **IF-Bench**, the first high-quality benchmark designed for evaluating multimodal understanding of infrared images. IF-Bench consists of 499 images sourced from 23 infrared datasets and 680 carefully curated visual question-answer pairs, covering 10 essential dimensions of image understanding. Based on this benchmark, we systematically evaluate over 40 open-source and closed-source MLLMs, employing cyclic evaluation, bilingual assessment, and hybrid judgment strategies to enhance the reliability of the results. Our analysis reveals how model scale, architecture, and inference paradigms affect infrared image comprehension, providing valuable insights for this area. Furthermore, we propose a training-free generative visual prompting (**GenViP**) method, which leverages advanced image editing models to translate infrared images into semantically and spatially aligned RGB counterparts, thereby mitigating domain distribution shifts. Extensive experiments demonstrate that our method consistently yields significant performance improvements across a wide range of MLLMs.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e249d3e9cdd89edc953b21e15aa83226CVF · CVPR 2026

Incentivizing Versatile Video Reasoning in MLLMs via Data-Efficient Reinforcement Learning

Wang, Xiaodong, Wu, Zhirong, Huang, Langling, Zheng, Yuxi, Peng, Peixi

原文摘要

Multimodal Large Language Models (MLLMs) have made great progress in video understanding tasks. However, when it comes to understanding complex or lengthy videos, MLLMs tend to overlook details or produce hallucinations. To alleviate these issues, recent work has attempted to leverage reinforcement learning (RL) to boost models' deep linguistic reasoning of complex videos. But these methods have two main problems: First, the RL framework they used has unstable training, high training costs, and is difficult to train satisfactory video reasoning models; Second, the linguistic reasoning process is difficult to guarantee the reliability of visual information. To alleviate these problems, we propose to use multimodal elements for reasoning, and we design a novel framework to build and enhance versatile video reasoning capabilities on MLLMs. We carefully design a multi-task cold start and multi-task reinforcement learning to improve the model's visual perception and proficiency in multiple capabilities. In the inference phase, we leverage multimodal reasoning and dynamic sampling to further improve the performance. We verified the efficiency of the framework on a base MLLM (Qwen2-VL-7B-Base). Through cold-start with 3k data and reinforcement learning training with 5k data, combined with inference design, our final model significantly outperforms the base model on seven public video benchmarks, even surpassing and approaching the state-of-the-art Instruct Models such as Qwen2.5-VL-7B-Instruct trained with large-scale data. Our code will be available at https://github.com/Wang-Xiaodong1899/VideoReasoner.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e37d8455044f7263b803f1fb77c7d577CVF · CVPR 2026

Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models

Li, Hengzhuang, Zhang, Xinsong, Peng, Qiming, Luo, Bin, Hu, Han, Jiang, Dengyang, Ye, Han-Jia, Zhang, Teng 等 9 位作者

原文摘要

Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in multimodal tasks.Despite their impressive performance, MLLMs suffer from the modality imbalance issue, where visual information is often underutilized compared to textual representations in deeper layers, leading to degraded visual performance or hallucinations.This issue stems from the predominant reliance on next-text-token-prediction during training, which fails to provide direct visual supervisory signals, resulting in progressive homogenization of visual representations throughout the layers.To this end, we propose Latent Visual Reconstruction (LaVer), a novel training framework that facilitates MLLMs in learning more discriminative visual representations via masked image modeling in the joint latent semantic space of LLM.Our method offers direct visual activation to MLLMs, which exhibit increased visual attention allocation, indicating enhanced utilization of visual information.Extensive experiments across diverse benchmarks prove the superiority of our approach in various scenarios, especially those requiring dense visual capabilities.Code will be publicly available upon publication.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e42c02ccce24fa2cc0e73572fc6cf550CVF · CVPR 2026

R2-Seg: Training-Free OOD Medical Tumor Segmentation via Anatomical Reasoning and Statistical Rejection

Shen, Shuaike, Liu, Ke, Xie, Jiaqing, Gao, Shangde, Shen, Chunhua, Liu, Ge, Crispin-Ortuzar, Mireia, Gao, Shangqi

原文摘要

Foundation models for medical image segmentation struggle under out-of-distribution (OOD) shifts, often producing fragmented false positives on OOD tumors. We introduce **R^2-Seg**, a **training-free** framework for robust OOD tumor segmentation that operates via a two-stage **Reason-and-Reject** process. First, the **Reason** step employs an LLM-guided anatomical reasoning planner to localize organ anchors and generate multi-scale ROIs. Second, the **Reject** step applies two-sample statistical testing to candidates generated by a frozen foundation model (BiomedParse) within these ROIs. This statistical rejection filter retains only candidates significantly different from normal tissue, effectively suppressing false positives. Our framework requires no parameter updates, making it compatible with zero-update test-time augmentation and avoiding catastrophic forgetting. On multi-center and multi-modal tumor segmentation benchmarks, **R^2-Seg** substantially improves Dice, specificity, and sensitivity over strong baselines and the original foundation models.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e456af22f7d93c5d9cdeaddcb9c97bdeCVF · CVPR 2026

PointThinker: Point-Incentivized Parallel Thinking for Multimodal Large Language Model

Hu, Zhengdong, Wang, Chao, Rao, Fengyun, LYU, Jing, Fan, Hehe, Yang, Yi

原文摘要

This paper explores parallel thinking for Multi-modal Large Language Models (MLLMs), aiming to improve Chain-of-Thought (CoT) through multiple diverse reasoning paths. We guide the model to list multiple visual key points and develop an independent reasoning path for each. Therefore, we term this method PointThinker, which is characterized by starting each thinking path with a point. PointThinker offers two key advantages. (1) It amplifies the benefits of parallel thinking. While parallel thinking naturally benefits from multiple reasoning paths, explicitly listing key points further amplifies these benefits by eliminating redundancy and promoting path diversity, enabling the model to explore problems from more varied perspectives. (2) It uses a novel dense (point-wise) reward for reinforcement learning. We observe that during parallel thinking, some points are helpful while others are invalid, yet popular methods assign them the same rewards. Therefore, we propose allocating differentiated rewards to different points within the same chain-of-thought. This is implemented via a self-verification mechanism called Group Points Policy Optimization (GPPO), which combines rollout-level and point-level validation for reward assignment. On challenging benchmarks such as HallusionBench, PointThinker achieves 58.7% accuracy, improving reasoning quality and answer accuracy. Experimental results demonstrate that parallel thinking with point improves performance, and GPPO further contributes non-trivial gains.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e47f22ce2a4135b30f65a4882ddb48e7CVF · CVPR 2026

Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection

Fu, Jinhu, Lou, Yihang, Si, Qingyi, Zhang, Shudong, Su, Sen

原文摘要

Large Vision-Language Models (LVLMs) have achieved impressive performance across multimodal understanding and reasoning tasks, yet their internal safety mechanisms remain opaque and poorly controlled. In this work, we present a comprehensive framework for diagnosing and repairing unsafe channels within LVLMs. We first perform causal mediation analysis to identify neurons and layers that are causally responsible for unsafe behaviors. Based on these findings, we introduce a dual-modal safety subspace projection method that learns generalized safety subspaces for both visual and textual modalities through generalized eigen-decomposition between benign and malicious activations. During inference, activations are dynamically projected toward these safety subspaces via a hybrid fusion mechanism that adaptively balances visual and textual corrections, effectively suppressing unsafe features while preserving semantic fidelity.Extensive experiments on multiple LVLM safety benchmarks demonstrate that our causal-subspace repair framework significantly enhances safety robustness without degrading general multimodal capabilities, outperforming prior activation steering and alignment-based baselines. Moreover, our approach demonstrates robust transferability, effectively defending against unseen and adaptive attacks.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e480b2567a25cfd76c0216dabdf39db6CVF · CVPR 2026

Make it SING: Analyzing Semantic Invariants in Classifiers

Yadid, Harel, Levi, Meir Yossef, Betser, Roy, Gilboa, Guy

原文摘要

All classifiers, including state-of-the-art vision models, possess invariants, partially rooted in the geometry of their linear mappings. These invariants, which reside in the null-space of the classifier, induce equivalent sets of inputs that map to identical outputs. The semantic content of these invariants remains vague, as existing approaches struggle to provide human-interpretable information. To address this gap, we present Semantic Interpretation of the Null-space Geometry (SING), a method that constructs equivalent images, with respect to the network, and assigns semantic interpretations to the available variations. We use a mapping from network features to multi-modal vision language models. This allows us to obtain natural language descriptions and visual examples of the induced semantic shifts. SING can be applied to a single image, uncovering local invariants, or to sets of images, allowing a breadth of statistical analysis at the class and model levels. For example, our method reveals that ResNet50 leaks relevant semantic attributes to the null space, whereas DINO-ViT, a ViT pretrained with self-supervised DINO, is superior in maintaining class semantics across the invariant space. Code is available at https://tinyurl.com/github-SING.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e4ca2f71d368769e64c599ac28f6143cCVF · CVPR 2026

ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving

Peng, Qihang, Chen, Xuesong, Yang, Chenye, Shi, Shaoshuai, Li, Hongsheng

原文摘要

Autonomous driving requires generating safe and reliable trajectories from complex multimodal inputs. Traditional modular pipelines separate perception, prediction, and planning, while recent end-to-end (E2E) systems learn them jointly. Vision-language models (VLMs) further enrich this paradigm by introducing cross-modal priors and commonsense reasoning, yet current VLM-based planners face three key challenges: (i) a mismatch between discrete text reasoning and continuous control, (ii) high latency from autoregressive chain-of-thought decoding, and (iii) inefficient or non-causal planners that limit real-time deployment. We propose ColaVLA, a unified vision-language-action framework that transfers reasoning from text to a unified latent space and couples it with a hierarchical, parallel trajectory decoder. The Cognitive Latent Reasoner compresses scene understanding into compact, decision-oriented meta-action embeddings through ego-adaptive selection and only two VLM forward passes. The Hierarchical Parallel Planner then generates multi-scale, causality-consistent trajectories in a single forward pass. Together, these components preserve the generalization and interpretability of VLMs while enabling efficient, accurate and safe trajectory generation. Experiments on the nuScenes benchmark show that ColaVLA achieves state-of-the-art performance in both open-loop and closed-loop settings with favorable efficiency and robustness.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗