跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1新增论文
+1其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-13 07:07 · 全站最近入库：2026-11-13 06:39各信源进度 →

统计口径与数据新鲜度

统计 2026-11-12 07:26 — 2026-11-13 07:26（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-13 07:07最近发现新增版本：2026-11-13 06:39库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,676 已收录论文18,324 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 5,655 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-13 07:05（北京时间）
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e4ca2f71d368769e64c599ac28f6143cCVF · CVPR 2026

ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving

Peng, Qihang, Chen, Xuesong, Yang, Chenye, Shi, Shaoshuai, Li, Hongsheng

原文摘要

Autonomous driving requires generating safe and reliable trajectories from complex multimodal inputs. Traditional modular pipelines separate perception, prediction, and planning, while recent end-to-end (E2E) systems learn them jointly. Vision-language models (VLMs) further enrich this paradigm by introducing cross-modal priors and commonsense reasoning, yet current VLM-based planners face three key challenges: (i) a mismatch between discrete text reasoning and continuous control, (ii) high latency from autoregressive chain-of-thought decoding, and (iii) inefficient or non-causal planners that limit real-time deployment. We propose ColaVLA, a unified vision-language-action framework that transfers reasoning from text to a unified latent space and couples it with a hierarchical, parallel trajectory decoder. The Cognitive Latent Reasoner compresses scene understanding into compact, decision-oriented meta-action embeddings through ego-adaptive selection and only two VLM forward passes. The Hierarchical Parallel Planner then generates multi-scale, causality-consistent trajectories in a single forward pass. Together, these components preserve the generalization and interpretability of VLMs while enabling efficient, accurate and safe trajectory generation. Experiments on the nuScenes benchmark show that ColaVLA achieves state-of-the-art performance in both open-loop and closed-loop settings with favorable efficiency and robustness.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e551fda5a55b2f237c4fe317c2421479CVF · CVPR 2026

From Failure to Feedback: Group Revision Unlocks Hard Cases in Object-Level Grounding

Liu, Yuyuan, Ji, Yiping, Le, Anjie, Zhu, Jiayuan, Pan, Jiazhen, Peng, Can, Deng, Jiajun, Liu, Fengbei 等 9 位作者

原文摘要

Finetuning Large Vision-Language Models with reinforcement learning has emerged as a promising approach to enhance their capability in object-level grounding. However, existing methods, mainly based on GRPO, assign rewards at the response level. Such sparse reward leads to minimal learning signals when all candidate responses are failed in challenging scenarios.In this work, we propose a group-revision optimisation paradigm that enhances learning on hard cases. It begins with a sampled initial response and generates a set of revised candidates to explore improved grounding outcomes. Inspired by reward shaping, we introduce a consolidation process that quantifies each candidate's improvement over the initial attempt and converts it into informative shaping signals.These signals are used to both refine the reward and modulate the advantage, amplifying the influence of high-quality revisions.Our method achieves consistent gains across referring and reasoning segmentation, REC, and counting benchmarks compared with prior GRPO-based models. Code is available at https://github.com/yyliu01/GroupRevision.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e55c616ea700dd8a88c491f0a730adc3CVF · CVPR 2026

Learning What Matters: Prioritized Concept Learning via Relative Error-driven Sample Selection

Yang, Qian, Chandhok, Shivam, Mañas, Oscar, Jain, Kanishk, Agrawal, Aishwarya, Sigal, Leonid

原文摘要

Instruction tuning has been central to the success of recent vision-language models (VLMs), but it remains expensive-requiring large-scale datasets, high-quality annotations, and large compute budgets. We propose PRioritized cOncept learninG via Relative Error-driven Sample Selection (PROGRESS), a data- and compute-efficient framework that enables VLMs to dynamically select what to learn next based on their evolving needs during training. At each stage, the model tracks its learning progress across skills and selects the most informative samples-those it has not already mastered and that are not too difficult to learn at the current stage of training. This strategy effectively controls skill acquisition and the order in which skills are learned. Specifically, we sample from skills showing the highest learning progress, prioritizing those with the most rapid improvement. Unlike prior methods, PROGRESS requires no upfront answer annotations, queries answers only on a need basis, avoids reliance on additional supervision from auxiliary VLMs, and does not require compute-heavy gradient computations for data selection. Experiments across multiple instruction-tuning datasets of varying scales demonstrate that PROGRESS consistently outperforms state-of-the-art baselines with much less data and supervision. Additionally, we show strong cross-architecture generalization and transferability to larger models, validating PROGRESS as a scalable solution for efficient learning.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e55cca754751681bc3e465612150782fCVF · CVPR 2026

SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models

Zhao, Yifei, Lou, Qian, Zheng, Mengxin

原文摘要

The public accessibility of Large Vision-Language Models (LVLMs) raises serious concerns about unauthorized model reuse and intellectual property infringement. Existing ownership verification approaches often rely on semantically abnormal queries or out-of-distribution responses as fingerprints, which are easily recognized and removed by adversaries. We first expose this vulnerability through the Semantic Divergence Attack (SDA), which detects and filters fingerprint checks by measuring semantic divergence between a stolen model and a reference model, showing that existing fingerprints are not semantic-preserving, easy to detect and bypass, and lacking robustness. To address these weaknesses, we propose SIF (Semantically In-Distribution Fingerprints), a non-intrusive ownership verification framework requiring no parameter modification. SIF introduces Semantic-Aligned Fingerprint Distillation (SAFD), which distills text-generation watermark signals--originally designed for text ownership protection rather than model protection--into the visual modality, enabling semantically coherent yet fingerprinted responses. Robust-Fingerprint Optimization (RFO) further simulates worst-case representation perturbations, ensuring resilience to model modifications such as fine-tuning and quantization. Extensive experiments on LLaVA-1.5 and Qwen2.5-VL demonstrate that SIF achieves superior stealthiness and robustness, providing a practical solution for LVLM copyright protection.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e5ba97bd2e93d43e61e2f8de166cd63dCVF · CVPR 2026

Joint Learning of General and Diverse Patterns with Mixture of Memory Experts for Weakly-Supervised Video Anomaly Detection

Sun, Bo, Chen, Junxi, Wu, Zhe, Gao, Feng, Yang, Fan, Su, Li, Wang, Yaowei

原文摘要

Weakly-supervised Video Anomaly Detection (wVAD) aims to detect abnormal events using only binary labels, making it challenging to capture both the diversity of anomalies and their shared semantic cues. Existing methods either focus on a generic anomaly pattern, achieving strong generalization but weak discrimination, or rely on class-level diversity modeling, which ignores shared semantics and suffers from limited generalization. To overcome these limitations, we propose the Mixture of Memory Experts (MoME), a unified framework that jointly learns general and diverse patterns. Each expert in MoME possesses an internal memory for fine-grained specialization and shares an external memory for general knowledge aggregation. To enhance semantic diversity and improve generalization beyond coarse class-level supervision, we introduce an Anomaly Prototype Router that leverages large language models to construct generalized anomaly prototypes for semantically guided expert routing. Moreover, balanced routing, expert diversity, and enhanced pattern discriminability are ensured by the regularization loss for APR, the distinctiveness loss for experts, and reconstruction together with memory tasks, respectively. Extensive experiments on UCF-Crime and XD-Violence demonstrate that our approach achieves state-of-the-art performance, validating the effectiveness of jointly modeling generality and diversity for robust anomaly detection.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e5e362e4b25f2caf450b08430cd5b008CVF · CVPR 2026

VLM4RSDet: Collaborative Optimization with Vision-Language Model for Enhancing Remote Sensing Object Detection

Shi, Shuohao, Fang, Qiang, Xu, Xin

原文摘要

Closed-set object detection in remote sensing imagery has made significant progress, but achieving high detection accuracy remains challenging. Vision-Language Models (VLMs), which possess rich prior knowledge, offer a promising solution to this challenge. However, most existing VLMs are designed for open-vocabulary tasks and exhibit inherent limitations when directly applied to closed-set scenarios, such as notable accuracy degradation and high deployment costs. To address these issues, we propose VLM4RSDet, a novel collaborative training framework that leverages vision-language model to enhance the performance of conventional closed-set remote sensing object detectors. Notably, during inference, VLM4RSDet only retains the standard object detection architecture, thus avoiding any additional deployment overhead. Furthermore, we introduce a Global-Local Cross-Attention (GLCA) module and a Learnable Hierarchical Prediction Strategy (LHPS) to further improve collaborative training performance. Extensive experiments on five benchmark datasets demonstrate the effectiveness and robustness of our approach. In particular, our method outperforms the state-of-the-art by 7.5% in mAP_ 0.5:0.95 on the VisDrone2019 dataset. Our code is available at \href https://github.com/cszzshi/VLM4RSDet https://github.com/cszzshi/VLM4RSDet .

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e6b8b07f955360eb1d57f7909935d5dcCVF · CVPR 2026

Evolutionary Multimodal Reasoning via Hierarchical Semantic Representation for Intent Recognition

Zhou, Qianrui, Xu, Hua, Gu, Yunjin, Wang, Yifan, Li, Songze, Zhang, Hanlei

原文摘要

Multimodal intent recognition aims to infer human intents by jointly modeling various modalities, playing a pivotal role in real-world dialogue systems. However, current methods struggle to model hierarchical semantics underlying complex intents and lack the capacity for self-evolving reasoning over multimodal representations. To address these issues, we propose HIER, a novel method that integrates hierarchical semantic representation with evolutionary reasoning based on Multimodal Large Language Model (MLLM). Inspired by human cognition, HIER introduces a structured reasoning paradigm that organizes multimodal semantics into three progressively abstracted levels. It starts with modality-specific tokens capturing localized semantic cues, which are then clustered via a label-guided strategy to form mid-level semantic concepts. To capture higher-order structure, inter-concept relations are selected using JS divergence scores to highlight salient dependencies across concepts. These hierarchical representations are then injected into MLLM via CoT-driven prompting, enabling step-wise reasoning. Besides, HIER utilizes a self-evolution mechanism that refines semantic representations through MLLM feedback, allowing dynamic adaptation during inference. Experiments on three challenging benchmarks show that HIER consistently outperforms state-of-the-art methods and MLLMs with 1-3% gains across all metrics. Code and more results are available at https://github.com/thuiar/HIER.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e6c761a24522ad1c4ee4bdf3f4709299CVF · CVPR 2026

Language-driven Fine-grained Retrieval

Wang, Shijie, Yu, Xin, Luo, Yadan, Wang, Zijian, Zhang, Pengfei, Huang, Zi

原文摘要

Existing fine-grained image retrieval (FGIR) methods learn discriminative embeddings by adopting semantically sparse one-hot labels derived from category names as supervision. While effective on seen classes, such supervision overlooks the rich semantics encoded in category names, hindering the modeling of comparability among cross-category details and, in turn, limiting generalization to unseen categories. To tackle this, we introduce LaFG, a Language-driven framework for Fine-Grained Retrieval that converts class names into attribute-level supervision using large language models (LLMs) and vision-language models (VLMs). Treating each name as a semantic anchor, LaFG prompts an LLM to generate detailed, attribute-oriented descriptions. To mitigate attribute omission in these descriptions, it leverages a frozen VLM to project them into a vision-aligned space, clustering them into a dataset-wide attribute vocabulary while harvesting complementary attributes from related categories. Leveraging this vocabulary, a global prompt template selects category-relevant attributes, which are aggregated into category-specific linguistic prototypes. These prototypes supervise the retrieval model to steer it toward pinpointing visual details consistent with linguistic descriptions, thus modeling comparability among object details. Extensive evaluations show that LaFG achieves impressive performance on both fine- and coarse-grained benchmarks and generalizes well to unseen categories.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e73c7380876f5cf73a7e679787360672CVF · CVPR 2026

4DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Models

Lu, Yiting, Luo, Wei, Tu, Peiyan, Li, Haoran, Zhu, Hanxin, Yu, Zihao, Wang, Xingrui, Chen, Xinyi 等 11 位作者

原文摘要

World Generation Models are emerging as a cornerstone of next-generation multimodal intelligence systems. Unlike traditional 2D visual generation, World Models aim to construct realistic, dynamic, and physically consistent 3D/4D worlds from images, videos, or text. These models not only need to produce high-fidelity visual content but also maintain coherence across space, time, physics, and instruction control, enabling applications in virtual reality, autonomous driving, Embodied Intelligence, and content creation.However, prior benchmarks, however, each emphasize different evaluation dimensions and lack a unified assessment of world-realism capability.To systematically evaluate World Models, we introduce the 4DWorldBench, which measures models across four key dimensions: Perceptual Quality, Condition-4D Alignment, Physical Realism, and 4D Consistency. The benchmark covers tasks such as Image-to-3D/4D, Video-to-4D, Text-to-3D/4D. Beyond these, we innovatively introduce adaptive conditioning across multiple modalities, which not only integrates but also extends traditional evaluation paradigms. To accommodate different modality-conditioned inputs, we map all modality conditions into a unified textual space during evaluation, and further integrate LLM-as-judge, MLLM-as-judge, and traditional network-based methods. This unified and adaptive design enables more comprehensive and consistent evaluation of alignment, physical Realism, and cross-modal coherence.Preliminary human studies further demonstrate that our adaptive tool selection achieves closer agreement with subjective human judgments.We hope this benchmark will serve as a foundation for objective comparisons and improvements, accelerating the transition from "visual generation" to "world generation." Our project can be found at https://yeppp27.github.io/4DWorldBench.github.io/.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e79151d259f36d4d5321bd5e4c323348CVF · CVPR 2026

Reasoning Diffusion for Unpaired Test Time Out-of-distribution Text-Image to Video Generation

Pan, Zirui, Wang, Xin, Zhang, Yipeng, Chen, Hong, Zheng, Kecheng, Zhu, Wenwu

原文摘要

Text-image to video generation aims to synthesize a video conditioned on the given text-image inputs. Nevertheless, existing methods generally assume that the semantic information carried in the input text and image tends to be perfectly paired and temporally aligned, occurring simultaneously in the generated video. As such, existing literature struggles with out-of-distribution (OOD) "unpaired" text-image inputs in the more universal and realistic scenario where i) the semantic information carried by the text and image may occur at different timestamps and ii) the condition image can appear at an arbitrary position rather than the first frame of the synthesized video. Video generation under this OOD setting poses an urgent need to conduct reasoning over the intrinsic connections between the given textual description and referred image, which is challenging and remains unexplored. To address the challenge, in this paper we study the problem of unpaired text-image to video generation for the first time, proposing ReasonDiff, a novel model for accurate video generation from unpaired text-image inputs. Specifically, ReasonDiff designs a VisionNarrator module to harness the powerful reasoning abilities of a multi-modal LLM to analyze the unpaired text-image inputs, producing coherent per-frame narratives that temporally align them. Building upon this VisionNarrator module, ReasonDiff further introduces a novel AlignFormer module, which employs a Multi-stage Temporal Anchor Attention mechanism to predict frame-wise latent representations. These reasoning-enhanced latents are subsequently fused with the condition frame, providing structured guidance throughout the video generation process. Extensive experiments and ablation studies demonstrate that ReasonDiff beats state-of-the-art baselines in terms of video generation quality with unpaired text-image inputs.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e79b13cca0f7089ff072f623e7e2515bCVF · CVPR 2026

RNED: Rotary Number Encoding and Decoding for Medical VLMs

Liu, Fengbei, Kwak, Sunwoo, Nizam, Nusrat, Richter, Ilan, Beecy, Ashley, Raikhelkar, Jayant, Estrin, Deborah, Sabuncu, Mert R.

原文摘要

Vision-Language Models (VLMs) are increasingly adopted for medical applications, but their clinical utility is limited by a core weakness in quantitative reasoning. This limitation affects tasks ranging from regression of lesion sizes to prediction of bounding-box coordinates and stems from the discrete tokenization schemes underlying Large Language Models (LLMs). To address this, we propose Rotary Number Encoding and Decoding (RNED), a principled method for embedding continuous numerical values directly in the representation space of a VLM. Analogous to rotary position encoding, RNED represents a scalar by applying a number-specific rotation matrix to a dedicated numeric token embedding. This norm-preserving transformation maintains ordinal structure over a wide numerical range and integrates seamlessly with pretrained model weights. For decoding, we introduce a robust score-matching-based scheme to recover continuous values from hidden states in the presence of stochastic noise. We evaluate RNED on two quantitative tasks: radiological measurement estimation and medical visual grounding. On both internal and public benchmarks, RNED consistently outperforms existing VLM baselines. Together, these results show that RNED offers a robust, generalizable solution for numerical reasoning in medical VLMs, enabling models that are both quantitatively reliable and clinically applicable.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e7af06cb2460ce08a04204ca5f547367CVF · CVPR 2026

Learning to Learn Weight Generation via Local Consistency Diffusion

Guan, Yunchuan, Liu, Yu, Zhou, Ke, Shen, Zhiqi, Hwang, Jenq-Neng, Li, Lei

原文摘要

Diffusion-based algorithms have emerged as promising techniques for weight generation. However, existing solutions are limited by two challenges: generalizability and missing local supervision targets. The first challenge stems from the inherent lack of cross-task transferability in existing single-level optimization methods, which limits model performance on new tasks. The latter challenge lies in existing research modeling only global optimal weights, neglecting the supervision signals in local target weights. Furthermore, naively assigning local target weights leads to inconsistency between local and global objectives. To address these issues, we propose Mc-Di, which integrates the diffusion algorithm with meta-learning for better generalizability. Additionally, we extend the vanilla diffusion into a local consistency diffusion algorithm. Our theoretical analysis and experimental results demonstrate that the model can learn from local targets while preserving consistency with the global optimum. We validate Mc-Di's superior accuracy and inference efficiency on tasks that require frequent weight updates, including transfer learning, few-shot learning, domain generalization, and language model fine-tuning.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e7d31bce89bda86fd4397785397fc1f6CVF · CVPR 2026

KVSmooth: Mitigating Hallucination in Multi-modal Large Language Models through Key-Value Smoothing

Jiang, Siyu, Chen, Feiyang, Zhang, Xiaojin, He, Kun

原文摘要

Despite the significant progress of Multi-modal Large Language Models (MLLMs) across diverse tasks, hallucination, which corresponds to the generation of visually inconsistent objects, attributes, or relations, remains a major obstacle to their reliable deployment. Unlike pure language models, MLLMs must ground their generation process in visual inputs; However, existing models often suffer from semantic drift during decoding, causing outputs to diverge from visual facts as the sequence length increases.To address this, we propose KVSmooth, a training-free, plug-and-play method that mitigates hallucination by performing attention-entropy-guided adaptive smoothing on hidden states. Specifically, KVSmooth applies an exponential moving average (EMA) to both keys and values in the KV-Cache while dynamically quantifying the sink degree of each token through its attention distribution entropy to adaptively adjust the smoothing strength. Unlike computationally expensive retraining or contrastive decoding methods, KVSmooth operates efficiently during inference without additional training or model modification. Extensive experiments demonstrate that KVSmooth significantly reduces hallucination (\mathit CHAIR _ S from 41.8 -> 18.2) while improving overall performance (F_1 score from 77.5 -> 79.2), achieving higher precision and recall simultaneously, whereas prior methods often sacrifice one for the other, thereby validating the effectiveness and generality of our method.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e8062fd20f01c09ae754dfd83e4714fcCVF · CVPR 2026

Interpretable Debiasing of Vision-Language Models for Social Fairness

An, Na Min, Jang, Yoonna, Hirota, Yusuke, Hachiuma, Ryo, Augenstein, Isabelle, Shim, Hyunjung

原文摘要

The rapid advancement of Vision-Language models (VLMs) has raised growing concerns that their black-box reasoning processes could lead to unintended forms of social bias. Current debiasing approaches focus on mitigating surface-level bias signals through post-hoc learning or test-time algorithms, while leaving the internal dynamics of the model largely unexplored. In this work, we introduce an interpretable, model-agnostic bias mitigation framework, DeBiasLens, that localizes social attribute neurons in VLMs through sparse autoencoders (SAEs) applied to multimodal encoders. Building upon the disentanglement ability of SAEs, we train them on facial image or caption datasets without corresponding social attribute labels to uncover neurons highly responsive to specific demographics, including those that are underrepresented. By selectively deactivating the social neurons most strongly tied to bias for each group, we effectively mitigate socially biased behaviors of VLMs without degrading their semantic knowledge. Our research lays the groundwork for future auditing tools, prioritizing social fairness in emerging real-world AI systems.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e80ff98e8bdea1ec1b34d6ca97b62c07CVF · CVPR 2026

Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure

Yun, Jooyeol, Choo, Jaegul

原文摘要

Scalable Vector Graphics (SVG) are central to modern web design, and the demand to animate them continues to grow as web environments become increasingly dynamic.Yet automating the animation of vector graphics remains challenging for vision-language models (VLMs) despite recent progress in code generation and motion planning.VLMs routinely mis-handle SVGs, since visually coherent parts are often fragmented into low-level shapes that offer little guidance of which elements should move together. In this paper, we introduce a framework that recovers the semantic structure required for reliable SVG animation and reveals the missing layer that current VLM systems overlook. This is achieved through a statistical aggregation of multiple weak part predictions, allowing the system to stably infer semantics from noisy predictions.By reorganizing SVGs into semantic groups, our approach enables VLMs to produce animations with far greater coherence. Our experiments demonstrate substantial gains over existing approaches, suggesting that semantic recovery is the key step that unlocks robust SVG animation and supports more interpretable interactions between VLMs and vector graphics.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e821d76d9e8b60ba15c702dab17fe840CVF · CVPR 2026

Boosting Vision-Language Models Towards Cross-Domain Incremental Object Detection

Wang, Xu, Lin, Zihan, Zhang, Yixin, Wang, Zilei

原文摘要

Incremental Object Detection (IOD) aims to equip detectors with the ability to handle dynamic environments and emerging object categories, and the rise of vision-language models has substantially advanced this goal. However, existing studies often oversimplify real-world scenarios by assuming the incremental tasks come from a single general domain. To better investigate vision-language models under IOD, it is necessary to explore more generalized scenarios that encompass both novel categories and domains. To this end, we propose Cross-Domain Incremental Object Detection (CDIOD), a new benchmark that assesses the ability to continuously adapt to diverse object detection tasks across domains. CDIOD reveals that existing methods struggle to balance between adaptivity and stability under substantial domain shifts. To tackle this challenge, we propose Dynamic Group Subspace (DGS), a novel framework that dynamically groups tasks by distribution to promote knowledge sharing and prevent task collisions; progressively consolidates adapters to build shared subspaces and control parameter growth; and implements a dynamic training pipeline to maintain a proper stability-adaptivity balance. DGS enables vision-language models to effectively handle task streams of various distribution shifts. Extensive experiments across three benchmarks demonstrate that DGS achieves SOTA performance, highlighting its robustness in diverse incremental learning scenarios. Code is available at https://github.com/Never-wx/dgs.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e82f9ff48f2d30c640c1e51295dcb239CVF · CVPR 2026

VKG-QA: Visual Knowledge Graph-based Question Answer for Large Multimodal Models

Du, Yuntao, Wang, Yiming, Yuan, Renshuo, Yue, Jincheng, Chen, Yijing, Fan, Yue, Zhang, Bo, Li, Qian 等 9 位作者

原文摘要

Understanding and reasoning over structured knowledge is a fundamental capability for intelligent systems. While Large Language Models (LLMs) have leveraged textual knowledge graphs for relational reasoning, linearizing graph structures into text often leads to token inefficiency and loss of higher-order relational cues. Inspired by the advances of Large Multimodal Model to capture higher-order relational structures explicitly novel paradigm of visualized knowledge representation, where knowledge graphs are transformed into graphical visualizations that LMMs can directly perceive and reason over. To systematically evaluate this capability, we introduce VKG-QA, a benchmark for Visual Knowledge Graph-based Question Answering, covering three major categories and fourteen subtasks. VKG-QA is constructed via a semi-automatic pipeline ensuring high-quality, semantically aligned, and visually clear data. We evaluate 19 representative LMMs on VKG-QA and perform extensive quantitative and qualitative analyses. Results reveal that current models struggle with visualized relational understanding, graph-specific comprehension remains challenging, and closed-source models significantly outperform open-source counterparts. VKG-QA thus highlights critical limitations in current LMMs and provides a scalable platform for advancing graph-aware visual reasoning.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e85331dbe904bd4ae5b6cd0b4d82cec5CVF · CVPR 2026

TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs

Zhang, Jun, Wang, Teng, Ge, Yuying, Ge, Yixiao, Li, Xinhao, Wang, Limin

原文摘要

This paper does not introduce a novel method but instead establishes a straightforward, incremental, yet essential baseline for video temporal grounding (VTG), a core capability in video understanding. While multimodal large language models (MLLMs) excel at various video understanding tasks, the recipes for optimizing them for VTG remain under-explored. In this paper, we present TimeLens, a systematic investigation into building MLLMs with strong VTG ability, along two primary dimensions: data quality and algorithmic design. We first expose critical quality issues in existing VTG benchmarks and introduce TimeLens-Bench, comprising meticulously re-annotated versions of three popular benchmarks with strict quality criteria. Our analysis reveals dramatic model re-rankings compared to legacy benchmarks, confirming the unreliability of prior evaluation standards. We also address noisy training data through an automated re-annotation pipeline, yielding TimeLens-100K, a large-scale, high-quality training dataset. Building on our data foundation, we conduct in-depth explorations of algorithmic design principles, yielding a series of meaningful insights and effective yet efficient practices. These include interleaved textual encoding for time representation, a thinking-free reinforcement learning with verifiable rewards (RLVR) approach as the training paradigm, and carefully designed recipes for RLVR training. These efforts culminate in TimeLens models, a family of MLLMs with state-of-the-art VTG performance among open-source models and even surpass proprietary models such as GPT-5 and Gemini-2.5-Flash. All codes, data, and models will be released to facilitate future research.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e876bff0950b82a0c9b7c3fd038e58d6CVF · CVPR 2026

AdapAction: Adaptive Target Action Backdoor Attack against GUI Agents

Chen, Baicheng, Zhang, Mingda, Zhang, Min, Li, Haizhou, Wu, Baoyuan

原文摘要

Autonomous Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) are increasingly vital for complex task automation. However, their capacity for self-driven decision-making introduces significant, yet underexplored, security risks, among which backdoor attacks pose a particularly stealthy and high-impact threat. Prior work has shown GUI agents vulnerable to such attacks, but existing methods rely on static trigger-action mappings that execute fixed, context-agnostic behaviors, making them highly detectable. To address this limitation, we introduce **AdapAction**, a novel backdoor attack that subverts the agent's decision-making by embedding an **adaptive, context-aware policy**. Unlike traditional approaches, AdapAction enables the agent to autonomously select environmentally coherent malicious actions based on the current GUI state and user instruction, thereby evading detection while preserving functional utility. Extensive experiments on the Android-In-The-Zoo (AitZ) and AndroidControl benchmarks show that AdapAction achieves up to 100% Attack Success Rate (ASR) while preserving benign task utility. More critically, AdapAction consistently evades a multi-principle-based LLM defense evaluating instruction alignment, visual coherence, and safety, whereas traditional fixed-action attacks are easily detected. This resilience stems from AdapAction's contextually grounded malicious actions, which are semantically and visually indistinguishable from legitimate operations. As a result, AdapAction exhibits exceptional stealth and poses a significantly greater real-world threat to LLM-powered GUI agents.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e880c47761afa65ae1f155c84867374dCVF · CVPR 2026

Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients

Xiang, Ziwei, Zeng, Fanhu, Fang, Hongjian, Wang, Rui-Qi, Chen, Renxing, Zhu, Yanan, Chen, Yi, Yang, Peipei 等 9 位作者

原文摘要

Large Vision Language Models (LVLMs) have achieved remarkable success in a wide range of downstream tasks that require multimodal interaction, but their powerful capabilities come with substantial computational and memory overhead, which hinders practical deployment. Among numerous acceleration techniques, post-training quantization is a popular and effective strategy for reducing memory cost and accelerating inference. However, existing LVLM quantization methods typically measure token sensitivity at the modality level, which fails to capture the rich and complex cross-token interactions and falls short in quantitatively measuring the quantization error at the token level. As tokens interact within the model, the distinction between modalities gradually diminishes, suggesting the need for fine-grained calibration. Inspired by axiomatic attribution in mechanistic interpretability, we introduce a fine-grained quantization strategy based on Quantization-aware Integrated Gradients (QIG), which leverages integrated gradients to quantitatively evaluate token sensitivity and push the granularity from modality level to token level, reflecting both inter-modality and intra-modality dynamics. Extensive experiments on multiple LVLMs under both W4A8 and W3A16 settings show that our method consistently improves accuracy across models and benchmarks with negligible latency overhead. For example, under 3-bit weight-only quantization, our method improves the average accuracy of LLaVA-onevision-7B by 1.60%, reducing the gap to its full-precision counterpart to only 1.33%. The code is available at https://github.com/ucas-xiang/QIG.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e8b80dd9f22381e3c7e1a611b1377ef1CVF · CVPR 2026

iSHIFT: Lightweight Slow-Fast GUI Agent with Adaptive Perception

Mehrotra, Sarthak, Rebbapragada, Sairam VC, Bonthu, Mani, Balasubramanian, Vineeth N.

原文摘要

Multimodal Large Language Models (MLLMs) show strong potential for interpreting and interacting with complex, pixel-rich Graphical User Interface (GUI) environments. However, building agents that are both efficient for high-level tasks and precise for fine-grained interactions remains challenging. GUI agents must perform routine actions efficiently while also handling tasks that demand exact visual grounding, yet existing approaches struggle when accuracy depends on identifying specific interface elements. These MLLMs also remain large and cannot adapt their reasoning depth to the task at hand. In this work, we introduce iSHIFT: Implicit Slow-fast Hybrid Inference with Flexible Tokens, a lightweight agent that integrates latent thinking (implicit chain-of-thought) with a perception control module. iSHIFT enables an MLLM to switch between a slow mode, which leverages detailed visual grounding for high precision and a fast mode that uses global cues for efficiency. Special perception tokens guide attention to relevant screen regions, allowing the model to decide both how to reason and where to focus. Despite its compact 2.5B size, iSHIFT matches state-of-the-art performance on multiple benchmark datasets.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e8e1f0d72a09f3f6af864a83e9895b56CVF · CVPR 2026

VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models

Deng, Juncan, Huang, Kejie

原文摘要

Post-training quantization (PTQ) serves as a vital technique for efficiently compressing large-scale models, with weight-compensation methods such as GPTQ (symmetric calibration) and GPTAQ (asymmetric calibration) showing remarkable success. However, directly applying these methods to Vision-Language Models (VLMs) exposes two notable shortcomings: 1) the standard rounding-to-nearest (RTN) method is suboptimal for the asymmetric objective, failing to account for residual-induced shifts in the optimal quantization target; and 2) all input channels are processed uniformly across modalities, overlooking the distinct information densities. In this paper, we introduce VLM-PTQ, an asymmetric post-training quantization framework for VLMs. First, we derive a closed-form correction term that shifts the quantization target, which explicitly accounts for the output residual and the corresponding inverse Hessian column, yielding a better local optimum than RTN. Second, we propose a modality-aware quantization that differentiates channel importance between vision and language tokens, allowing the quantizer to pre-compute better quantization parameters through a lightweight search. Our method extends weight-compensation methods with minimal overhead while achieving significant performance improvements in low-bit scenarios. Extensive experiments demonstrate that VLM-PTQ achieves competitive results compared to existing methods, effectively compressing models from 1B to 72B parameters on a single GPU.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e90783e9a7be189ab8456c20ccff2978CVF · CVPR 2026

MODIX: A Training-Free Multimodal Information-Driven Positional Index Scaling for Vision-Language Models

Huang, Ruoxiang, Yuan, Zhen

原文摘要

Vision-Language Models (VLMs) have achieved remarkable progress in multimodal understanding, yet their positional encoding mechanisms remain suboptimal. Existing approaches uniformly assign positional indices to all tokens, overlooking variations in information density within and across modalities, which leads to inefficient attention allocation where redundant visual regions dominate while informative content is underrepresented. We identify positional granularity as an implicit resource and propose MODIX (Multimodal Information-Driven Positional IndeX Scaling), a training-free framework that dynamically adapts positional strides based on modality-specific contributions. MODIX jointly models intra-modal density via covariance-based entropy and inter-modal interaction via cross-modal alignment to derive unified scores, which rescale positional indices to allocate finer granularity to informative modalities while compressing redundant ones, without requiring any modification to model parameters or architecture. Experiments across diverse architectures and benchmarks demonstrate that MODIX consistently improves multimodal reasoning and adaptively reallocates attention according to task-dependent information distributions, suggesting that positional encoding should be treated as an adaptive resource in Transformers for multimodal sequence modeling.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e910e1a1af63a2ff681e81a09cf8c152CVF · CVPR 2026

ParaUni: Enhance Generation in Unified Multimodal Model with Reinforcement-driven Hierarchical Parallel Information Interaction

Tan, Jiangtong, Liu, Lin, Huang, Jie, Zhang, Xiaopeng, Tian, Qi, Zhao, Feng

原文摘要

Unified multimodal models significantly improve visual generation by combining vision-language models (VLMs) with diffusion models. However, existing methods struggle to fully balance sufficient interaction and flexible implementation due to vast representation difference. Considering abundant and hierarchical information in VLM's layers from low-level details to high-level semantics, we propose ParaUni. It extracts features from variants VLM's layers in a Parallel way for comprehensive information interaction and retains a flexible separation architecture to enhance generation in Unified multimodal model. Concretely, visual features from all VLM's layers are fed in parallel into a Layer Integration Module (LIM), which efficiently integrates fine-grained details and semantic abstractions and provides the fused representation as a condition to the diffusion model. To further enhance performance, we reveal that these hierarchical layers respond unequally to different rewards in Reinforcement Learning (RL). Crucially, we design a Layer-wise Dynamic Adjustment Mechanism (LDAM) to facilitate multiple reward improvements that aligns the hierarchical properties of these layers using RL. Extensive experiments show ParaUni leverages complementary multi-layer features to substantially improve generation quality and shows strong potential for multiple reward advances during RL stages.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e91bbc9c9b43b88b4ae1fb62cb26e163CVF · CVPR 2026

Models as Lego Builders: Assembling Malice from Benign Blocks via Semantic Blueprints

Li, Chenxi, Liu, Xianggan, Shen, Dake, Du, Yaosong, Yao, Zhibo, Jiang, Hao, Jiang, Linyi, Cao, Chengwei 等 12 位作者

原文摘要

Despite the rapid progress of Large Vision-Language Models (LVLMs), the integration of visual modalities introduces new safety vulnerabilities that adversaries can exploit to elicit biased or malicious outputs. In this paper, we demonstrate an underexplored vulnerability via semantic slot filling, where LVLMs complete missing slot values with unsafe content even when the slot types are deliberately crafted to appear benign. Building on this finding, we propose StructAttack, a simple yet effective single-query jailbreak framework under black-box settings. StructAttack decomposes a harmful query into a central topic and a set of benign-looking slot types, then embeds them as structured visual prompts (e.g., mind maps, tables, or sunburst diagrams) with small random perturbations. Paired with a completion-guided instruction, LVLMs automatically recompose the concealed semantics and generate unsafe outputs without triggering safety mechanisms. Although each slot appears benign in isolation (local benignness), StructAttack exploits LVLMs' reasoning to assemble these slots into coherent harmful semantics. Extensive experiments on multiple models and benchmarks show the efficacy of our proposed StructAttack.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e93fdc8cb5cdb49546d5a376d806a987CVF · CVPR 2026

Hugging Visual Prompt and Segmentation Tokens: Consistency Learning for Fine-Grained Visual Understanding in MLLMs

Yang, Jing, Yang, Sen, Duan, Boqiang, Dai, Ming, Zhang, Wei, Tan, Xiao, Chen, Kunbin, He, Wei 等 10 位作者

原文摘要

Recently, multimodal large language models (MLLMs) have achieved remarkable success in general multimodal tasks. Increasing attention has been given to leveraging MLLMs for fine-grained visual understanding, such as region-level captioning and pixel-level grounding. However, most existing approaches are task-specific. While some recent unified approaches attempt to handle both types simultaneously, they still fall short of deeply exploring the underlying associations across tasks. To bridge this gap, we propose FCLM, a large multimodal model designed to jointly support fine-grained visual understanding through consistency learning. The central premise is that pixel-level captioning and grounding are mutually beneficial and complementary, each enhancing the other in achieving a fine-grained understanding of visual content. Specifically, FCLM analyzes the representation features - visual prompt and segmentation tokens - required for the two types of visual tasks, and achieves advanced reasoning and perception through a newly-designed consistency learning loss and a two-stage training framework. Moreover, a hybrid region extractor is designed to enhance visual prompt embeddings, yielding semantically discriminative representations for detailed captioning. Furthermore, a novel detailed localized referring expression segmentation (DL-RES) task is introduced to evaluate the model's ability to localize targets from detailed textual descriptions. Extensive experiments on seven visual understanding tasks demonstrate the superior performance and strong generalization of FCLM.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e961df3b794c8152275a715d6a221fbfCVF · CVPR 2026

Learning Multi-View Spatial Reasoning from Cross-View Relations

Jeong, Suchae, Song, Jaehwi, Lee, Haeone, Kim, Hanna, Kim, Jian, Lee, Dongjun, Shin, Dong Kyu, Kim, Changyeon 等 12 位作者

原文摘要

Vision-language models (VLMs) have achieved impressive results on single-view vision tasks, but lack the multi-view spatial reasoning capabilities essential for embodied AI systems to understand 3D environments and manipulate objects across different viewpoints. In this work, we introduce Cross-View Relations (XVR), a large-scale dataset designed to teach VLMs spatial reasoning across multiple views. XVR comprises 100K vision-question-answer samples derived from 18K diverse 3D scenes and 70K robotic manipulation trajectories, spanning three fundamental spatial reasoning tasks: Correspondence (matching objects across views), Verification (validating spatial relationships), and Localization (identifying object positions). VLMs fine-tuned on XVR achieve substantial improvements on established multi-view and robotic spatial reasoning benchmarks (MindCube and RoboSpatial). When integrated as backbones in Vision-Language-Action models, XVR-trained representations improve success rates on RoboCasa. Our results demonstrate that explicit training on cross-view spatial relations significantly enhances multi-view reasoning and transfers effectively to real-world robotic manipulation.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e9677eaece50842fd9c5e52047611ef5CVF · CVPR 2026

Visual Document Understanding and Reasoning: A Multi-Agent Collaboration Framework with Agent-Wise Adaptive Test-Time Scaling

Yu, Xinlei, Xu, Chengming, Chen, Zhangquan, Zhang, Yudong, Lu, Shilin, Yang, Cheng, Zhang, Jiangning, Yan, Shuicheng 等 9 位作者

原文摘要

The dominant paradigm of monolithic scaling in Vision-Language Models (VLMs) is failing for understanding and reasoning in documents, yielding diminishing returns as it struggles with the inherent need of this domain for document-based procedural reasoning, cognitive complexity, and factual accuracy. To this end, we introduce MACT, a Multi-Agent Collaboration framework with agent-wise adaptive Test-time scaling that pioneers a paradigm shift to procedural scaling, adapting dynamically to the functional entities of visual documents understanding and reasoning. MACT decomposes the visual document processing flow into four specialized agents, i.e., planning, execution, judgment, and answer, to resolve cognitive overload and introduce a critical self-correction loop for factual grounding. This collaborative architecture is amplified by an agent-wise adaptive test-time scaling strategy that intelligently allocates computational resources based on the complexity and redundancy of each functionality. Evaluated on multiple visual document understanding benchmarks, MACT achieves superior performance with a smaller parameter scale, adapting effectively to various document scenarios without compromising its general or mathematical reasoning capabilities. The three variants consistently attain top-three average performance rankings, with average performance enhancements of 9.9-11.5% over the base models.

cs.CV · cs.CL · cs.MA

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e96aec42007e700df1fffc848901acf7CVF · CVPR 2026

Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding

Clark, Christopher, Zhang, Jieyu, Ma, Zixian, Park, Jae Sung, Tripathi, Rohun, Lee, Sangho, Salehi, Mohammadreza, Ren, Jason 等 20 位作者

原文摘要

Today's strongest video-language models (VLMs) remain proprietary, and the strongest open-weight models often rely on synthetic data from proprietary VLMs and do not disclose their training data or recipe. As a result, the open-source community lacks the foundations needed to improve on the state-of-the-art video (and image) language models. Crucially, many downstream applications require grounding--either by pointing or by tracking in pixels. Even proprietary models lack this capability. We present Molmo2, a new family of VLMs that are state-of-the-art among open-source models and demonstrate exceptional new capabilities in point-driven grounding in single image, multi-image, and video tasks. Our key contribution is a collection of 7 new video datasets and 2 multi-image datasets, including a dataset of highly detailed video captions for pre-training, a free-form video Q&A dataset for fine-tuning, a new object tracking dataset with complex queries, and an innovative new video pointing dataset, all collected without the use of closed VLMs. We also present a training recipe for this data utilizing an efficient packing and message-tree encoding scheme, and show bi-directional attention on vision tokens and a novel token-weight strategy improves performance. Our best-in-class 8B model outperforms others in the class of open weight and data models on short videos, counting, and captioning, and is competitive on long-videos. On video-grounding Molmo2 significantly outperforms existing open-weight models like Qwen3-VL (35.5 vs 29.6 accuracy on video counting) and surpasses proprietary models like Gemini 3 Pro on some tasks (38.4 vs 20.0 F1 on video pointing and 56.2 vs 41.1 J&F on video tracking). Our model weights, new datasets, and source code are available at https://allenai.org/blog/molmo2.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:e995415ed28532537a83063e8c98ee73CVF · CVPR 2026

Vibe Spaces for Creatively Connecting and Expressing Visual Concepts

Yang, Huzheng, Xu, Katherine, Lu, Andrew, Grossberg, Michael D., Bai, Yutong, Shi, Jianbo

原文摘要

Creating new visual concepts often requires connecting distinct ideas through their most relevant shared attributes--their vibe. We introduce Vibe Blending, a novel task for generating coherent and meaningful hybrids that reveals these shared attributes between images. Achieving such blends is challenging for current methods, which struggle to identify and traverse nonlinear paths linking distant concepts in latent space. We propose Vibe Space, a hierarchical graph manifold that learns low-dimensional geodesics in feature spaces like CLIP, enabling smooth and semantically consistent transitions between concepts. To evaluate creative quality, we design a cognitively inspired framework combining human judgments, LLM reasoning, and a geometric path-based difficulty score. We find that Vibe Space produces blends that humans consistently rate as more creative and coherent than current methods.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗