跳到正文
研踪PaperTrail

论文库

检索计算机领域论文，按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+0新增论文
+0其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-15 23:30 · 全站最近入库：2026-11-14 15:09各信源进度 →

统计口径与数据新鲜度

统计 2026-11-14 23:57 — 2026-11-15 23:57（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-15 23:30最近发现新增版本：2026-11-14 15:09库内论文最新更新时间：2026-11-13 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

23,348 已收录论文19,580 AI 相关6,729 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 23,348 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-15 23:27（北京时间）
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4c3f97e833eddbf3bf50424a6f9ee51fCVF · CVPR 2026

Rounded or Streamlined Head? Bridging Concept Bottleneck Models and Attribute-Described Object Parts

Liu, Yang, Zhang, Jiajin, Hu, Yaojun, Hao, Bingguang, Cao, Xin, Xia, Yingda, Tu, Danyang, Gu, Shi 等 9 位作者

原文摘要

A faithful decision-making process requires models to ground human-understandable concepts both spatially (where they appear in the image) and causally (how they influence the prediction). Recent advances in Vision-Language Models (VLMs) enable concept-level alignment and have inspired Concept Bottleneck Models (CBMs), which explain predictions by mapping image representations to human-understandable concepts, allowing users to trace decisions through explicit semantic reasoning. However, existing CBMs suffer from two key inconsistencies. First, semantic inconsistency: VLMs often fail to localize fine-grained part-attribute concepts, producing noisy or incomplete masks. Second, object inconsistency: object-agnostic concepts such as "head: streamlined front profile" may describe multiple categories (e.g., fish or human); without enforcing object identity, non-targeted regions can introduce spurious evidence that corrupts the bottleneck representation. To address these challenges, we propose a new Object-Aware Concept Bottleneck Model (OA-CBM) that jointly enforces semantic- and object-level consistency. Specifically, (1) we redefine concepts as part-attribute pairs to enhance VLM robustness at the semantic level, and (2) introduce class-agnostic object clustering to suppress irrelevant visual evidence. We further annotate two grounding datasets with part-attribute descriptions and conduct extensive experiments. Results demonstrate that OA-CBM produces more faithful and robust explanations while maintaining competitive predictive performance.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4c4ee967f5769399a6b699628dd2171aCVF · CVPR 2026

Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation

Shi, Chuancheng, Li, Shangze, Guo, Shiming, Xie, Simiao, Wu, Wenhua, Dou, Jingtong, Wu, Chao, Xiao, Canran 等 12 位作者

原文摘要

Multilingual text-to-image (T2I) models have advanced rapidly in terms of visual realism and semantic alignment, and are now widely utilised. Yet outputs vary across cultural contexts: because language carries cultural connotations, images synthesized from multilingual prompts should preserve cross-lingual cultural consistency. We conduct a comprehensive analysis showing that current T2I models often produce culturally neutral or English-biased results under multilingual prompts.Analyses of two representative models indicate that the issue stems not from missing cultural knowledge but from insufficient activation of culture-related representations. We propose a probing method that localizes culture-sensitive signals to a small set of neurons in a few fixed layers. Guided by this finding, we introduce two complementary alignment strategies: (1) inference-time cultural activation that amplifies the identified neurons without backbone fine-tuned; and (2) layer-targeted cultural enhancement that updates only culturally relevant layers. Experiments on our CultureBench demonstrate consistent improvements over strong baselines in cultural consistency while preserving fidelity and diversity.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4caab821108a03c009219513df1093ffCVF · CVPR 2026

RF4D:Neural Radar Fields for Novel View Synthesis in Outdoor Dynamic Scenes

Zhang, Jiarui, Li, Zhihao, Wang, Chong, Wen, Bihan

原文摘要

Neural fields (NFs) have achieved remarkable success in scene reconstruction and novel view synthesis. However, existing NF approaches that rely on RGB or LiDAR inputs often struggle under adverse weather conditions, limiting their robustness in real-world outdoor environments such as autonomous driving. In contrast, millimeter-wave radar is inherently resilient to environmental variations, yet its integration with NFs remains largely underexplored. Moreover, outdoor driving scenes frequently involve dynamic objects, making spatiotemporal modeling crucial for temporally consistent novel view synthesis. To address these challenges, we present RF4D, a radar-based neural field framework tailored for novel view synthesis in outdoor dynamic scenes. RF4D explicitly incorporates temporal information into its representation, enabling more accurate modeling of object motion. A dedicated scene flow module further predicts temporal offsets between adjacent frames, enforcing temporal occupancy coherence during dynamic scene reconstruction. Moreover, we propose a radar-specific power rendering formulation grounded in radar sensing physics, improving both synthesis accuracy and interpretability. Extensive experiments on public radar datasets demonstrate that RF4D substantially outperforms existing methods in radar measurement synthesis and occupancy estimation accuracy, with particularly strong gains in dynamic outdoor environments.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4caf32154a611963d5347ce466449db5CVF · CVPR 2026

CLaD: Planning with Grounded Foresight via Cross-Modal Latent Dynamics

Jeong, Andrew, Kim, Jaemin, Lee, Sebin, Yoon, Sung-Eui

原文摘要

Robotic manipulation involves kinematic and semantic transitions that are inherently coupled via underlying actions. However, existing approaches plan within either semantic or latent space without explicitly aligning these cross-modal transitions. To address this, we propose CLaD, a framework that models how proprioceptive and semantic states jointly evolve under actions through asymmetric cross-attention that allows kinematic transitions to query semantic ones. CLaD predicts grounded latent foresights via self-supervised objectives with EMA target encoders and auxiliary reconstruction losses, preventing representation collapse while anchoring predictions to observable states. Predicted foresights are modulated with observations to condition a diffusion policy for action generation. On LIBERO-LONG benchmark, CLaD achieves 94.7% success rate, competitive with large VLAs with significantly fewer parameters.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4cd43440083dd82acca765e595b9f4a9CVF · CVPR 2026

HAD: Hallucination-Aware Diffusion Priors for 3D Reconstruction

Liu, Xi, Sun, Weiwei, Ren, Zhou, Broaddus, Chris, Huang, Siyu, Guigues, Laurent

原文摘要

Diffusion priors have recently demonstrated strong capability in enhancing the quality of sparse-view 3D reconstruction by augmenting training views at novel viewpoints, but they inevitably introduce hallucinated content-- artifacts inconsistent with the input views -- into the final 3D model. To address this challenge, we propose Hallucination-Aware Diffusion prior (HAD), which estimates pixel-wise hallucination score maps for augmented images by leveraging multi-view reasoning capabilities from a feedforward novel view synthesis (NVS) network pre-trained on large-scale 3D data. These hallucination scores enable selective masking of unreliable pixels during the progressive 3D reconstruction procedure, preventing the introduction of non-existent artifacts into the 3D model. To further enhance performance, we create multiple versions of augmented images at each novel view by conditioning the diffusion prior on different input views, which are then fused into a final image that leverages the broader context across all input views. We show that our method substantially reduces hallucination artifacts in diffusion-assisted 3D reconstruction, thereby achieving state-of-the-art performance across multiple benchmarks on novel view synthesis.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4ce05d013e667b6b13234daea7715b9bCVF · CVPR 2026

Shedding Light on VLN Robustness: A Black-box Framework for Indoor Lighting-based Adversarial Attack

Li, Chenyang, Tang, Wenbing, Huang, Yihao, Zhan, Sinong Simon, Hu, Ming, Jia, Xiaojun, Liu, Yang

原文摘要

Vision-and-Language Navigation (VLN) agents have made remarkable progress, but their robustness remains insufficiently studied. Existing adversarial evaluations often rely on perturbations that manifest as unusual textures rarely encountered in everyday indoor environments. Errors under such contrived conditions have limited practical relevance, as real-world agents are unlikely to encounter such artificial patterns. In this work, we focus on indoor lighting, an intrinsic yet largely overlooked scene attribute that strongly influences navigation. We propose Indoor Lighting-based Adversarial Attack (ILA), a black-box framework that manipulates global illumination to disrupt VLN agents. Motivated by typical household lighting usage, we design two attack modes: Static Indoor Lighting-based Attack (SILA), where the lighting intensity remains constant throughout an episode, and Dynamic Indoor Lighting-based Attack (DILA), where lights are switched on or off at critical moments to induce abrupt illumination changes. We evaluate ILA on two state-of-the-art VLN models across three navigation tasks. Results show that ILA significantly increases failure rates while reducing trajectory efficiency, revealing previously unrecognized vulnerabilities of VLN agents to realistic indoor lighting variations. Our code is available at https://github.com/LiChenyang0820/ILA4VLN.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4d1010f18bf895d3cdda8c09a7d60fbbCVF · CVPR 2026

Lafite: A Generative Latent Field for 3D Native Texturing

Chen, Chia-Hao, Guo, Yuan-Chen, Zou, Zi-Xin, Yuan, Ze, Luo, Guan, Qi, Xiaojuan, Liang, Ding, Cao, Yan-Pei 等 9 位作者

原文摘要

Generating high-fidelity, seamless textures directly on 3D surfaces, a process we term 3D-native texturing, is a fundamental open challenge, promising to overcome the limitations of traditional UV-based and multi-view projection methods. While promising, existing native approaches are bottlenecked by the lack of a powerful and versatile latent representation, severely limiting the fidelity of generated results. In this work, we identify this representation gap as the central roadblock to progress. We introduce Lafite, a framework that resolves this challenge by learning to generate textures as a 3D generative sparse latent color field. At its core, Lafite leverages a variational autoencoder (VAE) to encode complex surface appearance into a sparse, structured latent space and then decoded into a continuous color field. This novel representation achieves unprecedented fidelity, i.e., yielding a >10 dB PSNR improvement in reconstruction over state-of-the-art methods, by effectively disentangling texture from mesh topology and UV parameterizations. Building on this superior representation, a conditional rectified flow model synthesizes textures with state-of-the-art quality and coherence. Extensive experiments show that Lafite not only sets a new standard for native texturing but also enables flexible downstream applications like editing and material synthesis, paving the way for the next generation of 3D content creation workflows.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4d1dbe0b132c2dcf8cf7bcaaeaa6aa5eCVF · CVPR 2026

MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering

Xiao, Junbin, Chen, Jiajun, Sun, Tianxiang, Yang, Xun, Yao, Angela

原文摘要

Long streaming video QA remains challenging due to growing visual tokens and limited reasoning length of large language models (LLMs). KV-caching stores the Key-Value (KV) of the historical tokens via LLM prefill and enables more efficient streaming QA. However, existing methods cache every one or two frames, causing redundant memory usage and losing fine-grained spatial details within frame or temporal contexts across frames. This paper proposes MuKV, a method that features a multi-grained KV cache compression module and a semi-hierarchical retrieval approach to improve both efficiency and accuracy for long streaming VideoQA. For the offline KV cache, MuKV extracts visual representations at patch-, frame-, and segment-levels. The multiple levels of granularity preserve both local cues and global temporal context, while maintaining efficiency with a dual signal token compression mechanism guided by self-attention and frequency. For online QA, MuKV designs a semi-hierarchical retrieval method to retrieve relevant KV caches for answer generation. Experiments on long-streaming VideoQA benchmarks show that MuKV significantly improves answer accuracy, without sacrificing memory and online QA efficiency. Moreover, our compression mechanism alone brings consistent benefits across answer accuracy, memory, and QA efficiency over baselines, showcasing highly effective contribution.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4d2303f4a439e046ec7153f89c4e7634CVF · CVPR 2026

RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark

Shi, Yang, Dong, Yuhao, Ding, Yue, Wang, Yuran, Zhu, Xuanyu, Zhou, Sheng, Liu, Wenting, Tian, Haochen 等 25 位作者

原文摘要

The integration of visual understanding and generation into unified multimodal models represents a significant stride toward general-purpose AI. However, a fundamental question remains unanswered by existing benchmarks: does this architectural unification actually enable synergetic interaction between the constituent capabilities? Existing evaluation paradigms, which primarily assess understanding and generation in isolation, are insufficient for determining whether a unified model can leverage its understanding to enhance its generation, or use generative simulation to facilitate deeper comprehension. To address this critical gap, we introduce RealUnify, a benchmark specifically designed to evaluate bidirectional capability synergy. RealUnify comprises 1,000 meticulously human-annotated instances spanning 10 categories and 32 subtasks. It is structured around two core axes: 1) Understanding Enhances Generation, which requires reasoning (e.g., commonsense, logic) to guide image generation, and 2) Generation Enhances Understanding, which necessitates mental simulation or reconstruction (e.g., of transformed or disordered visual inputs) to solve reasoning tasks. A key contribution is our dual-evaluation protocol, which combines direct end-to-end assessment with a diagnostic stepwise evaluation that decomposes tasks into distinct understanding and generation phases. This protocol allows us to precisely discern whether performance bottlenecks stem from deficiencies in core abilities or from a failure to integrate them. Through large-scale evaluations of 12 leading unified models and 6 specialized baselines, we find that current unified models still struggle to achieve effective synergy, indicating that architectural unification alone is insufficient. These results highlight the need for new training strategies and inductive biases to fully unlock the potential of unified modeling.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4d2eeb876bbd56acfeca855c05d1fb9dCVF · CVPR 2026

Featurising Pixels from Dynamic 3D Scenes with Linear In-Context Learners

Araslanov, Nikita, Sundermeyer, Martin, Matsuki, Hidenobu, Tan, David Joseph, Tombari, Federico

原文摘要

One of the most exciting applications of vision models involve pixel-level reasoning. Despite the abundance of vision foundation models, we still lack representations that effectively embed spatio-temporal properties of visual scenes at the pixel level. Existing frameworks either train on image-based pretext tasks, which do not account for dynamic elements, or on video sequences for action-level reasoning, which does not scale to dense pixel-level prediction. We present a framework that learns pixel-accurate feature descriptors from videos, LILA. The core element of our training framework is linear in-context learning. LILA leverages spatio-temporal cue maps--depth and motion--estimated with off-the-shelf networks. Despite the noisy nature of those cues, LILA trains effectively on uncurated video datasets, embedding semantic and geometric properties in a temporally consistent manner. We demonstrate compelling empirical benefits of the learned representation across a diverse suite of vision tasks: video object segmentation, surface normal estimation and semantic segmentation.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4d492f8b21de479f14553d9d6154f90eCVF · CVPR 2026

CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation

Su, Xia, Chen, Ruiqi, Liu, Benlin, Ma, Jingwei, Di, Zonglin, Krishna, Ranjay, Froehlich, Jon

原文摘要

Vision-Language Models (VLMs) have shown remarkable progress in Vision-Language Navigation (VLN), offering new possibilities for navigation decision-making that could benefit both robotic platforms and human users. However, real-world navigation is inherently conditioned by the agent's mobility constraints. For example, a sweeping robot cannot traverse stairs, while a quadruped can. We introduce Capability-Conditioned Navigation (CapNav), a benchmark designed to evaluate how well VLMs can navigate complex indoor spaces given an agent's specific physical and operational capabilities. CapNav defines five representative human and robot agents, each described with physical dimensions, mobility capabilities, and environmental interaction abilities. CapNav provides 45 real-world indoor scenes, 473 navigation tasks, and 2365 QA pairs to test if VLMs can traverse indoor environments based on agent capabilities. We evaluate 13 modern VLMs and find that current VLM's navigation performance drops sharply as mobility constraints tighten, and that even state-of-the-art models struggle with obstacle types that require reasoning on spatial dimensions. We conclude by discussing the implications for capability-aware navigation and the opportunities for advancing embodied spatial reasoning in future VLMs. The benchmark is available at https://makeabilitylab.github.io/CapNav/

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4d5d9332e76691abb36b49c69b2d013cCVF · CVPR 2026

ImmerIris: A Large-Scale Dataset and Benchmark for Off-Axis and Unconstrained Iris Recognition in Immersive Applications

Mi, Yuxi, Yuan, Qiuyang, Zhong, Zhizhou, Zhao, Xuan, Zhou, Jiaogen, Zhu, Fubao, Guan, Jihong, Zhou, Shuigeng

原文摘要

Recently, iris recognition is regaining prominence in immersive applications such as extended reality as a means of seamless user identification. This application scenario introduces unique challenges compared to traditional iris recognition under controlled setups, as the ocular images are primarily captured off-axis and less constrained, causing perspective distortion, intra-subject variation, and quality degradation in iris textures. Datasets capturing these challenges remain limited. This paper fills this gap by presenting a large-scale iris dataset collected via head-mounted displays, termed ImmerIris. It contains 499,791 ocular images from 546 subjects, and is, to our knowledge, the largest public iris dataset to date and among the first dedicated to immersive applications. It is accompanied by a comprehensive set of evaluation protocols that benchmark recognition systems under various challenging conditions. This paper also draws attention to a shared obstacle of current recognition methods, the reliance on a pre-processing, normalization stage, which is fallible in off-axis and unconstrained setups. To this end, this paper further proposes a normalization-free paradigm that directly learns from minimally adjusted ocular images. Despite its simplicity, it outperforms normalization-based prior arts, indicating a promising direction for robust iris recognition.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4d607b85ff7ea4220a19f9a957c0e595CVF · CVPR 2026

Gastric-X: A Multimodal Multi-Phase Benchmark Dataset for Advancing Vision-Language Models in Gastric Cancer Analysis

Li, Yuanzhe, Chen, Hao, Yin, Rui, Ba, Juyan, Zhang, Yu, Lu, Sheng

原文摘要

Recent vision-language models (VLMs) have shown strong generalization and multimodal reasoning abilities in natural domains. However, their application to medical diagnosis remains limited by the lack of comprehensive and structured datasets that capture real clinical workflows. To advance the development of VLMs for clinical applications, particularly in gastric cancer, we introduce Gastric-X, a large-scale multimodal benchmark for gastric cancer analysis providing 1.7K cases. Each case in Gastric-X includes paired resting and dynamic CT scans, endoscopic image, a set of structured biochemical indicators, expert-authored diagnostic notes, and bounding box annotations of tumor regions, reflecting realistic clinical conditions. We systematically examine the capability of recent VLMs on five core tasks: Visual Question Answering (VQA), report generation, cross-modal retrieval, disease classification, and lesion localization. These tasks simulate critical stages of clinical workflow, from visual understanding and reasoning to multimodal decision support. Through this evaluation, we aim not only to assess model performance but also to probe the nature of VLM understanding: Can current VLMs meaningfully correlate biochemical signals with spatial tumor features and textual reports? We envision Gastric-X as a step toward aligning machine intelligence with the cognitive and evidential reasoning processes of physicians, and as a resource to inspire the development of next-generation medical VLMs.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4d7c456cc7631ead2610720978453d7eCVF · CVPR 2026

GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation

Deng, Tianchen, Chen, Xuefeng, Chen, Yi, Chen, Qu, Xu, Yuyao, Yang, Lijin, Xu, Le, Zhang, Yu 等 11 位作者

原文摘要

Driving World Models (DWMs) have been developing rapidly with the advances of generative models. However, existing DWMs lack 3D scene understanding capabilities and can only generate content conditioned on input data, without the ability to interpret or reason about the driving environment. Moreover, current approaches represent 3D spatial information with point cloud or BEV features do not accurately align textual information with the underlying 3D scene. To address these limitations, we propose a novel unified DWM framework based on 3D Gaussian scene representation, which enables both 3D scene understanding and multi-modal scene generation, while also enabling contextual enrichment for understanding and generation tasks. Our approach directly aligns textual information with the 3D scene by embedding rich linguistic features into each Gaussian primitive, thereby achieving early modality alignment. In addition, we design a novel task-aware language-guided sampling strategy that removes redundant 3D Gaussians and injects accurate and compact 3D tokens into LLM. Furthermore, we design a dual-condition multi-modal generation model, where the information captured by our vision-language model is leveraged as a high-level language condition in combination with a low-level image condition, jointly guiding the multi-modal generation process. We conduct comprehensive studies on the nuScenes, and NuInteract datasets to validate the effectiveness of our framework. Our method achieves state-of-the-art performance. We will release the code publicly on GitHub.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4db7ad17e2b7d02e0a1d1a1097c498c9CVF · CVPR 2026

A Provable Energy-Guided Test-Time Defense Boosting Adversarial Robustness of Large Vision-Language Models

Mirza, Mujtaba Hussain, D'Orazio, Antonio, Melamed, Odelia, Masi, Iacopo

原文摘要

Despite the rapid progress in multimodal models and Large Visual-Language Models (LVLM), they remain highly susceptible to adversarial perturbations, raising serious concerns about their reliability in real-world use. While adversarial training has become the leading paradigm for building models that are robust to adversarial attacks, Test-Time Transformations (TTT) have emerged as a promising strategy to boost robustness at inference.In light of this, we propose Energy-Guided Test-Time Transformation (ET3), a lightweight, training-free defense that enhances the robustness by minimizing the energy of the input samples.Our method is grounded in a theory that proves our transformation succeeds in classification under reasonable assumptions. We present extensive experiments demonstrating that ET3 provides a strong defense for classifiers, zero-shot classification with CLIP, and also for boosting the robustness of LVLMs in tasks such as Image Captioning and Visual Question Answering. Code is available at github.com/OmnAI-Lab/Energy-Guided-Test-Time-Defense .

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4dc0046986e317c7dee52068b9514adcCVF · CVPR 2026

CanonCGT: Reference-Based Color Grading via Canonical Pivot Representation

Ko, Jinwon, Ko, Keunsoo, Kim, Chang-Su

原文摘要

Reference-based color grading aims to reproduce the tonal mood and lighting of a reference while preserving color harmony and scene structure. Existing photorealistic and filter-based methods often produce unstable tone mappings --- over-shifting or inconsistently retaining colors --- leading to unnatural results. We propose CanonCGT, a two-stage framework built on a canonical pivot --- a style-neutral intermediate representation for stable color mapping. The first stage canonicalizes the input by removing intrinsic tonal bias, and the second color-grades it to match the reference style. A dual-phase training scheme, DP-CGT, combines supervised preset learning with self-supervised refinement on unpaired photographs. CanonCGT delivers photorealistic and tonally consistent results across diverse datasets, surpassing state-of-the-art methods in stability and visual fidelity. Our codes are available at \href https://github.com/Jinwon-Ko/CanonCGT https://github.com/Jinwon-Ko/CanonCGT .

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4dcbd75f9a7751cd4184ae4eccee4039CVF · CVPR 2026

CodeMMR: Bridging Natural Language, Code, and Image for Unified Retrieval

Geng, Jiahui, Li, Qing, Cai, Fengyu, Karray, Fakhri

原文摘要

Code search, framed as information retrieval (IR), underpins modern software engineering and increasingly powers retrieval-augmented generation (RAG), improving code discovery, reuse, and the reliability of LLM-based coding.Yet existing code IR models remain largely text-centric and often overlook the visual and structural aspects inherent in programming artifacts such as web interfaces, data visualizations, SVGs, schematic diagrams, and UML.To bridge this gap, we introduce MMCoIR, the first comprehensive benchmark for evaluating multimodal code IR across five visual domains, and show through extensive evaluation the task is challenging.Therefore, we then propose CodeMMR, a unified retrieval model that jointly embeds natural language, code, and images into a shared semantic space through instruction-based multimodal alignment.CodeMMR achieves strong generalization across modalities and languages, outperforming competitive baselines (e.g., UniIR, GME, VLM2Vec) by an average of 10 points on nDCG@10.Moreover, integrating CodeMMR into RAG enhances code generation fidelity and visual grounding on unseen code generation tasks, underscoring the potential of multimodal retrieval as a core enabler for next-generation intelligent programming systems.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4deabc751073c01f9641f6f96f5f4cb1CVF · CVPR 2026

Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generation

Xue, Bowen, Duan, Zheng-Peng, Yan, Qixin, Wang, Wenjing, Liu, Hao, Guo, Chun-Le, Li, Chongyi, Li, Chen 等 9 位作者

原文摘要

Generating high-fidelity human videos that match user-specified identities is important yet challenging in the field of generative AI.Existing methods often rely on an excessive number of training parameters and lack compatibility with other AIGC tools.In this paper, we propose Stand-In, a lightweight and plug-and-play framework for identity preservation in video generation.Specifically, we introduce a conditional image branch into the pre-trained video generation model.Identity control is achieved through restricted self-attentions with conditional position mapping.Thanks to these designs, which greatly preserve the pretrained prior of the video generation model, our approach is able to outperform other full-parameter training methods in video quality and identity preservation, even with just 1% additional parameters and only 2000 training pairs.Moreover, our framework can be seamlessly integrated for other tasks, such as subject-driven video generation, pose-referenced video generation, stylization, and face swapping.Code and dataset will be available to the community.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4df31166211406d797b1af4121241e89CVF · CVPR 2026

CIGMA: Causal Information-Gain Mechanistic Attribution of Attention Heads in Vision Transformers

Maliha, Maisha, Hougen, Dean F.

原文摘要

Vision Transformers often rely on spurious background correlations rather than foreground object features. While prior model pruning approaches focus solely on improving accuracy, they lack interpretability and fail to verify whether predictions are actually made by focusing on the main foreground object, providing no causal validation of which components drive spurious behavior. We introduce Causal Information Gain Mechanistic Attribution (CIGMA), a general framework for explaining the internal computation of Vision Transformers. CIGMA provides a mechanistic, information theoretic explanation by quantifying the importance of each attention head and determining whether it supports the main object or routes spurious background cues. It ranks attention heads by measuring object versus context reliance with Jensen Shannon based information gain computed from the model's full predictive distributions after two complementary edits, removing the object region and removing the surrounding context, which reveals a spurious subnet that carries background signals and a complementary set of evidence aligned heads. Evaluated on CIFAR-10, CIFAR-100, and Tiny-ImageNet across three VLM architectures (InternVL2-26B, LLaVA-1.6, LLaVA-1.5-13B), CIGMA improves accuracy by 7.6 to 24.8 percentage points over unmodified models while reducing background reliance by 79.5% to 88.1%, substantially outperforming all baselines, demonstrating that causal head-level interventions enable more effective spurious correlation mitigation than token pruning or retraining approaches.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e0fd25c30f55c14774e86dcf9a4d325CVF · CVPR 2026

Stereo World Model: Camera-Guided Stereo Video Generation

Sun, Yang-Tian, Huang, Zehuan, Niu, Yifan, Ma, Lin, Cao, Yan-Pei, Ma, Yuewen, Qi, Xiaojuan

原文摘要

We present StereoWorld, a camera-conditioned stereo world model that jointly learns appearance and binocular geometry for end-to-end stereo video generation.Unlike monocular RGB or RGBD approaches, StereoWorld operates exclusively within the RGB modality, while simultaneously grounding geometry directly from disparity. To efficiently achieve consistent stereo generation, our approach introduces two key designs: (1) a unified camera-frame RoPE that augments latent tokens with camera-aware rotary positional encoding, enabling relative, view- and time-consistent conditioning while preserving pretrained video priors via a stable attention initialization; and (2) a stereo-aware attention decomposition that factors full 4D attention into 3D intra-view attention plus horizontal row attention, leveraging the epipolar prior to capture disparity-aligned correspondences with substantially lower compute. Across benchmarks, StereoWorld improves stereo consistency, disparity accuracy, and camera-motion fidelity over strong monocular-then-convert pipelines, achieving more than 3x faster generation with an additional 5% gain in viewpoint consistency. Beyond benchmarks, StereoWorld enables end-to-end binocular VR rendering without depth estimation or inpainting, enhances embodied policy learning through metric-scale depth grounding, and is compatible with long-video distillation for extended interactive stereo synthesis.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e10b8ebc0acb2fb4dfad37e7a461847CVF · CVPR 2026

OMoBlur: An Object Motion Blur Dataset and Benchmark for Real-World Local Motion Deblurring

Yu, Dingchuan, Li, Jiatong, Zhou, Jingwen, Zhuge, Zhengyue, Chen, Yueting, Li, Qi

原文摘要

Object motion blur in static scenes is spatially heterogeneous, differing from conventional deblurring problems yet frequently occurring in real handheld capture scenarios. Existing datasets either rely on costly beam-splitting capture with residual misalignment or employ synthetic blur that fails to model the continuous photon-integration process during exposure. To overcome these limitations, we introduce OMoBlur, a physically grounded dataset that emulates realistic exposure integration via programmable sensor control, ensuring close alignment between synthetic and real blur distributions. OMoBlur provides 20,000 blur-sharp-mask pairs covering diverse object motion types. Leveraging this dataset, we further propose OMDNet, an object-motion-aware deblurring network that integrates a Motion-Appearance Extract Block, a Flow-Guided Gate Predictor, and an Adaptive Gated Fusion mechanism. This design enables the network to selectively restore blurred regions while preserving static backgrounds, without requiring pixel-accurate mask annotations. Extensive experiments demonstrate that OMoBlur's physically faithful data collection and large-scale diversity significantly enhance the network's generalization to real-world motion blur, establishing OMoBlur and OMDNet as a robust benchmark and practical solution for local motion deblurring. The dataset and code is publicly released at https://yudingchuan.github.io/OMoBlur_homepage/.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e132934f3b55aac6d3015fe44e2eab9CVF · CVPR 2026

Act2See: Emergent Active Visual Perception for Video Reasoning

Ma, Martin Q., Qu, Yuxiao, Agrawal, Aditya, Guo, Willis, Liang, Paul Pu, Salakhutdinov, Ruslan, Morency, Louis-Philippe

原文摘要

Vision-Language Models (VLMs) typically rely on static initial frames for video reasoning, restricting their ability to incorporate essential dynamic information as the reasoning process evolves. Existing methods that augment Chain-of-Thought (CoT) with additional frame information often exhibit suboptimal CoT quality and lack the crucial ability to synthesize visual information for hypothetical or counterfactual scenarios. We introduce Act-to-See (Act2See), a novel framework that enables active visual perception by empowering VLMs to actively interleave video frames within text CoTs. Act2See is developed via Supervised Fine-Tuning (SFT) on a high-quality dataset of reasoning traces generated by a frontier VLM. These traces integrate active calls to either retrieve existing frames or generate new ones, and are rigorously verified against human-annotated CoTs to ensure quality. This approach cultivates an emergent capability: at inference time, the model actively determines when to search for or synthesize the necessary visual evidence. Act2See establishes new state-of-the-art results on challenging benchmarks, including VideoEspresso and ViTIB, and outperforms comparable or larger models on Video-MME, EgoNormia, and VCR-Bench, demonstrating an advancement in enabling VLMs with active visual perception for video reasoning. Code: https://github.com/martinmamql/act2see.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e25d23e85c7d843462b2325624b71eeCVF · CVPR 2026

NOVA: Sparse Control, Dense Synthesis for Pair-Free Video Editing

Pan, Tianlin, Dai, Jiayi, Yuan, Chenpu, Lv, Zhengyao, Yang, Binxin, Yin, Hubery, Li, Chen, Lyu, Jing 等 10 位作者

原文摘要

Recent video editing models have achieved impressive results, but most still require large-scale paired datasets. Collecting such naturally aligned pairs at scale remains highly challenging and constitutes a critical bottleneck, especially for local video editing data. Existing workarounds transfer image editing to video through global motion control for pair-free video editing, but such designs struggle with background and temporal consistency. In this paper, we propose NOVA: Sparse Control & Dense Synthesis, a new framework for video editing. Specifically, the sparse branch provides semantic guidance through user-edited keyframes distributed across the video, and the dense branch continuously incorporates motion and texture information from the original video to maintain high fidelity and coherence. Moreover, we introduce a degradation-simulation training strategy that enables the model to learn motion reconstruction and temporal consistency by training on artificially degraded videos, thus eliminating the need for paired data. Our extensive experiments demonstrate that NOVA outperforms existing approaches in edit fidelity, motion preservation, and temporal coherence.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e49b64c80a1beefce02704358b311d0CVF · CVPR 2026

Taming Preference Mode Collapse via Directional Decoupling Alignment in Diffusion Reinforcement Learning

Chen, Chubin, Hu, Sujie, Zhu, Jiashu, Wu, Meiqi, Chen, Jintao, Li, Yanxun, Huang, Nisha, Fang, Chengyu 等 11 位作者

原文摘要

Recent studies have demonstrated significant progress in aligning text-to-image diffusion models with human preference via Reinforcement Learning from Human Feedback. However, while existing methods achieve high scores on automated reward metrics, they often lead to Preference Mode Collapse (PMC)-a specific form of reward hacking where models converge on narrow, high-scoring outputs (e.g., images with monolithic styles or pervasive overexposure), severely degrading generative diversity. In this work, we introduce and quantify this phenomenon, proposing DivGenBench, a novel benchmark designed to measure the extent of PMC. We posit that this collapse is driven by over-optimization along the reward model's inherent biases. Building on this analysis, we propose Directional Decoupling Alignment (D^2-Align), a novel framework that mitigates PMC by directionally correcting the reward signal.Specifically, our method first learns a directional correction within the reward model's embedding space while keeping the model frozen.This correction is then applied to the reward signal during the optimization process, preventing the model from collapsing into specific modes and thereby maintaining diversity. Our comprehensive evaluation, combining qualitative analysis with quantitative metrics for both quality and diversity, reveals that D^2-Align achieves superior alignment with human preference.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e630011af6fa2a4150cd15014eadae6CVF · CVPR 2026

Hier-COS: Making Deep Features Hierarchy-aware via Composition of Orthogonal Subspaces

Sani, Depanshu, Anand, Saket

原文摘要

Traditional classifiers treat all class labels as mutually independent, thereby considering all negative classes to be equally incorrect. This approach fails severely in many real-world scenarios, where a known semantic hierarchy defines a partial order of preferences over negative classes. While hierarchy-aware feature representations have shown promise in mitigating this problem, their performance is typically assessed using metrics like Mistake Severity (MS) and Average Hierarchical Distance (AHD). In this paper, we highlight important shortcomings in existing hierarchical evaluation metrics, demonstrating that they are often incapable of measuring true hierarchical performance. Our analysis reveals that existing methods learn sub-optimal hierarchical representations, despite competitive MS and AHD scores. To counter these issues, we introduce Hierarchical Composition of Orthogonal Subspaces (Hier-COS), a novel framework for unified 'hierarchy-aware fine-grained' and 'hierarchical multi-label' classification. We show that Hier-COS is theoretically guaranteed to be consistent with the given hierarchy tree. Furthermore, our framework implicitly adapts the learning capacity for different classes based on their position within the hierarchy tree -- a vital property absent in existing methods. Finally, to address the limitations of evaluation metrics, we propose Hierarchically Ordered Preference Score (HOPS), a ranking-based metric that demonstrably overcomes the deficiencies of current evaluation standards. We benchmark Hier-COS on four challenging datasets, including the deep and imbalanced tieredImageNet-H (12-level) and iNaturalist-19 (7-level). Through extensive experiments, we demonstrate that Hier-COS achieves state-of-the-art performance across all hierarchical metrics for every dataset, while simultaneously beating the top-1 accuracy in all but one case. Lastly, we show that Hier-COS can effectively learn to transform the frozen features extracted from a pretrained backbone (ViT) to be hierarchy-aware, yielding substantial benefits for hierarchical classification performance.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e6eccf72cc00ecf7a2c274b998e1c46CVF · CVPR 2026

FoV-Net: Rotation-Invariant CAD B-rep Learning via Field-of-View Ray Casting

Ballegeer, Matteo, Benoit, Dries F.

原文摘要

Learning directly from boundary representations (B-reps) has significantly advanced 3D CAD analysis. However, state-of-the-art B-rep learning methods rely on absolute coordinates and normals to encode global context, making them highly sensitive to rotations. Our experiments reveal that models achieving over 95% accuracy on aligned benchmarks can collapse to as low as 10% under arbitrary SO(3) rotations. To address this, we introduce FoV-Net, the first B-rep learning framework that captures both local surface geometry and global structural context in a rotation-invariant manner. Each face is represented by a Local Reference Frame (LRF) UV-grid that encodes its local surface geometry, and by Field-of-View (FoV) grids that capture the surrounding 3D context by casting rays and recording intersections with neighboring faces. Lightweight CNNs extract per-face features, which are propagated over the B-rep graph using a graph attention network. FoV-Net achieves state-of-the-art performance on B-rep classification and segmentation benchmarks, demonstrating robustness to arbitrary rotations while also requiring less training data to achieve strong results.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e7637fff8d2ebb020adbaea2e536399CVF · CVPR 2026

NeuroRule: Bridging Vision and Logic with Differentiable Rule Induction

Zarar, Muhammad, Zhang, Mingzheng, Zhang, Xiaowang, Feng, Zhiyong

原文摘要

Scene Graph Generation (SGG) aims to structurally represent visual scenes by detecting objects and their pairwise relationships. Despite significant progress, current models encode visual knowledge with ambiguous visual context and logically inferred implicit relations due to their purely neural, pipeline-based nature. This limitation underscores the need to advance beyond identifying what relations exist to explaining why they exist and how they can be compositionally reasoned about through logical rule chaining. To address these challenges, we introduce NeuroRule, the first Neurally-Guided Rule Induction Network that integrates Mask2Former pixel-precise visual understanding with a differentiable rule induction engine. Our proposed method enables automatic learning of compositional logical rules directly from visual data while providing transparent explanations for relational predictions. NeuroRule introduces three key innovations: (1) a neural-symbolic bridge that maps visual features to probabilistic symbolic representations; (2) a differentiable rule-learning mechanism that automatically discovers interpretable first-order logic rules without manual engineering; and (3) a compositional chain rule system that enables complex inference while propagating confidence scores through an end-to-end trainable pipeline. Extensive experiments on the benchmark datasets, including Visual Genome (VG), Panoptic Scene Graph (PSG), and OpenPSG, demonstrate that NeuroRule achieves state-of-the-art performance. Our method significantly improves few-shot relation extraction while maintaining full interpretability in its rule-based explanations.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e79445ebe81dcce8d233423d51fa332CVF · CVPR 2026

Hoi! - A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation

Engelbracht, Tim, Zurbrügg, René, Wohlrapp, Matteo, Büchner, Martin, Valada, Abhinav, Pollefeys, Marc, Blum, Hermann, Bauer, Zuria

原文摘要

We present a dataset for force-grounded, cross-view articulated manipulation that couples what is seen with what is done and what is felt during real human interaction. The dataset contains 3048 sequences across 381 articulated objects in 38 environments. Each object is operated in four embodiments - (i) human hand, (ii) human hand with a wrist-mounted camera, (iii) handheld UMI gripper, and (iv) a custom Hoi! gripper, where the tool embodiment provides end-effector forces and tactile sensing. Our dataset offers a holistic view of interaction understanding from video, enabling researchers to evaluate how well methods transfer between human and robotic viewpoints, but also investigate underexplored modalities such as interaction forces. The Project Website can be found at https://hoi-dataset.ethz.ch.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4e9b08ccf467c14a8da5113793bce586CVF · CVPR 2026

Vision-Language Attribute Disentanglement and Reinforcement for Lifelong Person Re-Identification

Xu, Kunlun, Cheng, Haotong, Li, Jiangmeng, Zou, Xu, Zhou, Jiahuan

原文摘要

Lifelong person re-identification (LReID) aims to learn from varying domains to obtain a unified person retrieval model. Existing LReID approaches typically focus on learning from scratch or a visual classification-pretrained model, while the Vision-Language Model (VLM) has shown generalizable knowledge in a variety of tasks. Although existing methods can be directly adapted to the VLM, since they only consider global-aware learning, the fine-grained attribute knowledge is underleveraged, leading to limited acquisition and anti-forgetting capacity. To address this problem, we introduce a novel VLM-driven LReID approach named Vision-Language Attribute Disentanglement and Reinforcement (VLADR). Our key idea is to explicitly model the universally shared human attributes to improve inter-domain knowledge transfer, thereby effectively utilizing historical knowledge to reinforce new knowledge learning and alleviate forgetting. Specifically, VLADR includes a Multi-grain Text Attribute Disentanglement mechanism that mines the global and diverse local text attributes of an image. Then, an Inter-domain Cross-modal Attribute Reinforcement scheme is developed, which introduces cross-modal attribute alignment to guide visual attribute extraction and adopts inter-domain attribute alignment to achieve fine-grained knowledge transfer. Experimental results demonstrate that our VLADR outperforms the state-of-the-art methods by 1.9%-2.2% and 2.1%-2.5% on anti-forgetting and generalization capacity. Our source code is available at https://github.com/zhoujiahuan1991/CVPR2026-VLADR

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:4ea5fdfbb6a498ad5b7f925e1d8fd61eCVF · CVPR 2026

Bootstrapping Video Semantic Segmentation Model via Distillation-assisted Test-Time Adaptation

Kim, Jihun, Kwon, Hoyong, Kweon, Hyeokjun, Yoon, Kuk-Jin

原文摘要

Fully supervised Video Semantic Segmentation (VSS) relies heavily on densely annotated video data, limiting practical applicability. Alternatively, applying pre-trained Image Semantic Segmentation (ISS) models frame-by-frame avoids annotation costs but ignores crucial temporal coherence. Recent foundation models such as SAM2 enable high-quality mask propagation yet remain impractical for direct VSS due to limited semantic understanding and computational overhead. In this paper, we propose DiTTA (Distillation-assisted Test-Time Adaptation), a novel framework that converts an ISS model into a temporally-aware VSS model through efficient test-time adaptation (TTA), without annotated videos. DiTTA distills SAM2's temporal segmentation knowledge into the ISS model during a brief, single-pass initialization phase, complemented by a lightweight temporal fusion module to aggregate cross-frame context. Crucially, DiTTA achieves robust generalization even when adapting with highly limited partial video snippets (e.g., initial 10%), significantly outperforming zero-shot refinement approaches that repeatedly invoke SAM2 during inference. Extensive experiments on VSPW and Cityscapes demonstrate DiTTA's effectiveness, achieving competitive or superior performance relative to fully-supervised VSS methods, thus providing a practical and annotation-free solution for real-world VSS tasks.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗