跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1新增论文
+1其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-13 07:07 · 全站最近入库：2026-11-13 06:39各信源进度 →

统计口径与数据新鲜度

统计 2026-11-12 09:06 — 2026-11-13 09:06（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-13 07:07最近发现新增版本：2026-11-13 06:39库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,676 已收录论文18,324 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 5,655 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-13 07:05（北京时间）
首次观测 · 记录 #1CVF · CVPR 2026:cvf:f9ef5aa823c91384287f9ffb42b2f436CVF · CVPR 2026

Obstruction Reasoning for Robotic Grasping

Jiao, Runyu, Bortolon, Matteo, Giuliari, Francesco, Fasoli, Alice, Povoli, Sergio, Mei, Guofeng, Wang, Yiming, Poiesi, Fabio

原文摘要

Successful robotic grasping in cluttered environments not only requires a model to visually ground a target object but also to reason about obstructions that must be cleared beforehand. While current vision-language embodied reasoning models show emergent spatial understanding, they remain limited in terms of obstruction reasoning and accessibility planning. To bridge this gap, we present UNOGrasp, a learning-based vision-language model capable of performing visually-grounded obstruction reasoning to infer the sequence of actions needed to unobstruct the path and grasp the target object. We devise a novel multi-step reasoning process based on obstruction paths originated by the target object. We anchor each reasoning step with obstruction-aware visual cues to incentivize reasoning capability. UNOGrasp combines supervised and reinforcement finetuning through verifiable reasoning rewards. Moreover, we construct UNOBench, a large-scale dataset for both training and benchmarking, based on MetaGraspNetV2, with over 100k obstruction paths annotated by humans with obstruction ratios, contact points, and natural-language instructions. Extensive experiments and real-robot evaluations show that UNOGrasp significantly improves obstruction reasoning and grasp success across both synthetic and real-world environments, outperforming generalist and proprietary alternatives.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fa2c285aae2f1c0449f7f8dbd4df6dd3CVF · CVPR 2026

LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models

Huang, Guolei, Peng, Qinzhi, Xu, Gan, Huang, Yao, Lu, Yuxuan, Shen, Yongjun

原文摘要

As Vision-Language Models (VLMs) move into interactive, multi-turn use, safety concerns intensify for multimodal multi-turn dialogue, which is characterized by concealment of malicious intent, contextual risk accumulation, and cross-modal joint risk. These characteristics limit the effectiveness of content moderation approaches designed for single-turn or single-modality settings. To address these limitations, we first construct the Multimodal Multi-turn Dialogue Safety (MMDS) dataset, comprising 4,484 annotated dialogues and a comprehensive risk taxonomy with 8 primary and 60 subdimensions. As part of MMDS construction, we introduce Multimodal Multi-turn Red Teaming (MMRT), an automated framework for generating unsafe multimodal multi-turn dialogues. We further propose LLaVAShield, which audits the safety of both user inputs and assistant responses under specified policy dimensions in multimodal multi-turn dialogues. Extensive experiments show that LLaVAShield significantly outperforms state-of-the-art VLMs and existing content moderation tools while demonstrating strong generalization and flexible policy adaptation. Additionally, we analyze vulnerabilities of mainstream VLMs to harmful inputs and evaluate the contribution of key components, advancing understanding of safety mechanisms in multimodal multi-turn dialogues.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fa64739b4ee897ff7420b1cc6bc9be2cCVF · CVPR 2026

Masking Matters: Unlocking the Spatial Reasoning Capabilities of LLMs for 3D Scene-Language Understanding

Jeon, Yerim, Lee, Miso, Moon, WonJun, Heo, Jae-Pil

原文摘要

Recent advances in 3D scene-language understanding have leveraged Large Language Models (LLMs) for 3D reasoning by transferring their general reasoning ability to 3D multi-modal contexts. However, existing methods typically adopt standard decoders from language modeling, which rely on a causal attention mask. This design introduces two fundamental conflicts in 3D scene understanding: sequential bias among order-agnostic 3D objects and restricted object-instruction attention, hindering task-specific reasoning. To overcome these limitations, we propose 3D Spatial Language Instruction Mask (3D-SLIM), an effective masking strategy that replaces the causal mask with an adaptive attention mask tailored to the spatial structure of 3D scenes. Our 3D-SLIM introduces two key components: a Geometry-adaptive Mask that constrains attention based on spatial density rather than token order, and an Instruction-aware Mask that enables object tokens to directly access instruction context. This design allows the model to process objects based on their spatial relationships while being guided by the user's task. 3D-SLIM is simple, requires no architectural modifications, and adds no extra parameters, yet it yields substantial performance improvements across diverse 3D scene-language tasks. Extensive experiments across multiple benchmarks and LLM baselines validate its effectiveness and underscore the critical role of decoder design in 3D multi-modal reasoning.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:faab9c8db7078640c58f791c3353d8e1CVF · CVPR 2026

An Empirical Study on How Video-LLMs Answer Video Questions

Gou, Chenhui, Ma, Ziyu, Duan, Zicheng, He, Haoyu, Chen, Feng, Liu, Akide, Zhuang, Bohan, Cai, Jianfei 等 9 位作者

原文摘要

Taking advantage of large-scale data and pretrained language models, Video Large Language Models (Video-LLMs) have shown strong capabilities in answering video questions. However, most existing efforts focus on improving performance, with limited attention to understanding their internal mechanisms. This paper aims to bridge this gap through a systematic empirical study. To interpret existing VideoLLMs, we adopt attention knockouts as our primary analytical tool and design three variants: Video Temporal Knockout, Video Spatial Knockout, and Language-to-Video Knockout. Then, we apply these three knockouts on different numbers of layers (window of layers). By carefully controlling the window of layers and types of knockouts, we provide two settings: a global setting and a fine-grained setting. Our study reveals three key findings: (1) Global setting indicates Video information extraction primarily occurs in early layers, forming a clear two-stage process--lower layers focus on perceptual encoding, while higher layers handle abstract reasoning; (2) In the fine-grained setting, certain intermediate layers exert an outsized impact on video question answering, acting as critical outliers, whereas most other layers contribute minimally; (3) In both settings, we observe that spatial-temporal modeling relies more on language-guided retrieval than on intra- and inter-frame self-attention among video tokens, despite the latter's high computational cost. Finally, we demonstrate that these insights can be leveraged to reduce attention computation in Video-LLMs. To our knowledge, this is the first work to systematically uncover how Video-LLMs internally process and understand video content, offering interpretability and efficiency perspectives for future research.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fad880dea5c0b8330e24788f8dbf4f90CVF · CVPR 2026

When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm

Leng, Ye, Chu, Junjie, Li, Mingjie, Lin, Chenhao, Shen, Chao, Backes, Michael, Shen, Yun, Zhang, Yang

原文摘要

Recently, multimodal large language models (MLLMs) have emerged as a unified paradigm for language and image generation. Compared with diffusion models, MLLMs possess a much stronger capability for semantic understanding, enabling them to process more complex textual inputs and comprehend richer contextual meanings. However, this enhanced semantic ability may also introduce new and potentially greater safety risks. Taking diffusion models as a reference point, we systematically analyze and compare the safety risks of emerging MLLMs along two dimensions: unsafe content generation and fake image synthesis. Across multiple unsafe generation benchmark datasets, we observe that MLLMs tend to generate more unsafe images than diffusion models. This difference partly arises because diffusion models often fail to interpret abstract prompts, producing corrupted outputs, whereas MLLMs can comprehend these prompts and generate unsafe content. For current advanced fake image detectors, MLLM-generated images are also notably harder to identify. Even when detectors are retrained with MLLMs-specific data, they can still be bypassed by simply providing MLLMs with longer and more descriptive inputs. Our measurements indicate that the emerging safety risks of the cutting-edge generative paradigm, MLLMs, have not been sufficiently recognized, posing new challenges to real-world safety.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fb5598277e05bc36fb692ebde030a427CVF · CVPR 2026

DuoGen: Towards Autonomous Interleaved Multimodal Generation

Shi, Min, Zeng, Xiaohui, Huang, Jiannan, Cui, Yin, Ferroni, Francesco, Li, Jialuo, Li, Zhaoshuo, Balaji, Yogesh 等 15 位作者

原文摘要

Interleaved multimodal generation enables capabilities beyond unimodal generation models, such as step-by-step instructional guides, visual planning, and generating visual drafts for reasoning. However, the quality of existing interleaved generation models under general instructions remains limited by insufficient training data and base model capacity. We present DuoGen, an interleaved generation framework that systematically addresses data curation, architecture design, and evaluation. On the data side, we build a large-scale, high-quality instruction-tuning dataset by combining multimodal conversations rewritten from curated raw websites, and diverse synthetic examples covering everyday scenarios. Architecturally, DuoGen leverages the strong visual understanding of a pretrained multimodal LLM and the visual generation capabilities of a diffusion transformer (DiT) pretrained on video generation, avoiding costly unimodal pretraining and enabling flexible base model selection. A two-stage decoupled strategy first instruction-tunes the MLLM, then aligns DiT with it using curated interleaved image-text sequences. Across public and newly proposed benchmarks, DuoGen outperforms prior open-source models in text quality, image fidelity, and image-context alignment, and also achieves state-of-the-art performance on text-to-image and image editing among unified generation models. Data and code are released at https://research.nvidia.com/labs/dir/duogen/

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fb8cdbaa7e058817416ca1d370a151b0CVF · CVPR 2026

DeepAlign: Mitigating Modality Conflict through Modality-Specific Alignment

Li, Shuo, Miao, Bingchen, Bu, Wendong, Li, Juncheng, Zhang, Hanwang, Wu, Fei

原文摘要

Multimodal Large Language Models (MLLMs) have demonstrated promising advancements in augmenting the capabilities of LLMs to comprehend visual input. However, modality misalignment between vision and text remains a key challenge in MLLM, which can be attributed to two aspects: misalignment of modality-specific representations and depletion of modality-specific details. To address the issue of modality misalignment, we propose DeepAlign, a novel multimodal alignment framework to mitigate modality conflict, which employs representation intervention and structure-induced knowledge distillation to prevent the misalignment and depletion of modality-specific information. Extensive experiments demonstrate that DeepAlign significantly mitigates modality conflicts, leading to substantial performance improvements compared to backbone models across multiple vision-language tasks. It also stimulates some emergent abilities in MLLMs, such as multimodal in-context learning on interleaved text-image sequences.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fbcedea7487baa0474d90b55ad5ea60dCVF · CVPR 2026

Do You See What I Am Pointing At? Gesture-Based Egocentric Video Question Answering

Choi, Yura, Miles, Roy, Potamias, Rolandos Alexandros, Elezi, Ismail, Deng, Jiankang, Zafeiriou, Stefanos

原文摘要

Understanding and answering questions based on a user's pointing gesture is essential for next-generation egocentric AI assistants. However, current Multimodal Large Language Models (MLLMs) struggle with such tasks due to the lack of gesture-rich data and their limited ability to infer fine-grained pointing intent from egocentric video.To address this, we introduce EgoPointVQA, a dataset and benchmark for gesture-grounded egocentric question answering, comprising 4000 synthetic and 400 real-world videos across multiple deictic reasoning tasks.Built upon it, we further propose Hand Intent Tokens (HINT), which encode tokens derived from 3D hand keypoints using an off-the-shelf reconstruction model and interleaves them with the model input to provide explicit spatial and temporal context for interpreting pointing intent.We show that our model outperforms others in different backbones and model sizes.In particular, HINT-14Bachieves 68.1% accuracy, on average over 6 tasks, surpassing the state-of-the-art, InternVL3-14B, by 6.6%.To further facilitate the open research, we will release the code, model, and dataset.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fc20e0981dd18c7a27efd4320d4f048aCVF · CVPR 2026

Towards Unified Human Perception and Machine Understanding: Token Flow Guided Compression Framework

Xu, Li, Zhang, Yingfu, Xu, Kepeng, He, Gang, Li, Yunsong

原文摘要

With the rapid rise of Large Vision Language Models (LVLMs) for image understanding, the objective of image compression is gradually shifting from human visual perception to machine-oriented semantic understanding. However, conventional learned compression techniques are optimized for pixel-level fidelity and typically operate at fixed or rigid bitrate points, misaligned with the semantic consistency and flexible bitrate control. This gap becomes critical in ultra-low-bitrate regimes, where latent representations often ignore semantic relevance and struggle to disentangle meaningful content from redundant visual details as the bitrate varies. To address these challenges, we develop a token-based flexible compression framework, Token Flow Guided Compression (TFGC), which unifies human- and machine-oriented objectives. TFGC supports variable bitrate control in ultra-low bitrate regimes and enables LVLMs to directly process compressed token without image reconstruction. Specifically, we explore token flow phenomenon in 1D token sequences and exploit it to design token flow propagation, which predicts missing tokens by propagating contextual information from unmasked tokens. Moreover, token semantic guidance aligns compressed representations with LVLM semantic space, while a progressive semantic alignment training strategy further bridges the gap between perceptual reconstruction and semantic reasoning. Experiments show that our framework achieves state-of-the-art LVLMs understanding at comparable bitrates while maintaining satisfactory perceptual quality.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fc5107a0b4b53d9b92718e7cc8a4feb2CVF · CVPR 2026

Retrieving Counterfactuals Improves Visual In-Context Learning

Xiong, Guangzhi, Sinha, Sanchit, He, Zhenghao, Zhang, Aidong

原文摘要

Vision-language models (VLMs) have achieved impressive performance across a wide range of multimodal reasoning tasks, but they often struggle to disentangle fine-grained visual attributes and reason about underlying causal relationships. In-context learning (ICL) offers a promising avenue for VLMs to adapt to new tasks, but its effectiveness critically depends on the selection of demonstration examples. Existing retrieval-augmented approaches typically rely on passive similarity-based retrieval, which tends to select correlated but non-causal examples, amplifying spurious associations and limiting model robustness. We introduce CIRCLES (Composed Image Retrieval for Causal Learning Example Selection), a novel framework that actively constructs demonstration sets by retrieving counterfactual-style examples through targeted, attribute-guided composed image retrieval. By incorporating counterfactual-style examples, CIRCLES enables VLMs to implicitly reason about the causal relations between attributes and outcomes, moving beyond superficial correlations and fostering more robust and grounded reasoning. Comprehensive experiments on four diverse datasets demonstrate that CIRCLES consistently outperforms existing methods across multiple architectures, especially on small-scale models, with pronounced gains under information scarcity. Furthermore, CIRCLES retrieves more diverse and causally informative examples, providing qualitative insights into how models leverage in-context demonstrations for improved reasoning. Our code is available at https://github.com/gzxiong/CIRCLES.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fc823183aaf55cb756a85c90c43851bdCVF · CVPR 2026

POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs

Wang, Haicheng, Liu, Yuan, Liu, Yikun, Yu, Zhemeng, Zhao, Zhongyin, You, Yangxiu, Yu, Zilin, Tian, Le 等 12 位作者

原文摘要

Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable capabilities in cross-modal understanding and generation. However, the rapid growth of visual token sequences--especially in long-video and streaming scenarios--poses a major challenge to their scalability and real-world deployment. Thus, we introduce POINTS-Long, a native dual-mode MLLM featuring dynamic visual token scaling inspired by the human visual system. The model supports two complementary perception modes: focus mode and standby mode, enabling users to dynamically trade off efficiency and accuracy during inference. On fine-grained visual tasks, the focus mode retains the optimal performance, while on long-form general visual understanding, the standby mode retains 97.7-99.7% of the original accuracy using only 1/40-1/10th of the visual tokens. Moreover, POINTS-Long natively supports streaming visual understanding via a dynamically detachable KV-cache design, allowing efficient maintenance of ultra-long visual memory. Our work provides new insights into the design of future MLLMs and lays the foundation for adaptive and efficient long-form visual understanding. Model and code are available at https://anakin-skywalker-joseph.github.io/POINTS-Long-Webpage.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fca5adbc83001601ccd4dde7f636e87bCVF · CVPR 2026

VecGlypher: Unified Vector Glyph Generation with Language Models

Huang, Xiaoke, Gauri, Bhavul, Ng, Kam Woh, Ng, Tony, Xu, Mengmeng, Liu, Zhiheng, Ren, Weiming, An, Zhaochong 等 15 位作者

原文摘要

Vector glyphs are the atomic units of digital typography, yet most learning-based pipelines still depend on carefully curated exemplar sheets and raster-to-vector postprocessing, which limits accessibility and editability. We introduce VecGlypher, a single multimodal language model that generates high-fidelity vector glyphs directly from text descriptions or image exemplars. Given a style prompt, optional reference glyph images, and a target character, VecGlypher autoregressively emits SVG path tokens, avoiding raster intermediates and producing editable, watertight outlines in one pass. A typography-aware data and training recipe makes this possible: (i) a large-scale continuation stage on 39K noisy Envato fonts to master SVG syntax and long-horizon geometry, followed by (ii) post-training on 2.5K expert-annotated Google Fonts with descriptive tags and exemplars to align language and imagery with geometry; preprocessing normalizes coordinate frames, canonicalizes paths, de-duplicates families, and quantizes coordinates for stable long-sequence decoding. On cross-family OOD evaluation, VecGlypher substantially outperforms both general-purpose LLMs and specialized vector-font baselines for text-only generation, while image-referenced generation reaches a state-of-the-art performance, with marked gains over DeepVecFont-v2 and DualVector. Ablations show that model scale and the two-stage recipe are critical and that absolute-coordinate serialization yields the best geometry. VecGlypher lowers the barrier to font creation by letting users design with words or exemplars, and provides a scalable foundation for future multimodal design tools.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fccccce0f4c872f041f14580b4b2008dCVF · CVPR 2026

UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifying

Bai, Chengyu, Chen, Jintao, Bai, Xiang, Chen, Yilong, She, Qi, Lu, Ming, Zhang, Shanghang

原文摘要

Recent advances in diffusion models and vision-language models (VLMs) have significantly enhanced the controllability of image editing. Methods like FlowEdit enable step-by-step editing along a visible, noise-free trajectory, where each intermediate result is a clear image, eliminating the need for full noise inversion. However, these approaches still operate in pixel space or VAE latent space, where intermediate outputs often suffer from visual artifacts, distortions, or unrealistic details--making reliable semantic evaluation difficult. Furthermore, they remain open-loop systems, applying static edits without feedback to guide or correct the editing process adaptively. We propose UniEdit-I, the first training-free, closed-loop image editing framework that operates entirely within the semantic latent space of a unified VLM by introducing an Understanding-Editing-Verifying (UEV) loop:(1) Understanding: parses the source image and editing instruction into a structured source prompt and a minimal target specification;(2) Editing: applies dynamic semantic offsets, with a configurable feedback weighting mechanism that adaptively modulates editing intensity based on real-time alignment feedback;(3) Verifying: leverages the VLM's own multimodal reasoning capability to evaluate the intermediate output along multiple semantic dimensions and trigger early stopping or refinement. By transforming the VLM from a post-hoc evaluator into an in-process conductor, UniEdit-I establishes the first semantics-driven, self-correcting closed-loop image editing pipeline. Evaluated on GEdit-Bench, UniEdit-I achieves state-of-the-art performance without any fine-tuning or architectural modifications, and even surpasses several large-scale pre-trained editors.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fcf28201ff00a815dfc1ef96be524ad2CVF · CVPR 2026

OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarios

Gao, Hong, Wu, Jingyu, Xu, Xiangkai, Xie, Kangni, Zhang, Yunchen, Zhong, Bin, Gao, Xurui, Zhang, Min-Ling

原文摘要

Spatio-Temporal Video Grounding (STVG) aims to localize target objects in videos based on natural language descriptions. While Multimodal Large Language Models have shown promise, a significant gap remains between current models and real-world demands involving diverse objects and complex queries. We attribute this to limited benchmark scope, causing models to exhibit category bias, oversimplified reasoning, and poor linguistic robustness.To address these limitations, we introduce OmniGround, a comprehensive benchmark with 3,475 videos spanning 81 categories and complex real-world queries. We propose the Forward-Backward-Refinement (FBR) annotation pipeline for high-quality labels and DeepSTG, a systematic evaluation framework quantifying dataset quality beyond superficial statistics. Evaluations reveal performance average drops of 10.4% on complex real-world scenes, particularly with small/occluded objects and intricate spatial relations. Motivated by these, we propose PG-TAF, a training-free two-stage framework decomposing STVG into high-level temporal grounding and fine-grained spatio-temporal propagation. Experiments demonstrate PG-TAF achieves 25.6% and 35.6% improvements in m_tIoU and m_vIoU on OmniGround with consistent gains across four benchmarks.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fd4854e0c84c8bfdcd9ea23e325732f8CVF · CVPR 2026

Grounded 3D-Aware Spatial Vision-Language Modeling

Cheng, An-Chieh, Fu, Yang, Ji, Yatai, Zhu, Ligeng, Zhan, Guanqi, Zhang, Zhuoyang, Yang, Zhaojing, Han, Song 等 15 位作者

原文摘要

We present GR3D, a spatial vision language model equipped with three complementary grounding capabilities--explicit 2D grounding, implicit 2D grounding, and monocular 3D grounding--within a single framework. GR3D introduces an implicit grounding mechanism that identifies entity mentions during generation and inserts the corresponding region tokens into the text stream, allowing the model to reference visual evidence on the fly when producing spatial chain-of-thought responses. In parallel, a region-prompted monocular 3D grounding design predicts 3D bounding boxes in the camera view from grounded region queries, supported by intrinsic-aware normalization and dense geometric supervision. Together, these grounding capabilities enable GR3D to decompose complex spatial understanding problems into grounded 2D perception followed by 3D inference. GR3D achieves consistent improvements across grounded and non-grounded spatial benchmarks, demonstrating grounding as an effective inductive bias for strengthening spatial understanding in VLMs. These grounding capabilities collectively enhance general spatial understanding beyond the grounding task itself.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fda836b140a8435071e9d9a20881b9e6CVF · CVPR 2026

TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detection

Jiang-Lin, Jian-Yu, Huang, Kang-Yang, Zou, Ling, Lo, Ling, Yang, Sheng-Ping, Tseng, Yu-Wen, Lin, Kun-Hsiang, Chen, Chia-Ling 等 14 位作者

原文摘要

Advances in generative modeling have made it increasingly easy to fabricate realistic portrayals of individuals, creating serious risks for security, communication, and public trust. Detecting such person-driven manipulations requires systems that not only distinguish altered content from authentic media but also provide clear and reliable reasoning. In this paper, we introduce TriDF, a comprehensive benchmark for interpretable DeepFake detection. TriDF contains high-quality forgeries from advanced synthesis models, covering 16 DeepFake types across image, video, and audio modalities. The benchmark evaluates three key aspects: Perception, which measures the ability of a model to identify fine-grained manipulation artifacts using human-annotated evidence; Detection, which assesses classification performance across diverse forgery families and generators; and Hallucination, which quantifies the reliability of model-generated explanations. Experiments on state-of-the-art multimodal large language models show that accurate perception is essential for reliable detection, but hallucination can severely disrupt decision-making, revealing the interdependence of these three aspects. TriDF provides a unified framework for understanding the interaction between detection accuracy, evidence identification, and explanation reliability, offering a foundation for building trustworthy systems that address real-world synthetic media threats.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fe03613b86a6dc152c72ed92898a1aa7CVF · CVPR 2026

Phrase-Grounding-Aware Supervised Fine-Tuning for Chart Recognition via Side-Masked Attention

Ito, Koichiro

原文摘要

Recent advances in chart recognition have been driven by supervised fine-tuning (SFT) of vision-language models (VLMs), which unify multiple related tasks, and by diversifying training corpora. In parallel, research on leveraging large language models (LLMs) for object detection has shown that jointly training phrase grounding alongside SFT enhances a model's generative capabilities. Inspired by this, we hypothesize that chart recognition can also benefit from phrase grounding, which aligns textual phrases with chart regions. However, this setting remains underexplored due to the lack of corresponding datasets. In this work, we introduce phrase-grounding-aware SFT via a Side-Masked Attention Module (SMAM) inserted into each transformer layer of the LLM. SMAM performs masked attention within the annotated region aligned with each phrase to produce an additional logit. This logit is supervised and serves as a reference signal to guide the LLM's output prediction during fine-tuning, alongside the standard SFT objective. To enable this approach, we also develop an automated pipeline for generating phrase-to-region alignments, which augments existing datasets. Experiments show that our method effectively incorporates phrase grounding into VLM fine-tuning for chart recognition.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:fe392b8a0bfab0fc5a557da67fa858d3CVF · CVPR 2026

ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps

Feng, Sicheng, Wang, Song, Ouyang, Shuyi, Kong, Lingdong, Song, Zikai, Zhu, Jianke, Wang, Huan, Wang, Xinchao

原文摘要

Multimodal large language models (MLLMs) have demonstrated significant progress in semantic scene understanding and text-image alignment, with reasoning variants enhancing performance on more complex tasks involving mathematics and logic. However, their proficiency in tasks requiring both fine-grained visual understanding and spatial reasoning remains underexplored. To bridge this gap, we introduce ReasonMap, a novel benchmark specifically designed to evaluate these capabilities. ReasonMap encompasses high-resolution transit maps from 30 cities and includes 1,008 question-answer pairs spanning two question types and three templates. Furthermore, we design a two-level evaluation pipeline that properly assesses answer correctness and quality. Our comprehensive evaluation of 16 popular MLLMs reveals a counterintuitive pattern: among open-source models, base variants outperform their reasoning-tuned counterparts, whereas the opposite trend is observed in closed-source models. Further analysis under the visual-masking setting confirms that strong performance necessitates direct visual grounding, rather than relying solely on language priors. We further establish a training baseline with reinforcement fine-tuning, providing a reference for future exploration. We hope this benchmark study offers new insights into visual reasoning and helps investigate the gap between open- and closed-source models.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:ff1cbf82ff0ee863a29b8a62ce2d1ff5CVF · CVPR 2026

Beyond Layer-Wise Merging: Chain-of-Merging for Vision-Language Models

Zhang, Xinyu, Dong, Yuxuan, Zhang, Lingling, Jia, Chengyou, Dang, ZhuoHang, Yao, Yixing, Wu, Yaqiang, Fernando, Basura 等 9 位作者

原文摘要

While model merging has demonstrated remarkable success for large language models (LLMs), its application to vision-language models (VLMs) remains largely underexplored. Recent methods attempt to enhance VLM reasoning capabilities by integrating specialized LLM parameters through layer-wise merging. However, existing paradigms suffer from two critical limitations: (1) strict positional correspondence, enforcing rigid one-to-one layer alignment, and (2) uniform merging weights applied across all layers. These constraints fail to account for substantial functional disparities between VLM and LLM layers, potentially misaligning incompatible layers and leading to detrimental parameter combinations. To address these, we propose Chain-of-Merging (CoM), a framework that adaptively adjusts merging plans for different images and questions, comprising two key stages: (1) Adaptive Layer Matching, which identifies optimal layer pairings based on structural and semantic matching scores while filtering incompatible pairings, and (2) Dynamic Weight Merging, which determines layer-specific merging weights based on matching scores and employs spherical linear interpolation to minimize memory overhead. Extensive experiments demonstrate that CoM achieves substantial performance improvements, with Qwen2.5-VL-7B + Qwen2.5-Math-7B attaining a 4.4% average improvement on mathematical reasoning benchmarks while enhancing general visual understanding, significantly outperforming existing training-free methods.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:ff3c36df99da2da93748d50980bfe4aeCVF · CVPR 2026

SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs

Yip, Koonting, Zhao, Qiyan, Yu, Wenhao, Yuen, Liangyu, Li, Mingkai, Zhang, Xiaofeng, Ji, Jianmin, Zhang, Yanyong 等 10 位作者

原文摘要

3D Large Vision-Language Models (3D LVLMs) built upon Large Language Models (LLMs) have achieved remarkable progress across various multimodal tasks. However, their inherited position-dependent modeling mechanism, Rotary Position Embedding (RoPE), remains suboptimal for 3D multimodal understanding. The vanilla RoPE formulation fails to preserve essential three-dimensional spatial structures when encoding 3D tokens, and its relative distance computation overlooks angular dependencies hindering the model's ability to capture directional variations in visual representations. To overcome these limitations, we introduce Spherical Coordinate-based Positional Embedding (SoPE). Our method maps point-cloud token indices into a 3D spherical coordinate space, enabling unified modeling of spatial locations and directional angles. This formulation preserves the inherent geometric structure of point-cloud data, enhances spatial awareness, and yields more consistent and expressive geometric representations for multimodal learning. In addition, we introduce a multi-scale frequency mixing strategy to fuse feature information across different frequency domains. Experimental results on multiple 3D scene benchmarks validate the effectiveness of our approach, while real-world deployment experiments further demonstrate its strong generalization capability.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:ff8d0ffa3a5b230c68d1e2efeb031c68CVF · CVPR 2026

Multi-Paradigm Collaborative Adversarial Attack Against Multi-Modal Large Language Models

Li, Yuanbo, Xu, Tianyang, Hu, Cong, Zhou, Tao, Wu, Xiao-Jun, Kittler, Josef

原文摘要

The rapid progress of Multi-Modal Large Language Models (MLLMs) has significantly advanced downstream applications. However, this progress also exposes serious transferable adversarial vulnerabilities. In general, existing adversarial attacks against MLLMs typically rely on surrogate models trained within a single learning paradigm and perform independent optimisation in their respective feature spaces. This straightforward setting naturally restricts the richness of feature representations, delivering limits on the search space and thus impeding the diversity of adversarial perturbations. To address this, we propose a novel Multi-Paradigm Collaborative Attack (MPCAttack) framework to boost the transferability of adversarial examples against MLLMs. In principle, MPCAttack aggregates semantic representations, from both visual images and language texts, to facilitate joint adversarial optimisation on the aggregated features through a Multi-Paradigm Collaborative Optimisation (MPCO) strategy. By performing contrastive matching on multi-paradigm features, MPCO adaptively balances the importance of different paradigm representations and guides the global perturbation optimisation, effectively alleviating the representation bias. Extensive experimental results on multiple benchmarks demonstrate the superiority of MPCAttack, indicating that our solution consistently outperforms state-of-the-art methods in both targeted and untargeted attacks on open-source and closed-source MLLMs. The code is released at https://github.com/LiYuanBoJNU/MPCAttack.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:ffd4177497dde9e413787bfe30514498CVF · CVPR 2026

RNN as Linear Transformer: A Closer Investigation into Representational Potentials of Visual Mamba Models

Yang, Timing, Wang, Feng, Wei, Guoyizhe

原文摘要

Mamba, originally introduced for language modeling, has recently garnered attention as an effective backbone for vision tasks. However, its underlying mechanism in visual domains remains poorly understood. In this work, we systematically investigate Mamba's representational properties and make three primary contributions. First, we theoretically analyze Mamba's relationship to Softmax and Linear Attention, confirming that it can be viewed as a low-rank approximation of Softmax Attention and thereby bridging the representational gap between Softmax and Linear forms. Second, we introduce a novel binary segmentation metric for activation map evaluation, extending qualitative assessments to a quantitative measure that demonstrates Mamba's capacity to model long-range dependencies. Third, by leveraging DINO for self-supervised pretraining, we obtain clearer activation maps than those produced by standard supervised approaches, highlighting Mamba's potential for interpretability. Notably, our model also achieves a 78.5% linear probing accuracy on ImageNet, underscoring its strong performance. We hope this work can provide valuable insights for future investigations of Mamba-based vision architectures.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1CVF · CVPR 2026:cvf:ffe518d430e60f6ad39d1c0fb18b61b0CVF · CVPR 2026

IF-Prune: Information-Flow Guided Token Pruning for Efficient Vision-Language Models

Sun, Guohao, Wang, Yufei, Ma, Sizhuo, Xie, Yuege, Cheng, Yuting, Tao, Zhiqiang, Wang, Jian

原文摘要

Vision-language models (VLMs) with dynamic resolution vision encoders achieve strong performance, but face significant efficiency challenges due to long input sequences. A common approach is to assess the importance of tokens and prune those that are less informative. Recent methods utilizing a small VLM to provide the importance map of visual tokens have outperformed existing rule-based and similarity-driven pruning approaches, particularly under high pruning ratios. However, directly using the small VLM remains unreliable, as it utilizes the aggregated visual attention weights as importance score, which can lead to noisy guidance if the generated tokens are incorrect.To address this, we invert the approach by having it detect non-informative visual tokens according to the user's input query. By adding a variational information bottleneck in the small VLM, we can approximate the entropy of each visual token as pruning guidance. Such a posteriori-guided pruning method allows the large VLM to retain its reasoning capacity with improved efficiency. Extensive experiments on eight benchmarks demonstrate the effectiveness of our approach. With only 5% of visual tokens retained, the large VLM preserves 95% of its original performance, outperforming the state of the art by 8%.

cs.CV · cs.CL

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

查看论文
CVF · CVPR 2026 ↗PDF ↗
首次观测 · 记录 #1软件学报:jos:7557软件学报

基于大语言模型的非功能需求生成方法

欧阳柳波, 叶巧莹, 孟心如, 杜漫茹

原文摘要

在软件工程领域中, 非功能需求(non-functional requirement, NFR)获取一直是需求工程实践中的重要内容, 但容易被忽视. 传统的NFR获取方法主要依赖需求工程师的经验和人工分析, 不仅效率低下, 而且容易出现遗漏和不一致. 近年来, 大语言模型在自然语言处理领域取得突破性进展, 为自动化获取非功能需求提供了新的技术手段. 然而, 直接使用大语言模型生成非功能需求常面临知识幻觉、领域专业性不足等问题. 为此, 提出一种基于大语言模型的非功能需求自动获取方法, 实现高质量的非功能需求生成. 构建包含3856条功能需求和5723条非功能需求的结构化关联数据集, 形成22647对FR-NFR关联关系. 通过融合检索增强生成(retrieval-augmented generation, RAG)技术, 构建包含3个核心模块的系统化解决方案: 基于语义相似度的案例检索模块、面向非功能需求生成的提示工程模块和基于大语言模型的需求生成模块. 通过软件工程专家的专业评分和对BLEU、ROUGE等自动评分指标的多维度评估, 实验结果表明该方法在需求完整性、准确性、可测试性和一致性方面优于现有方法.

cs.* · cs.CL

软件学报 · 2025 · 正式发表

查看论文
软件学报 ↗PDF ↗
版本更新 · v2arXiv:2505.22657Google Research

3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model

Wenbo Hu, Yining Hong, Yanjun Wang, Leison Gao, Zibu Wei, Xingcheng Yao, Nanyun Peng, Yonatan Bitton 等 10 位作者

原文摘要

Humans excel at performing complex tasks by leveraging long-term memory across temporal and spatial experiences. In contrast, current Large Language Models (LLMs) struggle to effectively plan and act in dynamic, multi-room 3D environments. We posit that part of this limitation is due to the lack of proper 3D spatial-temporal memory modeling in LLMs. To address this, we first introduce 3DMem-Bench, a comprehensive benchmark comprising over 26,000 trajectories and 2,892 embodied tasks, question-answering and captioning, designed to evaluate an agent's ability to reason over long-term memory in 3D environments. Second, we propose 3DLLM-Mem, a novel dynamic memory management and fusion model for embodied spatial-temporal reasoning and actions in LLMs. Our model uses working memory tokens, which represents current observations, as queries to selectively attend to and fuse the most useful spatial and temporal features from episodic memory, which stores past observations and interactions. Our approach allows the agent to focus on task-relevant information while maintaining memory efficiency in complex, long-horizon environments. Experimental results demonstrate that 3DLLM-Mem achieves state-of-the-art performance across various tasks, outperforming the strongest baselines by 16.5% in success rate on 3DMem-Bench's most challenging in-the-wild embodied tasks.

cs.CV · cs.AI · cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2512.10968Google Research

Benchmarking Automatic Speech Recognition Models for African Languages

Alvin Nahabwe, Sulaiman Kagumire, Denis Musinguzi, Bruno Beijuka, Jonah Mubuuke Kyagaba, Peter Nabende, Andrew Katumba, Joyce Nakatumba-Nabende

原文摘要

Automatic speech recognition (ASR) for African languages remains constrained by limited labeled data and the lack of systematic guidance on model selection, data scaling, and decoding strategies. Large pre-trained systems such as Whisper, XLS-R, MMS, and W2v-BERT have expanded access to ASR technology, but their comparative behavior in African low-resource contexts has not been studied in a unified and systematic way. In this work, we benchmark four state-of-the-art ASR models across 13 African languages, fine-tuning them on progressively larger subsets of transcribed data ranging from 1 to 400 hours. Beyond reporting error rates, we provide new insights into why models behave differently under varying conditions. We show that MMS and W2v-BERT are more data efficient in very low-resource regimes, XLS-R scales more effectively as additional data becomes available, and Whisper demonstrates advantages in mid-resource conditions. We also analyze where external language model decoding yields improvements and identify cases where it plateaus or introduces additional errors, depending on the alignment between acoustic and text resources. By highlighting the interaction between pre-training coverage, model architecture, dataset domain, and resource availability, this study offers practical and insights into the design of ASR systems for underrepresented languages.

cs.CL · cs.SD · eess.AS

查看论文
arXiv ↗PDF ↗
版本更新 · v5arXiv:2504.13837Google Research

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, Gao Huang

原文摘要

Reinforcement Learning with Verifiable Rewards (RLVR) has recently demonstrated notable success in enhancing the reasoning performance of large language models (LLMs), particularly on mathematics and programming tasks. Similar to how traditional RL helps agents explore and learn new strategies, RLVR is believed to enable LLMs to continuously self-improve, thus acquiring novel reasoning abilities beyond those of the corresponding base models. In this study we critically examine the current state of RLVR by systematically probing the reasoning capability boundaries of RLVR-trained LLMs across various model families, RL algorithms, and math, coding, and visual reasoning benchmarks, using pass@k at large k values as the evaluation metric. Surprisingly, we find that the current training setup does not elicit fundamentally new reasoning patterns. While RLVR-trained models outperform their base models at small k (e.g., k = 1), the base models achieve a higher pass@k score when k is large. Coverage and perplexity analyses show that the observed reasoning abilities originate from and are bounded by the base model. Treating the base model as an upper bound, our quantitative analysis shows that six popular RLVR algorithms perform similarly and remain far from optimal in leveraging the potential of the base model. By contrast, we find that distillation can introduce new reasoning patterns from the teacher and genuinely expand the model's reasoning capabilities. Overall, our findings suggest that current RLVR methods have not yet realized the potential of RL to elicit truly novel reasoning abilities in LLMs. This highlights the need for improved RL paradigms, such as continual scaling and multi-turn agent-environment interaction, to unlock this potential.

cs.AI · cs.CL · cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2507.16795Google DeepMind

Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning

Helena Casademunt, Caden Juang, Adam Karvonen, Samuel Marks, Senthooran Rajamanoharan, Neel Nanda

原文摘要

Fine-tuning large language models (LLMs) can lead to unintended out-of-distribution generalization. Standard approaches to this problem rely on modifying training data, for example by adding data that better specify the intended generalization. However, this is not always practical. We introduce Concept Ablation Fine-Tuning (CAFT), a technique that leverages interpretability tools to control how LLMs generalize from fine-tuning, without needing to modify the training data or otherwise use data from the target distribution. Given a set of directions in an LLM's latent space corresponding to undesired concepts, CAFT works by ablating these concepts with linear projections during fine-tuning, steering the model away from unintended generalizations. We successfully apply CAFT to three fine-tuning tasks, including emergent misalignment, a phenomenon where LLMs fine-tuned on a narrow task generalize to give egregiously misaligned responses to general questions. Without any changes to the fine-tuning data, CAFT reduces misaligned responses by 10x without degrading performance on the training distribution. Overall, CAFT represents a novel approach for steering LLM generalization without modifying training data.

cs.LG · cs.AI · cs.CL

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2502.09863Berkeley BAIR

闭式训练动力学揭示类Word2Vec模型中学到的特征与线性结构

Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models

Dhruva Karkada, James B. Simon, Yasaman Bahri, Michael R. DeWeese

AI 速读 基于摘要

作者对word2vec损失的四次Taylor近似进行解析求解，得到仅由语料统计和超参数表达的梯度流训练动力学与最终词嵌入，并据此解释线性子空间与语义概念的形成。

cs.LG · cs.CL · stat.ML

arXiv ↗PDF ↗
查看原文摘要

Self-supervised word embedding algorithms such as word2vec provide a minimal setting for studying representation learning in language modeling. We examine the quartic Taylor approximation of the word2vec loss around the origin, and we show that both the resulting training dynamics and the final performance on downstream tasks are empirically very similar to those of word2vec. Our main contribution is to analytically solve for both the gradient flow training dynamics and the final word embeddings in terms of only the corpus statistics and training hyperparameters. The solutions reveal that these models learn orthogonal linear subspaces one at a time, each one incrementing the effective rank of the embeddings until model capacity is saturated. Training on Wikipedia, we find that each of the top linear subspaces represents an interpretable topic-level concept. Finally, we apply our theory to describe how linear representations of more abstract semantic concepts emerge during training; these can be used to complete analogies via vector addition.

版本更新 · v2arXiv:2510.06124Google Research

Taxonomy of User Needs and Actions

Renee Shelby, Fernando Diaz, Vinodkumar Prabhakaran

原文摘要

The growing ubiquity of conversational AI highlights the need for frameworks that capture not only users' instrumental goals but also the situated, adaptive, and social practices through which they achieve them. Existing taxonomies of conversational behavior either overgeneralize, remain domain-specific, or reduce interactions to narrow dialogue functions. To address this gap, we introduce the Taxonomy of User Needs and Actions (TUNA), an empirically grounded framework developed through iterative qualitative analysis of 1193 human-AI conversations, supplemented by theoretical review and validation across diverse contexts. TUNA organizes user actions into a three-level hierarchy encompassing behaviors associated with information seeking, synthesis, procedural guidance, content creation, social interaction, and meta-conversation. By centering user agency and appropriation practices, TUNA enables multi-scale evaluation, supports policy harmonization across products, and provides a backbone for layering domain-specific taxonomies. This work contributes a systematic vocabulary for describing AI use, advancing both scholarly understanding and practical design of safer, more responsive, and more accountable conversational systems.

cs.HC · cs.CL

查看论文
arXiv ↗PDF ↗