跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1072新增论文
+845其中 AI 相关
+94已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-09 14:26 · 全站最近入库：2026-11-09 13:22各信源进度 →

统计口径与数据新鲜度

统计 2026-11-08 14:48 — 2026-11-09 14:48（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-09 14:26最近发现新增版本：2026-11-09 13:22库内论文最新更新时间：2026-11-07 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

18,172 已收录论文15,506 AI 相关4,527 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 6,491 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-09 14:25（北京时间）
新论文 · v1arXiv:2611.04524

E2I-Flow: EEG-to-Image Reconstruction via Wavelet-Temporal Cross-Attention and Conditional Flow Matching

Haowei Pan, Hwa Hui Tew, Junn Yong Loo, Fuad Noman, Hernando Ombao, Chee-Ming Ting

原文摘要

Electroencephalography (EEG)-based visual decoding and reconstruction aims to recover visual information from non-invasive brain signals. But there are challenges due to the complex temporal-frequency characteristics of EEG signals and the difficulty of discrete denoising framework in mapping neural activity to meaningful visual representations. Here, we propose E2I-Flow, an EEG-to-image decoding and reconstruction framework that combines a novel EEG encoder for time-frequency representation learning from EEG features, and conditional flow matching for visual decoding. A stationary-wavelet frequency branch and a temporal-mixing branch are integrated through bidirectional cross-attention and adaptive fusion, with the resulting EEG representations aligned to ground-truth image embeddings using contrastive and mean-squared-error loss supervisions. The aligned EEG representation is then used as a condition for a Transformer-based flow-matching model to predict the corresponding visual embedding for image synthesis. Experimental results show strong zero-shot EEG-to-image retrieval and improved visual reconstruction performance, outperforming several state-of-the-art baselines.

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2508.12969

Compact Attention: Exploiting Structured Spatio-Temporal Sparsity for Fast Video Generation

Qirui Li, Guangcong Zheng, Qi Zhao, Jie Li, Bin Dong, Yiwu Yao, Xuewei Li, Zequn Qin 等 9 位作者

原文摘要

The quadratic computational complexity of self-attention mechanisms imposes a critical constraint on transformer-based video generation, particularly for synthesizing ultra-long and high-definition sequences. While most sparse attention techniques employ fixed or heuristic patterns, they often struggle to accommodate the heterogeneous and layer-specific redundancies inherent in video diffusion transformers (VDiTs). We find that different attention heads consistently concentrate most of their attention mass in different coarse query-key regions, a behavior we call functional specialization. Although individual attention values vary with the generated content, these regions remain similar across prompts and random seeds. Leveraging these insights, we introduce Compact Attention, a hardware-aware sparse attention framework for efficient long-video synthesis. We quantify this stability and use it to construct searchable, reusable, head-specific masks. Based on the observation, we propose a deformable dual-window sparse pattern mechanism that flexibly approximates local and axial attention regions within one tile-aligned mask family. To avoid manually choosing a mask for each head, we develop a recall-driven offline auto-search algorithm that identifies high-quality sparse masks while leveraging temporal stability for mask reuse. Extensive evaluations on Wan2.1 (14B) and Hunyuan-Video demonstrate that Compact Attention achieves 2.25x/3.03x attention speedups over FlashAttention-2 and 1.20x/1.74x over FlashAttention-3, respectively, while maintaining visual quality and temporal consistency comparable to full-attention baselines. Project Page: https://yo-ava.github.io/Compact-Attention.github.io/

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04498

3DGS-VBench+ and 3DGSV-Assessor: A Multi-Dimensional Subjective Benchmark and a Reference-Flexible Objective Model for 3D Gaussian Splatting Video Quality Assessment

Yuke Xing, William Gordon, Jiarui Wang, Yujie Zhang, Guangtao Zhai, Yiling Xu

原文摘要

3D Gaussian Splatting (3DGS) enables real-time novel view synthesis (NVS), but its heavy storage makes compression essential, and both training and compression introduce representation-specific distortions. Since generative compression in many methods treats geometric and color attributes via different parameters, it can induce dimension-specific distortions, motivating multi-dimensional assessment. Moreover, real-world captures generally lack an aligned ground-truth video for a synthesized, continuous camera trajectory, leaving only sparse captured views as references. To enable multi-dimensional quality assessment of 3DGS videos with scarce aligned references, we first present 3DGS-VBench+, a large-scale subjective video quality assessment (VQA) benchmark comprising 3{,}036 3DGS reconstructions, each paired with a rendered video, across 11 real-world and 22 synthetic scenes, generated by 6 representative compression algorithms at systematically designed levels and annotated with 9{,}108 MOSs covering overall, geometry, and color quality. Based on 3DGS-VBench+, we propose 3DGSV-Assessor, a reference-flexible, large multimodal framework that scores all three dimensions in one generation pass and fuses a ranking-based reference cue into the frozen regressor through a bounded gate, supporting both reduced-reference (RR) and no-reference (NR) assessment. Extensive experiments demonstrate state-of-the-art correlation with human perception on 3DGS-VBench+ and strong zero-shot generalization to other NVS benchmarks. The dataset and code will be released at https://github.com/YukeXing/3DGSV-Assessor.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2604.01569

VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification

Jiahao Meng, Yue Tan, Qi Xu, Haochen Wang, Zhongwei Ren, Weisong Liu, Yuhao Wang, Ray Zhang 等 12 位作者

原文摘要

Video multimodal large language models achieve strong results on existing benchmarks, but answer accuracy alone does not establish whether they can locate the evidence needed to answer a question. We introduce VideoZeroBench, a challenging long-video benchmark with manually annotated question-answer pairs spanning 13 video domains. Questions target fine-grained cues, fleeting events, and evidence distributed across multiple segments. Temporal intervals and key-frame boxes are annotated where applicable. All questions undergo two rounds of cross-verification for answer validity and evidence quality. Our five-level diagnostic protocol compares answering with and without evidence hints, then combines answer correctness with independently evaluated temporal and spatial grounding. Across 19 evaluated models, the best standard QA accuracy is 24.8% (Level-3), achieved by Gemini-3.7-Flash. No model exceeds 1.8% when correct answers and accurate spatio-temporal localization are jointly required (Level-5). Analyses of atomic abilities, evidence spans, input modalities, and thinking-with-videos inference further characterize where the evaluated systems struggle. These findings motivate more precise evidence search and localization for long-video question answering. Our code and data are publicly released.

cs.CV · cs.MM

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04492

Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation

Jimmy Wang, Yifan Chou, Kaifeng Zhang, Yixuan Wang, Binghao Huang, Tao Yu, Saleh Nabi, Yunzhu Li

原文摘要

Predicting how the world evolves in response to robot actions is central to policy learning, evaluation, and planning. However, many action-conditioned world models predict future observations in 2D image or video space without explicitly modeling the underlying 3D scene, making spatial consistency across viewpoints and over long-horizon interactions challenging. We introduce 3D World Simulator, an action-conditioned multimodal world modeling framework that jointly predicts appearance, geometry, and tactile observations from a unified 3D representation. Our approach combines a hybrid particle-voxel autoencoder that compresses point clouds into compact 3D voxel latents with an action-conditioned latent dynamics model that predicts their evolution. The shared 3D representation supports geometrically consistent rendering from arbitrary viewpoints and flexible integration of spatially aligned tactile signals. Our experiments demonstrate interactive, long-horizon, multimodal prediction at 50 Hz. We then show how our model enables scalable data generation for policy training and evaluation using tactile and multi-view visual observations, including wrist-mounted and novel camera views. Furthermore, we scale our method to large datasets containing diverse objects and demonstrate model-based planning with unseen objects and robot configurations. Together, these results highlight the potential of hybrid particle-voxel dynamics as a foundation for multimodal 3D world simulation.

cs.RO · cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04491

SPA-Q: Structure-Preserving Adaptive Post-Training Quantization for Monocular Depth Estimation

Jaemin Choi, Jincheol Yang, Nahyun Lim, Yun-Seong Jeong, Matti Zinke, Hyunwoo Yu, Suk-Ju Kang

原文摘要

Monocular depth estimation (MDE) has advanced rapidly with the emergence of foundation models such as Depth Anything. Their transformer-based architectures provide strong generalization across diverse scenes and domains, but also incur high computational and memory cost, making efficient deployment challenging. Post-Training Quantization (PTQ) provides an efficient and practical solution for model compression, yet low-bit PTQ remains challenging for MDE models. When applying PTQ to Depth Anything, we identify two key challenges: (i) independently minimizing the quantization error of the query and key projections fails to preserve the attention maps induced by their interaction, and (ii) quantization errors accumulate across layers, resulting in intermediate feature distribution shifts. To address these issues, we propose SPA-Q, a Structure-Preserving Adaptive PTQ framework for MDE, consisting of two main components. First, Attention-Preserving Calibration (APC) determines query and key quantization parameters by matching the full-precision attention distribution. Second, Channel-Wise Distribution Alignment (CWDA) learns channel-wise affine transformations to mitigate quantization-induced distribution shifts, and the learned parameters are absorbed into the weights after training. Experimental results show that SPA-Q consistently outperforms existing PTQ methods under 4-bit quantization, achieving an average 25.9% reduction in AbsRel and a 17.3% improvement in $δ_1$ across NYUv2 and KITTI datasets.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04490

Polarimetric Scattering-Structure Contrastive Learning for Few-Shot HRRP Recognition

Fangyi Wang, Peng Zhao, Haipeng Wang

原文摘要

Few-shot high-resolution range profile (HRRP) recognition is challenged by strong aspect-dependent variation and sparse novel-class support. We propose polarimetric scattering-structure contrastive learning (PSS-CL), which supplies an episodic HRRP encoder with a structured physical training view. Dominant local responses are represented by their soft affinities to canonical polarimetric mechanisms, normalized range locations, and energy contributions. A symmetric cross-view objective transfers this scattering structure to the learned HRRP representation, while episodic prototype learning supports novel-class inference without the physical branch. We conduct Random-aspect and Cross-aspect experiments on a six-class measured dataset. In Cross-aspect 1-, 3-, and 5-shot recognition, PSS-CL achieves 81.73%, 86.92%, and 90.98% mean class accuracy (MCA), respectively, outperforming the best competing results by 5.62%, 3.09%, and 3.56%.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04489

Bridging CLIP with DINO: Cross-Modal Information Maximization for Online Test-Time Adaptation

Taehoon Kim, Wonjun Hwang, Hyunsouk Cho

原文摘要

Vision-language models such as CLIP enable zero-shot classification from arbitrary class names, but under test-time corruptions, their visual encoder projects distorted features onto incorrect textual anchors with high confidence: a phenomenon we term modality bias. Standard entropy minimization reinforces these overconfident errors, which creates a confirmation-bias loop and collapses the predicted label distribution. We propose an online test-time adaptation (TTA) method that breaks this loop at three levels. At the representation level, a frozen self-supervised backbone (e.g., DINO v2) is combined with CLIP through a cross-covariance logit ensemble. CLIP text prototypes are projected into the feature space of this text-free backbone, so the self-supervised branch classifies native image features and is less affected by the text bias of CLIP. At the optimization level, a gated test-time information-maximization objective pairs per-sample sharpening with batch-level diversity, while an adaptive gate excludes unreliable samples from the gradient. At the prediction level, an online logit adjustment corrects the residual class-distribution skew. CDIM outperforms the compared CLIP-based online TTA baselines on ImageNet-C, CIFAR10-C, and CIFAR100-C, and remains competitive in continual and mixed-domain TTA. Extensive analyses and ablation studies verify the contribution of each of its components. Our code is available at https://github.com/B1ueC1oud/CDIM

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04476

KeyWise: Efficient Video-Only Piano Transcription with a Lightweight Per-Key Architecture

Anastasios Chatzigiannis, Natalia Panagou, Maria Koziri

原文摘要

Audio-based piano transcription is hampered by the sustain pedal and by noisy or missing soundtracks, problems that video-only transcription avoids, since a top-view video shows directly when each key is pressed and released. Most visual transcription networks, however, describe the whole keyboard with one global feature vector and must learn where each key is. In this paper, we propose KeyWise, a lightweight per-key architecture that exploits the known geometry of the keyboard instead of learning it. The rectified keyboard is divided into 88 equal tiles that serve as regions of interest (ROIs), and one small recurrent network, shared by all keys, follows each key over time. On PianoVAM, KeyWise reaches the onset accuracy of the state-of-the-art V2N (95.0% vs. 94.7% onset F1 at 50 ms and 97.8% vs. 97.0% at 100 ms) with 1.3 M instead of 28.2 M parameters, and it runs 4.7 times faster on the same GPU. At 100 ms, it is also significantly more accurate when the key releases (97.0% vs. 95.5% F1) or the velocities (89.4% vs. 84.5% F1) must be correct as well. On PianoYT, KeyWise pre-trained on PianoVAM reaches 69.4% onset F1 at 100 ms, comparable to the best published result, although it needs fine-tuning for new recording conditions.

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.35706

TriTile: Three-Tier Attention for Video Diffusion Transformers

Yeonjae Kim, Omin Kwon, Seong Hoon Seo, Jae W. Lee

原文摘要

Attention over tens of thousands of visual tokens dominates the inference of video diffusion transformers (DiTs). Existing methods reduce this cost through sparsity, low precision, or both, but use a single precision for all kept tiles. We observe that the output error introduced by approximation varies widely across tiles. Many tiles can be skipped with little error, others must be computed but gain little beyond low precision, and a few retain substantial error even at low precision. We present TriTile, a training-free attention method that assigns each tile one of three approximation tiers: skipping it by replacing it with its block means, skimming it in NVFP4, or refining it with NVFP4 residual correction. A lightweight router predicts each tile's error under each tier from block statistics and spends extra compute on a tile only where it removes enough error; a single knob, a price on compute, controls the whole speed-quality trade-off. A fused kernel executes all three tiers in a single pass on the FP4 tensor cores. Over dense FlashAttention, TriTile speeds up generation by 1.38-1.83x on Wan2.2 at 23.0-27.6 dB PSNR, 2.28-2.32x on HunyuanVideo at 34.5-36.0 dB, and 1.84x on LTX-2.5 at 34.1 dB, and in every setting it Pareto-dominates every baseline in speed and PSNR.

cs.LG · cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04472

Beyond Token Representations: Explicit Visual Object Grounding for Video Reasoning Segmentation

Jun Huang, Junyang Zhang, Zijie Yue, Bingkun Wang, Yong Luo, Miaojing Shi

原文摘要

Video Reasoning Segmentation (VRS) aims to infer and segment target objects specified by reasoning queries in videos. Existing methods typically first infer the target and generate a few segmentation tokens for it, which are then fed into a mask decoder for mask prediction. However, such segmentation tokens tend to be both semantically monotonous and spatially ambiguous, making them largely insufficient to represent the target to guide mask prediction. In this paper, we propose an explicit visual object grounding framework for VRS. Specifically, we first propose a spatial-aware prompt generation and refinement scheme, in which we query a multimodal large language model to generate explicit spatial prompts (i.e., bounding box and point set) for the target and then refine them through multi-step dialogue. Furthermore, we introduce a query diversification and alignment module to generate auxiliary queries that describe the same target from different perspectives. We then enforce cross-query consistency between their predicted spatial prompts to alleviate the training bias caused by query limitations. Finally, we select several reliable frames using a prompt-based keyframe selection strategy to support mask decoding and propagation. Extensive experiments on eight datasets demonstrate that our method significantly outperforms state-of-the-art methods. The code is available at https://github.com/viscom-tongji/EVOG.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04470

Using GeoAI and machine learning tools for consistent high-resolution land cover mapping based on time-series NAIP imagery

Jie Liu, Xusheng Tang, Chao Wang, Zhengxiao Yan, Yuchen Dai, Qi Zhang, Conghe Song

原文摘要

High-resolution land-cover maps support ecological monitoring, but consistent historical time series remain scarce because aerial imagery varies across years and sensors. We developed a dual-track workflow to produce temporally coherent 1-m land-cover maps from National Agricultural Imagery Program (NAIP) imagery without historical training samples. For higher-quality imagery from 2009-2017, we used a pretrained U-Net/ResNet-34 model and refined boundaries with the Segment Anything Model (SAM). For lower-quality imagery from 2004-2008, we reconstructed land cover through an NLCD-based spatiotemporal bridging and label back-casting pipeline. Evaluation across six counties in North Carolina and Pennsylvania yielded overall accuracies of 0.887 (2014), 0.886 (2017), and 0.733 (2004). Structure, Water, Wetland, and Cropland consistently achieved high F1-scores. The workflow integrates GeoAI and multiple data sources to address temporal heterogeneity in historical aerial imagery, supporting high-resolution land-change analysis and landscape planning across the conterminous United States.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04467

Is Forgetting Unified? Exploring Concept Unlearning in Unified Multimodal Models

Zhongkai Xue, Junkai Chen, Mingcong Lei, Rumo Zhang, Yixiao Liu, Zhimin Sun, Shu Wu, Qiang Liu 等 10 位作者

原文摘要

Unified multimodal models (UMMs) integrate image generation and visual understanding, but are existing single-pathway unlearning methods sufficient to suppress target concepts across both? In this work, we study cross-pathway concept unlearning. Our goal is to prevent a model from both generating images of a target concept and identifying it in input images, while preserving other capabilities. To this end, we introduce UUBench, a diagnostic benchmark covering 30 target concepts and 60 semantically related neighboring concepts. Experiments on representative UMMs show that the evaluated single-pathway methods achieve limited joint forgetting: generation-side editors strongly suppress target generation but yield limited forgetting in visual understanding, while understanding-side baselines struggle to achieve substantial forgetting even in their own pathway. Their direct composition also yields limited joint forgetting. We therefore propose UniForget, achieving joint forgetting rates of 80.0% on BAGEL-7B-MoT and 92.7% on Janus-Pro-7B, outperforming representative baselines. Further analysis reveals trade-offs between joint forgetting and capability preservation.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2609.05351

MEOX: Modality-Conditioned Routing Before Fusion in a Compact Earth Observation Encoder

Mohanad Albughdadi

原文摘要

Multimodal Earth Observation models often fuse radar and optical inputs early, limiting sensor-conditioned processing, or maintain separate sensor streams through the encoder, increasing computational cost. We present MEOX (Multimodal Earth Observation with eXperts), which delays fusion until after a shared mixture-of-experts block that processes Sentinel-2 and ascending and descending Sentinel-1 streams independently. Co-located patches are then fused with validity-aware weights into a single sequence processed by fourteen shared blocks, reducing the sequence length from 3N + 5 to N + 5 tokens for N patches. We pretrain the 2.939M-parameter encoder on 1.228M MMEarth64 samples using masked radar-optical reconstruction. Under the reported frozen-transfer protocols, MEOX reaches 90.56% average class accuracy on EuroSAT and 64.42% mean intersection over union (mIoU) on cashew segmentation, 2.26 and 5.02 points above the strongest reported cross-sensor mixture-of-experts (CSMoE) variants. Full-encoder adaptation increases BigEarthNet micro-average precision from 57.22% to 72.95%. In a controlled same-corpus test using synthetic random patch removal, masking 75% of Sentinel-2 patches reduces WorldCover mIoU from 30.45% to 26.53% when both Sentinel-1 streams remain available; with Sentinel-2 alone, mIoU falls to 13.81%. For identical zeroed optical patches, declaring them missing through the validity mechanism improves mIoU from 19.42% to 26.53%. These results characterize a compact route-then-fuse encoder that retains sensor-conditioned processing and uses complementary radar observations when optical inputs are incomplete.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.33518

SceneScaffold: Active Scene-State Construction for Unified 3D Scene Understanding

Xiangqi Li, Libo Huang, Jiarui Zhao, Weilun Feng, Chuanguang Yang, Zhulin An, Yongjun Xu

原文摘要

Recent 3D large multimodal models (3D-LMMs) rely on a visual bottleneck to compress complex 3D scene evidence into a limited number of visual tokens compatible with large language models (LLMs). Current visual bottlenecks, however, often passively compress heterogeneous 3D evidence into a homogeneous object-centric token sequence, leaving the spatial organization of the scene under-represented. This under-representation forces the LLM to recover spatial relations from a flattened token sequence, leading to unstable reasoning in relation-intensive and spatially ambiguous scenes. To address this issue, we propose SceneScaffold, an active scene-state construction framework for unified 3D scene understanding. SceneScaffold reformulates the visual bottleneck from a passive feature compressor into an active scene organizer, constructing a role-aware spatial scaffold before language reasoning. Specifically, SceneScaffold organizes superpoint-level visual evidence into scene-state components with distinct structural roles: entity states preserve core object semantics, scene-frame states maintain spatial references via boundary and region anchors, relation states encode object-environment interaction cues, and a global summary provides compact context. Through this role-aware construction, SceneScaffold provides the LLM with a spatially organized scene representation before language reasoning. Experiments on unified 3D scene understanding tasks, including 3D visual grounding, question answering, and dense captioning, demonstrate the effectiveness of SceneScaffold, while diagnostic results further show its applicability to relation-intensive and spatially ambiguous cases. Code is available at https://github.com/lixiangqi707/SceneScaffold.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04452

DynOSeg: Dynamic Object Segmentation via Motion-Geometry Fusion

Xu Han, Yiduo Wang, Yiran Wang, Viorela Ila

原文摘要

Moving Object Segmentation (MOS) detects and separates dynamic objects from the static background in a video, which plays a fundamental role in dynamic-scene understanding. Most MOS methods rely on pure visual motion cues alone, such as optical flow or point tracks, which is inherently ambiguous due to the compounded camera and object motions as well as motion-induced appearance changes, e.g. shadows and reflections. Naive fusion of additional information such as depth and semantics, however, cannot leverage their complementarity to the fullest extent. We therefore propose DynOSeg, a novel MOS approach that takes on a two-stage fusion structure. DynOSeg first combines a geometry-aware residual map and depth encoding into a geometric representation of motion, delineating camera motions and object dynamics. We then further incorporate semantic features to refine object understanding. We have additionally curated a diverse training corpus including CARLA-MOS, a new synthetic urban-driving dataset for MOS training. Our experiments show that DynOSeg consistently outperforms published state-of-the-art MOS methods across a diverse range of datasets. Our code and pretrained models can be found at https://github.com/ACFR-RPG/DynOSeg

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.31033

SlotMemory: Object-Centric KV Memory for Streaming Long-Video Generation

Weijia Dou, Hui Li, Jiahao Cui, Lei Zhou, Jingdong Wang, Siyu Zhu

原文摘要

Streaming video generation models typically rely on temporal-centric memory, organizing historical context as latent queues, compressed context tokens, or unclustered token sets. This often causes identity drift and semantic inconsistency when entities exit the frame or during interactive prompt transitions. To address these limitations, we propose SlotMemory, an object-centric Key-Value (KV) memory mechanism for streaming video diffusion. Our approach shifts memory abstraction from ``when'' an event occurred to ``what'' is represented by decomposing the transformer's KV manifold into discrete, reusable semantic slots. Because the slots serve only as routing addresses and the stored content remains the original transformer KV tokens, the mechanism adds entity-level persistence and prompt-aware retrieval across long horizons. Evaluated on 60-second interactive narratives using the Wan2.1-T2V-1.3B backbone, SlotMemory achieves the highest Dynamic Degree among evaluated streaming methods and the second-highest aggregate Quality Score, preserving prompt adherence rather than trading it away for consistency. At the 30-second horizon, SlotMemory leads in Total, Quality, and Dynamic Degree, improving Dynamic Degree by 31.2% over MemFlow, the strongest memory-based baseline. Our results support treating structured semantic representation, rather than raw temporal capacity, as a critical primitive for persistent long-form video synthesis.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04442

Hybrid Quantum--Classical Pauli-Correlator Architecture for Image Reconstruction

Sparsho Chakraborty, Siddhartha Patra

原文摘要

We introduce the Pauli-Correlator Autoencoder (PCAE), a hybrid quantum--classical architecture for image reconstruction. PCAE splits an image into patches and compresses each patch as a matrix product state (MPS). A small quantum circuit turns each compressed patch into a few measured numbers, and a classical neural network rebuilds the patch from these numbers and the patch position. A learnable energy term, taken from physics, is added to training and tracked as training proceeds. Two versions place the six qubits on a line (PCAE1D) or on a grid (PCAE2D). An averaging step makes the PCAE2D features follow rotations and reflections of the image exactly. Trained on one image at a time, PCAE2D rebuilds images from six public datasets with a peak signal-to-noise ratio of 25.8 to 41.6 decibels. On CIFAR-10 images not used in training, a formula-based stand-in for the PCAE2D features performs within 1 decibel of several simpler classical methods. Five trained models also run on real IBM quantum computers without retraining, and rebuild images at 25.90 to 31.52 decibels. A short argument and a synthetic test show that, with a linear final layer, measuring qubits in pairs is needed for targets built from products of inputs. These results show that PCAE works as a hybrid model that rebuilds varied images and runs on today's quantum hardware at the six-qubit scale.

quant-ph · cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.34033

Fovea-JEPA: Overcoming Geometric Blind Spots in V-JEPA for Camera Motion Recognition

Xiangkun Shi, Hanhui Li, Shiyue Zhang, Xiaodan Liang

原文摘要

Camera motion recognition requires separating viewpoint changes caused by camera translation, rotation, or zoom from object motion and scene dynamics. Video-language models often rely on semantic cues and struggle with the geometric evidence needed for this task. Joint-Embedding Predictive Architectures such as V-JEPA offer strong spatio-temporal representations, but our experiments suggest that they underrepresent geometric cues in some regions. Under temporal-reversal supervision, V-JEPA improves overall but degrades on several motion families, whereas a weaker geometry-oriented VGGT predictor improves across all families. We refer to these geometrically underrepresented regions as geometric blind spots and propose Fovea-JEPA to identify and address them. Using V-JEPA as the base model and VGGT as a geometry expert, we construct intervention pseudo-labels from their prediction errors to weakly supervise a blind-spot predictor. The predictor produces token-wise response maps, which are aggregated into a clip-level intervention score to guide adaptive fusion of visual and geometric features. This selective intervention introduces complementary geometric information while preserving reliable visual predictions. Experiments on CameraMotionDataset, CameraBench, and ShotBench demonstrate the effectiveness of Fovea-JEPA, with gains of 9.67, 8.67, and 24.38 percentage points over previously reported state-of-the-art baselines, respectively. Code and models will be released upon acceptance.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2510.26441

A-TPT: Angular Diversity Calibration Properties for Test-Time Prompt Tuning of Vision-Language Models

Shihab Aaqil Ahamed, Udaya S. K. P. Miriya Thanthrige, Ranga Rodrigo, Muhammad Haris Khan

原文摘要

Test-time prompt tuning (TPT) has emerged as a promising technique for adapting large vision-language models (VLMs) to unseen tasks without relying on labeled data. However, the lack of dispersion between textual features can hurt calibration performance, which raises concerns about VLMs' reliability, trustworthiness, and safety. Current TPT approaches primarily focus on improving prompt calibration by either maximizing average textual feature dispersion or enforcing orthogonality constraints to encourage angular separation. However, these methods may not always have optimal angular separation between class-wise textual features, which implies overlooking the critical role of angular diversity. To address this, we propose A-TPT, a novel TPT framework that introduces angular diversity to encourage uniformity in the distribution of normalized textual features induced by corresponding learnable prompts. This uniformity is achieved by maximizing the minimum pairwise angular distance between features on the unit hypersphere. We show that our approach consistently surpasses state-of-the-art TPT methods in reducing the aggregate average calibration error while maintaining comparable accuracy through extensive experiments with various backbones on different datasets. Notably, our approach exhibits superior zero-shot calibration performance on natural distribution shifts and generalizes well to medical datasets. We provide extensive analyses, including theoretical aspects, to establish the grounding of A-TPT. These results highlight the potency of promoting angular diversity to achieve well-dispersed textual features, significantly improving VLM calibration during test-time adaptation. Our code will be made publicly available.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04438

Entropy-Gated Dynamic Rectification for Object Hallucination in Large Vision-Language Models

Riasad Alvi, Md. Asaduzzaman Shuvo, Md. Ahashan Habib, Md. Faiyaz Abdullah Sayeedi

原文摘要

Large Vision-Language Models (LVLMs) routinely name objects that are absent from the image. Latent steering can reduce this failure in one model forward pass, but a fixed gate treats early and late decoding steps by the same criterion. We observe that predictive entropy varies sharply within free-running generation, while states that commit to short answers occupy a narrow, confident band. We exploit this temporal mismatch by using uncertainty as a causally available, token-local control signal for an existing steering gate. We propose Entropy-Gated Dynamic Rectification (EGDR), which lowers the intervention threshold when entropy derived from the preceding step's already available logits is high and retains the calibrated operating point when it is low. On LLaVA-1.5-7B, the underlying static gate lowers CHAIR CS/CI from 53.00/15.30 to 32.00/14.30; replacing only its constant threshold with EGDR further lowers them to 29.00/13.60. At a matched 58% intervention rate, the corresponding static control obtains 30.50/14.00. The gain over the static gate holds on LLaVA-NeXT-7B and grows with the generation budget, while POPE scores and general multimodal capability remain effectively unchanged. EGDR requires no additional model forward pass and adds roughly 2% measured latency. Our code is available at https://github.com/alvi-uiu/EGDR.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04435

A Systematic Point Cloud Edge Detection Framework for Automatic Aircraft Skin Milling

Zijie Wu, Yaonan Wang, He Xie, Mingtao Feng, Haotian Wu, Chao Ding, Ajmal Mian

原文摘要

The edge detection technique is an essential step for aircraft skin milling in aviation manufacturing. Most of the current detection methods focus on traditionally defined edge extraction tasks but disregard the crucial systematic requirement of edge milling. In this paper, we proposed a novel edge detection framework for automatic edge milling of aircraft skins. First, an edge probability detector is proposed by the spatial tangent continuity (STC) to provide the essential reference. Second, we propose a hierarchical branch searching (HBS) method to hierarchically strip the desired milling edges from the raw point cloud, which consists of three graded progressive steps: branch backbone generation, branch extension, and branch pruning. We demonstrate the performance of the proposed method on both synthetic models and aircraft skin workpieces. The proposed method outperforms the other baselines and shows accurate edges for the edge milling task.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04423

SWAP-INR: Implicit Neural Representation with Switchable Attention Paths for Time-varying Multivariate Scientific Data Compression

Hyunsoo Son, Jeonghyun Noh, Chaoli Wang, Won-Ki Jeong

原文摘要

Visualizing multivariate scientific simulation data is essential for understanding complex physical phenomena, yet the massive scale of such data imposes a severe storage bottleneck in high-performance computing environments. Implicit neural representations (INRs) have emerged as a promising continuous alternative to traditional lossy compressors, but existing methods struggle to explicitly capture cross-variable interactions and to adapt to the diverse spectral characteristics of physical variables. To address these limitations, we propose SWAP-INR, an architecture for high-fidelity compression of multivariate, time-varying simulation data. SWAP-INR introduces two key components: (1) The switchable attention path (SWAP) module explicitly models cross-variable interactions through a cross-variable attention layer, then routes each variable to an attention or non-attention path by a deterministic, loss-driven criterion, and prunes the inactive layers for structural compression. (2) The adaptability amplification SIREN (A2-SIREN) replaces rigid frequency scaling with channel-wise learnable modulation, adapting to heterogeneous physical features with negligible overhead. Combined with post-training 16-bit quantization, SWAP-INR achieves state-of-the-art performance on small modular reactor (SMR) datasets and remains competitive on public physics simulation datasets, while maintaining a constrained memory footprint. Our code is https://github.com/hvcl/SWAP-INR.

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04422

GVT3D: Gesture-Vision-Text Token Enrichment for 3D Embodied Reference Understanding

Vimarsh Shah, Dulanga Weerakoon, Ajinkya Deshpande, Saksham Agarwal, Nilesh Bhatia, Neel Kiran Naik, Parth Shah, Sravan Danda 等 12 位作者

原文摘要

Natural human-robot interaction requires robots to interpret language and gestures within a 3D environment to support tasks such as pick-and-place and object retrieval. 3D Embodied Reference Understanding (3D-ERU) supports this by identifying the 3D location of a target object from a scene (point-cloud) containing a human pointing gesture and an accompanying language description. However, current 3D-ERU models lack alignment of cross-modal representations and incur high processing costs that increase with the point-cloud's size. To address this, we propose GVT3D, which employs an Embodied Fusion module that jointly models language, gesture, and vision inputs within a shared embedding space, promoting cross-modal alignment. We additionally introduce a foveation approach that selectively processes the point cloud within a coarse region around the estimated pointing direction to reduce computations. GVT3D achieves ~8% higher accuracy and foveation enables 2.95x faster execution while maintaining comparable accuracy, relative to prior 3D-ERU models.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2610.39861

M-PFT: Angular Margin for Calibrated Prompt-Tuning of Fine-Tuned Vision-Language Models

Shihab Aaqil Ahamed, Mohamed Ansaff, Nang Khin Myat Lay Phyu, Udaya Sampath K. Perera Miriya Thanthrige, Chamira U. S. Edussooriya, Ranga Rodrigo, Muhammad Haris Khan

原文摘要

Prompt-tuning has emerged as a parameter-efficient technique for adapting large-scale vision-language models (VLMs) to downstream tasks while keeping the pre-trained weights frozen. However, prompt-tuned VLMs often suffer from a base-to-novel dual miscalibration problem, where models typically show underconfidence on base classes and overconfidence on novel classes. Current VLM calibration approaches address this dual miscalibration either separately or jointly, but lack a geometric understanding of the hyperspherical embedding space and thus fail to exploit angular discriminative properties to adjust distorted decision boundaries. To address this, we propose M-PFT, a novel angular-margin approach for prompt tuning that introduces two complementary objectives to promote class discriminability and semantic anchoring. This enlarges the angular margin by 1) minimizing the maximum cosine similarity between tunable class embeddings to reduce embedding collapse and 2) minimizing the angular distance between each tunable embedding and its corresponding zero-shot embedding to preserve semantic geometry. Extensive experiments across 9 prompt-tuning methods with various backbones on 15 diverse datasets show that our approach reduces the average calibration error on novel classes, while remaining competitive on base classes and maintaining comparable accuracy. Further analyses reveal that M-PFT effectively adjusts distorted decision boundaries induced by prompt tuning while preserving the pre-trained embedding space, improving calibration with negligible computational overhead.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04418

VidUEU-Agent: A Data-Curation Agent for Multimodal Understanding, Editing, and Unified Tasks

Tengju Ru, Weitong Lian, Zecong Tang, Lingyi Meng, Haoran Li, Zhejun Cui, Yichen Zhu, Hangshuo Cao 等 15 位作者

原文摘要

The advancement of multimodal learning is heavily constrained by the scarcity and high annotation cost of high-quality instruction data. While raw videos offer an abundant and dynamic source of visual knowledge, transforming them into precise instruction data remains highly challenging. To bridge this gap, we present VidUEU-Agent, an automated pipeline that synthesizes instruction data from raw videos for multimodal understanding, editing, and unified tasks. VidUEU-Agent follows a four-stage pipeline: it performs global perception to filter low-quality shots, mines keyframes and multimodal signals, constructs metadata and routes the metadata to synthesize instruction samples. More importantly, our agent framework can autonomously iterate the data distribution based on downstream training results, thereby optimizing data quality, balance, and training effectiveness. Scaling this pipeline, we construct VidUEU-100K, a large-scale dataset of 100K training samples spanning multiple domains and tasks, providing a robust foundation for developing comprehensive multimodal capabilities. Extensive experiments demonstrate that fine-tuning baseline models on VidUEU-100K yields significant and consistent performance gains across diverse representative benchmarks, validating the effectiveness of our proposed agent framework and dataset.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2603.29167

JDCNet Uses Confidence Gated Privileged Modality Distillation for Cost Preserving X Ray Inference

Bo Ma, Weiqi Yan, Jinsong Wu, Kun Liu

原文摘要

Training-only privileged-modality learning uses an expensive modality during training while keeping a single-modality deployment path. We study chest radiograph (X-ray) classification with same-patient computed tomography (CT) available only during training. JDCNet adds an auxiliary CT-teacher target, the teacher's argmax label or its softmax distribution through a Kullback--Leibler term (soft-KL), only on samples whose teacher confidence exceeds a threshold; the deployed model has the same ResNet-18 computation graph as the supervised baseline. On 510 same-patient pairs from the public BIMCV-COVID19 releases of the Valencian Region Medical ImageBank (113 COVID-19, 397 non-COVID), with patient-level five-fold cross-validation and three seeds, 15 of 16 adaptively explored JDCNet configurations improved mean balanced accuracy (BA) over the supervised baseline (average $+0.020$), but none remained significant after Holm adjustment. In a pre-registered experiment with 240 batch-matched supervised restarts on the same splits, batch size had no detectable effect on the baseline (BA $0.620$ versus $0.625$), and when both methods selected one of 16 runs per cell on the validation partition, JDCNet exceeded supervised training by $+0.037$ BA (95\% confidence interval $+0.016$ to $+0.058$; patient-level $+0.007$ to $+0.068$). Logit distillation and contrastive alignment did not help. Teacher diagnostics indicate that the gate acts mainly as CT-informed sample weighting. All models, including JDCNet, performed at chance on an external radiograph set. Confidence-gated CT supervision thus gives a small within-cohort advantage on one public cohort and a binary task; generalization requires independent paired replication.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04416

Efficient and Faithful High-Resolution Image Editing with Sparse Cross-Resolution Denoising

Maosen Zhao, Xianfang Zeng, Pengtao Chen, Peng Ye, Gang Yu, Jiayuan Fan, Tao Chen

原文摘要

Instruction-based image editors still primarily operate at 1K, whereas practical editing increasingly requires faithful 2K/4K outputs. Dense high-resolution adaptation preserves the editing trajectory but incurs expensive target-source attention; editing at 1K followed by external refinement is cheaper but can separate detail restoration from the editing trajectory. We introduce FLASH, a native high-resolution editor based on sparse cross-resolution denoising. A frozen 1K branch provides a semantic and structural anchor, while a target-resolution branch reconstructs local detail within the same Transformer. An asymmetric topology retains full 1K attention, guides high-resolution queries with spatially aligned low-resolution noisy-target states, and confines high-resolution target-reference interaction to local Halo neighborhoods. Resolution-specific diffusion-state mapping and spatial YaRN adapt the pretrained editor to larger grids. An optional, training-free progressive path replaces early high-resolution denoising with a 1K prefix and completes refinement in the same editor. On VINS-4KEval, the direct model achieves the strongest structural fidelity among the compared methods at both 2K and 4K. A separate 4K forward benchmark with R = 1 records 4.2x faster total forward execution and 25.3x faster attention than dense target-reference attention. Progressive inference reduces end-to-end 4K latency by 30.5%, with a modest quality cost.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.04414

SphereFlow: Missing Modality Imputation via Geometric Transport on Hypersphere

Huihan Wang, Juncheng Wang, Yuxiang Feng, Wenlong Hou, Guoqi Yu, Chao Xu, Yang Liu, Baigui Sun 等 9 位作者

原文摘要

Missing modality imputation aims to synthesize unobserved modalities from observed ones. Existing methods treat each modality as living on its own latent manifold, forcing the generator to learn a long, unstructured jump between them. We present SphereFlow, which takes a fundamentally different view: by projecting all modalities into a shared hyperspherical latent space via a frozen self-supervised encoder, cross-modal imputation reduces to short-range geometric transport between nearby points on the sphere. Realizing this idea with flow matching, however, exposes two geometric conflicts: the standard Gaussian noise source is catastrophically mismatched with the unit-norm data manifold, and the linear interpolation path departs the sphere surface into a region where the decoder has no training signal. We resolve both with simple, geometry-aware modifications: replacing the noise source with the observed-modality latent so that the velocity field learns only the cross-modal displacement, and training the decoder with a time-weighted loss that extends its domain to the near-surface region traversed by the flow. We prove that this data-as-source formulation reduces the worst-case off-manifold deviation from a quantity that grows with the latent dimension to a small, dimension-independent bound governed by inter-modality similarity. Experiments on BraTS multi-modal MRI imputation and CT-MRI translation demonstrate state-of-the-art generation fidelity across all settings, while downstream brain tumor segmentation with our imputed data closes roughly eighty percent of the gap to the full-modality oracle. Code is available at https://github.com/WTHH031230/SphereFlow.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2607.28907

Domain-Adaptive Deep Joint Source-Channel Coding for Image Classification

Yishen Li, Xuechen Chen, Xiaoheng Deng, Hao Zhang

原文摘要

Deep joint source--channel coding (Deep JSCC) can suffer from domain shift between training and deployment data. We study single-source domain adaptation for image classification over wireless channels. A Gaussian example shows that improving channel quality does not necessarily improve target-domain accuracy under a fixed source-trained classifier. We further proposed an information objective to reduce domain information while preserving class information. Motivated by the analysis, we develop a domain-adaptive Deep JSCC method that combines class-wise adversarial alignment with pseudo-label supervised contrastive learning. The former uses all target samples to reduce class-conditional domain differences, while the latter selects high-confidence target samples to obtain class-discriminative received representations. We relate these losses to the information objective, and analyze the effects of pseudo-label errors and confidence-based selection. Experiments on SVHN--MNIST and PACS over AWGN and Rayleigh channels demonstrate improved target-domain classification without additional inference-time networks. The proposed method achieves 98.15\% accuracy on SVHN $\rightarrow$ MNIST at 10~dB CSNR.

cs.IT · cs.CV · eess.SP

查看论文
arXiv ↗PDF ↗