跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1072新增论文
+845其中 AI 相关
+94已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-09 22:47 · 全站最近入库：2026-11-09 13:22各信源进度 →

统计口径与数据新鲜度

统计 2026-11-08 23:47 — 2026-11-09 23:47（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-09 22:47最近发现新增版本：2026-11-09 13:22库内论文最新更新时间：2026-11-07 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

18,172 已收录论文15,506 AI 相关4,527 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 6,491 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-09 22:43（北京时间）
新论文 · v1arXiv:2610.36371

BoxWorld: Let World Engines Control and Video Generators Render

Zike Wu, Ahmed Hatem, Chintan Trivedi, Wallace Michel Pinto Lira, Xuebin Qin, Lele Wang, Renjie Liao

原文摘要

We tackle the challenge of building real-time interactive video worlds with user-defined behaviors. Such worlds must track object states, apply interaction rules, and render the resulting consequences as users act. Yet existing end-to-end video world models typically learn interaction dynamics within the generator, making it difficult to specify object-level rules and reason about their consequences. In this work, we introduce BoxWorld, which separates interaction reasoning from generative rendering. Given user-specified rules, a coding agent creates a Box World Engine that determines interaction outcomes and exposes the resulting object states through a lightweight interface of layout boxes, persistent object attributes, and dynamic status variables, leaving appearance and physical effects to generative rendering. A Box-State Generative Renderer then renders this evolving state stream into video autoregressively, enabling online control over object motion, state switches, and conditional events. Rendering this state stream is non-trivial: the renderer must respond to object-level state updates in real time while remaining coherent with the generated history. To this end, we introduce a three-stage training strategy that learns object grounding from open-domain videos and state transitions from gameplay videos, both annotated automatically, and finally performs rollout distillation on self-generated histories. We further propose PairedStateBench, a paired benchmark for fine-grained interaction control. BoxWorld substantially outperforms end-to-end world models and remains competitive across multiple public benchmarks. Project page: https://boxworld-web.github.io.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36364

When Slides Met Scans: Cross-Attention for Renal Cancer Prognosis

Anusha Krishnamoorthy, Basudha Pal

原文摘要

Histopathology and computed tomography (CT) provide complementary views of renal cancer. We connect these modalities through bidirectional cross-attention between 16 histology prototypes and up to 32 kidney-centered CT tokens from frozen foundation models. In 210 TCGA-KIRC patients with 46 disease-specific survival (DSS) events, 5-fold evaluation yielded a C-index of $0.760\pm0.081$ and 3-year AUROC of 0.773 (95\% CI 0.678--0.860), the highest averages among evaluated unimodal and fusion baselines. External testing in 180 CPTAC-CCRCC patients yielded DSS C-index of 0.629 with CT withheld. Attribution analysis linked predictions to image regions while exposing acquisition artifacts. These findings support further study of cross-scale fusion and independent multimodal validation.

eess.IV · cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36337

CRLoMTL: Conflict-Resolution through Low-rank matrices for Multi-Task Learning

Ahmed Agiza, Tasneem Shaffee, Rahil Badkul, Sherief Reda

原文摘要

Multi-task learning (MTL) improves model efficiency by sharing representations across related vision tasks, but it often suffers from negative transfer when task updates interfere with one another. Existing conflict-mitigation methods typically operate in gradient space, requiring explicit task-gradient manipulation and additional optimization steps. We propose Conflict-Resolution through Low-Rank Matrices for Multi-Task Learning (CRLoMTL), a parameter-efficient framework that mitigates task interference in the low-rank parameter-update space. CRLoMTL learns task-specific LoRA updates through task-specific passes, where each task updates its own low-rank matrices, and integrates these updates through global shared-subspace fusion. Specifically, we introduce a shared-subspace resolver based on singular value decomposition (SVD), referred to as the SVD resolver, which extracts a common low-dimensional orthonormal basis across task adapters and projects task updates into this shared space before merging them into a single deployment-time update. The resulting update can be folded into the backbone, preserving the inference structure of the original model without additional runtime branches. Experiments with Swin Transformer backbones on PASCAL-Context, NYUD-v2, and CelebA show that CRLoMTL achieves a favorable accuracy-parameter trade-off, improving over single-task baselines by up to 2.77% on PASCAL-Context, 5.03% on NYUD-v2, and 3.50% on CelebA, while training only a small fraction of the parameters used by single-task models.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2604.10385

GTASA: Ground Truth Annotations for Spatiotemporal Analysis, Evaluation and Training of Video Models

Nicolae Cudlenco, Mihai Masala, Marius Leordeanu

原文摘要

Game engines hold what video models struggle to learn: a complete, explicit world state behind every frame. We turn one into a data instrument. GEST-Engine, our production-grade open-source system, deterministically executes Graphs of Events in Space and Time (GESTs), whether procedurally generated or derived from text, into videos of synchronized multi-actor scenarios, recording ground truth as it renders: 3D entity and camera state, pairwise spatial relations, event-to-frame mappings, instance segmentation, and long descriptions, at zero marginal annotation cost. With it we release GTASA, a 938-video sample of what the system can generate at arbitrary scale, carrying, to our knowledge, the densest spatial-relation coverage of any video dataset: a complete entity-pair relation graph at every frame, ~84x denser than the state of the art, frame-for-frame. We validate GTASA both qualitatively, through human evaluation of physical validity and semantic alignment where frontier neural generators, given the same prompts, largely fail, and quantitatively, with GTASA pretraining improving VideoLLaMA3 video captioning. Probing six frozen video encoders across 11 spatio-temporal tasks enabled by GTASA's exact 3D ground truth, a previously untestable inter-entity relational probe of frozen video features, reveals that who-is-near-whom barely rises above chance for all of them. We release the engine, the corpus, and the benchmark, making this gap a measurable, trainable target.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36327

Training-Free Inference Acceleration for Video Large Language Models: Methods, Trade-offs, and Evaluation

Yuqi Li, Chuanyue Yan, Neil Rodrigues, Xianglong Wang, Xiaoqin Feng, Junming Liu, Xi Xiao, Zhongwei Cheng 等 11 位作者

原文摘要

Video Large Language Models (VLMs) have demonstrated strong capabilities in video understanding and reasoning, but processing long visual sequences incurs substantial computational, latency, and memory overhead. This survey examines training-free inference acceleration, focusing on reducing inference costs while preserving task-relevant spatiotemporal evidence. We organize existing methods by their intervention stages in the inference pipeline, covering input, visual encoding, projector, prefill, and decoding. Post-training quantization (PTQ) is discussed separately as a cross-stage extension with varying calibration requirements. We analyze how intervention position affects evidence preservation and computational savings, and why local efficiency improvements may not translate into end-to-end acceleration. We complement this analysis with quantitative evidence from prior studies and our own experiments under a shared evaluation setting to assess accuracy-latency trade-offs.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2608.27860

From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation

Rit Gangopadhyay, Alex Wong

原文摘要

Vision foundation models are capable of generalizing across 3-dimensional (3D) scenes with high-fidelity estimates; their empirical success can be attributed to training on large-scale datasets of perspective images. However, when transferred to wide field-of-view (FoV) images, such as those captured by fisheye cameras, they return erroneous outputs due to a covariate shift stemming from the radial distortion on the image pixels. We propose a method to generalize vision foundation models to fisheye cameras. The crux of our method lies in a set of learnable parameters, termed Distortion Extenders (DEX), that model the fisheye distortion coefficients and the distributional shift between fisheye and perspective images encoded in the latent space. By minimizing a self-supervised alignment loss, DEX transforms the latent embeddings of fisheye images to resemble those of perspective images to recover high-fidelity estimates. DEX is architecture- and task-agnostic: We demonstrate DEX on monocular depth estimation and open-vocabulary segmentation for convolution- and Transformer-based architectures, where we consistently improve over baselines across indoor and outdoor fisheye datasets. As a byproduct, the activations of DEX can also be decoded to distortion coefficients to support camera calibration. Code available at: https://github.com/Suchisrit/DEX.

cs.CV · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37709

Towards Myocardial Infarction Diagnosis: An Investigation into Machine Learning and Deep Learning Algorithms using ECG Data

Rafid Ahnaf, Husne Mubarak, Fahim Ahsan, G. M. Refatul Islam, Rafeed Rahman, Nabuat Zaman Nahim

原文摘要

Myocardial Infarction (MI) is a severe and potentially fatal cardiac condition caused by myocardial injury due to ischemia, and timely detection can substantially reduce its mortality. Electrocardiography (ECG) is the primary diagnostic modality for MI, but manual interpretation is susceptible to human error. In this work, we propose Visionary V2, a multi-step, adaptive filtration method for ECG signal normalization, and use it to study machine learning (ML) and deep learning (DL) approaches to MI detection on the PTB Diagnostic ECG Database using all 15 available leads. On statistical features extracted from the filtered signals, such as the R-R interval, a Bagging ensemble of decision trees reaches 96.6% accuracy, compared with 87.7% when the signals are processed with BioSPPy. We further generate an image dataset from the filtered signals, on which a custom ResNet-50 misclassifies only 1 of 7,000 validation images and a custom CNN reaches an F1 score of about 98.7%. Finally, we apply LIME, Grad-CAM and Integrated Gradients to verify that the image models attend to the ECG waveform rather than the background. These results indicate that careful signal normalization combined with ML and DL models can support fast and accurate automated MI screening.

eess.IV · cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.38136

CLeaR: A Unified Framework for Resolving the Leakage-Degradation Dilemma in Style Transfer

Teng Zhou, Yunhao Chen

原文摘要

Style transfer aims to render target content in the style of a reference image, but existing methods often suffer from content leakage, where objects, layouts, or semantics from the style reference appear in the generated output. Although prior data-driven and training-free methods can reduce leakage, they often face a leakage-degradation dilemma: stronger content suppression may weaken style fidelity, while richer style preservation may reintroduce unwanted reference content. We identify this dilemma across the full style-transfer pipeline, including feature separation, feature-space grounding, and diffusion generation. To address these issues, we propose CLeaR, a training-free framework for content-leakage-resistant style transfer. CLeaR first uses Orthogonal Subspace Projection to define content-reduced style targets in each vision foundation model (VFM) feature space. It then performs Ensemble Inversion, which optimizes a shared pixel-space style anchor satisfying style constraints across multiple VFMs. Finally, Energy-Guided Calibration maintains style alignment during diffusion sampling by steering the denoising trajectory toward the ensemble-defined style manifold. We further provide a theoretical analysis showing that the style-anchor estimation error decreases with the number of VFMs. Experiments on StyleBench demonstrate that CLeaR improves style alignment, reduces content leakage, and achieves better LLM-as-Judge evaluation compared with existing methods. The code is available at \href{https://github.com/0606zt/CLeaR}{https://github.com/0606zt/CLeaR}.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36269HF 社区推荐

Cross-modal Steering Vectors Exist Across Both Image and Text Generation

Donghao Zeng, Nishant Subramani, Tamar Rott Shaham, Benno Krojer

原文摘要

Are concepts and mechanisms unified across modalities in current AI models, or instead implemented separately for each one? We operationalize this broad question of modality unification by studying cross-modal steering: 1) finding vectors that successfully steer generation on tokens of one modality, which then 2) transfer zero-shot to steer generation when added to tokens of another modality. Our study goes beyond simple correlational alignment (e.g. representational similarity) because steering is fundamentally causal. We study cross-modal steering in several unified multimodal models (UMMs), since they possess shared parameters for language and image generation, focusing on these two modalities. Across diverse steered concepts such as emotion and object size, we show that steering vectors transfer zero-shot in both directions (text-to-image & image-to-text). Even for concepts that are highly unique to one modality (e.g. formal vs. colloquial language), steering vectors often induce a consistent, interpretable effect in the other modality, such as formal clothing and design. This transfer has a geometric signature too: despite a clear modality gap separating text and image representations, steering vectors with strong cross-modal transferability tend to point in similar directions for the same concept. Our findings shed light on how vision and language interact inside unified multimodal models and point towards more general, modality-agnostic approaches for controlling their behavior.

cs.CV · cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.30405

VisCurate: Evaluating Behavioral Equivalence of Visual Skills

Md Farhan Ishraq, Md Farhan Ishmam, Reaz Hassan Joarder, Md Hasanul Kabir

原文摘要

Modern vision-based agents have shifted toward skill-based problem-solving, where a skill is an executable image-to-image operation, e.g., cropping, thresholding, or edge detection. As these skill libraries grow, they accumulate duplicated, mislabeled, and defective skills. The signals normally used to curate them (a skill's name, description, or source code) say little about how the skill visually changes the image. We present VisCurate, a framework and benchmark that curates visual skill libraries from observed execution rather than from text. VisCurate runs each candidate skill pair on a shared probe battery and assigns one of six behavioral relations, each tied to a library edit that is safe to make. The benchmark injects seven defect types into a clean library at controlled rates, producing 300 reproducible instances with ground-truth relations and repairs. While text baselines accept up to five of six false merges on hard negatives, our verifier accepts none and recovers 51.2% of the pairs that are truly mergeable. On two real, uncurated libraries, VisCurate finds that 92% of the skills a package shares with its downstream copy are redundant, while execution confirms only 10 of 17 published equivalence claims. The curation problem also remains open, as the strongest of the nine automated curators recovers only about half of the repairs. We emphasize that visual skill libraries should be curated by behavior, not by description.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36249

Hierarchical Multimodal Interaction Reasoning for Dyadic Relationship Recognition

Wang Tang, Fethiye Irmak Dogan, Linbo Qing, Ce Zhu, Hatice Gunes

原文摘要

Dyadic relationship recognition aims to identify the latent social relationship between two individuals from their multimodal interactions. Existing methods typically project heterogeneous cues into a shared representation and optimise a flat classification objective, overlooking that relationships may rely on different interaction patterns. For instance, distant relationships can often be inferred from direct behavioural correspondence, whereas close relationships are better reflected by coordinated affective, behavioural, and multimodal responses. To address this limitation, we propose HiRe-DRR, a Hierarchical Multimodal Interaction Reasoning framework that organises relationship evidence from three complementary perspectives: feature level, interaction type, and relationship hierarchy. HiRe-DRR decomposes facial, bodily, acoustic, and textual cues into low- and high-level representations and constructs homogeneous and heterogeneous cross-participant interactions. We further introduce the Mixture-of-Interaction Transformer (MoI-Former), which treats diverse interactions as specialised experts and employs relation-aware sparse routing to identify relationship-relevant evidence while preserving contextual interaction information. The proposed Hierarchical Conditional Loss (HC-Loss) exploits the hierarchical nature of social relationships through coarse-to-fine supervision, enabling structured relationship reasoning. Extensive experiments on three dyadic relationship benchmarks demonstrate that HiRe-DRR outperforms recent representative image-, video-, and multimodal methods. Our routing analysis further reveals that distant relationships tend to rely on low-level homogeneous interactions, whereas closer relationships focus on high-level heterogeneous interactions. Our code is publicly available at: https://github.com/tw-repository/HiRe-DRR.

cs.CV · cs.MM

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2607.03740

Multimodal Knowledge Aggregation for Microbial Keratitis: Geographic Confounding and Shortcut Risk in Pooled Multicountry Data

Yiqing Wang, Maria A. Woodward, Ziyun Yang, N. Venkatesh Prajna, Chunming He, Leslie M. Niziol, Mercy Pawar, Ming-Chen Lu 等 14 位作者

原文摘要

To develop an open-source multimodal model for bacterial-versus-fungal microbial keratitis classification and assess shortcuts from geographic pooling. Participants: 1,645 US and Indian patients contributed 17,158 slit-lamp images under blue-light, sclerotic-scatter, and white-light illumination. Methods: The model combined cross-modality contrastive learning, modality-specific fine-tuning, and fusion with 7 clinical variables. Diagnoses used smear, culture, and polymerase chain reaction results, with expert review of culture-negative cases. The model and 5 comparators used 5 patient-disjoint outer folds stratified by country and organism, with inner validation. Secondary analyses examined within-country performance, a country-only reference, ablations, and imbalance mitigation. Code: https://github.com/yqwang01/TPMKA. Main Outcome Measures: Patient-level macro-F1, class-specific F1, accuracy, and area under the receiver operating characteristic curve (AUROC), overall and by country. Results: The model had the highest overall macro-F1 (83.06%; 95% CI, 81.58-84.56) and accuracy (84.50%; 95% CI, 83.16-85.84) among matched comparators, with 89.9% AUROC (95% CI, 88.7-91.1). Paired differences were not consistently significant. AUROC was 83.9% in India and 67.0% in the US; bacterial F1 in India was 47.78%, and fungal F1 in the US was 40.97%. A country-only reference achieved 81.88% accuracy and 78.2% AUROC overall despite 50.0% AUROC within each country. Resampling and group distributionally robust optimization improved some underrepresented groups without removing the geographic pattern. Conclusions: The model compared favorably with matched methods, but geographic prevalence differences made pooled performance more reassuring than within-country discrimination. Independent geographic validation is needed.

q-bio.QM · cs.CV · eess.IV

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2510.07441

A Closer Look at Evaluation of Text-to-Video Models Under Camera Motion

Nithin C. Babu, Aniruddha Mahapatra, Harsh Rangwani, Rajiv Soundararajan, Kuldeep Kulkarni

原文摘要

Existing text-to-video (T2V) evaluation benchmarks, such as VBench and EvalCrafter, predominantly contain either subject-centric prompts or static camera scenes, overlooking the importance of camera motion, essential for producing cinematic shots with realistic scenes. Due to this, the metrics developed in these works often produce incorrect results in the presence of camera motion. To fill this gap, we introduce DynamicEval, a benchmark comprising of systematically curated prompts that emphasize dynamic camera motion, paired with 45k human annotations on video pairs. DynamicEval evaluates two key dimensions of video quality: the consistency of the foreground subjects and the consistency of the background scene. To address the shortcomings of existing metrics under camera motion, we introduce two novel metrics: 1) Tracker-FG, which evaluates subject consistency by assessing the keypoint tracks of the subject. 2) Debiased Motion Smoothness, for background consistency, devoid from issues of occlusions/disocclusions arising from camera and foreground subject movements. Extensive experiments on the large-scale DynamicEval benchmark demonstrate that our proposed metrics achieve stronger correlations with human preferences at both the video level, and the model level with an increase of 0.32 in average Pearson Linear Correlation.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2604.23018

AmaraSpatial-10K: A Spatially and Semantically Aligned 3D Dataset for Spatial Computing and Embodied AI

Mohammad Sadegh Salehi, Alexander J. P. Perkins, Igor Pardo Maurell, Ashkan Dabbagh, Raymond Wong

原文摘要

Web-scale 3D asset collections are abundant but rarely ready to ingest without cleanup, suffering from arbitrary metric scaling, incorrect pivots, brittle geometry, and incomplete textures, defects that limit their use in embodied AI, robotics, and spatial computing. We present AmaraSpatial-10K, a dataset of 10,071 synthetic 3D assets standardised for direct ingestion. Each asset ships as a metric-scaled, deterministically anchored .glb with separated PBR maps, a convex collision hull, a paired reference image, and multi-sentence text metadata. Alongside the dataset we introduce a reusable evaluation suite for 3D asset banks, a continuous Scale Plausibility Score (SPS), a keyword-based Concept Density metric, anchor-error auditing, and a cross-modal CLIP coherence protocol, and apply it to AmaraSpatial-10K alongside matched subsets of Objaverse, HSSD, ABO, and GSO. We check the annotations independently of the LLM/VLM pipeline that produced them: measured ABO object heights fall inside our scale intervals 88.7-90.7% of the time, and three non-author raters auditing a stratified sample of 314 assets judge 95.5% plausibly scaled and 99.7% correctly anchored. AmaraSpatial-10K improves CLIP Recall@5 by 3.4x over Objaverse (0.612 vs. 0.181, median rank 267 -> 3), achieves a 99.1% settling-stability rate under Habitat-Sim in static placement and collision tests, with ~20x lower per-load wall time, and as a drop-in Holodeck asset bank improves planned-scale matching and is preferred by blinded human raters in 27 of 40 paired scenes. Controlled ablations on the same asset bank attribute the retrieval gain to description richness.

cs.CV · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36210

Diagnosing and Correcting Bias in MLLM for Long Video Understanding

Xusheng Liang, Jianqiao Sun, Hao Zhang, Yulei Niu, Hengshuang Zhao, Jiawei Ma

原文摘要

Long-video question answering is challenging since the answer often hinges on a few decisive moments scattered throughout the video, while memory constraints force Multimodal Large Language Models (MLLMs) to sample frames at extremely low rates, resulting in sparsity that risks omitting key evidence. Though aggressively enlarging the context size is intuitive, we argue that performance remains fundamentally constrained by the bias, \emph{i.e.}, the misleading cues caused by spurious \textit{linguistic} correlations and salient yet irrelevant \textit{visual} observations. In this paper, we first introduce {S-MME}, a benchmark designed to systematically diagnose shortcut-biased behavior in MLLMs. To mitigate this failure, we further propose a training-free framework, Counterfactual Long-video Evidence-Aware Reasoning (CLEAR). By treating the full video-question pair as a complete view, CLEAR estimates bias effects via augmented views by only keeping the question or locally salient visual clips. Then, we intervene hidden states to mitigate the discrepancy at inference such that the correct answer can be predicted confidently. With experimental studies across five benchmarks, we show that our method is generalizable and can be integrated in different MLLMs, yielding consistent gains in prediction and robustness. We hope CLEAR can offer a complementary direction in test-time scaling for long-video understanding.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.24021

Token-to-Token Alignment of Text Embeddings for Semantic Blending

Saar Huberman, Ron Mokady, Or Patashnik, Daniel Cohen-Or

原文摘要

In modern generative models, images are specified and controlled through text prompts. In practice, images are generated from sequences of tokens derived from these prompts. However, the space of token sequences lacks a consistent accessible structure: semantically similar images may correspond to sequences that differ in wording, ordering, and placement of concepts, while similar token sequences may encode very different semantics. This apparent lack of structure makes it difficult to perform smooth transitions in this space, hindering applications such as image blending and continuous control of edits. We argue that this limitation stems not from the absence of semantic structure, but from misalignment between representations. To address this misalignment, we introduce Token-to-Token alignment, a framework that establishes explicit semantic correspondence between tokens across prompts. Our approach transforms prompts into a structured representation in which semantically corresponding concepts are mapped to consistent positions across prompts, and then aligns their token embeddings based on semantic similarity. Concretely, the method consists of two stages: a structural alignment that rephrases prompts into a shared structured form, followed by an embedding-level alignment that matches token representations across prompts. With this alignment in place, simple linear interpolation becomes a meaningful operation, producing smooth and coherent semantic transitions and enabling applications such as blending and continuous editing. Our results show that text embedding spaces in text-to-image models implicitly encode a continuous semantic structure that becomes accessible once representations are properly aligned, suggesting that semantic control can be achieved by organizing existing representations rather than modifying the generative model.

cs.CV · cs.GR

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36203

UT-HMR: Uncertainty-Guided Temporal Modeling for Robust 3D Human Mesh Recovery

Zhibo Wang, Sirisha Rambhatla, Yuhao Chen, Maria Koshkina, David Clausi

原文摘要

Monocular 3D human mesh recovery (HMR) remains unreliable under weak observations caused by long camera-to-person distances, occlusion, and motion blur. In challenging scenarios such as sports, these factors make the problem highly ambiguous, as different 3D poses may yield similar 2D projections. This ambiguity is especially severe for one-stage HMR, which recovers all meshes directly from the full image without person crops or external 2D cues. To handle this ambiguity, we present UT-HMR ( Uncertainty-guided Temporal HMR), to our knowledge the first one-stage HMR framework to capture correlated pose uncertainty across joints. Each person query predicts a Gaussian distribution over the full SMPL pose with a compact low-rank-plus-diagonal covariance, which is converted into joint-wise precision weights for heteroscedastic supervision while keeping inference deterministic. Temporal early fusion further injects neighboring-frame evidence before pose decoding, supplying the visual evidence needed to resolve the ambiguity that the uncertainty captures. With a lightweight ViT-B backbone, UT-HMR achieves state-of-the-art results on VIP-HockeyPose-v2 (ice hockey) and WorldPose (soccer), reducing MPJPE, PA-MPJPE, and PVE by up to 4.7%, 3.8%, and 5.6%, and remains competitive on 3DPW against methods with larger backbones and higher input resolutions.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36192HF 社区推荐

Beyond the Estimate: Wiener-Informed Decoding for Easier Training of Pixel-Space Flow Models

Ziqiao Wang, Yang You

原文摘要

Learning velocity fields directly in pixel space is difficult and slow. We show that coarse second-order image statistics can ease this task: frozen Wiener estimates improve velocity learning, with gains extending beyond the linear predictions they explicitly provide. Moreover, we observe similar but complementary gains from frozen bases and pixel decoding. These observations motivate Wiener-informed Decoding (WiD), which uses statistical knowledge both for direct prediction and to organize decoder computation without modifying native blocks. WiD combines a trainable Wiener prediction path with coefficient interfaces that inject features into the native decoder and return its predicted coefficient corrections, with these components initialized with data statistics. All introduced parameters, including the spectrum and coordinate transforms, are trained jointly under the original pixel-space velocity objective. Across model scales and decoder architectures, WiD accelerates training and improves generation quality with modest parameter overhead. It also yields smoother gradient trajectories and remains effective with representation alignment. WiD improves class-conditional and text-to-image generation across different resolutions. On ImageNet 256 x 256, PixelDiT-XL with WiD reaches FID 1.53 after 500 epochs, achieving comparable quality to PixelDiT with 37.5% fewer training epochs. Statistical denoisers contribute beyond the estimate: their computational structure can also ease pixel-space training.

cs.CV · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36190

Bayesian Evidence Accumulation for Graph-Based Learning in Dynamic Data-Driven Applications Systems

Moises Ponce, Sarah Harkins Dayton, Ioannis Schizas, Vasileios Maroulas

原文摘要

Graph Neural Networks (GNNs) are widely used in modern machine learning, yet most existing frameworks remain fundamentally static, limiting their applicability in Dynamic Data Driven Application Systems (DDDAS) where observations arrive sequentially under uncertainty. We introduce B-EAGLe (Bayesian Evidence Accumulation for Graph-Based Learning), a probabilistic framework inspired by CBGT-Net (cortico-basal ganglia-thalamic network) that reformulates graph classification as a sequential evidence accumulation process. B-EAGLe combines a GNN encoder with a Bayesian accumulator that incrementally updates posterior beliefs over class labels as new observations arrive, committing to a decision only when sufficient confidence is achieved. Experiments on CIFAR10 demonstrate competitive performance together with principled confidence estimation under sequential observations.

cs.LG · cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.15796

DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracing

Artyom Mazur, Nina Konovalova, Aibek Alanov

原文摘要

While transcoder-based circuit tracing has recently enabled detailed causal analyses of large language models, it remains unclear how semantic information propagates across denoising steps and how text and image representations interact within multimodal diffusion transformers (MM-DiT) for image generation. Attention maps expose only a limited view of token interactions, and sparse autoencoders find interpretable features but do not reveal how nonlinear MLP layers transform and compose them. In this work, we extend transcoder-based circuit tracing to multimodal diffusion transformers. We train timestep-conditioned transcoders that faithfully approximate the input-output behavior of the MLP sublayers in the double-stream blocks of FLUX.1[schnell]. By replacing MLPs with transcoders and linearizing the remaining computation, we obtain exact feature-to-feature attribution and build compact attribution graphs over interpretable features. Empirically, our transcoders match or slightly outperform sparse autoencoders on the sparsity-faithfulness tradeoff. The graphs recover the known shift from prompt-driven to image-driven computation across denoising steps, point to candidate mechanisms of attribute binding and cross-stream semantic propagation, and yield intervention-testable explanations for systematic generation errors. On prompt grids labeled blind, graph-guided interventions reach effects that single-feature steering does not; at a matched budget, features chosen from the graph fix a color prior more often than the target feature alone and destroy the object less often than features chosen by co-activation. FLUX.1[dev] reproduces the temporal and color-prior results, and SD3.5-medium reproduces them in direction. Our results demonstrate that transcoder-based circuit analysis is feasible for state-of-the-art multimodal diffusion transformers.

cs.CV · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.18541

LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift

Haozhe Si, Yuxuan Wan, Yuqing Wang, Minh N. Do, Han Zhao

原文摘要

Modeling hyperspectral imagery (HSI) across different sensors presents a fundamental challenge due to variations in wavelength coverage, band sampling, and channel dimensionality. As a result, models trained under a fixed spectral configuration often fail to generalize to other sensors. Existing Vision Transformer (ViT) approaches either rely on implicit spectral modeling with fixed channel assumptions or adopt explicit spatial-spectral attention with prohibitive computational cost, leading to a fundamental trade-off between efficiency and expressiveness. In this work, we introduce Low-rank Efficient Spatial-Spectral ViT (LESSViT), a sensor-flexible architecture for cross-spectral generalization. LESSViT is built on LESS Attention, a structured low-rank factorization that models joint spatial-spectral interactions through separable spatial and spectral components, reducing the complexity of full spatial-spectral attention from $\mathcal{O}(N^2 C^2)$ to $\mathcal{O}(rNC)$, where $N$ is the number of spatial tokens, $C$ is the number of spectral channels, and $r$ is the rank of the low-rank approximation. We further incorporate channel-agnostic patch embedding and wavelength-aware positional encoding to support flexible spectral inputs. To enable efficient and robust pretraining, we introduce a hyperspectral masked autoencoder (HyperMAE) with decoupled spatial-spectral masking and hierarchical channel sampling. We evaluate LESSViT under a cross-spectral generalization setting that simulates cross-sensor variability. Experiments on the SpectralEarth benchmark demonstrate that LESSViT improves robustness under spectral shifts while remaining competitive in-distribution, and explicit and efficient spatial-spectral modeling is essential for scalable and generalizable hyperspectral representation learning. The code is available through the project page: https://uiuctml.github.io/LESSViT/

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.02366

PRIMA: Boosting Animal Mesh Recovery with Taxonomic Priors and Adaptation-Driven Data Bootstrapping

Xiaohang Yu, Ti Wang, Mackenzie Weygandt Mathis

原文摘要

Recovering articulated 3D animal meshes from monocular images is challenging due to the long-tailed distribution of species and limited large-scale 3D supervision. Existing methods rely on small annotated datasets and thus regress to average shapes and poses, leading to poor generalization to rare species and articulations. We present PRIMA (PRIors for Mesh Adaptation), a framework that tackles these limitations with two components: taxonomic shape priors and adaptation-driven data bootstrapping. First, we derive taxonomic shape priors from BioCLIP image features, which are learned under taxonomic supervision, to inject species-level knowledge into mesh reconstruction, improving shape estimation across diverse quadrupeds. Second, we propose an adaptation-driven bootstrapping scheme that turns abundant 2D animal annotations into reliable pseudo-3D supervision via lightweight instance-level mesh refinement and quality-aware filtering. This yields Quadruped3D, a large-scale pseudo-3D dataset constructed from real-world animal images, containing approximately 9K pseudo-3D annotations. Together, these components alleviate the long-tail problem: extensive experiments on Animal3D, CtrlAni3D, Quadruped2D, and Animal Kingdom demonstrate state-of-the-art performance for mesh reconstruction, with particularly strong gains on challenging poses and underrepresented species. The code is available at https://github.com/AdaptiveMotorControlLab/PRIMA.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2603.21819

Ctrl-A: Control-Driven Machine Learning using Adaptive Online Data Augmentation

Jesper B. Christensen, Ciaran Bench, Spencer A. Thomas, Hüsnü Aslan, David Balslev-Harder, Nadia A. S. Smith, Alessandra Manzin

原文摘要

We introduce ControlAugment (Ctrl-A), an automated data augmentation algorithm for image-vision tasks, which incorporates principles from control theory for online adjustment of augmentation strength distributions during model training. Ctrl-A eliminates the need for initialization of individual augmentation strengths. Instead, augmentation strength distributions are dynamically, and individually, adapted during training based on a control-loop architecture and what we define as relative operation response curves. Using an operation-dependent update procedure provides Ctrl-A with the potential to reduce the use of augmentation operations that negatively impact model performance, alleviating the need to manually engineer augmentation policies for new image classification tasks. Experiments on the CIFAR-10, CIFAR-100, SVHN-core, and Tiny ImageNet benchmark datasets using WideResNet-28-10 and ResNet architectures demonstrate that Ctrl-A provides competitive performance across a range of image classification benchmarks.

cs.CV · cs.AI · cs.LG · eess.SY

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36150

Any3R: Feedforward 3D Reconstruction from Any Camera Type

Jingong Chen, Qingwen Zhang, Youming Deng, Chulwoo Pack, Olaf Wysocki, Boris Jutzi, Kwanghee Won, Kyle Gao

原文摘要

Geometry foundation models recover camera poses and dense geometry from a set of images with a feed-forward network, but they expect pinhole images. That assumption is inside the backbone too: the positional encoding and the training loss both bind an image position to a direction in space, and both depend on the lens. We condition both on the dense ray field, the unit direction through every pixel, which describes any central camera without a lens type, distortion coefficients or a bound on field of view. Angular coordinates give the rotary encoding the measured ray angle of each patch, evaluated per view, and solid-angle weighting scales the point loss by the area each pixel covers on the sphere, so neither the encoding nor the supervision depends on the lens. Any3R is therefore a feed-forward model that takes any central camera, and is robust to mixing many camera types per scene, with no retraining per lens. On a benchmark of thirteen central cameras across three datasets, where every method reconstructs with its own predicted camera poses, it is the most consistently accurate method. Any3R enables scalable 3D reconstruction from heterogeneous crowdsourced and archival imagery collections sourced across diverse camera types. Project page: https://jingong-chen.github.io/Any3R/

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.26970

Computer Vision for MOBA Analytics: A Dataset and Baseline for Visibility Analysis in Dota 2

Ricardo da Rocha Carvalho, Eloísa Oliveira, Luiz Bernardo Martins Kummer, Emerson Cabrera Paraiso, Rayson Laroca

原文摘要

Introduction: Most Multiplayer Online Battle Arena (MOBA) analytics studies rely on structured data, which does not directly capture what each team could actually see during a match. Objective: This work introduces Dota2-Vis, a video-based dataset, and a baseline pipeline for visibility analysis in professional Dota 2 matches. Methodology: The dataset comprises all 144 matches from The International 2025, recorded from both team perspectives, totaling 288 Full HD videos, together with 2,477 manually annotated minimap images. We evaluate multiple variants of a modern object detector for player-icon detection and use the best-performing model to estimate opponent-visible player presence over time. Results: YOLO11l (large) achieved the best overall performance, reliably identifying player icons even in dense and visually cluttered minimap scenes. The resulting visibility curves reveal player, hero, role, and team-level patterns that complement conventional MOBA analytics, highlighting behavioral differences that are difficult to obtain from structured data alone. The Dota2-Vis dataset is publicly available at https://github.com/RicardoRCarvalho/dota2-vis/.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36079

Closed-Loop Aware Planning via Imitation-Free Speed Scoring for End-to-End Autonomous Driving

Emad Aghajanzadeh, Thomas Schwarzfischer, Marc Kaufeld, Moritz Thoma, Ryan Pégoud, Manoj Rohit Vemparala, Pierpaolo Mori, Nael Fasfous 等 11 位作者

原文摘要

End-to-end autonomous driving has predominantly relied on open-loop imitation learning for training, which presents two limitations. First, expert demonstrations show only one behavior per scenario, so the model cannot learn alternative valid behaviors and, in turn, cannot adapt to user-specified preferences at test time. Second, noise or ambiguity in these demonstrations can produce inconsistent, unreliable supervision for the end-to-end models. Driving is inherently multi-modal: multiple different behaviors can be valid responses to the same scene, and this diversity arises largely from speed rather than path. We introduce a planning paradigm that learns the path through imitation while scoring a set of candidate speeds across multiple criteria with a learned function supervised by physically grounded safety constraints. We realize this paradigm as Closed-Loop Aware Planning for Autonomous Driving (CLAP-AD), a plug-in planning head that integrates with existing end-to-end driving models and, to our knowledge, is the first to enable explicit test-time control over their speed selection without speed-conditioned training. On Bench2Drive-Speed, CLAP-AD sets a new state of the art Speed-Adherence Score of 82.53, topping TFv6, the strongest baseline on this metric, by 12.9 points while retaining strong closed-loop driving performance. On Bench2Drive, it improves driving score by +28.5 and +32.7 over UniAD and VAD, and on NAVSIM v2, built on real-world driving data, EPDMS by +6.1 over its backbone. Project page: https://clap-ad.github.io

cs.RO · cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.36077HF 社区推荐

The Trinity:Capability-Driven Recursive Self-Improvement for Multimodal Agents

Weidong Tang, Ruiheng Wang, Jingxuan Zhang, Hexin Wang, Haoyuan Chen, Zhengfeiyang Zhang, Yixing Ma, Zichen Jiang 等 20 位作者

原文摘要

Multimodal intelligence can be viewed as human-guided co-evolution between worlds and agents through interaction. However, scaling this process faces two bottlenecks: constructing heterogeneous, verifiable environments is costly, and manual Harness iteration cannot keep pace with growing task diversity and complexity. A key insight is that tasks with different modalities and surface forms can share the same underlying capability requirements. We therefore introduce The Trinity, a recursive self-improvement framework coupling world generation with Harness evolution through three roles: Creator, Explorer, and Evolver. Drawing on verified execution trajectories, Evolver shifts the basis of task organization from surface categories to shared requirements, reorganizing the original task tree into a hierarchical capability tree. We introduce Capability-Driven Real2Sim2Real: for heterogeneous tasks at the same node, Creator abstracts their common requirements into simplified, verifiable simulated worlds for capability learning, while evaluation on original tasks measures transfer. Along this capability tree, Explorer progressively evolves reusable capability modules through selective inheritance, forming a Harness tree whose validated updates accumulate across iterations within a fixed topology. The resulting Harnesses specialize in different capabilities, motivating a Capability-Aware Harness Router that uses public task information to select compatible module configurations from the tree. Our evolved Harnesses improve mean performance by 34 points across 28 training benchmarks and 26 points across 18 unseen benchmarks, including two real-world embodied tasks.

cs.AI · cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.35857

Do vision models know that three is three?

Ankit Sonthalia, Arnas Uselis, Seong Joon Oh

原文摘要

Do vision models encode the transition $2{\to}3$ the same way whether it appears in handwritten digits or in hand gestures, and do they "understand" that clean${\to}$dirty is the same transformation regardless of whether it is applied to a kitchen or a carpet? The linear representation hypothesis (LRH) answers this for language models: concept changes correspond to directions in (language) embedding space. In vision, the same concept can manifest in visually unrelated "regimes" that rarely co-occur: images of digits and hands do not share much in common in pixel space. To understand whether vision models still encode the underlying abstract concepts, we study 46 vision encoders across architectures, training objectives, and scales, on 10 concept families including numerosity, age, object identity, and MIT-States attributes. Within each regime, we find statistically significant concept directions near-universally across our tested models. However, two distinct regimes do not always share the same concept direction. Several models learn multiple linear structures for the same concept transition, split across regimes. We call this phenomenon "fragmented linearity": strong local linearity coexisting with near-absent global linearity. Fragmentation is, however, not fixed: models with stronger local linearity progressively escape it (local-global linearity correlated with $τ\in [0.38, 0.89]$ across all concepts), and where global linearity does emerge, concept directions can transfer as linear probes across regimes.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.35854

FL4N: Feed-forward Latent Model for 4D Novel View Synthesis

Eunbeen Hong, Hyeonseo Yu, Honggyu An, Jooho Son, Hyunah Ko, Soowon Son, Chaehyun Kim, Jahyeok Koo 等 11 位作者

原文摘要

4D novel view synthesis (NVS) aims to render dynamic scenes across viewpoints and timestamps, which requires jointly inferring 3D structure and temporal dynamics. Existing monocular feed-forward methods often rely on explicit 4D Gaussian representations, which are prone to temporal inconsistency and severe rendering artifacts. We present FL4N, the first feed-forward latent 4D NVS model. Instead of relying on explicit 4D primitives, FL4N encodes a 4D scene prior into a unified latent representation across space and time. Our core insight is that learning temporal dynamics as residuals on top of a pretrained static latent prior enables effective 4D modeling with limited dynamic training data. To this end, we introduce a lightweight dynamic encoder that leverages static scene features to disentangle object motion from camera-induced parallax. Trained solely with an image synthesis objective, FL4N achieves state-of-the-art 4D NVS quality while rendering in real time. Its learned latent representation generalizes to real-world videos and disentangles viewpoint from time.

cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.35850

OSWAM: An Effective World Action Model with One-Step Action Generation

Guowen Zhang, Liyi Chen, Xu Hu, Pengfei Wang, Jingfan Chen, Chenhang He, Lei Zhang

原文摘要

Direct-policy World Action Models (WAM) have demonstrated the benefits of decoupling action generation from explicit future visual prediction for efficient inference in autonomous driving and embodied control. However, they still require tens of denoising steps to achieve competitive action quality, incurring substantial latency and hindering real-time deployment. Simply reducing the number of denoising steps will collapse the nonlinear multi-step denoising trajectory into coarse updates, substantially degrading action quality. To reduce the denoising steps while enabling real-time action generation, we present an effective world action model with one-step action generation, namely OSWAM, for autonomous driving and embodied control. We argue that it is the absence of structured action priors in random noise initialization that forces direct-policy WAMs to recover coherent action sequences through iterative denoising. Hence, we construct imagination tokens to initialize the diffusion process, providing physical priors and action-conditioned representations. We further introduce privileged information distillation with future conditioning to enhance future-aware action generation. As a result, OSWAM can efficiently and effectively generate high quality actions in just one diffusion step. Our experiments demonstrate that OSWAM achieves comparable or even better results in both autonomous driving and embodied control than direct-policy WAM and Joint-WAM that require tens of steps. It obtains approximately $3\times$ speedup over direct-policy WAM baselines and supports real-time compiled inference with latency as low as 40ms on an NVIDIA RTX 5090 GPU. The project page is available at https://gwenzhang.github.io/OSWAM/

cs.CV · cs.RO

查看论文
arXiv ↗PDF ↗