跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文,不再错过。

覆盖计算机全学科,以 AI 为快捷入口。关注方向、作者和关键词,把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+0新增论文
+0其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描:2026-11-09 07:12 · 全站最近入库:2026-11-08 06:02各信源进度 →

统计口径与数据新鲜度

统计 2026-11-08 08:03 — 2026-11-09 08:03(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-09 07:12最近发现新增版本:2026-11-08 06:02库内论文最新更新时间:2026-11-06 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

17,100 已收录论文14,661 AI 相关4,057 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 6,302 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-09 07:10(北京时间)
新论文 · v1arXiv:2611.05451

From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models

Gayatri Deshmukh, Onkar Susladkar, Adheesh Juvekar, Halil Kilicoglu, Ismini Lourentzou

原文摘要

While discrete tokenization in Unified Multimodal Models (UMMs) establishes strong semantic reasoning, it introduces a strict quantization bottleneck that degrades fine-grained visual details, often resulting in structurally accurate but artifact-heavy images. We introduce FlowShift, a parameter-efficient post-training framework that bypasses this bottleneck by converting pre-trained discrete UMMs into continuous rectified-flow generators. Instead of discarding the backbone's semantic priors, FlowShift replaces discrete visual interfaces with continuous latent projections and applies LoRA to the attention layers. To efficiently boost visual quality without semantic drift, we propose Dual-Objective Velocity Vector Distillation (DOVD), which aligns the model with a high-fidelity continuous teacher and ground-truth continuous trajectories. This approach drastically improves perceptual quality while preserving prompt adherence and reasoning. When applied to leading discrete UMMs, FlowShift significantly improves COCO-30K FID and aesthetic scores, while retaining over 99.5% of multimodal understanding capabilities. Ablations confirm DOVD drives generation quality. Ultimately, FlowShift achieves high-fidelity continuous generation without sacrificing pre-trained semantic grounding.

cs.CV

版本更新 · v2arXiv:2605.07695

OphEdit:无需训练的眼科手术视频文本引导编辑

OphEdit: Training-Free Text-Guided Editing of Ophthalmic Surgical Videos

Ritul Jangir, Arkya Jyoti Bagchi, Mangalton Okram, Saurabh Seetaram Korgaonkar, Aiman Farooq, Deepak Mishra

AI 速读 基于摘要

提出首个无需训练、以文本引导编辑眼科手术视频的框架,通过二阶ODE反演与注意力Value张量注入,在保持眼部解剖结构的同时实现器械替换与手术阶段变化。

cs.CV

查看原文摘要

High-fidelity surgical video generation holds great promise for medical training and AI development, yet precise, controllable editing of such videos remains a formidable challenge. Modifying surgical attributes such as instrument, tissue interactions or procedural phases is particularly difficult due to the strict anatomical and temporal constraints of ophthalmic procedures. We introduce OphEdit, the first training-free framework for text-guided editing of ophthalmic surgical videos. Our approach uses a deterministic second-order ODE inversion pipeline to capture attention Value tensors from the source video and selectively injects them into the conditional branch of classifier-free guidance during denoising, preserving the intricate anatomical geometry of the eye while mapping text-driven semantic modifications onto the video. Clinical evaluation by an ophthalmologist shows that OphEdit handles complex transformations, such as instrument swaps and procedural variations, with superior structural fidelity and temporal consistency compared to adapted general-domain video editors, offering a scalable route to diverse surgical training data. The code and prompts can be accessed at https://github.com/ophedit/OphEdit

新论文 · v1arXiv:2611.05294

Wonder 4:面向图像质量增强的混合专家方案

Wonder 4: A Mixture-of-Experts Recipe for Image Quality Enhancement

Yunan Wu, Bin Sun, Partha Acharjee, Xiaoyu Wang

AI 速读 基于摘要

作者提出 Wonder 4,一种混合专家(MoE)模型,可同时修复真实照片中多种退化,并让每种修复作用于其发生区域。

cs.CV

查看原文摘要

A real photograph is often blurry, grainy, compressed, scratched, and low-resolution at the same time. Each defect has its own severity and its own place in the image. The corrections for these defects conflict with each other. Exist- ing models describe the whole image with one description, so they cannot say where each correction belongs. To solve this problem, we present Wonder 4, a mixture-of-experts (MoE) model for image quality enhancement. It corrects several defects of a real photograph at the same time, each in the region where it occurs. Every block of the model holds a mixture of N = 13 experts. A router, trained with the model, sends every image patch to k = 5 of them. No degradation classifier, mask, prompt, detector, or task label is used. Another issue is that standard MoE training, taken from language models, pushes every expert toward equal use. Equal use applies every correction everywhere, even where it is not needed. We replace this rule with a bias that only sets a minimum share of use. Across all patches, an expert used less than this minimum gets higher router scores until it is selected again. An expert above the minimum is not changed, so no expert is forced to be used as often as the others. The bias needs no extra loss. The method adds no extra module and no extra processing step. We evaluate Wonder 4 on DIV2K and Flickr2K, against five published methods and three open-weight pipelines. It reaches the best LPIPS, DISTS, and NIQE on all three test settings (DIV2K x4, DIV2K x8, and Flickr2K x4).

新论文 · v1arXiv:2611.05291

低密度三维点云分类

Low-Density 3D Point Cloud Classification

Ahmed Baha Ben Jmaa, Faten Chaieb

AI 速读 基于摘要

论文提出面向低密度三维点云的分类架构 LD-PointNet++,并报告其在 ModelNet 基准上跨密度保持 90% 以上准确率。

cs.CV

查看原文摘要

3D point cloud classification, a task in computer vision, has recently gained substantial interest due to its extensive applications across various domains such as augmented and virtual reality, robotics, and autonomous driving. However, classifying 3D point clouds becomes challenging in real-world scenarios where the density levels vary. These variations are due to limitations inherent to current sensing technologies and variable environmental conditions, which often prevent the capture of detailed and comprehensive geometric representations. Existing classification methods, which have demonstrated impressive results on dense point clouds, may be affected and could experience reduced performance when dealing with low-density point clouds. Addressing this challenge, this paper introduces LD-PointNet++, an architecture optimized for the classification of low-density point clouds. To pave the way for LD-PointNet++, we conduct the first benchmarking study that systematically evaluates state-of-the-art point cloud classification methods across varying densities, revealing significant performance declines in low-density environments. Experimental results on the ModelNet benchmark dataset demonstrate that LD-PointNet++ not only outperforms these methods in low-density scenarios but also demonstrates consistent accuracy above 90% across different density levels.

新论文 · v1arXiv:2611.05286

锚定排序对抗微调预训练视觉位置识别模型

Anchored Rank-Adversarial Fine-Tuning Pretrained Visual Place Recognition Models

Mostafa Kira, Bader Rasheed, Manuel Mazzara

AI 速读 基于摘要

提出ARAFT方法,通过对对抗查询的最难正样本施加列表排序损失并锚定冻结预训练模型的干净描述子,在预设干净精度预算内显著提升VPR模型的对抗鲁棒性。

cs.CV

查看原文摘要

Visual place recognition (VPR) localises a camera by retrieving similar geo-tagged images, and current foundation-model VPR models are accurate across cities, seasons and illumination, but not robust to adversarial perturbations: under a white-box attack on the ranking at an imperceptible budget of 1/255, SuperVLAD and BoQ keep at most 0.2% top-1 recall (R@1) on four benchmarks. We propose anchored rank-adversarial fine-tuning (ARAFT), which fine-tunes a released model with a listwise rank loss on each adversarial query's hardest positive and an anchor to the clean descriptor of the frozen pretrained model. Checkpoints are selected under a clean-accuracy budget fixed in advance, and the full trade-off is reported. Within five validation R@1 points of clean cost, plain adversarial training has no admissible checkpoint, whereas ARAFT raises robust R@1 from zero to 47.3 on SuperVLAD and 55.3 on BoQ; plain adversarial training exceeds this only at about three times the clean cost. Against unbudgeted plain adversarial training on the test sets, ARAFT matches its robust R@1 at 2/255 with about ten more clean R@1 points. At 4/255, plain adversarial training is 6-9 robust R@1 points ahead, while ARAFT still keeps its ten-point clean advantage. Developed on SuperVLAD, ARAFT transfers to BoQ without re-tuning, and its robustness holds against stronger optimisers, with no sign of gradient masking.

新论文 · v1arXiv:2611.05280

YANAMI:面向多轮有状态交互的大规模多模态数据集与基准

YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction

Shengyu Wang, Tengyue Nong, Junming Liu, Haosong Peng, Siyu Li, Ruiqi Lei, Fushuo Huo, Yuhao Pan 等 13 位作者

AI 速读 基于摘要

作者提出首个面向多模态有状态交互的大规模多轮对话数据集 YANAMI-400K 与评测基准 YANAMI-Bench,并报告微调 BAGEL 后在图像编辑、生成与有状态交互上均有提升。

cs.CV

查看原文摘要

Unified Multimodal Models (UMMs) integrate visual understanding, generation, and editing within a single unified framework. However, existing unified multimodal datasets and benchmarks remain dominated by single-turn tasks. Even in recent multi-turn datasets, many turns can be solved independently without tracking information across turns. Consequently, they provide limited supervision and evaluation for multimodal stateful interaction, which requires models to understand, maintain, update, and faithfully realize evolving dialogue states across turns. To address this gap, we introduce YANAMI, the first large-scale multimodal dialogue dataset and benchmark for stateful interaction, comprising YANAMI-400K and YANAMI-Bench. YANAMI-400K contains 400K multi-turn multimodal dialogues, with more than 2M images and 2.6M dialogue turns. The dialogues span four representative scenarios: Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning, and require models to continuously track, preserve, and revise evolving multimodal dialogue states across turns. YANAMI-Bench comprises 400 multi-turn dialogues for systematically evaluating multimodal stateful interaction. It jointly evaluates Dialogue State Understanding and Visual Realization, characterizing the gap between models' ability to understand evolving dialogue states and their ability to faithfully realize them in visual outputs. Experiments show that fine-tuning BAGEL on YANAMI-400K improves image editing, generation, and multimodal stateful interaction by 10.3%, 2.0%, and 13.23%, respectively, while preserving its general visual understanding capabilities. Evaluations on YANAMI-Bench further reveal that existing UMMs still struggle to maintain, update, and visually realize evolving multimodal dialogue states.

新论文 · v1arXiv:2611.05279

Möbius:统一理解与生成的运动-语言模型

Möbius: A Unified Motion-Language Model Bridging Understanding and Generation

Jiazhi Shu, Chengfeng Zhao, Jinyi Wang, Yubo Zhao, Zhiyang Dou, Ronghui Li, Yu Lu, Xinwei Jiang 等 10 位作者

AI 速读 基于摘要

Möbius 将扩散生成过程直接放入与运动理解共享的多模态表示空间,在连续帧级原始运动上统一实现运动描述、文本生成运动、时序补全与指令引导编辑。

cs.CV

查看原文摘要

We aim to build a truly unified motion model that can understand, generate, and edit human motion within a single framework. Existing motion UMMs remain only partially unified: motion understanding is handled by a language model, while generation is often delegated to a separate diffusion module operating over learned motion latents. We introduce Möbius, which brings the entire diffusion process into the same multimodal representation space used for motion understanding. Clean motion, noisy motion, and text jointly participate in shared attention, while role-specific parameters process understanding and generation streams. Importantly, Möbius operates directly on continuous frame-level raw motion, requiring neither discrete tokenization nor a pretrained motion VAE. This simple formulation naturally supports motion captioning, text-to-motion generation, temporal completion, and instruction-guided editing within a single model. On HumanML3D, Möbius achieves state-of-the-art performance among unified motion models. Scaling to 1.7B parameters and 1,700 hours of motion data further demonstrates that our framework is both effective and scalable.

新论文 · v1arXiv:2611.05275

Route Anything:面向通用人工智能系统的模型与策略路由研究

Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems

Pengfei Zhou, Yixing Ma, Hexin Wang, Weidong Tang, Zhenglin Wan, Wangbo Zhao, Yang You

AI 速读 基于摘要

该工作提出统一框架 Route Anything,研究在质量—成本权衡下如何为每个任务动态选择并协调专用模型、工具与策略,并构建跨模态大规模评测矩阵来检验路由的潜力。

cs.CV

查看原文摘要

Artificial general intelligence (AGI) requires a method that can understand, generate, reason, and act across diverse domains and modalities. Rather than relying on a single model to master every capability, a promising path toward AGI now is to orchestrate a heterogeneous ecosystem of specialized models, tools, and policies (candidates) into a unified AI system. At the heart of this vision lies a fundamental question: how can an AI system dynamically select and coordinate the most capable candidate for each task while balancing performance and computational cost? In this work, we present Route Anything, a unified framework for studying model and policy routing across general-purpose AI systems. We develop a unified theoretical foundation that characterizes optimal routing under quality--cost trade-offs and establishes how the information available to a router determines its attainable performance. To systematically investigate routing across diverse capabilities, we first construct an omni-modal trajectory matrix spanning 25 benchmarks across language understanding, image, video, and speech generation, coding and workplace agents, and robotic manipulation, comprising 374,401 scored runs from 452 candidate configurations. Building on this framework, we train and test different routers to investigate the potential of omni-modal model routing, the capabilities of learned routers, and the conditions under which routing improves efficiency. Our results reveal substantial opportunities for capability orchestration: per-input oracle routing outperforms the best fixed candidate by 5.4 points, and learned routers outperform it by 0.3 points across 16 benchmarks, by 2.1 points on video generation, and by 1.1 points on robot manipulation while using 24% less policy time. More importantly, routing raises efficiency by 8.82x on average compared with the reference model.

新论文 · v1arXiv:2611.05449

SO(3)-Equivariant Learning on CAD Boundary Representations

Matteo Ballegeer, Dries F. Benoit

原文摘要

Learning on boundary representations (B-reps) requires robustness to the pose of a part, since CAD models are not stored in a canonical orientation. Existing B-rep networks either operate on world-frame coordinates and normals, and thus degrade under rotation, or encode each face in a local reference frame. Local frames yield invariance, but reduce inter-face messages to scalars and discard the relative orientation of adjacent faces. We propose SO3brep, an SO(3)-equivariant graph neural network on the B-rep face-adjacency graph. Node features are computed in local frames and mapped into a shared frame by Wigner D-matrices. Clebsch-Gordan tensor products couple these features with the directions between adjacent faces, so that relative orientation is preserved throughout message passing and the output remains SE(3)-invariant. Across three face-segmentation and two classification benchmarks with randomly rotated test parts, SO3brep achieves the highest accuracy among the compared B-rep networks and equivariant point cloud networks, with the fewest parameters. Its advantage is largest with few labeled parts, a setting typical of industrial CAD. Code is available at https://github.com/UGent-CVAMO/so3brep.

cs.CV · cs.LG

版本更新 · v2arXiv:2607.00861

TrajLoc:面向多目标运动控制的轨迹注意力定位

TrajLoc: Trajectory-Attention Localization for Multi-Object Motion Control

Omer Sela, Inbar Huberman-Spiegelglas, Michael Rotman, Sagie Benaim, Avi Ben-Cohen

AI 速读 基于摘要

TrajLoc 通过在注意力层中为每个对象 token 注入以目标位置为中心的高斯热图,实现图像到视频生成中多对象轨迹的独立空间约束。

cs.CV · cs.GR

查看原文摘要

Controlling the motion of multiple objects in image-to-video (I2V) generation requires preserving object identities while enforcing adherence to distinct target trajectories. This becomes particularly challenging as the number of objects increases and their paths intersect or occlude one another. Existing approaches entangle multiple trajectories within a shared, dense conditioning signal, making object-level correspondence difficult to preserve in crowded scenes. We depart from this paradigm and enforce a strict, per object spatial constraint that isolates instances independently. Our method, TrajLoc, achieves this directly within the attention layers by substituting the cross-attention weights of each object token with a Gaussian heatmap centered on its target location at every frame. The same per object token interface carries trajectory and depth through a learned embedding and preserves identity by encoding first frame appearance in place of an object token. Evaluations across six datasets, featuring up to 20 simultaneously controlled objects and out of distribution real world scenes, demonstrate that our method consistently improves both visual fidelity and trajectory adherence. Applied to two architecturally distinct backbones (CogVideoX 5B and WaN 2.1 14B), our approach achieves average gains of +4.3 dB PSNR and a 51% reduction in trajectory end point error compared to the strongest baselines. Project page: https://sela-omer.github.io/traj-loc/

版本更新 · v2arXiv:2606.01643

手语生成中的条件坍缩:一项诊断与覆盖度论证

Conditional Collapse in Sign Language Production: A Diagnostic and a Coverage Argument

Rui Hong, Jana Košecká

AI 速读 基于摘要

作者提出用三个独立层级诊断手语生成模型的条件坍缩问题,并论证目标忠实度取决于训练句对测试条件的覆盖密度而非语料规模。

cs.CV

查看原文摘要

The goal of Sign Language Production (SLP) is to translate natural language text into corresponding sign language movements performed by a virtual avatar. The quality of the generated motion is typically evaluated by a motion-space Frechet distance (FID) and back-translation BLEU score on benchmarks such as How2Sign. The existing approaches can improve both metrics substantially while the underlying generator fails to represent the input sentence, a failure we call conditional collapse. In this work we propose to evaluate the generated motion at three independent levels: (tau1) initial-pose conditioning, (tau2) output diversity, and (tau3) target faithfulness. We compute these as pairwise-distance ratios using latent representations of a frozen motion autoencoder (MoAE), judged against each dataset's own GT-vs-GT reference. We evaluate diffusion and regression SLP models on How2Sign, including a re-implemented Neural Sign Actors, and show that tau3 faithfulness is never attained, while FID varies by nearly two orders of magnitude and is uncorrelated with faithfulness. We then apply the same models and diagnostic to Phoenix-2014T, CSL-Daily, and gloss-level ASL3DWord and show that faithfulness varies proportionally to how densely the training sentences cover the test conditions (conditioning coverage) and not corpus size. The smaller but densely covered Phoenix-2014T and ASL3DWord datasets generate faithful sign motion, while CSL-Daily follows the input text yet falls short of sign language faithfulness. Using the largest, open-domain How2Sign dataset, the models collapse at every scale. We introduce two measurements of the text data that rank the corpora in the order of how densely the training sentences fill the embedding space, and how close each test sentence lies to its nearest training sentence.

新论文 · v1arXiv:2611.05231

组合诊断:通过组合已学放射学征象检测和定位未见疾病

Diagnosis by Composition: Detecting and Localizing Unseen Diseases by Composing Learned Radiological Findings

Kaihua Hou, Wei Liu, Yi Huang, Hao Zhang, Haibo Jin, Yifei Sun, Yuan gao, Zuoou Li 等 13 位作者

AI 速读 基于摘要

提出诊断即组合(DbC)框架,通过学习放射学征象词汇并依据指南组合征象图来诊断和定位未见疾病,在肝脏CT上优于对比方法。

cs.CV

查看原文摘要

Deep models for radiological diagnosis are typically trained to predict disease labels directly from images, which limits them to diseases seen during training and makes their decisions hard to interpret in radiologists' terms. Radiology, however, proceeds from visual findings to disease through published guidelines that make this decision rule explicit, interpretable, and reproducible. We introduce diagnosis by composition (DbC), a representation learning framework that encodes this findings-first logic. DbC learns a vocabulary of radiological findings by aligning image features with the text embeddings of the findings, each finding represented as a dense map over the scan. A disease is defined by a set of required findings derived from its guidelines, and its diagnosis map is a fixed, parameter-free composition of the corresponding finding maps. Because the finding-to-disease mapping is derived from guidelines rather than learned, DbC is interpretable and auditable by design, can extend to new diseases by specifying their findings without retraining, and can diagnose unseen diseases by composing their findings. On multi-phase liver CT, DbC detects and localizes unseen diseases, rare diseases from a different institution, and an external cohort significantly better than all comparators, including models that align images with disease names. DbC also improves detection and localization in the fully supervised setting and remains superior across training-set sizes. Although demonstrated on liver CT, DbC applies wherever radiological guidelines define diseases through findings.

新论文 · v1arXiv:2611.05223

PLANTEUR:面向热带地区的作物类型分割基准

PLANTEUR: A Crop-Type Segmentation Benchmark for Tropical Regions

Piper Wolters, Gabriel Tseng, Favyen Bastani, Joseph Redmon, Yawen Zhang, Hadrien Sablon, Ando Shah

AI 速读 基于摘要

作者提出 PLANTEUR,一个基于法国海外领地卫星影像的作物类型制图基准,涵盖六个热带区域、多源卫星序列与跨岛屿泛化评测协议。

cs.CV · cs.LG

查看原文摘要

We introduce PLANTEUR, a benchmark for crop-type mapping from satellite images in the French overseas territories. Building on the PASTIS design, PLANTEUR provides georeferenced and aligned Sentinel-2, Sentinel-1, and Landsat-8 sequences with parcel-derived labels spanning six tropical regions and an 8-class label space. We define both a standard test split and a leave-one-island-out protocol to measure cross-territory generalization, and evaluate fully supervised models, linear probes, and fine-tuned foundation models. The benchmark code and data are open source and available on GitHub and Hugging Face.

版本更新 · v2arXiv:2609.18772

手语手形的零样本跨语言识别

Zero-Shot Cross-Lingual Recognition of Sign Language Handshapes

Marcel Granero-Moya, Carolina del Corral Farrarós, Gloria Haro, Coloma Ballester, Ricardo Marques

AI 速读 基于摘要

作者提出首个手形零样本跨语言框架,将 ASL 模型迁移到加泰罗尼亚手语(LSC),并借助共享的音系特征分解实现无目标语言训练标签的识别。

cs.CL · cs.CV

查看原文摘要

Sign language processing advances rapidly for high-resource languages such as American Sign Language (ASL), yet most of the world's sign languages lack the phonological annotations new methods require. We present the first zero-shot cross-lingual framework for handshape recognition, transferring from ASL to Catalan Sign Language (LSC). Our approach leverages the decomposition of handshapes into five phonological features -- selected fingers, flexion, spread, thumb position, and thumb contact -- shared across both languages, to decode LSC handshapes from predicted features via a composite phonological distance metric. We evaluate three architectures (MLP, SL-GCN, SHuBERT) trained on two ASL corpora (PopSign, Sem-Lex) against a 37-handshape, single-signer LSC benchmark. Zero-shot transfer proves viable once recording-format disparities are harmonized, reaching 80.0% phonological feature accuracy and 54.5% expected handshape accuracy. Phonological decomposition thus offers a bridge for extending sign language technologies to low-resource languages without any target-language video training labels.

版本更新 · v2arXiv:2609.01800

利用大型深度神经网络改进合成孔径声呐图像中的自动目标识别

Improved Automatic Target Recognition in Synthetic Aperture Sonar Imagery Using Large Deep Neural Networks

C. J. Moore, Alex Hurt, Jordan Malof

AI 速读 基于摘要

该研究探讨在合成孔径声呐自动目标识别中使用高容量深度神经网络,并在小数据集上系统评估多种正则化策略的影响。

cs.CV

查看原文摘要

Automatic Target Recognition (ATR) in Synthetic Aperture Sonar (SAS) is often underpinned by deep neural networks (DNNs). Recent research has focused on relatively low-capacity DNNs (i.e., relatively fewer trainable parameters) to avoid overfitting on the small training datasets that are typically available for SAS. However, recent research has found that increasing DNN capacity often improves performance, even when training data is scarce. Motivated by this emerging finding - termed the double descent phenomenon - this work investigates the use of especially high-capacity DNNs for SAS-ATR. We evaluate the performance of several state-of-the-art DNN architectures as we increase their model capacity. We examine performance in a realistic leave-one-geo-location-out cross-validation scheme. We find that increasing model capacity often increases performance substantially, and is never harmful. All of our large models significantly outperform two baseline models from recent publications. Training high-capacity models on small dataset however requires careful regularization, and we also perform systematic experiments showing the impact of several regularization strategies: pretraining on other imaging modalities, data augmentation (including some new to SAS), DropPath, and weight decay. We show that each of these strategies independently improves the performance of the large models.

版本更新 · v5arXiv:2507.22017

Cyst-X:用于胰腺囊肿风险分层与跨机构AI的多中心MRI基准

Cyst-X: A Multicenter MRI Benchmark for Pancreatic Cyst Risk Stratification and Cross-Institutional AI

Hongyi Pan, Gorkem Durak, Halil Ertugrul Aktas, Ziliang Hong, Zhixiang Wang, Elif KelesDeniz Seyithanoglu, Zheyuan Zhang, Alpay Medetalibeyoglu 等 30 位作者

AI 速读 基于摘要

作者构建了覆盖七中心、764例患者的胰腺囊肿MRI公开基准,并提供分割与IPMN风险预测的参考AI模型及联邦学习模拟结果。

eess.IV · cs.CV

查看原文摘要

Intraductal papillary mucinous neoplasms (IPMNs) are pancreatic cancer precursors, and guideline-based risk stratification both over-treats indolent lesions and misses progression. We present Cyst-X, a public benchmark of 1,461 T1-weighted (T1W) and T2-weighted (T2W) MRI volumes from 764 patients at seven centers, with expert pancreas masks, risk labels, splits and reference AI risk-prediction models. PanSegNet segmented the pancreas with Dice of 86.34% (T1W) and 89.26% (T2W) on held-out centers. For the primary task, high-risk versus low-risk IPMN, a 3D DenseNet-121 achieved cross-validated AUCs of 74.59% and 73.45%; complementary broad binary and three-class tasks reached 78.67--81.28% and 73.47--75.76%. Because patient images often cannot be pooled across institutions, we also simulated federated learning across the seven centers. On T2W MRI, FedProx was similar to centralized training in center-averaged external AUC (71.00% versus 71.18%) but not in pooled AUC (62.35% versus 77.49%). Radiologist comparisons showed cohort-dependent sensitivity--specificity tradeoffs. Cyst-X supports reproducible method development and identifies within-IPMN risk discrimination and cross-center score comparability as open challenges.

新论文 · v1arXiv:2611.05205

TAQ-MOTR: Tracking-Aware Detection Query Formation for End-to-End Multiple Object Tracking

Hanyang Liu, Hong Zhang, Ding Yuan, Yang Han, Yifan Yang

原文摘要

Tracking-by-propagation (TBP) methods perform end-to-end multi-object tracking by propagating track queries across frames, jointly handling detection and identity association without heuristic post-processing while naturally modeling temporal information. However, as detection and association are tightly coupled, detection queries in TBP are expected to discover currently untracked objects and provide complementary current-frame evidence for challenging tracks whose queries and identities are retained. Existing methods largely inherit detection-query formation from single-frame detection and construct such queries before the decoder with little awareness of trajectory states. Consequently, currently untracked objects may be missed, while challenging tracked objects with reduced visibility under occlusion may lack complementary evidence, causing association failures or duplicate responses. To address these issues, we propose TAQ-MOTR, a tracking-aware detection query formation framework that employs a pre-decoder Hybrid Query Generator (HQG) to fuse current-frame image features, track memory, and learnable memory. By conditioning detection queries on trajectory states, TAQ-MOTR adapts them to \rev{the discovery of currently untracked objects and tracked-object complementation We further introduce a training-only auxiliary detection branch whose one-to-one label assignment is decoupled from that of the main tracking branch, providing additional detection supervision to strengthen the shared HQG and decoder for the discovery of currently untracked objects. TAQ-MOTR improves performance on DanceTrack, SportsMOT, and BFT, achieving 72.0 HOTA, 63.1 AssA, and 76.6 IDF1 on the DanceTrack test set. Code is available at https://github.com/Magicmiya/TAQ-MOTR

cs.CV

版本更新 · v2arXiv:2611.03533

CoLa3D: Composable Latent 3D Decomposition

Xudong Cai, Yongcai Wang, Deying Li, Chuanxia Zheng

原文摘要

We introduce CoLa3D, a simple and effective model for composable 3D scene decomposition that, given a single whole-scene latent from either a single image or an existing scene mesh as input, directly outputs a set of object meshes with consistent spatial layout that can be assembled back into a coherent scene. CoLa3D introduces a novel paradigm for 3D scene decomposition within a shared whole-scene latent, unlike alternatives that perform per-object generation in canonical spaces and then glue the objects back together with fragile pose and scale estimation. The network is a lightweight, flexible, and scalable query-based transformer that accepts various types of queries, including 2D masks, 3D points, and learnable queries, and learns to decompose the scene into objects in a compact latent space. On both synthetic and real-world datasets, CoLa3D achieves state-of-the-art performance on composable 3D scene decomposition, substantially improving global scene-level consistency and object-level quality while running 2 times faster and using less than 1% of SAM 3D's training data. Visible depth on par with MoGe2 further confirms that latent decomposition preserves accurate visible scene geometry while producing physically complete, decomposable object meshes.

cs.CV

新论文 · v1arXiv:2611.05197

Robust Reconstruction of Power Lines from Incomplete Mobile and Aerial LiDAR Data

Mohamed Gaha, Damos Ayobo Abongo, Philippe Massicotte, Eya Ben Kacem, Wael Jaafar, Rami Langar, Christian Buteau

原文摘要

Accurate vegetation-encroachment assessment and risk mitigation require electric utilities to accurately locate and characterize distribution-network assets, including poles and power lines, and distinguish among medium-voltage, low-voltage, neutral, and telecommunication lines. Mobile and aerial Light Detection and Ranging (LiDAR) point clouds provide detailed 3D representations of these assets and their surroundings. However, power-line detection and reconstruction remain challenging because conductor returns are often sparse, incomplete, noisy, or occluded by vegetation. These challenges are especially significant in distribution networks, where lower pole heights and complex line configurations place conductors close to trees. In addition, mobile and aerial LiDAR datasets differ substantially in point density, viewing geometry, spatial coverage, and occlusion patterns. We propose a robust and automated reconstruction method combining scaled density-based clustering, robust polynomial fitting, and iterative fragment merging to recover individual conductors under severe data incompleteness and occlusion. Following semantic classification, the same reconstruction pipeline processes both mobile and aerial LiDAR data without modality-specific code or parameter tuning. The method is evaluated through degradation experiments, benchmarking, parameter sensitivity analysis, and large-scale validation. On the aerial dataset, it achieves 96.87% precision, 96.66% recall, a 96.76% F$_1$-score, and a geometric RMSE of 0.127 m. On the mobile dataset, where conductor-level reference annotations were unavailable, it achieves a point-to-model RMSE of 0.045 m and a reconstruction coverage of 95.27%. Source code, documentation, and evaluation data are publicly available at https://github.com/varen-project.

cs.CV

新论文 · v1arXiv:2611.05192

From Natural Images to Medical Scans: Transferability of Diffusion Models for Medical Image Generation

Efthymios Georgiou, Ioannis Kakogeorgiou, Sarah Brüningk

原文摘要

Transfer learning from natural images is standard in supervised and increasingly adopted in self-supervised medical image analysis, but its role in generative modelling, and in particular diffusion denoisers, remains underexplored. Medical diffusion models typically train the denoiser from scratch, with no systematic evaluation of natural-image transfer. We analyse this across latent and pixel paradigms, model sizes (131M-952M) and two radiology datasets (RadImageNet, BraTS), and ask which aspects of transfer support generation. We find that: i) Natural-image initialisation consistently improves generation, by 21-34% relative gFID in pixel space against 14% in latent space, where the tokeniser supplies much of the prior. gFID gains mirror downstream pathology classification and radiomic-feature FD, both proxies for clinical fidelity. ii) Benefits reflect spatial structure, not semantics: semantic probing does not predict generation quality, whereas a spatial-structure metric adapted to internal denoiser representations tracks gFID at $|r|=0.85$, positioning spatial organisation as a channel through which pretraining manifests. iii) Mixed-domain denoiser training shows that natural and medical images share similar layer-wise representations up to the late blocks, and mixing natural images into medical training improves fidelity. iv) Transfer benefit is tied to natural image initialisation, outperforming in-domain medical pretraining and several reinitialisation schemes. Overall, natural image denoisers transfer positively to medical ones in our setup. Code is available at https://github.com/CAIROLab-Bern/rad-dit

cs.CV

新论文 · v1arXiv:2611.05181

What Does the Model See? Representation Validity in Multimodal LLM-Based Video Analysis

Christian Pipal, Eva-Maria Vogel, Morgan Wack

原文摘要

Multimodal large language models (MLLMs) allow social scientists to analyze video at scale, but researchers must decide which parts of a video to provide as input. We use the term representation validity to examine how text, audio, video, and combined inputs affect measurement and subsequent analysis. We compare input representations across six open-weight MLLMs and three human-annotated video datasets. No representation consistently performs best. On seven news-related constructs, native-video performance varies little across models (macro-F1 = .70-.77), while extracted-text performance ranges from .52 to .80. Three models perform better with text and three with video. Combining inputs rarely improves performance. Higher classification accuracy also does not reliably produce estimates closer to those based on human codes. The highest-F1 representation best reproduces human-coded regression coefficients in 17 of 42 comparisons. Even when each model uses its highest-F1 deployable representation, 45% of sample-prevalence estimates differ from the human-coded benchmark by more than ten percentage points. Input representation therefore requires validation for both the construct being measured and the quantity estimated from the resulting labels.

cs.CV · cs.CL

新论文 · v1arXiv:2611.05172

PadLoc: Training-Free Floor-Plan Localization by Sparse Geometric Matching

Marian Bookhahn, Frank Neumann

原文摘要

We propose PadLoc, a training-free localization back-end that places a moving sensor within a known floor plan. Floor plans are cheap, persistent, and robust to appearance change, but recent learned-depth methods localize against them by filtering per-frame depth through dense $(x,y,θ)$ pose grids. PadLoc keeps the swappable depth front-end and replaces the dense grid back-end with sparse geometric matching: dominant wall orientations define a small pose-candidate set, masked normalized cross-correlation scores observed free space against the plan, and a moving-pose Gaussian-mixture Bayes filter plus consensus readout decodes the trajectory. Holding each competitor's depth fixed, PadLoc matches or beats UnLoc's forward filter and the F3Loc filter at every Gibson trajectory length, and UnLoc's offline refinement from $T=35$ on; it exceeds Z-FLoc's published results with a shared MapAnything front-end. On Bonn LiLocBench, PadLoc improves pooled SR@1m over UnLoc by 23-32 pp under wheel odometry and 35-43 pp with ground-truth motion; with full-FOV LiDAR plus wheel odometry the same readout reaches 100% SR@1m at 9-10 cm median error on the static sequence. On the Hilti-Trimble Localization Challenge PadLoc localizes a camera-only fisheye rig on construction floors hundreds of metres across at 0.90-0.98 online SR@2m on three of five sessions; adding learned wall isolation, applied to every session, covers all five at 0.67-0.92 with the localizer unchanged throughout. PadLoc thus matches or outperforms dense grid back-ends while preserving sensor modularity and a compact state that does not grow with trajectory length.

cs.CV · cs.RO

新论文 · v1arXiv:2611.05158

Perturb, Repair, Verify: Self-Play Vision-Language Verifiers for Compositional Understanding

Zijun Huang, Yingjun Du, Cees G. M. Snoek

原文摘要

Vision-language verifiers judge whether an image faithfully matches a textual description, yet existing training pipelines typically rely on static supervision from stronger external judges or curated hard-negative data. Such supervision requires external generation or annotation, is teacher-bounded, and is difficult to adapt as model failures evolve. We introduce PROVE (Perturb-and-Repair Optimization for Vision-Language Evaluation), a self-play framework that trains verifiers through a closed perturb-verify-repair-verify loop. Starting from a known-correct image-caption pair, a Mutator introduces a typed perturbation over objects, attributes, counts, or relations. The verifier then judges the perturbed caption and provides textual feedback, which a Repairer uses to recover a faithful caption for re-verification. These two verification rounds produce structured rewards for informative mismatches and visually grounded repairs. Since perturbation, verification, and repair are role-conditioned behaviors of the same evolving multimodal backbone, updating the shared parameters also improves the verifier itself. Experiments show that PROVE improves fine-grained compositional verification across model scales and families and strengthens visual-detail robustness on the reference backbone, with limited, benchmark-dependent changes on broader multimodal tasks.

cs.CV · cs.LG

新论文 · v1arXiv:2611.05145

When Cell Crops Share a Parent: Measuring and Isolating Acquisition Leakage in Cervical Cytology Benchmarks

Snehasish Ghosh, Soham Mitra, Antareep Majumdar, KC Santosh, Nibaran Das

原文摘要

Public cervical cytology benchmarks release many cell crops cut from far fewer microscope fields or slides and are often split at the crop level, so test cells share acquisition parents with training cells. In a preregistered study, crop-level splitting raised SIPaKMeD accuracy by 0.97 percentage points across twelve architectures and twenty seeds (95% CI 0.59-1.39), and CRIC accuracy by 12.08 points (11.16-12.99). On SIPaKMeD, a factorial that fixes training crop and parent counts attributes the gain to sibling exposure (1.73 points, 0.95-2.51), not to the extra parent diversity of crop-level splits (0.00). Parent identity is decodable at about 100 times chance, yet erasing 82% of its above-floor linear recoverability left the gap unchanged, and Grad-CAM did not distinguish leaking from clean models; matching test cells' color statistics to an unseen same-class field removed most of the advantage. Decodability and saliency cannot substitute for a parent-disjoint split.

cs.CV

版本更新 · v3arXiv:2608.28207

Explainable Diabetic Retinopathy Classification Using Vision Foundation Models

Abhishek Verma, Anila Krishna, Abhishek Gajanan Bankar, Juan Miguel Lopez Alcaraz

原文摘要

Diabetic retinopathy (DR) is a leading cause of preventable blindness, and automated screening must be both accurate and trustworthy. We study an explainable DR classification framework built on vision foundation models and three transfer learning strategies. Three backbones, DINOv2, CLIP, and Vision Transformer (ViT), were adapted by full fine-tuning, linear probing, and Low-Rank Adaptation (LoRA). Models were trained and internally evaluated on the ODIR dataset and externally evaluated on APTOS to assess generalization. DINOv2-LoRA achieved the highest internal AUROC (0.758), whereas DINOv2 and ViT full fine-tuning achieved the highest external AUROC (0.920). Probability calibration was assessed with reliability analysis after isotonic regression. For explainability, Grad-CAM and HiResCAM maps were compared with expert-annotated lesion masks from the IDRiD dataset using Dice, Intersection over Union (IoU), and Pointing Game metrics. The results show that foundation models, particularly DINOv2, provide strong predictive performance, and that LoRA is a parameter-efficient alternative to full fine-tuning. Spatial overlap between explanation maps and annotated lesions was limited (mean Dice of about 0.13 for the combined lesion mask), which highlights the value of quantitatively testing whether model attention corresponds to clinically relevant retinal lesions.

cs.CV · cs.LG

版本更新 · v3arXiv:2601.04153

Diffusion-DRF: Free, Rich, and Differentiable Reward for Video Diffusion Fine-Tuning

Yifan Wang, Yanyu Li, Gordon Guocheng Qian, Sergey Tulyakov, Yun Fu, Anil Kag

原文摘要

Video diffusion alignment has been heavily relied on scalar rewards. These rewards are typically derived from learned reward models in human preference datasets, requiring additional training and extensive collection. Moreover, scalar rewards provide coarse, global supervision, offering limited prompt-generation mismatch credit assignment and making models prone to reward exploitation and unstable optimization. We propose Diffusion-DRF, a free, rich, and differentiable reward framework for video diffusion fine-tuning. Diffusion-DRF employs a frozen, off-the-shelf Vision-Language Model (VLM) as the critic, eliminating the need for reward model training. Instead of relying on a single scalar reward, it decomposes each user prompt into multi-dimensional questions with freeform dense VQA explanation queries, yielding information-rich feedback. By direct differentiable optimization over this rich feedback, Diffusion-DRF achieves stable reward-based tuning without preference datasets collection. Diffusion-DRF achieves significant gains both quantitatively and qualitatively, outperforming state-of-the-art Flow-GRPO by 4.74% in overall performance on unseen VBench-2.0.

cs.CV

版本更新 · v9arXiv:2508.15568

Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment

Youjia Zhang, Youngeun Kim, Young-Geun Choi, Hongyeob Kim, Huiling Liu, Sungeun Hong

原文摘要

Test-time adaptation (TTA) enhances the zero-shot robustness under distribution shifts by leveraging unlabeled test data during inference. Despite notable advances, several challenges still limit its broader applicability. First, most methods rely on backpropagation or iterative optimization, which limits scalability and hinders real-time deployment. Second, they lack explicit modeling of class-conditional feature distributions. This modeling is crucial for producing reliable decision boundaries and calibrated predictions, but it remains underexplored due to the lack of both source data and supervision at test time. In this paper, we propose ADAPT, an Advanced Distribution-Aware and backPropagation-free Test-time adaptation method. We reframe TTA as a Gaussian probabilistic inference task by modeling class-conditional likelihoods using gradually updated class means and a shared covariance matrix. This enables closed-form, training-free inference. To correct potential likelihood bias, we introduce lightweight regularization guided by CLIP priors and a historical knowledge bank. ADAPT requires no source data, no gradient updates, and no full access to target data, supporting both online and transductive settings. Extensive experiments across diverse benchmarks demonstrate that our method achieves state-of-the-art performance under a wide range of distribution shifts with superior scalability and robustness.

cs.CV · cs.LG

版本更新 · v4arXiv:2605.12309

G$^2$TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models

Junxian Li, Kai Liu, Zizhong Ding, Zhixin Wang, Zhikai Chen, Renjing Pei, Yulun Zhang

原文摘要

The development of separate-encoder Unified multimodal models (UMMs) comes with a rapidly growing inference cost due to dense visual token processing. In this paper, we focus on understanding-side visual token reduction for improving the efficiency of separate-encoder UMMs. While this topic has been widely studied for MLLMs, existing methods typically rely on attention scores, text-image similarity and so on, implicitly assuming that the final objective is discriminative reasoning. This assumption does not hold for UMMs, where understanding-side visual tokens must also preserve the model's capabilities for editing images. We propose G$^2$TR, a generation-guided visual token reduction framework for separate-encoder UMMs. Our key insight is that the generation branch provides a task-agnostic signal for identifying understanding-side visual tokens that are not only semantically relevant but also important for latent-space image reconstruction and generation. G$^2$TR estimates token importance from consistency with VAE latent, performs balanced token selection, and merges redundant tokens into retained representatives to reduce information loss. The method is training-free, plug-and-play, and applied only after the understanding encoding stage, making it compatible with existing UMM inference pipelines. Experiments on image understanding and editing benchmarks show that G$^2$TR substantially reduces visual tokens and prefill computation by 1.94x while maintaining both reasoning accuracy and editing quality, outperforming baselines on almost all benchmarks. Code is at: https://github.com/lijunxian111/G2TR.

cs.CV

新论文 · v1arXiv:2611.05106

Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for 3D Mitral Valve Segmentation

Javeria Javed, Ahmed Salim, Muhammad Mahboob Ur Rahman

原文摘要

Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert- annotated volumetric datasets. To address this critical limitation, we investigated Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images (from platforms like Chat- GPT, Google Gemini, and Microsoft MAI) as annotation-free style donors for 3D mitral valve segmentation. We employed a DeepMitral residual U-Net model on the public MVSeg2023 dataset, comparing six conditions over three runs: real-only training, traditional augmentation, FDA with each generative source, and a non-AI FDA control using real-image style donors, plus an optimization-matched real-only control and a \b{eta} sensitivity analysis. Case-level paired testing showed that FDA with these generative sources led to significant reductions in boundary- and surface-related errors across all three donor conditions relative to the primary real-only baseline. However, the impact on the core segmentation metric, Generalized Dice Score (GDS), proved generator-dependent. ChatGPT-FDA showed a significant GDS advantage over the optimization-matched control but not over the primary baseline, and Gemini-FDA showed no significant GDS difference. Notably, only MAI-FDA provided a statistically robust and significant increase in GDS compared to the primary baseline. The \b{eta} sensitivity was not robust across training runs. These findings confirm that FDA remains an important augmentation mechanism. While generative-AI images offer a practical means of acquiring style diversity without new annotations, the non-AI control performed comparably to ChatGPT-FDA, so we do not conclude that this approach is superior to training with real-image donors.

eess.IV · cs.CV

新论文 · v1arXiv:2611.05443

SurgTransfer: Multi-Step Latent Action Modeling for Action Transfer in Surgical World Model

Saurabh Koju, Love Panta, Deepak Thapa, Rudra P. K. Poudel, Prajwal Ghimire, Angus Watson, Yash Raj Shrestha, Prashnna Kumar Gyawali 等 9 位作者

原文摘要

Surgical world models have the potential to provide intra-operative guidance, accelerate risk-free training and facilitate transfer of expert surgical maneuvers. However, the use of world models for transferring surgical expertise has been largely unexplored. Moreover, recent approaches in this domain have mainly relied on ground truth kinematics actions or expert annotations for specific tasks, limiting their applicability beyond the narrow range of tasks with ground-truth annotations. In contrast, existing works in the natural domain have shown promising results in learning latent actions solely from unlabeled videos and transferring the learned actions across contexts. We observe that this paradigm is short-sighted for the surgical domain, failing to capture the hierarchical and goal-directed structure of surgical procedures. Thus, we propose a novel multi-step action learning framework to learn context-invariant latent actions that are easily transferable across various surgical contexts. We perform extensive experiments on the publicly available datasets, SurgToolLoc-2022 and PitVis-2023, verifying our approach with both quantitative metrics and qualitative evaluation by surgical experts. We have published our code and the implementation details at https://github.com/bhattarailab/Surgical-Vision-World-Model.

cs.CV · cs.AI · cs.LG