跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1085新增论文
+884其中 AI 相关
+130已有论文修订
+94新增 AI 导读

arXiv 最近完成扫描:2026-11-19 06:59 · 全站最近入库:2026-11-19 06:59各信源进度 →

统计口径与数据新鲜度

统计 2026-11-18 08:37 — 2026-11-19 08:37(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-19 06:59最近发现新增版本:2026-11-19 06:59库内论文最新更新时间:2026-11-18 08:00

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

26,871 已收录论文22,375 AI 相关8,156 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 108 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-19 06:57(北京时间)
新论文 · v1arXiv:2611.23189

面向通用医学图像分割的检索增强校正

Retrieval-Augmented Correction for Generalist Medical Image Segmentation

Vincenzo Marcianò, Xiaoming Zhang, Michela Antonelli, Sebastien Ourselin, Maria A Zuluaga

AI 速读 基于摘要

GRACE 用共享的已验证病例知识库为掩膜校正提供检索上下文,训练一次即可跨多器官、多模态编辑分割掩膜。

cs.CV · cs.AI

查看原文摘要

Automated segmentation is routinely used as the first step in quantitative image-based clinical workflows, where predicted masks are subsequently reviewed and corrected by human operators. Automated correction of the residual structural errors is currently addressed one anatomy at a time, so the number of trained correctors scales with the number of segmentation targets. We present GRACE, a retrieval-augmented refiner that edits an input mask under context drawn from a shared knowledge base of verified prior cases. The refiner is trained once on a multi-organ corpus spanning compact organs, cavities, branching vessels, and tumour lesions. In the held-out dataset-transfer evaluation, three held-out datasets query a fixed knowledge base built from training data only. A separate leave-one-anatomy-out experiment tests parameter-free index extension: verified cases of an unseen solid organ enter the knowledge base, and the refiner stays frozen. A light shape-aware projection head on a frozen multimodal encoder maps embedding similarity to the shape agreement of the masks, so the knowledge base returns cases that the refiner can use as a shape prior. We evaluate GRACE on 18 public datasets spanning CT, MRI and ultrasound, using nine refinement configurations and seven upstream segmentors that span weak, mid, and strong operating regimes. We report each change with the DSC of its input mask. On the weak tier, the input DSC is 0.386 and GRACE adds up to +0.35 DSC. On a strong segmentor, the input DSC is 0.932 and the mean change is small. Transfer to an imaging modality that is absent from training and from the index is partial and depends on the anatomy, and remains an open research direction. Our results suggest retrieval-conditioned refinement is a practical alternative to anatomy-specific correction models. Code: https://github.com/robustml-eurecom/grace

新论文 · v1arXiv:2611.23113

LUTBench:文本驱动摄影风格迁移基准测试

LUTBench: Benchmarking Text-Driven Photographic Look Transfer

Chuanzhi Xu, Boyu Wei, Chengkun Yue, Xuanhua Yin, Yiyang Li, Chupeng Liu, Zihan Deng, Weidong Cai

AI 速读 基于摘要

作者构建了基于91个3D LUT与FiveK照片的文本驱动摄影调色基准,用统一协议比较16种方法在目标保真度、输入转换敏感性和泛化性上的表现。

cs.CV

查看原文摘要

Photographic color grading shapes the tone and color of a photograph to achieve a desired look. Text-driven photographic look transfer lets users specify these adjustments in natural language, with the goal of reproducing the requested look across photographs while preserving their content. However, prior research on text-driven color grading uses different datasets, text instructions, target definitions, and metrics. These differences make it difficult to compare how faithfully the methods reproduce the same look. To compare methods under a shared protocol, we introduce LUTBench. We collect 91 three-dimensional lookup tables (3D LUTs) and define one photographic look per LUT with a specified input conversion. Applying these looks to 5,000 MIT-Adobe FiveK photographs produces 455,000 input-target image pairs with reproducible target images. For each look, we use templates to turn measured tone and color changes on fixed color samples into candidate clauses. We use training image measurements to verify applicable clauses before composing the text description. The evaluation measures target fidelity, tests sensitivity to predefined input conversions, and separates generalization to unseen looks from reproduction of seen looks on new photographs. Across 16 methods, stronger text alignment does not guarantee target fidelity. However, most supervised variants improve fidelity on both seen and unseen looks.

版本更新 · v2arXiv:2610.36509

ARISE: Authentic Reference-Informed Stability Enhancement for Continual Multimodal Forgery Detection

Yuchen Zhang, Yaxiong Wang, Yujiao Wu, Lianwei Wu, Qi Zhang, Li Zhu, Chaofeng Dong, Le Wu 等 9 位作者

原文摘要

Multimodal forgery detection is commonly studied under a static assumption: training attacks are expected to approximate those encountered after deployment. This assumption becomes increasingly fragile as generators, editing systems, and multimodal large language models continually introduce new forgery mechanisms and forensic evidence. We formulate this problem as attack-incremental continual multimodal forgery detection, where the binary real/fake interface remains fixed while emerging attacks must be learned without forgetting previous ones. We introduce Fable-Bench, which organizes attack evolution through complementary joint-modality, text-side, and image-side protocols. We further propose ARISE (Authentic Reference-Informed Stability Enhancement), motivated by the open-ended forged class and the comparatively reusable geometry of authentic content. ARISE fits a real-only Gaussian mixture model to projected authentic replay representations as a compact historical reference. Meanwhile, the proposed Gaussian-Responsibility-Aware Stability Preservation (GRASP) derives a sample-adaptive anisotropic metric that strongly constrains historically stable representation directions while retaining plasticity elsewhere. Our Vulnerability-Aware Replay (VAR) further estimates stage-wise vulnerability from authentic-density degradation, forged-to-authentic boundary confusion, and authentic-mode occupancy drift, then assigns bounded replay emphasis to fragile stages. All adaptive signals are obtained from training-time replay statistics, while inference uses only the original multimodal backbone and binary classification head. Experiments across three attack-evolution protocols and multiple detector architectures demonstrate improved historical retention and final average performance over representative continual-learning baselines.

cs.CV

新论文 · v1arXiv:2611.21973

How Many Persistent Features Does a Biometric Need? A Law Linking Equal Error Rate to Feature Count and Persistence

Oleg Komogortsev, Mehedi Hasan Raju

原文摘要

A biometric designer controls the number of features and their temporal persistence. Earlier empirical work linked the equal error rate (EER) to persistence and to the required number of features by straight lines. We derive the law behind these regularities. For $N$ independent standardized Gaussian features with test-retest correlation $ρ$, scored by cosine similarity, Fisher's $z$-transform turns the genuine and impostor scores into two normals, giving $\mathrm{EER}=Φ(-\frac{1}{2}\sqrt{N}\,\operatorname{atanh}ρ)$ and, inversely, $N=(2q/\operatorname{atanh}ρ)^2$ with $q=Φ^{-1}(1-\mathrm{EER})$; intercorrelated features enter through an effective count $N_{\mathrm{eff}}$. The central extension lets persistence vary across comparisons with spread $τ$ on Fisher's scale: $\mathrm{EER}=Φ(-m/(\sqrt{τ^2+v}+\sqrt{v}))$ with $v=1/(N_{\mathrm{eff}}-2)$, which for a normal spread has a floor $Φ(-m/τ)$ that no number of features can lower. With no fitted parameter the simple law reproduces published simulations and the EER-ICC line, and predicts the KCC-EER relation of eye-movement embeddings on GazeBase ($R^2=0.88$), though less well on GazeBaseVR ($0.63$). Read from the genuine and impostor scores, the general form matches the EER of eye movements, keystroke dynamics, the ECG and face embeddings over 101 conditions with $R^2=0.98$ and a median error of $0.6$ points, against $0.87$ and $1.9$ points for the simple law. The 61 eye-movement, keystroke and ECG conditions check consistency on the same scores; the 40 face conditions use held-out subjects, with the same accuracy. The product $τ\sqrt{N_{\mathrm{eff}}-2}$ gives a design rule: adding or decorrelating features pays fully while it is well below one and little once it exceeds about three, where only higher or more uniform persistence helps.

cs.CV

新论文 · v1arXiv:2611.21812

Sketch, Don't Paint: Allocating Test-Time Compute in Generative Visual Reasoning

Sheng Lu, Guangpin Tao, Tong Lu, Ying Tai

原文摘要

Multimodal models increasingly reason by synthesizing pixels, sketching maze paths within interleaved chains of thought or rolling out video solutions frame by frame. Each visual thought runs the full diffusion schedule inherited from photorealistic synthesis, so iterative denoising dominates inference cost. We study how to allocate test-time compute along two axes: when to render visual thoughts rather than thinking in text or answering directly, and how many denoising steps each thought requires. Controlled interventions that vary only how the thought is rendered uncover three regimes. Causal thoughts carry the plan and become legible early: an 8-step sketch nearly matches a 50-step render in accuracy, commits to the same plan as often as two independent full renders agree, and in video models mostly fails by stalling short of the goal, so the rollout abstains. Decorative thoughts can be omitted without loss, whereas harmful thoughts lower accuracy. Because the regime depends on both model and task, always drawing is suboptimal. Across 14 model-benchmark pairs of three interleaved reasoners, answering directly, without any thinking, wins 9 pairs, and choosing among direct, text-only, and visual tiers from ten labeled examples per pair raises mean accuracy from 60.7% (always drawing) to 65.1%. Where drawing pays, we propose sketch-and-vote, which samples 8-step sketches (a step count chosen without labels) from independent noises until one answer reaches three votes. With this one setting, sketch-and-vote beats full rendering with fewer denoising steps on an image reasoner and two video reasoners, by 8.6 points on VSP spatial planning and by 9.6 and 22.1 points on mazes.

cs.CV

新论文 · v1arXiv:2611.21283

PLECTA: Overlap-aware instance segmentation of dense strand networks from binary masks

Oday Allan, Igor Ostanin

原文摘要

Many materials and tissues, including carbon-nanotube sheets, collagen and paper, comprise overlapping strands whose arrangement influences their effective properties. Measuring this geometry from micrographs requires individual strands to be traced through crossings and signal gaps, which merge distinct strands or split one strand into disconnected fragments. We present PLECTA (from Latin plecta, a braid), a deterministic method that segments individual strands using only a binary image whose foreground pixels mark strand centrelines. At each junction, PLECTA selects which strand segments continue into one another while allowing segments to terminate. It formulates this decision as an exact maximum-weight partial-matching problem, with an explicit penalty for unpaired segments and no restriction on the number of segments entering a junction. The output permits crossing pixels to belong to multiple strands. On synthetic scenes, PLECTA resolved strand identities more accurately than three released alternatives at every density tested. The same ranking held on an independently published generator run at its own settings. A separate stage uses the corresponding greyscale micrograph to estimate which strand lies on top at each crossing. The complete workflow is demonstrated on scanning electron micrographs of carbon nanotube sheets, from raw images to individual strand instances with estimated crossing order, without manual annotations for training or as pipeline inputs.

cs.CV · cond-mat.soft

新论文 · v1arXiv:2611.20841

Vision-Language Model-Based Phase-OTDR Event Classification

Ahmet Bestami Kose, Mehmet Sergen Catal, Muhammet Cagri Yeke, Kivilcim Yuksel, Abdurrahman Gumus

原文摘要

Phase-sensitive optical time-domain reflectometry (Phase-OTDR) enables distributed vibration sensing, but reliable event classification has traditionally relied on fully supervised models that require extensive annotated data. This study examines the adaptability of a Vision-Language Model (VLM) for Phase-OTDR event recognition through few-shot learning. Raw samples are converted into Short-Time Fourier Transform (STFT) dB-magnitude spectrogram mosaics, color-mapped and resized to RGB images, and processed by a Contrastive Language-Image Pre-training (CLIP) image encoder; few-shot adaptation optimizes only the Tip-Adapter-F cache keys, leaving both encoders unchanged. Under a leakage-controlled protocol (train-side support/validation pool of 512 images per class, fixed held-out test of 120 per class, no test-set access during selection, 20 distinct support-sampling trials), the selected configuration (the ViT-L-14 Vision Transformer encoder with OpenAI weights, 128 shots per class) reaches 96.81 +/- 0.42% accuracy and a macro-averaged F1 score of 96.79 +/- 0.42% on the held-out test set; a label-efficient 16-shot setting, that is, 96 labeled support images at adaptation time across the six classes, already attains 90.17 +/- 0.81%. Tip-Adapter-F clearly outperforms both zero-shot CLIP and the standard training-free Tip-Adapter, and prompt and backbone analyses show that performance is stable across Vision Transformer encoders and insensitive to prompt wording at the selected operating point. Overall, CLIP visual features carry a transferable structure that a lightweight support cache turns into a label-efficient Phase-OTDR event classifier. The code and the corresponding image dataset are made publicly available on GitHub to support further research: https://github.com/miralab-ai/vlm-phase-otdr.

cs.CV · cs.LG

新论文 · v1arXiv:2611.20786

ThermoSight: From Appearance to Internal Thermofluid Fields via Physics-Guided Neural Reconstruction

Yuhao Fang, Chenglin Lu, Hefei Ling, Chun Lou

原文摘要

Most neural reconstruction methods focus on reproducing scene appearance, yet inferring physical states from visual observations is essential for machine understanding of the physical world. Reconstructing the internal temperature and soot fields of radiating thermofluids such as flames is challenging yet essential for improving energy efficiency and assessing heat exposure and fire spread. In flames, temperature and soot concentration jointly govern visible emission, and soot self-absorption alters the light that reaches each camera. With limited views, changes in the two fields can offset one another in the recorded RGB images, making temperature and soot difficult to disentangle. ThermoSight, a physics-guided neural framework for scientific imaging, jointly reconstructs continuous three-dimensional temperature and soot volume fraction fields from limited calibrated flame images without field-level ground truth during optimization. A differentiable spectral model links these fields to measured RGB through thermal emission, self-absorption, and calibrated camera responses. Temperature-dependent emission spectra and wavelength-dependent attenuation constrain shared fields across RGB channels, reducing temperature-soot ambiguity. Chromaticity consistency, observation-derived spatial constraints, and soot regularization complement intensity fitting. In a synthetic ablation, wavelength-resolved attenuation reduces temperature RMSE in the hot, sooty region from 234 to 112 K relative to a three-channel approximation, with nearly identical RGB PSNR (65.23 vs. 65.24 dB). Measured data yield coherent fields and competitive RGB reconstructions. These results demonstrate that physics-guided neural fields recover interpretable internal physical quantities beyond visually consistent appearance.

cs.CV · physics.app-ph

新论文 · v1arXiv:2611.20769

Deep Learning for Satellite-Based Mountain Hazard Screening: A Retrospective Failure Analysis on the 2021 Chamoli Rock-Ice Avalanche

Snigdhajyoti Rakshit

原文摘要

The approach utilizes 10 bands from Sentinel-2 imagery, along with terrain information, in a deep learning pipeline to classify potential hazard sources in any given mountainous region. The performance of our model has been evaluated against the data collected during the rock-ice avalanche event in the Chamoli region in the Indian Himalayas on 7th February 2021. U-Net performs segmentation of the detected land surface in the satellite image, detects changes in the identify candidate hazard zones, and then the following steps are performed to calculate the potential energy using the Monte Carlo sampling for steepest descent (D8) flow paths, finding the exposed structures in the downstream locations, and ranking the zones depending on the risk score. In comparison to the simple spectral rule, our approach achieved the higher macro F1 score of 0.648 (as compared to 0.519). But the F1 score in case of rock and soil was low, being in the range of 0.3s. In case of optical change detection in winter, it's only that northeast-facing slope which couldn't be detected, as it was under 73% of shade prior to the incident. The parameters have been set for this particular scene alone and not independently; hence, this is a retrospective test, and this test locates a failure which has occurred earlier. The primary contribution of this paper is locating the pipeline leak.

cs.CV

新论文 · v1arXiv:2611.20767

UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

Tao Tao, Anke Tang, Enneng Yang, Zhi Hou, Xiaogang Wang, Dacheng Tao

原文摘要

Embodied capabilities such as visual grounding, spatial reasoning, navigation, and driving understanding are often developed in separate vision-language specialists fine-tuned from a common pretrained model.Deploying the specialists separately multiplies storage, whereas merging them into one dense model forces conflicting task updates to share parameters and loses part of each specialist's expertise.We introduce UNITE, a framework that consolidates such specialists into a compact shared-backbone mixture of experts. UNITE keeps the feed-forward block of each specialist as a routable expert and represents the expert bank through a shared-anchor one-bit factorization: a shared floating-point anchor plus expert-specific, row-scaled binary deviations.The anchor is optimized jointly with the scales and signs through closed-form alternating updates, using only source weights and no calibration data.Router adaptation then adjusts token-level expert selection with the backbone and experts frozen, updating fewer than $0.6$M parameters ($<0.01\%$ of the $8.8$B active parameters). Built from four Qwen3-VL-8B specialists, the complete UNITE model occupies $20.3$ GB, $1.16\times$ the storage of one dense specialist and $59.6\%$ below the uncompressed MoE. Across $22$ benchmarks and R2R Val-Unseen, UNITE matches six static merging methods in mean retention of best-specialist performance, leads all of them on Scan2Cap, ScanQA, and all four navigation metrics, and surpasses every source specialist on all five 3D scene understanding benchmarks (up to $+19.1$ on Scan2Cap) and on R2R navigation. Relative to the uncompressed MoE, factorization is nearly lossless on $21$ of $22$ benchmarks; the exception is VSI-Bench, where frame-count experiments suggest impaired use of cross-frame evidence.UNITE thus consolidates complementary embodied capabilities near single-model storage.

cs.LG · cs.CV

版本更新 · v4arXiv:2611.12657

When Does Sliced Inference Pay Off? Resolution, Tiling and Dense Fusion for Small Object Detection

Stefan Becker, Jens Bayer, David Münch, Wolfgang Hübner

原文摘要

Sliced inference such as SAHI is the standard remedy for small objects in high-resolution imagery, but its gains are usually measured against low-resolution baselines. We revisit sliced inference with scale as the explicit variable and ask when tiling actually pays off. We compare nine dense detectors, eight YOLO models of four generations and FCOS, on VisDrone and DOTA, under pre-specified hypotheses, calibration-only tuning and a single evaluation on held-out test data. Two findings emerge. First, much of the benefit attributed to slicing is a resolution effect: on VisDrone, a single pass trained at a higher resolution outperforms tiling with box fusion (merging decoded boxes) for every detector that could be trained at that resolution, and at equal compute a higher resolution helps more than a larger model. Second, tiles carry complementary information that box fusion wastes. We introduce SCAHI (Slicing with Coverage-Aware Hyper Inference), which accumulates the dense outputs of all passes with weights that give every image position the same total coverage, and reads objects off a peak grid fine enough to keep crowded neighbours apart. On identical passes, SCAHI outperforms box fusion for every YOLO detector on both datasets and matches it for FCOS. On DOTA, whose images far exceed the resolution a single pass can be trained at, tiles with SCAHI beat the best single pass for every detector, by 4.3 to 5.6 mAP50 at 2048 px, and replacing only SAHI's merge step by SCAHI adds 2.3 to 4.0 mAP50. An ablation attributes the gain to the border weighting and the resolution of the peak grid. Source code is available at https://github.com/beckerio/scahi.

cs.CV

版本更新 · v2arXiv:2611.16408

Gravity and Flight: Learning to Navigate Fine-Grained Remote Sensing Scenes

Yuheng Zong, Minghua Wang, Xin Zhao, Xiaobin Liu, Yongjian Deng, Chanho Eom

原文摘要

Vision-language models are increasingly capable, yet their reasoning in complex remote sensing scenes can remain coarse-grained and insufficiently grounded in task-relevant visual evidence. We propose Gravity and Flight (G&F), a framework for scene specialization with limited supervision that separates visual and reasoning verification while coupling policy updates. It combines Evidence-of-Thought supervised fine-tuning (EoT-SFT) and Relational Transfer Policy Optimization (RTPO). EoT-SFT teaches fine-grained observation and faithful reasoning through description, analysis, and supported conclusions. RTPO independently samples responses to scene-matched image pairs with fine-grained visual differences under a shared task. Its rubric-based verification combines Gravity, which checks visual fidelity and description coverage, and Flight, which takes descriptions as premises to assess evidence use and relations between paired judgments. Their scores and task correctness yield a shared pair-level advantage that couples policy updates from both responses. Using 1,008 demonstrations and 500 training pairs for harbor specialization, G&F achieves a mean score of 69.12 across six evaluation sets, exceeding the backbone and supervised initialization by 9.76 and 4.56 points. Matched controls show gains over single-image and joint-input RL, and removing either verification view reduces performance. Applying G&F to urban specialization also improves on the base model and supervised initialization across three evaluation subsets. Project details and updates are available at https://github.com/Z0ngL1ng/Gravity-and-Flight.

cs.CV

新论文 · v1arXiv:2611.17334

拆解多模态视觉编码

Unbundling Multimodal Vision Encoding

Ellis Brown, Chris Hoang, Shengyi Qian, Shang-Wen Li, Mike Lewis, Marjan Ghazvininejad, Luke Zettlemoyer, Lucas Beyer 等 12 位作者

AI 速读 基于摘要

在统一数据与匹配总计算量的受控框架下,作者报告视觉先验并非必需,真正关键的是专用视觉容量,且该容量可置于多模态融合早期而非融合前。

cs.CV · cs.LG

查看原文摘要

How multimodal language models should encode visual input remains up for debate. Most models use a pretrained vision encoder, while some recent recipes train the encoder from scratch with the language model or remove it entirely. Vision encoders bundle together three design choices: a visual prior from upstream pretraining, dedicated vision capacity, and pre-fusion placement of that capacity (i.e., before the multimodal backbone). Because published models also differ in data, architecture, and compute, comparing them cannot isolate the effect of each choice. We disentangle these choices via a controlled framework where we pretrain all models from scratch on common data and compare them on visual question answering and language benchmarks at matched lifetime compute. We find that the visual prior is not necessary; an encoder trained jointly from scratch is on par with a pretrained one, without a separate training stage. What matters instead is dedicated vision capacity: an encoder-free model trails on visual understanding even when its backbone is enlarged by the encoder's parameter count. Yet that capacity need not precede multimodal fusion, as vision-only modules added in-fusion to early backbone layers perform comparably to a pre-fusion encoder and can lower time to first token. These results also hold across backbone sizes, when encoder pretraining is treated as free, and when starting from a pretrained language model. Overall, our findings favor dedicated vision capacity trained jointly with the backbone and placed early within it.

版本更新 · v2arXiv:2608.07314

TEMPO:面向视觉-语言-动作模型的语义-动作解耦强化学习后训练

TEMPO: Semantic-Action Decoupled RL Post-Training for Vision-Language-Action Models

Ziheng Liu, Quantao Yang

AI 速读 基于摘要

TEMPO 提出一种语义与动作解耦的双时间尺度强化学习后训练框架,冻结预训练视觉-语言主干,分别以不同频率更新语义投影层和动作专家,以缓解潜在动作漂移并提升策略性能。

cs.RO · cs.CV

查看原文摘要

Vision-language-action (VLA) models are commonly adapted to downstream manipulation tasks via supervised fine-tuning (SFT) or online reinforcement learning(RL) post-training. SFT is prone to distribution mismatch, and existing RL approaches typically optimize all trainable components within a single RL loop at a uniform update rate, although these components play distinct functional roles. We propose TEMPO, a semantic-action decoupled, two-timescale RL post-training framework for VLA models. TEMPO freezes the pretrained vision-language backbone and adapts two components with dedicated RL loops: the semantic projection layer, which produces the latent action conditioning the action expert, and the low-level action expert itself. The semantic projection layer is updated infrequently and the action expert frequently, which aims to limit latent-action drift while allowing the action expert to learn from online feedback. Our analyses show that, compared with equal update frequencies, less frequent semantic updates reduce latent-action drift and improve policy performance. Experiments on the CALVIN benchmark and real-world manipulation tasks demonstrate that TEMPO outperforms both pretrained state-of-the-art VLA models and the RL post-training baseline, while reaching and maintaining higher evaluation rewards on three real-world tasks.

新论文 · v1arXiv:2611.16061

Pothole Risk Assessment with Geometrically Grounded Vision Language Models

Abdalmalek Aburaddaha, Zaid A. El Shair, Ali Al-Bustami, Samir A. Rawashdeh

原文摘要

Potholes impose billions of dollars in annual vehicle damage, yet detectors report only where potholes are, not how severe they are or what to do. Judging severity from a single RGB image is ill-posed, and ungrounded vision-language model (VLM) assessments tend to be over-conservative. We present a five-stage modular framework for the offline assessment of already-detected potholes that combines geometric measurements with VLM reasoning: perspective transformation, zero-shot SAM segmentation, geometric feature extraction, Gemini 2.5 Pro severity classification with composite risk scoring, and natural-language warning generation. We evaluate it on 3,092 potholes in 1,119 dashcam scenes against independent references. Severity agreement with a prediction-free three-annotator panel is weak (Cohen's $κ= 0.139$, 95% CI [0.00, 0.28]; $n = 101$) and far below the annotators' agreement with one another ($κ= 0.605$); against physically measured depth on an independent RGB-D dataset, it is also weak ($κ= 0.098$), although the predicted classes preserve the order of measured depths. The severity classes are therefore perceived visual categories, not depth measurements. Geometric grounding does change the warnings: relative to an image-only baseline, it moves 26.7% of scenes out of the high-urgency tier and raises critical-tier assignments by 109% (Stuart-Maxwell $p < 10^{-15}$), and in a blind study, three raters preferred the grounded warnings 3.5:1. A six-arm prompt ablation attributes the hybrid prompt's rule adherence to the per-pothole measurements rather than to the rubric or worked examples. At USD 0.105 per scene, it suits large-scale offline infrastructure assessment; its maneuver suggestions are advisory research outputs, not validated vehicle guidance.

cs.CV

新论文 · v1arXiv:2611.15610

Contrastive Bootstrap Representation Learning for Label-Efficient Agricultural Image Classification

Hazem Ahmed, Ali Hamdi, Ammar Mohammed

原文摘要

Expert annotation remains a major bottleneck in agricultural image classification, where collecting images is often easier than obtaining reliable labels from domain specialists. Self-supervised learning can reduce this dependence by pretraining transferable representations on unlabelled imagery. Contrastive and bootstrap methods provide two established but distinct training signals, motivating an empirical study of a formulation that applies both signals within one representation path under severe downstream label scarcity. We present Contrastive Bootstrap Representation Learning (CBRL), which jointly optimises SimCLR-style NT-Xent contrastive learning and bidirectional BYOL-style exponential-moving-average target prediction within a shared online projection space. Using a ViT-L/16 backbone and 10% of the downstream labels, we report single-seed point estimates on four primary agricultural test benchmarks plus an exploratory 33-image New Plant Diseases evaluation. CBRL records the highest macro-F1 point estimate on the four primary benchmarks. On DeepWeeds, its 79.77% test macro-F1 is 32.29 points above the random-initialisation, label-only baseline and 15.78 points above the next-highest comparison point estimate; on PlantVillage, its 98.14% macro-F1 is 10.43 and 0.43 points higher, respectively. On PlantDoc, however, all methods fail to reach practically useful performance. These results identify the evaluated joint formulation as a promising candidate for label-limited transfer, but they do not establish objective synergy, statistical superiority, or robustness across stochastic runs.

cs.CV · cs.AI · cs.LG

版本更新 · v2arXiv:2609.34480

When Does an Image Determine the Answer? Benchmarking Visual Answerability across Charts and Scenes

Sungguk Cha, Mintae Kim, Youngsub Han, Byoung-Ki Jeon, Sangyeob Lee

原文摘要

Reliable visual question answering requires correct answers when evidence is sufficient and abstention when it is not. We introduce a benchmark of 3,000 question groups: 1,000 each from PlotQA charts, CLEVR rendered scenes, and GQA photographs. Each group keeps the question fixed across original, answer-preserving, and evidence-removing images; charts and CLEVR also include answer-changing edits. Models receive each image independently. A group succeeds only if every supported answer and every required abstention is correct. For charts and CLEVR, executable witnesses establish ambiguity under the declared task semantics: two admissible complete charts or scenes give different answers but yield the same masked observation. GQA scoring compares source labels with a permissive rule that also credits masked-image answers compatible with the full-image reference. Across 72,000 responses from six model configurations, the best complete-question success is 57.0% on charts, 43.5% on CLEVR, and 33.7% on GQA under source labels. On charts, the strongest configuration makes correct answer-or-abstain decisions on 96.2% of individual images and on all five images of 83.5% of questions. Requiring correct answers lowers success to 57.0%: in 26.5% of all chart questions, every decision is correct but at least one answer is wrong. Adding answer correctness also reverses a chart ranking based on decision accuracy. Evaluating all response obligations together reveals failures that per-image decision accuracy conceals.

cs.CV

版本更新 · v3arXiv:2609.13504

RIGOR: Rig-Informed Geometry for Omnidirectional Reconstruction

Tingjun Huang, Dmitry Rudshin, Mathieu Meyer, Pietro Bonazzi, Marc Pollefeys, Emilia Szymanska

原文摘要

Recent developments in feed-forward 3D reconstruction resulted in models which can recover dense scene representations and camera motion solely from an image stream. However, such predictions are prone to becoming inconsistent over long trajectories, specifically in demanding environments with repetitive structures, weak textures and dynamic objects or people. One way to mitigate those challenges is to use an omnidirectional camera, which provides wide spatial coverage and captures richer visual information. Yet, the majority of models do not offer support for 360-degree imagery or require additional fine-tuning. To bridge these two aspects, we present RIGOR: a large-scale reconstruction pipeline for gravity-aligned omnidirectional videos that retains a frozen feed-forward perspective backbone and exploits each panorama as a four-view virtual rig. The rig structure is used to detect and repair locally inconsistent predictions, to retrieve loop closures through cyclic four-view consensus, and to geometrically verify candidate revisits before global optimization. Verified constraints drive a Sim(3) pose graph that corrects accumulated rotation, translation, and scale drift along the sequence. We demonstrate that the proposed consistency mechanisms improve both trajectory accuracy and reconstructed geometry over a feed-forward baseline on challenging construction-site sequences. The code is made available under this link: https://github.com/TangentH/RIGOR.

cs.CV

新论文 · v1arXiv:2611.15149

From Concepts to Interactions: Dynamic Graph Semantic Recomposition for Safe Visual Generation

Qinghui Gong, Jinshan Lai, Xunlei Chen, Hua Meng, Zhengchun Zhou

原文摘要

Recent advances in flow-matching Transformers have greatly improved Text-to-Image (T2I) and Text-to-Video (T2V) generation, but unconstrained generation risks producing illegal or unethical content. Safety alignment must therefore suppress unsafe content while preserving legitimate generation. A concept's safety depends on context: a knife used for cooking is safe, whereas its use in violence is harmful. Yet most concept erasure methods indiscriminately suppress static, isolated representations, overlooking evolving semantic interactions. Consequently, safe scenes containing potentially sensitive terms may be unintentionally suppressed, compromising generation quality. To address this, we propose Dynamic Graph Semantic Recomposition (DGSR), which dynamically constructs and updates a semantic interaction graph from evolving text visual states during sampling. Context aware graph reasoning captures semantic dependencies, while paired safe/unsafe supervision distinguishes harmful from legitimate interactions, enabling precise identification of and selective intervention on high risk semantic relationships while preserving benign semantics. DGSR freezes the pretrained backbone and supports both image and video generation. Experiments show that DGSR achieves the lowest Attack Success Rate (ASR) in 10 of 12 T2I evaluations and reduces the average T2V unsafe frame rate by 41.2\% relative to the strongest baseline, while maintaining high generation fidelity and temporal consistency.

cs.CV

版本更新 · v3arXiv:2603.22570

CanViT: Toward Active-Vision Foundation Models

Yohaï-Eliel Berreby, Sabrina Du, Audrey Durand, B. Suresh Krishna

原文摘要

Active computer vision promises efficient, biologically plausible perception through sequential, localized glimpses, but lacks scalable general-purpose architectures and pretraining pipelines, leaving Active-Vision Foundation Models (AVFMs) underexplored. We introduce the Canvas Vision Transformer (CanViT), the first task- and policy-agnostic AVFM. CanViT uses scene-relative RoPE to bind a retinotopic Vision Transformer backbone and a spatiotopic scene-wide latent workspace, the canvas. Efficient interaction with this high-capacity working memory is supported by Canvas Attention, a novel asymmetric cross-attention mechanism. We decouple thinking (backbone-level) and memory (canvas-level), eliminating canvas-side self-attention and fully-connected layers to achieve fast sequential inference and scalability to high output resolutions. We propose a label-free active vision pretraining scheme, policy-agnostic passive-to-active dense latent distillation: reconstructing scene-wide DINOv3 embeddings from sequences of low-resolution glimpses with randomized locations, zoom levels, and lengths. We pretrain CanViT-B from a random initialization on 13.2 million ImageNet-21k scenes and 1 billion random glimpses, in 166 hours on a single H100. On ADE20K segmentation, a frozen CanViT-B achieves 38.5% mIoU in a single low-resolution glimpse, outperforming the best active model's 27.6% with 20x fewer inference FLOPs as well as its FLOP- or input-matched DINOv3 teacher. Given additional glimpses, CanViT-B reaches 45.9% ADE20K mIoU. On ImageNet-1k classification, CanViT-B also sets a new active-vision state of the art, with 84.5% top-1 accuracy after fine-tuning. CanViT generalizes to longer rollouts, larger scenes, and new policies. Our work narrows the wide gap between passive and active computer vision, demonstrating the potential of task- and policy-agnostic AVFM pretraining.

cs.CV

版本更新 · v2arXiv:2609.04726

Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation

Siyuan Liu, Xuze Zhang, Yongshun Wang, Licong Pan, Hang Liu, Huihui Li

原文摘要

Synthetic Aperture Radar (SAR) remains available through cloud and darkness, yet speckle and non-intuitive scattering make it hard to inspect and interpret. SAR-to-optical (S2O) translation renders radar observations as optical-like images for visual access, while semantic segmentation on SAR remains necessary since land-cover decisions cannot rely on synthesised pixels. We propose BMT, a hierarchical Vision Transformer coupling both tasks on a shared encoder via multi-scale one-hot conditioning, a gated LocalViTBlock, a colour-calibrating generation head, ControlNet-style zero-initialized injection, and uncertainty-based task weighting. The joint framework's value is not label-free training: it uses paired optical images and the same pixel annotations as a dedicated SAR segmentor, supervising the SAR decoder and teacher-forcing the generator with ground-truth one-hot maps. At test time, the model takes only SAR and returns an optical-like readout and a radar-decided land-cover map, with the translation objective supplying dense appearance supervision that speckle lacks. On WHU-OPT-SAR, the predicted-condition model achieves the best SSIM and PSNR among compared translators; its land-cover map surpasses the same backbone trained for labelling alone and is within about one point of DeepLabV3+.

cs.CV

新论文 · v1arXiv:2611.14365

从视频帧到机器人元帧:面向世界动作模型的统一潜在接口

From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models

Guoqiang Liang, Yiqi Lin, Yanzhe Chen, Kevin Yuchen Ma, Mike Zheng Shou

AI 速读 基于摘要

作者提出 MetaWAM,将机器人各类低维信号统一折叠进一个频域元帧,以对齐预训练视频扩散模型的潜在先验,并在仿真与真实机器人任务上报告了性能与效率结果。

cs.CV · cs.RO

查看原文摘要

Pretrained video diffusion models (VDMs) offer a powerful foundation for World Action Models (WAMs), which jointly predict robot actions and future observations. Building such models requires injecting low-dimensional robot signals (e.g. proprioception, actions, and values) into a latent space shaped by natural videos, and a key design choice is the interface through which these signals enter: existing schemes treat it as a formatting detail, spending a separate latent frame on each signal or bolting on dedicated encoders and decoders. We argue that signal injection is instead a distribution-alignment problem: how well a representation matches the prior of the pretrained VDMs during denoising. Using the relative reconstruction error of a frozen VDM denoiser as a probe, we find that conventional spatial encodings sit far off this prior, whereas frequency-domain packing yields latents that the denoiser corrects far less. Motivated by this diagnostic result, we propose MetaWAM, which folds all robot signals into a single unified meta-frame: each signal is assigned to a disjoint frequency-domain channel band and lifted to the spatial latent by an inverse DCT, yielding a decodable interface aligned with the pretrained VDM prior without any added module or architectural change. A fixed band layout together with slot-aware routing lets the same meta-frame support policy, world-model, and value prediction over a shared set of robot variables. On RoboCasa, MetaWAM reaches 68.9% success while collapsing the per-signal frames into one and cutting inference latency by 39%, and attains 85.0% success on three real-world Franka tasks. Code and models are available at https://github.com/showlab/MetaWAM

新论文 · v1arXiv:2611.14058

CURRENT: A Recurrent Neural Network for Concurrent Spatial and Temporal Super-Resolution of 4D Flow MRI

Luuk Jacobs, Pietro Dirix, Sebastian Kozerke

原文摘要

Objective: To improve the spatial and temporal resolution of aortic 4D flow MRI through a unified deep learning framework for post-acquisition super-resolution (SR) of reconstructed velocity vector fields. Methods: We propose CURRENT, a recurrent neural network that performs spatial and temporal SR with a single network architecture. Supervised training was conducted using paired low- and high-resolution datasets generated by embedding computational fluid dynamics simulations of aortic flow into in vivo thoracic MRI backgrounds, followed by controlled spatial and temporal downsampling. CURRENT was evaluated for two-fold spatial and temporal SR against 4DFlowNet and for two-fold spatiotemporal SR against sequential spatial and temporal SR pipelines based on 4DFlowNet. Image-based and k-space-based quantitative metrics were considered and conventional interpolation were used as baselines. In addition, the impact of CURRENT's spatiotemporal SR on physics-informed relative pressure estimation was assessed. Finally, the ability of CURRENT to generalize from synthetic training data to in vivo acquisitions was investigated. Results: CURRENT achieved superior spatial SR performance and comparable temporal SR performance to dedicated 4DFlowNet models while providing a unified solution for both tasks. Simultaneous spatiotemporal SR using CURRENT yielded more accurate relative pressure estimates than either spatial or temporal SR alone and outperformed sequential 4DFlowNet-based spatial and temporal SR approaches. Furthermore, the proposed model demonstrated promising generalization to in vivo 4D flow MRI data despite being trained exclusively on synthetic datasets. Conclusion: CURRENT provides a unified framework for spatial and temporal SR of 4D flow MRI. The proposed approach has the potential to enhance physics-based analysis and broaden the applicability of accelerated 4D flow MRI.

cs.CV · physics.med-ph

新论文 · v1arXiv:2611.11831

End-to-End Joint Training of Super-Resolution and Compression Artifact Removal for Industrial Machine Vision

Dang BuiDinh, Sang NguyenQuang, Anh NguyenPhuong, Wen-Hsiao Peng, Xiem HoangVan

原文摘要

In industrial machine vision, images captured by edge devices are often downsampled and compressed to reduce bandwidth, introducing resolution loss and compression artifacts that severely degrade the performance of downstream tasks, such as defect detection. Existing quality enhancement methods typically apply super-resolution (SR) or compression artifact removal (CAR) independently or in a sequential manner and are largely optimized for human visual fidelity rather than machine perception. This paper proposes an end-to-end quality enhancement framework that integrates SR and CAR with a unified training strategy that optimizes both human-perception and machine-perception objectives. We investigate fixed-weight and uncertainty-based dynamic-weight loss functions to balance reconstruction quality and machine vision task performance. Extensive experiments on PCB Defect, DeepPCB, and NEU-DET datasets show that the proposed jointly trained model generally outperforms standalone and separately trained SR-CAR pipelines. In particular, under severe JPEG compression, our method improves defect detection performance from 0.461 to 0.887 mAP@50 on PCB Defect over sequential SR-CAR baselines, with only a slight reduction in PSNR. These results demonstrate that joint optimization is critical for compression-aware industrial machine vision systems.

cs.CV

新论文 · v1arXiv:2611.11087

SoftPAT3D: Physics-Augmented Text-to-3D Scene Generation with Deformables

Lei Shu, Alberta Longhini, Leonidas Guibas, Gordon Wetzstein, Jeannette Bohg

原文摘要

We present SoftPAT3D, a physics-augmented framework for text-to-3D scene generation in which deformable objects, such as cloth, stuffed toys and ropes, are generated as physically valid, simulation-ready bodies and placed in intersection-free contact with the furniture that supports them, with one another and with themselves. Existing physics-augmented pipelines simulate rigid bodies only and cannot accommodate a generated deformable object, which text-to-3D generation returns as a geometrically invalid mesh, sometimes self-intersecting, that has no rest shape, no material model and no admissible initial state. These are the inputs a deformable simulation requires, and we construct each from information already in the pipeline. The rest shape comes from how the object is manufactured rather than from the generated physically inconsistent deformed configuration, and a material library assigns each deformable object its initial elastic parameters, which can then be optimized. The occluded support is regenerated and converted into a watertight collision mesh, each object is released from an optimizable placement above the supporting part that the prompt specifies, and the complete scene is driven to static equilibrium under one unified contact model, with completion verified by geometric checks and a vision-language model. When a scene fails the prompt, a grid search over placements corrects it, and the adjoint of a differentiable simulation of the same contact energy reaches the targets the grid does not, places several bodies jointly and recovers and optimizes material parameters. Multi-body scenes equilibrate in a single simulation, and the pipeline also runs from the text prompt alone. On all sixty-five prompts, SoftPAT3D produces no undesired interpenetration of any kind, whereas every other system does, and its scenes directly support scene editing and robotic manipulation.

cs.CV · cs.GR · cs.RO

新论文 · v1arXiv:2611.10672

Heterogeneous Asymmetric Contrastive Distillation

Qi Wang, Jinjia Zhou

原文摘要

Knowledge distillation (KD) between architectural families, e.g., from a Transformer teacher to a CNN student, is difficult because the two networks learn different representations. Logits share a label space, but heterogeneous teachers differ in logit offset, scale, and sharpness, and their non-target classes are less semantically organized. Conventional logit distillation matches these values sample by sample, asking the student to imitate how another architecture expresses its predictions. We argue that a heterogeneous teacher should pass on the relational structure of its predictions, i.e., how it arranges samples relative to one another, and propose Heterogeneous Asymmetric Contrastive Distillation (HACD) to transfer this structure. HACD maps teacher and student logits to centered, temperature-softened probability directions on a shared unit hypersphere, which removes architecture-dependent offsets and lets the temperature control how sharply peaked outputs enter the representation while keeping informative non-target classes. Each student output must then identify its own teacher target within the batch. Because every teacher target also acts as a negative for the others, HACD computes the targets on a moderately augmented view and contrasts both a moderate and a strong student view against them. HACD adds no learnable parameters and needs one teacher pass per sample. It achieves the best or second-best accuracy on 11 of 12 heterogeneous pairs on CIFAR-100, without ground-truth labels, and on 14 of 15 pairs on ImageNet-1K, improving over the reported vanilla KD results by 8.82 and 1.33 percentage points on average. Checkpoints are available at https://github.com/BTAEYANG/HACD-Heterogeneous-Asymmetric-Contrastive-Distillation; the code will be released upon publication.

cs.CV

版本更新 · v3arXiv:2609.03080

Exemplar:数秒内完成的小样本生物医学分割

Exemplar: Few-Shot Biomedical Segmentation That Fits in Seconds

Michal Průšek, Adam Novozámský, Filip Šroubek

AI 速读 基于摘要

Exemplar 用闭式解拟合线性分割读出层,结合冻结的 DINOv3 特征与经典滤波响应,在数秒内完成小样本生物医学分割,并随新增标注精确更新。

cs.CV

查看原文摘要

Segmenting a new microscopy dataset usually means training a deep network on its annotations, and training again for every added image. Exemplar solves a segmentation readout in closed form and updates it exactly with each new mask in seconds, as accurately as networks trained for minutes. The updated readout is identical to a fit from all masks at once. Its linear readout combines frozen DINOv3 features, read at a magnification chosen from the masks, with classical filter responses at native resolution. Its regularisation and its pixel weighting are chosen exactly by leaving out one annotated image at a time, so none is set aside for validation. On ten biomedical datasets from cells to vessels and filaments, Exemplar is more accurate than every few-shot method that only applies a pretrained model, and on average than nnU-Net trained on the same masks. A fine-tuned Cellpose-SAM, the strongest baseline, is as accurate on average, but each of its fits takes a median of 25 minutes against 8.5 seconds, and each new mask means fine-tuning it again.

新论文 · v1arXiv:2611.09845

AdaptivePnP:用于反卷积的快速自适应调参即插即用方法

AdaptivePnP: Fast Auto-Tuned Plug-and-Play for Deconvolution

Haley Duba-Sullivan, Emma J. Reid, Charles A. Bouman, Gregery T. Buzzard

AI 速读 基于摘要

提出 AdaptivePnP 框架,在每次迭代中从数据拟合残差联合估计 PnP 的步长 γ 与噪声水平 σ,从而免去网格搜索调参和已知噪声假设。

eess.IV · cs.CV

查看原文摘要

Plug-and-play (PnP) methods reconstruct images by alternating a data-fitting step with an image denoiser. Their performance depends on two parameters: the step size $γ$ and the measurement noise level $σ$. In practice, $γ$ is often tuned by grid search over validation images and must be retuned whenever the problem setting changes, and $σ$ is assumed known or estimated separately. We propose AdaptivePnP, a PnP framework that jointly estimates $σ$ and $γ$ at every iteration from the data-fitting residual. To do so, we apply the PnP virtual model to the data-fitting proximal map and show that the residual is zero-mean Gaussian with covariance parameterized by $γ$ and $σ$. However, when one parameter is much larger than the other, the residual carries little information about the smaller one and using the maximum likelihood (ML) estimator can degrade the PnP iterations. We therefore introduce the identifiable maximum likelihood (IML) estimator, which restricts the ML estimator to an identifiable window in which the normalized Cramér--Rao bound of every parameter is bounded by a user-specified tolerance. For the virtual model, this window is an interval of ratios $ρ= γ/σ$, so computing the IML estimate reduces to a one-dimensional search over $ρ$. We apply AdaptivePnP to two standard PnP methods, yielding algorithms that require neither a tuned $γ$ nor a known $σ$. On image deblurring and super-resolution, AdaptivePnP matches or outperforms PnP baselines tuned with the true $σ$. Code is available at https://github.com/hdubasullivan/AdaptivePnP.

版本更新 · v3arXiv:2512.02458

Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Exploration

Zhongyi Cai, Yi Du, Chen Wang, Yu Kong

原文摘要

Embodied agents are expected to assist humans by actively exploring unknown environments and reasoning about spatial contexts. When deployed in real life, agents often face sequential tasks where each new task follows the completion of the previous one and may include infeasible objectives, such as searching for non-existent objects. However, most existing research focuses on isolated goals, overlooking the core challenge of sequential tasks: the ability to reuse spatial knowledge accumulated from previous explorations to guide subsequent reasoning and exploration. In this work, we investigate this underexplored yet practically significant embodied AI challenge. Specifically, we propose 3DSPMR, a 3D SPatial Memory Reasoning framework that utilizes Field-of-View (FoV) coverage as an explicit geometric prior. By integrating FoV-based constraints, 3DSPMR significantly enhances an agent's memory, reasoning, and exploration capabilities across sequential tasks. To facilitate research in this area, we further introduce SEER-Bench, a novel Sequential Embodied Exploration and Reasoning Benchmark that spans two foundational tasks: Embodied Question Answering (EQA) and Embodied Multi-modal Navigation (EMN). SEER-Bench uniquely incorporates both feasible and infeasible tasks to provide a rigorous and comprehensive evaluation of agent performance. Extensive experiments verify that 3DSPMR achieves substantial performance gains on both sequential EQA and EMN tasks.

cs.CV

版本更新 · v2arXiv:2604.08711

Deep Learning-Based Tracking and Lineage Reconstruction of Ligament Breakup

Vrushank Ahire, Vivek Kurumanghat, Mudasir Ganaie, Lipika Kabiraj

原文摘要

The disintegration of liquid sheets into ligaments and droplets involves highly transient, multi-scale dynamics that are complex to quantify from high-speed shadowgraphy images. Identification of droplets, ligaments, and blobs formed by the breakup of a liquid sheet, and their tracking from frame to frame, is an essential part of spray analysis. Analysis of such processes poses a fundamental challenge for conventional multi-object tracking frameworks, which enforce strict one-to-one temporal associations and cannot represent one-to-many fragmentation events. We present a two-stage deep learning framework that detects droplets and ligaments and identifies their temporal relationships across successive frames, modeling ligament deformation, fragmentation, and parent-child lineage. In the first stage, a Faster R-CNN detector with a ResNet-50 backbone and Feature Pyramid Network classifies ligaments and droplets in recordings of two impinging Carbopol gel jets. A morphology-preserving synthetic data generation strategy enriches the training set without physically implausible configurations, achieving a held-out test F1 score of up to 0.872 across fourteen original-to-synthetic data configurations, with robustness tested on different atomization conditions and liquid compositions. In the second stage, a Transformer-augmented multilayer perceptron classifies inter-frame object associations into continuation, fragmentation (one-to-many), and non-association using physics-informed geometric features. Despite severe class imbalance, the model achieves 86.1% accuracy, 93.2% precision, and perfect recall (1.00) for fragmentation events on the held-out test set. Together, the two stages preserve parent-child lineage and enable extraction of breakup statistics, including fragment multiplicity and droplet size distributions, offering an automated analysis of the primary mode of atomization.

cs.CV · cs.AI · cs.LG