让下一篇重要论文，不再错过。

新论文 · v1arXiv:2611.08262

更新于 2026-11-06

模拟视角的几何结构：在多模态语言模型中分离表征与行为

The Geometry of Simulated Perspectives: Dissociating Representation and Behaviour in a Multimodal Language Model

Mathis Immertreu, Fitim Abdullahu, Thomas Kinfe, Claus Metzner, Helmut Grabner, Patrick Krauss, Achim Schilling

AI 速读基于摘要

作者报告在 Qwen3-VL-8B-Instruct 中，人格提示的措辞比人格身份更强地改变内部表征，而人格内容对行为评分影响更大，呈现表征与行为的分离。

cs.AI · cs.CL · cs.CV

查看原文摘要

Multimodal language models can simulate different observers, yet it remains unclear how assigned perspectives reshape their internal representations and whether these changes govern their judgements. Here, we investigate the representational geometry of persona-conditioned visual interestingness in Qwen3-VL-8B-Instruct. Across 100 images and 53 prompting conditions, we systematically vary persona content and the linguistic framing used to introduce it, while recording behavioural ratings and layer-wise hidden states. We observe a systematic reorganisation of representational geometry across processing depth: early layers primarily encode image identity, whereas persona-specific structures emerge in later layers. Strikingly, prompt framing reshapes internal representations more strongly than persona identity, whereas persona content has a greater impact on behavioural judgements. This dissociation is further illustrated by minimal identity statements that alter representations even without changing responses. These findings demonstrate that representational distance does not necessarily reflect behavioural relevance. More broadly, they reveal how simulated perspectives emerge as distinct structures within the representational space of artificial cognitive systems, highlighting a fundamental distinction between the geometry of internal representations and their functional consequences.

新论文 · v1arXiv:2611.04438

更新于 2026-11-03

Entropy-Gated Dynamic Rectification for Object Hallucination in Large Vision-Language Models

Riasad Alvi, Md. Asaduzzaman Shuvo, Md. Ahashan Habib, Md. Faiyaz Abdullah Sayeedi

原文摘要

Large Vision-Language Models (LVLMs) routinely name objects that are absent from the image. Latent steering can reduce this failure in one model forward pass, but a fixed gate treats early and late decoding steps by the same criterion. We observe that predictive entropy varies sharply within free-running generation, while states that commit to short answers occupy a narrow, confident band. We exploit this temporal mismatch by using uncertainty as a causally available, token-local control signal for an existing steering gate. We propose Entropy-Gated Dynamic Rectification (EGDR), which lowers the intervention threshold when entropy derived from the preceding step's already available logits is high and retains the calibrated operating point when it is low. On LLaVA-1.5-7B, the underlying static gate lowers CHAIR CS/CI from 53.00/15.30 to 32.00/14.30; replacing only its constant threshold with EGDR further lowers them to 29.00/13.60. At a matched 58% intervention rate, the corresponding static control obtains 30.50/14.00. The gain over the static gate holds on LLaVA-NeXT-7B and grows with the generation budget, while POPE scores and general multimodal capability remain effectively unchanged. EGDR requires no additional model forward pass and adds roughly 2% measured latency. Our code is available at https://github.com/alvi-uiu/EGDR.

cs.CV

新论文 · v1arXiv:2611.03952

更新于 2026-11-03

Do We Still Need Rotation-Invariant Architectures? A Statistical Robustness Benchmark for Point Cloud Classification

Abdulla Nasir Chowdhury, Gulpi Qorik Oktagalu Pratamasunu, Kazuhiro Fukui

原文摘要

A substantial line of work builds rotation invariance directly into point cloud architectures. We benchmark 10 architectures (PointNet, PointNet++, DGCNN, RIConv, RIConv++, RISurConv, PointMLP, PointTransformer, and the equivariant VN-PointNet and VN-DGCNN) across 5 domains under pose, noise, and occlusion, and analyze the results with confirmatory hypothesis tests rather than point estimates. The three rotation-invariant (RI) architectures achieve the best pose robustness but rank among the lowest on noise robustness in 4 of 5 domains. We ask whether this cost is intrinsic to invariance or specific to how invariance is obtained. We retrain all 10 architectures unchanged, with a simple invariance loss on the embedding and three training seeds each, and evaluate all models on the independently designed ModelNet40-C. Invariance training transfers narrowly but reliably. It improves exactly the corruption families its perturbations target wherever the baseline has headroom (mean occlusion +10.5, lidar +13.4 percentage points of accuracy). At matched training budget the best standard backbone and the best RI architecture end on par on unseen corruptions (within 2.5 percentage points in every seed), and invariance-trained RIConv++ is the most seed-stable top model. Once rotation augmentation is widened, standard backbones concede only a flat, angle-independent accuracy offset under full-sphere rotation. We release the benchmark along with a package for analyzing point cloud robustness and reproducing our results.

cs.CV · cs.LG

新论文 · v1arXiv:2611.02744

更新于 2026-11-02

Subspace Representations for Synthetic-to-Real Transfer in 3-D Surface Inspection

Abdulla Nasir Chowdhury, Naish Ibrahim, Gulpi Qorik Oktagalu Pratamasunu, Kazuhiro Fukui

原文摘要

This paper studies what subspace representations contribute to synthetic-to-real transfer in 3-D surface inspection, where models learn from synthetic defects and the only real data before deployment are scans of good parts. We use three classical constructions as instruments: the nearest-subspace classifier, one-class template subspaces and the generalized difference subspace (GDS). On 1,197 MVTec 3D-AD scans without target annotation we find that every decision rule ranks real defects at a similar window AUC, that image-level detection stays at chance, and that no rule keeps its in-domain threshold. We show that the margin of a nearest-subspace rule moves by at most the part of the feature shift outside its class subspaces, and that this leakage, measured from unlabeled good scans, bounds the displacement observed on unseen scans. On 597 Real3D-AD objects, a one-class subspace of registered templates evaluated at every location on frozen features reaches point AUROC 0.957 on pre-registered scans and a category-mean object AUROC of 78 to 81 with no training on the benchmark. Our results suggest that a subspace helps under shift in proportion to the shift it spans, and that in reference-based inspection the reference and its registration matter more than the subspace.

cs.CV · cs.LG

版本更新 · v3arXiv:2605.02206

更新于 2026-11-01

Validity-Gated Evaluation of Multimodal Machine Unlearning

Abdullah Ahmad Khan, Hamid Laga, Ferdous Sohel

原文摘要

Machine unlearning aims to remove the influence of training data without retraining a model from scratch. Evaluation of vision-language models relies on forget accuracy (FA), retain accuracy (RA), membership inference, and similarity to a retain-only reference, but evaluation pipelines are rarely tested. We propose a validity-gated unlearning audit (VGUA) with three checks: the forget set must be memorized without duplicate leakage into the retain set, the scorer must measure correctness, and the training code must implement the intended procedure. Applying these checks reveals that random MMBench splits leak duplicate questions, a lenient scorer hides memorization, and two coding errors remove the signal while training completes. After correcting these problems, we evaluate 234 unlearned models using six methods, three learning rates, and three backbones. Each model is compared with a retain-only reference trained using low-rank adaptation (LoRA). The results depend on update strength. Among models with RA at or above 0.70, lower FA generally indicates closer agreement with the retrained FA-RA operating point. Aggressive updates overshoot this point, collapse RA, and reverse method rankings. Relative to the original model, 78% of healthy models move closer to the retrained model's scores, whereas 94% move farther from it in activation space. Membership-inference advantage does not reliably track retraining alignment. Under option-shuffled training, unlearned models answer 25% of reordered forgotten ScienceQA questions, compared with 6% for the retain-only reference. These results show that multimodal unlearning conclusions depend on pipeline validity, update strength, and the comparison criterion.

cs.CV · cs.LG

版本更新 · v3arXiv:2503.15625

更新于 2026-10-30

EarthScape：地理域偏移下地质填图与地表感知学习的多模态基准

EarthScape: A Multimodal Benchmark for Geologic Mapping and Surface-Aware Learning under Geographic Domain Shift

Matthew A. Massey, Nusrat Munia, Abdullah-Al-Zubaer Imran

AI 速读基于摘要

作者提出 EarthScape，一个融合航空影像、LiDAR 高程、多尺度地形衍生特征与上下文矢量数据的多模态基准，用于在地理域偏移下评估地质填图与地表感知学习。

cs.CV

查看原文摘要

Understanding natural Earth surface processes requires integrating topography, imagery, hydrology, and human infrastructure while remaining robust to geographic variation, yet few benchmarks support both challenges. We introduce EarthScape, a multimodal benchmark for geologic mapping and surface-aware learning under geographic domain shift. EarthScape combines aerial imagery, LiDAR-derived elevation, multi-scale terrain derivatives, and contextual vector data into 37 co-registered input channels per sample, together with pixel-level geologic masks. Its geographically separated regions share a common label space, enabling controlled evaluation of in-domain and cross-region generalization under realistic class imbalance and spatial complexity. Benchmark experiments show that engineered terrain representations transfer more robustly than raw elevation, while spatial support and multimodal terrain inputs strongly influence performance. Fusion effectiveness depends on backbone and evaluation domain, with more complex cross-attention providing no consistent advantage over simpler strategies. These representation trends extend across geospatial foundation models and semantic segmentation, while class-level results show that no single terrain feature or scale is optimal across geologic classes. Together, EarthScape provides a benchmark for studying representation design, multimodal fusion, and geographic robustness in surface-aware geospatial learning. Our source code is available at https://github.com/masseygeo/earthscape and the dataset at https://doi.org/10.13023/kgs.data.05.01.2025.

版本更新 · v2arXiv:2603.14271

更新于 2026-10-26

Adaptation of Foundation Models in Medical Image Analysis: Mechanisms, Evidence, and Deployment Considerations

Karma Phuntsho, Abdullah, Kyungmi Lee, Ickjai Lee, Euijoon Ahn

原文摘要

Foundation models, pretrained on diverse data, objectives, or modalities for reuse across downstream tasks, are increasingly applied to medical image analysis. Their performance depends not only on pretraining but also on how they are adapted to specific tasks, data regimes, and deployment settings. Whereas previous reviews have considered adaptation within broader syntheses, this review focuses specifically on post-pretraining interventions applied to existing FMs. Using a structured narrative review with a systematic, auditable workflow, we identified 719 eligible reports representing 705 independent studies. Of these, 473 provided controlled comparative evidence: 322 formally published studies formed the primary synthesis and 151 preprint-only studies a sensitivity layer. Adaptation was organized into five non-mutually exclusive families: parameter-level, representation-level, objective-level, data-centric, and architectural/sequence-level adaptation, with model use and update regime analyzed separately. Controlled comparisons showed no consistent ranking among frozen, parameter-efficient, partial, or full updating, or across broader adaptation families; outcomes varied by task, data regime, comparator, and evaluation setting. Clinical-readiness evidence remained limited: 173/705 studies (24.5\%) had confirmed independent external validation, 10 (1.4\%) reported calibration or uncertainty evidence, and 97/322 formally published comparative studies (30.1\%) included a qualifying externally validated arm. Overall, FM adaptation is a context-dependent design space rather than a hierarchy of preferred strategies.

cs.CV

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.1915ACL Anthology · ACL 2026

来源日期 2026-07-01（仅到月份）

Multimodal Safety Evaluation in Generative Agent Social Simulations

Alhim Adonai Vera Gonzalez, Carlos Hinojosa King Abdullah University of Science and Technology, Karen Sanchez King Abdullah University of Science and Technology, Haidar Bin Hamid University of Cincinnati, Donghoon Kim University of Cincinnati, Bernard Ghanem King Abdullah University of Science and Technology

原文摘要

Can generative agents be trusted in multimodal environments? Despite recent advances, agents remain limited in their ability to reason about safety, coherence, and trust across modalities. We introduce a reproducible simulation framework to evaluate generative agents in three aspects: (1) safety improvement over time via iterative plan revision in multimodal scenarios; (2) detection of unsafe activities across social contexts; and (3) social dynamics, measured through interaction and acceptance rates. These multimodal agents are evaluated using metrics that quantify plan revisions and unsafe-to-safe conversions. Experiments show that while agents detect direct multimodal contradictions, they often fail to align local revisions with global safety, achieving only a 55% success rate in correcting unsafe plans. We release a dataset of 1,000 multimodal plans, yielding more than 600,000 simulation steps. Notably, 45% of unsafe actions are accepted when paired with misleading visual cues, revealing a strong tendency to overtrust visual content. Code is available at https://github.com/AdonaiVera/X-CASE

cs.CL · cs.CV

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表

ACL Anthology · ACL 2026 ↗PDF ↗

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-srw.81ACL Anthology · ACL 2026

来源日期 2026-07-01（仅到月份）

Tonal Salience in Cognitive Decline: In-Context MCI Detection with Multimodal LLMs

Christopher Song, Abdullah P. Rashed Ahmed Brown University

原文摘要

Speech-based screening for mild cognitive impairment offers a highly accessible way to detect early cognitive decline. While most existing work focuses on English, cross-linguistic research is emerging to examine how cognitive decline manifests across languages. Studies on the Interspeech 2024 TAUKADIAL dataset, comprising English and Chinese speech recordings, consistently report higher classification performance on Chinese, yet the cause of this cross-lingual discrepancy remains unexplored. We examine this gap using Gemini 2.5 Pro, a multimodal large language model, using zero-shot and in-context-learning (ICL) paradigms. We hypothesize that this disparity is rooted in language typology: in tonal languages like Chinese, pitch encodes lexical meaning in every syllable (tone), whereas in non-tonal languages like English, pitch carries no lexical function. To test this, we pitch-flattened audio from TAUKADIAL and compared how classification performance changed across both languages. We found that Chinese classification degraded significantly under both zero-shot and ICL conditions (-4.78 and -5.92 UAR, respectively), while English performance increased (+0.11 and +2.98 UAR), implicating tonal pitch as the cross-lingual advantage. These findings suggest language typology should inform the design of audio-based cognitive screening tools, with raw audio preferred for tonal languages and text for non-tonal languages, a distinction critical for developing equitable cross-linguistic screening.

cs.CL · cs.CV

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop) · 2026 · 正式发表

ACL Anthology · ACL 2026 ↗PDF ↗

首次观测 · 记录 #1CVF · CVPR 2026:cvf:166a199006fedd020b9940ad10c87d3aCVF · CVPR 2026

来源日期 2026-01-01（仅到年份）

Prompt-Free Unknown Label Generation for Open World Detection in Remote Sensing

Azeem, Abdullah, Wang, Ruisheng, Li, Qingquan, Siddique, Abubakar

原文摘要

Autonomous object detection in remote sensing requires systems that can discover new categories and assign them usable labels during deployment. Existing Open-World Object Detectors identify unknown objects but leave them unnamed until manual annotation. In contrast, Open-Vocabulary Detectors recognize unseen categories only with provided prompts at test time, lacking autonomous discovery or naming. This work presents HSGDet, a detector that achieves both discovery and semantic assignment at test time without external prompts. This method introduces DHGA that navigates a hierarchical semantic graph to perform scene-conditioned coarse-to-fine classification of detected objects. It leverages spatial co-occurrence patterns from surrounding scene context to produce classification confidence scores. High-scoring regions are identified as known objects, while low-scoring regions are flagged as unknown detections. Unknown regions pass to CR2T, which synthesizes text embeddings by fusing visual features, hierarchical parents, and scene context, enabling prompt-free labeling and vocabulary expansion. This approach enables prompt-free semantic labeling and supports autonomous vocabulary expansion without requiring external models. Results demonstrate that HSGDet outperforms state-of-the-art methods by a large margin of 6.6 points in Known mAP and 9.9 points in Unknown Recall. It also reduces Wilderness Impact by 36%, enabling scalable and autonomous aerial monitoring.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

首次观测 · 记录 #1CVF · CVPR 2026:cvf:17657feac3b32207b7a54ab656c41efaCVF · CVPR 2026

来源日期 2026-01-01（仅到年份）

TIACam: Text-Anchored Invariant Feature Learning with Auto-Augmentation for Camera-Robust Zero-Watermarking

Tanvir, Abdullah, Dasgupta, Agnibh, Zhong, Xin

原文摘要

Camera recapture introduces complex optical degradations, such as perspective warping, illumination shifts, and Moire interference, that remain challenging for deep watermarking systems. We present TIACam, a text-anchored invariant feature learning framework with auto-augmentation for camera-robust zero-watermarking. The method integrates three key innovations: (1) a learnable auto-augmentor that discovers camera-like distortions through differentiable geometric, photometric, and Moire operators; (2) a text-anchored invariant feature learner that enforces semantic consistency via cross-modal adversarial alignment between image and text; and (3) a zero-watermarking head that binds binary messages in the invariant feature space without modifying image pixels. This unified formulation jointly optimizes invariance, semantic alignment, and watermark recoverability. Extensive experiments on both synthetic and real-world camera captures demonstrate that TIACam achieves state-of-the-art feature stability and watermark extraction accuracy, establishing a principled bridge between multimodal invariance learning and physically robust zero-watermarking.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

首次观测 · 记录 #1CVF · CVPR 2026:cvf:314836a48b53ff1b0e4614b40eba48b2CVF · CVPR 2026

来源日期 2026-01-01（仅到年份）

DASH: A Meta-Attack Framework for Synthesizing Effective and Stealthy Adversarial Examples

Al Nomaan Nafi, Abdullah, Rahaman, Habibur, Haider, Zafaryab, Mahfuz, Tanzim, Suya, Fnu, Bhunia, Swarup, Chakraborty, Prabuddha

原文摘要

Numerous techniques have been proposed for generating adversarial examples under strict Lp-norm constraints. However, such norm-bounded examples often fail to align well with human perception, and only a few methods specifically explore perceptually aligned adversarial examples. Moreover, it remains unclear whether insights from Lp-constrained attacks can be effectively leveraged to improve perceptual efficacy. In this paper, we introduce DASH, a differentiable meta-attack framework that generates effective and perceptually aligned adversarial examples by strategically composing existing Lp-based attack methods. DASH operates in a multi-stage fashion: at each stage, it aggregates candidate adversarial examples from multiple base attacks using learned, adaptive weights and propagates the result to the next stage. A meta-loss function guides this process by jointly minimizing misclassification loss and perceptual distortion, enabling the framework to dynamically modulate the contribution of each base attack throughout the stages. We evaluate DASH on adversarially trained robust models across CIFAR-10, CIFAR-100, and ImageNet while considering visual perception metrics (e.g. SSIM, FID, LPIPS) in the perturbation budget (instead of Lp-norm). Despite relying solely on Lp-constrained based methods, DASH significantly outperforms state-of-the-art perceptual attacks such as AdvAD, achieving higher attack success rates (e.g., 20.63% improvement) and superior visual quality, as measured by SSIM, LPIPS, and FID (improvements of 11, 0.015, and 5.7, respectively). DASH generalizes well to unseen defenses and different white-box/black-box scenarios, making it a practical and strong baseline for evaluating robustness.

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表

首次观测 · 记录 #1CVF · CVPR 2026:cvf:a57b5067084da007f8cdd05e6817e722CVF · CVPR 2026

来源日期 2026-01-01（仅到年份）

Unleashing Stealthy Backdoor Pandemic by Infecting a Single Diffusion Model

Al Nahian, Mohaiminul, Almalky, Abeer Matar, Ahmed, Sabbir, Al Arafat, Abdullah, Rizve, Mamshad Nayeem, Rakin, Adnan Siraj

原文摘要

The remarkable success of modern Deep Neural Networks (DNNs) can be primarily attributed to having access to compute resources and high-quality labeled data, which is often costly and challenging to acquire. Recently, text-to-image Diffusion Models (DMs) have emerged as powerful data generators to augment training datasets. Machine learning practitioners often utilize off-the-shelf third-party DMs for generating synthetic data without domain-specific expertise or adaptation. Such a practice leads to a novel and insidious threat: a diffusion model infected with a backdoor can effectively spread into a large number of downstream models, causing a backdoor pandemic. To achieve this for the first time, we propose Eidolon, designed and optimized to stealthily transfer the backdoor injected into a single diffusion model into virtually an unlimited number of downstream models without any active attacker role in the downstream training tasks. Proposed Eidolon not only makes the attack stealthier and effective, but it also enforces a strict threat model for injecting a backdoor into the downstream model compared to conventional backdoor attacks. We propose four necessary tests that a successful backdoor attack on the diffusion model should pass to cause a backdoor pandemic. Our evaluation across a wide range of benchmark datasets and model architectures exhibits that only our attack successfully passes these tests, causing widespread pandemic across many downstream models. Code is available at https://github.com/ML-Security-Research-LAB/Eidolon

cs.CV

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表