跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1112新增论文
+875其中 AI 相关
+133已有论文修订
+95新增 AI 导读

arXiv 最近完成扫描:2026-11-20 00:30 · 全站最近入库:2026-11-19 19:09各信源进度 →

统计口径与数据新鲜度

统计 2026-11-19 00:54 — 2026-11-20 00:54(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-20 00:30最近发现新增版本:2026-11-19 19:09库内论文最新更新时间:2026-11-19 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

27,982 已收录论文23,249 AI 相关8,559 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 17 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-20 00:28(北京时间)
新论文 · v1arXiv:2611.24445

Retroactive Concept Bottleneck Models for Exemplar-Free Class-Incremental Learning

Yang Ji, Ying Sun, Yuanxin Zhuang, Yuting Zhang

原文摘要

Exemplar-free class-incremental learning (EFCIL) requires models to learn new classes sequentially without retaining training samples from previous tasks. Continual concept bottleneck models (CBMs) make their predictions interpretable by identifying visual concepts in an image and using them to score each class. However, these methods typically focus on preserving learned concepts or approximating past classes with generated features, which can leave new concepts underused in old-class predictions. We propose RetroCBM, which retains a small set of vocabulary-independent statistics relating frozen visual features to the embedding space of a pretrained vision-language model. Even after earlier training images have been discarded, we can use these statistics to supervise new concepts. To do so, we develop a retroactive regression rule that yields the same predictor for each new concept as class-balanced ridge regression on all images seen so far. We then reproject the stored statistics to recover the full-history squared classification loss and thus learn how much each new concept should contribute to old-class predictions. Since the classifier may use systematic concept errors to distinguish nearby classes, we further introduce a graph-weighted penalty on the margin shifts caused by those errors. Across six datasets, RetroCBM outperforms state-of-the-art interpretable methods in all class-incremental settings, with an 18.1\% relative gain in mean average incremental accuracy over the strongest exemplar-free baseline.

cs.CV

版本更新 · v2arXiv:2611.19803

Pros4D: Modeling Evolving 4D Worlds in Latent Space for Dynamic Spatial Reasoning

Rongxin Gao, Dongxuan Liu, Chu Li, Chengyang Ji, Jialing Huang, Runyu Chen, Kairun Wen, Chenxin Li 等 15 位作者

原文摘要

Dynamic spatial reasoning from monocular video enables vision-language models (VLMs) to understand how 3D environments evolve over time, supporting physical world understanding and embodied intelligence. Existing efforts primarily address observed-dynamics understanding, whereas agents in dynamic environments also require future-dynamics prediction beyond the observation boundary. Moreover, external geometric processing and textual descriptions can increase inference complexity or lose information when representing continuous dynamics. These limitations motivate a shared internal state that preserves observed dynamics and remains predictive beyond the input video. Accordingly, we present Pros4D, a framework for modeling evolving 4D worlds in latent space that unifies retrospective understanding of observed-dynamics with prospective reasoning for future-dynamics prediction. Specifically, Pros4D-Engine derives geometry-grounded supervision and constructs paired examples for both tasks, while Pros4D-Bench evaluates them across eight spatial task categories spanning dynamic cameras, objects, and scenes. At the model level, Pros4D compresses observed dynamics into latent observed states and autoregressively rolls them forward into latent future states. 4D Latent Alignment (4DLA) grounds these states through complementary visual-semantic and geometric supervision, while joint language supervision connects them to task reasoning. Experiments across benchmarks show consistent improvements over strong baselines in both capabilities.

cs.CV

新论文 · v1arXiv:2611.22724

ConserveEdit: Source-Anchored Transactions for Drift-Bounded Iterative Image Editing

Shuyang Jiang, Fucheng Deng, Yuchuan Luo, Zhenyu Wu

原文摘要

Instruction-driven image editors produce convincing local edits in one pass, but inside an iterative refinement loop every accepted output becomes the next input, so unintended changes outside the requested region accumulate. We present ConserveEdit, an editor-agnostic controller that treats each refinement step as a transaction against a contract compiled once from the source image. The contract separates what may change (the authorized region of each target, with a bounded halo for shadows and reflections) from what must survive (protected content with explicit tolerances). Candidates from a frozen editor are composed through a source-anchored mask and committed only if target progress, drift on protected content, and image quality pass as separate tests; otherwise the step is rolled back and repaired. For any editor, this keeps every pixel outside the authorized region bit-exact and bounds drift independently of the number of iterations, whereas no fixed scalar trade-off reproduces the separate tests. On all 1053 MagicBrush test edits with a frozen Qwen-Image-Edit, masking is necessary but not sufficient. A direct call already alters the rest of the image (12.1 dB PSNR outside the mask); composed through the source-anchored mask, it stays bit-exact there and its DINO similarity to the ground truth rises from 0.67 to 0.85, matching ConserveEdit, so this gain comes from the mask. Next to automatic masks, however, masked direct editing changes a protected face or text on 48% of turns, and masked scalar selection avoids this mostly by abstaining (23% constrained success). Because its drift test names the violated entity, ConserveEdit shrinks the support instead (2% violations, 53% constrained success); in a simulator with known ground truth it raises constrained success from 55.0% to 61.8%. Automatic grounding remains the main bottleneck. Code and logs will be released.

cs.CV

版本更新 · v2arXiv:2611.06518

EmoReward: Self-Evolving Reward Calibration for Emotional Video Generation RL

Mingzhe Zheng, Yue Wu, Jing Tan, Chang Zou, Dengyang Jiang, Zeyue Tian, Xuanhua He, Ryan Xu 等 22 位作者

原文摘要

Group Relative Policy Optimization (GRPO) has advanced generative model post-training in verifiable domains such as code and mathematics, but evaluating video quality, especially emotional expression, often depends on subjective aesthetics and has no uniquely correct target. Existing approaches typically rely on fixed rewards for visual quality, motion quality, and text alignment that poorly capture emotional intent and can invite reward hacking, while RLHF is difficult to scale and subject to individual aesthetic differences that complicate the calibration of facial, body motion, and chromatic cues across narrative contexts. We introduce EmoReward, a new self-evolving training mechanism that distills evidence reliability from reward traces collected during an actual training run to adaptively guide multi-reward weight allocation in subsequent training by determining which emotional evidence to trust and how much to weight it. EmoReward first plans specific grounding cues from the emotional prompt to guide evaluation, then uses heterogeneous signal primitives to score the generated video during the act stage. Reflective Reward Gating calibrates their contributions through instance-level gates for phase, face confidence, conflict, and subtlety, then renormalizes weights over the active set to produce the sample's fused reward. RewardMemory then estimates each primitive's reliability from rollout scores and outcomes within each emotional regime in the finished training run, then fixes bounded multipliers that calibrate fusion weights in the next training run without updating reward models. Experiments on HunyuanVideo 1.5 show improvements across multiple RL algorithms and compare EmoReward with other agentic reward methods under SAGE-GRPO. Ablations and transfer experiments on SD 3.5 and FLUX further support adaptive reward calibration for visual generation.

cs.CV · cs.AI

版本更新 · v2arXiv:2508.16569

A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer

Yuhui Tao, Zhongwei Zhao, Zilong Wang, Xufang Luo, Feng Chen, Kang Wang, Chuanfu Wu, Xue Zhang 等 20 位作者

原文摘要

The non-invasive assessment of renal masses remains a critical challenge in urologic oncology, where diagnostic uncertainty frequently causes overtreatment. Here, we develop RenalCLIP, a vision-language foundation model for precision oncology in kidney cancer. Utilizing 27,866 computed tomography scans from 8,809 patients across diverse multi-center cohorts, we employ a two-stage pre-training strategy to align domain-specific visual and textual representations. RenalCLIP achieves enhanced performance and generalizability across ten core clinical tasks, spanning anatomical assessment, diagnostic classification, and survival prediction, significantly outperforming state-of-the-art general-purpose foundation models. Furthermore, RenalCLIP demonstrates strong data efficiency in diagnostic classification, achieving peak baseline performance using only 20% of the training data. The model also exhibits robust zero-shot diagnostic capabilities, effective image-text retrieval, and high-quality medical report generation. Our findings establish RenalCLIP as a powerful, generalizable tool to enhance diagnostic precision, refine prognostic stratification, and personalize the management of renal masses.

eess.IV · cs.AI · cs.CV

版本更新 · v4arXiv:2602.22507

Rethinking Floorplan Generation with Space Syntax: From Design Theory to Model Improvement

Zhuoyang Jiang, Dongqing Zhang, Qingxiang Liu, Yuxuan Liang

原文摘要

Architectural design increasingly adopts generative AI (GenAI) for floorplan generation, an early-stage design task exploring how spaces should be organized for use. Yet fidelity objectives inherited from native visual and geometric generators do not automatically capture this design purpose. This organization remains largely implicit in generated representations, limiting direct evaluation and feedback. Drawing on space syntax, an established architectural design theory, the Space Syntax Integration Operator (SSIO) recovers convex-space access graphs across heterogeneous formats. This theory-grounded shared intermediate representation, which we call a design container, supports evaluation and model understanding of spatial design. Applying SSIO to four residential corpora spanning 15 countries reveals public spaces' integration levels are highest, with corpus-dependent distributions. These patterns underpin the RPLAN-SSI benchmark, an empirical design reference system assessing population-level functional-profile alignment (IPD) and distributional alignment of within-plan public-space advantage (PIAD). Comparison with this residential reference reveals a design gap: public spaces in generated floorplans have a smaller integration advantage over other spaces. To address this gap, Space Syntax-Guided Post-Training (SSPT) provides a plug-and-play design-learning interface: SSIO-derived RL feedback reduces IPD and PIAD across five heterogeneous diffusion backbones while preserving native mechanisms. We further discuss how this approach may inform GenAI applications in other design domains.

cs.LG · cs.CV

新论文 · v1arXiv:2611.15100

Keep Images in Sight: Visual Position Re-Anchoring for Long-Context MLLMs

Juntao Xu, Qichai Zhao, Rui Sun, Zhouyang Jiang, Shiwei Wu, Zhi Li, Kaisheng Ma, Yu Liu

原文摘要

Long-context multimodal intelligence requires revisiting earlier images as conversations and documents accumulate, yet retaining an image does not guarantee reliable access to its details. By varying context length, redistributing a fixed text passage around the image, and changing positions without adding text, we identify the positional relation between images and questions as a direct influence on visual access. To address this gap, we propose Visual Position Re-Anchoring (VPR), a training-free method that shifts cached visual-key rotary phases toward a short-context reference before question processing. It operates directly on the existing KV cache, preserving standard attention computation without extracting attention maps or adding model forward passes. We develop static and continuous compensation schedules, using the static schedule by default. Across ten checkpoints and ten benchmarks, static VPR improves 97 of 100 model--task scores. Further analysis shows greater attention to regions containing answer evidence across six checkpoints, with minimal runtime overhead under fixed rotary frequencies. These findings demonstrate an efficient way to sustain visual access through positional adjustment within the cache.

cs.CV

新论文 · v1arXiv:2611.12353

QCC-WM: Linking Macro-State Dynamics and Visual Generation via Quotient Causal Construction

Zhizhong Fu, Wei Zhou, Zhaoyang Jiang, Yudong Lin, Songlun Li, Zhi Li, Feifan Chen, Gaoming Li 等 10 位作者

原文摘要

A key challenge in action-conditioned world modeling is to construct macro-states from high-dimensional observations that support recursive prediction under actions, while keeping their predicted evolution consistent with visual changes in generated observations. We propose QCC-WM, a visual world model based on Quotient-based Causal Construction. QCC-WM uses action responses to guide macro-variable construction for recursive action-conditioned prediction. A shared reference quotient measurement constrains the consistency between predicted states and observed changes, allowing macro-level predictions to provide structured constraints for future visual generation. Using translational and rotational actions and their predicted responses, the model aggregates local visual features into macro-nodes with spatial correspondence through causal abstraction, and recursively predicts node content and motion under a shared reference measurement. Macro-level predictions guide future observation generation, while predicted motion transports historical appearance information. Node-level errors between generated observations and macro-level targets are then mapped back to the corresponding spatial regions for correction, improving consistency between visual generation and macro-level predictions. On the US-VLA dataset, QCC-WM achieves an ROI-MSE of 0.01861 for four-step prediction. Compared with the best baseline at each prediction horizon, it reduces LPIPS by 48.59%, 41.68%, and 30.23% for four-step prediction and eight- and sixteen-step recursive extrapolation, respectively. Ablation studies and mechanism analyses show that motion-guided reference transport mainly improves pixel reconstruction, while node-level error feedback further improves perceptual quality and consistency between generated observations and macro-level predictions.

cs.CV

新论文 · v1arXiv:2611.10678

MGE: A Membership Guided Data Extraction Attack against Diffusion Models

Yuchang Mei, Longqiang Wang, Haoyang Jiang, Mingyang Yi, Shaolei Zhang, Keman Huang, Wuqiong Pan, Ju Fan

原文摘要

Recently, diffusion models have achieved remarkable performance in image generation. However, their inherent memory properties introduce a potential safety risk of training data leakage, i.e., data extraction attack. To illustrate such risk, we propose a membership guided extraction (MGE) framework to effectively extract training data from a pre-trained diffusion model. Our core idea is to formulate the data extraction as a conditional generation, where the condition signal is from a membership inference classifier. To improve the success rate and efficiency of MGE, we propose adversarial gradient and parallel branch decoding. Our experiments on image generation benchmarks demonstrate that our MGE can extract hundreds of images, which shows that more attention should be paid to privacy-preserving of diffusion model.

cs.CV

版本更新 · v2arXiv:2611.06748

EvoWorld: Towards Out-of-Sight World Evolution in Generative Video Models

Yequan Bie, Dengyang Jiang, Zhixuan Chen, Xiaozhuan Liang, Haoxuan Che, Quande Liu, Xi Chen, Hao Chen

原文摘要

Generative video models are increasingly used as visual world simulators, yet their primary strength lies in rendering plausible pixels rather than reasoning about how a world should evolve. Coherent world evolution requires more than visual fidelity: when an event occurs, a model must infer its consequences for the entities in the world and then render the resulting state. For example, after a wild beast appears, the model should not only depict the wild beast, but also infer whether a nearby person should flee, take cover, or fight back based on the realized scene. Existing diffusion-based video models lack an explicit mechanism for maintaining world state and reasoning over such event-conditioned consequences, often producing visually plausible but behaviorally static or causally inconsistent outputs. To make this reasoning gap concrete and measurable, we propose a new problem, Out-of-Sight Causal Evolution, which requires a model to render an event, reason about its consequences for the evolving world state, and reveal a causally plausible outcome as the world continues to evolve, even out of sight. As a baseline reference solution, we introduce EvoWorld, a brain-renderer collaboration framework in which a vision-language model serves as the world brain: it observes generated segments, maintains an evolving account of the world, and reasons about plausible state transitions, while a segment-autoregressive video renderer realizes them visually. We further introduce EvoWorld-Data and EvoWorld-Bench for training and evaluating event realization, causal reactions, and revisit consistency. Experiments show that EvoWorld substantially improves event-conditioned causal evolution over recent models while preserving visual quality and consistency, establishing a reference point for reasoning-guided, autonomous world evolution beyond the current field of view.

cs.CV · cs.AI

新论文 · v1arXiv:2611.06061

TouchéWorld tech report : Towards Online Unified Camera--Hand Exploration in Generative Worlds

Jiaao Yu, Yanjian Zhang, Haoyang Jiang, Guillaume Wisniewski, Nadi Tomeh, Shijie Jia

原文摘要

Recent video world models allow users to explore generated environments through camera navigation. Continuous hand motion provides a fine-grained action channel for interacting with the people and objects encountered during exploration. We present TouchéWorld, a framework that integrates gesture-driven interaction into a camera-controlled video world model. Given an initial image and a scene-description prompt, the model generates video conditioned on temporally aligned camera and hand controls. We construct joint camera--hand training data from heterogeneous egocentric videos through annotation completion, consistency-based filtering, and temporal alignment. Building on a pretrained camera-controlled world model, we combine spatial hand-skeleton conditioning with numerical hand-motion features through parameter-efficient adaptation. Causal adaptation and few-step distillation support chunk-level online generation, allowing users to update their controls based on generated visual feedback. We further introduce TouchéBench to evaluate hand-motion following, camera--hand coordination, online interaction, and object response. Together, these components establish a framework for studying continuous exploration and hand-driven interaction within generative worlds.

cs.CV · cs.AI

版本更新 · v3arXiv:2609.35203

NucShare: Lightweight Nuclei Instance Segmentation with Foundation-Model Accuracy

Wenyan Li, Xiangyang Jia, Wenlong Zhong, Xu Chen

原文摘要

Nuclei instance segmentation is fundamental to quantitative pathology. Yet high-accuracy methods mostly build on large vision transformers (ViTs) or heavy convolutional backbones, whose parameter count and compute cost hinder deployment in clinical and edge settings. We present NucShare, a lightweight model whose shared decoder handles nuclear pixel segmentation, horizontal-vertical distance, and classification through learnable transposed convolutions for upsampling and three lightweight heads, using 37% of independent decoders' parameters with no loss in accuracy. Output-level distillation then transfers knowledge from NucShare-T (UNI2-h) to NucShare-S (ConvNeXt V2-Tiny), reducing 690.5M parameters to 34.7M. On PanNuke they reach mPQ 0.524 and 0.519, above the prior best CFR-SAM-H at 0.511, and NucShare-S surpasses HoVer-Net and Mask R-CNN by 5.6 and 15.0 points; on zero-shot MoNuSeg it attains bPQ 0.668 against CFR-SAM-H's 0.662. Code: https://github.com/L-iMeng-L/Newidea

cs.CV

新论文 · v1arXiv:2610.36756

Efficient Remote Sensing Image-Text Retrieval via Interaction-Free Recall and Text-Guided Reranking

Xi Chen, Xu Chen, Xiamgyang Jia, Wei Wang, Xu Zhang, Zhenyuan Shun

原文摘要

Remote sensing (RS) image-text retrieval plays a critical role in understanding massive RS imagery. However, the dense multi-object distribution and complex backgrounds in RS imagery make it difficult to simultaneously achieve fine-grained cross-modal alignment and efficient retrieval. Existing methods either rely on complex cross-modal interactions that lead to low retrieval efficiency, or depend on large-scale vision-language adaptation, which requires massive data and computational resources. To address these issues, we develop a fast-then-fine (FTF) retrieval framework centered on interaction-free multi-granular recall and parameter-free text-guided cross-modal reranking with the Balanced Text-guided Interaction Block (BTIB). Specifically, interaction-free multi-granular recall pre-computes gallery embeddings and matches them with the corresponding query embedding without full-gallery token-level text-image interaction, while text-guided cross-modal reranking with BTIB refines local semantic correspondence within the top-K candidate set. Furthermore, an inter- and intra-modal loss is designed to jointly optimize cross-modal alignment across multi-granular representations. Extensive experiments on public benchmarks demonstrate that the FTF achieves competitive retrieval accuracy while significantly improving retrieval efficiency compared with existing methods.

cs.CV

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.1626ACL Anthology · ACL 2026

Specializing Large Models for Oracle Bone Script Interpretation via Component-Grounded Multimodal Knowledge Augmentation

Jianing Zhang, Runan Li, Honglin Pang, Ding Xia The University of Tokyo, Zhou Zhu, Qian Zhang, Chuntao Li Jilin University, Xi Yang Jilin University

原文摘要

Deciphering ancient Chinese Oracle Bone Script (OBS) is a challenging task that offers insights into the beliefs, systems, and culture of the ancient era. Existing approaches treat decipherment as a closed-set image recognition problem, which fails to bridge the “interpretation gap”: while individual characters are often unique and rare, they are composed of a limited set of recurring, pictographic components that carry transferable semantic meanings. To leverage this structural logic, we propose an agent-driven Vision-Language Model (VLM) framework that integrates a VLM for precise visual grounding with an LLM-based agent to automate a reasoning chain of component identification, graph-based knowledge retrieval, and relationship inference for linguistically accurate interpretation. To support this, we also introduce OB-Radix, an expert-annotated dataset providing structural and semantic data absent from prior corpora, comprising 1,022 character images (934 unique characters) and 1,853 fine-grained component images across 478 distinct components with verified explanations. By evaluating our system across three benchmarks of different tasks, we demonstrate that our framework yields more detailed and precise decipherments compared to baseline methods.

cs.CL · cs.CV

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.2068ACL Anthology · ACL 2026

Benchmarking Fine-Grained Error Detection in Multimodal Reasoning

Chi-Min Chan, Han Zhu, Chunyang Jiang, Jiaming Ji, Juntao Dai, Wei Xue Hong Kong University of Science and Technology, Sirui Han The Hong Kong University of Science and Technology, Yike Guo Hong Kong University of Science and Technology and Imperial College London

原文摘要

Multimodal Process Reward Models (MPRMs) have emerged as a pivotal framework for enhancing the reasoning capabilities of Multimodal Large Language Models (MLLMs). However, the research community currently lacks a dedicated benchmark to rigorously assess the error discernment capabilities of these models.To address this gap, we introduce PRMBench-V, a novel benchmark specifically designed to evaluate MPRMs’ proficiency in detecting erroneous reasoning steps across diverse error categories. Leveraging a semi-automated annotation pipeline augmented with human verification, we construct a comprehensive dataset comprising 907 unique queries, each annotated with nine distinct error types, resulting in 8,163 test cases with fine-grained step-level error labels.Through extensive experiments involving over 15 open- and closed-source models, we uncover several key findings: (1) even the strongest existing MPRMs achieve only \textasciitilde30\% accuracy in error identification; (2) while partial error detection achieves moderate precision and recall (\textasciitilde60\%), overall accuracy remains low (\textasciitilde20\%); and (3) benchmark scores exhibit a strong correlation with downstream task performance gains (r=0.86). Furthermore, we demonstrate that PRMBench-V can inform the development of more robust MPRMs: by introducing the Bayesian Rater Reliability Process Reward Model (BR2-PRM), we achieve up to a 4.8% performance improvement through test-time scaling.We believe that PRMBench-V will serve as a valuable resource for advancing MPRM research, enabling more rigorous evaluation and fostering the development of models with fine-grained multimodal reasoning capabilities.

cs.CL · cs.CV

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.367ACL Anthology · ACL 2026

GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Models

Xiangdong Hu, Yangyang Jiang, Qin Hu Shandong University, Xiaojun Jia Nanyang Technological University

原文摘要

Multimodal Large Language Models (MLLMs) have become widely deployed, yet their safety alignment remains fragile under adversarial inputs. Previous work has shown that increasing inference steps can disrupt safety mechanisms and lead MLLMs to generate attacker-desired harmful content. However, most existing attacks focus on increasing the complexity of the modified visual task itself and do not explicitly leverage the model’s own reasoning incentives. This leads to them underperforming on reasoning models (Models with Chain-of-Thoughts) compared to non-reasoning ones (Models without Chain-of-Thoughts). If a model can think like a human, can we influence its cognitive-stage decisions so that it proactively completes a jailbreak? To validate this idea, we propose GAMBIT (Gamified Adversarial Multimodal Breakout via Instructional Traps), a novel multimodal jailbreak framework that decomposes and reassembles harmful visual semantics, then constructs a gamified scene that drives the model to explore, reconstruct intent, and answer as part of winning the game. The resulting structured reasoning chain increases task complexity in both vision and text, positioning the model as a participant whose goal pursuit reduces safety attention and induces it to answer the reconstructed malicious query. Extensive experiments on popular reasoning and non-reasoning MLLMs demonstrate that GAMBIT achieves high Attack Success Rates (ASR), reaching 92.13% on Gemini 2.5 Flash, 91.20% on QvQ-MAX, and 85.87% on GPT-4o, significantly outperforming baselines. Warning: This paper contains unsafe and offensive examples.

cs.CL · cs.CV

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.770ACL Anthology · ACL 2026

MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application

Xueqing Peng Yale University, Lingfei Qian Yale University, Yan Wang Yale University, Ruoyu Xiang, Yueru He Columbia University, Yang Ren Yale University, Mingyang Jiang, Vincent Jim Zhang 等 47 位作者

原文摘要

Real-world financial analysis involves information across multiple languages and modalities, from reports and news to scanned filings and meeting recordings. Yet most existing evaluations of LLMs in finance remain text-only, monolingual, and largely saturated by current models. To bridge these gaps, we present MultiFinBen, the first expert-annotated multilingual (five languages) and multimodal (text, vision, audio) benchmark for evaluating LLMs in realistic financial contexts. MultiFinBen introduces two new task families: multilingual financial reasoning, which tests cross-lingual evidence integration from filings and news, and financial OCR, which extracts structured text from scanned documents containing tables and charts. Rather than aggregating all available datasets, we apply a structured, difficulty-aware selection based on advanced model performance, ensuring balanced challenge and removing redundant tasks. Evaluating 21 leading LLMs shows that even frontier multimodal models like GPT-4o achieve only 46.01% overall, stronger on vision and audio but dropping sharply in multilingual settings. These findings expose persistent limitations in multilingual, multimodal, and expert-level financial reasoning. All datasets, evaluation scripts, and leaderboards are publicly released.

cs.CL · cs.CV

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表