跳到正文
研踪PaperTrail

版本更新

查看已发布论文的最新修订，在详情页对照实际观测到的版本。

部分采集异常

全站更新概况

+0新增论文
+0其中 AI 相关
+0已有论文修订
+93新增 AI 导读

arXiv 最近完成扫描：2026-11-16 01:34 · 全站最近入库：2026-11-14 15:09各信源进度 →

统计口径与数据新鲜度

统计 2026-11-15 03:53 — 2026-11-16 03:53（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-16 01:34最近发现新增版本：2026-11-14 15:09库内论文最新更新时间：2026-11-13 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

23,348 已收录论文19,580 AI 相关6,729 有修订112 HF 社区发现

论文修订列表

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 6,729 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-16 01:33（北京时间）
版本更新 · v2arXiv:2609.08277

Adaptively Incorporating Directional Hints into Zeroth-Order Optimization

Alexander Ryabchenko, Jian Qian, Wenlong Mou

原文摘要

We study zeroth-order optimization of nonconvex functions with the aid of directional hints, which are cheap but potentially inaccurate approximations of the true gradient direction, given by linear subspaces at each iteration. To leverage these hints adaptively while maintaining robustness to their quality, we introduce Control-Variate Zeroth-Order Descent (CV-ZOD), a new framework that refines the classical zeroth-order gradient estimator with a control variate that can be set based on the directional hints. We first show that the oracle algorithm that optimally sets the reference vector and step size at each iteration achieves a convergence rate that interpolates between the first-order $O(1/T)$ rate and the zeroth-order $O(d/T)$ rate, depending on the quality of the hints along the trajectory. We then develop a practical variant of CV-ZOD that achieves the same oracle guarantee up to logarithmic factors, without any prior knowledge of the hint quality. We validate the method empirically on simulation-based scientific optimization tasks, demonstrating sustained progress on nonconvex landscapes where zeroth-order descent is slower and existing guided methods stall as guidance deteriorates.

cs.LG · math.OC

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.08864

Adaptive Scheduling Pipeline for Multi-Instance Asynchronous Reinforcement Learning

Salah Chikhi, Luis Zuin, Entong Li, Zeng Li

原文摘要

We present the Adaptive Scheduling Pipeline (ASP), an offline rollout scheduler for multi-instance asynchronous reinforcement learning (RL) of large language models (LLMs). ASP jointly reduces three sources of rollout underutilization - padding bubbles from intra-minibatch length heterogeneity, skewness bubbles from long-tail outputs, and sorting bubbles from fluctuations in training-queue occupancy induced by output-length-sorted dispatch - through a single offline preprocessing pass. This enables efficient multi-instance rollout that overcomes the early scaling plateau of single-instance inference while reducing fluctuations in the supply of generated samples to the trainer. We integrate the scheduler into an asynchronous architecture that decouples rollout from training while keeping policy lag observable and bounded. Within our asynchronous architecture, ASP's scheduling reduces mean iteration time by 10.9%. The complete system improves end-to-end throughput by up to 3.22 times and reduces iteration time by up to 76.8% relative to MindSpeed-RL, while reducing iteration time by up to 62.5% relative to verl. It also scales more robustly with additional generation resources than these baselines and shows no visible convergence degradation in the measured bounded-lag regime. ASP has since been used in an internal telecom-domain RL post-training workflow, reducing the reported training period from more than three months to two months and producing a domain-enhanced model for knowledge answering and tool calling.

cs.DC · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.35374

GIFT: Representation Geometry Matters for Single-Domain Generalized Object Detection

Yin Zhang, Yaoyue Zheng, Yongqiang Zhang, Bogdan Raducanu, Dan Liu

原文摘要

Single-Domain Generalized Object Detection (S-DGOD) aims to generalize from a single source domain to unseen domains under distribution shifts. Existing efforts mainly focus on simulating unseen domains or suppressing domain-specific factors. However, they largely overlook the intrinsic generalization capability embedded in Vision Foundation Models (VFMs). In this work, we show that unconstrained fine-tuning disrupts the geometric structure of pre-trained representations, which encodes domain-invariant semantics. Motivated by this, we propose Geometric-Invariance Fine-Tuning (GIFT), a geometric-aware fine-tuning method for VFMs. Specifically, we introduce a series of learnable Householder reflections to the pre-trained weights to encourage geometry-consistent adaptation under domain shift. In addition, we propose a lightweight channel-wise scaling to facilitate effective task-specific adaptation. Extensive experiments demonstrate that GIFT consistently outperforms state-of-the-art (SOTA) approaches on two S-DGOD benchmarks. Notably, by introducing fewer than 1% additional trainable parameters into the frozen backbone, the proposed method achieves improvements of +7.9% and +2.9% in mPC over the previous SOTA on the Cityscapes-C and Diverse Weather Dataset, respectively. This finding highlights that preserving representation geometry is crucial for robust cross-domain generalization, establishing GIFT as a principled and efficient solution to domain shift.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.02248

Geometric Latent Reasoning Induces Shorter Generations in LLMs

Shashi Kumar, Yacouba Kaloga, Petr Motlicek, Ina Kodrasi, Andrea Cavallaro

原文摘要

Large language models solve complex problems by generating lengthy chains of explicit reasoning tokens. While effective, this makes reasoning expensive, length-sensitive, and constrained to (discrete) natural language. While latent reasoning offers a continuous alternative, determining useful structures for intermediate latent states is an open challenge. In this paper, we formulate latent reasoning as a geometric path-approximation problem within the model's pretrained token-embedding space. We introduce Geometric Latent Reasoning (GLR), which uses a lightweight transition head to predict iterative direction updates in embedding space. Using textual chain-of-thought traces as anchors, GLR learns to approximate discrete reasoning trajectories while permitting continuous deviations from exact token embeddings. Evaluations on mathematical reasoning benchmarks using Qwen3 models reveal an emergent phenomenon: geometric latent reasoning induces substantially shorter generations without an explicit length objective. By replacing early explicit reasoning with continuous latent steps, models often reach correct answers using substantially fewer total generation steps. These findings suggest that continuous trajectories act as compact intermediate reasoning states, exposing a new tradeoff between latent computation budget, output length, and accuracy. Our code is publicly available at: https://github.com/idiap/GLR. The project page is available at: https://glr-llm.github.io/.

cs.CL

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.21743

Pairwise AUC Optimization Needs Corrective Power: A Unified View

Jia Chen, Zhiyong Yang, Shilong Bao, Qianqian Xu, Qingming Huang

原文摘要

Direct pairwise AUC-surrogate training is attractive because it trains toward the ranking metric used at test time, especially under class imbalance. This creates a direct alignment between the training objective and the evaluation metric, but such metric alignment alone does not guarantee that stochastic training remains corrective. We study this gap through Corrective Power: whether pairwise mini-batch updates produce stronger and more reliable improvements for more severe positive-negative ranking errors. Our main observation is that direct pairwise training can enter regions where severe errors remain common but no longer produce useful corrective motion. In such regions, even the mini-batch noise that would normally help SGD move along the steepest severe-error correction direction can become much weaker. In this paper, we seek a theoretical explanation for this by exploring the degenerate property of the U-statistics formed from the pairwise gradients therein. Our results provide a mechanistic explanation for several empirical patterns: (a) squared-hinge loss is often an effective AUC surrogate; (b) cross-entropy warm-up can help models that enter the AUC phase from a poor starting region; and (c) parameter-efficient fine-tuning from a pretrained model can reduce that need by starting closer to a favorable ranking geometry. Experiments on 8 CV tasks and 8 NLP tasks, using ResNet50, DenseNet121, DistilBERT-base, and Qwen2.5-1.5B-Instruct, support these predictions.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2602.13930

MamaDino: A Hybrid Vision Model for Breast Cancer 3-Year Risk Prediction

Ruggiero Santeramo, Igor Zubarev, Florian Jug

原文摘要

Breast cancer screening programmes increasingly seek to move from one-size-fits-all intervals to risk-adapted and personalized strategies. Deep learning (DL) has enabled image-based risk models with stronger 1- to 5-year prediction than clinical models, but leading systems (e.g., Mirai) typically use convolutional backbones, high-resolution inputs (>1M pixels) and simple multi-view fusion, with limited explicit modelling of contralateral asymmetry. We hypothesised that combining complementary inductive biases (convolutional and transformer-based) with explicit contralateral asymmetry modelling would allow us to match state-of-the-art 3-year risk prediction performance on lower-resolution mammograms, suggesting that using less detail in a more structured way can recover state-of-the-art accuracy. We present MamaDino, a mammography-aware multi-view attentional DINO model. MamaDino fuses frozen self-supervised DINOv3 ViT-S features with a trainable CNN encoder at 512x512 resolution and aggregates bilateral breast information via a BilateralMixer to output a 3-year breast cancer risk score. We train on 53,883 women from OPTIMAM (UK) and evaluate on matched 3-year case-control cohorts: an in-distribution test set from four screening sites and an external out-of-distribution cohort from an unseen site. At breast level, MamaDino achieves discrimination comparable to Mirai on internal and external tests while using ~13x fewer input pixels. Adding the BilateralMixer improves patient-level discrimination to AUC 0.736 (vs Mirai 0.713) in-distribution and 0.677 (vs Mirai 0.676) out-of-distribution, with overlapping confidence intervals. Performance was evaluated across age, ethnicity, scanner manufacturer, tumour type and grade. These findings suggest that complementary inductive biases and explicit contralateral modelling can achieve Mirai-comparable discrimination at lower resolution.

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2412.17330

EcoSearch: A Constant-Delay Best-First Search Algorithm for Program Synthesis

Théo Matricon, Nathanaël Fijalkow, Guillaume Lagarde

原文摘要

Many approaches to program synthesis perform a combinatorial search within a large space of programs to find one that satisfies a given specification. To tame the search space blowup, previous works introduced probabilistic and neural approaches to guide this combinatorial search by inducing heuristic cost functions. Best-first search algorithms ensure to search in the exact order induced by the cost function, significantly reducing the portion of the program space to be explored. A key limitation of existing best-first search algorithms is that they slow down over time: the delay between outputting successive programs grows with the number of programs already generated. We ask whether best-first search algorithms with constant delay exist. We design Eco Search, a new best-first bottom-up search algorithm. It combines the cost tuple representation of Bee Search with the per non-terminal data structures of Heap Search, thanks to a new frugal expansion strategy, and it uses bucket queues instead of priority queues, which is possible because the costs stored in each queue always lie in a window of constant width. We prove that Eco Search is the first constant-delay best-first search algorithm for pre-generation cost functions. Experimentally, we evaluate Eco Search on two classic program-synthesis benchmarks (DeepCoder and FlashFill) and observe that Eco Search solves 12% to 70% more tasks than its predecessors, and reaches the number of tasks they solve 1.6 to 3.2 times faster. This shows that constant delay can be achieved for best-first search in program synthesis, and that it makes a difference in practice.

cs.LG · cs.AI · cs.PL

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2605.21706

Latent-space Attacks for Refusal Evasion in Language Models

Giorgio Piras, Raffaele Mura, Fabio Brau, Maura Pintor, Luca Oneto, Fabio Roli, Battista Biggio

原文摘要

Safety-aligned language models are trained to refuse harmful requests, yet refusal behavior can be suppressed by steering their internal representations. Existing methods do so by ablating a refusal direction from model activations, aiming to remove refusal from the model's residual stream. Despite their empirical success, these methods lack a principled account of the latent-space transformation they induce and why it suppresses refusal. In this work, we recast refusal suppression as a latent-space evasion attack against linear probes trained to separate refused from answered prompts. Under this view, prior work's difference-in-means direction naturally defines such a probe, and its ablation is exactly a projection onto its decision boundary, i.e., a minimum-confidence evasion attack. This perspective not only explains the empirical success of prior work but also admits a key limitation: evasion stops at the decision boundary, motivating the need to push representations further into the compliant region, i.e., where the model answers. We leverage this by proposing a Controlled Latent-space Evasion attack that projects representations past the boundary with an optimized confidence. Across 15 instruction-tuned, multimodal, and reasoning language models, we achieve higher attack success rates than existing refusal-ablation baselines, also outperforming jailbreak attacks.

cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2603.23439

SIGMA: A Physics-Based Benchmark for Gas Chimney Understanding in Seismic Images

Bao Truong, Quang Nguyen, Baoru Huang, Jinpei Han, Van Nguyen, Ngan Le, Minh-Tan Pham, Doan Huy Hien 等 9 位作者

原文摘要

Seismic images reconstruct subsurface reflectivity from field recordings, guiding exploration and reservoir monitoring. Gas chimneys are vertical anomalies caused by subsurface fluid migration. Understanding these phenomena is crucial for assessing hydrocarbon potential and avoiding drilling hazards. However, accurate detection is challenging due to strong seismic attenuation and scattering. Traditional physics-based methods are computationally expensive and sensitive to model errors, while deep learning offers efficient alternatives, yet lacks labeled datasets. In this work, we introduce \textbf{SIGMA}, a new physics-based dataset for gas chimney understanding in seismic images, featuring (i) pixel-level gas-chimney mask for detection and (ii) paired degraded and ground-truth image for enhancement. We employed physics-based methods that cover a wide range of geological settings and data acquisition conditions. Comprehensive experiments demonstrate that SIGMA serves as a challenging benchmark for gas chimney interpretation and benefits general seismic understanding.

cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.19943

Probabilistic Tiny Recursive Model

Amin Sghaier, Ali Parviz, Alexia Jolicoeur-Martineau

原文摘要

Tiny Recursive Models (TRM) solve complex reasoning tasks with a fraction of the parameters of modern large language models (LLMs) by iteratively refining a latent state and final answer. While powerful, their deterministic recursion can lead to convergence at suboptimal solutions, without escape mechanism. A common workaround relies on task-specific input perturbations at test time combined with answer aggregation via voting. We introduce Probabilistic TRM (PTRM), a task-agnostic framework for test-time compute scaling that addresses this limitation through stochastic exploration. PTRM injects Gaussian noise at each deep recursion step, enabling parallel trajectories to explore diverse solution basins, and selects among them using the model's existing Q head (used for early stopping in the original TRM). Without requiring retraining or task-specific augmentations, PTRM enables substantial accuracy gains across benchmarks, including Sudoku-Extreme (87.4% to 98.75%) and on various puzzles from Pencil Puzzle Bench (62.6% to 91.2%). On the latter, PTRM achieves nearly double the accuracy of frontier LLMs (91.2% vs. 55.1%) at less than 0.0001x the cost, using only 7M parameters.

cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.27767

Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization

Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef Mroueh

原文摘要

Optimizing functionals over the space of probability measures is now ubiquitous in machine learning. A widely used approach is to perform the optimization directly over the Wasserstein space, but many objective functionals of practical interest are non-convex along Wasserstein geodesics, making the analysis of standard first-order methods challenging. In this work, we study a class of objectives over the Wasserstein space that admit a difference-of-convex (DC) decomposition and we lift the classical convex-concave procedure (CCCP) to this setting. Under smoothness and strong convexity assumptions on the convex components of the decomposition, we prove almost stationarity along the iterates of the resulting algorithm. Our main focus is on the Maximum Mean Discrepancy (MMD) and the Energy Distance (ED) functionals, for which we develop explicit Wasserstein DC decompositions, and establish local convergence of the scheme under mild assumptions. Empirically, we show that well-chosen DC decompositions yield faster and more stable convergence than Wasserstein gradient descent on these MMD objectives.

cs.LG · math.OC

查看论文
arXiv ↗PDF ↗
版本更新 · v7arXiv:2502.12756

Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planning

Jaike van Twiller, Yossiri Adulyasak, Erick Delage, Djordje Grbic, Rune Møller Jensen

原文摘要

Reinforcement learning (RL) has successfully solved various deterministic and stochastic planning problems. However, conventional RL struggles with complex real-world constraints, particularly when feasibility is explicit and depends on the current state or trajectory. In this work, we address stochastic sequential decision-making with state-dependent constraints through a real-world case study of the master stowage planning problem in container shipping, which aims to optimize revenue and costs under demand uncertainty and operational constraints. We propose a deep RL framework with an encoder-decoder model that integrates problem instance, solution, and uncertainty information to guide planning. We introduce differentiable projection layers that enforce convex polyhedral constraints, while Jacobian corrections offset the projections to yield unbiased policy gradient estimates. Experiments show that our model efficiently finds adaptive, feasible solutions that generalize across distribution shifts and scale to longer planning horizons, outperforming state-of-the-art baselines in constrained RL and stochastic programming. As such, our policies enable adaptive, uncertainty-aware planning that can support resilient and sustainable supply chains.

cs.LG · math.OC

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.04571

Ground-Wing: A Parametric 3D Wing-in-Ground-Effect Dataset for Benchmarking Neural Surrogate Models

Andrea Cenzato, Lukas Karch, Andre Luckow, Yoran Pichon, Francesco Pasqua, Abouzar Ghasemi, Rishikesh Ranade, Ira Shokar 等 9 位作者

原文摘要

This paper presents a CFD dataset of a highly parametrized 3D wing operating in ground effect, designed for benchmarking neural surrogate models for aerodynamic prediction, using a wing shape and flow conditions typically encountered in motorsport applications. The dataset comprises 770 RANS simulations of 194 unique wing configurations, sampled across 27 airfoil profile variants and 4 distances from the ground, spanning variations in wing sweep, chord, taper, angle of attack, and tip twist. Six neural network architectures representing distinct paradigms --- convolutional (DoMINO), graph-based (X-MeshGraphNet), transformer-based (AB-UPT, GeoTransolver, GAOT), and JEPA-based (AeroJEPA) --- are evaluated on their ability to predict surface pressure coefficient and wall shear stress distributions, from which integral drag and lift coefficients are recovered. The dataset is intended to be used as a challenging benchmark for surrogate CFD models, due to the uniqueness of the ground effect being absent from other wing and airfoil datasets. The dataset is available for download at https://huggingface.co/datasets/andreacenzato/Ground-Wing

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2606.23357

SOAP-Bubbles $\circ$\ding{109} Structured Weight Uncertainty for Neural Networks

Adrian Robert Minut, Nico Daheim, Marco Miani, Mohammad Emtiyaz Khan, Wu Lin, Thomas Möllenhoff

原文摘要

Structured weight-uncertainty can improve many aspects of deep learning, but it remains costly to estimate and difficult to implement. Here, we show that these issues can be addressed by adapting the SOAP optimizer. Our key idea is to run IVON, an existing diagonal-covariance variational method, in the eigenspace of SOAP's preconditioner and then use the preconditioner to transform the diagonal estimate into a non-diagonal covariance. The resulting method has costs similar to those of SOAP and requires no drastic changes to training pipelines. We call the posteriors obtained in this way SOAP-Bubbles and our new optimizer Eigenspace-VON (EVON). We show that, for logistic regression, EVON can match the full Gaussian covariance and that, for language model pretraining, it yields significantly better results than existing diagonal-covariance methods. Our work makes it easier to estimate more expressive posterior distributions for deep learning at scale.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2508.14948

LFM4Ads: Large Foundation Model for Ads Recommendation

Jie Jiang, Shangyu Zhang, Shijie Quan, Zhongren Wang, Tianqu Zhuang, Junwei Pan, Bo Fu, Yilong Sun 等 35 位作者

原文摘要

Recent work leverages large-scale pretrained foundation models (FMs) to transfer generalizable representations to downstream advertising tasks. Most existing approaches transfer user representations (URs) and item representations (IRs), while underutilizing the user-item cross representations (CRs) generated by deeper interaction layers. Although a recent work transfers target-aware embeddings (TAEs), it requires real-time upstream FM inference alongside the downstream, introducing additional engineering complexity and overhead. To address these limitations, we propose Large Foundation Model for Ads Recommendation (LFM4Ads), an efficient pretrain-then-transfer paradigm for cross-domain ad recommendation. LFM4Ads jointly exploits URs, IRs, and CRs from the upstream FM. It extracts sample-level CRs from pretrained interaction layers and aggregates them into user-level CRs using a time-aware exponential moving average, thereby decoupling upstream inference from downstream online serving. To use CRs efficiently, we propose three downstream usages at different levels of granularity: feature-level gating to fuse CRs into downstream embeddings, module-level fine-tuning to yield downstream CRs, and model-level retrieval by matching CRs with IRs. LFM4Ads has been deployed on Tencent's advertising platform since Q4 2024. It has supported 10+ launches across major scenarios, yielding a platform-wide gross merchandise value (GMV) lift of 2.45% and annual revenue gains in the hundreds of millions of dollars. Offline experiments on KuaiRand-1K further demonstrate our effectiveness. Our code is available at https://github.com/ZhuangQu/LFM4Ads.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2602.16507

On objective mismatch in molecular retrieval from tandem mass spectrometry

Gaetan De Waele, Marek Wydmuch, Krzysztof Dembczyński, Wojciech Kotłowski, Willem Waegeman

原文摘要

Identifying molecular structures from LC-MS/MS spectra is a central problem in computational metabolomics, often approached by predicting molecular fingerprints and retrieving candidates via similarity search. Despite widespread use, the relationship between training objectives for fingerprint prediction and downstream retrieval performance remains poorly understood. In this work, we investigate the alignment between these objectives and show that they can be misaligned under uncertainty. Adopting a decision-theoretic perspective, we derive novel regret bounds that characterize when Bayes-optimal predictors for fingerprint similarity diverge from those for molecular retrieval. Our analysis reveals that optimizing standard similarity-based losses can degrade retrieval performance, and that the extent of this mismatch depends on the similarity structure of candidate sets. Empirically, we validate our theory on the MassSpecGym and Enveda-180 datasets, observing a Pareto frontier between fingerprint accuracy and retrieval metrics across commonly used loss functions. These findings highlight potential trade-offs in fingerprint-based molecular identification and provide principled guidance for the design of learning objectives in small molecule mass spectrometry.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.09114

MAAM: Anchor-Preserving Compression and Contextual Calibration for Chinese Discriminatory Language Detection

Yuxin Fu, Shijing Si

原文摘要

Chinese discriminatory-language detection is challenging because harmful intent is often implicit and context-dependent. We propose MAAM (Myopia--Astigmatism Anchor Mechanism), a lightweight, model-agnostic framework inspired by functional visual blur: rather than preserving every token equally, MAAM retains discrimination-relevant semantic anchors and calibrates them with C--I--S contextual priors (Contextual Tone, Group Identity, and Stance Polarity). We also introduce ChLGBT, to our knowledge the first Chinese LGBT-focused discriminatory-language dataset, with 8,120 manually annotated samples and three ordinal labels: explicit bias, implicit bias, and emotional intensity. Across strong encoder baselines, MAAM improves all three prediction dimensions, with consistent gains in accuracy, F1, Brier score, and expected calibration error. Compared with frontier LLM baselines under zero-shot and few-shot prompting protocols, MAAM remains competitive while offering stronger compactness and stability. These results suggest that interpretable anchor preservation and contextual calibration provide a practical alternative to heavier model scaling for Chinese discriminatory-language assessment.

cs.CL

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2607.22595

OrigaMI: An LLM Serving System with First-Class Interpretability Apps

Michael Blum, Mark Silberstein, Yaniv David

原文摘要

Mechanistic interpretability (MI) enables monitoring and steering LLMs by operating on their internal state during inference. MI applications remain difficult to develop, expensive to execute, and challenging to manage at runtime. We present OrigaMI, a system that integrates MI applications as first-class components of LLM serving, enabling simplified development, efficient execution, and flexible deployment. Guided by a comprehensive analysis of modern MI applications, OrigaMI streamlines the MI applications development via a simple Python-based Domain Specific Language (DSL) for accessing intra- and inter-layer runtime model state at well-defined trigger points during inference. OrigaMI enables efficient access to ephemeral attention scores via a novel FlashInspect kernel that retains the performance benefits of FlashAttention. OrigaMI's compiler integrates multiple MI applications into the performance-optimized inference execution flow while ensuring conflict-free coexistence. OrigaMI's runtime supports dynamic, per-request activation and deactivation of MI applications without service interruption or performance degradation when inactive, leveraging OrigaMI's dynamic memory management and CUDA graph-compatible execution. We integrate OrigaMI with vLLM and evaluate ten MI applications, including sparse autoencoders, steering, and jailbreak detection, across four state-of-the-art LLMs. OrigaMI incurs negligible execution overhead, up to two orders of magnitude lower than existing frameworks, while enabling per-request activation and deactivation up to 1,000 times faster than alternatives. These results demonstrate that OrigaMI makes MI applications practical for deployment in production LLM serving systems.

cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2602.12021

Improved state mixing in higher-order and block diagonal linear recurrent networks

Igor Dubinin, Antonio Orvieto, Felix Effenberger

原文摘要

Linear recurrent networks (LRNNs) and linear state space models (SSMs) promise computational and memory efficiency on sequence modeling tasks, yet their diagonal state transitions limit expressivity. Dense and/or nonlinear architectures (e.g., LSTMs) on the other hand are provably more expressive, but computationally costly. Here, we explore how expressivity in LRNNs can be increased via richer state mixing across time and channels while maintaining competitive efficiency. Specifically, we introduce two structured LRNN architectures: (i) Higher-order Linear Recurrent Units (H-LRU), which generalize recurrences to arbitrary order, mixing multiple past states, and (ii) Block-Diagonal LRUs (BD-LRU), which enable dense intra-block channel mixing. To ensure stable training, we introduce a selective gate normalization scheme that allows for scalable window and block sizes. To maintain efficiency, we implement block-diagonal parallel scans in PyTorch and Triton that keep the throughput competitive with diagonal LRNNs for moderate orders (H-LRU) and block sizes (BD-LRU). Consistent with prior theoretical studies on the limitations of diagonal models, we empirically demonstrate that our architectures benefit from the increased expressivity of structured state mixing, on synthetic sequence modeling and on language modeling tasks that require tracking structure. Our results show that the structure of state mixing is a critical driver of performance in LRNNs, offering a practical pathway to closing the efficiency-expressivity gap in linear sequence models.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2605.03631

Design and Analysis of Quantum Dual-Containing CSS LDPC Codes based on Quasi-Dyadic Matrices

Alessio Baldelli, Marco Baldi, Massimo Battaglioni, Franco Chiaraluce, Paolo Santini

原文摘要

Quantum error correcting codes are essential to achieve fault-tolerant quantum computation. This work introduces two constructions of high-rate, dual-containing (DC) Calderbank--Shor--Steane low-density parity-check (LDPC) codes based on quasi-dyadic matrices. We characterize the automorphism group of such codes, investigate their minimum distance behavior, and provide several theoretical results on their cycle properties. Monte Carlo simulations under depolarizing and phenomenological noise show better finite-length logical error rates than the considered DC benchmark codes and competitive performance against several state-of-the-art quantum LDPC code families. Finally, we employ an automorphism-ensemble belief propagation decoder to improve their decoding performance.

cs.IT · quant-ph

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.09221

The Pokémon Theorem: Fairness Criteria Can't Catch 'Em All

Daniel Matsui Smola, Alex Smola

原文摘要

Debates around algorithmic fairness rarely end. Models considered "fair" by one criterion are soon shown to be unfair by another. We prove that this is unavoidable. Satisfying any finite checklist of fairness criteria never certifies fairness itself, since the model may always violate yet another criterion. We call this the Pokémon theorem: you can't catch them all. In fact, trying to catch all criteria recovers the classical impossibility theorems. Since these violations come from differences between the groups' data, a natural remedy would be to make the data fair by learning fair features. Unfortunately, this, too, is impossible: features that scrub out sensitive group information also scrub out all predictive signal. While exact fairness is out of reach, approximate fairness is not. We derive the Pareto frontier between fairness and predictive signal for approximately fair features, and show how to attain it.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.41158

Lifelong Learning of Scheduling Rules with Surrogate-assisted Genetic Programming for Dynamic Flexible Job Shop Scheduling

Luyao Zhu, Fangfang Zhang, Yi Mei, Ke Chen, Mengjie Zhang

原文摘要

Genetic programming (GP) has been widely used to learn scheduling rules for dynamic flexible job shop scheduling (DFJSS). However, trained rules tend to only generalise to tasks of similar scales and distributions, and when task characteristics change, learned rules often suffer significant performance degradation. Moreover, fine-tuning pre-trained rules on new tasks often leads to catastrophic forgetting, where prior knowledge is lost. This motivates the need for GP methods that can adapt to new tasks while preserving past knowledge, leading to a lifelong learning paradigm. However, lifelong learning in GP for DFJSS remains largely under-explored. To address this gap, this paper proposes a surrogate-assisted GP method for lifelong learning to handle sequential heterogeneous DFJSS tasks with job or batch arrivals. The framework includes three key components: an adaptive two-stage training strategy to balance plasticity and stability, a task-similarity-based weighting mechanism to regulate knowledge reuse, and an improved surrogate model with distance constraints to enhance estimation accuracy. Experimental results show that the proposed method effectively mitigates catastrophic forgetting while maintaining strong adaptability, outperforming several compared methods and demonstrating the potential of lifelong GP in dynamic scheduling environments.

cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.06081

Fast Gauss-Newton for Multiclass Cross-Entropy

Mikalai Korbit, Mario Zanon

原文摘要

In multiclass softmax cross-entropy, the full generalized Gauss-Newton (GGN) curvature couples all output logits through the softmax covariance, making curvature-vector products harder to scale as the number of classes grows. We show that the standard multiclass GGN can be decomposed exactly into a true-vs-rest term and a positive semidefinite within-competitor covariance term. Fast Gauss-Newton (FGN) retains the first term and drops the second, yielding a positive semidefinite under-approximation of the multiclass GGN that is exact for binary classification. The derivation uses an exact true-vs-rest scalar-margin representation of softmax cross-entropy: the loss and gradient are unchanged, and the approximation enters only at the curvature level. Exploiting the FGN curvature structure, the damped update can be written as an equivalent whitened row-space system with one row per mini-batch example. We solve this system matrix-free by conjugate gradient using Jacobian-vector and vector-Jacobian products of the scalar margin map. Targeted mechanism experiments show how the dropped covariance grows with competitor dispersion and how damping can reduce its effect on the resulting steps. In frozen-feature head optimization and end-to-end CIFAR-100 training, FGN achieves similar test accuracy to a full GGN baseline with faster updates.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2608.30556

AdaPath: Query-Adaptive Path-Finding via Path-Bank for Multi-Hop Implicit Biomedical KGQA

Jun Hyeong Kim, Dongki Kim, Yinhua Piao, Sung Ju Hwang

原文摘要

Path-finding over knowledge graphs has become an effective way to ground LLM reasoning on multi-hop questions. However, biomedical QA introduces two distinct challenges that general-domain methods are not designed for: (i) queries do not expose intermediate reasoning and can be answered through multiple valid pathways, and (ii) biomedical knowledge graphs are densely connected, so path-finding methods easily take wrong turns. To address these challenges, we propose AdaPath, a path-finding framework that retrieves query-adaptive meta-paths from Path-Bank, which captures both query semantics and biomedical knowledge graph structure. AdaPath provides the missing cues in biomedical queries while effectively pruning dense knowledge graph neighborhoods during multi-hop reasoning. We further release BioStrat-QA, a biomedical KGQA benchmark that stratifies multi-hop queries by how much intermediate reasoning they expose. Across biomedical KGQA benchmarks, AdaPath consistently outperforms baselines, sustaining meaningful path-finding even when multi-hop queries expose less surface information. The source code is available at https://github.com/Jun-Hyeong-Kim/AdaPath.

cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.02577

ROMAN: A Multiscale Routing Operator for Convolutional Time Series Models

Gonzalo Uribarri

原文摘要

We introduce ROMAN (ROuting Multiscale representAtioN), a deterministic operator for time series that maps temporal scale and coarse temporal position into an explicit channel structure while reducing sequence length. ROMAN builds an anti-aliased multiscale pyramid, extracts fixed-length windows from each scale, and stacks them as pseudochannels. ROMAN provides a training-free mechanism to control the inductive bias of downstream models: it can reduce temporal invariance, make temporal pooling implicitly coarse-position-aware, and expose multiscale interactions through channel mixing, while often improving computational efficiency by shortening the processed time axis. We formally analyze the operator and evaluate it as preprocessing for four convolutional classifiers: MiniRocket, MultiRocket, a standard CNN-based classifier, and a fully convolutional network (FCN) classifier. Synthetic mechanism studies that isolate coarse position, long-range correlation, multiscale interaction, and full positional invariance show that ROMAN behaves as intended, and comparisons with coordinate augmentation and single-scale routing distinguish the benefits of positional routing from those of the multiscale cascade across the evaluated backbones. On long-sequence subsets of the UCR and UEA archives, ROMAN's effect on accuracy varies across datasets while its effect on efficiency is often favorable. At \(S=4\) on UCR, accuracy changes are positively correlated across the pooled backbones, consistent with the proposed role of task structure. We give practical guidance for choosing the operator's two hyperparameters and illustrate its use on very long sequences, where it is both more accurate and several times cheaper, and as a source of representation diversity for ensembles. The ROMAN operator library is available at \href{https://github.com/gon-uri/ROMAN}{github.com/gon-uri/ROMAN

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.08839

Functional Dependency beyond Similarity: Hierarchical Decomposition of Training Data Attribution in Music Generative Models

Changheon Han, Hugh Xuechen Liu, Ashkan Panahi, Kıvanç Tatar

原文摘要

Copyright analysis focuses on the similarity between two works. However, generative AI models learn from training data through functional relations beyond similarity. Training Data Attribution (TDA) estimates the influence of each training instance on a generated output and may serve as a basis for remuneration. However, prior studies find global influence, where a few training instances rank as the most influential for almost every generation. Used directly, TDA scores would distribute part of every remuneration to these instances regardless of what was generated. To address this, we propose a new approach in which TDA scores are decomposed into three levels of influence, namely global indicating the influence of a training data to a trained model, group for its influence on a group of generated outputs (that could be clustered together), and specific for its influence on a single generated output. Our study finds that, across several music generative models and TDA methods, global- and group-level influence is non-negligible in every case. Based on these observations, we propose the term functional dependency for the relation of training data to a trained model or to its outputs. This notion both encompasses and distinguishes dependencies of different scopes, allowing TDA scores to be interpreted as distinct forms of contribution in legal, economic and ethical discussions on remuneration.

cs.CY · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2602.01849

Self-Rewarding Sequential Monte Carlo for Masked Diffusion Language Models

Ziwei Luo, Ziqi Jin, Lei Wang, Lidong Bing, Thomas B. Schön

原文摘要

This work presents self-rewarding sequential Monte Carlo (SMC), an inference-time scaling algorithm enabling effective sampling of masked diffusion language models (MDLMs). Our algorithm stems from the observation that most existing MDLMs rely on a confidence-based sampling strategy, where only tokens with the highest prediction confidence are preserved at each step. This restricts the generation to a noise-sensitive, greedy decoding paradigm, resulting in an inevitable collapse in the diversity of possible paths. We address this problem by launching multiple interacting diffusion processes in parallel, referred to as particles, for trajectory exploration. Importantly, we introduce the trajectory-level confidence as a self-rewarding signal for assigning particle importance weights. During sampling, particles are iteratively weighted and resampled to systematically steer generation towards globally confident, high-quality samples. Our self-rewarding SMC is verified on various masked diffusion language models and benchmarks, achieving significant improvement without extra training or reward guidance, while effectively converting parallel inference capacity into improved sampling quality. Our code is available at https://github.com/Algolzw/self-rewarding-smc.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.02283

MLoRA:Continual Low-Rank Adaptation via Relative-Response Subspace Selection and Momentum Orthogonalization

Junkang Liu

原文摘要

Low-rank adaptation (LoRA) provides a parameter-efficient approach to continual learning, but its nominal rank can conceal a loss of effective adaptation capacity. We identify \emph{spectral plasticity collapse}: during sequential adaptation, update energy becomes concentrated in a small subset of singular modes, leaving much of the available low-rank space underutilized. This exposes a limitation of interference avoidance alone: protecting historical representations does not ensure that the remaining adaptation capacity is responsive to new tasks or effectively utilized. To address this problem, we propose \texttt{MLoRA}, which jointly controls capacity allocation and utilization. First, historical whitening identifies input directions with strong current-task response relative to accumulated historical response, yielding a task-adaptive basis that remains fixed during training. Second, approximate polar orthogonalization of momentum updates reduces spectral concentration within theselected space. An orthonormal basis connects these mechanisms by transferring the factor-update spectrum exactly tothe induced weight update. We establish a max--min characterization of exact subspace selection and derive cumulative spectral bounds under controlled cross-step anisotropy. Across five class-incremental benchmarks and eight incremental settings, \texttt{MuLoRA} achieves the highest mean accuracy in 15 of 16 reported metrics.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.24209

Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities

Chenming Shang, Yujin Tang, Jun Jie Ou Yang, Ruize Xu, Adam Breuer, Nikhil Singh

原文摘要

The Platonic Representation Hypothesis (PRH) claims that independently trained models converge on a shared statistical model of reality, yet recent work finds that the evidence weakens under some conditions. In this paper, we measure what models share through the directions (displacement vectors) between samples. Under a single orthogonal alignment limited to just rotation and reflection, these displacement vectors are substantially preserved across 44 independently trained vision and language encoders spanning modalities and asymmetric capability pairs, consistent with the PRH. Pointwise similarity, however, is lower. Representations split into a shared semantic component that is linearly aligned across models, and a private component that is not, which we interpret as having model-specific capabilities. We trace this geometry to concept-level structure: within a model, parent concepts are orthogonal to their child variation vectors; across models, concept displacements are parallel. We predict (and experiments confirm) that fine-tuning preserves Pointwise similarity but reduces displacement similarity, and relational distillation does the opposite. Because semantics align linearly but capabilities do not, we propose importing capabilities from one model to another using a single cached forward pass through the source. We call this Shadow Casting. As a proof of concept, our ShadowCLIP instantiation outperforms strong fine-tuned baselines at far less compute. A cache can be released alongside open model weights, letting one model's capabilities be downloaded and imported into other models with frozen backbones.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.09424

MINE: Entropy-coded Memory-efficient Inference for Large Language Models

Emil Schätzle, Markus Püschel

原文摘要

We present MINE (Memory-efficient INference Engine), a lossless entropy coding approach that accelerates single-stream inference of Large Language Models (LLMs), where existing lossless compression methods yield speedups only for specific data types such as BF16 and under additional assumptions. Single-stream decoding is strongly memory-bound, since every weight must be streamed from memory once per generated token and requests cannot be batched in local or low-latency deployments. At the core of MINE is dtANS-LLM, a GPU entropy coding scheme adapted from the prior dtANS to the needs of LLM inference. It exploits the non-uniform distribution of weights, which persists even after quantization, and exposes hyper-parameters that trade compression ratio against decoding cost. We auto-tune these parameters for each tensor and GPU to obtain the Pareto front between model size and decoding latency. MINE integrates into vLLM as a plugin and selects the best kernel configuration for a given memory budget. On Llama 3.1 8B Instruct with a 2.1-bit EntQuant checkpoint, MINE achieves speedups of 2.52x and 1.48x over the fastest baseline on an RTX 4060 Ti and an RTX 5090, at effective bitrates of 2.24 and 3.43 bits per weight. MINE is available as open source.

cs.LG · cs.DC · cs.PF

查看论文
arXiv ↗PDF ↗