跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+2新增论文
+2其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-13 05:02 · 全站最近入库：2026-11-12 06:37各信源进度 →

统计口径与数据新鲜度

统计 2026-11-12 06:03 — 2026-11-13 06:03（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-13 05:02最近发现新增版本：2026-11-12 06:37库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,675 已收录论文18,323 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 4,903 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-13 05:00（北京时间）
版本更新 · v3arXiv:2601.03910

An Algebraic Representation Theorem for Linear GENEOs in Machine Learning

Francesco Conti, Patrizio Frosini, Nicola Quercioli

原文摘要

Geometric and Topological Deep Learning enhance machine learning by exploiting geometric and topological structures in data. In this context, Group Equivariant Non-Expansive Operators (GENEOs) provide a principled framework for encoding symmetries and designing efficient, interpretable neural architectures. Originally introduced in Topological Data Analysis and later adopted in Deep Learning, GENEOs unify geometric and topological approaches and include, as a special case, the operator computing persistence diagrams. Their theoretical foundations rely on group actions, equivariance, and compactness properties of operator spaces. GENEOs are defined between data spaces endowed with transformation groups, referred to as perception pairs. While previous representation results characterized linear GENEOs acting on identical perception pairs, many applications require operators between heterogeneous ones. In this work, we introduce and prove a new algebraic representation theorem for linear GENEOs acting between different perception pairs. Under mild assumptions on data domains and group actions, this result provides a complete characterization of such operators and implicitly yields a computational method to generate all linear GENEOs between two perception pairs. The proof combines tools from the theory of stochastic matrices, group actions, and invariant measures. In our paper, we also show that the space of linear GENEOs is compact and convex, making easier their use for learning and optimization. Our new theorem enlarges the applicability of GENEOs, and allows for their composition with a broader class of current architectures. We show this with an application in image reconstruction.

math.RT · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.34427

LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization

Shihao Zhang, Weiting Liu, Siyu Shao, Yitian Chen, Jianfeng Feng, Dongdong Ge, Yinyu Ye

原文摘要

Scaling LLM-based optimization from textbook-scale instances to real-world, industrial tasks remains a critical open challenge. Existing approaches are predominantly evaluated on small, self-contained textual problems and often commit to a solver-integrated paradigm, limiting their ability to handle the scale and structural diversity of practical optimization workloads. In this work, we propose a practical framework for training open-source LLMs to tackle real-world, industrial-scale optimization. We first show empirically that solver-integrated reasoning, exact combinatorial algorithm, and heuristic search exhibit complementary strengths across different problem structures and scales. Motivated by this, we introduce Strategy-Diverse Reinforcement Learning (SDRL), which trains LLMs as adaptive optimization meta-solvers. SDRL leverages this complementarity through a correctness-gated hierarchical diversity reward that promotes robust exploration across varying strategies and within each strategy, effectively preventing premature strategy collapse. We further introduce a mixed-format training scheme that jointly supports both self-contained textual problems and file-grounded instances. Across comprehensive evaluations, our framework outperforms existing fine-tuned methods and frontier models including DeepSeek-V4-Pro and GPT-5.5, both on average across benchmarks and on industrial-scale optimization tasks. We release our code and training data to facilitate future research https://github.com/tokentideAI/LLMMetaSolver.

cs.LG · cs.CL

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.32740

Strongly Adaptive Online Learning with Time-Varying Movement Costs

Andrew Jacobsen, Emmanuel Esposito, Hao Qiu, Mengxiao Zhang

原文摘要

We study online convex optimization (OCO) with time-varying movement costs, where the learner incurs both a loss and a time-dependent penalty for changing its decisions. While previous work has established dynamic regret guarantees for this setting, obtaining strongly adaptive guarantees that hold simultaneously over arbitrary intervals remains an open challenge. We address this by developing a new algorithm which, for every interval $[a,b]\subseteq[T]$ and arbitrary comparator sequence, achieves a data-dependent regret bound of $ \widetilde{\mathcal{O}}\left( \sqrt{(D^2+DP_{[a,b]})\sum_{t \in [a,b]}(\|g_t\|^2+λ_t\|g_t\|)} \right) $, where $D$ is the diameter of the decision domain, $P_{[a,b]}$ is the path length of the comparator sequence over $[a,b]$, $g_t$ is the gradient of the loss, and $λ_t$ is the movement-cost coefficient at round~$t$. Key to our approach is a novel reduction to sleeping experts with movement costs, a previously unexplored variant that is of independent interest. As a result of our development, we further obtain strongly-adaptive dynamic regret guarantees under arbitrary delayed feedback with a refined dependence on interval-restricted cumulative delays. Our approach also naturally extends to OCO with time-varying memory, where we obtain strongly-adaptive guarantees which improve the best-known bounds.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10705

AbductionBench: Benchmarking Abductive Reasoning in Large Language Models

Moein Salimi, Amir Qeysarbeigi, Babak Hosseini Mohtasham, Faraz Doagoye Tehrani, Mahdi Naieni, Hanieh Sartipi, Ali Nouri, Amin Aghakasiri 等 11 位作者

原文摘要

Abductive reasoning is the inference of plausible explanations for observed evidence. Evaluating this capability requires more than a single task: a system may need to generate an explanation, discriminate among competing hypotheses, revise its hypotheses as evidence arrives, or decide what evidence to seek next. Yet these capabilities are typically evaluated through isolated benchmarks with different task formulations and scoring procedures, so claims that a model is ``good at abduction'' may reflect only a narrow aspect of this capability. We introduce AbductionBench, a unified framework whose taxonomy characterizes abductive tasks. We instantiate this framework on 27 datasets spanning ten domains and evaluate eight LLMs from four model families evaluated on single-choice selection tasks. Beyond harmonizing heterogeneous tasks, AbductionBench separates final-answer evaluation from analysis of the observable reasoning traces and interaction histories produced alongside those answers. We analyze written traces as observable model outputs rather than faithful representations of internal computation, characterizing evidence use, hypothesis comparison, uncertainty, contradiction handling, and answer commitment. Jev matches or exceeds small LLMs when selecting among explanations over compact evidence but falls substantially behind on longer cases, suggesting that candidate discrimination and extended evidence integration place different demands on a system. In interactive tasks, the proportion of actions judged relevant declines at later steps, and a relevant first question is associated with a correct final answer; in the passive task, models rarely revise their initial hypothesis as new evidence arrives. Together, these findings highlight the value of evaluating abductive reasoning through both final answers and observable evidence use, hypothesis revision, and information gathering.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2606.14299

What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

Jiazhen Huang, Xiao Chen, Zhiming Liu, Yaru Sun, Jingyan Jiang, Zhi Wang

原文摘要

Vision-Language Models (VLMs) such as CLIP have become a standard backbone for open-vocabulary recognition, yet their zero-shot predictions remain vulnerable to distribution shifts encountered at deployment. Test-Time Adaptation (TTA) has recently been extended to CLIP as a lightweight solution, leading to a rapidly growing body of TTA4CLIP methods. However, empirical progress in this area has largely outpaced our understanding of what truly drives adaptation, where their gains originate, and under which shifts they remain reliable. In this paper, we take a step back from the pursuit of state-of-the-art accuracy and conduct a systematic controlled study of TTA4CLIP. We first organize existing methods into three unified paradigms according to what is updated at test time. We then introduce TTABC, an open-source TTA Benchmark for CLIP, which standardizes evaluation protocols and integrates more than 20 representative methods. Our controlled empirical analysis focuses on three key areas. First, we determine the driving factors in parameter-based methods, revealing that adaptation gains are primarily driven by test-time evidence and reliable proxies rather than heavy optimization. Second, we explore evidence utilization beyond heavy parameter tuning, showing that competitive and efficient performance can be achieved through cross- or current-sample evidence and lightweight prototype updates. Finally, we demonstrate that there is no silver bullet for TTA: no single adaptation paradigm is universally optimal, and the preferred paradigm depends on the nature of shift. We hope our benchmark and study provide a clearer understanding of the current TTA4CLIP landscape and establish a foundation for further research. The project page is available at https://walawalagoose.github.io/TTABC/

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2512.13935

Informing Acquisition Functions via Foundation Models for Molecular Discovery

Qi Chen, Fabio Ramos, Alán Aspuru-Guzik, Florian Shkurti

原文摘要

Bayesian Optimization (BO) is a key methodology for accelerating molecular discovery by estimating the mapping from molecules to their properties while seeking the optimal candidate. Typically, BO iteratively updates a probabilistic surrogate model of this mapping and optimizes acquisition functions derived from the model to guide molecule selection. However, its performance is limited in low-data regimes with insufficient prior knowledge and vast candidate spaces. Large language models (LLMs) and chemistry foundation models offer rich priors to enhance BO, but high-dimensional features, costly in-context learning, and the computational burden of deep Bayesian surrogates hinder their full utilization. To address these challenges, we propose a likelihood-free BO method that bypasses explicit surrogate modeling and directly leverages priors from general LLMs and chemistry-specific foundation models to inform acquisition functions. Our method also learns a tree-structured partition of the molecular search space with local acquisition functions, enabling efficient candidate selection via Monte Carlo Tree Search. By further incorporating coarse-grained LLM-based clustering, it substantially improves scalability to large candidate sets by restricting acquisition function evaluations to clusters with statistically higher property values. We show through extensive experiments and ablations that the proposed method substantially improves scalability, robustness, and sample efficiency in LLM-guided BO for molecular discovery.

cs.LG · cs.AI · q-bio.QM

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.30258

Temporal Gradient Inversion for Private Trajectory Reconstruction in Embodied Reinforcement Learning

Sudip Bhujel, Shanghao Shi, Ruiquan Huang, Ning Zhang, Yang Xiao

原文摘要

Distributed learning in embodied reinforcement-learning agents offers a degree of privacy by retaining raw sensor data on-device and transmitting only policy gradients to the server. Yet temporal structure can amplify this leakage beyond single-frame attacks. We introduce Temporal Reconstruction Attack on Consecutive Encodings (TRACE), an amortized temporal gradient-inversion attack that autoregressively reconstructs the sequence of private observation-action trajectories from per-step policy-learning gradients. The attack exploits two structural signals ignored by prior single-frame methods: (i) cross-time correlation between successive embodied gradients, which we formalize via a conditional mutual-information bound, and (ii) closed-form action recovery from policy-head gradient structure, which we prove exact when standard entropy regularization is sufficiently small. On held-out embodied scenes, TRACE reaches $18.8$ dB PSNR with near-perfect action recovery at $3$-$4.5$ ms per reconstructed frame, dominating the learning-based baseline across all reconstruction metrics and exceeding optimization attacks while running orders of magnitude faster. Further evaluation demonstrates TRACE's broader applicability across recurrent, residual, and compact transformer victim architectures, multi-modal inputs, and larger discrete action spaces. Defense experiments suggest that protecting temporal gradient streams may require sequence-aware privacy mechanisms.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.10163

Coarsening Linear Non-Gaussian Causal Models with Cycles

Francisco Madaleno, Francisco C. Pereira, Alex Markham

原文摘要

Recent work on causal abstraction, in particular graphical approaches focusing on causal structure between clusters of variables, aims to summarize a high-dimensional causal structure in terms of a low-dimensional one. Existing methods for learning such summaries from data assume that both structures are acyclic. Acyclicity is helpful for causal effect identification and reasoning, but it excludes the many systems of scientific interest that contain feedback loops. We show that in the linear non-Gaussian (LiNG) setting, the acyclicity assumption on the high-dimensional structure can be relaxed while still allowing recovery of a low-dimensional causal directed acyclic graph (DAG). LiNG models with cycles are observationally identifiable only up to an equivalence class whose members differ only within their feedback loops. Our low-dimensional DAG, which is invariant across all members of this class, is therefore identifiable from observational data alone. While existing approaches for learning this equivalence class have exponential time complexity, our low-dimensional summary is learned in worst-case cubic time and comes with finite-sample guarantees. We provide open source code and experiments on synthetic data to corroborate our theoretical results.

stat.ML · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10700

Spectral Bias Reduction in Local Gated ReLU Architectures

Victorita Dolean

原文摘要

Neural networks trained by gradient-based methods learn the low-frequency content of a target function faster than its high-frequency content. This spectral bias is well documented, but it is much less clear which architectural choices mitigate it and which cannot. We study the question for shallow ReLU models linear in their trainable parameters, where each mode is learned at a rate given by a Gram-matrix eigenvalue, and for a general symmetric bilinear form $a(\cdot,\cdot)$, so that least-squares regression ($a=$ the $L^2$ inner product) and variational or physics-informed losses ($a=$ the $H^1$ energy form) are two instances of one analysis. Within this framework we propose gated local ReLU dictionaries, obtained by multiplying each ReLU feature by a hat partition-of-unity weight, in the spirit of partition-of-unity finite elements and overlapping Schwarz methods. The resulting Gram matrices are block-banded for any local bilinear form, with a block-Toeplitz interior whose matrix-valued symbol governs the spectrum. The two forms then behave very differently. For $L^2$ regression, the symbol is bounded away from zero, the ratio is uniform in the number of windows, and the measured spread of frequency-dependent learning rates falls from $1.9\times10^{5}$ to $4.5$ at matched model size. For the $H^1$ stiffness matrix, the symbol vanishes at zero frequency: gating leaves the $Θ(N^2)$ conditioning intact and inverts the frequency ordering instead of removing the bias. A preconditioner built from the same interior stencil will fix the spectral bias, showing that architecture alone is not enough. Experiments on $1$D regression and Poisson problems confirm the dichotomy, and the regression effect persists when the hidden-layer parameters are trained as well.

cs.LG · math.NA

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.05227

Regret Equals Integrated Covariance: Exact Identities for the Value of Information in Stochastic Optimization

Irene Aldridge

原文摘要

For a stochastic program $\min_{z\in Z} c^T z + h(z)$ with a random cost vector $c$, regret, or the expected value of perfect information, is usually computed by re-solving the problem scenario by scenario. We show that regret is determined by the covariance between the cost vector and the optimal decision $π^s(c)$. The identity is exact and holds without regularity assumptions once the covariance is integrated along the path from the mean to each scenario. With $c_s=E[c]+s(c-E[c])$, $Reg=-\int_0^1 s^{-1} Cov(c_s,π^s(c_s))\,ds$. Three consequences follow. First, the plain covariance is always an upper bound: $0\le Reg\le-Cov(c,π^s(c))$. Second, when the optimal decision rule is affine, as in unconstrained and equality-constrained (e.g., budget-constrained Markowitz) quadratic programs, regret equals exactly one half of the covariance. For purely linear objectives, an affine rule forces zero regret. Third, the ratio of regret to covariance lies in $[0,1]$ and is governed by the shape of the covariance profile along the path. The noise level at which this profile first becomes positive is a decision-stability margin that tightens the bound. The identity also holds scenario by scenario for any predicted cost, so it applies to linear and combinatorial programs with discontinuous decision rules. We give dimension-free concentration bounds and a central limit theorem for the plug-in estimator. Across 7 problem classes, 8 cost distributions, and the shortest-path benchmark of Elmachtoub and Grigas (2022) with $10^6$ draws per configuration, the identities hold to within quadrature error, and the covariance bound holds in every scenario.

econ.EM · cs.IT · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.33012

Recursive Semantic Divergence for LLM Agent Consistency

Harshavardhan Abichandani, Penny Chong, Atin Ghosh, Daniel Dahlmeier

原文摘要

Large Language Model (LLM) agents with tool use can behave inconsistently across independent runs from identical states. They may call different tools, produce contradictory outputs, or follow divergent strategies.This variability compounds over time and undermines deployment reliability. Existing evaluation metrics primarily measure task correctness via completion-based scores and do not capture consistency in agent behavior across executions. Separately, agent behavior is analyzed either at the token level using probability distributions or at the single-response level using semantic clustering, but neither captures how inconsistency propagates across future turns of a trajectory. We introduce Recursive Semantic Divergence (RSD), a trajectory-level consistency metric adapted from bisimulation metrics in reinforcement learning. RSD measures the expected cumulative semantic disagreement between two independent rollouts of the same agent from the same state, using natural language inference to detect logical inconsistency. It is defined recursively over future states and approximated by a neural network trained on independent rollouts. The learned metric estimates future divergence from the current state, providing a consistency score at each turn without requiring the full trajectory to complete. When used as a reward signal for fine-tuning, RSD reduces AUC coefficient of variation (CV) by up to 30.9% on tau2-bench and 50.5% on ToolSandbox. Across Qwen-2.5 and Qwen-3.5 models, RSD reduces AUC CV in seven out of eight model-benchmark settings, while improving the final task progress in all eight settings.

cs.CL · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2511.15856

GLOBE: Accurate Surrogates for Boundary-Driven PDEs via Domain-Inspired Architectures and Equivariance

Peter Sharpe

原文摘要

We introduce GLOBE, a nonlinear neural surrogate for boundary-driven, homogeneous, weakly nonlinear PDEs that draws inductive bias from boundary-element methods and equivariant ML, representing solutions as superpositions of learnable Green's-function-like kernels evaluated from boundary faces to targets. The architecture is translation-, rotation-, and parity-equivariant; discretization-invariant; and units-invariant via rigorous nondimensionalization. On AirFRANS (steady incompressible RANS over NACA airfoils), a compact ~117k-parameter GLOBE model has lower mean-squared error than published values for the strongest reported ML surrogates on both the interpolation and low-data splits, in single-run comparisons under the released AirFRANS normalization. The model evaluates fields at arbitrary points, handles non-watertight meshes, and generalizes under Reynolds number and angle-of-attack distribution shifts. We further introduce a hierarchical Barnes-Hut acceleration that exploits the kernel's far-field decay to reduce evaluation cost from quadratic to near-linear in the number of boundary elements, with a tunable opening angle that recovers exact dense evaluation in the zero-angle limit. Quantitative validation focuses on 2D AirFRANS; on 3D DrivAerML cars, GLOBE predicts surface and volume fields on the full meshes, and we compare it with a matched Transolver baseline. Together, these results show that boundary-element-inspired inductive biases can substantially improve the accuracy of ML surrogates for boundary-driven PDEs while scaling to large 3D meshes.

cs.LG · physics.flu-dyn

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2608.18495

Physics-Unrolled Neural Operator for Accelerating High-Fidelity Wireless Ray Tracing

Rafid Umayer Murshed, Saif Ur Rahman, Mingyue Tang, Elahe Soltanaghai

原文摘要

Site-specific wireless design relies on ray tracing to predict how signal strength varies across a floorplan, but a converged indoor radio map takes tens of seconds to simulate and planning a single building requires thousands of them. A learned surrogate can remove most of that cost, yet training one runs into a second problem: high-fidelity labels are exactly what is too expensive to produce at scale, so the affordable labels come from finite-ray simulations that carry residual Monte Carlo noise. We address this with RayNO, a physics-unrolled neural operator that reconstructs the converged output of a ray tracer from one cheap low-fidelity run and scene priors. Rather than treating a radio map as a generic image, RayNO refines the field through three stages matched to reflection, diffraction, and scattering, each supervised by a label of matching physical complexity. We prove that when the label error is conditionally unbiased the learned operator can end up closer to the high-fidelity reference than the labels it was trained on, and we verify that condition directly in the simulator using repeated independent ray seeds. Across diverse floorplans, RayNO outperforms image-to-image baselines, wireless learning models, monolithic neural operators, and classical multi-fidelity regression on both image-quality and wireless deployment metrics, while reducing the cost of a high-fidelity map from roughly 28 s to 0.11 s.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10691

A Geometric Theory for Nonequilibrium Transformer Dynamics

Aythami Bethencourt de León, Darryl D. Holm

原文摘要

We develop a variational and geometric theory based on statistical physics for the joint evolution of Transformer attention and token representations at fixed model parameters. To this end, we introduce a free energy for the attention and model the evolution of the attention weights by its Fisher--Rao gradient flow. At fixed token representations, we prove that the softmax matrix is the unique free-energy minimiser and that every positive attention trajectory converges to it. For prescribed time-dependent representations, we show that the attention dynamics becomes linear after a change to logit variables. This result yields tracking bounds relative to the instantaneous softmax and sufficient conditions for attention to converge to a fixed matrix. Hamiltonian dynamics for the representations and their momenta are then coupled to the attention relaxation driven by the same energy. This coupling results in an energy-dissipation law. We also develop two alternative Hamiltonians incorporating the attention value map. We prove global well-posedness of the attention flow and all three coupled systems.

cs.LG · math-ph · math.PR · nlin.AO · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2603.28764

Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds

N Alex Cayco-Gajic, Arthur Pellegrino

原文摘要

Similarity measures such as centred kernel alignment (CKA) or representational similarity analysis (RSA) are widely used to compare the representational geometries employed by different neural networks to solve the same task. Yet, because existing methods compare the extrinsic geometry of representations in state space, rather than their intrinsic geometry, they may fail to capture subtle yet crucial distinctions between fundamentally different computations. Here, we introduce metric similarity analysis (MSA), a novel method which leverages tools from Riemannian geometry to compare the intrinsic geometry of neural representations under the manifold hypothesis. We show that MSA can be used to (i) disentangle neural computations of deep networks in rich vs. lazy learning regimes, (ii) compare the dynamics of recurrent neural networks and state-space models, and (iii) investigate the statistical manifolds of diffusion models. Overall, we introduce a mathematically grounded and broadly applicable framework to understand the mechanisms behind neural computations by contrasting their intrinsic geometries.

cs.LG · cs.AI · math.DG · q-bio.NC

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10684

Audit Before You Merge: Provenance, Probing, and Continual LoRA Composition

Lenny Aharon, Neta Glazer, Lior Aharon

原文摘要

Post-hoc merging of public LoRA adapters is an attractive gradient-free path to continual adaptation, but reliable measurement of catastrophic forgetting in this setting depends on infrastructure that prior work has not addressed. We treat each adapter merge as a sequential task in a continual-learning setting, with the instruction-tuned base model's capabilities as Task 0, and study forgetting on Llama-3.1-8B-Instruct. Public adapter hubs contain modules that fail basic provenance checks: silent base-version mismatches and near-zero delta norms whose inclusion inflates measured interference by ~10pp. We propose a two-check audit (base-model match, non-trivial delta norm) and a leakage-free probing protocol as preconditions for any forgetting study in this setting. On an audited 4-adapter set, naive merging causes -10.5pp on GSM8K and -18.3pp on HumanEval, while Task Arithmetic at lambda=0.5 sits within a standard error of the unmerged baseline. A coordinate search including negative coefficients identifies the math adapter as probe-optimal at lambda=-0.5, outside the reach of positive-only mergers, suggesting forgetting has structure that negation-aware merging may exploit. The same protocol replicates on Mistral-7B-Instruct-v0.3, where 2 of 4 candidate adapters fail the audit and the audited merge reproduces the same merge-method ranking on GSM8K, supporting that our findings are not Llama-specific.

cs.AI · cs.CR · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2512.23818

Energy-Tweedie: Score meets Score, Energy meets Energy

Andrej Leban

原文摘要

Denoising and score estimation are classically linked through Tweedie's formula, which relates the posterior mean under Gaussian noise to the Stein score of the noisy marginal. In this work, we extend this perspective beyond Gaussian noise to a broad class of Gibbs (energy-based) noise distributions, with the generalized Gaussian family as an example. We derive the Energy-Tweedie identity: when the denoising posterior is viewed through the lens of scoring rules, the path derivative of a kernel scoring rule defined by the noise potential recovers the Stein score of the noisy marginal. Thus, the familiar correspondence between Gaussian noise, posterior means, squared loss, and Tweedie's formula is lifted to a distributional correspondence between Gibbs noise distributions, full posterior laws, kernel scoring rules, and the Energy-Tweedie identity, which gives a Tweedie-style relation for each noise potential. Among its consequences, this identity gives a posterior-samples-to-score route to score estimation, yields a principled criterion for estimating unknown noise parameters, and enables diffusion-style sampling along user-chosen paths through the noise-parameter space, supplying the score-based perspective on recent generative methods trained with scoring rules.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.10654

Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weights

Jixiang Qing, Henry Moss, Matthias Sachs

原文摘要

We consider the active learning problem where the goal is to learn an unknown function with low prediction error under an unknown Boltzmann distribution induced by the function itself. This self-induced weighting arises naturally in problems such as potential energy surface (PES) modeling in computational chemistry, yet poses unique challenges as the target distribution is unknown and its partition function is intractable. We propose \texttt{AB-SID-iVAR}, a Gaussian Process-based acquisition function that approximates the intractable Bayesian target distribution in closed form while avoiding partition function estimation, and is applicable to both discrete and continuous input domains. We also analyze a Thompson sampling alternative (\texttt{TS-SID-iVAR}) as a higher variance Monte Carlo variant. Despite the unknown target, under mild conditions, we establish that the terminal prediction error vanishes with high probability, and provide a tighter average-case guarantee. We demonstrate consistent improvements over existing approaches in this setting on synthetic benchmarks and real-world PES modeling and drug discovery tasks. Our code is available at https://github.com/TsingQAQ/SIDAL

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.08740

PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise

Mihaly Petreczky, Mohamad Al Ahdab, John Leth

原文摘要

In this paper we derive a Probably Approximately Correct (PAC)-Bayesian error bound for partially observed linear time-invariant (LTI) stochastic dynamical systems in state-space form with inputs and sub-Gaussian noise. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. The bound derived in this paper relates the expectation of prediction errors with the prediction error generated by the model on the data used for learning. In addition, we show that it can also be used to derive bounds for the parameter estimation error. In turn, this allows us to provide finite-sample error bounds for the prediction error and parameter estimation error for a wide class of system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.

cs.LG · math.OC

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10677

Safe Offline Reinforcement Learning using Behavior Regularisation and Latent Feasibility Guidance

Mahesh Keswani, Parth Bhardwaj, Ankur Kumar, Raunak P. Bhattacharyya

原文摘要

Safe offline reinforcement learning (RL) seeks to learn policies that maximize return while satisfying safety constraints, using a static dataset. To characterize state-wise safety, reachability analysis identifies the feasible region, defined as the set of states from which constraints can be satisfied at every future step. Policies are commonly extracted from learned value functions with advantage-weighted regression (AWR). However, AWR relies on exponentiated weights that amplify value estimation errors, whereas behavior-regularized policy gradient methods directly utilize the action gradient of the value function and often make more effective use of learned critics. We propose Feasibility-Optimized Conditional Actor Learning (FOCAL), a method that incorporates behavior-regularized policy gradients into a conditional variational autoencoder (CVAE) actor. FOCAL optimizes this policy using a first-order objective partitioned by feasibility, updating the actor to increase return on transitions estimated to be feasible and to reduce constraint violations on transitions estimated to be infeasible. During inference, FOCAL dynamically adjusts the scale of the latent prior according to the estimated feasibility of the state to adjust the conservatism of action sampling. Theoretically, we show that for AWR, the policy update variance depends exponentially on the advantage error, whereas for FOCAL it scales linearly with the variance of the value estimation error. Across the DSRL benchmark, FOCAL satisfies the constraints on all evaluated tasks while achieving competitive or higher returns compared to baselines.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2604.08039

LINE: LLM-based Iterative Neuron Explanations for Vision Models

Vladimir Zaigrajew, Michał Piechota, Gaspar Sekula, Paweł Gelar, Przemysław Biecek

原文摘要

Interpreting the concepts encoded by individual neurons in deep neural networks is a crucial step towards understanding their complex decision-making processes and ensuring AI safety. Despite recent progress in neuron labeling, existing methods often limit the search space to predefined concept vocabularies or produce overly specific descriptions that fail to capture higher-order, global concepts. We introduce LINE, a novel, training-free iterative approach tailored for open-vocabulary concept labeling in vision models. Operating in a strictly gray-box setting, LINE leverages a large language model and a text-to-image generator to iteratively propose and refine concepts in a closed loop, guided by activation history. We demonstrate that LINE achieves state-of-the-art performance across multiple model architectures, yielding AUC improvements of up to 0.15 on ImageNet and 0.05 on Places365, while discovering, on average, 25% of new concepts missed by predefined vocabularies. Beyond identifying the top concept, LINE provides a complete generation history, enabling polysemanticity evaluation and producing supporting visual explanations that rival gradient-dependent activation maximization methods. The LINE source code is available at https://github.com/CredibleAI/LINE

cs.CV · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2505.06169

On the Depth of Monotone ReLU Neural Networks and ICNNs

Egor Bakaev, Florestan Brunck, Christoph Hertrich, Daniel Reichman, Amir Yehudayoff

原文摘要

We study two models of $\mathsf{ReLU}$ neural networks: monotone networks ($\mathsf{ReLU}^+$) and input convex neural networks ($\mathsf{ICNN}$). Our focus is on expressivity, mostly in terms of depth, and our motivation is to gain a better understanding of the depth required to exactly represent functions using neural networks with ReLU activations, a subject that has received a lot of attention lately. We prove several lower bounds on the depth required to represent functions by monotone ReLU networks and $\mathsf{ICNN}$. For the maximum function $\mathsf{MAX}_n$ computing the maximum of $n$ real numbers, we show that $\mathsf{ReLU}^+$ networks cannot compute $\mathsf{MAX}_n$, or even approximate it. We prove a sharp lower bound of $n-1$ on the $\mathsf{ICNN}$ depth complexity of $\mathsf{MAX}_n$. We also prove depth separations between $\mathsf{ReLU}$ networks and $\mathsf{ICNN}$s; for every $k$, there is a depth-$2$ $\mathsf{ReLU}$ network of size $O(k^2)$ that cannot be simulated by a depth-$k$ $\mathsf{ICNN}$. The proofs are based on deep connections between neural networks and polyhedral geometry, and also use isoperimetric properties of triangulations.

cs.LG · cs.DM · cs.NE · math.CO

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10668

Causal Heteroscedastic Structure Learning From Incomplete Temporal data

Abdellah Rahmani, Pascal Frossard

原文摘要

Discovering causal structures in multivariate time series (MTS) is critical in domains such as finance and neuroscience, yet remains challenging in practice due to three distinct obstacles. First, MTS are routinely corrupted by missing values, where naive impute-then-discover pipelines distort the underlying data distribution and yield spurious causal graphs. Second, identifying instantaneous effects is difficult even on complete data, and missingness further entangles imputation with graph recovery. Third, real-world data frequently exhibits heteroscedastic noise with variance driven by both instantaneous and lagged causes, violating the core assumptions of standard discovery methods. Existing approaches fail to address these challenges concurrently, typically presuming complete data, homoscedastic noise, or strictly time-lagged relationships. Crucially, we show through ablation studies that resolving these issues sequentially, rather than jointly, leads to a substantial degradation in performance. To overcome this, we introduce Cheesefill, an optimal transport framework for causal structure learning under missingness and heteroscedastic noise. Cheesefill parameterizes causal mechanisms using conditional normalizing flows and jointly learns a stochastic correction map that refines naive imputations into trajectories consistent with the inferred causal mechanisms. Theoretically, we prove that optimizing over stochastic correction maps is equivalent to a conditional kernel reformulation of the Kantorovich problem, from which our neural parameterization yields a tractable objective. Extensive experiments on synthetic and real-world benchmarks demonstrate that Cheesefill consistently outperforms existing baselines across diverse missingness regimes.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2605.06377

Independent Learning of Nash Equilibria in Partially Observable Markov Potential Games with Decoupled Dynamics

Philip Jordan, Maryam Kamgarpour

原文摘要

We study Nash equilibrium learning in partially observable Markov games (POMGs), a multi-agent reinforcement learning framework in which agents cannot fully observe the underlying state. Prior work in this setting relies on centralization or information sharing, and suffers from sample and computational complexity that scales exponentially in the number of players. We focus on a subclass of POMGs with independent state transitions, where agents remain coupled through their rewards, and assume that the underlying fully observed Markov game is a Markov potential game. For this class, we present an independent learning algorithm in which players, observing only their own actions and observations and without communication, jointly converge to an approximate Nash equilibrium. Due to partial observability, optimal policies may in general depend on the full action-observation history. Under a filter stability assumption, we show that policies based on finite history windows provide sufficient approximation guarantees. This enables us to approximate the POMG by a surrogate Markov game that is near-potential, leading to quasi-polynomial sample and computational complexity for independent Nash equilibrium learning in the underlying POMG.

cs.GT · cs.LG · cs.MA

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2607.01022

Seahorse: A Unified Benchmarking Framework for Spatiotemporal Event Modeling

Yahya Aalaila, Sebastian Vollmer, Gerrit Großmann

原文摘要

Spatiotemporal point processes (STPPs) model event data in continuous time and space, with applications in mobility, epidemiology, and public safety. Recent neural STPPs span expressive intensities, density models, continuous-time latent dynamics, normalizing-flow spatial decoders, and score-based generative mechanisms. Yet comparison remains fragile because implementations differ in preprocessing, coordinate normalization, splits, likelihood conventions, and evaluation protocols. We present Seahorse, a unified framework for reproducible training and evaluation of STPPs. Seahorse provides an encode-evolve-decode interface and an executable benchmarking contract that standardizes dataset handling, tuning, raw-coordinate likelihood reporting, and run artifacts across classical baselines, neural likelihood models, continuous-time Neural STPP variants, and sample-based generative models. On 13 real datasets and 19 presets, no model dominates: likelihood, calibration, and next-event prediction favor different models, and the strongest models take from minutes to hours to train. Using HawkesNest as a controlled synthetic stress-test suite, we show how diagnostic benchmarking can probe neural STPP behavior under increasing spatiotemporal entanglement. Seahorse provides reusable infrastructure for comparing, diagnosing, and extending STPP models beyond implementation-specific leaderboards. The implementation, including preset configurations and reproduction scripts, is available at https://github.com/YahyaAalaila/seahorse.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2602.09297

Laplacian Heads Improve Transformers by Smoothing Token Representations

Yuchong Zhang, Vardan Papyan

原文摘要

Transformers update token representations through multi-head attention and residual connections: $X \leftarrow X + \sum_{i} P^{(i)}\text{Norm}(X)W_{V_i}W_{o_i}$, where $P^{(i)}$ is the softmax probability matrix in head $i$. We propose replacing some $P^{(i)}$'s with $I - P^{(i)}$, giving $X \leftarrow X + \sum_{i \in \mathcal{A}} P^{(i)}\text{Norm}(X)W_{V_i}W_{o_i} + \sum_{i \in \mathcal{L}} (I - P^{(i)})\text{Norm}(X)W_{V_i}W_{o_i}$. We call heads that use $I - P^{(i)}$ Laplacian heads. Our proposal has two motivations. First, it allows attention heads to update the mean of token representations, while Laplacian heads can directly control within-sequence variance. Second, if tokens are viewed as nodes in a graph with normalized edge weights $P^{(i)}$, then $I - P^{(i)}$ is a graph Laplacian operator, and the update can be interpreted as one step of heat diffusion on the graph. We show that this simple modification improves performance across supervised learning, causal language modeling, and vision self-supervised learning tasks. To investigate why, we examine the token representations learned with and without Laplacian heads. In supervised learning, Laplacian heads collapse token representations within the same sequence and align the sequence means with the geometry of Neural Collapse. In language modeling, they increase the separability of token representations that share the same next-token prediction. In vision self-supervised learning, they produce token representations whose top principal components better capture segmentation structure. Across modalities, they also lead to faster-decaying spectra, indicating stronger token smoothing. Overall, our findings challenge the prevailing view that token oversmoothing is inherently harmful, showing instead that certain forms of smoothing can be beneficial.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2605.15960

Imperfect World Models are Exploitable

Logan Mondal Bhamidipaty, Esmeralda S. Whitammer, David Abel, Mykel J. Kochenderfer, Subramanian Ramamoorthy

原文摘要

We propose a novel definition of model exploitation in reinforcement learning. Informally, a world model is exploitable if it implies that one policy should be strictly preferred over another while the environment's true transition model implies the reverse. We analogize our definition with a prior characterization of reward hacking but show that the associated proof of inevitability does not transfer to exploitation. To overcome this obstruction, we develop a general theory of reward hacking and model exploitation that proves that exploitation is essentially unavoidable on large policy sets and yields the corresponding claim for hacking as a special case. Unfortunately, we also find that the conditions that guarantee unhackability in finite policy sets have no counterpart that precludes exploitation. Consequently, we introduce a relaxed notion of exploitation and derive a safe horizon within which it can be avoided. Taken together, our results establish a formal bridge between reward hacking and model exploitation and elucidate the limits of safe planning in world models.

cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2503.07324

Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamics

Zhiyu He, Saverio Bolognani, Florian Dörfler, Michael Muehlebach

原文摘要

Distribution shifts have long been regarded as troublesome external forces that a decision-maker should either counteract or conform to. An intriguing feedback phenomenon termed decision dependence arises when the deployed decision affects the environment and alters the data-generating distribution. In the realm of performative prediction, this is encoded by distribution maps parameterized by decisions due to strategic behaviors. In contrast, we formalize an endogenous distribution shift as a feedback process featuring nonlinear dynamics that couple the evolving distribution with the decision. Stochastic optimization in this dynamic regime provides fertile ground to examine the various roles played by dynamics in the composite problem structure. To this end, we develop an online decision-making algorithm that both adapts to and shapes the dynamic distribution. Throughout the paper, we adopt a distributional perspective and demonstrate how this view facilitates characterizations of distribution dynamics and the optimality and generalization performance of the proposed algorithm. We showcase the theoretical results in an opinion dynamics context, where an opportunistic party maximizes the affinity of a population with evolving, polarized opinions, and in a recommender system scenario, featuring performance optimization with discrete distributions in the probability simplex.

math.OC · cs.LG · eess.SY

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2505.21938

Practical Adversarial Attacks on Stochastic Bandits via Fake Data Injection

Qirun Zeng, Eric He, Richard Hoffmann, Xuchuang Wang, Jinhang Zuo

原文摘要

Adversarial attacks on stochastic bandits have traditionally relied on some unrealistic assumptions, such as per-round reward manipulation and unbounded perturbations, limiting their relevance to real-world systems. We propose a more practical threat model, Fake Data Injection, which reflects realistic adversarial constraints: the attacker can inject only a limited number of bounded fake feedback samples into the learner's history, simulating legitimate interactions. We design effective attack strategies under this model, explicitly addressing both magnitude constraints (on reward values) and temporal constraints (on when and how often data can be injected). Our theoretical analysis shows that these attacks can mislead a class of bandit algorithms into selecting a target arm in nearly all rounds while incurring only sublinear attack cost. Experiments on synthetic and real-world datasets validate the effectiveness of our strategies, revealing vulnerabilities in stochastic bandit algorithms under practical adversarial scenarios.

cs.LG · cs.AI · cs.CR

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10658

The Delta Gap: Cellular World Models for Perturbation Response Prediction Are Far Harder Than Global Correlation Implies

Junhui Gao

原文摘要

Cellular world models aim to predict how cells respond to genetic perturbations, analogous to world models in reinforcement learning. Perturbation response models are evaluated almost exclusively by global Pearson correlation between predicted and observed transcriptomes. We show this metric is dominated by thousands of non-responsive genes and severely inflates apparent performance. On the Replogle K562 screen (509 unseen perturbations, 5000 genes), GEARS achieves global Pearson 0.977, but its delta Pearson is only 0.076. A trivial baseline predicting mean training delta achieves 0.267, which is 3.5x better than GEARS. Our primary contribution is methodological: we identify and quantify this delta gap as a systematic evaluation artifact, and propose delta-space evaluation as the primary benchmark. We provide two post-hoc corrections (RWR on co-expression graphs and kNN matching) and show that 2.8x of the remaining gap lies in perturbation representation geometry, not decoder capacity. We further show the gap is partially addressable end-to-end: delta-aligned training (lambda = 3.0) raises delta Pearson to 0.179 on Replogle (2.35x over GEARS) and surpasses the trivial baseline on direction accuracy (0.680 vs 0.567) and gene recovery (1.52 vs 1.19); combined with kNN it reaches 0.217. An amplitude-calibration variant (eta = 0.3) further raises gene recovery to 2.16 while reaching 0.218. Critically, these representations transfer zero-shot across cell lines: from Replogle K562 to unseen Norman A549, the delta-aligned model reaches 0.328 on 741 shared genes, 4.7x a trivial baseline and 2.1x standard GEARS. This is the first systematic quantification of the delta gap, and the first diagnostic framework localizing the residual gap to perturbation representation geometry via an oracle upper bound.

cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗