跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1673新增论文
+1257其中 AI 相关
+198已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-14 02:49 · 全站最近入库：2026-11-13 20:44各信源进度 →

统计口径与数据新鲜度

统计 2026-11-13 03:19 — 2026-11-14 03:19（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-14 02:49最近发现新增版本：2026-11-13 20:44库内论文最新更新时间：2026-11-13 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

23,348 已收录论文19,580 AI 相关6,727 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 5,444 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-14 02:48（北京时间）
新论文 · v1arXiv:2611.10397

Regularization Amplifies Subliminal Learning

Lawrence Feng, Jacob Mitchell Springer, Gaurav R. Ghosal, Ziqian Zhong, Aditi Raghunathan

原文摘要

Recent work has shown that hidden biases can be transferred through data with no apparent semantic relationship to those biases. Can regularization limit this covert influence? By constraining adaptation, regularization might be expected to suppress behavioral changes on seemingly unrelated inputs. We systematically study this hypothesis in subliminal learning, as well as in related settings of covert influence, including phantom transfer and logit-linear selection. We find the opposite: regularization amplifies the transfer of hidden biases. In particular, subliminal learning is typically absent under full fine-tuning, but low-rank adaptation or explicit penalties can induce strong transfer. Phantom transfer and logit-linear selection show similar trends. Adding more data, an implicit form of regularization, also strengthens covert influence. Together, these findings suggest that the simplicity biases induced by regularization may facilitate covert influence. We further hypothesize that the level of regularization that maximizes covert transfer depends on task complexity.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.41191

When AI Investigators Cannot Trust Their Tools: Conflict-Aware Evidence Verification for Cyber Threat Investigation

Yuchen Zhang

原文摘要

LLM agents increasingly triage security alerts by combining the outputs of several detection tools, implicitly treating those tools as trustworthy. Unlike prior work on conflicts between a model's knowledge and its context, we study conflicts among the tools themselves: what happens when this machine-generated evidence is conflicting or wrong. On the corrected CIC-IDS2017 dataset, four heterogeneous detectors emit labels and confidences that we corrupt under a threat model varying the number of corrupted tools, the persistence of the corruption, and the confidence attached to wrong outputs. Across open-weight LLMs from 3B to 14B parameters, unverified investigators are fragile: one corrupted detector out of four lowers naive accuracy to .50-.72, three lower every unverified investigator to .13 or less, and honest disagreement alone makes them abstain on up to 63% of flows. We propose Conflict-Aware Evidence Verification (CAV), built from established components: an equal-weight vote that ignores reported confidences, an evidence-conflict gate, and an independent nearest-neighbour trust check that tests the leading candidates. With its prompt frozen before testing, CAV-LLM improves on the naive investigator in every corrupted condition ($p < 0.001$), reaching .84-.88 with three corrupted detectors against at most .08, while calling the LLM on 0.4% of clean traffic. A fixed verification rule is as accurate or better unless most tools are corrupted. When we make the independent channel itself noisy, CAV inherits its errors: it stays ahead when two or more tools are corrupted, but adding the channel as an extra voter is better when at most one is. Verification thus moves trust from the tools to the check, which must be measured and protected in turn.

cs.CR · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v11arXiv:2208.00335

Interpretable Machine Learning: Can the EnglishLike Programming Language be the Future of ChatIPC and ChatIPCv2?

Caleb Princewill Nwokocha

原文摘要

The development of interpretable machine learning requires more than the ability to produce predictions accompanied by explanations. It requires computational systems whose internal representations, decision procedures, constraints, and execution paths can be inspected, reproduced, and modified by humans. This paper investigates whether EnglishLike, an English-like, object-oriented programming language implemented in C++17, could provide the symbolic programming and execution foundation for the ChatIPC and its subsequent implementation, ChatIPCv2. The investigation is based on a comparative examination of the publicly available ChatIPC.cpp and ChatIPCv2.cpp implementations in the PCNTechnologyEnterprise Hugging Face repository, together with the EnglishLike 2.2 implementation available in the associated project materials. ChatIPC combines incrementally learned token-transition relationships with dictionary derived semantic information, part-of-speech information, Jaccard similarity, grammatical heuristics, repetition penalties, and deterministic candidate selection. ChatIPCv2 introduces a different generation strategy based on token identifiers, historical-context bitsets, multi-depth graph exploration, n-gram backoff, and parallel candidate evaluation. EnglishLike provides a complementary computational abstraction: a structured language for expressing declarations, functions, classes, interfaces, control flow, and executable rules. Its compiler, bytecode verifier, and virtual machine provide a potentially useful foundation for representing and executing symbolic reasoning procedures.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10393

Finite Time Analysis of Risk-Sensitive RL via Noisy Power Iteration

Waqar Mirza, Yashaswini Murthy, Laixi Shi, Eric Mazumdar, Adam Wierman

原文摘要

Estimating the principal eigenpair of a positive operator from stochastic evaluations gives rise to a noisy form of normalized power iteration: each step applies a sampled operator estimate and then renormalizes. We establish finite-time stochastic-approximation guarantees for this procedure, working in Hilbert's projective metric, which is well-suited to the multiplicative geometry of Perron--Frobenius eigenproblems. Our main application is risk-sensitive average-cost reinforcement learning (RL) with exponential utility. In this setting the Bellman equations are multiplicative, and value functions are characterized by nonlinear eigenvalue problems rather than additive fixed points. We cast both policy evaluation and control in this framework, obtaining risk-sensitive TD- and Q-learning algorithms that learn from a single Markovian trajectory. Under explicit positivity, mixing, and linear function approximation assumptions, we prove finite-time bounds on the recovered eigenvector and eigenvalue with $\tilde{O}(ε^{-2})$ sample complexity up to problem-dependent constants. The results cover both tabular and linear function approximation regimes, and through the duality between exponential utility and KL-robust control yield finite-time guarantees for KL-robust average-cost RL as well.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.03787

Dimension-Free Sinkhorn for Optimal Transport: Bypassing Extremal Costs

Kun He

原文摘要

The Sinkhorn--Knopp (SK) algorithm is a cornerstone method for matrix scaling and entropically regularized optimal transport (EOT). Despite its empirical efficiency, existing bounds for achieving marginal accuracy $\varepsilon$ deteriorate severely in the presence of extreme entries, bottlenecked either by the global maximum regularized cost $η\|C\|_\infty$ (where $η$ is the regularization parameter and $C$ the cost matrix) or the matrix's minimum-to-maximum entry ratio $ν$. These bounds may be overly sensitive to rare extreme entries that do not dominate the overall matrix structure. We identify a broad structural regime in which the dependence on the magnitude of global extreme entries can be circumvented. For EOT, we introduce well-boundedness, a local mass property that captures the row- and column-wise bulk structure of the underlying matrix while maintaining robustness against scattered entrywise outliers. We prove that, governed by this structural condition, SK converges to the target transport plan for a problem of dimension $n$ in $O(\log n + \log(1/\varepsilon))$ iterations. This complexity bound is independent of the maximum regularized cost $η\|C\|_\infty$. A computationally negligible pre-scaling step eliminates this dimensional dependence, reducing the iteration complexity to a strictly dimension-free $O(\log(1/\varepsilon))$. Beyond EOT, we establish a sharp phase transition for general $(\boldsymbol{u},\boldsymbol{v})$-scaling governed by a critical matrix density threshold. We prove that when a matrix's density exceeds this threshold, the iteration complexity is strictly independent of $ν$. Conversely, when the density falls below this threshold, the dependence on $ν$ becomes unavoidable; in this sub-critical regime, we construct instances where SK requires $Ω(n/\varepsilon)$ iterations.

cs.DS · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2602.12413

Soft Contamination Means Benchmarks Test Shallow Generalization

Ari Spiesberger, Juan J. Vazquez, Nicky Pochinkov, Tomáš Gavenčiak, Peli Grietzer, Gavin Leech, Nandi Schoots

原文摘要

Training on benchmark test items is a classic way to overestimate generalization to unseen problems. Standard decontamination methods miss semantic duplicates (SemDs): training items equivalent in content to a test item, but distant in string space. We call training on such items soft contamination; when the resulting gains extend to held-out items of the same benchmark (but not to related benchmarks), we call the result shallow generalization. We first embed Olmo 3's entire post-training dataset and search it for SemDs of benchmark items; we then finetune Olmo 3 and Qwen models on natural and synthetic semantic duplicates. We find (1) that soft contamination is widespread: the Olmo corpus contains SemDs of 76% of the CodeForces test set and 100% of MBPP's; (2) that finetuning on SemDs of a benchmark raises accuracy on its held-out items by up to 22pp in controlled experiments and 10-11pp at realistic contamination rates, though the effect varies across benchmarks; (3) that the gains from training on SemDs mostly do not transfer to conceptually similar benchmarks, i.e. we find shallow generalization; (4) that finetuning on items with similar embeddings that are not SemDs produces no such gain, indicating that the mechanism is not mere topical similarity; and (5) that at realistic contamination rates, soft contamination can raise held-out performance about as much as exact duplicates (or more). Benchmark gains here are therefore confounded - in a way that standard decontamination does not address.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.16423

Shifting the Gradient: Understanding How Defensive Training Methods Protect Language Model Integrity

Satchel Grant, Victor Gillioz, Jake Ward, Thomas McGrath

原文摘要

Defensive training methods such as positive preventative steering (PPS) and inoculation prompting (IP) offer surprising results through seemingly similar processes: both add trait-inducing objects to large language models (LLMs) during training, and both defend the LLM against acquiring the trait. The success of these methods raises the questions: how do they work, and are they doing the same thing? We provide behavioral and mechanistic analyses of PPS and IP, using the acquisition of an ``evil'' persona by Qwen2.5-7B-Instruct as an in-depth case study and replicating key measurements across six traits and four models. Behaviorally, neither method operates through a purely associative mechanism, and both methods prevent trait acquisition; but PPS can also substantially reduce pre-existing trait expression, whereas IP does so only modestly. Mechanistically, PPS induces a concentrated change in the activation gradient and shifts the signal along the trait direction from amplifying to attenuating, whereas IP changes the gradient more diffusely and shifts it close to zero along the trait direction. Gradient manipulations show that the direction of this component is causally relevant to trait acquisition, although it is one effect rather than a complete account of either defense. We then show that IP lowers the next-token prediction loss on trait-expressing data, whereas PPS can raise it at the intensities at which it defends; we show how this analysis raises the possibility that the loss changes and the gradient shift are two summaries of the same training signal rather than completely separate mechanisms. Together, our results show that PPS and IP are not interchangeable, and they highlight open questions about a complete mechanistic picture.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2602.05214

Disentangled Representation Learning via Flow Matching

Jinjin Chi, Taoping Liu, Mengtao Yin, Ximing Li, Yongcheng Jing, Jialie Shen, Leszek Rutkowski, Dacheng Tao

原文摘要

Disentangled representation learning aims to capture the underlying explanatory factors of observed data, enabling a principled understanding of the data-generating process. Recent advances in generative modeling have introduced new paradigms for learning such representations. However, existing diffusion-based methods encourage factor independence via inductive biases, yet frequently lack strong semantic alignment. In this work, we propose a flow matching-based framework for disentangled representation learning, which casts disentanglement as learning factor-conditioned flows in a compact latent space. To enforce explicit semantic alignment, we introduce a non-overlap (orthogonality) regularizer that suppresses cross-factor interference and reduces information leakage between factors. Extensive experiments across multiple datasets demonstrate consistent improvements over representative baselines, yielding higher disentanglement scores as well as improved controllability and sample fidelity.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.34045

Minimax Regret for Sobolev Kernel-Based Reinforcement Learning

Haichen Hu, David Simchi-Levi

原文摘要

We study the minimax regret rate for episodic kernel-based reinforcement learning, where the expected reward and transition density belong to a Sobolev reproducing kernel Hilbert space (RKHS), with continuous state and action spaces, and adversarial initial states. For Sobolev smoothness parameter $ν>0$, state dimension $d_S$, and action dimension $d_A$, we prove that the minimax expected regret over $T$ episodes is $\widetildeΘ(T^{\max\{e_A,e_B\}})$, where $e_A=\frac{ν+d_A}{2ν+d_A}$, $e_B=\frac{ν+d_A+3d_S/2}{2ν+d_A+2d_S}$ with the other model parameters fixed. The two exponents reflect the distinct difficulties of action selection and learning across states. We introduce Sobolev Optimal Value Iteration (\texttt{SOVI}), which attains this rate from observed rewards and transitions. For fixed smoothness and reference-kernel lengthscale, its regret bound has a prefactor polynomial in the state-action dimension $d=d_S+d_A$, horizon $H$, reward and transition RKHS norm bounds $B_r,B_p$, and the norm-comparison factor $Λ$, up to logarithmic factors; $Λ=1$ for normalized Matérn kernels. The algorithm separates initial action selection from continuation-value estimation and combines local polynomial approximation with the shared RKHS structure of the approximation errors. Our matching upper and lower bounds determine the optimal dependence on the number of episodes for Sobolev kernel-based reinforcement learning regret minimization. Thus, we partially address the open problem in Vakili(2024) under Sobolev kernel RL.

cs.LG · math.ST · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.22100

When the next byte's type is fixed: why entropy patching skips computed values under tight budgets

Nicolás Vera Zúñiga

原文摘要

The Byte Latent Transformer (BLT) runs a large model once per patch and starts patches where a small model's next-byte entropy is high. Under tight budgets this rule skips computed values, such as the number after "= " in math. On BLT-1B at inference we trace the miss to the cue before a value: when the preceding bytes fix the next byte's type, the patcher's entropy is capped near log2 10 bits, close to a 10% budget's threshold. Entropy then starts a patch at 0% of values after a dollar sign and 19% after "= "; writing "is " instead of "= " raises the 19% to 66%. Version 1's two-byte lookup table, motivated by LZ novelty, acts as a learned detector of cues that tend to precede new values in its fitting text: on the solutions rewritten as prose it covers every value after a dollar sign and none after a plain space, and its margin over entropy falls from +42.1 to -8.9 points, partly because prose gives entropy more starts per value. In BLT-style models trained from scratch (1M-50M parameters, 128-byte context, 3 seeds per rule) and scored by teacher-forced exact match on answer spans, the table beats entropy at 50M by 36.8 points on computed results and 29.6 on final answers, which mostly copy the last computed result. A hand-written marker rule is not distinguishable from the table at 1M and 12M; its 50M control was not run. At 1M and 12M the table ran on a T4 GPU in fp16 and the baselines on an A40 in bf16; at 50M all ran on the A40. End to end, no 1M-50M model solves more than 2 of 100 GSM8K problems. A label-free probe of BLT-1B's patcher finds prose values, mostly after a dollar sign, but costs final answers. In six domains, no rule we tested beat entropy on exact match of whole words and numbers in more than one domain. The positive findings concern tight budgets on cue-formatted text. This version corrects version 1; the work is closed, with no venue intended.

cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10389

From Task Affinities to Optimal Allocations: Probabilistic Graphical Modelling for Supervised Finetuning

Prateek Chanda, Saral Sureka, Krishnateja Killamsetty, Ganesh Ramakrishnan

原文摘要

Supervised fine-tuning over a large pool of heterogeneous tasks is often computationally expensive, leading practitioners to train language models on a fixed training budget of carefully chosen SFT tasks. Thus, achieving strong downstream performance hinges on selecting an effective subset of tasks, or equivalently, identifying an optimal task mixture under a fixed budget constraint. In practice, supervised finetuning mixtures are often determined using simple heuristics (e.g., uniform or size-proportional sampling) that ignore task interactions, which can hurt transfer and waste budget on redundant task sources. We introduce TASKMIXPGM, a framework for learning continuous task mixtures through energy-based modelling over SFT tasks. Each task forms a node in a Markov random field, with unary potentials capturing task utility and pairwise potentials modelling inter-task relationships via behavioral divergences between predictive distributions of single-task fine-tuned models. Optimizing this objective yields task mixtures that balance utility, coverage, and redundancy. We further show that the induced set function defined over the set of such tasks is weakly submodular under budget constraints, enabling approximation guarantees for discrete task selection. Empirically, we provide a fixed, benchmark-independent mixture that improves over heuristic mixing across multiple model families and different training budgets. TASKMIXPGM also uncovers interpretable structure in task interactions, revealing complementarities and redundancies that inform effective SFT mixtures.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10386

Pruning DNA Sequence

Faias Satter, Shaker Islam, Aminul Islam

原文摘要

Data pruning can substantially reduce training cost by removing examples that contribute little additional learning signal, but it has not been studied for DNA language-model pretraining. We introduce Sequence Pruning (SP), a dynamic pruning framework that removes genomic sequences as they become easy for the model to predict. SP uses per-sequence pretraining accuracy computed directly from the model's forward pass, requiring no separate scoring model or additional pass over the corpus. We evaluate SP across five pretraining settings spanning two corpora, three model families, and both MLM and CLM objectives, with downstream evaluation on five genomic benchmarks. SP reduces pretraining data processing by up to 96% while largely preserving downstream performance. At moderate savings, pruning improves performance by up to 1.2 MCC points over full-data pretraining; at the highest savings, pretraining uses only 4% of the baseline GPU hours with little performance loss. SP also outperforms the dynamic pruning methods InfoBatch and BPAS across most comparisons. Beyond accuracy, our framework supports alternative criteria such as perplexity and EL2N, although accuracy yields the strongest downstream performance at matched savings. These results show that genomic pretraining does not require repeatedly processing all available sequences and that dynamic sequence-level pruning can substantially improve its data and computational efficiency. Code is available at https://github.com/FaiasPromit/pruning-dna-sequence.

cs.LG · q-bio.GN

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2601.16906

The Trajectory Alignment Coefficient in Two Acts: From Reward Tuning to Reward Learning

Calarina Muslimani, Yunshu Du, Kenta Kawamoto, Kaushik Subramanian, Peter Stone, Peter Wurman

原文摘要

The success of RL is fundamentally tied to having a reward function that accurately reflects the task objective. Yet, designing reward functions is time-consuming and prone to misspecification. To address this, our first goal is to understand how to support RL practitioners in specifying reward functions. We leverage the Trajectory Alignment Coefficient (TAC), a metric that evaluates how closely a reward function's induced preferences match those of a domain expert. To test whether TAC provides meaningful support, we evaluated its impact in a user study where practitioners designed reward functions for Lunar Lander. We found that iterative TAC feedback allowed participants to design more performant reward functions while reporting lower cognitive workload than unguided reward design. However, the study also underscored that manual reward design, even with TAC, remains labor-intensive. This motivated our second goal: to learn a reward model that maximizes TAC directly. Specifically, we propose Soft-TAC, a differentiable approximation of TAC that can be used as a loss function to train reward models from human preferences. Our theoretical analysis guarantees that under Soft-TAC optimization: (1) the optimal reward function belongs exclusively to the set of human-aligned reward functions, and (2) this optimal reward remains invariant whether trained on clean or noisy preference data. Empirically, we first evaluate reward models trained with Soft-TAC across three robotic tasks, demonstrating that they consistently match or outperform those trained with the Cross-Entropy loss. Next, we evaluate Soft-TAC in the GT7 driving simulator, and find that Soft-TAC-trained reward models more successfully capture preference-specific driving styles. Ultimately, this work demonstrates that TAC can serve as both a practical tool for guiding reward design and a reward learning objective in complex domains.

cs.LG · cs.HC

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10385

A Topological Encoder Decoder Framework for Temporal Graph Learning

Ronan Buck, Kiarash Shamsi, Tran Gia Bao Ngo, Astrit Tola, Baris Coskunuzer, Cuneyt Gurcan Akcora

原文摘要

Most temporal graph learning methods reduce future prediction to discriminative inference over past interactions, focusing on edges or node labels for a largely persistent set of nodes. This view breaks down when new nodes and edges appear, since the future graph becomes a new object with changing size, composition, and structure. We address this gap by framing temporal graph prediction as an inverse topology problem. Instead of predicting edges directly, we first predict a multiscale topological descriptor of the future graph and then reconstruct a plausible future snapshot that realizes this descriptor under inductive constraints. This approach makes global structure and node churn explicit prediction targets and produces forecasted graphs that downstream tasks can evaluate without retraining. Across 14 temporal graph datasets, we evaluate our method, TOPOGED, on node, link, and graph property prediction and compare it against state-of-the-art temporal graph models. TOPOGED significantly improves node-level forecasting accuracy over the strongest baseline, with a macro average of 0.49 versus 0.05. It also outperforms baselines in 60% of graph-structure metric evaluations and substantially increases macro-average edge prediction metrics, from near-zero to 0.12. Our results show that topology-guided graph forecasting can predict inductive future snapshots whose structure supports multiple downstream evaluations.

cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2605.13230

Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence

Xinyu Liu, Kechen Jiao, Chunyang Xiao, Runsong Zhao, Junhao Ruan, Bei Li, Jiahao Liu, Qifan Wang 等 13 位作者

原文摘要

On-policy distillation (OPD) has become a promising paradigm for reasoning-oriented post-training of large language models (LLMs), especially when combined with reinforcement learning from verifiable rewards (RLVR). Existing OPD methods rely on reverse KL (RKL)-based teacher supervision over trajectories sampled from the student policy. However, we identify a critical limitation: under large teacher--student policy divergence, RL-driven exploration often produces trajectories outside the teacher distribution, resulting in uninformative negative feedback. To address this, we propose Teacher-Guided Policy Optimization (TGPO), an on-policy reasoning distillation method that remains effective under large policy divergence settings. Rather than relying solely on evaluative supervision, TGPO uses teacher to directly guide token level generation conditioning on student-generated contexts; together with RLVR-style trajectory level rewards, TGPO steers exploration toward improved continuations. Experiments on reasoning benchmarks show that TGPO consistently outperforms existing RKL-based OPD methods and remains robust across different teacher models.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.04072

Finite-Sample Convergence in Networked Average Reward MARL: Decentralization Pitfalls and Entropy Remedies

Yizhou Zhang, Yashaswini Murthy, Laixi Shi, Adam Wierman

原文摘要

We study the problem of average reward Multi-Agent Reinforcement Learning (MARL) where agents interact within a network. Each agent in the network conducts local updates with information within its $k$-hop neighborhood and collaboratively maximizes the overall reward of the entire network. We first provide impossibility results in achieving convergence to the global optimal joint policies in our setting. These impossibility results highlight the challenges of decentralized learning with local information in multi-agent systems, namely the issues of multiagency, where each agent independently optimizes its policy, and partial observability, where agents have limited visibility into the global state. Given these challenges which indicate that decentralized policy optimization can generally be certified only up to stationarity, we provide finite-sample convergence guarantees for a decentralized actor-critic algorithm with linear function approximation that converges to an approximate stationary point with small gradient norm. To overcome such impossibility results, we further show that adding fixed-state entropy regularization reshapes the objective landscape which leads to even stronger global convergence guarantees for our proposed actor-critic algorithm.

cs.LG · cs.MA

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10376

PhysDNet: Physics-Driven Gradient Amplification for Real-Time Image Dehazing

Rayen Dhahri, Lei Sun, Victor Javier Kartsch Morinigo, Alice Pagnoux, Luca Benini, Danda Pani Paudel, Luc Van Gool

原文摘要

Modern dehazing networks are often computationally expensive and overfit to synthetic haze, limiting real-time deployment. This underscores an urgent demand for efficient, high-fidelity restoration methods that can operate on resource-constrained edge devices in practical, dynamic environments. We introduce PhysDNet, a lightweight multimodal architecture that fuses NIR imagery with sparse LiDAR depth and is trained with a physics-constrained dual-head objective. Differentiating through a fixed Koschmieder inversion induces density-adaptive gradient amplification, acting as an implicit resource allocation mechanism that concentrates learning on heavily degraded regions without increasing inference cost. PhysDNet delivers superior restoration with minimal capacity and enhanced training efficiency, facilitating real-time deployment on edge devices without increasing inference overhead. On the Seeing Through Fog benchmark, PhysDNet outperforms competitors by 1.67 dB at 5.2 times lower compute while recovering 54.9% of fog-induced detection loss. Ablations and cross-domain evaluations show that physics-guided training improves generalization, including under non-homogeneous haze that violates the Koschmieder assumption. PhysDNet further demonstrates compatibility with fixed-function NPUs, running at 29/45/99 FPS at <4 W across L/M/S variants. Code, deployment scripts and pre-trained checkpoints are available at https://github.com/Heatdh/PhysDnet.

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2602.04042

Partition Tree: Conditional Density Estimation over General Outcome Spaces

Felipe Angelim, Alessandro Leite

原文摘要

We propose Partition Tree, a tree-based framework for conditional density estimation over general outcome spaces that supports both continuous and categorical variables within a unified formulation. Our approach models conditional distributions as piecewise-constant densities on data-adaptive partitions and learns trees by greedily minimizing a surrogate for conditional log-loss. This yields a scalable alternative to existing probabilistic trees that does not make parametric assumptions about the target distribution. We further introduce Partition Forest, an ensemble extension obtained by averaging conditional densities. Empirically, we demonstrate improved probabilistic prediction over CART-style trees and competitive performance compared to state-of-the-art probabilistic tree methods and Random Forests.

cs.LG · stat.ME · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.04266

Explaining and Preventing Alignment Collapse in Iterative RLHF

Etienne Gauthier, Francis Bach, Michael I. Jordan

原文摘要

Reinforcement learning from human feedback (RLHF) typically assumes a static or non-strategic reward model (RM). In iterative deployment, however, the policy generates the data on which the RM is retrained, creating a feedback loop. Building on the Stackelberg game formulation of this interaction, we derive an analytical decomposition of the policy's true optimization gradient into a standard policy gradient and a parameter-steering term that captures the policy's influence on the RM's future parameters. We show that standard iterative RLHF, which drops this steering term entirely, suffers from alignment collapse: the policy systematically exploits the RM's blind spots, producing low-quality, high-reward outputs whose feedback reinforces the very errors it exploits. To mitigate this, we propose foresighted policy optimization (FPO), a mechanism-design intervention that restores the missing steering term by regularizing the policy's parameter-steering effect on RM updates. We instantiate FPO via a scalable first-order approximation and demonstrate that it prevents alignment collapse on both controlled environments and an LLM alignment pipeline using Llama-3.2-1B.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10373

SAMBoost: Spatially Aware Mesh Boosting

Mulah Moriah, Franck Vermet, Pierre Ailliot, Juliette Legrand

原文摘要

We propose SAMBoost (Spatially Aware Mesh Boosting), a gradient boosting method that combines tree-based covariate modelling with a stochastic partial differential equation (SPDE) latent spatial field through a single penalised objective. At each boosting iteration, the predictor is decomposed into separate covariate and spatial components. The method is motivated by applications in which the spatial effect is not a nuisance to be removed but a quantity of interest that must be recovered alongside the covariates. In insurance, for example, practitioners need both flexible nonlinear covariate effects and an interpretable geographic risk surface that can be reported on its own. Existing approaches often involve trade-offs, offering flexible covariate modelling but no explicit spatial field, rigorous spatial inference with a fixed additive covariate structure, or a combination of both at computational costs that limit practicality for large datasets. SAMBoost addresses these goals within the boosting loop, and the underlying mesh produces a sparse precision structure whose cost is largely independent of sample size. On simulations and on an open benchmark, SAMBoost matches or improves on the predictive accuracy of joint Gaussian process and tree boosting approaches. It is also faster and recovers the spatial field and covariate effects more accurately. When covariate effects include interactions, it outperforms additive Bayesian spatial models. Applied to a flood insurance portfolio with 4.9 million records, SAMBoost trains on the full training set without subsampling and runs about sixteen times faster than a Bayesian reference, with a consistent spatial risk surface. A public Python implementation is available.

stat.ME · cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.27946

Is Backpropagation Optimal? When Synthetic Gradients Improve Sample Efficiency

Yibo Jacky Zhang, Zeyu Tang, Sanmi Koyejo

原文摘要

Backpropagation is the default learning rule for artificial neural networks and is often treated as the settled approach whenever differentiability is available. In this work, we revisit this convention through a theoretical lens of sample efficiency. We introduce a unified vectorized feedback framework for loss-based and reward-based learning on computational graphs, in which synthetic gradients emerge as a natural alternative to backpropagation. We characterize the conditions under which synthetic gradients can achieve a lower gradient-estimation mean squared error than backpropagation. We construct examples illustrating that this sample efficiency advantage can be arbitrarily large. Experiments on contextual bandits and reinforcement learning tasks demonstrate the potential of our theoretical findings.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2609.28870

When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse

Yiyu Liu, Minlan Yu, Juncheng Yang

原文摘要

Long-running LLM applications and autonomous agents repeatedly send growing context, making prefix caching critical for reducing prefill cost. This raises a fundamental question: do decades of traditional cache-management techniques transfer directly to agentic and conversational LLM serving? We analyze production traces comprising over 20 billion processed tokens and evaluate 14 eviction algorithms across capacity-constrained HBM and large memory-pool settings. We find that simple Least Recently Used (LRU) is surprisingly robust, offering competitive performance against sophisticated traditional policies, though substantial headroom to Belady's optimum remains. The reason is structural: prefix reuse is dominated by the steady pacing of active sessions, making recency an unusually strong predictor for eviction. Nevertheless, prefix caching diverges from traditional workloads in ways that recency ignores, notably the prevalence of single-turn requests, extreme session footprint skew, and highly non-uniform miss costs because attention computation scales with token depth. To close the gap to optimality, we establish several design principles for prefix-cache management: keeping recency as the baseline while incorporating quick demotion to filter one-hit prompts, compute-aware eviction to prioritize expensive misses, and capacity-adaptive eviction granularity. Our traces and simulator are publicly available.

cs.DC · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2507.12412

Contrastive Nonmyopic Objective Cost-Tradeoff Acquisition for Longitudinal Data

Dzung Dinh, Boqi Chen, Yunni Qu, Marc Niethammer, Junier Oliva

原文摘要

In many critical domains, features are not freely available at inference time: each measurement may come with a cost of time, money, and risk. Longitudinal prediction further complicates this setting because both features and labels evolve over time, and measurements missed at earlier timepoints may become permanently unavailable. We propose CAPE (Contrastive Acquisition-Plan Embeddings), a partitioning acquisition policy for longitudinal active feature acquisition. CAPE first defines NOCT, an oracle-based objective that scores a set of future feature-time acquisitions by its expected predictive loss together with its acquisition cost. Based on NOCT, CAPE learns a contrastive acquisition-utility manifold of masked partial observations, where proximity reflects the expected utility of future acquisitions. Offline, training states are embedded and partitioned at each rollout timepoint; candidate future plans are scored within each partition using plug-in NOCT, and the best partition-level acquisition plans are cached. During inference, CAPE retrieves the cached plan via nearest-centroid lookup, enabling low-latency adaptive acquisitions. Experiments on synthetic and real-world healthcare datasets demonstrate that CAPE outperforms nonparametric, greedy, and RL-based alternatives, achieving higher accuracy at lower acquisition costs. Code is publicly available at https://github.com/dzungdinhh/CAPE.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10364

Bias, Measurement Error, and Double-Dipping: When Can GNN Convolutions Help Brain Connectome Prediction?

Tommaso Castellani, Jiaqi Li, Muriah D. Wheelock, Rezwana Rosen Razzaque, Helmut Laufs, Hong Chen, Claire Donnat

原文摘要

Graph Neural Networks (GNNs) are an important tool for fMRI prediction, where empirical functional connectivity matrices serve both as features and for graph construction. However, recent studies and our own experiments show that these methods often fail to outperform simpler graph-agnostic baselines. We propose a statistical explanation for this failure by formulating connectome prediction as an errors-in-variables (EIV) problem: the true connectome is latent, while the observed covariance matrix is a noisy finite-time-series estimate. In a linear asymptotic setting, we decompose the effect of graph convolution into three mechanisms: smoothing-induced bias and variance reduction, graph estimation error, and a double-dipping effect arising when the same noisy matrix determines both graph and features. Under a latent community model adapted to neuroscience, we identify a narrow favorable regime, set by the brain community structure, the predictive signal, the graph construction and the recording length, in which message passing can improve prediction. Simulations validate this decomposition, while real-data experiments illustrate that connectome GNN pipelines often fall outside this regime. Our results provide a first step towards explaining when graph convolutions can help brain connectome prediction, and why they often do not.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2608.14689

A Reproducibility Study of Partial Residual Ablations in Pre-LN Transformers

Pratikkumar Babariya

原文摘要

Residual connections are a fundamental component of transformer architectures, yet the roles of the attention and feed-forward residual pathways remain poorly understood when considered independently. This paper presents a reproducibility study of partial residual ablations in Pre-LN GPT-style transformers trained at two scales (10M and 124M parameters). I compare four architectural configurations by selectively removing the attention residual connection, the feed-forward residual connection, or both. At 10M scale, a controlled 8-seed deterministic sweep shows a clear asymmetry: removing the attention residual (FFNOnly) reaches the No-Residual collapse regime (3.350 +/- 0.002), whereas removing the feed-forward residual (AttnOnly) remains well separated from it (1.580 +/- 0.003). A subsequent controlled 124M sweep establishes that the direction of this asymmetry persists under matched seeds: across five seeds each, AttnOnly reaches 6.037 +/- 0.355 versus 7.569 +/- 0.001 for FFNOnly, with no overlap between the observed ranges. AttnOnly nevertheless remains substantially degraded relative to FullResidual (4.577 +/- 0.012) and exhibits markedly greater seed sensitivity at 124M. During the investigation, I identified and corrected an experimental measurement confound in runtime gain scaling and retained an intermediate reproduction failure rather than excluding it. I propose cross-position routing through self-attention as a falsifiable hypothesis for the observed asymmetry; no clean causal test of this mechanism has yet been completed. To support reproducibility, I release the source code, experiment configurations, training logs, and experimental results, including intermediate non-reproducing runs.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10362

Information-Theoretic Generalization for Set-Input Optimization-Valued Objectives

Futoshi Futami, Masahiro Fujisawa

原文摘要

Modern machine learning is increasingly evaluated using criteria such as multicalibration, CVaR, and Cressie-Read DRO, which target subgroup reliability, tail risk, and distributional robustness rather than averages of fixed per-example losses. Existing information-theoretic generalization bounds for pointwise losses do not directly apply, because these empirical criteria depend on the entire batch and involve a post-hoc auxiliary optimization over witnesses, thresholds, or reweightings. We formalize this structure as set-input optimization-valued objectives. In the balanced supersample setting, we bound the expected optimized validation-training gap in terms of an objective-dependent sub-Gaussian parameter and the square root of the prediction-based conditional mutual information (fCMI) between the learned supersample prediction table and the selector vector. Combining this with fixed-predictor population-validation comparisons yields unified algorithm-dependent generalization bounds. We verify sub-Gaussianity of the selector gap using stability and convex L2-Lipschitz certificates, obtaining concrete bounds for multicalibration, CVaR, and Cressie-Read DRO, with a stability-based single-index refinement.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2603.24511

Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs

Alexander Panfilov, Peter Romov, Igor Shilov, Yves-Alexandre de Montjoye, Jonas Geiping, Maksym Andriushchenko

原文摘要

We show that AI agents are capable of discovering novel algorithms for adversarial attacks against LLMs, advancing the state of the art on white-box jailbreaking and prompt injection evaluations. We deploy frontier agents, such as Claude Code and Codex, in an autoresearch loop with access to a library of 30+ prior methods and an evaluation script with a fixed compute budget. We show this pipeline to be effective in jailbreaking OpenAI's GPT-OSS-Safeguard-20B and in prompt injections against Meta-SecAlign-70B, an adversarially robust model. For GPT-OSS-Safeguard, the best agent-discovered method achieves up to 80\% attack success rate on CBRN queries, compared to <50\% for existing methods. For SecAlign, it achieves 100\% ASR, while the best prior automated methods only achieve 82\%. Notably, in our setting, attack methods are developed on unrelated surrogate models for a pure random-target token-forcing task, yet generalize directly to prompt injection on the adversarially trained model. Finally, we trace the lineage of methods developed during autoresearch, characterizing the agents' strategies and failure modes. Adversarial ML has long held that defenses must be evaluated against attacks tailored to them; autoresearch automates this principle, and we argue it should be the minimum bar for defense evaluation going forward.

cs.LG · cs.AI · cs.CR

查看论文
arXiv ↗PDF ↗
版本更新 · v5arXiv:2602.00387

Singular Bayesian Neural Networks

Mame Diarra Toure, David A. Stephens

原文摘要

Bayesian neural networks promise calibrated uncertainty but require $O(mn)$ parameters for standard mean-field Gaussian posteriors. We argue this cost is often unnecessary, particularly when weight matrices exhibit fast singular value decay. By parameterizing weights as $W = AB^{\top}$ with $A \in \mathbb{R}^{m \times r}$, $B \in \mathbb{R}^{n \times r}$, we induce a posterior that is \emph{singular} with respect to the Lebesgue measure, concentrating on the rank-$r$ manifold. This singularity captures structured weight correlations through shared latent factors, geometrically distinct from mean-field's independence assumption. We derive PAC-Bayes generalization bounds whose complexity term scales as $\sqrt{r(m+n)}$ instead of $\sqrt{m n}$, and prove loss bounds that decompose the error into optimization and rank-induced bias using the Eckart-Young-Mirsky theorem. We further adapt recent Gaussian complexity bounds for low-rank deterministic networks to Bayesian predictive means. Empirically, across MLPs, LSTMs, and Transformers on standard benchmarks, our method achieves competitive predictive performance while using up to $33\times$ fewer parameters than 5-member Deep Ensembles. It substantially improves OOD detection and often improves calibration relative to mean-field and perturbation baselines, while Deep Ensembles can still be stronger on in-distribution likelihood-based metrics.

stat.ML · cs.LG · stat.AP

查看论文
arXiv ↗PDF ↗
版本更新 · v5arXiv:2602.14697

Evolutionary System Prompt Learning for Reinforcement Learning in LLMs

Lunjun Zhang, Ryan Chen, Bradly C. Stadie

原文摘要

Building agentic systems that can autonomously self-improve from experience is a longstanding goal of AI. Large language models (LLMs) today primarily self-improve via two mechanisms: self-reflection for context updates, and reinforcement learning (RL) for weight updates. In this work, we propose Evolutionary System Prompt Learning (E-SPL), a method for jointly improving a model's experiential context and its weights. In each RL iteration, E-SPL samples trajectories under multiple system prompts in parallel, then uses them to jointly update the weights with RL and evolve the system prompts with self-reflection. E-SPL encourages a natural division between procedural knowledge in weights and interpretable declarative knowledge in system prompts. Most notably, in an easy-to-hard generalization setting (AIME $\rightarrow$ BeyondAIME), E-SPL improves RL success rate from 38.8% $\rightarrow$ 45.1%. E-SPL also improves RL on AIME 2025 (56.3% $\rightarrow$ 60.6%), HMMT 2025 (50.0% $\rightarrow$ 52.7%), and agentic search (44.2% $\rightarrow$ 48.6% on gpt-oss-120b). Across all settings, E-SPL outperforms both RL-only and evolution-only baselines, demonstrating that weight updates and context updates are deeply synergistic and can together yield gains in generalization that neither achieves alone. Code is available at https://github.com/LunjunZhang/E-SPL.

cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10359

Spiking Sensory Networks

Lleyton Ariton

原文摘要

Effective sensory reception is critical for survival, yet the computational mechanisms behind how peripheral sensory neurons (SNs) transduce external signals into spiking representations for downstream brain circuits remain to be fully understood. Numerous theoretical frameworks have been proposed to model sensory processing, with Bayesian approaches among the most prominent. However, existing machine learning attempts to apply Bayesian theory to spiking neurons remain largely phenomenological, disconnected from empirical biology, and poorly validated on non-stationary temporal data. Conversely, neuroscientific models often rely on strong inductive experimental biases or explicit kinetic dynamics fit post hoc to match spiking data. Here, we introduce a class of recurrent Poisson VAE models, termed Spiking Sensory Networks (SSNs), that bridge these paradigms by reconciling variational inference with biologically inspired spike transduction for time-varying stimuli. Evaluated on real-world odor plume recordings from locust olfactory receptor neurons (ORNs), SSNs match or exceed the reconstruction fidelity of Spiking Neural Network (SNN) and GLM baselines while exhibiting superior representational characteristics. Furthermore, SSNs are well-suited for neuroscientific investigation, learning empirical in vivo ORN response motifs and exhibiting emergent spike-frequency adaptation under unsupervised reconstruction without manual tuning. Moreover, we theoretically link the recurrent decoding objective to mutual information maximization, and derive a formal thermodynamic lower bound connecting the Poisson Kullback-Leibler regularizer to the excess non-equilibrium metabolic power of biophysical homeostasis. Ultimately, our work synthesizes principles from machine learning theory and empirical neuroscience, providing a viable bridge that concretizes Bayesian models with biological mechanisms.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗