跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+2新增论文
+2其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-13 02:57 · 全站最近入库：2026-11-12 06:37各信源进度 →

统计口径与数据新鲜度

统计 2026-11-12 03:20 — 2026-11-13 03:20（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-13 02:57最近发现新增版本：2026-11-12 06:37库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,675 已收录论文18,323 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 4,903 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-13 02:56（北京时间）
版本更新 · v2arXiv:2607.09645

Deep Gaussian Processes on Directed Acyclic Graphs

Federico L. Perlino, Oliver Hamelijnck, Adam M. Johansen, Theodoros Damoulas

原文摘要

Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \citep{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theory and methodology at scale and under DAG misspecification. We then model a latent-collider DAG, a protein signalling network, and a multi-fidelity \mbox{heavy-ion} collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability over the simulator hierarchy.

stat.ML · cs.LG · math.ST · stat.CO · stat.ME

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10993

DPDP: Diffusion Policy-Steered Diffusion Planner for Offline Reinforcement Learning

Ruyi An, Mingyuan Zhou

原文摘要

In offline reinforcement learning, diffusion models fill two roles with complementary strengths. Typically guided by a critic, a diffusion policy is value-aware and favors high-return, behavior-supported actions, but it is short-sighted: it never lays out the states it intends to visit. A diffusion planner is far-sighted, synthesizing such a long-horizon trajectory, but it is only behavior-driven at execution, realizing each planned transition using only offline behavior.Each is strong exactly where the other is weak. We introduce DPDP to bring them together at the planner's execution step, where a planned transition becomes an action. Reading the planner's inverse-dynamics executor as a posterior over actions: the environment's transition likelihood relates an action to the planned next state, while the behavioral prior determines its weight under the offline data. Replacing this prior with a value-optimized diffusion policy preserves the planned transition likelihood and favors valuable actions that can serve the plan. We target the resulting clean-action distribution in two ways. A training-free sampler composes policy and inverse-dynamics scores at matched action noise levels and applies Langevin correction; this route uses a shared action noise schedule and an approximately common terminal prior. A preference objective instead amortizes the transfer into a single diffusion executor. Across D4RL, DPDP improves the planner it builds on, staying competitive with reactive diffusion policies on locomotion and leading on long-horizon navigation.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10992

Hydro-Dataset: A Multi-Modal Open-Source Pipeline for Nordic Hydrological Research and Foundation Model Benchmarking

Getnet Demil, Md Moinul Islam, Mourad Oussalah

原文摘要

Despite world-class snow monitoring by the Swedish Meteorological and Hydrological Institute (SMHI) and Finnish Meteorological Institute (FMI), no station-anchored, multi-parameter snow benchmark dataset exists for the Nordic boreal domain. We present Hydro-Dataset v1.1, a reproducible open-source pipeline that harmonizes 582 SMHI meteorological observation stations across nine hydrological winters (2015/16-2023/24) into a CF-1.9 NetCDF product with 1.8~million annotated station-days. Six derived snow parameters, such as Snow Water Equivalent (SWE), snow phenology, melt rates, degree-day melt factor, rain-on-snow flags, and freeze-thaw cycles, supply benchmark-ready labels for hydrological foundation model training. ERA5-Land cross-validation (Pearson R=0.78, bias=-262mm) confirms observation-anchored SWE provides a more reliable benchmark than reanalysis for machine learning training.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10991

Sensitivity-Gated Accumulator Caching for On-Device Learning of Low-Bit Hand Networks

Soroor Ghandali, Kanika Mamgain, Manjot Kaur, Akanksha Shukla, Shashira Guntuka

原文摘要

Small hand-tracking networks are deployed as 4- and 8-bit codes, and they should keep adapting to a user's camera after deployment, on the device and within roughly the memory of the model itself. Quantization-aware training cannot do this cheaply, because it keeps a 32-bit master copy of every weight together with its optimizer state, about 96 bits per weight. Training the codes directly with stochastic rounding needs no copy but injects rounding noise at every step, and the methods that remove this noise keep a residual buffer for every weight. We introduce sensitivity-gated accumulator caching (SGAC), which keeps residual state for only a small and changing set of weight groups. A set-associative cache of 8-bit accumulators collects the sub-step updates of the groups with the largest Fisher-weighted squared code step and writes them into the codes as whole steps; an evicted line is flushed by stochastic rounding, and a group without a line does not move until it is admitted. We prove that the write-back is unbiased and show, under a local quadratic model, that rounding a sub-step update costs loss of the same order as the update's own gain, so that rounding pays only where the expected gradient is large relative to the curvature. On a 383k-parameter hand-keypoint network, SGAC trains with 8.1 bits of state per weight. Scored on the 4/8-bit network that the device runs after adaptation, it has the highest mean accuracy of nine deployed low-bit networks on two simulated cameras. On twelve ImageNet-C corruptions it has the highest average of corrupted and clean accuracy of seven deployed methods, ahead of the PEQA-style, ECO-style and direct low-bit baselines with confidence intervals that exclude zero, and it matches an 8-bit accumulator kept for every weight with 56% of that state.

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2602.00474

Persistent-Transient Policy Evaluation for Markov Chains via Minimal Peripheral Quotients

Yang Xu, Vaneet Aggarwal

原文摘要

We study fixed-policy evaluation for finite Markov chains that may be reducible and periodic. Classical evaluation methods with gain and bias decomposition are not always diagnostic: the gain records only invariant Cesàro averages, while persistent phase-dependent behavior is absorbed into the bias together with genuinely transient effects. We identify the real peripheral invariant subspace $\mathcal{K}(P)$ of the transition matrix $P$ as the source of this ambiguity. Quotienting by $\mathcal{K}(P)$ is the minimal exact quotient that removes all non-decaying modes and makes the remaining dynamics strictly stable. After choosing a gauge projection $Π$ with kernel $\mathcal{K}(P)$, the reward admits a unique decomposition $r = g_Π^\star + (I-P)v_Π^\star$, where $g_Π^\star$ is a persistent regime profile and $v_Π^\star$ is a gauge-fixed transient component. An exact comparison with classical normalized gain and bias shows that the new pair reallocates the same information into a persistent profile and a gauge-fixed representative of the transient quotient. This decomposition reconstructs finite-horizon returns, recovers statewise average reward, admits a transient-cost interpretation, and yields a stable estimator under a generative model.

stat.ML · cs.LG · math.NA

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10987

Horizon-Scalable Q-Learning

Haoran Xu, Pragya Srivastava, Xinjie Liu, Siddhant Agarwal, Amy Zhang

原文摘要

Learning accurate off-policy state-action value functions (Q-functions) from offline datasets becomes harder as the task horizon grows. One-step temporal-difference (TD) learning bootstraps after every transition, so small per-step errors accumulate along the horizon, and early states can be badly misvalued even when the TD loss is small. Multi-step TD shortens this chain by following the dataset trajectory for several steps; however, since the behavior policy is often suboptimal, every followed action adds a bias, and no fixed number of steps suits a whole trajectory. In this paper, we propose Horizon-Scalable Q-Learning (HQL), a simple method that enables accurate long-horizon value learning by automatically balancing between one-step and multi-step returns. HQL extends a target through a dataset action only when the critic values it at least as highly as the best-of-N actions sampled from the BC policy, and otherwise falls back to a one-step target. We show that this rule is a principled off-policy correction that needs no action densities, which makes HQL compatible with expressive flow and diffusion policies. Experiments show that HQL avoids the failures of both one-step and multi-step TD on a combination-lock task across horizons, and outperforms prior horizon-reduction methods on long-horizon OGBench and OCBench tasks.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2605.18445

What's Holding Back Latent Visual Reasoning?

André G. Viveiros, Nuno Gonçalves, André F. T. Martins, Matthias Lindemann

原文摘要

Humans can approach complex visual problems by mentally simulating intermediate visual steps, rather than reasoning through language alone. Inspired by this, several works on Vision-Language Models have recently explored chain-of-thought reasoning with continuous latent tokens as intermediate visual imagination steps. In this work, we investigate how recent models leverage such latent tokens. Surprisingly, we find that model accuracy is unaffected when latent tokens are replaced by uninformative dummy tokens. This indicates that latent tokens play a minimal causal role in the model's final prediction. To better understand this phenomenon, we analyze both the training signal provided by oracle latent representations and the quality of the latent tokens generated at inference time. Our experiments reveal two crucial issues holding back latent visual reasoning: First, in most existing datasets, oracle latent tokens provide limited additional information beyond the original image and do not substantially simplify the task, leading models to ignore them during training and effectively bypassing them at inference time. When fine-tuned on a diagnostic dataset, in which latent tokens provide sufficient support for the final prediction, we show that models can causally rely on them. Second, the latent tokens produced at inference time deviate from their corresponding oracle representations, collapsing to a narrow region and preventing benefits even when the model relies on them. Overall, our findings suggest that future progress in latent visual reasoning depends on two key pillars: high-quality datasets with informative intermediate steps and more precise latent token prediction.

cs.CV · cs.AI · cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10984

Network Contention State Aware Dynamic Optimizer for Accelerator Collective Communication Libraries

Rajat Bisht, Tevfik Kosar, Anthony Skjellum, Gene Cooperman, MD S Q Zulkar Nine

原文摘要

Collective Communication Libraries (CCL) are critical for large-scale distributed AI/ML training, yet their configurations are determined by static heuristics that do not account for real-time network conditions. Our empirical study shows that these heuristics perform near-optimally in contention-free environments, but degrade substantially under background traffic. To address this, we present a cooperative multi-agent run-time optimizer for accelerator collective communication. Our approach employs a two-agent architecture: (i) detector- a Long Short Term Memory(LSTM) recurring neural network fed classifier head with a sequential probability ratio test that distinguishes sustained contention from transient spikes; and (ii) selector- a configuration selector built on a Dueling Double DQN (D3QN) with Conservative Q-Learning that adapts configuration parameters in response. We implemented this dynamic tuner over NCCL, a widely used CCL developed by NVIDIA. When evaluated on Chameleon Cloud's isolated bare-metal cluster, this dynamic tuning approach reduces AllReduce latency by 36% over NCCL and 32% over a fully converged AutoCCL. We further evaluated it across six real AI workloads spanning convolutional networks, transformers, and recommendation architectures. Our multi-agent tuner approach delivers up to 65% speedups on Delta cluster with NVIDIA A100 GPUs and Slingshot-11 fabric internode connectivity.

cs.DC · cs.LG · cs.MA

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2602.04131

Decoupling Time and Risk: Risk-Sensitive Reinforcement Learning with General Discounting

Mehrdad Moghimi, Anthony Coache, Hyejin Ku

原文摘要

Distributional reinforcement learning (RL) is a powerful framework increasingly adopted in safety-critical domains for its ability to optimize risk-sensitive objectives. However, the role of the discount factor is often overlooked, as it is typically treated as a fixed parameter of the Markov decision process or tunable hyperparameter, with little consideration of its effect on the learned policy. In the literature, it is well-known that the discounting function plays a major role in characterizing time preferences of an agent, which an exponential discount factor cannot fully capture. Building on this insight, we propose a novel framework that supports flexible discounting of future rewards and optimization of risk measures in distributional RL. We provide a technical analysis of the optimality of our algorithms, show that our multi-horizon extension fixes issues raised with existing methodologies, and validate the robustness of our methods through extensive experiments. Our results highlight that discounting is a cornerstone in decision-making problems for capturing more expressive temporal and risk preferences profiles, with potential implications for real-world safety-critical applications.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2604.00449

Convergence of Byzantine-Resilient Gradient Tracking via Probabilistic Edge Dropout

Amirhossein Dezhboro, Fateme Maleki, Arman Adibi, Erfan Amini, Jose E. Ramirez-Marquez

原文摘要

We study distributed optimization over networks with Byzantine agents that may send arbitrary adversarial messages. We propose Gradient Tracking with Probabilistic Edge Dropout (GT-PD), which combines a universal self-centred projection that clips incoming messages to a radius-$τ$ ball around the receiver with a fully decentralized probabilistic dropout rule driven by a threshold-gated, dual-metric trust score in the decision and tracking channels. These identity-free defences bound adversarial perturbations while preserving doubly stochastic mixing. The projection preserves the honest network average exactly in every realization, even when honest messages are clipped. Under complete Byzantine isolation ($p_b=0$), GT-PD converges linearly to a neighbourhood of order $ασ^2/(μn_{\mathcal H})$ plus higher-order network terms, matching centralized stochastic gradient descent to first order in the step size. We also give a radius computable from problem data that ensures honest messages remain unclipped with high probability. Under partial isolation ($p_b>0$), a leaky variant, GT-PD-L, controls accumulated tracking perturbations and converges linearly to a bounded neighbourhood governed by stochastic variance and the clipping-to-leak ratio, under explicit conditions on the step size, leak, and aggregate Byzantine mixing weight. With two-tier dropout and $p_h=1$, Byzantine isolation adds no variance to honest consensus dynamics. MNIST experiments under Sign Flip, ALIE, and Inner Product Manipulation attacks show that GT-PD-L outperforms coordinate-wise trimmed mean by up to 4.3 percentage points under stealth attacks.

cs.LG · cs.MA · eess.SY

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2605.20555

LAPO: KL-Free Logit-Averaged Policy Optimization to Leverage RL and SFT in LLM Post-Training

Xingwei Gan, Ying Zhu

原文摘要

From a classical statistical perspective, model averaging and parameter shrinkage (e.g., Ridge regression with an $l_2$-penalty) are deeply connected mechanisms designed to reduce estimator variance and lower mean squared error. Motivated by this insight, we introduce Logit-Averaged Policy Optimization (LAPO), a KL-free and critic-free post-training framework that replaces explicit KL regularization with logit averaging inside Group Relative Policy Optimization (GRPO); in particular, averaging the logits of the trainable policy and a frozen reference policy functions as an implicit structural regularizer that stabilizes rollout trajectories. For selecting the mixing weight, we propose a compute-efficient adaptive algorithm that dynamically balances exploration and structural anchoring throughout training, evaluated against a static fixed-weight baseline. This adaptive mechanism eliminates explicit KL penalty backpropagation and expensive, fragile hyperparameter sweeps while reducing the incidence of uninformative zero-reward rollout groups. Mathematically, embedding logit averaging within the reinforcement learning loop operates as an in-training Product of Experts (PoE): the multiplicative geometric mean enables the reference anchor to act as a soft logical "AND" that vetoes non-compliant tokens, enforcing schema rigor and safety guardrails without the catastrophic forgetting typical of sequential fine-tuning. Evaluated across Qwen2.5-Instruct models (1.5B, 3B, and 7B), LAPO consistently outperforms canonical KL-regularized GRPO, achieving a 7%-15% accuracy gain on MATH and a 2%-5% gain on cn_k12 and MMLU. Finally, the resulting mixed teacher policy can be distilled back into a single student network, fully restoring standard $1\times$ inference latency.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.14029

Utility-Constrained Policy Optimization

Mehrdad Moghimi, Bernardo Avila Pires

原文摘要

Constrained MDPs (CMDPs) are a widely adopted framework for incorporating safety into RL agents; however, the framework does not support risk-sensitive constraints. This can be problematic: For example, CMDPs allow for optimal solutions that, in order to satisfy the risk-neutral constraints, mix infrequent catastrophic behaviors and frequent, overly conservative ones. Moreover, prior empirical results suggest that enforcing stricter, risk-sensitive constraints can improve performance even under risk-neutral evaluation. The natural framework to incorporate risk-sensitive constraints is utility-constrained MDPs (UCMDPs), but no practical solutions for this problem existed. In this work, we introduce a simple yet powerful methodology for UCMDPs and constrained RL. Besides allowing for risk-sensitive constraints, our framework does not require us to fix constraint limits in advance of training the agent, provided that a sensible range is known. This increases policy flexibility and, in practice, allows for adjustments to these limits at no extra training cost. Besides benefiting from the generality of the framework, our agent shows strong performance in practice, consistently matching or outperforming existing baselines in several Safety Gymnasium benchmark tasks.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10981

An Analysis of Reinforcement Learning with Function Approximation

Francisco S. Melo, Sean P. Meyn, M. Isabel Ribeiro

原文摘要

We address the problem of computing the optimal Q-function in Markov decision problems with infinite state-space. We analyze the convergence properties of several variations of Q-learning when combined with function approximation, extending the analysis of TD-learning in Tsitsiklis and Van Roy (1996) to stochastic control settings. We identify conditions under which such approximate methods converge with probability 1. We conclude with a brief discussion on the general applicability of our results and compare them with several related works.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.30213

Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs

Benjamin Walker, Alexandre Bloch, Lingyi Yang, Sam Morley, Terry Lyons

原文摘要

Continuous-time models are a natural choice for irregular and asynchronous data. A central design choice is how to embed discrete observations into continuous time. Interpolation- and imputation-based embeddings reconstruct a continuous observation path, making the model sensitive to the choice of reconstruction. We show that this reconstruction step is unnecessary; under mild conditions, compact-set universality on the model input space transfers to the data space whenever the embedding from data to input is continuous and injective. Guided by this result, and building on the rectilinear control path for Neural Controlled Differential Equations (NCDEs), we introduce a continuous and injective embedding for Log-NCDEs, a universal class of continuous-time models. Our approach records observations as increments and composes them over arbitrary query intervals to directly form log-signatures. This provides interval-level summaries without first interpolating the observed variables, while supporting online computation. Experiments on synthetic controlled dynamics and real-world time-series datasets show that the representation is accurate, efficient, and robust to irregular, asynchronous, and sparse observations.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10977

DM-Call: Decision-Model-Guided Function Calling with Cumulative Verification Feedback

Minhyeong Yu, Wonduk Seo, Seunghyun Lee

原文摘要

Reliable function calling requires Large Language Models (LLMs) to select appropriate functions and generate arguments consistent with user intent. However, syntactically valid calls may still be semantically incorrect due to inappropriate function selection or mismatched arguments. In this paper, we propose Decision-Model-Guided Function Calling (DM-Call), a framework that combines local schema validation with separate decision-model verification of function selection and argument correctness. For rejected candidates, DM-Call accumulates verification probabilities, binary judgments, and schema errors across attempts. A hybrid feedback representation further combines numerical probabilities with verbal correctness labels to guide iterative refinement without executing external tools. Extensive evaluation across BFCL and API-Bank (first-call) demonstrates DM-Call boosts function-calling accuracy versus direct generation, at modest additional cost.

cs.IR · cs.AI · cs.CL · cs.LG · cs.SE

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.03199

Fast Organic Crystal Structure Prediction with Unit Cell Flow Matching

Alston Lo, Luka Mucko, Austin H. Cheng, Andy Cai, Alastair J. A. Price, Wojciech Matusik, Alán Aspuru-Guzik

原文摘要

Organic crystal structure prediction (CSP) is a requirement for computational modelling of organic solids, but traditionally costs several CPU-years per molecule. Generative models such as OXtal dramatically reduce this cost by sampling stable organic crystal structures directly. However, OXtal forgoes explicit lattice parametrization in favour of modelling large crops of the bulk material with expensive triangle layers, which can incur a computational cost of minutes per molecule. In this paper, we reduce this to seconds with Clari, a large-scale flow matching model that generates redundancy-free unit cells and replaces triangle layers with pair-bias attention. Clari requires only atom types and bonds as input and does not need an RDKit-sanitizable input molecule, which expands its applicability to challenging chemistries such as fullerenes, metal complexes, and atom clusters. We further ablate key design choices such as auxiliary losses, timestep distributions, noise priors, and self-conditioning. On OXtal's test sets, we surpass OXtal's solve rate while obtaining a speedup of 15-30$\times$. Because Clari also models explicit hydrogens, it supports inference-time scaling via direct energy ranking, without any decoration or relaxation step. When generating 150 crystals and selecting the top-30 by energy, we further improve solve rate while maintaining a speedup of 5-8$\times$. We also introduce the CSD Teaching Subset as a new test split of diverse and complex molecules for future benchmarking. Our contributions enable CSP within seconds, making large-scale virtual screening of organic solids practical. Code is available at https://github.com/the-matter-lab/clari.

cs.LG · physics.chem-ph

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2603.23994

Understanding the Challenges in Iterative Generative Optimization with LLMs

Allen Nie, Xavier Daull, Zhiyi Kuang, Abhinav Akkiraju, Anish Chaudhuri, Max Piasevoli, Ryan Rong, YuCheng Yuan 等 13 位作者

原文摘要

Generative optimization uses large language models (LLMs) to iteratively improve artifacts (such as code, workflows or prompts) using execution feedback. It is a promising approach to building self-improving agents, yet in practice remains brittle: despite active research, only 9% of surveyed agents used any automated optimization. We argue that this brittleness arises because, to set up a learning loop, an engineer must make "hidden" design choices: What can the optimizer edit and what is the ``right'' learning evidence to provide at each update? We investigate three factors that affect most applications: the starting artifact, the credit horizon for execution traces, and batching trials and errors into learning evidence. Through case studies in MLAgentBench, Atari, and BigBench Extra Hard, we find that these design decisions can determine whether generative optimization succeeds, yet they are rarely made explicit in prior work. Different starting artifacts determine which solutions are reachable in MLAgentBench, truncated traces can still improve Atari agents, and larger minibatches do not monotonically improve generalization on BBEH. We then study these factors jointly in harness optimization on TBLite, evaluated on Terminal-Bench 2.0. We also propose a batch-size and credit-horizon rule that, calibrated on TBLite, transfers to DeepSWE after adjusting to its token costs and comes within 1% of the best configuration tested at each context budget.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10974

SWEET: Mitigating Spurious Feature Reliance in LLM Reasoning via Weight Editing

Zhizhang Fu, Lucas Zhao, Michalis Korakakis, Mengru Ding, Lukasz Szpruch, Yue Zhang

原文摘要

Reasoning in large language models can be disrupted by simple input perturbations (e.g. adding irrelevant distractors), suggesting reliance on spurious correlations. Identifying and suppressing the spurious features involved in long chain-of-thought reasoning remains challenging. We introduce Spurious-feature WEight EdiTing (SWEET), a training-free method that combines counterfactual interventions with mechanistic interpretability to identify and suppress spurious-features. SWEET extracts token- and sequence-level behavioral directions from activation contrasts between original problems and their counterfactual variants - capturing local and global patterns of spurious reasoning. It then localizes the responsible MLP neurons and attention heads and suppresses their contributions along these directions through weight edits. Using only 20 original-counterfactual pairs for direction extraction, SWEET improves accuracy by an average of 11.8 percentage points on perturbed inputs and 2 points on original inputs across five open-weight reasoning models and five reasoning benchmarks. It also reduces the relative gap between original and perturbed accuracies from 17.6% to 2.5%. Analyses identify two recurring forms of spurious reasoning, i.e. over-integrating irrelevant conditions and overthinking, and provide mechanistic evidence that SWEET improves reasoning robustness by suppressing both behavioral directions.

cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10972

JUBILEE: A Benchmark Environment for Off-Policy Evaluation in Debt Collections

Diego Silva Prado, Tiago Amaral Penteado de Jesus, Bruno Caldas de Souza, Cristiano Bevitori Maffia De Oliveira

原文摘要

Off-policy evaluation (OPE) estimates the value a candidate decision policy would achieve, using only data logged by the policy already in production. In debt collections, unrecorded factors can influence both actions and outcomes. This paper presents JUBILEE, a benchmark environment built around a debt-collections simulator that generates million-loan panels with competing risks, rare recovery events, and hidden factors driving both outcomes and assignment. The evaluation includes an 80-run comparison of tabular and gradient-boosted estimators under two synthetic outcome mechanisms and a 190-run parameter grid. Independent policy simulations provide reference values with estimated Monte Carlo uncertainty for a synthetic reward score and for simulated cash. With hidden factors influencing action assignment, several tabular methods agree closely while underestimating the field-visit policy's score, an error that also appears with boosted models. Intervals from tabular doubly robust estimation usually contain the reference without hidden assignment dependence but mostly miss it when that dependence is present. Estimates from randomized decisions do not show the same large error, although their limited precision leaves policy choice uncertain. The decision consequences depend on how policy value is defined. The field-visit policy ranks last under the score and first under simulated cash, which excludes channel costs. Under cash, log-based estimators miss this best candidate in nearly every run with hidden assignment dependence in the paired study. JUBILEE supports reproducible study of these failure modes under specified collections mechanisms.

cs.LG · q-fin.RM · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.07130

Teaching Agents the "Principle", Not the "Reward": Prescriptive Policy Learning

Shan Yang

原文摘要

Reinforcement learning has achieved strong performance in many tasks by letting agents learn from nothing more than a reward signal. However, in more complex tasks, agents still struggle to learn good, generalizable policies, even with carefully designed rewards and abundant samples. We argue that this difficulty stems from the reward decoding burden: designers encode their knowledge of the desired behavior into rewards and penalties, and agents must infer this knowledge from long sequences of noisy scalars. To address this burden, we propose prescriptive policy learning (PPL), which directly teaches agents higher-level knowledge about the tasks, termed principles of behavior, such as how to maintain balance, how to accumulate energy, and how to land smoothly. Through appropriately selected key quantities and references, PPL can directly indicate at every step whether the agent follows the principles and to what extent. Our experiments show that agents can learn good, generalizable policies without any reward. Taught only simple heuristic principles, PPL agents outperform reward-based methods with far fewer samples. Their policies also transfer directly to harder settings and keep performing well, demonstrating their strong generalization ability. In summary, PPL provides a new approach in which rough knowledge of the desired behavior is enough for agents to learn good, generalizable policies, even when the system dynamics cannot be accurately modeled.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10969

Rule-table structure of life-like cellular automata: activity, asymmetry, sensitivity, and the propagation storage factorisation

Yair Cohen

原文摘要

What structure on the rule space of a cellular-automaton family exposes which rules can support which behaviours? For the 262,144 life-like rules -- the two-dimensional outer-totalistic family containing Conway's Game of Life -- the standard coordinate, Langton's activity parameter, is long known to be too coarse. I organise the family by three symmetry-derived rule-table coordinates: an unweighted activity, a birth-survival asymmetry (odd under the swap of birth and survival), and the Vichniac-Binder Boolean-derivative sensitivity in closed form, whose centre-bit component vanishes exactly on the centre-blind (B=S) submanifold. The cardinality coordinates lambda, delta are necessary but never sufficient to certify behaviour; the centre-bit sensitivity labels the B=S submanifold on which the paper's one exact exclusion holds. Combined with an exhaustive enumeration of every rule's compact gliders, still lifes and oscillators (three-by-three seed, period at most sixteen), they expose a hierarchical factorisation: the birth condition is a narrow gate deciding whether anything can propagate, the survival condition tunes storage, and every compact-glider-supporting rule is birth-light and survival-rich. One exclusion is exact: no B=S rule supports a compact glider at that footprint, proven by exhaustive enumeration and closing only for compact-footprint patterns -- 36 B=S rules ship at four-by-four seed and 40 at five-by-five. Two dynamical results frame the whole: glider-capable rules that random sampling never expresses show that capability is certifiable from the table while its expression is only estimable from dynamics; and every uncatalogued rule producing a richly Life-like ecology turns out to be a Conway or HighLife variant on the neighbour counts its trajectories actually visit.

q-bio.PE · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2501.00087

High-Dimensional Markov-Switching Ordinary Differential Processes

Katherine Tsai, Mladen Kolar, Sanmi Koyejo

原文摘要

Complex systems often evolve in continuous time but are observed only at discrete time points with measurement noise. When the interactions can change across unobserved states, recovering both the state transitions and the differential equations is important for mechanistic insight and for identifying time-varying directed graphs. However, this recovery is challenging in high dimensions. In this paper, we study estimation and graph recovery for high-dimensional Markov-switching ordinary differential equation models from noisy discrete observations. We model the switching between states by a latent continuous-time Markov chain, and within each state the drift is a sparse additive function with unknown nonlinear components. We approximate each component by a finite basis expansion and estimate the parameters under group sparsity. Our procedure estimates the latent trajectories by wavelet smoothing and then runs a penalized surrogate expectation-maximization algorithm. Under stationarity and geometric $β$-mixing, given a sufficiently accurate estimated trajectory and other local regularity conditions, we show that the algorithm converges geometrically to a neighborhood of the true parameters. Finally, in an application to resting-state fMRI data, the procedure identifies multiple connectivity states, whose occupation times we compare descriptively between the children with combined-type ADHD and the typically developing children used in the fit.

stat.ME · cs.LG · math.ST · stat.AP

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10965

Bayesian transfer for offline Q-learning with multi-task Gaussian processes

Wenxing Guo

原文摘要

Offline reinforcement learning can benefit from data collected in related tasks, yet indiscriminate knowledge transfer may lead to negative transfer under task mismatch and limited target-data coverage. We propose a target-aware Bayesian transfer framework for offline Q-learning based on multi-task Gaussian processes that jointly model task-specific Q-functions. Source tasks are assessed using target predictive evidence and maximum mean discrepancy (MMD), while posterior uncertainty and behavior-supported action sets guide conservative Bellman updates and policy extraction. To account for the dependence induced by Bellman bootstrapping, we distinguish Bellman innovations from bootstrap errors and establish stability and predictive source-selection results under explicit regularity conditions. We further derive selection-adaptive posterior contraction and offline policy-performance bounds, identifying the structural conditions under which source information can be pooled to achieve minimax-optimal rates in a shared-function Gaussian sequence experiment. Simulation studies with heterogeneous source tasks demonstrate improved target Q-function estimation and reduced material negative transfer compared with indiscriminate source pooling. An application to human decision-making data further shows higher average expected immediate payoff than both target-only learning and matched-size random source selection. These results highlight the benefits of combining target-aware source selection with uncertainty-aware conservative learning for reliable and sample-efficient offline reinforcement learning.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10961

Improving Analytic Constants via Interpretable AI-Guided Lean Proof Refinement

Rahul Saha, Alan Li, Anton Xue, Swarat Chaudhuri, Adam Klivans, Pravesh K. Kothari, Raghu Meka

原文摘要

AI agents are increasingly capable of discovering new mathematical results, yet their reasoning and generated proofs can be difficult to understand and verify. We introduce an interpretable framework for AI-guided mathematical discovery, focused on improving quantitative bounds on analytic constants. Our key insight is that existing proofs often contain slack in intermediate lemmas, which can be tightened without reconstructing the entire argument. To exploit this structure, we represent proofs as partially-formalized Lean blueprints: human-interpretable dependency graphs of definitions, lemmas, and theorems. An AI agent then iteratively refines the Lean code while preserving the blueprint's dependency structure, allowing local improvements to automatically propagate to a verified final bound. This interpretable structure allows humans to inspect the agent's refinements and understand how local improvements propagate to the final theorem. For Grothendieck's constant ($\mathcal{K}_G$), we obtain $$ \mathcal{K}_G \leq \fracπ{2\log(1+\sqrt{2})} - 10^{-17}, $$ an explicit numerical improvement over Krivine's 1977 bound. For Korenblum's constant ($\mathcal{K}_K$), we obtain $$ \mathcal{K}_K \geq 0.356565, $$ improving Wang's 2025 bound of $0.3554$. Our experiments demonstrate the potential of both partial formalization and iterative agentic refinement for interpretable mathematical discovery.

cs.AI · cs.LG · math.NA

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2602.06208

Emergent Low-Rank Training Dynamics in MLPs with Smooth Activations

Alec S. Xu, Can Yaras, Matthew Asato, Qing Qu, Laura Balzano

原文摘要

Recent empirical evidence has demonstrated that the training dynamics of large-scale deep neural networks occur within low-dimensional subspaces. While this has inspired new research into low-rank training, compression, and adaptation, theoretical justification for these dynamics in nonlinear networks remains limited. %compared to deep linear settings. To address this gap, this paper analyzes the learning dynamics of multi-layer perceptrons (MLPs) under gradient descent (GD). We demonstrate that the weight dynamics concentrate within invariant low-dimensional subspaces throughout training. Theoretically, we precisely characterize these invariant subspaces for two-layer networks with smooth nonlinear activations, providing insight into their emergence. Experimentally, we validate that this phenomenon extends beyond our theoretical assumptions. Leveraging these insights, we empirically show there exists a low-rank MLP parameterization that, when initialized within the appropriate subspaces, matches the classification performance of fully-parameterized counterparts on a variety of classification tasks.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.19804

Stitched Value Model for Diffusion Alignment

Hyojun Go, Hyungjin Chung, Prune Truong, Goutam Bhat, Li Mi, Zhaochong An, Zixiang Zhao, Dominik Narnhofer 等 11 位作者

原文摘要

Reward-based alignment of diffusion and flow models requires estimating the value function at noisy intermediate latents, although pretrained reward models operate on clean images. Common approaches use Tweedie-style clean-image proxies or Monte Carlo rollouts, introducing approximation bias or substantial computational cost. Directly learning the value function offers an alternative, but matching the performance and generalization of foundation-scale pixel-space reward models on noisy latents remains challenging. We address this gap by transferring pretrained reward capabilities to noisy latents. We propose \textbf{StitchVM} (Stitched Value Model), a model stitching framework that connects a frozen diffusion-backbone prefix to a pretrained reward-model suffix through a compatible feature interface. A closed-form feature fit selects the interface, and lightweight distillation adapts the stitched model, combining noise-aware diffusion features with pretrained reward capability. For example, adapting CLIP ViT-L with SD~3.5 Medium at $512\times512$ resolution takes approximately 10 GPU-hours on a single GH200. Once trained, StitchVM provides an amortized alternative to Tweedie-style and Monte Carlo approximations of the value function: it estimates value directly from noisy latents, avoiding clean-image reconstruction and additional denoising rollouts for each evaluation. We demonstrate benefits across inference-time steering and post-training. StitchVM improves the quality and efficiency of FK steering and DRaFT, accelerates DPS by $3.7\times$ with half the peak GPU memory, and accelerates DiffusionNFT by $2.3\times$.

cs.CV · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.08553

VeriContest: A Competitive-Programming Benchmark for Verifiable Code Generation

Zichen Xie, Mrigank Pawagi, Yuxin Liu, Aaditi Rai, Lize Shao, John Berberian, Sicong Che, Wenxi Wang

原文摘要

Large language models can generate useful code from natural language, but their outputs come without correctness guarantees. Verifiable code generation offers a path beyond testing by requiring models to produce not only executable code, but also formal specifications and machine-checkable proofs. Progress in this direction, however, is difficult to measure: existing benchmarks are often small, cover only part of the pipeline, lack ground-truth proofs or rigorous specification validation, or target verification settings far from mainstream software development. We present VeriContest, a benchmark of 1,006 competitive-programming problems from LeetCode and Codeforces for verifiable code generation in Rust with Verus. Each problem includes a natural language description, expert-validated formal specifications, judge-accepted Rust code, Verus-checked proofs, and positive and negative test suites. VeriContest is constructed through a three-phase pipeline that scales from manually verified seed problems to semi-automated expansion with human-in-the-loop review. To strengthen benchmark quality, we also use testing to validate postcondition completeness. VeriContest supports both isolated and compositional evaluation of specification, code, and proof generation, and end-to-end verified program synthesis. Evaluating ten state-of-the-art models reveals a sharp gap between ordinary coding and verifiable code generation: the strongest model reaches 97.51% on natural-language-to-code generation, but only 57.16% on specification generation, 13.82% on proof generation, and 6.76% end-to-end. These results identify proof and specification generation as the central bottlenecks for current models and establish VeriContest as a rigorous platform for evaluating and training future systems that generate code with machine-checkable correctness.

cs.SE · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.13740

SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices

Timi Owolabi, Hikmah Olawore, Frances Adelakun, Ayotomiwa Oyewumi

原文摘要

World-action models (WAMs) jointly predict robot actions and future observations, but their video diffusion backbones are costly for edge deployment. We introduce SmolWAM, a compact latent WAM derived from the 2B-parameter Cosmos Policy using single-frame encoding, cached text features with preserved attention sinks, depth pruning, and token compaction. With 12 of 28 transformer blocks and 1,008 tokens, SmolWAM predicts actions and future latents in one rectified-flow step. After 3.6 A100 GPU-hours of training, it achieves 95.5% success over 400 LIBERO episodes at 57ms, 2.3TFLOP, and 2.4GiB per 16-action chunk on an A100: 11x lower latency and 3.2x lower memory than the five-step teacher. Its world head predicts scene motion; joint training improves success by 6.2 percentage points over an action-only ablation.

cs.RO · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10956

Risk-Sensitive Reinforcement Learning for an Options Market Maker: An Empirical Study with Real S&P 500 Options

Jacky Chen, Ing-Haw Cheng, John Hull, Landis Li, Zissis Poulos, Justin Shi, Andreas Veneris, Zeyu Wang

原文摘要

Options market makers absorb client order flow and must hedge the resulting inventory risk under trading frictions. We study whether an option hedging reinforcement learning (RL) agent trained on historical market data outperforms three alternatives in tail-risk protection: (i) traditional delta hedging, (ii) simple hedge-ratio rules, and (iii) the same agent trained in simulated markets. Most RL hedging agents are trained in model-generated markets that do not capture the crashes and volatility regimes present in real data. We introduce an empirical replay environment using S&P 500 option data from 1996 to 2017 and compare against a matched geometric Brownian motion (GBM) environment, a stochastic-volatility (SABR) simulator, and a hybrid that values historical paths with Black-Scholes, all sharing the same portfolio mechanics, stochastic arrivals, costs, and test period. We train a distributional Twin Delayed Deep Deterministic Policy Gradient (TD3) agent and evaluate it out of sample over 2018-2023. The agent trained on empirical data reduces Conditional Value-at-Risk (CVaR) tail loss by 3.1% compared to delta hedging and outperforms every hedge-ratio rule chosen during the training period and every alternative training environment. The reduction reaches 8.9% in volatile and falling markets, but reverses to a cost of 7-10% in rising markets. Empirical training induces a state-dependent short net delta: the policy under-hedges negative-delta books, most strongly after prices have fallen. This tilt disappears with GBM training and is weaker with SABR or hybrid training. These findings suggest that exposure to historical market dynamics can materially change a tail-risk hedging policy, although the evidence rests on a single test path and the edge is regime-conditional.

q-fin.RM · cs.LG · q-fin.CP

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2603.29264

Linear Generator Networks for Nonlinear Partial Differential Equations

Shafayeth Jamil, Rehan Kapadia

原文摘要

Most physical systems are governed by nonlinear partial differential equations, yet the analytical tools that make linear systems transparent, such as eigenspectra, dispersion relations and modal decomposition, remain unavailable in nonlinear settings. We introduce Linear Generator Network-Koopman (LGN-KM), a neural operator that learns a nonlinear lift and a continuous-time linear generator for the lifted dynamics through the decomposition $L_k = S - D_k$, where $S$ is skew-symmetric (conservative coupling) and $D_k$ is positive-definite diagonal (mode-selective dissipation). The decomposition guarantees stability by construction and exposes the eigenstructure directly, making linear-systems analysis available on a nonlinear PDE. On 2D Navier-Stokes turbulence, the generator yields a complete multi-branch dispersion relation and models trained independently at viscosities $100\times$ apart learn the same coupling spectrum ($R^2 = 0.87$). The same architecture shows the corresponding diffusive damping on FitzHugh-Nagumo reaction-diffusion, and for pulse propagation in optical fibers, the oscillation frequencies of the learned generator match the fiber's dispersion relation, giving the group-velocity dispersion to within $3.1\%$ with no fitted constant. The architecture additionally enables long-horizon stability, $O(1)$ continuous-time evaluation, and physics-informed cross-regime model transfer.

cs.LG · physics.flu-dyn

查看论文
arXiv ↗PDF ↗