跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+2新增论文
+2其中 AI 相关
+1已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-12 18:21 · 全站最近入库：2026-11-12 06:37各信源进度 →

统计口径与数据新鲜度

统计 2026-11-11 19:09 — 2026-11-12 19:09（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-12 18:21最近发现新增版本：2026-11-12 06:37库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,675 已收录论文18,323 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 4,903 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-12 18:18（北京时间）
新论文 · v1arXiv:2611.11819

When Should AI Overturn a Human Decision? Controversy at the 2026 World Cup and a $\sqrt{2/π}$ Solution

Amir Ali Ahmadi, Parsa Alavi, Radeen Alavi

原文摘要

We study how to combine a human binary call (a referee's offside or onside decision) with an AI numerical measurement (VAR's estimate of an attacker's position relative to the offside line). We derive an explicit maximum-likelihood rule that determines when the referee's call should stand and when a conflicting VAR measurement should overturn it. Our model treats the true offside margin as fixed and unknown and assumes independent additive Gaussian errors in the human's perception and the AI measurement. When both errors have zero means and known standard deviations $σ_H$ and $σ_{AI}$, respectively, the reversal threshold is $\sqrt{2/π}\,σ^2_{AI}/σ_H$. An onside call is overturned precisely when VAR's measured offside margin exceeds this threshold---not merely because VAR places the attacker beyond the offside line. We also derive the rule when either source has a systematic bias. We prove that the maximum-likelihood estimate of the true margin exists and is unique, while the final binary decision requires only a threshold comparison, without computing that estimate numerically. We illustrate the rule through numerical examples and show how to estimate the error parameters from calibration data. The same analysis applies beyond soccer to binary classification problems in which a human call and an AI measurement satisfy the stated model. This paper is intended for educational purposes: we aim to illustrate how undergraduate students and advanced high school students familiar with basic probability and calculus can investigate research questions in AI and its applications.

cs.AI · cs.LG · math.OC · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.14098

Pause and Reflect: Conformal Aggregation for Chain-of-Thought Reasoning

Yu Gu, Zijun Yu, Vahid Partovi Nia, Masoud Asgharian

原文摘要

Chain-of-thought (CoT) reasoning with self-consistency improves performance by aggregating multiple sampled reasoning paths. In this setting, correctness is no longer tied to a single reasoning trace but to the aggregation rule over a pool of candidate paths, making aggregation uncertainty the central challenge. This issue is critical where confidently incorrect answers are far more costly than abstentions. We introduce a conformal procedure for CoT reasoning that directly addresses aggregation uncertainty. Our approach replaces majority voting with weighted score aggregation over reasoning paths and calibrates an abstention rule using conformal risk control. This approach leads to finite-sample guarantees on the confident-error rate--the probability that the system answers and is wrong. We further identify score separability as the key condition under which abstention provably improves selective accuracy, and derive closed-form expressions that predict accuracy gains from calibration data alone. The method is fully inference-time, and requires no retraining. Across four benchmarks, four open-source models, and three score classes, realized confident-error rates are consistent with the prescribed targets up to calibration-split and test-set variability. Our method achieves $90.1\%$ selective accuracy on GSM8K by abstaining on fewer than $5\%$ of problems, compared with $82.3\%$ accuracy under majority voting and $87.4\%$ under score-weighted voting without abstention, while conformal risk control keeps the expected rate of confidently wrong answers below $10\%$.

stat.ML · cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.13013

FaST: Factored Synthesis of Transitions for Offline Reinforcement Learning

Daniel Campa, Spandan Das, Srinjoy Das

原文摘要

Synthetic data augmentation promises to scale offline reinforcement learning beyond fixed logged datasets, but leading diffusion-based generators model the transition tuple (state, action, next state, reward, terminal) as a monolith: no single factor can be natively redrawn in isolation. We present FaST (Factored Synthesis of Transitions), which factors transitions via the chain rule into state, action, dynamics, reward, and terminal stages, fitting each with an estimator matched to its statistical properties. Across twelve D4RL locomotion and maze benchmarks, TD3+BC and IQL agents trained on FaST transitions match the SynthER diffusion baseline while generating transitions over 90x faster. The explicit action-given-state conditional can also be re-queried for a fixed state, enabling targeted action synthesis: drawing N candidate actions per generated state and keeping the highest-advantage one under a value function fit to the same data, thereby synthesizing improved behavior during generation rather than filtering for it post-hoc. Best-of-10 synthesis beats training on real data on 11 of 12 medium and medium-replay settings (+8.9 D4RL points on average, up to +28.4), and is competitive with advantage-filtered SynthER at a fraction of its generation cost. On expert mixtures both mechanisms degrade identically, tracing the failure to the advantage estimate itself rather than to either targeting mechanism.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.08811

Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity

Zhongjie Shi, Wenjing Liao

原文摘要

This paper investigates the learning theory of Transformer networks for regression tasks on the compact Euclidean domain $[0,1]^d$ and $d$-dimensional compact Riemannian manifolds. We propose a novel constructive approximation framework for Transformers that builds local approximations of the target function and aggregates them into a global approximation via softmax partition of unity (POU). This approach leverages the attention mechanism to achieve spatial localization through affine transformations of the input. The softmax activation plays a crucial role in aggregating local approximations to a global output. From an approximation perspective, we prove that a dense Transformer equipped with only two encoder blocks and standard single-hidden-layer point-wise feed-forward networks can achieve a uniform $\varepsilon$-approximation error for $α$-Hölder continuous functions with $α\in (0,1]$ using $\mathcal{O}(\varepsilon^{-d/α})$ total parameters. Building upon this approximation guarantee, we establish a near minimax-optimal generalization error bound of order $\mathcal{O}\big(n^{-\frac{2α}{2α+d}} \log n\big)$ for the empirical risk minimizer, where $n$ is the training data size. The Transformer architecture studied in this paper uses native softmax attention and sinusoidal positional encodings, with a fixed depth of two encoder blocks. Finally, numerical experiments reveal spatially localized, Gaussian-like attention weights that emerge during training, resembling our Softmax POU weights.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11808

Efficient Online Submodular Maximization with Bandit Feedback

Iasonas Nikolaou, Evimaria Terzi

原文摘要

We study online monotone submodular maximization under a cardinality constraint with bandit feedback. On each round, the learner selects at most $k$ elements from a ground set of size $n$ and observes only the value of the selected set. We develop an efficient algorithm achieving $O\!\left(k\sqrt{nT\log(n/k)}\right)$ expected $(1-1/e)$-regret over $T$ rounds. To our knowledge, this is the first polynomial-time algorithm with $\sqrt{T}$ dependence on the horizon for arbitrary monotone submodular functions in the bandit setting. Our approach uses randomized sampling and a carefully constructed potential to reduce online submodular maximization to online linear optimization over the probability simplex.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11805

Operator Learning of Microstructure-Dependent Internal-Variable Models for Inelastic Homogenization: Approximation and Complexity

Lianghao Cao, George Stepaniants, Andrew M. Stuart

原文摘要

We study the learning of homogenized constitutive laws for inelastic heterogeneous materials, aiming to replace costly concurrent unit-cell simulations with surrogates that generalize across loading histories and microstructures. We focus on the recurrent neural operator framework, motivated by the explicit form of homogenized Kelvin--Voigt linear viscoelasticity. In this framework, internal variables enable a Markovian approximation of loading-history dependence, and neural operators represent the microstructure-dependent constitutive response. For periodic Kelvin--Voigt viscoelasticity, we prove a universal approximation theorem showing that the model can approximate the homogenized constitutive law to any prescribed accuracy, uniformly in time and over a wide set of admissible strain trajectories and microstructural property fields, in two and three dimensions. This comprises a significant extension of the existing, somewhat limited, one-dimensional theory that uses explicit solutions for piecewise constant materials. Instead, in two and three dimensions, we adopt a more versatile approximation based on the spectral representation of the memory kernel that arises from homogenization. This also enables us to give an explicit bound on the internal-variable dimension, sufficient for approximation, exhibiting logarithmic growth in the inverse relative stress error and in material contrast. Numerical experiments on viscoelasticity and crystal plasticity complement the theory, illustrating the roles of internal-variable dimension and sample complexity in learning homogenized responses from unit-cell data. Theoretical results are formally verified in Lean 4.

math.NA · cond-mat.mtrl-sci · cs.LG · math.DS

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2508.18741

Statistical Complexity of Soft Bellman Residual Minimization

Enoch H. Kang, Kyoungseok Jang

原文摘要

Offline reinforcement learning and offline inverse reinforcement learning aim to recover near-optimal value functions or reward models from a fixed batch of logged trajectories, yet current practice still struggles to enforce Bellman consistency scalably. Bellman residual minimization (BRM) has emerged as an attractive remedy. While globally convergent gradient-based soft (i.e., entropy-regularized) BRM methods for neural networks have recently been established, their statistical complexity under stochastic gradient descent remains largely unknown. In this paper, we address this theoretical gap for soft BRM. Through a Lyapunov-based analysis under a uniform PL condition in the auxiliary variable, we establish an $\mathcal{O}(1 / n)$ primal argument stability bound up to optimization error and an $\mathcal{O}(1 / n)$ statistical complexity for the soft BRM objective under the stated sampling conditions.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2602.17739

GeneZip: Region-Aware Compression for Long Context DNA Modeling

Jianan Zhao, Xixian Liu, Zhihao Zhan, Xinyu Yuan, Hongyu Guo, Jian Tang

原文摘要

Long-context DNA modeling is computationally expensive, motivating existing approaches to reduce sequence length through uniform downsampling. However, functional information and sequence redundancy are unevenly distributed across the genome, calling for finer resolution in functionally dense regions and stronger compression of redundant sequence. We introduce GeneZip, a region-aware DNA compression framework that learns where to allocate a limited token budget. Building on H-Net-style dynamic routing, GeneZip combines a Region-Aware Ratio objective, which uses regional labels derived from GENCODE gene annotations to specify compression targets during training, with bounded routing for stable token counts. At inference, it compresses raw DNA without annotations. GeneZip provides three main benefits. First, it preserves predictive utility under high compression: GeneZip-70M achieves 137.6-fold compression with a validation perplexity increase of 0.31 relative to uncompressed Mamba2-84M, and the best average rank across four DNALongBench tasks among compression models with leakage-controlled pretraining. Second, it learns region-aware allocation with emergent repeat-aware compression. Alternative regional priors yield task-dependent performance trade-offs. Without repeat-specific supervision, GeneZip compresses transposable-element-derived and tandem repeats more strongly than matched non-repeat sequence; its tandem-repeat compression preference transfers to a nine-species evaluation panel without further training. Third, it enables efficient long-context modeling: on 450-kb inputs, GeneZip-70M achieves 36.1-fold higher inference throughput than JanusDNA-7.7M on a single H100 GPU.

q-bio.GN · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2602.13804

Attention in Constant Time: Vashista Sparse Attention for Long-Context Decoding with Exponential Guarantees

Vashista Nobaub

原文摘要

Large language models spend most of their inference cost on attention over long contexts, yet empirical behavior suggests that only a small subset of tokens meaningfully contributes to each query. We formalize this phenomenon by modeling attention as a projection onto the convex hull of key vectors and analyzing its entropic (softmax-like) relaxation. Our main theoretical contribution is a face-stability theorem showing that, under a strict complementarity margin (a support gap (Δ) certified by KKT multipliers), entropic attention concentrates on a constant-size active face: the total mass assigned to inactive tokens decays exponentially as (\exp(-Ω(Δ/\varepsilon))), while the error on the active face scales linearly in the temperature/regularization parameter (\varepsilon). This yields a practical criterion for when sparse long-context decoding is safe and provides a principled knob to trade accuracy for compute. Building on these guarantees, we introduce Vashista Sparse Attention, a drop-in mechanism that maintains a small candidate set per query through a paging-style context selection strategy compatible with modern inference stacks. Across long-context evaluations, we observe stable constant-size effective support, strong wall-clock speedups, and minimal quality degradation in the regimes predicted by the support-gap diagnostics. Finally, we discuss deployment implications for privacy-sensitive and air-gapped settings, where interchangeable attention modules enable predictable latency and cost without external retrieval dependencies.

cs.AI · cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2605.09214

Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability

Qingyue Zhao, Kaixuan Ji, Heyang Zhao, Quanquan Gu

原文摘要

\emph{Kullback-Leibler} (KL) regularization is ubiquitous in reinforcement learning algorithms in the form of \emph{reverse} or \emph{forward} KL. Recent studies have demonstrated $ε^{-1}$-type fast rates for decision making under reverse KL regularization, in contrast to the standard $ε^{-2}$-type sample complexity. However, for forward-KL-regularized objectives, existing statistical analyses are either not applicable or result in $\tilde{O}(ε^{-2})$ slow rates. We take the first step towards addressing this problem via a streamlined analysis of forward-KL-regularized offline CBs. We give the first $\tilde{O}(ε^{-1})$ upper bounds in tabular and general function approximation settings, both under notions of \emph{single-policy concentrability}. In particular, our convex-analytical pipeline unifies these settings by exploiting the pessimism principle in a novel way and completely bypasses the proof routines in previous works based on the mean value theorem, which might be of independent interest. Moreover, we provide rate-optimal lower bounds, manifesting the tightness of our upper bounds in terms of statistical rates. Our lower bounds also demonstrate that the forward-KL-regularized sample complexity recovers the unregularized slow rate in the low-regularization regime, similarly to the reverse-KL regularization.

cs.LG · cs.AI · cs.IT · math.ST · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11794

ChemInsight: A Comprehensive Benchmark for Evaluating and Diagnosing Chemistry Understanding and Reasoning in Large Language Models

Changyu Hu, Yiming Zou, Xin Tang, Yiru Pan, Zicheng Huang, Deng Zhao, Qing Cui, Zhenduo Zhang 等 10 位作者

原文摘要

We introduce ChemInsight, a benchmark designed to evaluate large language models (LLMs) on chemistry tasks that require multi-step reasoning and structured chemical understanding. We propose a two-dimensional scoring system (scope and depth) to assess both task coverage and reasoning completeness. ChemInsight consists of 203 challenging chemistry questions organized into 50 bundles, which support the two-dimensional scoring evaluation scheme. We evaluate ten representative LLMs using ChemInsight and observe consistently low performance across models, with scope scores below 30% and depth scores below 11%, substantially lower than human expert performance. We then proceed to ask (1) What factor limits the advancement of current LLMs in chemistry? (2) To what extent are statistically driven language-model training methods effective at modeling chemistry symbolic language such as SMILES? And (3) Can LLM performance in chemistry be improved through intensive training on large, domain-specific corpora?

cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11792

APO2CS: Advantage-Aware Preference Optimization for Two-Level Constructive Search in Combinatorial Optimization

Kaixi Yang, Linbo Qiao, Wei Shang, Yanqi Shi, Dongsheng Li

原文摘要

Preference optimization has emerged as a promising alternative to reward-based training for neural combinatorial optimization. However, existing methods typically rely on binary pairwise labels, which discard the magnitude of solution-quality differences and provide supervision only at the complete-solution level. We propose Advantage-aware Preference Optimization for Two-level Constructive Search (APO2CS), a preference-based training objective for constructive neural solvers that integrates both macro-level and micro-level supervision. At the macro level, APO2CS enriches pairwise preferences with an advantage term derived from objective-value differences. At the micro level, it compares partial trajectories, thereby providing denser supervision throughout the sequential construction process. The resulting objective can be used either to train a policy from scratch or to fine-tune a pretrained one. Experimental results show that APO2CS improves the expressiveness and search capability of preference learning for combinatorial optimization, while demonstrating strong effectiveness and adaptability across multiple combinatorial optimization tasks.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.02267

Overcoming Demand Censoring through Randomization for Risk-Sensitive Newsvendor Learning

Zhengchao Wang, Nam Ho-Nguyen

原文摘要

We study online newsvendor learning with bounded demand, censored observations, and convex nonlinear shortage costs. Because stockouts can conceal the shortage magnitudes needed by gradient-based algorithms, we use randomized ordering to construct unbiased subgradient estimates. We characterize admissible randomization distributions within this framework and develop an online inventory policy that combines the resulting subgradient estimates with gradient-based updates. Our regret analysis quantifies the cost of randomization via the estimation variance and an additional loss term. We characterize the randomization distributions that minimize this objective. For piecewise-linear shortage costs, the optimal distribution is computed through a linear-time recursion; for costs with bounded curvature density, it is characterized by a scalar ordinary differential equation. The policy achieves expected regret of $O(T^{2/3})$ against the best fixed order in hindsight over $T$ periods. Under independent demand and suitable curvature of expected losses, the bound improves to $O(\sqrt{T})$. Matching minimax lower bounds establish rate optimality over the corresponding classes with a fixed marginal-cost bound. Both rates extend to nonperishable inventory under uniformly positive conditional expected demand. Numerical experiments on synthetic and retail data show improvements over alternative policies using censored observations.

cs.LG · math.OC

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2507.04380

Transferring Visual Explainability from Self-Explaining to Prediction-Only Vision Transformers via Task Arithmetic

Yuya Yoshikawa, Ryotaro Shimizu, Takahiro Kawashima, Yuki Saito

原文摘要

In image classification scenarios where both prediction and explanation efficiency are required, self-explaining models that perform both tasks in a single inference are effective. However, for users who already have prediction-only models, training a new self-explaining model from scratch is costly in both labeling and computation. This study proposes a method to transfer the visual explanation capability of self-explaining Vision Transformer (ViT) models learned in a source domain to prediction-only, zero-shot-capable ViT models based on vision-language models (VLMs) in a target domain via task arithmetic, without target-side explanation supervision. The proposed method endows target models with explanation capability by adding an explainability vector induced by explanation supervision in the source domain based on the task arithmetic framework. Experiments on 10 diverse target datasets show that a single explainability vector learned on ImageNet-1k augmented with patch-level explanation supervision transfers consistently, improving explanation quality while largely preserving classification accuracy. Within-model faithfulness evaluation and comparisons with control vectors (e.g., random vectors of the same norm) show that the transferred explanations are faithful to the model's own predictions and that the effect is specific to the explainability vector. Beyond the transfer itself, we further investigate whether transfer success can be anticipated a priori from model-internal features alone, and find that, for a fixed source, a small set of such features carries non-trivial signal about the transfer outcome on held-out targets. Code is available at https://github.com/yuyay/explainability_vector.

cs.CV · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11786

Payoff-Aware Prediction of Population Game Dynamics

Kang Wang, Xiao Wang, Bo Li, Renzhe Xu

原文摘要

Population game dynamics describe how aggregate behavior evolves in response to payoff signals. In many applications, the payoff function may change, and the goal is to predict the population trajectory induced by a new payoff design before trajectory data under that design are available. We study this problem as $\textbf{payoff-aware prediction of population game dynamics}$. Standard dynamics learning methods fit the motion observed under training payoffs, but do not separate the payoff-dependent incentive from the payoff-independent response structure needed for prediction under new payoffs. Building on the existing framework of Riemannian game dynamics, we propose PA-RmD, a payoff-aware learning method that keeps payoff information explicit and learns the payoff-independent response structure from data. Theoretically, we show that the learned model preserves fixed-payoff Nash equilibria and derive finite-time one-step prediction bounds under shared-response and payoff-coverage conditions. Empirically, we evaluate PA-RmD on synthetic zero-sum and potential games, as well as a YouTube Trending multi-country dataset, showing that our method performs well under changing payoff designs.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11782

CABrain: A Correlation-Aware Brain Foundation Model for EEG Decoding

SuGil Choi, Mooseop Kim, Chi Yoon Jeong

原文摘要

Electroencephalography (EEG) foundation models have emerged as promising paradigms for decoding brain activity through large-scale self-supervised pretraining. Notably, most existing approaches employ masked patch reconstruction, in which randomly selected EEG patches are masked and recovered by the network. However, purely random masking may limit representation learning as informative patches from highly correlated channels often remain unmasked, causing the network to underutilize weakly correlated channels or serial dependencies. Furthermore, because these models are typically based on transformer architectures, their performance is influenced by the choice of positional encoding. Although positional encoding has been extensively explored in natural language processing and computer vision, few studies have adapted it to the unique characteristics of EEG signals. To address these limitations, this study introduces a conditional positional encoding method based on graph convolutional networks (GCNs) to explicitly model inter-channel correlations, along with an adaptive masking strategy that enhances representation learning by modulating the difficulty of masked patch reconstruction. To the best of our knowledge, this is the first study to integrate graph-based representations into conditional positional encoding and introduce adaptive masking within EEG foundation model pretraining. Experimental results on ten benchmark datasets demonstrate that the proposed GCN-based positional encoding and adaptive masking strategies, both individually and in combination, achieve state-of-the-art or competitive performance, highlighting their effectiveness in EEG foundation modeling.

cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11780

Planners Exploit Deterministic Error: Dithered Low-Bit World Models

Yong Hwi Song, Keon Myung Lee

原文摘要

Sampling-based planners such as MPPI and CEM query a learned world model thousands of times per control step, so low-bit world models are an obvious way to make model-based control cheaper on a robot. With round-to-nearest activation quantization and no calibration, we find that planning can collapse, by up to 100 points of success, while the world model's latent one-step prediction error stays no larger than that of a stochastic rounding scheme that plans well. At the same bit width, what decides the outcome is not the size of the rounding error but whether it persists across planner iterations and is shared across candidates. Round-to-nearest leaves an error that persists, and an iterating planner drifts toward the candidates that this error favors; in TD-MPC2, holding the same dither fixed within a step or an episode brings the drift back. Dither shared by all candidates and redrawn at every iteration removes most of the damage at the same bit width and without training: at least half of it in all 19 affected Meta-World cells (task $\times$ bit width) of TD-MPC2 (median 95%), in 4 of 5 DINO-WM cells, and in all 8 affected tasks of a 317M-parameter multi-task TD-MPC2. Captured as CUDA graphs on an Ampere GPU, a W8A4 integer TD-MPC2 world model runs its planning rollouts 1.9-3.3$\times$ and the whole planning step 1.3-1.5$\times$ faster than FP32, though not faster than BF16 with pre-converted weights; shared dither adds at most 2.3% latency. In TD-MPC2, block-scaled NVFP4 removes most of the damage and MXFP4 part of it, which ties the failure to coarse grids with a shared scale. Hypotheses, thresholds and analysis code were fixed before the data that test them, and the hypotheses that failed are reported.

cs.RO · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2602.05243

CORP: Closed-Form One-shot Representation-Preserving Structured Pruning

Boxiang Zhang, Baijian Yang

原文摘要

Transformers achieve strong accuracy but incur high compute and memory cost. Structured pruning reduces inference cost, but most methods rely on retraining or multi-stage optimization, which limits post-training deployment. We propose CORP, a closed-form one-shot structured pruning method that removes MLP dimensions and attention substructures using only unlabeled calibration data without gradients or fine-tuning. CORP formulates structured pruning as a representation recovery problem. It models removed components as affine functions of retained components and derives closed-form ridge regression solutions that fold compensation into model weights. This minimizes a layer-local affine/logit reconstruction objective under the calibration distribution. Experiments on ImageNet with DeiT reveal strong redundancy in both MLP and attention representations. With CORP, models retain high accuracy under aggressive sparsity. On DeiT-Huge, CORP achieves 83.24% Top-1 accuracy after pruning 50% of both MLP and attention structures.

cs.LG · cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v5arXiv:2605.27991

Gradient-Flow Optimization as Dynamic Random-Effects Inference: Testing and Early Stopping with Applications to Deep Learning

Minhao Yao, Ruoyu Wang, Xihong Lin, Lin Liu, Zhonghua Liu

原文摘要

Gradient-flow optimization is usually viewed as an algorithmic procedure for minimizing empirical loss, with training duration selected by validation or heuristic early stopping rules. We develop a statistical inference framework for gradient-flow training. We show that whenever fitted values evolve through a time-invariant positive semidefinite training operator, the output at each time is equivalent to the best linear unbiased predictor under a corresponding random-effects model. Training time then becomes a variance-component parameter governing variance reallocation from residual noise to structured signal. This turns two training decisions into inferential problems: whether training is needed becomes a variance-component test for signal beyond initialization, and how long to train becomes restricted maximum likelihood (REML) estimation of the training-time variance component. We show that the REML-guided early stopping rule selects the time at which optimized spectral losses become decorrelated from the training-operator eigenvalues. The asymptotic prediction optimality of the REML-guided early stopping time is established for fixed-design in-sample risk and random-design out-of-sample risk. Deep learning models in fixed-kernel gradient regimes provide canonical instantiations for our results. Numerical experiments and a UK Biobank proteomics application show competitive accuracy of the REML-guided early stopping time with reduced reliance on validation splits and repeated checkpoint evaluation.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.29061

OccJEPA: Occlusion as Masking for Amodal Video Segmentation

Genpei Zhang, Runmin Jiang, Xingzu Zhan, Yibo Zhong, Ruien Li, Zhou Zhang

原文摘要

Amodal video segmentation asks for the complete mask of an object in every frame, including the parts hidden behind other objects. The best current methods fine-tune video diffusion models to generate these masks; they need a depth network and tens of denoising steps, and their outputs vary from sample to sample. Self-supervised video world models of the JEPA family are pre-trained on a closely related task, predicting the representations of masked space-time regions from the visible ones, yet their predictor is usually discarded after pre-training. We reuse it by treating occlusion as masking. Tokens not covered by the object's visible mask are dropped, the V-JEPA 2.1 predictor is fine-tuned to fill them in, and a lightweight decoder turns all features into amodal masks. The resulting model, OccJEPA (Occlusion-as-masking JEPA), is deterministic and 32x faster per frame than Diffusion-VAS, the diffusion-based state of the art. It is also more accurate on SAIL-VOS (+3.70 occluded-region mIoU over all occluded frames), on TAO-Amodal (+4.00 AP50 on a held-out test split, +4.32 with SAM2-tracked masks) and on Kubric. Controlled experiments trace most of the gain to the predictor's pre-trained weights rather than its capacity, and the predictor helps at every encoder size tested (87M to 1.85B parameters).

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.37364HF 社区推荐

EmbeddingGemma 2: Unified and Efficient Multimodal Representations

Henrique Schechter Vera, Sahil Dua, Jovan Andonov, Sara Smoot, Sebastian Russo, Ivan Llanos, Max Matveev, Jae Hyeon Yoo 等 77 位作者

原文摘要

We present EmbeddingGemma 2, an open multimodal embedding model which natively represents text, images, audio, video, and combinations thereof in a unified space. Our approach introduces improvements such as simplified per-layer embeddings and a multi-stage curriculum with modality and objective progression which ensure strong quality across unimodal, cross-modal, and mixed-modal evaluations. Notably, in comprehensive embedding benchmarks across all modalities, EmbeddingGemma 2 establishes a new Pareto frontier, outperforming both specialist models and models two to twelve times larger. Furthermore, our design offers flexibility for resource-constrained environments, supporting selective modality encoder loading, adjustable vision token budgets, truncatable embedding dimensions, and quantized weights. We present ablations analyzing key decisions and release EmbeddingGemma 2 to promote further research.

cs.LG · cs.CL · cs.CV · cs.SD

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11752

Spectral Rank and Step-Direction Dynamics in Language-Model Post-Training

Khurram Khalil, Khaza Anuarul Hoque

原文摘要

Language-model post-training modifies weights through cumulative updates and successive displacements, yet existing studies report seemingly conflicting spectral properties: nearly full-rank updates and low stable-rank changes. We show these observations are compatible and characterize update directions during training. For singular values following $σ_i \propto i^{-α}$ with $0 < α\leq 1/2$, we prove that stable rank becomes negligible relative to fixed-energy rank as dimension increases. Full singular value decompositions of selected OLMo-2-7B updates confirm this separation: three proximal policy optimization (PPO) attention updates have stable ranks of 9.6-18.3 but require 532-1,222 singular components to capture 80% of their energy. We next compute consecutive checkpoint-displacement cosines from full-model inner products. Across nine publicly released reinforcement learning from verifiable rewards (RLVR) stages, all 131 adjacent pairs have negative cosines on their native checkpoint grids, including PPO and Group Relative Policy Optimization (GRPO) trajectories. Supervised fine-tuning comparisons across four model families show configuration-dependent alignment, including negative late-training cosines for Mistral and FP32 Llama. Finally, a controlled OLMo-2-1B study with three paired seeds and a 2,048-token response cap compares group-relative advantages against a learned critic with generalized advantage estimation (GAE). At matched 60-rollout windows, late-training mean cosines are +0.259 and +0.012, respectively; the paired critic-minus-group difference is -0.247 (seed-level 95% interval [-0.358, -0.137]). These findings distinguish spectral concentration, energy coverage, and trajectory direction, and demonstrate that the advantage system influences directional alignment under matched training conditions.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.03180

GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations

Jonggwon Park, Seongeun Lee, Junhyun Park, Hannah Yun, Hyunwoong Kim, Sohyun Jeong, Hyewon Kang, Byungmu Yoon 等 9 位作者

原文摘要

Vision-language models (VLMs) for radiology have emerged as a scalable paradigm by leveraging image-report pairs naturally produced in clinical workflows. However, this pairing reveals a mismatch in scale: each finding occupies only a small region of the image, yet supervision is provided only at the global image-report level. This poses a central challenge: prior approaches spread weight densely across all patches rather than concentrating on the sparse subset relevant to a given query. To address this, we present GLINT (Gated Language-Image alignmeNT), a framework that models this sparse correspondence. On the alignment side, we introduce Sparsely Gated Alignment, a novel architecture in which a sigmoid gate over a separate gate embedding space activates only the patches relevant to each textual query, yielding sparsity. On the representation side, we add Dense Feature Regularization, which anchors the trainable encoder's intermediate features to a frozen self-supervised learning (SSL) teacher, preserving the fine-grained patch features that the gate relies on. The same recipe applies to both 2D chest X-ray (CXR) and 3D chest computed tomography (CT), built with DINOv3 and V-JEPA 2.1, respectively. GLINT enables zero-shot classification, grounding, and segmentation from free-text queries, and to our knowledge is the first to demonstrate zero-shot segmentation on 3D CT volumes without mask supervision. Notably, the most pronounced gains arise on zero-shot grounding and segmentation, where sparse, query-specific localization is required, consistent with our design intent. In downstream evaluation, GLINT outperforms both SSL encoders and medical VLMs on classification, report generation, and segmentation. Code is available at https://github.com/deepnoid-ai/GLINT.

cs.CV · cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11744

Breaking the $\sqrt{d}$ Communication Barrier in Federated Sampling with Adaptive Hamiltonian Monte Carlo

Jiajun Liang, Linxuan Wang, Guang Lin, Qifan Song

原文摘要

This work proposes the Adaptive Federated Hamiltonian Monte Carlo (AFHMC) for Bayesian federated learning (FL) tasks. The prior work on HMC applications for FL achieves $O(\sqrt{d}/ε)$ communication cost under log-convex distributions, already demonstrating its advantage over the Langevin dynamic-based counterpart. Based on a refined analysis, this paper further reveals the benefit of HMC for the FL regime by incorporating a control mechanism for local node shifts. We prove that AFHMC at most requires $O(\{d/ε^2\}^{1/3})$ communication cost up to a logarithmic term, which is significantly better than the existing results. Moreover, the proposed method adapts to client heterogeneity. Under low-heterogeneity settings (i.e., the target sampling precision $ε^2$ is higher than the heterogeneity level), the communication rate of AFHMC can be as good as $O(\log(d/ε^2))$, resembling the communication costs for FL optimization tools. To the best of our knowledge, this provides the first logarithmic communication complexity result for federated sampling, even though it requires low heterogeneity. Numerically, we also justify AFHMC's advantage under the low-heterogeneity regime. Our code is available at https://github.com/laaaarrywang/Adaptive-Federated-HMC

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2410.03756

The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainability

Judah Goldfeder, Victoria Dean, Zixin Jiang, Xuezheng Wang, Bing dong, Hod Lipson, John Sipple

原文摘要

Commercial buildings account for 17\% of U.S. carbon emissions, with roughly half of that from Heating, Ventilation, and Air Conditioning (HVAC). HVAC devices form a complex thermodynamic system, and while model predictive control and reinforcement learning have been used to optimize control policies, scaling to thousands of buildings remains a significant unsolved challenge. Most current approaches are over-optimized for specific buildings and rely on proprietary data or hard-to-configure simulators. We present the Smart Buildings Control Suite, the first open source interactive HVAC control benchmark with a focus on solutions that generalize across building. It has 3 components: real-world data from 11 buildings over 6 years, a lightweight data-driven simulator, and a modular Physically Informed Neural Network (PINN) building model. The buildings span multiple climates, management systems, and sizes, and both the simulator and PINN easily transfer to new buildings, ensuring solutions using this benchmark are robust to these factors and only reliant on fully scalable building models. This represents a major step towards scaling HVAC optimization from the lab to buildings everywhere. To facilitate use, our benchmark is compatible with the Gym standard, and our data is available at our webpage. Code at https://github.com/google/sbsim

cs.AI · cs.CY · cs.DC · cs.LG · eess.SY

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11732

Particle Filtering with Generative Trajectory Models for State Estimation and Control

Lukas Lao Beyer, Samvit Das, Sertac Karaman

原文摘要

Generative models can support both state estimation and control in robotics. For control, a common approach is to use the model as a policy that directly samples actions conditioned on observation histories. However, models that jointly represent states, actions, and observations can also predict how the robot state and its environment evolve. We investigate whether these predictions can improve state estimation and control through a change in inference procedure, without modifying the pretrained model. To this end, we use particle filtering to maintain competing hypotheses about state histories. A single pretrained diffusion model generates these hypotheses along with their predicted observations and associated action proposals. As new measurements arrive, particles are reweighted according to how well their predicted observations match the measurements. While particle filtering conventionally uses these weights only for state estimation, we also use them to guide action selection, favoring proposals associated with hypotheses that better explain the observations. Simulation experiments in partially observed navigation and vision-based manipulation demonstrate improvements in both state estimation and task performance compared to action generation without filtering, using the same pretrained model and no additional test-time information.

cs.RO · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2606.16933

A Unified Taxonomy and Evaluation Framework for Distributional Shifts in Reinforcement Learning

Ardianto Wibowo, Paulo E Santos, Amer Baghdadi, Matthew Stephenson, Karl Sammut, Jean-Philippe Diguet

原文摘要

Distributional shift in reinforcement learning (RL) is defined inconsistently across the literature, and it is studied alongside out-of-distribution (OOD) generalization and non-stationarity without a clear account of how the three terms are related. Methods are also often described as robust after evaluation under only a few particular changes, leaving the scope of such claims unclear. We develop a unified taxonomy that brings the three terms under one definition of distributional shift. We factorize the joint distribution of a one-step interaction into five components, each produced by its own generating function: initial-state distribution, observation mechanism, policy implementation (distinct from the policy being optimized), transition dynamics, and reward mechanism. A shift is specified by what changes (the changed components) and when it changes (the interaction regime: OOD, non-stationary, or combined), while changes in interaction data and performance are treated as observed effects. We complement the taxonomy with an evaluation framework that measures immediate generalization and subsequent adaptation, summarizing adaptation in a shift adaptation score (SAS). Published benchmark results, controlled Gridworld experiments with DQN, A2C, and PPO, and a mapping of selected studies show that method responses and rankings vary with the shifted component, and that existing evaluations cover only some components. Robustness claims should therefore be bounded by the components and regimes evaluated, since strong performance under one shift does not establish robustness under others. Identifying an unknown component-level source from observed interaction data remains a direction for future research.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11726

The Dark Matter of Clinical Data: An Identification Limit on Disentangling Physiology from Care Process in electronic health records

Mohammad Shahin, Krittaphas Chaisutyakorn, Dimitrios Proios, David Restrepo, Vishala Mishra, Boya Zhang, Aadarsh Sivaraman, Sparsh Roy 等 14 位作者

原文摘要

Representations learned from electronic health record (EHR) data encode patient physiology together with the clinical process that decides what is measured, when, and for whom. We formalize the data-generating process with latent physiology, ordering intent, laboratory values that exist only when measured, a learned representation, and clinical environments, and analyze two strategies for undoing this entanglement. First, no division of the information carried by laboratory "value" and "ordering" channels is identified from a single dataset: when ordering depends on the analyte, its distribution among untested patients is not recoverable, and a sentinel code for the undefined recorded value makes it carry all the entropy of the ordering indicator, fixing partial information decomposition (PID) for every redundancy measure satisfying the Williams-Beer axioms and invariant under injective re-encoding. Second, cross-site invariance of representation-conditional outcomes is a falsifiable implication of biological invariance taken jointly with exclusion and representation-sufficiency conditions, but not an identification strategy: we construct observationally equivalent models that disagree about biological invariance, whether the invariance test fails or passes. For binary physiology this happens exactly when case mix differs, the representation is lossy, and physiology affects the outcome. A worked example yields a 9.6-percentage-point cross-site gap under an unchanged biological mechanism. Tests performed for every patient remove the ordering gate, and measured tests can restore identification: biological invariance is identified when a test exactly summarizes outcome-relevant physiology, or, for binary physiology, when two tests with independent errors are available at sites with different case mix. Cross-site analyses should be reported as tests of a stated joint hypothesis.

cs.LG · stat.ME

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.11725

The Value of Not Knowing: How Fresh Eyes through Naive LLMs Fuel Creativity

David B. D'Ambrosio, Alon Albalak, Joel Lehman, Kenneth O. Stanley

原文摘要

Many major scientific advances come from approaching a problem with a new or novel perspective. While large language models (LLMs) with internet-scale knowledge seem poised to take such perspectives, their consistent generation remains elusive. Instead, models are drawn to dominant views and preexisting, in-field bias. To explore the potential of fresh eyes to increase creativity, this paper proposes naive LLMs, models that are made intentionally ignorant of a specific topic. Through deprivation in training or intentional forgetting, naive models are unburdened by existing dogma, allowing them to be free to make new associations and conjectures. These naive models can then be paired with a stronger mentor model that can refine their unique perspectives into concrete proposals. For the 1B and 7B models tested in this paper, naive models consistently produce seed ideas that are more novel and useful than baseline counterparts, and mentor refinement improves the ideas even further. Thus, contrary to the prevailing tradition of scaling up training data, we demonstrate a case where less information is actually more powerful, in particular when the aim is to discover genuinely outside-the-box solutions.

cs.AI · cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.19882

Multimodal Concept Bottleneck Models

Tongqing Shi, Ge Yan, Tuomas Oikarinen, Tsui-Wei Weng

原文摘要

Concept Bottleneck Models (CBMs) enhance the interpretability of deep neural networks by aligning the features extracted from images with human-interpretable concepts. However, conventional CBMs typically use a linear classifier that maps concept activations to a fixed set of predefined classes, limiting their flexibility in open-vocabulary settings. In this paper, we propose Multimodal Concept Bottleneck Model (MM-CBM) to address these issues and extend CBMs into CLIP structure. MM-CBM utilizes dual Concept Bottleneck Layers (CBLs) to align both the image and text embeddings into interpretable features. This allows us to perform new vision tasks like zero-shot classification or image retrieval in an interpretable way. Compared to existing methods, MM-CBM achieves 23.28% accuracy improvement on average across four standard benchmarks. Our method maintains competitive performance compared with black-box CLIP model while offering interpretability. Under classification tasks, MM-CBM's accuracy is on average 3.94% lower than CLIP in the target-dataset adaptation setting and 9.84% in the zero-shot transfer setting. Under image-text retrieval tasks, it achieves an average gap of 2.48% for image retrieval and 9.22% for text retrieval.

cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗