跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1新增论文
+1其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-13 07:07 · 全站最近入库：2026-11-13 06:39各信源进度 →

统计口径与数据新鲜度

统计 2026-11-12 07:16 — 2026-11-13 07:16（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-13 07:07最近发现新增版本：2026-11-13 06:39库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,676 已收录论文18,324 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 4,904 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-13 07:05（北京时间）
版本更新 · v3arXiv:2503.03283

Exploring specialization and sensitivity of convolutional neural networks under simultaneous image augmentations

Pavel Kharyuk, Sergey Matveev, Ivan Oseledets

原文摘要

Drawing parallels with the way biological networks are studied, the authors adapt the treatment--control paradigm to explainable artificial intelligence research and enrich it through multi-parametric input alterations. A framework for investigating internal inference impacted by input data augmentations is proposed to address the real-world scenario with multiple simultaneously applied treatments instead of observing a single effect. The internal changes in network operation are reflected in activation changes measured by variance, which can be decomposed into components related to each augmentation by employing Sobol indices and Shapley values. To facilitate distinguishing model-related specifics from the self-interactions of augmentations, an auxiliary variable responsible for their permutations has been incorporated into the analysis. The resulting quantities enable one to visualize sensitivity to different variables and use them as features for further analysis and processing, including guided masking of activations. In addition, a method for single-class sensitivity analysis is introduced through a pipeline that involves class selection according to their match with the prediction bias generated by targeted damaging of the activations. Three convolutional networks, namely AlexNet, VGG11, and ResNet18, have been evaluated via the designed framework, with two vision transformers, ViT-b-16 and Swin-t, partially complementing the findings. Two sets of augmentations, consisting of five and seven distinct input transformations applied simultaneously, have been utilized as sources of variance in the image domain. Relying on the observed parallels, the authors assume that the developed framework can potentially be transferred to the study of biological neural networks in complex environments, as well as enhance the design of autonomous decision-making systems.

stat.ML · cs.AI · cs.LG · math.NA

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10579

Predictive Modelling of Solid Waste Generation through Time Series Analysis

Mitra Tithi Dey, Punyasha Chatterjee, Amlan Chakrabarti

原文摘要

Reliable forecasting of Municipal Solid Waste (MSW) production is pivotal for sustainable urban management, as it enables efficient resource planning, optimized collection schedules, and reduced environmental impacts. Existing works on MSW forecasting have some limitations, such as unavailability of granular data from lower administrative units, small datasets, limited or sometimes no consideration of important factors that can affect MSW generation, including holidays, weekends, seasonality, changes in temperature or humidity. This work presents a comprehensive comparative analysis of four popularly used time series forecasting models, namely Seasonal Autoregressive Integrated Moving Average with Exogenous Regressors (SARIMAX), Prophet, Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU), using a real-world MSW dataset from Sri Lanka spanning 2012-2018. A detailed factor-wise analysis is conducted to examine waste generation behaviour considering holidays, weekday-weekend variations, seasonal patterns, temperature and humidity. The results show lower waste generation during holidays and weekends, higher generation during Summer and Winter, and comparatively lower generation during Monsoon and Post-Monsoon. Temperature and humidity do not exhibit a direct linear relationship with waste generation, although their seasonal variations are associated with the changes in waste generation across different seasons. All model performances are evaluated using Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Root Mean Squared Error (RMSE), and the coefficient of determination (R2). Among the evaluated models, LSTM and SARIMAX achieve the best overall forecasting performance.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.37361

Online Interval Selection with Binary Predictions: Optimal Consistency and a Continuous Robustness Tradeoff

Zhong Han

原文摘要

We study online interval selection with binary predictions of membership in a final optimal solution. Requests arrive in arbitrary order; accepted requests may be revoked, but rejected or revoked requests cannot be recovered. With rewards proportional to interval length, the best consistency under finite strict robustness is $d=(3+\sqrt5)/2$ when both guarantees hold almost surely on each fixed input, including for randomized algorithms. A continuous lower bound forces consistency strictly above $d$ at every robustness budget below $4\varphi+3\approx9.472$, whereas our deterministic algorithm attains $d$ with exact robustness $\varphi^5\approx11.090$, where $\varphi=(1+\sqrt5)/2$. Each held interval inherits a ledger recording positive prediction mass lost through irrevocable decisions. Tuning the replacement costs and both special thresholds gives a continuous family with exact consistency throughout and exact robustness on two parameter intervals, including $(C,R)=(25/9,10)$ and $(4,7)$. Randomization permits expected consistency approaching one with finite expected robustness, whereas almost-sure robustness forces expected consistency strictly above two at every finite budget.

cs.DS · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.37675

LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling

Biswajit Banerjee, Claudia Alvarez Carreno, Anton S. Petrov

原文摘要

Protein Language Models (PLMs) have made remarkable progress following scaling laws established in natural language processing across sequence- and structure-based tasks, yet the potential of tokenization remains underexploited. Unlike human language, proteins preserve structure despite extensive sequence variation, a property standard tokenization strategies fundamentally fail to capture. We introduce ZEST (Zoned Encoding of Sequence Traits), an evolution-informed vocabulary derived from conserved regions of multiple sequence alignments. ZEST allows embedding domain-level biological priors directly at the tokenization stage rather than learning them implicitly through scale. ZEST natively compresses sequences to an average token length of 3.3 residues, enabling our model to process $\geq3380$ residues within a standard 1024-token context window. Building on this, we present LEMON (Layered Extraction of Molecular Ordering from Nature), a compact 200M-parameter sequence-based model for detection of remote homology between protein sequences trained on a single H100 GPU for one week. Despite its modest size, LEMON outperforms models of 600M to 3B parameters and is on-par with TM-Vec. Our results demonstrate that evolution-informed tokenization can substitute for massive parameter scaling, opening a new direction for efficient, biologically-grounded protein representation learning. All code and model weights are publicly available (at \href{https://github.com/LDWLab/ZEST}{github.com/LDWLab/ZEST}) under the MIT license.

cs.LG · q-bio.BM

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.22893

L-FAME: Longitudinal Focused Attention Meditation EEG Dataset and Benchmark

Angqi Li, Ab Basit Rafi Syed, Hamzeh Alzweri, Taosheng Liu, Barry H. Cohen, Saiprasad Ravishankar

原文摘要

We introduce a novel Longitudinal Focused Attention Meditation Electroencephalography (L-FAME) dataset and an accompanying benchmark, designed to foster research into the neural effects of various meditation practices and the evolution of these effects over a six-week training period. The dataset includes baseline EEG recordings from 74 healthy college participants, of whom 44 completed the six-week intervention and returned for post-intervention recordings, along with psychological assessments. Participants were randomly assigned to one of three distinct meditation groups: two mantra-based techniques (SA-TA-NA-MA and Hare Krishna) and one Breath Focus practice. Leveraging this unique longitudinal and comparative dataset, we propose a benchmark suite comprising three distinct classification tasks: (1) cognitive state decoding to distinguish between resting and meditation states, (2) fine-grained classification of the specific meditation techniques, and (3) cross-session adaptation to evaluate model generalization across the longitudinal time gap. We provide comprehensive baseline results for these tasks utilizing a range of classical machine learning algorithms and deep learning architectures. The complete dataset, preprocessing pipelines, and benchmark evaluation code are being publicly released, offering a valuable resource and a standardized framework for the development and comparison of new analytical methods in computational meditation research and EEG-based machine learning. The dataset is available at https://huggingface.co/datasets/L-FAME-Dataset-Benchmark/L-FAME

eess.SP · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2608.01745

Lagrangian or Lyapunov? Multi-Agent Reinforcement Learning under Coupled Finite-Horizon Constraints

Yeonseo Jeong, Wonhyeok Ko, Songnam Hong

原文摘要

Radio resource management in dense networks is subject to budgets that must hold on every realized trajectory, carried by different agent groups: an energy budget at each base station (BS) and a handover budget at each user, coupled through the association decisions that consume both. Constrained multi-agent reinforcement learning (MARL) handles such budgets using Lagrangian multipliers updated across episodes or Lyapunov virtual queues updated per slot, but their relative behavior under coupled two-sided constraints remains unclear. We formulate this problem and introduce a windowed proportional-fairness objective that captures the temporal distribution of service beyond the classical horizon-average formulation. We characterize the distinct budget-regulation timescales of the two mechanisms through complementary theoretical bounds and identify queue-price buildup as a key mechanism governing finite-horizon budget pacing. Using a common heterogeneous MARL architecture, we compare three utility surrogates and both constraint-handling mechanisms. The results show that utility design primarily determines throughput and fairness, whereas constraint-price dynamics govern training efficiency and budget pacing. Under the default setting, the queue-based PF policy reaches near-final performance after a single training episode, but its unscaled penalty can lead to delayed pacing under tight budgets. Guided by the analysis, we show that queue-price calibration reduces the empirical drift constant by an order of magnitude and substantially narrows the performance gap to Lagrangian methods, which adapt constraint prices across episodes at higher training cost. These findings provide practical design guidelines for constrained MARL.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10570

Deep backward multistep high-order regression methods for high dimensional nonlinear parabolic partial differential equations

Qiang Han, Shaolin Ji, Zihao Xv

原文摘要

We propose a deep backward multistep high-order regression (MHDBR) method for semilinear parabolic partial differential equations through their backward stochastic differential equation representations. The method combines multistep time discretization with regression targets formed by conditional expectations. Under explicit regularity, starting-value, forward-approximation and network-moment assumptions, we prove an error bound separating time discretization, conditional Monte Carlo sampling and neural-network approximation. The analysis assumes exact population minimization; finite-training-sample generalization and optimization errors are not included. For generators independent of the gradient variable, suitably constrained GroupSort networks and sufficiently large conditional sample sizes give order $J$ in the stated nodal $L^2$ error norm. The required network sizes depend on dimension, so this result does not establish dimension-independent complexity. We present a reproducible time-discretization test satisfying the analytical assumptions and three broader neural-network benchmarks, whose relation to the assumptions is discussed explicitly.

stat.ML · cs.LG · math.AP

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2611.06228

Separation Is a Riesz Energy: How Many Environments Does ICA Need?

Yuda Bi, Yubo Guo, Jiaxiao Cai, Xiaonai Li, Zhuolin Li, Vince D. Calhoun

原文摘要

How many environments and samples does it take to separate $d$ Gaussian sources whose variances change across environments? Each source has a signature, its vector of variances. The Riesz $2$-energy $E_j$ of source $j$ sums the inverse Fisher informations for rotating it towards other sources. With known whitening, we prove a worst-case squared-angle risk lower bound of order $E_j/(n+E_j)$ for orthogonal estimators at every $n$ and $d$. A quadratic pencil given the signatures attains $E_j/n$ up to a constant above a separation-dependent sample threshold, even with estimated whitening. For well-spread signatures at fixed $K$, the energy grows as $d^2$, $d\log d$ and $d$ for $K=2$, $3$ and $K\ge4$, respectively. A design with $K\asymp\log d$ and pairwise Fisher information bounded below by a constant makes $d\log d$ samples sufficient. The eigenvectors of a single fixed linear combination have an average first-order error coefficient of at least order $d^2$. Simulations up to $d=1000$ support the first-order predictions, and on task fMRI the energy helps rank split-half instability.

cs.LG · cs.IT · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.37413

Displacement Moments and Path-Terminal Separation in Brownian Interpolant Sampling

Zhong Han

原文摘要

We prove early-stopped, target-to-output KL guarantees for Brownian interpolant Euler sampling from the true source under arbitrary Borel couplings, even with nonintegrable endpoints. At fixed stopping time and noise level, a fourth displacement moment yields an $O(N^{-1})$ discretization term. A second moment suffices for fixed-law convergence with grid refinement and vanishing drift error. For one fixed coupling with no displacement moment above two, the original and comparison paths have KL of orders one and $1/\log N$ relative to the exact-drift Euler path. On the same geometric grid, terminal KL is at most $C/N^2$. The path decomposition identifies the information about the source discarded by Markov projection. When the original bridge is Markov, current-state Euler sampling has an $O((M_2+d)/N)$ discretization term. We prove expected stopped-output KL consistency for a learner trained on iid endpoint pairs with fresh true-source initialization. In a spatial image control, iterative sampling has worse fair energy scores than direct prediction with resampled residuals. Uniform terminal rates for general second-moment couplings remain open.

cs.LG · math.PR

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.16632

Learning How to Cube

Ferhat Erata, Sam Kouteili, Thanos Typaldos, Timos Antonopoulos, Robert B. Jones, Byron Cook, Ruzica Piskac

原文摘要

Despite the effectiveness of Cube-and-Conquer (C&C) for solving challenging Boolean Satisfiability (SAT) problems, no prior work has shown that transformer-based models can learn effective cubing heuristics. We introduce a neuro-symbolic post-training framework for this task. We design an MCTS-based data curation pipeline that uses symbolic heuristics to explore splitting decisions over SAT competition formulas, producing preference data grounded in solver statistics and augmented with reasoning traces from a teacher model. Our two-stage post-training, supervised fine-tuning (SFT) followed by direct preference optimization (DPO), enables a 4B-parameter model to achieve a pass@5 score of 53 on 100 SAT competition benchmarks, surpassing frontier LLMs such as Claude-Sonnet-4 (50) and matching, with five attempts per benchmark, the best symbolic heuristic (53), which nonetheless remains preferable for a single attempt. Ablations show that SFT alone improves pass@5 from 46 to 51, with DPO adding 2 additional benchmarks; an entropy/agreement ablation on realized first-cube decisions further shows that SFT, not DPO, accounts for the root-level decision diversity that produces complementary per-run coverage over deterministic symbolic methods. This demonstrates that transformers can be trained to make effective cubing decisions in a domain traditionally dominated by symbolic methods.

cs.LG · cs.AI · cs.LO

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10563

Confronting Uncertainty in Mechanism-AI Coupled Dynamical Systems via Optimal Transport and Information Geometry: Part I. Distribution Learning

Pengfei Song, Yanni Xiao, Shuangshuang Yin

原文摘要

Mechanism-AI coupled dynamical systems are increasingly used to study complex biological systems. However, uncertainty associated with noisy, sparse, and partially observed data limits their reliability. This paper presents a systematic mathematical formulation for treating these uncertainties from the perspective of optimal transport and information geometry. We formulate the uncertainty quantification problem as the minimization of an energy functional on a probability density space equipped with a Riemannian metric, with the loss evaluated through a differential equation, and give a unified description of the gradient flows of the energy functional under specific metrics, such as the Wasserstein and Fisher-Rao metrics. Furthermore, we present corresponding numerical methods for solving these gradient flows using abstract adjoint sensitivity analysis. The formulation connects a range of Bayesian inference methods for dynamical systems; numerical experiments compare posterior approximation and predictive uncertainty under a common computational budget. Moreover, it also provides a geometric perspective on dynamical modeling and the study of heterogeneity evolution.

cs.LG · math.DS · math.MG · math.PR · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2603.05432

Ensembling Language Models with Sequential Monte Carlo

Robin Shing Moon Chan, Tianyu Liu, Samuel Kiegeland, Clemente Pasti, Jacob Hoover Vigly, Timothy J. O'Donnell, Ryan Cotterell, Tim Vieira

原文摘要

Combining the predictions of multiple models is one of the most effective strategies in machine learning. Thus, it is only natural to investigate the ensembling of language models (LMs). However, because LMs define distributions over strings, a proper ensemble requires aggregating predictions at the string level, which requires a computation that is generally intractable. In this work, we cast LM ensembling as an inference problem and propose the deployment of sequential Monte Carlo (SMC), an appropriate approximate inference scheme. Concretely, we introduce a unified framework for composing multiple LMs into ensemble distributions parameterized by a broad family of aggregation functions. To sample from these distributions, we introduce an SMC algorithm that operates in a shared byte space, enabling ensembles of models with mismatching vocabularies and consistent sampling in the limit. We evaluate a range of ensembles across prompt and model combinations for various text generation tasks, finding that consensus-seeking strategies such as the product consistently outperform probability averaging, and that better posterior approximations can yield better ensemble performance.

cs.CL · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.00686

A Study of Hidden-State Optimization Order in Predictive Coding Networks

Xueyuan Li, Danilo Vasconcellos Vargas

原文摘要

Local learning methods offer an alternative to end-to-end backpropagation, but their unstructured local objectives can produce weak feature learning in deep networks. We study whether the order of hidden-state optimization can address this limitation. We propose a boundary-first inference schedule that partitions a model into chunks, first coordinates hidden states at chunk boundaries, and then refines representations within each chunk. We instantiate this schedule in predictive coding networks (PCNs), a local-learning framework in which hidden activities and prediction errors are explicitly exposed during inference. On CIFAR-10, the resulting boundary-first predictive-coding instantiation improves accuracy over standard predictive coding by $9.77\%$ under a standard parametrization and by $5.51\%$ under a $μ$-parametrization. Diagnostic analyses further show more non-trivial early-layer updates, lower initial-to-final CKA, and more diverse layerwise gradients, consistent with stronger feature learning. These results support boundary-first, chunk-based inference as a practical design principle for predictive-coding training and motivate its study in broader local-learning systems.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10560

Large Language Models without QK-Normalization

Yongding Tao, Yitong Qiao, Tianqi Shao, Tian Wang, Zhixuan Chu, Xiaoming Wu, Yihong Dong, Ge Li

原文摘要

Query-Key Normalization (QK-Norm) is widely adopted in modern large language models to control the growth of attention logits and improve training stability. However, query and key norms are not merely sources of instability. The query norm controls the sharpness of the attention distribution, while the key norm determines how strongly a token attracts different queries. These norms therefore carry token-specific information about attention selectivity and token importance. By normalizing both, QK-Norm suppresses this information. This raises a natural question: can we control query-key representations without discarding their sensitivity to magnitude? To address this question, we propose QK-MAP, a Magnitude-Aware Projection for constructing queries and keys. Rather than removing magnitude through normalization, QK-MAP transforms the input representation using multiple learnable frequencies, so that changes in magnitude are encoded as bounded phase patterns. This allows magnitude to be represented through phase and direction rather than vector length alone. Its periodic features have a fixed norm while remaining sensitive to magnitude changes, separating magnitude encoding from norm growth. Multiple frequencies further help distinguish magnitudes and reduce periodic ambiguity. Experiments across model scales from 190M to 7B parameters show that QK-MAP achieves lower training and validation loss than both Standard attention and QK-Norm. Further analyses show that QK-MAP preserves sensitivity to input scale, assigns less attention to the first token, and produces stronger attention contrast for relevant evidence in synthetic long-context tests. These results demonstrate that query-key scale can be effectively controlled without sacrificing token-specific magnitude information.

cs.CL · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2605.08506

Learning Polyhedral Conformal Sets for Robust Optimization

Shuyi Chen, Wenbin Zhou, Shixiang Zhu

原文摘要

Robust optimization (RO) provides a principled framework for decision-making under uncertainty, but its performance critically depends on the choice of the uncertainty set. While large sets ensure reliability, they often lead to overly conservative decisions, whereas small sets risk excluding the true outcome. Recent data-driven approaches, particularly conformal prediction, offer finite-sample validity guarantees but remain largely task-agnostic, ignoring the downstream decision structure. In this paper, we propose a decision-aware conformal framework that learns uncertainty sets tailored to robust optimization objectives. Our approach parameterizes a flexible family of polyhedral sets via data-driven hyperplanes and learns their geometry by directly minimizing the induced robust loss, while preserving statistical validity through conformal calibration. To correct for data-dependent selection, we incorporate a re-calibration step on an independent dataset to restore coverage. The resulting sets capture directional and anisotropic uncertainty aligned with the decision objective while remaining computationally tractable. We provide finite-sample coverage guarantees and bounds on the sub-optimality gap to an oracle decision. This work bridges the gap between statistical validity and decision optimality, providing a principled framework for data-driven robust optimization.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10556

onóna: Twin Embeddings with Tied Unembeddings

Cheng Luo, Xinnan Dai, Zhongzhu Zhou, Zefan Cai, Ruijie Zhu, Tianyu Zhang, Junjie Hu

原文摘要

Weight tying reduces the parameter cost of language models, but the usual tied output head is limited to a single softmax. We introduce onóna, a two-table parameterization that increases output expressiveness while preserving input-output weight sharing. The Transformer takes the average of two embedding tables as input, and the output head combines the softmax distributions defined by those same tables. A gate conditioned only on the current input token determines the mixture weight. Its values can be precomputed for inference, and the Transformer body remains unchanged. We evaluate onóna across Dense, Gated DeltaNet hybrid, mixture-of-experts, and Attention Residual architectures at 142M, 330M, and 1B scales. Under matched token budgets, onóna lowers final validation loss and external perplexity in 11 of 12 comparisons and raises the average accuracy over eight downstream tasks in 10 of 12. The one exception on loss is the 1B MoE model. The cost is one extra embedding table and one extra vocabulary projection.

cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.07654

Reliable Chain-of-Thought via Prefix Consistency

Naoto Iwase, Yuki Ichihara, Mohammad Atif Quamar, Junpei Komiyama

原文摘要

Large language models often improve accuracy on reasoning tasks by sampling multiple chain-of-thought (CoT) traces and aggregating them with majority voting, a test-time technique called self-consistency. When we truncate a CoT partway through and regenerate the remainder, we observe that traces with correct answers reproduce their original answer more often than traces with wrong answers. We use this difference as a reliability signal, prefix consistency, that weights each candidate answer by how often it reappears under regeneration. It requires no access to token log-probabilities or self-rating prompts. Across five reasoning models and four math and science benchmarks, prefix consistency is the best correctness predictor in most settings, and reweighting votes by this signal reaches the plateau accuracy of majority voting with up to $21\times$ fewer tokens (median $4.6\times$). Our code is available at https://github.com/naoto-iwase/prefix-consistency.

stat.ML · cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10553

Meta-Reinforcement Learning with Zero-Shot Reinforcement Learning

Jake Grigsby, Siddhant Agarwal, Yu Lei, Leonidas Varveropoulos, Yuke Zhu

原文摘要

Meta-reinforcement learning (meta-RL) trains agents to adapt to novel tasks from test-time experience, but requires diverse training environments and reward functions. While environments are becoming cheaper to generate, it remains difficult to anticipate every objective an agent may be asked to pursue at test time. Behavior Foundation Models (BFMs) learn policies from reward-free data and support zero-shot RL (ZSRL): adaptation to new objectives at test time using a large reward-labeled dataset. Recent work has shown that BFM task inference can instead be performed online without prior task-specific data. We note that this capability positions BFMs and meta-RL as alternative approaches for online adaptation to new reward functions in a fixed environment. We then ask whether we can extend BFMs to compete with meta-RL more broadly, including in novel environments, and identify four challenges: environment identification, online data collection, test-time exploration, and learning from mixed reward supervision. We study each individually, then propose MetaBFM, a hybrid method that addresses all four. During training, it combines supervised reward-following with unsupervised learning; at test time, it can interpolate between exploration, exploitation, and ZSRL-style reward inference. We demonstrate that MetaBFM can recover the in-distribution performance of supervised meta-RL while generalizing to reward functions outside the training set. Our work suggests that BFMs and meta-RL are complementary research directions whose combined strengths allow adaptive agents to learn more diverse behaviors from a fixed set of supervised objectives.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.02986

Mitigating Reward Hacking in Reinforcement Learning from Human Feedback via Advantage Sign Robustness

Shinnosuke Ono, Johannes Ackermann, Soichiro Nishimori, Takashi Ishida, Masashi Sugiyama

原文摘要

Reward models (RMs) used in reinforcement learning from human feedback (RLHF) are vulnerable to reward hacking: as the policy maximizes a learned proxy reward, true quality plateaus or degrades. We make the assumption that reward hacking is often caused by flipped advantage signs: instead of reducing the likelihood of a bad response, a flipped sign causes the update to increase it. By considering an adversarial perturbation in the RM parameter space, we can derive a certified sign-preservation radius, which is the smallest perturbation that can flip the advantage sign during policy optimization. Based on this formulation, we propose Sign-Certified Policy Optimization (SignCert-PO), down-weighting non-robust completions in the policy gradient update. Unlike prior approaches that require multiple RMs or access to the RM training data, SignCert-PO is lightweight and operates purely at the policy optimization stage using only the RM parameters and on-policy completions. On TL;DR summarization and AlpacaFarm benchmarks, SignCert-PO consistently achieves a better win rate than baselines and reduces reward hacking.

cs.LG · cs.AI · cs.CL

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10550

Dual-Granularity Learning for Regression with Continuous Noisy Labels

Hui Guo, Boyu Wang, Grace Y. Yi

原文摘要

Learning with noisy labels has been extensively studied for classification, yet its regression counterpart remains relatively underexplored. Unlike classification, regression involves continuous-valued targets where label noise exhibits heterogeneous magnitudes across instances, which poses conflicting requirements on the model: correcting small-to-moderate deviations requires sensitivity to label values for instance-level refinement, whereas large deviations caused by corrupted or outlying annotations render exact label values unreliable, necessitating reduced reliance on exact labels and instead leveraging ordinal structure among samples. These requirements call for methods that capture both global structure and sample-specific deviations in a unified manner. To this end, we propose GRAIN, a dual-granularity framework for robust noisy regression. At the fine-grained level, we derive an explicit residual learning mechanism from a robust loss formulation, introducing instance-specific auxiliary variables to decouple label noise from predictive signals. At the coarse-grained level, we introduce a binning-based contrastive regularization that enforces an order-preserving feature structure aligned with label proximity, addressing noise beyond the capacity of local residual refinement. The two levels are coupled via a histogram-based bridging mechanism that enables bidirectional information flow: coarse-grained structural guidance establishes a well-ordered feature space that anchors residual learning, while fine-grained residual refinement corrects instance-level deviations that structural regularization alone cannot resolve. Experimental results show that GRAIN consistently improves robustness and generalization under diverse noise conditions.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2605.11473

TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing

Yuanpeng Li, Gefei Lin, Annie Qu, Rui Miao

原文摘要

Soft Actor-Critic (SAC) and its variants dominate Multi-Task Reinforcement Learning (MTRL) due to their off-policy sample efficiency, while on-policy methods such as Proximal Policy Optimization (PPO) remain underexplored. We diagnose that PPO in MTRL suffers from a previously overlooked issue: critic-side gradient ill-conditioning, which may cause tail tasks to stall while easy tasks dominate the value function's updates. To address this, we propose TOPPO (Tail-Optimized PPO), a reformulation of PPO via Critic Balancing -- a set of modules that improve gradient conditioning and balance learning dynamics across tasks. Unlike prior approaches that rely on modular architectures or large models, TOPPO targets the optimization bottleneck within PPO itself. Empirically, on the Meta-World+ MT50 benchmark, TOPPO achieves stronger mean and tail-task performance than prior SAC-family and ARS-family methods while using substantially fewer parameters and reaching their final performance in fewer environment steps. Notably, TOPPO matches or surpasses strong SAC baselines early in training and maintains superior performance at full budget. Ablations confirm the effectiveness of each module in TOPPO and provide insights into their interactions. Our results demonstrate that, with proper optimization, on-policy methods can rival or exceed off-policy approaches in MTRL, challenging the prevailing reliance on SAC and highlighting critic-side gradient conditioning as the central bottleneck.

cs.AI · cs.LG · cs.RO · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2602.00158

RAPTOR: Ridge-Adaptive Logistic Probes

Ziqi Gao, Yaotian Zhu, Qingcheng Zeng, Xu Zhao, Ziqing Wang, Feng Ruan, Kaize Ding

原文摘要

Probing studies what information is encoded in a frozen LLM's layer representations by training a lightweight predictor on top of them. Beyond analysis, probes are often used operationally in probe-then-steer pipelines: a learned concept vector is extracted from a probe and injected via additive activation steering by adding it to a layer representation during the forward pass. The effectiveness of this pipeline hinges on estimating concept vectors that are accurate, directionally stable under ablation, and inexpensive to obtain. Motivated by these desiderata, we propose RAPTOR (Ridge-Adaptive Logistic Probe), a simple L2-regularized logistic probe whose validation-tuned ridge strength yields concept vectors from normalized weights. Across extensive experiments on instruction-tuned LLMs and human-written concept datasets, RAPTOR matches or exceeds strong baselines in accuracy while achieving competitive directional stability and substantially lower training cost; these quantitative results are supported by qualitative downstream steering demonstrations. Finally, using the Convex Gaussian Min-max Theorem (CGMT), we provide a mechanistic characterization of ridge logistic regression in an idealized Gaussian teacher-student model in the high-dimensional few-shot regime, explaining how penalty strength mediates probe accuracy and concept-vector stability and yielding structural predictions that qualitatively align with trends observed on real LLM embeddings.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10544

Compact Quantum Channel Attention for Static ASL Recognition: A Controlled Study with Parameter-Matched Baselines

Ronak Bhalgami, Deep Kothadiya, Nikita Bhatt

原文摘要

Hybrid quantum-classical models often report good accuracy on image tasks, but it is rarely clear whether the quantum part is responsible. This paper proposes QCA-DenseNet, a static American Sign Language (ASL) recognition pipeline in which an eight-qubit variational circuit acts as a channel-attention gate on DenseNet121 features. The quantum channel attention (QCA) gate has 17,464 parameters, 7.51 times fewer than a squeeze-and-excitation (SE) block. The pipeline is tested on 1,815 MU HandImages images by holding out one of five signers at a time, and a frozen-backbone ablation uses small parts of the Kaggle ASL Alphabet data. On unseen signers, QCA-DenseNet reaches 94.50% mean accuracy, above SE (93.72%) and within 0.28 points of the best model. A matched classical MLP and a model without attention perform about the same, and no difference is significant. QCA gives a much smaller attention gate without a detectable loss in accuracy, but no quantum advantage.

quant-ph · cs.AI · cs.CV · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10543

Intra-Option Fitted Q-Evaluation: Evaluating Hierarchical Policies from Non-Hierarchical Data

Yunfu Deng, Josiah P. Hanna

原文摘要

Off-policy evaluation (OPE) estimates the expected return of a target policy from previously collected data without additional environment interaction. While OPE methods for flat Markovian policies are well studied, little work has addressed evaluating hierarchical policies in which a high-level policy selects temporally extended options that generate primitive actions until a termination condition is met. The primary existing approach applies per-decision importance sampling at the option level, but requires option-annotated trajectories and exhibits variance that still grows with the horizon. For non-hierarchical policies, fitted Q-evaluation (FQE) typically achieves lower error than importance sampling by leveraging the classic Bellman equation for policy evaluation; however, a naive extension of FQE to hierarchical policies results in a biased policy value estimate. In this paper, we introduce Intra-Option Fitted Q-Evaluation (IO-FQE), which estimates a value function in the augmented state-option space; this approach enables OPE of hierarchical policies even when we lack annotations of what option was run in the data. We develop two continuous-action instantiations of IO-FQE and show on long-horizon hierarchical continuous control tasks (AntMaze navigation and OGBench Puzzle manipulation) that IO-FQE eliminates the bias incurred by a naive application of FQE to hierarchical policies and substantially lowers mean squared-error compared to option-level importance sampling.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2608.17253

Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL

Yunhao Yang, Yuexin Bian, Yunjie Tian, Di Fu, Tianjin Huang, Yuanyuan Shi, Ziang Xiao, Nuno Vasconcelos 等 9 位作者

原文摘要

Reinforcement Learning (RL) has emerged as a powerful approach for improving reasoning in language and vision-language models, yet its strongest successes still depend heavily on ground-truth supervision (e.g., verifiable reward). Such annotations are costly to obtain and become increasingly scarce as reasoning capabilities advance beyond what humans can reliably evaluate. Self-rewarding RL reduces this dependence by enabling models to derive reward signals from their own completions. However, training solely on self-generated feedback can reinforce existing biases, reduce response diversity, and ultimately lead to homogenized responses and training collapse. In this work, we show that unsupervised reasoning can emerge through cooperative multi-agent training. We introduce Co-RL, a framework in which multiple decoupled models, sharing no parameters, are simultaneously optimized through RL using rewards derived from the other agent. We further show that increasing cohort diversity, through heterogeneous model families, sizes, and rephrased training samples, reduces the correlated errors that drive self-reinforcing feedback loops. This diversity consistently improves reasoning performance and mitigates training collapse. Without any ground-truth labels, Co-RL improves the base models by 3.0-7.9% on average across seven benchmarks and by 1.6-4.4% across four multimodal benchmarks. It also outperforms the prior state-of-the-art multi-agent RL method CoMAS by 4.0% and the self-rewarding vision-language RL method MM-UPT by 2.0%. Our code is available at https://github.com/Diverse-CoRL/Co-RL and website at https://williamium3000.github.io/Co-RL.

cs.LG · cs.AI · cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.25143

Stable Unsupervised Continual Chunking with Sheaf SyncMap

Xueyuan Li, Danilo Vasconcellos Vargas

原文摘要

Unsupervised Continual chunking is a fundamental problem in machine learning and neuroscience, where the goal is to identify groups of states that frequently co-occur in temporal sequences. A key challenge is to form accurate chunks while maintaining their stability over time. In this work, we propose sheaf regularization to reduce local inconsistencies in Decentralized SyncMap, a self-organizing system, and thereby stabilize its chunking dynamics. We introduce a radial sheaf structure that penalizes distance-dependent radial motion between pairs of variables. Experimental results show that the proposed method achieves the highest normalized mutual information (NMI) among the evaluated SyncMap variants on 12 of 18 probabilistic Continual General Chunking Problem (CGCP) graphs with two-state memory and on 17 of 18 graphs with dynamic memory. In the sequential adaptation experiment, Sheaf SyncMap also achieves high NMI after shifts in the input distribution, indicating that it can adapt to new knowledge while avoiding the negative transfer commonly observed in modern machine learning systems such as neural networks.

cs.LG · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2412.00101

Multi-Label Contrastive Learning : A Comprehensive Study

Alexandre Audibert, Aurélien Gauffre, Massih-Reza Amini

原文摘要

Multi-label classification, which involves assigning multiple labels to a single input, has emerged as a key area in both research and industry due to its wide-ranging applications. Designing effective loss functions is crucial for optimizing deep neural networks for this task, as they significantly influence model performance and efficiency. Traditional loss functions, which often maximize likelihood under the assumption of label independence, may struggle to capture complex label relationships. Recent research has turned to supervised contrastive learning, a method that aims to create a structured representation space by bringing similar instances closer together and pushing dissimilar ones apart. Although contrastive learning offers a promising approach, applying it to multi-label classification presents unique challenges, particularly in managing label interactions and data structure. In this paper, we conduct an in-depth study of contrastive learning loss for multi-label classification across diverse settings. These include datasets with both small and large numbers of labels, datasets with varying amounts of training data, and applications in both computer vision and natural language processing. Our empirical results indicate that the promising outcomes of contrastive learning are attributable not only to the consideration of label interactions but also to the robust optimization scheme of the contrastive loss. Furthermore, while the supervised contrastive loss function faces challenges with datasets containing a small number of labels and ranking-based metrics, it demonstrates excellent performance, particularly in terms of Macro-F1, on datasets with a large number of labels.

cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10538

Close and Sort: Exact Maximum-Likelihood Aggregation of Cyclic Rank-Two Preferences

Kehan Liu, Zhaoyun Chen, Xianquan Yan

原文摘要

Pairwise probabilities arise in rank aggregation, recommendation, and human or LLM evaluation, yet converting them into a maximum-likelihood list is a weighted Kemeny problem and is NP-hard in general. Known exact algorithms for rank-two tournaments use only comparison signs and do not optimize this likelihood. We show that the weighted problem is exactly solvable when the skew-symmetric log-odds matrix $W_{ij}=\log(p_{ij}/p_{ji})$ has rank two, including when the induced preferences contain cycles. Writing $W_{ij}=\det(h_i,h_j)$ and adding the closing vector $h_0=-\sum_i h_i$ turns the objective into twice the signed area of a closed polygon. This geometry yields CloseSort, which polar-sorts the $n+1$ edge vectors, cuts after $h_0$, and returns a global maximum-likelihood order in $O(n\log n)$ time from the factors. The same geometry identifies the runner-up and gives a stability certificate beyond exact rank two. When the smallest determinant $δ$ of two consecutive edges in the closed angular cycle is positive, it is exactly the best-to-runner-up log-likelihood gap. For $W=W_0+E$, the CloseSort order of $W_0$ has regret at most $\sum_{i<j}|E_{ij}|$ on $W$ and remains optimal whenever this sum is below $δ$, a worst-case tight threshold. In 5,120 determinant-logit simulations, increasing the observations per pair from 4 to 64 raises the fraction on which rank-two projection outperforms the empirical plug-in optimum from 68.6% to 85.7%. On 3,600 Arena-Hard LLM-judge profiles drawn from 63 systems and ten judges, the leading rank-two component captures 89.2% of the log-odds energy. On profiles containing a directed majority triangle, CloseSort reduces normalized regret by 44% relative to Borda and improves the mean normalized objective over each of eight baselines. With factors supplied, it is 8,407 times faster than exact dynamic programming on 18-item rankings.

cs.DS · cs.IR · cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.10536

In Search of Lost Time: Temporal Competence in Large Language Models

Ryan Liu, David Lee, Icey Siyi Ai, Marwa Abdulhai, Rachit Dubey, Thomas L. Griffiths

原文摘要

Today's large language models (LLMs) increasingly interact with time: AI agents schedule tasks, coordinate with people and other AI systems, and even decide how long to think. Yet LLMs only ever encounter time secondhand---in descriptions rather than experience. Does this hinder their reasoning and decisions that involve time? Our paper traces this question from a model's knowledge to its actions through four successive evaluations. We find that models know the rules of time, which can be read, but struggle to predict durations, which must be lived; while they are highly accurate (92.0-97.5\%) on a collection of questions built from the theorems of various time-related literatures, they underestimate their own completion times by up to 99.5\% and fail to identify which tasks will take them longer to run. And while they are better at estimating durations of human tasks, their duration judgments change based on how an event is worded, magnifying similar human biases by up to 1.3$\times$. Moving from estimation to action, we find that some LLMs disregard time costs completely when answering user queries, ignoring a 1,200-fold increase in wait time entirely. Together, our experiments reveal new challenges posed to LLMs as they navigate a dimension they cannot directly perceive, and highlight how their blind spots can even be simple cognitive abilities that humans take for granted.

cs.AI · cs.CL · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2608.21810

MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents

Md Asaduzzaman Jabin, Khoa Le, Lin Zhao, Tianming Liu

原文摘要

Clinical decision-making is inherently experience-driven: physicians progressively refine their reasoning by synthesizing patient history, multimodal observations, and prior diagnostic experiences across interactions. In contrast, current multimodal large language model (MLLM)-based medical AI agents largely operate as stateless inference systems, generating decisions independently for each interaction without retaining or internalizing experiential knowledge. This discrepancy limits their ability to progressively improve reasoning reliability through usage and adapt to longitudinal patient contexts in real-world clinical workflows. In this study, we propose Medical Structured Multimodal Memory (MSM-Mem), an agentic memory framework that enables medical AI agents to evolve through accumulated clinical experiences. MSM-Mem organizes heterogeneous clinical experiences into semantic, episodic, and visual memory and incrementally updates them during inference, allowing the agent to retrieve prior experiences to inform current reasoning and progressively refine decision-making over time. Evaluations on MoE-LLaVA backbones demonstrate consistent performance improve- ments with further gains observed through continued usage. In general, MSM-Mem offers a viable pathway toward medical AI agents capable of evolving their reasoning competence in a manner analogous to the way clinicians learn from practice over time.

cs.LG · cs.CV

查看论文
arXiv ↗PDF ↗