跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1新增论文
+1其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-13 07:07 · 全站最近入库：2026-11-13 06:39各信源进度 →

统计口径与数据新鲜度

统计 2026-11-12 08:37 — 2026-11-13 08:37（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-13 07:07最近发现新增版本：2026-11-13 06:39库内论文最新更新时间：2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

21,676 已收录论文18,324 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 4 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-13 07:05（北京时间）
版本更新 · v2arXiv:2608.08809

Tevatron-Elastic: A Unified Abstraction for Training Elastic Retrievers and Rerankers

Yu Wang, Shengyao Zhuang, Xueguang Ma, Zongyu Wu, Jimmy Lin, Vivek Srikumar, Zhichao Xu

原文摘要

A single model scale challenges the flexibility of a production retrieval system: some settings need it faster, others need a smaller index, and the right trade-off changes with the workload. In the context of information retrieval (IR), a transformer-based model can be made smaller in three ways --- using fewer layers, passing fewer tokens through the upper layers, or producing a shorter embedding --- and each way saves a different resource. These options have been studied in isolation, each as its own method with its own code and training setup, which makes them hard to combine or adapt to a new model. In this work, we present Tevatron-Elastic to bring all three under one unified abstraction: a single object names any size the model can run at, and a short schedule lists the sizes to train. Training then produces one checkpoint that serves all of those sizes, and at deployment the user has the liberty to pick any of them. The same abstraction covers both retrievers and rerankers and both encoder and decoder models, as it works through interfaces that Hugging Face transformers already expose; adding a new backbone can be a configuration change rather than a model rewrite. Prior methods --- Matryoshka embeddings, early exit, 2D Matryoshka (Starbucks), and layerwise token compression (LTC) --- become special cases of our unified abstraction. With Tevatron-Elastic, we also propose a new elastic algorithm Matryoshka~LTC (MLTC) to jointly train several token-compression ratios in one retriever checkpoint. To validate our framework, we train 20 checkpoints across three backbones and two tasks: the quality curves are smooth, full-point quality remains close to that of models trained for a single size, and a controlled study confirms the wall-clock speedups. We release the framework and all checkpoints to support elastic retrieval research.

cs.CL

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2608.00916

Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget

Zhichao Xu, Xueguang Ma, Shengyao Zhuang, Luyu Gao, Wenqian Ye, Yu Wang, Jamie Callan, Jimmy Lin

原文摘要

Training large dense and mixture-of-experts (MoE) rerankers is constrained by GPU memory and distributed-execution efficiency, which has kept academic reranker research largely on dense models. Tevatron trains rerankers through the Hugging Face Trainer with DeepSpeed and PyTorch FSDP1, a stack that stalls on three fronts: DeepSpeed ZeRO-2 has a known gradient-gathering failure on this path, FSDP1 full sharding is memory-safe but throughput-bound, and neither offers the expert parallelism (EP) that MoE training needs. We present Tevatron 3.0, which integrates Megatron-Core as a reranker training backend while preserving Tevatron's data processing, evaluation pipeline, and Hugging Face checkpoint format. For full-parameter dense 8B training, Megatron matches FSDP1 on BEIR-15 NDCG@10 with approximately 4% lower epoch time and twice the peak GPU memory under matched data-parallel and recomputation settings, and approximately 22% lower epoch time in its recommended configuration; it supports both LoRA and full-parameter fine-tuning. Expert parallelism makes it possible to train a 30B-parameter MoE reranker (Qwen3-30B-A3B) on two nodes. A controlled study of architecture, LoRA, and distillation on BEIR-15, with three first-stage retrievers and Hugging Face and vLLM serving, shows that the MoE reranker matches dense-8B quality at less than half the activated parameters and at higher inference throughput. We open-source the framework and trained checkpoints.

cs.IR · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.16656

Defragmenting Language Models: An Interpretability-based Approach for Vocabulary Expansion

Maitrey Mehta, Nishant Subramani, Zhichao Xu, Ashim Gupta, Vivek Srikumar

原文摘要

All languages are equal; when it comes to tokenization, some are more equal than others. Tokens are the hidden currency that dictates the cost and latency of contemporary LLM access. However, many languages written in non-Latin scripts observe a poor exchange rate: LLMs take several multiples of tokens to encode the same information in many languages as they do for English. We observe that this issue, known as token overfragmentation, persists in modern open-weight LLMs. The standard remedy is vocabulary expansion that adds target language items missing from the model's vocabulary. We focus on two core decisions in the vocabulary expansion process: What items should we add? and How should we initialize their corresponding input and output embeddings? We comprehensively study and advance the new research direction of interpretability-based vocabulary expansion along these two core decisions, making four key contributions: First, we question the conventional use of frequency-based methods to choose candidate vocabulary items to add (a decision long treated as settled), and show that interpretability-based methods offer a superior performance-token efficiency tradeoff. Second, we strengthen the case for interpretability-based embedding initialization by showing large gains (~20 pts) over baseline initialization methods for several languages written in non-Latin scripts. Third, we identify the phenomenon of "subword detokenization," where models progressively merge fragmented subword tokens into larger subwords across layers. Fourth, we propose FragMend, a method that leverages this phenomenon to push the efficiency ceiling of interpretability-based expansion, and validate its effectiveness.

cs.CL

查看论文
arXiv ↗PDF ↗
首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.69ACL Anthology · ACL 2026

Reinforcement Learning for Self-Improving Agent with Skill Library

Jiongxiao Wang University of Wisconsin - Madison, Qiaojing Yan Amazon, Yawei Wang Amazon, Yijun Tian, Soumya Smruti Mishra Amazon, Zhichao Xu Amazon, Megha Gandhi Amazon, Panpan Xu Amazon 等 9 位作者

原文摘要

Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in complex reasoning and multi-turn interactions but struggle to continuously improve and adapt when deployed in new environments. One promising approach is implementing skill libraries that allow agents to learn, validate, and apply new skills. However, current skill library approaches rely primarily on LLM prompting, making consistent skill library implementation challenging. To overcome these challenges, we propose a Reinforcement Learning (RL)-based approach to enhance agents’ self-improvement capabilities with a skill library. Specifically, we introduce Skill Augmented GRPO for self-Evolution (SAGE), a novel RL framework that systematically incorporates skills into learning. The framework’s key component, Sequential Rollout, iteratively deploys agents across a chain of similar tasks for each rollout. As agents navigate through the task chain, skills generated from previous tasks accumulate in the library and become available for subsequent tasks. Additionally, the framework enhances skill generation and utilization through a Skill-integrated Reward that complements the original outcome-based rewards. Experimental results on AppWorld demonstrate that SAGE, when applied to supervised-finetuned model with expert experience, achieves 8.9% higher Scenario Goal Completion while requiring 26% fewer interaction steps and generating 59% fewer tokens, substantially outperforming existing approaches in both accuracy and efficiency. Our code is available at https://github.com/amazon-science/SAGE.

cs.CL

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表