跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1200新增论文
+951其中 AI 相关
+132已有论文修订
+97新增 AI 导读

arXiv 最近完成扫描:2026-11-20 08:48 · 全站最近入库:2026-11-20 16:53各信源进度 →

统计口径与数据新鲜度

统计 2026-11-19 17:15 — 2026-11-20 17:15(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-20 08:48最近发现新增版本:2026-11-20 16:53库内论文最新更新时间:2026-11-20 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

28,609 已收录论文23,738 AI 相关8,785 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 3 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-20 08:46(北京时间)
版本更新 · v3arXiv:2609.27964

Learning from Transients: Linear RNN Scaling Laws in Stationary Prediction

Ziyan Chen, Linhe Zheng, Peilin Liu, Zhongzhu Zhou, Ding-Xuan Zhou

原文摘要

Does the organization of a fixed number of transitions into independent trajectories affect a fixed stationary prediction task? We study a Gaussian-sketched linear recurrent teacher--student model, where independently initialized training trajectories create transients that can excite weak spectral directions. For full-batch gradient descent with stable deterministic learning rates, we establish matching stationary approximation, bias, variance, and total-risk scaling laws from dependent trajectory data. When initialization decays at least as fast spectrally as the innovations, trajectory count and length enter the leading variance rate through their product. With a heavier initialization tail, transient excitation can improve the trajectory-length-dependent finite-time bias and, at fixed transition count, variance, while the stationary oracle shift remains controlled by representation approximation. Under token and compute caps, constant-length trajectories attain the optimal risk scaling order within the theorem-controlled range. Synthetic experiments provide finite-scale evidence in this stylized model.

cs.LG

版本更新 · v2arXiv:2605.17757

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization

Zhongzhu Zhou, Donglin Zhuang, Jisen Li, Ziyan Chen, Shuaiwen Leon Song, Ben Athiwaratkun, Xiaoxia Wu

原文摘要

INT2 KV-cache quantization is attractive for long-context LLM serving, yet making it both accurate and practical remains challenging. Data-oblivious rotations such as Hadamard transforms flatten channel outliers, but still degrade sharply at INT2 because they ignore how downstream attention queries the cache. We propose OSCAR, an ultra-low-bit KV-cache quantization method that estimates attention-aware covariance targets offline and uses them to derive fixed per-layer rotations and clipping thresholds. By aligning the quantization basis with the second-moment structures consumed by attention, OSCAR preserves attention fidelity at 2-bit precision. Beyond theoretical analysis under a frozen-error surrogate, we build a serving-ready OSCAR system with fused Triton write and decode kernels that preserve paged KV-cache layouts and prefix caching, integrating into modern serving engines such as SGLang and vLLM. We evaluate OSCAR on reasoning and coding benchmarks with generation traces of up to 32K tokens across five tasks. On Qwen3-4B-Thinking-2507 and Qwen3-8B, OSCAR narrows the BF16 accuracy gap to 3.78 and 1.21 points, respectively, where Hadamard-only INT2 collapses, and matches BF16 within run-to-run variance on Qwen3-32B and GLM-4.7 (358B parameters). On RULER-NIAH up to 128K tokens, OSCAR substantially outperforms QuaRot-style INT2 across context lengths. System-wise, OSCAR compresses KV-cache memory by nearly $8\times$, boosts multi-request throughput by up to $7.8\times$ at large batch sizes within the same memory budget, and accelerates batch-size-1 decoding by up to $3.1\times$ over BF16 by cutting KV memory traffic.

cs.LG · cs.AI · cs.DC · cs.PF

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.339ACL Anthology · ACL 2026

Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning

Qisheng Su, Shiting Huang, Zhen Fang University of Science and Technology of China, Ziyan Chen, Zehui Chen, Feng Zhao University of Science and Technology of China

原文摘要

In real-world Tool-Integrated Reasoning (TIR) scenarios, a major source of inefficiency is that the toolcalls create pauses between LLM requests and cause KV-cache eviction. Also, the long, unfiltered response returned by external tools inflates the KV-cache, so each decode step spends more time loading the growing cache and thus becomes steadily slower as context length increases. However, existing efficiency metrics like token counts and toolcall counts fail to capture this real computational cost. To address this, we introduce PTE (Prefill Token Equivalents), a hardware-aware TIR-efficiency metric that unifies internal reasoning and external tool-use costs while explicitly accounting for non-reusable KV-Cache and long-tool-response scenarios, thus better reflects real-world scenarios. We conduct extensive experiments across five TIR benchmarks, quantify their PTE costs, and identify four inefficiency patterns that appear in TIR. In a simulated high-concurrency industrial setting, PTE explains wall-clock latency significantly better than token-count metric. We also discover that trajectories with higher PTE costs tend to have lower reasoning correctness, indicating that simply using more tools does not improve the quality of the answer. PTE offers a new perspective on the efficiency of Tool-Integrated Reasoning. The code is available.

cs.CL

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表