CodeQuant2：面向低精度大型推理模型的增强型离群值缓解与自适应聚类
CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs
Xingyu Liu, Xiangyang Yin, Tianhua Xia, Haiyu Wang, Sai Qian Zhang
AI 速读 基于摘要
CodeQuant2 是一种基于聚类的查找表（LUT）量化框架，通过旋转与缩放缓解激活离群值、自适应码本大小并改进校准样本选择，以提升低精度大型推理模型的推理精度与速度。
cs.CL
查看原文摘要
Large reasoning models (LRMs) improve task performance by generating longer intermediate reasoning traces, but this substantially increases inference costs and makes low-precision quantization increasingly important. However, aggressively quantized models often perform poorly on reasoning tasks. We observe larger activation fluctuations during the first decoding steps of reasoning. Calibration that omits these early states may miss their activation fluctuations. To address these reasoning-specific quantization challenges, we propose \textbf{CodeQuant2}, a clustering-based LUT quantization framework for low-precision large reasoning models. CodeQuant2 mitigates activation outliers with rotation and scaling, uses clustering to represent non-uniform weight distributions, and adapts codebook sizes to the quantization difficulty of each layer. Guided by our observation of reasoning-time activation behavior, CodeQuant2 further improves calibration sample selection to better match actual decoding statistics. Combined with our customized LUT CUDA kernel, CodeQuant2 delivers substantial accuracy improvements on reasoning benchmarks while achieving up to $2.85\times$ speedup over BF16.