跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2610.28537 · v2

Beyond 4 bits: Data-Centric Post-training Quantization and Recovery

首次提交 2026-10-21 01:10
版本更新 2026-11-17 01:16
雷达观测 2026-11-17 16:10
arXiv ↗PDF ↗

摘要

Post-training quantization (PTQ) compresses the weights of a trained model with a limited compute and data budget. Down to 4 bits, state-of-the-art PTQ methods are near-lossless; below this limit, the tested scalar PTQ configurations collapse to near-random accuracy and quality becomes acutely sensitive to choices such as the quantization representation and calibration data. At such low bitwidths, recovery fine-tuning becomes crucial, and its effectiveness depends heavily on these same choices. Yet there is little systematic evidence on how the quantization method, the quality and task alignment of calibration data, and the design of the recovery procedure interact to determine the final 2-bit model. We present a systematic study of 2-bit quantization and recovery fine-tuning across model families, scales, and data distributions, and distill our findings into practical, architecture-dependent best practices for obtaining an accurate 2-bit model. This recipe, which combines a vector codebook representation, a small task-aligned calibration mixture, and a short codebook-only distillation stage, restores 63-68% average accuracy across seven question-answering tasks at 2 bits (a 71-73% BF16 ceiling), versus the near-random 29% of the evaluated scalar PTQ pipelines, at roughly half the compute of a full-model QAT baseline. Given the same calibration and recovery data, EfficientQAT, a trained scalar baseline, avoids the collapse. However, at a matched rate it trails the vector recipe on generation on every model (2.8-10.9 AvgGen; within noise on Qwen3-MoE) and on question answering on three of the four studied models (2.7-7.8 AvgQA). We also find that the best recovery data depends on the calibration data, an interaction confirmed on a held-out model. Our architecture-aware map of the 2-bit design space shows that representation, calibration data, and recovery data must be chosen jointly.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-17 01:16 · 最新观测版本

Beyond 4 bits: Data-Centric Post-training Quantization and Recovery

Post-training quantization (PTQ) compresses the weights of a trained model with a limited compute and data budget. Down to 4 bits, state-of-the-art PTQ methods are near-lossless; below this limit, the tested scalar PTQ configurations collapse to near-random accuracy and quality becomes acutely sensitive to choices such as the quantization representation and calibration data. At such low bitwidths, recovery fine-tuning becomes crucial, and its effectiveness depends heavily on these same choices. Yet there is little systematic evidence on how the quantization method, the quality and task alignment of calibration data, and the design of the recovery procedure interact to determine the final 2-bit model. We present a systematic study of 2-bit quantization and recovery fine-tuning across model families, scales, and data distributions, and distill our findings into practical, architecture-dependent best practices for obtaining an accurate 2-bit model. This recipe, which combines a vector codebook representation, a small task-aligned calibration mixture, and a short codebook-only distillation stage, restores 63-68% average accuracy across seven question-answering tasks at 2 bits (a 71-73% BF16 ceiling), versus the near-random 29% of the evaluated scalar PTQ pipelines, at roughly half the compute of a full-model QAT baseline. Given the same calibration and recovery data, EfficientQAT, a trained scalar baseline, avoids the collapse. However, at a matched rate it trails the vector recipe on generation on every model (2.8-10.9 AvgGen; within noise on Qwen3-MoE) and on question answering on three of the four studied models (2.7-7.8 AvgQA). We also find that the best recovery data depends on the calibration data, an interaction confirmed on a held-out model. Our architecture-aware map of the 2-bit design space shows that representation, calibration data, and recovery data must be chosen jointly.

查看此版本解读 →查看此版本原文 ↗