GPTQ-2D：三次时间复杂度的双边自适应舍入
GPTQ-2D: Cubic-Time Two-Sided Adaptive Rounding
Jiale Chen, Torsten Hoefler, Dan Alistarh
AI 速读 基于摘要
作者提出 GPTQ-2D，在保持与 YAQA 相同双边自适应舍入结果的前提下，把误差反馈的计算量从四次降为三次，并给出 GPU 实现与多 Kronecker 因子扩展。
cs.DS · cs.LG
查看原文摘要
Adaptive rounding methods such as GPTQ quantize the weights of a large language model (LLM) one at a time, propagating each rounding error to the weights not yet rounded. YAQA lowers the KL divergence from the original model with two-sided adaptive rounding under a Kronecker-factored Hessian, but its error feedback takes quartic work in the size of a square layer. We present GPTQ-2D, which provably computes the same rounding with cubic work, while still rounding the entries of each anti-diagonal in parallel. Its GPU implementation uses two levels of blocking to improve hardware utilization. We also extend the algorithm to Hessians with any number of Kronecker factors. In our experiments, GPTQ-2D is as accurate as YAQA and cuts the time to quantize Llama-3.1-70B-Instruct on a single NVIDIA RTX A6000 GPU from 8.6 days to 1.6 hours. Source code is available at https://github.com/IST-DASLab/GPTQ-2D.