GPTQ: One-Shot 3–4 Bit Quantization as Approximate Second-Order Optimization
Elias Frantar et al., 2022 — ICLR 2023
GPTQ quantizes a 175B model to 3–4 bits in a few GPU-hours by turning layer-wise quantization into a sequence of cheap Hessian-guided weight updates — and by noticing that the expensive part of the classic algorithm was never necessary.
repro: reproduced