Model·Foundations

Topic

inference

2 notes · all notes

Jul 05, 2026 quantization / inference / llm

GPTQ: One-Shot 3–4 Bit Quantization as Approximate Second-Order Optimization

Elias Frantar et al., 2022 — ICLR 2023

GPTQ quantizes a 175B model to 3–4 bits in a few GPU-hours by turning layer-wise quantization into a sequence of cheap Hessian-guided weight updates — and by noticing that the expensive part of the classic algorithm was never necessary.

repro: reproduced
Jun 28, 2026 attention / inference / systems

FlashAttention-2: Where the Second 2× Actually Comes From

Tri Dao, 2023 — arXiv 2307.08691

FlashAttention-1 made attention IO-aware; FlashAttention-2 gets another ~2× by fixing how the work is divided — fewer non-matmul FLOPs, parallelism over sequence length, and warp-level partitioning that stays out of shared memory.

repro: reproduced