GPTQ Post-Training Quantization (Frantar 2022年)（ジーピーティーキュー）
2022年Frantar et al.発表GPTQ・Industry-leading post-training quantization LLM + Industry-leading 4-bit INT4 quantization + Industry-leading second-order Hessian information + Industry-leading OPT/BLOOM/LLaMA 175B production quantization。
概要
GPTQ Post-Training Quantization は、2022年Frantar et al.発表GPTQ・Industry-leading post-training quantization LLM 2022年 + Industry-leading OPT/BLOOM/LLaMA 175B production quantization position確立。GPTQ specifications = Industry-leading post-training quantization LLM (Industry-leading GPTQ 2022 + Industry-leading post-training quantization LLM + Industry-leading PTQ flagship) + Industry-leading 4-bit INT4 quantization (Industry-leading 4-bit INT4 weight-only quantization + Industry-leading 4×memory reduction FP16→INT4) + Industry-leading second-order Hessian information (Industry-leading second-order Hessian quantization OBQ derivative + Industry-leading layer-wise calibration data quantization) + Industry-leading OPT/BLOOM/LLaMA 175B production quantization (Industry-leading 175B production quantization)。
主な特徴・仕組み
- Authors: Industry-leading Elias Frantar + Saleh Ashkboos + Torsten Hoefler + Dan Alistarh ETH Zurich + ISTA + Neural Magic
- Year: 2022年 (arXiv 2022年10月発表)
- Paper: Industry-leading "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers" arXiv 2210.17323
- Quantization Type: Industry-leading post-training quantization (PTQ) weight-only
- Bits: Industry-leading 4-bit INT4 standard (3-bit/2-bit also)
- Method: Industry-leading second-order Hessian information OBQ derivative + Industry-leading layer-wise calibration
- Calibration Data: Industry-leading 128-512 samples small calibration set
- Memory Reduction: Industry-leading 4×memory reduction FP16→INT4
- Speed: Industry-leading 3-4×inference speedup INT4 vs FP16
- Accuracy: Industry-leading <1% accuracy drop OPT-175B/BLOOM-176B
- Production Adoption: Industry-leading AutoGPTQ + ExLlamaV2 + vLLM + TGI + HuggingFace Transformers
- Industry-Leading: Industry-leading 4-bit PTQ LLM 175B production standard
スペック比較表
| LLM Quantization Method | Year | Bits | Method | Industry Position |
|---|---|---|---|---|
| GPTQ | 2022 | 4/3/2-bit | Second-order Hessian PTQ weight-only | Industry-leading 4-bit PTQ 175B |
| AWQ | 2023 | 4-bit | Activation-aware weight quantization | Industry-leading activation-aware quantization |
| SmoothQuant | 2022 | INT8 | Smoothing activation+weight quantization | Industry-leading W8A8 production |
| QuIP | 2024 | 2-bit | Incoherence processing + lattice quantization | Industry-leading 2-bit extreme quantization |
| AQLM | 2024 | 2-bit | Additive quantization with learned codebooks | Industry-leading 2-bit additive quantization |
具体例・対応製品
- GPTQ (2022年, Frantar et al.): Industry-leading post-training quantization LLM
- Industry-leading 4-bit INT4 quantization: Industry-leading 4-bit INT4 weight-only
- Industry-leading second-order Hessian information: Industry-leading second-order Hessian OBQ derivative
- Industry-leading 4×memory reduction: Industry-leading 4×memory reduction FP16→INT4
- Industry-leading AutoGPTQ + ExLlamaV2 + vLLM ecosystem: Industry-leading production ecosystem
- 競合 AWQ + SmoothQuant + QuIP + AQLM: Industry-leading quantization competitors
自作PCでの選び方・注意点
GPTQ Post-Training Quantization は「Industry-leading post-training quantization LLM + Industry-leading 4-bit INT4 quantization」「Industry-leading second-order Hessian information + Industry-leading OPT/BLOOM/LLaMA 175B production quantization」用途のIndustry-leading post-training quantization 2022年Frantar et al.発表product。Industry-leading 4-bit INT4 (Industry-leading 4-bit INT4 weight-only quantization + Industry-leading 4×memory reduction FP16→INT4 + Industry-leading 3-4×inference speedup INT4 vs FP16) で Industry-leading 4-bit INT4 + Industry-leading 4×memory reduction + Industry-leading 3-4×inference speedup。Industry-leading second-order Hessian (Industry-leading second-order Hessian quantization OBQ derivative + Industry-leading layer-wise calibration data + Industry-leading 128-512 samples calibration set + Industry-leading <1% accuracy drop OPT-175B/BLOOM-176B) で Industry-leading second-order Hessian + Industry-leading layer-wise calibration + Industry-leading 128-512 calibration + Industry-leading <1% accuracy drop。Industry-leading 175B production quantization (Industry-leading OPT-175B/BLOOM-176B/LLaMA 175B production quantization + Industry-leading PTQ flagship + Industry-leading 4-bit PTQ LLM 175B production standard) で Industry-leading 175B production + Industry-leading PTQ flagship + Industry-leading 4-bit standard。Industry-leading AutoGPTQ ecosystem (Industry-leading AutoGPTQ + ExLlamaV2 + vLLM + TGI + HuggingFace Transformers + Industry-leading production ecosystem adoption) で Industry-leading AutoGPTQ + ExLlamaV2 + vLLM + TGI + HuggingFace。但しIndustry-leading AWQ + SmoothQuant + QuIP + AQLM competition (Industry-leading AWQ activation-aware 4-bit 2023 + SmoothQuant W8A8 INT8 + QuIP 2-bit lattice 2024 + AQLM 2-bit additive 2024 vs GPTQ 4-bit Hessian PTQ 2022 trade-off) で Industry-leading 4-quantization method competitors vs GPTQ + Industry-leading 4-bit INT4 + second-order Hessian + 175B production + AutoGPTQ ecosystem first-mover advantage adoption alignment必須。
関連用語との違い
- vs AWQ (Activation-Aware 2023): GPTQはHessian + 4-bit PTQ・AWQはactivation-aware + salient channel保護 + 4-bit
- vs SmoothQuant (W8A8 2022): GPTQは4-bit weight-only・SmoothQuantはINT8 weight+activation + smoothing
- vs QuIP/AQLM (2-bit 2024): GPTQは4-bit standard PTQ・QuIP/AQLMは2-bit extreme quantization
よくある質問（FAQ）
Q1: GPTQ vs AWQ 違いは？ A: GPTQ (Industry-leading 4-bit INT4 + second-order Hessian + OBQ derivative + 128-512 calibration + OPT-175B/BLOOM-176B + 4×memory reduction + AutoGPTQ ecosystem) vs AWQ (Industry-leading 4-bit activation-aware weight quantization + salient channel protection + group-wise quantization + per-channel scaling + LLaMA/Mistral)・Industry-leading 4-bit + Hessian + 175B + AutoGPTQ = GPTQ + Industry-leading 4-bit + activation-aware + salient channel + LLaMA = AWQ preference judgment。
Q2: Industry-leading 4-bit INT4 quantization + 4×memory reduction value は？ A: Industry-leading 4-bit INT4 (Industry-leading 4-bit INT4 weight-only quantization + Industry-leading 4×memory reduction FP16→INT4 + Industry-leading 3-4×inference speedup INT4 vs FP16 + Industry-leading <1% accuracy drop)。
Q3: Industry-leading second-order Hessian information + 175B production value は？ A: Industry-leading second-order Hessian (Industry-leading second-order Hessian quantization OBQ derivative + Industry-leading layer-wise calibration data + Industry-leading 128-512 samples calibration + Industry-leading OPT-175B/BLOOM-176B/LLaMA 175B production quantization)。
まとめ
GPTQ = 2022年Frantar et al.発表のpost-training quantization LLM。Industry-leading 4-bit INT4 weight-only quantization + Industry-leading second-order Hessian information OBQ derivative + Industry-leading layer-wise calibration + Industry-leading 128-512 samples calibration set + Industry-leading 4×memory reduction FP16→INT4 + Industry-leading 3-4×inference speedup + Industry-leading <1% accuracy drop OPT-175B/BLOOM-176B + Industry-leading AutoGPTQ + ExLlamaV2 + vLLM + TGI + HuggingFace Transformers production ecosystem + Industry-leading 4-bit PTQ LLM 175B production standard position確立。