SmoothQuant W8A8 Quantization (Xiao 2022年)（スムースクォント）
2022年Xiao et al. (MIT)発表SmoothQuant・Industry-leading W8A8 INT8 quantization LLM + Industry-leading smoothing activation+weight + Industry-leading OPT-175B/BLOOM-176B W8A8 + Industry-leading TensorRT-LLM production W8A8。
概要
SmoothQuant W8A8 Quantization は、2022年Xiao et al. (MIT)発表SmoothQuant・Industry-leading W8A8 INT8 quantization LLM 2022年 + Industry-leading TensorRT-LLM production W8A8 position確立。SmoothQuant specifications = Industry-leading W8A8 INT8 quantization LLM (Industry-leading SmoothQuant 2022 + Industry-leading W8A8 INT8 quantization LLM + Industry-leading SmoothQuant flagship) + Industry-leading smoothing activation+weight (Industry-leading mathematically equivalent smoothing + Industry-leading activation outlier migration to weight) + Industry-leading OPT-175B/BLOOM-176B W8A8 (Industry-leading 175B W8A8 production) + Industry-leading TensorRT-LLM production W8A8 (Industry-leading TensorRT-LLM production W8A8)。
主な特徴・仕組み
- Authors: Industry-leading Guangxuan Xiao + Ji Lin + Mickael Seznec + Hao Wu + Julien Demouth + Song Han MIT + NVIDIA
- Year: 2022年 (arXiv 2022年11月発表)
- Paper: Industry-leading "SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models" arXiv 2211.10438
- Quantization Type: Industry-leading W8A8 INT8 weight + activation quantization
- Bits: Industry-leading W8A8 (8-bit weight + 8-bit activation)
- Method: Industry-leading mathematically equivalent smoothing + Industry-leading activation outlier migration to weight + Industry-leading per-channel scaling factor s
- Key Insight: Industry-leading "activations are harder to quantize than weights due to outliers"
- Speed: Industry-leading 1.56×inference speedup vs FP16
- Memory Reduction: Industry-leading 2×memory reduction FP16→INT8
- Accuracy: Industry-leading <0.5% accuracy drop OPT-175B/BLOOM-176B
- Production Adoption: Industry-leading NVIDIA TensorRT-LLM + FasterTransformer + LMDeploy + vLLM
- Industry-Leading: Industry-leading W8A8 INT8 quantization LLM production standard
スペック比較表
| LLM Quantization Method | Year | Bits | Method | Industry Position |
|---|---|---|---|---|
| SmoothQuant | 2022 | W8A8 INT8 | Smoothing activation+weight quantization | Industry-leading W8A8 production |
| GPTQ | 2022 | 4/3/2-bit | Second-order Hessian PTQ weight-only | Industry-leading 4-bit PTQ 175B |
| AWQ | 2023 | 4-bit | Activation-aware weight quantization | Industry-leading activation-aware quantization |
| QuIP | 2024 | 2-bit | Incoherence processing + lattice quantization | Industry-leading 2-bit extreme quantization |
| AQLM | 2024 | 2-bit | Additive quantization with learned codebooks | Industry-leading 2-bit additive quantization |
具体例・対応製品
- SmoothQuant (2022年, Xiao et al. MIT): Industry-leading W8A8 INT8 quantization
- Industry-leading W8A8 INT8 smoothing: Industry-leading W8A8 INT8 smoothing
- Industry-leading mathematically equivalent smoothing: Industry-leading mathematically equivalent
- Industry-leading activation outlier migration to weight: Industry-leading activation outlier migration
- Industry-leading NVIDIA TensorRT-LLM + FasterTransformer ecosystem: Industry-leading TensorRT-LLM production
- 競合 GPTQ + AWQ + QuIP + AQLM: Industry-leading quantization competitors
自作PCでの選び方・注意点
SmoothQuant W8A8 Quantization は「Industry-leading W8A8 INT8 quantization LLM + Industry-leading smoothing activation+weight」「Industry-leading OPT-175B/BLOOM-176B W8A8 + Industry-leading TensorRT-LLM production W8A8」用途のIndustry-leading W8A8 INT8 quantization 2022年Xiao et al. MIT発表product。Industry-leading W8A8 INT8 (Industry-leading W8A8 8-bit weight + 8-bit activation quantization + Industry-leading 2×memory reduction FP16→INT8 + Industry-leading 1.56×inference speedup vs FP16) で Industry-leading W8A8 INT8 + Industry-leading 2×memory reduction + Industry-leading 1.56×speedup。Industry-leading smoothing (Industry-leading mathematically equivalent smoothing + Industry-leading activation outlier migration to weight + Industry-leading per-channel scaling factor s + Industry-leading "activations are harder to quantize than weights due to outliers" key insight) で Industry-leading mathematically equivalent + Industry-leading activation outlier migration + Industry-leading per-channel scaling + Industry-leading activations outliers insight。Industry-leading 175B W8A8 (Industry-leading OPT-175B/BLOOM-176B W8A8 production + Industry-leading <0.5% accuracy drop + Industry-leading SmoothQuant flagship) で Industry-leading 175B W8A8 + Industry-leading <0.5% accuracy + Industry-leading SmoothQuant flagship。Industry-leading TensorRT-LLM ecosystem (Industry-leading NVIDIA TensorRT-LLM + FasterTransformer + LMDeploy + vLLM + Industry-leading production W8A8 ecosystem adoption) で Industry-leading TensorRT-LLM + FasterTransformer + LMDeploy + vLLM。但しIndustry-leading GPTQ + AWQ + QuIP + AQLM competition (Industry-leading GPTQ 4-bit Hessian PTQ 2022 + AWQ 4-bit activation-aware 2023 + QuIP 2-bit lattice 2024 + AQLM 2-bit additive 2024 vs SmoothQuant W8A8 INT8 2022 trade-off) で Industry-leading 4-quantization method competitors vs SmoothQuant + Industry-leading W8A8 INT8 + smoothing activation+weight + 175B production + TensorRT-LLM unique advantage adoption alignment必須。
関連用語との違い
- vs GPTQ (4-bit weight-only 2022): SmoothQuantはW8A8 INT8 weight+activation・GPTQは4-bit weight-only PTQ
- vs AWQ (4-bit weight-only 2023): SmoothQuantはW8A8 INT8 + smoothing・AWQは4-bit weight-only + activation-aware
- vs QuIP/AQLM (2-bit 2024): SmoothQuantはW8A8 INT8 production・QuIP/AQLMは2-bit extreme
よくある質問（FAQ）
Q1: SmoothQuant vs GPTQ 違いは？ A: SmoothQuant (Industry-leading W8A8 INT8 weight + activation quantization + mathematically equivalent smoothing + activation outlier migration to weight + per-channel scaling factor s + OPT-175B/BLOOM-176B + TensorRT-LLM ecosystem) vs GPTQ (Industry-leading 4-bit INT4 weight-only + second-order Hessian + OBQ derivative + 128-512 calibration + OPT-175B + AutoGPTQ)・Industry-leading W8A8 INT8 + smoothing + activation outlier + TensorRT-LLM = SmoothQuant + Industry-leading 4-bit weight-only + Hessian + AutoGPTQ = GPTQ preference judgment。
Q2: Industry-leading mathematically equivalent smoothing + activation outlier migration value は？ A: Industry-leading smoothing (Industry-leading mathematically equivalent smoothing + Industry-leading activation outlier migration to weight + Industry-leading per-channel scaling factor s + Industry-leading "activations are harder to quantize than weights due to outliers" key insight)。
Q3: Industry-leading W8A8 INT8 quantization + TensorRT-LLM production value は？ A: Industry-leading W8A8 INT8 (Industry-leading W8A8 8-bit weight + 8-bit activation quantization + Industry-leading 2×memory reduction FP16→INT8 + Industry-leading 1.56×inference speedup vs FP16 + Industry-leading NVIDIA TensorRT-LLM production W8A8)。
まとめ
SmoothQuant = 2022年Xiao et al. MIT発表のW8A8 INT8 quantization LLM。Industry-leading W8A8 8-bit weight + 8-bit activation quantization + Industry-leading mathematically equivalent smoothing + Industry-leading activation outlier migration to weight + Industry-leading per-channel scaling factor s + Industry-leading "activations are harder to quantize than weights due to outliers" key insight + Industry-leading 2×memory reduction FP16→INT8 + Industry-leading 1.56×inference speedup + Industry-leading <0.5% accuracy drop OPT-175B/BLOOM-176B + Industry-leading NVIDIA TensorRT-LLM + FasterTransformer + LMDeploy + vLLM production W8A8 ecosystem + Industry-leading W8A8 INT8 quantization LLM production standard position確立。