AWQ Activation-aware Weight Quantization (Lin 2023年)（エーダブリューキュー）
2023年Lin et al. (MIT)発表AWQ・Industry-leading activation-aware weight quantization LLM + Industry-leading 4-bit quantization + Industry-leading salient channel protection + Industry-leading LLaMA/Mistral production quantization。
概要
AWQ Activation-aware Weight Quantization は、2023年Lin et al. (MIT)発表AWQ・Industry-leading activation-aware weight quantization LLM 2023年 + Industry-leading LLaMA/Mistral production quantization position確立。AWQ specifications = Industry-leading activation-aware weight quantization LLM (Industry-leading AWQ 2023 + Industry-leading activation-aware weight quantization LLM + Industry-leading AWQ flagship) + Industry-leading 4-bit quantization (Industry-leading 4-bit activation-aware weight quantization) + Industry-leading salient channel protection (Industry-leading salient channel 1% protection + Industry-leading per-channel scaling) + Industry-leading LLaMA/Mistral production quantization (Industry-leading LLaMA/Mistral production quantization)。
主な特徴・仕組み
- Authors: Industry-leading Ji Lin + Jiaming Tang + Haotian Tang + Shang Yang + Wei-Ming Chen + Wei-Chen Wang + Guangxuan Xiao + Xingyu Dang + Chuang Gan + Song Han MIT + NVIDIA + UMass
- Year: 2023年 (arXiv 2023年6月発表)
- Paper: Industry-leading "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration" arXiv 2306.00978
- Quantization Type: Industry-leading activation-aware weight quantization
- Bits: Industry-leading 4-bit weight-only
- Method: Industry-leading salient channel protection 1% + Industry-leading per-channel scaling + Industry-leading group-wise quantization
- Key Insight: Industry-leading "weights aren't equally important" insight
- Salient Channel: Industry-leading top 1% salient channel keep FP16
- Speed: Industry-leading 3-4×inference speedup
- Memory Reduction: Industry-leading 4×memory reduction FP16→INT4
- Production Adoption: Industry-leading AutoAWQ + llama.cpp + vLLM + TGI + LMDeploy + MLC LLM
- Industry-Leading: Industry-leading activation-aware quantization LLM
スペック比較表
| LLM Quantization Method | Year | Bits | Method | Industry Position |
|---|---|---|---|---|
| AWQ | 2023 | 4-bit | Activation-aware weight quantization | Industry-leading activation-aware quantization |
| GPTQ | 2022 | 4/3/2-bit | Second-order Hessian PTQ weight-only | Industry-leading 4-bit PTQ 175B |
| SmoothQuant | 2022 | INT8 | Smoothing activation+weight quantization | Industry-leading W8A8 production |
| QuIP | 2024 | 2-bit | Incoherence processing + lattice quantization | Industry-leading 2-bit extreme quantization |
| AQLM | 2024 | 2-bit | Additive quantization with learned codebooks | Industry-leading 2-bit additive quantization |
具体例・対応製品
- AWQ (2023年, Lin et al. MIT): Industry-leading activation-aware weight quantization
- Industry-leading 4-bit activation-aware: Industry-leading 4-bit activation-aware
- Industry-leading salient channel 1% protection: Industry-leading salient channel 1% FP16 keep
- Industry-leading per-channel scaling + group-wise quantization: Industry-leading per-channel + group-wise
- Industry-leading AutoAWQ + llama.cpp + vLLM + LMDeploy ecosystem: Industry-leading production ecosystem
- 競合 GPTQ + SmoothQuant + QuIP + AQLM: Industry-leading quantization competitors
自作PCでの選び方・注意点
AWQ Activation-aware Weight Quantization は「Industry-leading activation-aware weight quantization LLM + Industry-leading 4-bit quantization」「Industry-leading salient channel protection + Industry-leading LLaMA/Mistral production quantization」用途のIndustry-leading activation-aware quantization 2023年Lin et al. MIT発表product。Industry-leading 4-bit activation-aware (Industry-leading 4-bit activation-aware weight quantization + Industry-leading 4×memory reduction FP16→INT4 + Industry-leading 3-4×inference speedup) で Industry-leading 4-bit activation-aware + Industry-leading 4×memory reduction + Industry-leading 3-4×inference speedup。Industry-leading salient channel protection (Industry-leading salient channel 1% protection + Industry-leading "weights aren't equally important" key insight + Industry-leading top 1% salient channel keep FP16 + Industry-leading per-channel scaling + Industry-leading group-wise quantization) で Industry-leading salient channel 1% + Industry-leading weights insight + Industry-leading 1% FP16 keep + Industry-leading per-channel + Industry-leading group-wise。Industry-leading LLaMA/Mistral production (Industry-leading LLaMA/Mistral production quantization + Industry-leading AWQ flagship + Industry-leading activation-aware quantization LLM standard) で Industry-leading LLaMA/Mistral + Industry-leading AWQ flagship + Industry-leading activation-aware standard。Industry-leading AutoAWQ ecosystem (Industry-leading AutoAWQ + llama.cpp + vLLM + TGI + LMDeploy + MLC LLM + Industry-leading production ecosystem adoption) で Industry-leading AutoAWQ + llama.cpp + vLLM + TGI + LMDeploy + MLC LLM。但しIndustry-leading GPTQ + SmoothQuant + QuIP + AQLM competition (Industry-leading GPTQ 4-bit Hessian PTQ 2022 + SmoothQuant W8A8 INT8 2022 + QuIP 2-bit lattice 2024 + AQLM 2-bit additive 2024 vs AWQ 4-bit activation-aware 2023 trade-off) で Industry-leading 4-quantization method competitors vs AWQ + Industry-leading 4-bit activation-aware + salient channel 1% + per-channel scaling + LLaMA/Mistral + AutoAWQ ecosystem unique advantage adoption alignment必須。
関連用語との違い
- vs GPTQ (4-bit Hessian 2022): AWQはactivation-aware + salient channel保護・GPTQはsecond-order Hessian + OBQ derivative
- vs SmoothQuant (W8A8 2022): AWQは4-bit weight-only + activation-aware・SmoothQuantはINT8 weight+activation
- vs QuIP/AQLM (2-bit 2024): AWQは4-bit activation-aware・QuIP/AQLMは2-bit extreme quantization
よくある質問（FAQ）
Q1: AWQ vs GPTQ 違いは？ A: AWQ (Industry-leading 4-bit activation-aware weight quantization + salient channel 1% protection + per-channel scaling + group-wise quantization + "weights aren't equally important" + LLaMA/Mistral + AutoAWQ ecosystem) vs GPTQ (Industry-leading 4-bit INT4 + second-order Hessian + OBQ derivative + 128-512 calibration + OPT-175B/BLOOM-176B + AutoGPTQ ecosystem)・Industry-leading 4-bit + activation-aware + salient channel + LLaMA = AWQ + Industry-leading 4-bit + Hessian + 175B + AutoGPTQ = GPTQ preference judgment。
Q2: Industry-leading salient channel 1% protection + per-channel scaling value は？ A: Industry-leading salient channel (Industry-leading salient channel 1% protection + Industry-leading "weights aren't equally important" key insight + Industry-leading top 1% salient channel keep FP16 + Industry-leading per-channel scaling + Industry-leading group-wise quantization)。
Q3: Industry-leading activation-aware weight quantization + LLaMA/Mistral production value は？ A: Industry-leading activation-aware (Industry-leading activation-aware weight quantization + Industry-leading 4-bit activation-aware quantization + Industry-leading LLaMA/Mistral production quantization + Industry-leading AWQ flagship)。
まとめ
AWQ = 2023年Lin et al. MIT発表のactivation-aware weight quantization LLM。Industry-leading 4-bit activation-aware weight quantization + Industry-leading salient channel 1% protection + Industry-leading "weights aren't equally important" key insight + Industry-leading top 1% salient channel keep FP16 + Industry-leading per-channel scaling + Industry-leading group-wise quantization + Industry-leading 4×memory reduction FP16→INT4 + Industry-leading 3-4×inference speedup + Industry-leading LLaMA/Mistral production quantization + Industry-leading AutoAWQ + llama.cpp + vLLM + TGI + LMDeploy + MLC LLM production ecosystem + Industry-leading activation-aware quantization LLM standard position確立。