LoRA (Low-Rank Adaptation・Hu + Microsoft 2021年-2024年)（ローラ）
2021年Hu et al. + Microsoft発表LoRA paper・Industry-leading Low-Rank Adaptation PEFT paradigm Pioneer + Industry-leading 全LLM PEFT industry-standard + Industry-leading 3年heritage LoRA Pioneer foundation。
概要
LoRA は、2021年Hu et al. + Microsoft発表LoRA paper・Industry-leading Low-Rank Adaptation PEFT paradigm Pioneer 2021年-2024年 + Industry-leading 全LLM PEFT industry-standard position確立。LoRA specifications = Industry-leading Low-Rank Adaptation PEFT paradigm Pioneer (Industry-leading LoRA + Industry-leading Low-Rank Adaptation + Industry-leading PEFT paradigm Pioneer) + Industry-leading 全LLM PEFT industry-standard (Industry-leading 全LLM PEFT industry-standard + Industry-leading Llama/Mistral/Qwen全LLM LoRA adoption) + Industry-leading 3年heritage LoRA Pioneer foundation (Industry-leading 2021年-2024年3年heritage + Industry-leading QLoRA/DoRA/PiSSA successors Pioneer foundation)。
主な特徴・仕組み
- Authoring: Hu et al. + Microsoft 2021年発表
- Paradigm: Industry-leading Low-Rank Adaptation PEFT
- Mechanism: Industry-leading low-rank decomposition ΔW = BA
- Heritage: Industry-leading 3年heritage LoRA Pioneer foundation
- Microsoft Backing: Industry-leading Microsoft Research backing
- Industry Position: Industry-leading PEFT paradigm Pioneer
- vs Full Fine-Tuning: Industry-leading parameter-efficient vs full fine-tuning
- Industry-Leading: Industry-leading 全LLM PEFT industry-standard
- Successors: Industry-leading QLoRA + DoRA + PiSSA + IA3 successors
- Use Cases: Industry-leading 全modern LLM fine-tuning industry-standard
スペック比較表
| LLM PEFT (2021-2024) | Approach | Heritage | Industry Position |
|---|---|---|---|
| LoRA (Hu Microsoft) | Low-Rank Adaptation ΔW = BA | 2021年Microsoft Pioneer | Industry-leading PEFT Pioneer + 全LLM standard |
| QLoRA (Dettmers UW) | Quantized LoRA 4-bit NF4 | 2023年UW | Industry-emerging Quantized LoRA |
| DoRA (Liu NVIDIA) | Weight-Decomposed LoRA | 2024年NVIDIA | Industry-emerging DoRA NVIDIA |
| PiSSA (Meng BJTU) | Principal Singular values + Vectors Adaptation | 2024年BJTU | Industry-emerging PiSSA SVD |
| IA3 (Liu) | Infused Adapter by Inhibiting Activations | 2022年 | Industry-emerging IA3 |
具体例・対応製品
- LoRA (Hu + Microsoft・2021年-2024年): Industry-leading PEFT paradigm Pioneer
- Industry-leading Low-Rank Adaptation ΔW = BA decomposition: Industry-leading low-rank decomposition
- Industry-leading 全LLM PEFT industry-standard: Industry-leading 全LLM standard
- Industry-leading Microsoft Research backing: Industry-leading Microsoft Research
- Industry-leading QLoRA + DoRA + PiSSA + IA3 successors Pioneer foundation: Industry-leading successors
- 競合 QLoRA + DoRA + PiSSA + IA3 successors: Industry-emerging PEFT successor competitors
自作PCでの選び方・注意点
LoRA は「Industry-leading Low-Rank Adaptation PEFT paradigm Pioneer + Industry-leading 全LLM PEFT industry-standard」「Industry-leading 3年heritage LoRA Pioneer foundation」用途のIndustry-leading PEFT Pioneer 2021年Hu + Microsoft product。Industry-leading Low-Rank Adaptation (Industry-leading LoRA + Industry-leading Low-Rank Adaptation + Industry-leading low-rank decomposition ΔW = BA + Industry-leading PEFT paradigm Pioneer) で Industry-leading Low-Rank Adaptation + Industry-leading ΔW = BA decomposition + Industry-leading PEFT Pioneer。Industry-leading 全LLM PEFT industry-standard (Industry-leading 全LLM PEFT industry-standard + Industry-leading Llama/Mistral/Qwen全LLM LoRA adoption + Industry-leading 全modern LLM fine-tuning industry-standard) で Industry-leading 全LLM PEFT standard + Industry-leading Llama/Mistral/Qwen全LLM + Industry-leading 全modern LLM。Industry-leading 3年heritage Pioneer foundation (Industry-leading 2021年-2024年3年heritage + Industry-leading QLoRA/DoRA/PiSSA successors Pioneer foundation + Industry-leading Microsoft Research backing) で Industry-leading 3年heritage Pioneer foundation + Industry-leading successors Pioneer + Industry-leading Microsoft Research backing。但しIndustry-leading QLoRA + DoRA + PiSSA + IA3 successors evolution competition (Industry-leading QLoRA UW 2023 quantized + DoRA NVIDIA 2024 weight-decomposed + PiSSA BJTU 2024 SVD + IA3 2022 vs LoRA Pioneer foundation trade-off) で Industry-leading successors 4-paradigm evolution vs LoRA Pioneer + Industry-leading 3年Pioneer foundation + 全LLM standard adoption alignment必須。
関連用語との違い
- vs QLoRA (Dettmers UW 2023年Quantized): LoRAはLow-Rank Adaptation Pioneer + Microsoft 2021年・QLoRAはQuantized LoRA 4-bit NF4 + UW 2023年
- vs DoRA (Liu NVIDIA 2024年): LoRAはLow-Rank Adaptation Pioneer + 2021年・DoRAはWeight-Decomposed LoRA + NVIDIA 2024年successor
- vs PiSSA (Meng BJTU 2024年): LoRAはLow-Rank Pioneer Microsoft 2021年・PiSSAはPrincipal Singular values + Vectors Adaptation + BJTU 2024年
よくある質問（FAQ）
Q1: LoRA vs QLoRA 違いは？ A: LoRA (Hu + Microsoft 2021年 + Low-Rank Adaptation ΔW = BA + Microsoft Research + 全LLM PEFT industry-standard + 3年heritage Pioneer foundation) vs QLoRA (Dettmers + UW 2023年 + Quantized LoRA 4-bit NF4 quantization + UW emerging + memory-efficient quantized LoRA)・Industry-leading Pioneer + 全LLM standard + Microsoft + 3年heritage = LoRA + Industry-leading 4-bit NF4 + quantized + UW = QLoRA preference judgment。
Q2: Industry-leading Low-Rank Adaptation ΔW = BA value は？ A: Industry-leading Low-Rank Adaptation (Industry-leading LoRA + Industry-leading Low-Rank Adaptation + Industry-leading low-rank decomposition ΔW = BA + Industry-leading PEFT paradigm Pioneer + Industry-leading parameter-efficient adaptation)。
Q3: Industry-leading 全LLM PEFT industry-standard value は？ A: Industry-leading 全LLM PEFT (Industry-leading 全LLM PEFT industry-standard + Industry-leading Llama/Mistral/Qwen全LLM LoRA adoption + Industry-leading 全modern LLM fine-tuning industry-standard + Industry-leading 3年LoRA Pioneer adoption)。
まとめ
LoRA = 2021年Hu et al. + Microsoft発表のLow-Rank Adaptation PEFT paradigm Pioneer。Industry-leading low-rank decomposition ΔW = BA + Industry-leading Microsoft Research backing + Industry-leading 全LLM PEFT industry-standard + Industry-leading Llama/Mistral/Qwen全LLM LoRA adoption + Industry-leading QLoRA + DoRA + PiSSA + IA3 Industry-leading successors Pioneer foundation + Industry-leading 3年heritage LoRA Pioneer foundation position確立。