YaRN (Yet another RoPE extensioN・2023年-2024年)（ヤーン）
2023年Bowen Peng et al.発表YaRN paper・Industry-leading Yet another RoPE extensioN paradigm + Industry-leading efficient RoPE scaling + Industry-leading NTK-Aware improved + Industry-leading Mistral + Llama 2 adoption + Industry-leading context length extension efficient paradigm。
概要
YaRN は、2023年Bowen Peng et al.発表YaRN paper・Industry-leading Yet another RoPE extensioN paradigm 2023年-2024年 + Industry-emerging efficient RoPE scaling paradigm Pioneer position確立。YaRN specifications = Industry-leading Yet another RoPE extensioN paradigm (Industry-leading Yet another RoPE extensioN paradigm + Industry-emerging YaRN efficient RoPE scaling paradigm Pioneer) + Industry-leading efficient RoPE scaling (Industry-leading efficient RoPE scaling + Industry-leading NTK-Aware improved efficiency + Industry-leading low-fine-tuning RoPE scaling paradigm) + Industry-leading NTK-Aware improved (Industry-leading NTK-Aware foundational + Industry-emerging YaRN improved paradigm) + Industry-leading Mistral + Llama 2 adoption (Industry-leading Mistral 7B Industry-leading YaRN adoption + Industry-leading Llama 2 Industry-leading YaRN extension) + Industry-leading context length extension efficient paradigm (Industry-leading 4k → 64k + 128k efficient context length extension paradigm)。
主な特徴・仕組み
- Authoring: Bowen Peng et al. 2023年発表
- Paradigm: Industry-leading Yet another RoPE extensioN
- Approach: Industry-leading efficient RoPE scaling
- Improvement: Industry-leading NTK-Aware improved efficiency
- LLM Adoption: Industry-leading Mistral 7B + Llama 2 adoption
- Context Extension: Industry-leading 4k → 64k/128k efficient
- Industry Position: Industry-emerging efficient RoPE scaling Pioneer
- Fine-tuning: Industry-leading low-fine-tuning paradigm
- Industry-Emerging: Industry-emerging YaRN paradigm Pioneer
- Use Cases: Industry-leading RoPE LLM efficient context extension
スペック比較表
| LLM Positional Encoding (2024) | Approach | LLM Adoption | Context Extension |
|---|---|---|---|
| YaRN | Efficient RoPE scaling | Mistral + Llama 2 | 4k → 64k/128k efficient |
| NTK-Aware RoPE Scaling | NTK theory-based scaling | RoPE community | 4k → 8k/16k/32k |
| LongRoPE | Microsoft 2M context | Phi-3 + Llama | 2M context Microsoft |
| RoPE (Rotary) | Rotation matrix relative | Llama + Mistral + Qwen + ChatGLM全LLM | Base paradigm |
| ALiBi | Linear bias attention | BLOOM + MPT | Length extrapolation |
具体例・対応製品
- YaRN (Bowen Peng・2023年-2024年): Industry-emerging efficient RoPE scaling
- Mistral 7B Industry-leading YaRN adoption: Industry-leading Mistral adoption
- Llama 2 Industry-leading YaRN extension adoption: Industry-leading Llama 2 extension
- Industry-leading 4k → 64k/128k efficient context: Industry-leading efficient context
- Industry-leading NTK-Aware improved efficiency: Industry-leading NTK-Aware improvement
- 競合 NTK-Aware + LongRoPE: Industry-leading RoPE extension competitors
自作PCでの選び方・注意点
YaRN は「Industry-leading Yet another RoPE extensioN paradigm + Industry-leading efficient RoPE scaling」「Industry-leading NTK-Aware improved + Industry-leading Mistral + Llama 2 adoption」「Industry-leading context length extension efficient paradigm」用途のIndustry-emerging efficient RoPE scaling 2023年Bowen Peng product。Industry-leading efficient RoPE scaling + NTK-Aware improved (Industry-leading efficient RoPE scaling + Industry-leading NTK-Aware improved efficiency + Industry-leading low-fine-tuning RoPE scaling paradigm) で Industry-leading efficient + Industry-leading NTK-Aware improved + Industry-leading low-fine-tuning paradigm。Industry-leading Mistral + Llama 2 adoption (Industry-leading Mistral 7B Industry-leading YaRN adoption + Industry-leading Llama 2 Industry-leading YaRN extension) で Industry-leading Mistral 7B + Industry-leading Llama 2 Industry-leading YaRN adoption。Industry-leading context length extension efficient (Industry-leading 4k → 64k + 128k efficient context length extension paradigm) で Industry-leading 4k → 64k/128k efficient context extension + Industry-leading NTK-Aware 32k improved efficient extension。但しIndustry-leading LongRoPE Microsoft 2M context Industry-leading aggressive extension competition (Industry-leading LongRoPE Microsoft 2M context + Phi-3 + Llama 2 adoption vs YaRN efficient 64-128k trade-off) で Industry-leading LongRoPE 2M aggressive vs YaRN efficient 64-128k + Industry-leading efficient + Mistral/Llama 2 adoption alignment必須。
関連用語との違い
- vs NTK-Aware RoPE Scaling: YaRNはNTK-Aware improved efficient + Mistral/Llama 2 adoption + Bowen Peng・NTK-AwareはFoundational + bloc97 community + community-driven
- vs LongRoPE (Microsoft 2024年): YaRNは2023年efficient 64-128k + Mistral/Llama 2・LongRoPEは2024年Microsoft 2M context + Phi-3/Llama
- vs Mistral 7B + Llama 2 (LLM): YaRNはRoPE extension paradigm・Mistral/Llama 2はLLM adopting YaRN
よくある質問（FAQ）
Q1: YaRN vs NTK-Aware 違いは？ A: YaRN (Bowen Peng 2023年 + efficient RoPE scaling + NTK-Aware improved + Mistral/Llama 2 adoption + 64-128k context) vs NTK-Aware (bloc97 community 2023年 + NTK theory foundational + RoPE community + 8-32k context)・Industry-leading efficient + LLM adoption = YaRN + Industry-leading foundational + community-driven = NTK-Aware preference judgment。
Q2: Industry-leading efficient RoPE scaling + low-fine-tuning value は？ A: Industry-leading YaRN efficient (Industry-leading efficient RoPE scaling + Industry-leading NTK-Aware improved efficiency + Industry-leading low-fine-tuning RoPE scaling paradigm + Industry-leading efficient context extension paradigm)。
Q3: Industry-leading Mistral 7B + Llama 2 adoption value は？ A: Industry-leading YaRN adoption (Industry-leading Mistral 7B Industry-leading YaRN adoption + Industry-leading Llama 2 Industry-leading YaRN extension adoption + Industry-leading commercial LLM Industry-leading YaRN adoption)。
まとめ
YaRN = 2023年Bowen Peng et al.発表のYet another RoPE extensioN paradigm。Industry-leading efficient RoPE scaling + Industry-leading NTK-Aware improved efficiency + Industry-leading low-fine-tuning RoPE scaling + Industry-leading Mistral 7B + Llama 2 adoption + Industry-leading 4k → 64k/128k efficient context length extension + Industry-emerging YaRN paradigm Pioneer position確立。