WizardLM Evol-Instruct (Xu 2023年)（ウィザードLMエボルインストラクト）
2023年Can Xu et al. (Microsoft+Peking University)発表WizardLM Evol-Instruct・Industry-leading Evol-Instruct evolved instructions + Industry-leading In-depth+In-breadth evolving + Industry-leading WizardLM 7B/13B/30B/70B + Industry-leading evolved instruction tuning。
概要
WizardLM Evol-Instruct は、2023年Can Xu et al. (Microsoft+Peking University)発表WizardLM Evol-Instruct・Industry-leading Evol-Instruct evolved instructions 2023年 + Industry-leading evolved instruction tuning position確立。WizardLM Evol-Instruct specifications = Industry-leading Evol-Instruct evolved instructions (Industry-leading WizardLM Evol-Instruct Xu 2023 + Industry-leading Evol-Instruct + Industry-leading WizardLM flagship) + Industry-leading In-depth+In-breadth evolving + Industry-leading WizardLM 7B/13B/30B/70B + Industry-leading evolved instruction tuning。
主な特徴・仕組み
- Authors: Industry-leading Can Xu + Qingfeng Sun + Kai Zheng + Xiubo Geng + Pu Zhao + Jiazhan Feng + Chongyang Tao + Daxin Jiang (Microsoft + Peking University)
- Year: 2023年 (arXiv 2023年4月)
- Paper: Industry-leading "WizardLM: Empowering Large Language Models to Follow Complex Instructions" arXiv 2304.12244 ICLR 2024
- Method Type: Industry-leading Evol-Instruct evolved instructions method
- Evol-Instruct: Industry-leading Evol-Instruct evolved instructions signature
- In-Depth Evolving: Industry-leading In-depth evolving signature (5 operations)
- In-Breadth Evolving: Industry-leading In-breadth evolving signature
- WizardLM Models: Industry-leading WizardLM 7B + 13B + 30B + 70B variants
- WizardCoder: Industry-leading WizardCoder code variant + WizardMath math variant
- Evolved Instructions: Industry-leading evolved instruction complexity progression
- Industry Impact: Industry-leading evolved instruction tuning methodology
- Industry-Leading: Industry-leading Evol-Instruct evolved instruction tuning Microsoft+Peking 2023
スペック比較表
| LLM Synthetic Data (2022-2024) | Year | Org | Method | Industry Position |
|---|---|---|---|---|
| WizardLM Evol-Instruct | 2023 | Xu et al. Microsoft+Peking | Evolved instructions | Industry-leading Evol-Instruct |
| Self-Instruct | 2022 | Wang et al. UW+AI2 | Self-generated 52K | Industry-leading synthetic foundation |
| Alpaca | 2023 | Taori et al. Stanford | 52K Alpaca dataset | Industry-leading Stanford Alpaca |
| Orca | 2023 | Mukherjee Microsoft | Explanation tuning | Industry-leading Microsoft Orca |
| Phi-3 textbooks | 2024 | Microsoft | Textbook quality data | Industry-leading Phi-3 textbooks |
具体例・対応製品
- WizardLM Evol-Instruct (2023年, Xu et al. Microsoft+Peking): Industry-leading Evol-Instruct evolved
- Industry-leading Evol-Instruct evolved instructions signature: Industry-leading Evol-Instruct
- Industry-leading In-depth evolving 5 operations signature: Industry-leading In-depth evolving
- Industry-leading In-breadth evolving signature: Industry-leading In-breadth evolving
- Industry-leading WizardLM 7B + 13B + 30B + 70B + WizardCoder + WizardMath: Industry-leading WizardLM family
- 競合 Self-Instruct + Alpaca + Orca + Phi-3 textbooks: Industry-leading synthetic data competitors
選び方・注意点
WizardLM Evol-Instruct は「Industry-leading Evol-Instruct evolved instructions + In-depth+In-breadth」「Industry-leading WizardLM 7B-70B + evolved instruction tuning」用途のIndustry-leading 2023年Xu et al. Microsoft+Peking発表product。Industry-leading Evol-Instruct (Industry-leading Evol-Instruct evolved instructions signature + Industry-leading WizardLM Evol-Instruct signature + Industry-leading evolved instruction methodology) で Industry-leading Evol-Instruct + Industry-leading WizardLM + Industry-leading evolved methodology。Industry-leading In-depth evolving (Industry-leading In-depth evolving 5 operations signature + Industry-leading WizardLM In-depth signature + Industry-leading add constraints + deepening + concretizing + increased reasoning + complicating input) で Industry-leading In-depth evolving + Industry-leading WizardLM In-depth + Industry-leading 5 operations。Industry-leading In-breadth evolving (Industry-leading In-breadth evolving signature + Industry-leading WizardLM In-breadth signature + Industry-leading topic diversity expansion) で Industry-leading In-breadth + Industry-leading WizardLM In-breadth + Industry-leading topic diversity。Industry-leading WizardLM family (Industry-leading WizardLM 7B + 13B + 30B + 70B variants + Industry-leading WizardCoder code + WizardMath math variants + Industry-leading WizardLM family ecosystem) で Industry-leading WizardLM family + Industry-leading WizardCoder + WizardMath + Industry-leading family ecosystem。Industry-leading Microsoft+Peking (Industry-leading Microsoft + Peking University collaboration + Industry-leading Can Xu + Qingfeng Sun + Daxin Jiang Microsoft+Peking authors + Industry-leading ICLR 2024 acceptance + Industry-leading evolved instruction tuning methodology) で Industry-leading Microsoft+Peking + Industry-leading Xu+Sun+Jiang + Industry-leading ICLR 2024 + Industry-leading methodology。但しIndustry-leading Self-Instruct + Alpaca + Orca + Phi-3 textbooks competition で Industry-leading 4-synthetic data competitors vs WizardLM Evol-Instruct + Industry-leading Evol-Instruct evolved instructions + In-depth evolving 5 operations + In-breadth evolving + WizardLM 7B + 13B + 30B + 70B + WizardCoder + WizardMath family + evolved instruction tuning methodology + Can Xu + Qingfeng Sun + Daxin Jiang Microsoft + Peking University ICLR 2024 unique advantage adoption alignment必須。
関連用語との違い
- vs Self-Instruct (Wang 2022): WizardLM Evol-InstructはEvol-Instruct evolved + In-depth/In-breadth・Self-Instructはself-generated 52K static foundation
- vs Alpaca (Stanford 2023): WizardLM Evol-InstructはEvol-Instruct evolved + Microsoft+Peking・Alpacaは52K static + Stanford + LLaMA
- vs Orca (Microsoft 2023): WizardLM Evol-InstructはEvol-Instruct evolved + Xu・Orcaはexplanation tuning + GPT-4 + Mukherjee
よくある質問（FAQ）
Q1: WizardLM Evol-Instruct vs Self-Instruct 違いは？ A: WizardLM Evol-Instruct (Industry-leading Evol-Instruct evolved instructions + In-depth evolving 5 operations + In-breadth evolving + WizardLM 7B + 13B + 30B + 70B + WizardCoder + WizardMath family + evolved instruction tuning methodology + Can Xu + Daxin Jiang Microsoft + Peking University ICLR 2024) vs Self-Instruct (Industry-leading self-generated instructions + 52K static + GPT-3 generated + 175 seed tasks + synthetic instruction tuning foundation + Wang+Hajishirzi UW+AI2 ACL 2023)・Industry-leading Evol-Instruct + In-depth + In-breadth + WizardLM family + evolved + Microsoft+Peking = WizardLM + Industry-leading self-generated + 52K static + GPT-3 + 175 seed + foundation + UW+AI2 = Self-Instruct preference judgment (evolved vs static differ)。
Q2: Industry-leading Evol-Instruct + In-depth+In-breadth value は？ A: Industry-leading Evol-Instruct + In-depth+In-breadth (Industry-leading Evol-Instruct evolved instructions + Industry-leading In-depth evolving 5 operations + Industry-leading In-breadth evolving + Industry-leading WizardLM triple signature)。
Q3: Industry-leading WizardLM family + Microsoft+Peking value は？ A: Industry-leading WizardLM family + Microsoft+Peking (Industry-leading WizardLM 7B + 13B + 30B + 70B + WizardCoder + WizardMath family + Industry-leading Microsoft + Peking University collaboration + Industry-leading Can Xu + Daxin Jiang authors ICLR 2024)。
まとめ
WizardLM Evol-Instruct = 2023年Xu et al. Microsoft+Peking University発表のevolved instruction tuning method。Industry-leading Evol-Instruct evolved instructions signature + Industry-leading In-depth evolving 5 operations + Industry-leading In-breadth evolving + Industry-leading WizardLM 7B + 13B + 30B + 70B variants + Industry-leading WizardCoder code + WizardMath math variants + Industry-leading evolved instruction tuning methodology + Industry-leading Can Xu + Qingfeng Sun + Kai Zheng + Xiubo Geng + Pu Zhao + Jiazhan Feng + Chongyang Tao + Daxin Jiang Microsoft + Peking University authors + Industry-leading "WizardLM: Empowering Large Language Models to Follow Complex Instructions" arXiv 2304.12244 ICLR 2024 + Industry-leading Evol-Instruct evolved instruction tuning Microsoft+Peking 2023 position確立。