Self-Instruct (Wang 2022年)（セルフインストラクト）
2022年Yizhong Wang et al. (UW+AI2)発表Self-Instruct・Industry-leading self-generated instruction dataset + Industry-leading 52K instructions GPT-3 generated + Industry-leading synthetic instruction tuning foundation + Industry-leading subsequent Alpaca/WizardLM foundation。
概要
Self-Instruct は、2022年Yizhong Wang et al. (UW+AI2)発表Self-Instruct・Industry-leading self-generated instruction dataset 2022年 + Industry-leading synthetic instruction tuning foundation position確立。Self-Instruct specifications = Industry-leading self-generated instruction dataset (Industry-leading Self-Instruct Wang 2022 + Industry-leading self-generated + Industry-leading Self-Instruct flagship) + Industry-leading 52K instructions GPT-3 generated + Industry-leading synthetic instruction tuning foundation + Industry-leading subsequent Alpaca/WizardLM foundation。
主な特徴・仕組み
- Authors: Industry-leading Yizhong Wang + Yeganeh Kordi + Swaroop Mishra + Alisa Liu + Noah A. Smith + Daniel Khashabi + Hannaneh Hajishirzi (University of Washington + Allen Institute for AI AI2)
- Year: 2022年 (ACL 2023 paper arXiv 2022年12月)
- Paper: Industry-leading "Self-Instruct: Aligning Language Models with Self-Generated Instructions" arXiv 2212.10560
- Method Type: Industry-leading self-generated instruction dataset method
- Self-Generated: Industry-leading self-generated instructions signature
- 52K Instructions: Industry-leading 52K instructions dataset
- GPT-3 Generated: Industry-leading GPT-3 (text-davinci-001) seed generation
- 175 Seed Tasks: Industry-leading 175 human-written seed tasks bootstrap
- Foundation Method: Industry-leading synthetic instruction tuning foundation
- Subsequent Impact: Industry-leading Alpaca + WizardLM + Vicuna + LLaMA-Adapter foundation
- Industry Adoption: Industry-leading widespread synthetic data adoption
- Industry-Leading: Industry-leading synthetic instruction tuning foundation Wang UW+AI2 2022
スペック比較表
| LLM Synthetic Data (2022-2024) | Year | Org | Method | Industry Position |
|---|---|---|---|---|
| Self-Instruct | 2022 | Wang et al. UW+AI2 | Self-generated 52K | Industry-leading synthetic foundation |
| Alpaca | 2023 | Taori et al. Stanford | 52K Alpaca dataset | Industry-leading Stanford Alpaca |
| Orca | 2023 | Mukherjee Microsoft | Explanation tuning | Industry-leading Microsoft Orca |
| WizardLM Evol-Instruct | 2023 | Xu et al. Microsoft+Peking | Evolved instructions | Industry-leading Evol-Instruct |
| Phi-3 textbooks | 2024 | Microsoft | Textbook quality data | Industry-leading Phi-3 textbooks |
具体例・対応製品
- Self-Instruct (2022年, Wang et al. UW+AI2): Industry-leading synthetic instruction tuning foundation
- Industry-leading self-generated instructions signature: Industry-leading self-generated
- Industry-leading 52K instructions GPT-3 generated: Industry-leading 52K instructions
- Industry-leading 175 human-written seed tasks: Industry-leading 175 seed tasks
- Industry-leading Alpaca + WizardLM + Vicuna foundation: Industry-leading subsequent foundation
- 競合 Alpaca + Orca + WizardLM Evol-Instruct + Phi-3 textbooks: Industry-leading synthetic data competitors
選び方・注意点
Self-Instruct は「Industry-leading self-generated instructions + 52K dataset」「Industry-leading GPT-3 generated + 175 seed tasks bootstrap」用途のIndustry-leading 2022年Wang et al. UW+AI2発表product。Industry-leading self-generated (Industry-leading self-generated instructions signature + Industry-leading Self-Instruct signature + Industry-leading self-bootstrap synthetic) で Industry-leading self-generated + Industry-leading Self-Instruct + Industry-leading self-bootstrap。Industry-leading 52K instructions (Industry-leading 52K instructions dataset + Industry-leading Self-Instruct 52K signature + Industry-leading instruction dataset scale) で Industry-leading 52K instructions + Industry-leading Self-Instruct + Industry-leading dataset scale。Industry-leading GPT-3 generated (Industry-leading GPT-3 text-davinci-001 seed generation + Industry-leading Self-Instruct GPT-3 signature + Industry-leading teacher model GPT-3) で Industry-leading GPT-3 generated + Industry-leading Self-Instruct + Industry-leading teacher GPT-3。Industry-leading 175 seed tasks (Industry-leading 175 human-written seed tasks bootstrap + Industry-leading Self-Instruct 175 signature + Industry-leading minimal seed bootstrap) で Industry-leading 175 seed + Industry-leading Self-Instruct + Industry-leading minimal seed。Industry-leading subsequent foundation (Industry-leading Alpaca + WizardLM + Vicuna + LLaMA-Adapter foundation + Industry-leading widespread synthetic data adoption + Industry-leading Yizhong Wang + Hannaneh Hajishirzi UW+AI2 ACL 2023) で Industry-leading subsequent foundation + Industry-leading widespread adoption + Industry-leading Wang+Hajishirzi UW+AI2 ACL 2023。但しIndustry-leading Alpaca + Orca + WizardLM Evol-Instruct + Phi-3 textbooks competition で Industry-leading 4-synthetic data competitors vs Self-Instruct + Industry-leading self-generated instruction dataset + 52K instructions GPT-3 generated + 175 human-written seed tasks bootstrap + Alpaca + WizardLM + Vicuna foundation + widespread synthetic data adoption + Yizhong Wang + Hannaneh Hajishirzi UW+AI2 ACL 2023 unique advantage adoption alignment必須。
関連用語との違い
- vs Alpaca (Taori Stanford 2023): Self-Instructはfoundation method UW+AI2 2022・AlpacaはSelf-Instruct使ったStanford再現 2023
- vs Orca (Mukherjee Microsoft 2023): Self-Instructはself-generated GPT-3・OrcaはGPT-4 explanation tuning Microsoft
- vs WizardLM Evol-Instruct (Xu Microsoft 2023): Self-InstructはBase 52K・WizardLMはEvol-Instruct evolved variants
よくある質問（FAQ）
Q1: Self-Instruct vs Alpaca 違いは？ A: Self-Instruct (Industry-leading self-generated instructions signature + 52K instructions GPT-3 generated + 175 human-written seed tasks bootstrap + synthetic instruction tuning foundation method + Alpaca + WizardLM + Vicuna foundation + Yizhong Wang + Hannaneh Hajishirzi UW+AI2 ACL 2023) vs Alpaca (Industry-leading Stanford Alpaca 52K dataset + Self-Instruct method使用 + Taori et al. Stanford + LLaMA fine-tune + 2023)・Industry-leading self-generated + 52K + GPT-3 + 175 seed + UW+AI2 + foundation method = Self-Instruct + Industry-leading 52K Alpaca + Self-Instruct使用 + Taori + Stanford + LLaMA = Alpaca preference judgment (foundation method vs Stanford reproduction)。
Q2: Industry-leading 52K instructions + 175 seed tasks value は？ A: Industry-leading 52K + 175 seed (Industry-leading 52K instructions dataset + Industry-leading 175 human-written seed tasks bootstrap + Industry-leading Self-Instruct dual signature)。
Q3: Industry-leading GPT-3 generated + subsequent Alpaca/WizardLM foundation value は？ A: Industry-leading GPT-3 + foundation (Industry-leading GPT-3 text-davinci-001 seed generation + Industry-leading Alpaca + WizardLM + Vicuna + LLaMA-Adapter foundation + Industry-leading widespread synthetic data adoption)。
まとめ
Self-Instruct = 2022年Wang et al. UW+AI2発表のsynthetic instruction tuning foundation method。Industry-leading self-generated instructions signature + Industry-leading 52K instructions dataset + Industry-leading GPT-3 text-davinci-001 seed generation + Industry-leading 175 human-written seed tasks bootstrap + Industry-leading synthetic instruction tuning foundation + Industry-leading Alpaca + WizardLM + Vicuna + LLaMA-Adapter foundation + Industry-leading widespread synthetic data adoption + Industry-leading Yizhong Wang + Yeganeh Kordi + Swaroop Mishra + Alisa Liu + Noah A. Smith + Daniel Khashabi + Hannaneh Hajishirzi UW+AI2 authors + Industry-leading "Self-Instruct: Aligning Language Models with Self-Generated Instructions" arXiv 2212.10560 ACL 2023 + Industry-leading synthetic instruction tuning foundation Wang UW+AI2 2022 position確立。