Alpaca (Taori Stanford 2023年)（アルパカ）
2023年Rohan Taori et al. (Stanford)発表Alpaca・Industry-leading Stanford Alpaca 52K dataset + Industry-leading Self-Instruct method使用 + Industry-leading LLaMA 7B fine-tune + Industry-leading open instruction tuning democratization。
概要
Alpaca は、2023年Rohan Taori et al. (Stanford)発表Alpaca・Industry-leading Stanford Alpaca 52K dataset 2023年 + Industry-leading open instruction tuning democratization position確立。Alpaca specifications = Industry-leading Stanford Alpaca 52K dataset (Industry-leading Alpaca Taori Stanford 2023 + Industry-leading 52K dataset + Industry-leading Alpaca flagship) + Industry-leading Self-Instruct method使用 + Industry-leading LLaMA 7B fine-tune + Industry-leading open instruction tuning democratization。
主な特徴・仕組み
- Authors: Industry-leading Rohan Taori + Ishaan Gulrajani + Tianyi Zhang + Yann Dubois + Xuechen Li + Carlos Guestrin + Percy Liang + Tatsunori B. Hashimoto (Stanford CRFM Center for Research on Foundation Models)
- Year: 2023年 (Stanford blog post 2023年3月)
- Blog Post: Industry-leading "Alpaca: A Strong, Replicable Instruction-Following Model" Stanford CRFM blog 2023年3月13日
- Method Type: Industry-leading instruction tuning LLaMA 7B
- 52K Dataset: Industry-leading 52K Alpaca instructions dataset
- Self-Instruct Used: Industry-leading Self-Instruct method使用
- OpenAI text-davinci-003: Industry-leading OpenAI text-davinci-003 teacher model
- LLaMA 7B Base: Industry-leading LLaMA 7B base model fine-tune
- $600 Total Cost: Industry-leading $600 total training cost democratization
- Open Release: Industry-leading Stanford open release democratization
- Industry Impact: Industry-leading open instruction tuning democratization 2023
- Industry-Leading: Industry-leading Stanford Alpaca open democratization 2023
スペック比較表
| LLM Synthetic Data (2022-2024) | Year | Org | Method | Industry Position |
|---|---|---|---|---|
| Alpaca | 2023 | Taori et al. Stanford | 52K Alpaca dataset | Industry-leading Stanford Alpaca |
| Self-Instruct | 2022 | Wang et al. UW+AI2 | Self-generated 52K | Industry-leading synthetic foundation |
| Orca | 2023 | Mukherjee Microsoft | Explanation tuning | Industry-leading Microsoft Orca |
| WizardLM Evol-Instruct | 2023 | Xu et al. Microsoft+Peking | Evolved instructions | Industry-leading Evol-Instruct |
| Phi-3 textbooks | 2024 | Microsoft | Textbook quality data | Industry-leading Phi-3 textbooks |
具体例・対応製品
- Alpaca (2023年, Taori et al. Stanford): Industry-leading Stanford Alpaca open democratization
- Industry-leading 52K Alpaca instructions dataset: Industry-leading 52K Alpaca dataset
- Industry-leading Self-Instruct method使用: Industry-leading Self-Instruct used
- Industry-leading OpenAI text-davinci-003 teacher: Industry-leading text-davinci-003 teacher
- Industry-leading $600 total training cost: Industry-leading $600 democratization
- 競合 Self-Instruct + Orca + WizardLM Evol-Instruct + Phi-3 textbooks: Industry-leading synthetic data competitors
選び方・注意点
Alpaca は「Industry-leading Stanford Alpaca 52K dataset + Self-Instruct method」「Industry-leading LLaMA 7B fine-tune + $600 democratization」用途のIndustry-leading 2023年Taori et al. Stanford発表product。Industry-leading Stanford Alpaca (Industry-leading Stanford Alpaca 52K dataset + Industry-leading Stanford CRFM signature + Industry-leading Alpaca dataset signature) で Industry-leading Stanford Alpaca + Industry-leading Stanford CRFM + Industry-leading Alpaca dataset。Industry-leading Self-Instruct used (Industry-leading Self-Instruct method使用 + Industry-leading Alpaca Self-Instruct adoption + Industry-leading building on Wang 2022 foundation) で Industry-leading Self-Instruct used + Industry-leading Alpaca adoption + Industry-leading Wang foundation。Industry-leading text-davinci-003 (Industry-leading OpenAI text-davinci-003 teacher model + Industry-leading Alpaca teacher signature + Industry-leading GPT-3.5-era teacher) で Industry-leading text-davinci-003 + Industry-leading Alpaca teacher + Industry-leading GPT-3.5-era teacher。Industry-leading LLaMA 7B base (Industry-leading LLaMA 7B base model fine-tune + Industry-leading Alpaca LLaMA 7B signature + Industry-leading Meta LLaMA leverage) で Industry-leading LLaMA 7B base + Industry-leading Alpaca signature + Industry-leading Meta LLaMA leverage。Industry-leading $600 democratization (Industry-leading $600 total training cost democratization + Industry-leading Stanford open release democratization + Industry-leading Rohan Taori + Percy Liang + Tatsunori Hashimoto Stanford CRFM + Industry-leading 2023年3月13日 blog post release) で Industry-leading $600 democratization + Industry-leading Stanford open release + Industry-leading Taori+Liang+Hashimoto Stanford CRFM + Industry-leading 2023年3月13日 release。但しIndustry-leading Self-Instruct + Orca + WizardLM Evol-Instruct + Phi-3 textbooks competition で Industry-leading 4-synthetic data competitors vs Alpaca + Industry-leading Stanford Alpaca 52K dataset + Self-Instruct method使用 + OpenAI text-davinci-003 teacher + LLaMA 7B base fine-tune + $600 total training cost + Stanford open release democratization + Rohan Taori + Percy Liang + Tatsunori Hashimoto Stanford CRFM Center for Research on Foundation Models 2023年3月13日 unique advantage adoption alignment必須。
関連用語との違い
- vs Self-Instruct (Wang UW+AI2 2022): AlpacaはStanford reproduction使用method 2023・Self-InstructはUW+AI2 foundation method 2022
- vs Orca (Mukherjee Microsoft 2023): AlpacaはStanford open + Self-Instruct + 52K・OrcaはMicrosoft + GPT-4 explanation tuning
- vs WizardLM Evol-Instruct (Xu Microsoft 2023): AlpacaはStanford 52K static + LLaMA 7B・WizardLMはEvol-Instruct evolved variants
よくある質問（FAQ）
Q1: Alpaca vs Self-Instruct 違いは？ A: Alpaca (Industry-leading Stanford Alpaca 52K instructions dataset + Self-Instruct method使用 + OpenAI text-davinci-003 teacher + LLaMA 7B base fine-tune + $600 total training cost + Stanford open release democratization + Rohan Taori + Percy Liang + Tatsunori Hashimoto Stanford CRFM 2023年3月13日) vs Self-Instruct (Industry-leading self-generated instructions + 52K instructions + GPT-3 generated + 175 seed tasks + synthetic instruction tuning foundation method + Wang+Hajishirzi UW+AI2 ACL 2023)・Industry-leading Stanford Alpaca + 52K + Self-Instruct使用 + text-davinci-003 + LLaMA 7B + $600 + Stanford 2023 = Alpaca + Industry-leading self-generated + 52K + GPT-3 + 175 seed + foundation method + UW+AI2 2022 = Self-Instruct preference judgment (foundation vs Stanford reproduction)。
Q2: Industry-leading 52K Alpaca + Self-Instruct method value は？ A: Industry-leading 52K Alpaca + Self-Instruct (Industry-leading Stanford Alpaca 52K dataset + Industry-leading Self-Instruct method使用 + Industry-leading Alpaca dual signature)。
Q3: Industry-leading $600 democratization + LLaMA 7B value は？ A: Industry-leading $600 + LLaMA 7B (Industry-leading $600 total training cost democratization + Industry-leading LLaMA 7B base model fine-tune + Industry-leading Stanford open release democratization)。
まとめ
Alpaca = 2023年Taori et al. Stanford CRFM発表のopen instruction tuning democratization。Industry-leading Stanford Alpaca 52K instructions dataset + Industry-leading Self-Instruct method使用 + Industry-leading OpenAI text-davinci-003 teacher model + Industry-leading LLaMA 7B base model fine-tune + Industry-leading $600 total training cost democratization + Industry-leading Stanford open release democratization + Industry-leading Rohan Taori + Ishaan Gulrajani + Tianyi Zhang + Yann Dubois + Xuechen Li + Carlos Guestrin + Percy Liang + Tatsunori B. Hashimoto Stanford CRFM authors + Industry-leading "Alpaca: A Strong, Replicable Instruction-Following Model" Stanford CRFM blog 2023年3月13日 + Industry-leading Stanford Alpaca open democratization 2023 position確立。