MetaMath Math Bootstrap (Yu 2023年)（メタマス）
2023年Yu et al. (NUS+SJTU)発表MetaMath・Industry-leading math reasoning LLM + Industry-leading bootstrap question augmentation + Industry-leading MetaMathQA 395K dataset + Industry-leading LLaMA-2 7B-70B fine-tuned。
概要
MetaMath Math Bootstrap は、2023年Yu et al. (NUS+SJTU)発表MetaMath・Industry-leading math reasoning LLM 2023年 + Industry-leading bootstrap question augmentation position確立。MetaMath specifications = Industry-leading math reasoning LLM (Industry-leading MetaMath 2023 + Industry-leading math reasoning + Industry-leading MetaMath flagship) + Industry-leading bootstrap question augmentation + Industry-leading MetaMathQA 395K dataset + Industry-leading LLaMA-2 7B-70B fine-tuned。
主な特徴・仕組み
- Authors: Industry-leading Longhui Yu + Weisen Jiang + Han Shi + Jincheng Yu + Zhengying Liu + Yu Zhang + James T. Kwok + Zhenguo Li + Adrian Weller + Weiyang Liu NUS + SJTU + Huawei
- Year: 2023年 (arXiv 2023年9月発表 + ICLR 2024)
- Paper: Industry-leading "MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models" arXiv 2309.12284
- Method Type: Industry-leading math reasoning LLM with bootstrap augmentation
- Question Augmentation: Industry-leading 4 ways (rephrasing/self-verification/FOBAR/Answer Substitution)
- Dataset: Industry-leading MetaMathQA 395K augmented dataset
- Base Model: Industry-leading LLaMA-2 7B + 13B + 70B fine-tuned
- GSM8K: Industry-leading GSM8K 82.3% pass@1 MetaMath-70B
- MATH: Industry-leading MATH 26.6% pass@1 MetaMath-70B
- Open Source: Industry-leading MetaMath GitHub meta-math/MetaMath
- HuggingFace: Industry-leading meta-math/MetaMath-70B-V1.0
- Industry-Leading: Industry-leading math reasoning LLM bootstrap NUS+SJTU 2023
スペック比較表
| LLM Math Reasoning (2023-2024) | Year | Org | Base Model | Industry Position |
|---|---|---|---|---|
| MetaMath | 2023 | Yu NUS+SJTU | LLaMA-2 7B-70B | Industry-leading bootstrap MetaMathQA 395K |
| WizardMath | 2023 | Microsoft+Peking | LLaMA-2 7B-70B | Industry-leading Reinforced Evol-Instruct |
| MAmmoTH | 2023 | Yue+Wang Waterloo | LLaMA + Code Llama 7B-70B | Industry-leading MathInstruct 260K |
| InternLM-Math | 2024 | Shanghai AI Lab | InternLM 7B-20B | Industry-leading InternLM-Math 8K context |
| DeepSeekMath | 2024 | DeepSeek AI | DeepSeek-Math 7B | Industry-leading DeepSeekMath GRPO |
具体例・対応製品
- MetaMath (2023年, Yu et al. NUS+SJTU): Industry-leading math reasoning LLM
- Industry-leading 4 augmentation rephrasing/self-verification/FOBAR/Answer Substitution: Industry-leading 4 augmentation methods
- Industry-leading MetaMathQA 395K augmented dataset: Industry-leading MetaMathQA 395K
- Industry-leading LLaMA-2 7B/13B/70B fine-tuned: Industry-leading LLaMA-2 7B/13B/70B
- Industry-leading GSM8K 82.3% + MATH 26.6% MetaMath-70B: Industry-leading GSM8K 82.3% + MATH 26.6%
- 競合 WizardMath + MAmmoTH + InternLM-Math + DeepSeekMath: Industry-leading math reasoning competitors
自作PCでの選び方・注意点
MetaMath Math Bootstrap は「Industry-leading math reasoning LLM + Industry-leading bootstrap question augmentation」「Industry-leading MetaMathQA 395K dataset + Industry-leading LLaMA-2 7B-70B fine-tuned」用途のIndustry-leading math reasoning 2023年Yu et al. NUS+SJTU発表product。Industry-leading bootstrap (Industry-leading bootstrap question augmentation + Industry-leading 4 augmentation methods rephrasing/self-verification/FOBAR/Answer Substitution + Industry-leading MetaMath bootstrap signature) で Industry-leading bootstrap + Industry-leading 4 augmentation + Industry-leading MetaMath signature。Industry-leading MetaMathQA 395K (Industry-leading MetaMathQA 395K augmented dataset + Industry-leading bootstrap dataset signature + Industry-leading large-scale augmented math QA) で Industry-leading MetaMathQA 395K + Industry-leading bootstrap dataset + Industry-leading large-scale augmented。Industry-leading LLaMA-2 (Industry-leading LLaMA-2 7B + 13B + 70B fine-tuned + Industry-leading MetaMath LLaMA-2 signature) で Industry-leading LLaMA-2 7B/13B/70B + Industry-leading MetaMath LLaMA-2 signature。Industry-leading GSM8K 82.3% + MATH 26.6% (Industry-leading GSM8K 82.3% pass@1 MetaMath-70B + Industry-leading MATH 26.6% pass@1 MetaMath-70B + Industry-leading MetaMath-70B performance) で Industry-leading GSM8K 82.3% + Industry-leading MATH 26.6% + Industry-leading 70B performance。Industry-leading NUS+SJTU (Industry-leading Longhui Yu + Weisen Jiang NUS + SJTU + Huawei authors + Industry-leading MetaMath GitHub meta-math/MetaMath + Industry-leading meta-math/MetaMath-70B-V1.0 HuggingFace) で Industry-leading Yu+Jiang NUS+SJTU + Industry-leading meta-math/MetaMath + Industry-leading MetaMath-70B-V1.0。但しIndustry-leading WizardMath + MAmmoTH + InternLM-Math + DeepSeekMath competition (Industry-leading WizardMath Microsoft+Peking Reinforced Evol 2023 + MAmmoTH Waterloo MathInstruct 260K 2023 + InternLM-Math Shanghai AI Lab 2024 + DeepSeekMath GRPO 2024 vs MetaMath NUS+SJTU bootstrap MetaMathQA 395K 2023 trade-off) で Industry-leading 4-math reasoning competitors vs MetaMath + Industry-leading bootstrap question augmentation + 4 augmentation methods + MetaMathQA 395K dataset + LLaMA-2 7B/13B/70B + GSM8K 82.3% + MATH 26.6% + NUS+SJTU+Huawei 2023 unique advantage adoption alignment必須。
関連用語との違い
- vs WizardMath (Microsoft 2023): MetaMathはbootstrap + 4 augmentation + MetaMathQA 395K・WizardMathはReinforced Evol-Instruct + Microsoft+Peking
- vs MAmmoTH (Waterloo 2023): MetaMathはbootstrap augmentation + GSM8K 82.3%・MAmmoTHはMathInstruct 260K + 14 datasets
- vs InternLM-Math/DeepSeekMath (2024): MetaMathはLLaMA-2 base 2023・InternLM-Math/DeepSeekMathは2024 newer base models
よくある質問（FAQ）
Q1: MetaMath vs WizardMath 違いは？ A: MetaMath (Industry-leading bootstrap question augmentation + 4 augmentation methods rephrasing/self-verification/FOBAR/Answer Substitution + MetaMathQA 395K augmented dataset + LLaMA-2 7B/13B/70B fine-tuned + GSM8K 82.3% pass@1 + MATH 26.6% MetaMath-70B + NUS+SJTU+Huawei 2023) vs WizardMath (Industry-leading Reinforced Evol-Instruct RLEIF + LLaMA-2 7B/13B/70B fine-tuned + GSM8K 81.6% + MATH 22.7% WizardMath-70B + Microsoft+Peking 2023)・Industry-leading bootstrap + 4 augmentation + MetaMathQA + 82.3% + NUS = MetaMath + Industry-leading RLEIF + 81.6% + 22.7% + Microsoft = WizardMath preference judgment。
Q2: Industry-leading bootstrap + MetaMathQA 395K value は？ A: Industry-leading bootstrap + 395K (Industry-leading bootstrap question augmentation + Industry-leading 4 augmentation methods rephrasing/self-verification/FOBAR/Answer Substitution + Industry-leading MetaMathQA 395K augmented dataset + Industry-leading bootstrap dataset signature)。
Q3: Industry-leading GSM8K 82.3% + MATH 26.6% value は？ A: Industry-leading GSM8K + MATH (Industry-leading GSM8K 82.3% pass@1 MetaMath-70B + Industry-leading MATH 26.6% pass@1 MetaMath-70B + Industry-leading MetaMath-70B performance + Industry-leading math reasoning evaluation)。
まとめ
MetaMath = 2023年Yu et al. NUS+SJTU+Huawei発表のmath reasoning LLM with bootstrap augmentation。Industry-leading bootstrap question augmentation + Industry-leading 4 augmentation methods rephrasing/self-verification/FOBAR/Answer Substitution + Industry-leading MetaMathQA 395K augmented dataset + Industry-leading LLaMA-2 7B + 13B + 70B fine-tuned + Industry-leading GSM8K 82.3% pass@1 MetaMath-70B + Industry-leading MATH 26.6% pass@1 MetaMath-70B + Industry-leading Longhui Yu + Weisen Jiang + James T. Kwok + Weiyang Liu NUS + SJTU + Huawei authors + Industry-leading MetaMath GitHub meta-math/MetaMath + Industry-leading meta-math/MetaMath-70B-V1.0 HuggingFace + Industry-leading math reasoning LLM bootstrap NUS+SJTU 2023 position確立。