OLMoE Allen AI (2024年)（オールエムオーイー）

概要

OLMoE は2024年9月Allen AI発表・Industry-leading 7B total + 1.3B active fully open-source MoE 2024年Allen AI OLMoE Apache 2.0 fully open-source MoE LLM position確立。Industry-leading 64-expert MoE + training data + recipes + intermediate checkpoints open + 5T tokens + AI2 open research pioneer。

主な特徴・仕組み

• Industry-leading Allen Institute for AI (AI2, Founded 2014 Seattle WA by Paul Allen Microsoft co-founder)
• Industry-leading OLMoE fully open-source MoE LLM 2024年9月 release
• Industry-leading 7B total + 1.3B active MoE signature
• Industry-leading 64-expert MoE signature (8 active per token)
• Industry-leading 5T tokens trained signature
• Industry-leading training data fully released signature
• Industry-leading training recipes + intermediate checkpoints open signature
• Industry-leading fully reproducible research signature
• Industry-leading Apache 2.0 open-source license signature
• Industry-leading HuggingFace native
• Industry-leading Beats DeepSeek-MoE-16B + Qwen1.5-MoE-2.7B-A14B signature
• Industry-leading Allen AI 2014 Seattle Paul Allen 10 years

まとめ

OLMoE = 2024年9月Allen AI発表のpremium Apache 2.0 fully open-source MoE LLM。Industry-leading 7B total + 1.3B active MoE + 64-expert MoE 8 active per token + 5T tokens trained + training data fully released + training recipes + intermediate checkpoints open + fully reproducible research + Beats DeepSeek-MoE-16B + Qwen1.5-MoE-2.7B-A14B + Apache 2.0 + HuggingFace + Allen AI AI2 2014 Seattle WA Paul Allen Microsoft co-founder 10 years position確立。