Llama 3.1 405B Meta (2024年)（ラマ3.1 405B）

概要

Llama 3.1 405B Meta は2024年7月Meta発表・Industry-leading 405B parameters open-weight frontier 2024年Meta Llama 3.1 405B Llama Community License open-weight frontier LLM position確立。Industry-leading 15.6T tokens trained + 128K context + 16384 H100 16K GPUs used + GPT-4o + Claude 3.5 Sonnet competitive。

主な特徴・仕組み

• Industry-leading Meta AI (Founded 2013 Menlo Park FAIR → Meta 2021) Llama 3.1 release 2024年7月
• Industry-leading Llama 3.1 405B open-weight frontier LLM signature
• Industry-leading 405B parameters open-weight signature (largest open-weight 2024)
• Industry-leading 15.6T tokens trained signature
• Industry-leading 128K context window signature
• Industry-leading 16384 H100 16K GPUs used signature ($1.5B+ training cost rumored)
• Industry-leading GPT-4o + Claude 3.5 Sonnet competitive signature
• Industry-leading MMLU 88.6 + HumanEval 89.0 + MATH 73.8
• Industry-leading 8 languages multilingual
• Industry-leading FP8 quantized variant signature
• Industry-leading Llama Community License signature (commercial use OK <700M MAU)
• Industry-leading Meta AI 2013 Menlo Park FAIR 11 years

まとめ

Llama 3.1 405B Meta = 2024年7月Meta発表のpremium open-weight frontier LLM。Industry-leading 405B parameters open-weight largest open-weight 2024 + 15.6T tokens trained + 128K context window + 16384 H100 16K GPUs used $1.5B+ training cost rumored + GPT-4o + Claude 3.5 Sonnet competitive + MMLU 88.6 + HumanEval 89.0 + MATH 73.8 + 8 languages multilingual + FP8 quantized variant + Llama Community License commercial use OK <700M MAU + Meta AI 2013 Menlo Park FAIR 11 years position確立。