Speculative Decoding (2024年)（スペキュレイティブデコーディング）
2024年成熟Speculative Decoding・Industry-leading draft model + target model verification + Industry-leading 2-3x inference speedup + Industry-leading Leviathan+Kalman+Matias DeepMind Speculative Decoding inference optimization 2024。
概要
Speculative Decoding は2024年成熟・Industry-leading draft model + target model verification 2024年DeepMind Speculative Decoding inference optimization position確立。Industry-leading 2-3x inference speedup + Leviathan+Kalman+Matias DeepMind 2023 paper → 2024 mature + Medusa + EAGLE evolutions。
主な特徴・仕組み
- Industry-leading Yaniv Leviathan + Matan Kalman + Yossi Matias (Google DeepMind) 2023年11月 paper → 2024 mature
- Industry-leading Speculative Decoding inference optimization signature
- Industry-leading draft model + target model verification signature (small draft + large verify)
- Industry-leading 2-3x inference speedup signature (vs autoregressive)
- Industry-leading lossless quality signature (target verifies)
- Industry-leading vLLM + TGI + llama.cpp implementations
- Industry-leading Medusa multi-head extension signature
- Industry-leading EAGLE auto-regressive draft signature 2024
- Industry-leading 2023年11月 ICML 2023 paper "Fast Inference from Transformers via Speculative Decoding"
- Industry-leading DeepMind 2010 London + Google Brain merger 2023
まとめ
Speculative Decoding = 2024年成熟Google DeepMind発表のpremium inference optimization。Industry-leading draft model + target model verification small draft + large verify + 2-3x inference speedup vs autoregressive + lossless quality target verifies + vLLM + TGI + llama.cpp implementations + Medusa multi-head extension + EAGLE auto-regressive draft 2024 + 2023年11月 ICML 2023 paper "Fast Inference from Transformers via Speculative Decoding" + Yaniv Leviathan + Matan Kalman + Yossi Matias Google DeepMind 2023→2024 mature position確立。