StreamingLLM (2024年)（ストリーミングエルエルエム）
2024年成熟StreamingLLM・Industry-leading attention sink + sliding window + Industry-leading infinite streaming + Industry-leading 4M tokens demonstrated + Industry-leading Meta + MIT StreamingLLM infinite streaming LLM 2024。
概要
StreamingLLM は2024年成熟・Industry-leading attention sink + sliding window 2024年Meta + MIT StreamingLLM infinite streaming LLM position確立。Industry-leading 4M tokens demonstrated + Llama 2/MPT/Pythia/Falcon support + 22.2x speedup vs sliding-window re-computation + 2023年9月 paper → 2024 mature。
主な特徴・仕組み
- Industry-leading Guangxuan Xiao + Yuandong Tian + Beidi Chen + Song Han + Mike Lewis (MIT + Meta + CMU) 2023年9月 paper → 2024 mature
- Industry-leading StreamingLLM infinite streaming LLM signature
- Industry-leading attention sink signature (first 4 tokens absorb attention)
- Industry-leading sliding window signature (recent context window)
- Industry-leading infinite streaming deployment signature
- Industry-leading 4M tokens demonstrated signature (vs Llama 2 4K limit)
- Industry-leading 22.2x speedup vs sliding-window re-computation
- Industry-leading Llama 2 + MPT + Pythia + Falcon support
- Industry-leading no fine-tuning required signature
- Industry-leading MIT license signature
- Industry-leading 2023年9月 ICLR 2024 paper "Efficient Streaming Language Models with Attention Sinks"
まとめ
StreamingLLM = 2024年成熟Meta + MIT発表のpremium infinite streaming LLM。Industry-leading attention sink first 4 tokens absorb attention + sliding window recent context + infinite streaming deployment + 4M tokens demonstrated vs Llama 2 4K limit + 22.2x speedup vs sliding-window re-computation + Llama 2 + MPT + Pythia + Falcon support + no fine-tuning required + MIT license + Guangxuan Xiao + Yuandong Tian + Beidi Chen + Song Han + Mike Lewis MIT + Meta + CMU 2023年9月 ICLR 2024 paper position確立。