vLLM PagedAttention (Kwon 2023年)（ブイエルエルエムページドアテンション）
2023年Woosuk Kwon et al. (UC Berkeley)発表vLLM・Industry-leading PagedAttention KV cache management + Industry-leading LLM inference high-throughput serving + Industry-leading vLLM open-source framework + Industry-leading SOSP 2023 best paper。
概要
vLLM PagedAttention は、2023年Woosuk Kwon et al. (UC Berkeley)発表vLLM・Industry-leading PagedAttention KV cache management 2023年 + Industry-leading SOSP 2023 best paper position確立。vLLM specifications = Industry-leading PagedAttention KV cache management (Industry-leading vLLM Kwon 2023 + Industry-leading PagedAttention + Industry-leading vLLM flagship) + Industry-leading LLM inference high-throughput serving + Industry-leading vLLM open-source framework + Industry-leading SOSP 2023 best paper。
主な特徴・仕組み
- Authors: Industry-leading Woosuk Kwon + Zhuohan Li + Siyuan Zhuang + Ying Sheng + Lianmin Zheng + Cody Hao Yu + Joseph Gonzalez + Hao Zhang + Ion Stoica (UC Berkeley)
- Framework: Industry-leading vLLM LLM inference framework 2023
- Year: 2023年 (vLLM release 2023年6月 + SOSP 2023 paper)
- Paper: Industry-leading "Efficient Memory Management for Large Language Model Serving with PagedAttention" SOSP 2023 arXiv 2309.06180
- PagedAttention: Industry-leading PagedAttention KV cache management signature
- Virtual Memory-like: Industry-leading virtual memory-like paging for LLM KV cache
- High-Throughput: Industry-leading 14-24x throughput vs HuggingFace Transformers
- Continuous Batching: Industry-leading continuous batching dynamic batching
- vLLM Open-Source: Industry-leading vLLM open-source Apache 2.0 framework
- SOSP 2023 Best Paper: Industry-leading SOSP 2023 best paper award
- Industry Adoption: Industry-leading widespread enterprise LLM serving deployment
- Industry-Leading: Industry-leading UC Berkeley vLLM PagedAttention SOSP 2023 best paper LLM serving framework
スペック比較表
| LLM Inference Framework (2023-2024) | Year | Org | Method | Industry Position |
|---|---|---|---|---|
| vLLM PagedAttention | 2023 | Kwon UC Berkeley | PagedAttention KV cache | Industry-leading vLLM high-throughput |
| SGLang | 2023 | Zheng UC Berkeley | RadixAttention + frontend | Industry-leading SGLang complex prompt |
| TensorRT-LLM NVIDIA H100 | 2024 | NVIDIA | TensorRT GPU optimization | Industry-leading NVIDIA H100 inference |
| Ollama | 2023 | Ollama | Local LLM CLI/server | Industry-leading Ollama local |
| llama.cpp | 2023 | Georgi Gerganov | C/C++ CPU + GGUF | Industry-leading llama.cpp CPU+GGUF |
具体例・対応製品
- vLLM PagedAttention (2023年, Kwon UC Berkeley): Industry-leading vLLM high-throughput serving
- Industry-leading PagedAttention KV cache management signature: Industry-leading PagedAttention KV cache
- Industry-leading virtual memory-like paging for LLM KV cache: Industry-leading virtual memory-like paging
- Industry-leading 14-24x throughput vs HuggingFace Transformers: Industry-leading 14-24x throughput vs HF
- Industry-leading SOSP 2023 best paper award: Industry-leading SOSP 2023 best paper
- 競合 SGLang + TensorRT-LLM + Ollama + llama.cpp: Industry-leading inference framework competitors
選び方・注意点
vLLM PagedAttention は「Industry-leading PagedAttention KV cache + 14-24x throughput」「Industry-leading SOSP 2023 best paper + Kwon UC Berkeley」用途のIndustry-leading 2023年 UC Berkeley product。Industry-leading PagedAttention (Industry-leading PagedAttention KV cache management signature + Industry-leading vLLM PagedAttention signature + Industry-leading virtual memory-like paging) で Industry-leading PagedAttention + Industry-leading vLLM + Industry-leading virtual memory-like paging。Industry-leading virtual memory-like (Industry-leading virtual memory-like paging for LLM KV cache + Industry-leading vLLM virtual memory signature + Industry-leading OS-inspired KV cache management) で Industry-leading virtual memory-like + Industry-leading vLLM + Industry-leading OS-inspired KV cache management。Industry-leading 14-24x throughput (Industry-leading 14-24x throughput vs HuggingFace Transformers + Industry-leading vLLM throughput signature + Industry-leading high-throughput LLM serving) で Industry-leading 14-24x throughput + Industry-leading vLLM + Industry-leading high-throughput LLM serving。Industry-leading continuous batching (Industry-leading continuous batching dynamic batching + Industry-leading vLLM continuous batching signature + Industry-leading dynamic batching innovation) で Industry-leading continuous batching + Industry-leading vLLM + Industry-leading dynamic batching innovation。Industry-leading UC Berkeley (Industry-leading Woosuk Kwon + Zhuohan Li + Siyuan Zhuang + Ying Sheng + Lianmin Zheng + Joseph Gonzalez + Hao Zhang + Ion Stoica UC Berkeley + Industry-leading "Efficient Memory Management for Large Language Model Serving with PagedAttention" SOSP 2023 arXiv 2309.06180 + Industry-leading 2023年6月 vLLM release + Industry-leading SOSP 2023 best paper award + Industry-leading widespread enterprise LLM serving deployment + Industry-leading vLLM open-source Apache 2.0) で Industry-leading UC Berkeley + Industry-leading Kwon+Li+Zhuang+Sheng+Zheng+Gonzalez+Zhang+Stoica + Industry-leading SOSP 2023 + Industry-leading 2309.06180 + Industry-leading 2023年6月 + Industry-leading SOSP 2023 best paper + Industry-leading widespread enterprise + Industry-leading Apache 2.0。但しIndustry-leading SGLang + TensorRT-LLM + Ollama + llama.cpp competition で Industry-leading 4-inference framework competitors vs vLLM PagedAttention + Industry-leading PagedAttention KV cache management + LLM inference high-throughput serving + virtual memory-like paging + 14-24x throughput vs HuggingFace Transformers + continuous batching dynamic batching + vLLM open-source Apache 2.0 + SOSP 2023 best paper award + widespread enterprise LLM serving + Woosuk Kwon + Zhuohan Li + Siyuan Zhuang + Ying Sheng + Lianmin Zheng + Joseph Gonzalez + Hao Zhang + Ion Stoica UC Berkeley + arXiv 2309.06180 SOSP 2023 + 2023年6月 unique advantage adoption alignment必須。
関連用語との違い
- vs SGLang (Zheng UC Berkeley 2023): vLLMはPagedAttention KV cache + 14-24x throughput・SGLangはRadixAttention + structured generation frontend
- vs TensorRT-LLM (NVIDIA 2023+): vLLMはopen-source Apache 2.0 vendor-agnostic・TensorRT-LLMはNVIDIA proprietary GPU optimized
- vs Ollama (2023): vLLMはhigh-throughput enterprise serving・Ollamaはlocal LLM CLI/server end-user
よくある質問（FAQ）
Q1: vLLM PagedAttention vs SGLang 違いは？ A: vLLM PagedAttention (Industry-leading PagedAttention KV cache management + virtual memory-like paging + 14-24x throughput vs HuggingFace + continuous batching + vLLM Apache 2.0 + SOSP 2023 best paper + Kwon + Li + Zhuang + Sheng + Zheng + Gonzalez + Zhang + Stoica UC Berkeley + 2309.06180 + 2023年6月) vs SGLang (Industry-leading RadixAttention KV cache reuse + structured generation frontend + complex prompts + Lianmin Zheng + Liangsheng Yin et al. UC Berkeley + arXiv 2312.07104 + 2023年12月)・Industry-leading PagedAttention + virtual memory-like + 14-24x throughput + continuous batching + SOSP 2023 best paper + UC Berkeley + 2023年6月 = vLLM + Industry-leading RadixAttention + structured generation frontend + complex prompts + UC Berkeley + 2023年12月 = SGLang preference judgment (同UC Berkeley Stoica lab だが PagedAttention 2023年6月 SOSP best vs RadixAttention 2023年12月 complex prompts differ)。
Q2: Industry-leading PagedAttention + 14-24x throughput value は？ A: Industry-leading PagedAttention + 14-24x throughput (Industry-leading PagedAttention KV cache management signature + Industry-leading 14-24x throughput vs HuggingFace Transformers + Industry-leading vLLM dual signature)。
Q3: Industry-leading SOSP 2023 best paper + vLLM Apache 2.0 value は？ A: Industry-leading SOSP 2023 best paper + Apache 2.0 (Industry-leading SOSP 2023 best paper award + Industry-leading vLLM open-source Apache 2.0 framework + Industry-leading Kwon UC Berkeley dual signature)。
まとめ
vLLM PagedAttention = 2023年Kwon et al. UC Berkeley発表のLLM inference high-throughput serving framework SOSP 2023 best paper。Industry-leading PagedAttention KV cache management + Industry-leading virtual memory-like paging for LLM KV cache + Industry-leading 14-24x throughput vs HuggingFace Transformers + Industry-leading continuous batching dynamic batching + Industry-leading vLLM open-source Apache 2.0 framework + Industry-leading SOSP 2023 best paper award + Industry-leading widespread enterprise LLM serving deployment + Industry-leading Woosuk Kwon + Zhuohan Li + Siyuan Zhuang + Ying Sheng + Lianmin Zheng + Cody Hao Yu + Joseph Gonzalez + Hao Zhang + Ion Stoica UC Berkeley authors + Industry-leading "Efficient Memory Management for Large Language Model Serving with PagedAttention" SOSP 2023 arXiv 2309.06180 + Industry-leading 2023年6月 vLLM release + Industry-leading UC Berkeley vLLM PagedAttention SOSP 2023 best paper LLM serving framework position確立。