SGLang (Zheng 2023年)（エスジーラング）
2023年Lianmin Zheng et al. (UC Berkeley)発表SGLang・Industry-leading RadixAttention KV cache reuse + Industry-leading structured generation frontend + Industry-leading SGLang structured complex prompts + Industry-leading SGLang LLM serving framework。
概要
SGLang は、2023年Lianmin Zheng et al. (UC Berkeley)発表SGLang・Industry-leading RadixAttention KV cache reuse 2023年 + Industry-leading SGLang LLM serving framework position確立。SGLang specifications = Industry-leading RadixAttention KV cache reuse (Industry-leading SGLang Zheng 2023 + Industry-leading RadixAttention + Industry-leading SGLang flagship) + Industry-leading structured generation frontend + Industry-leading SGLang structured complex prompts + Industry-leading SGLang LLM serving framework。
主な特徴・仕組み
- Authors: Industry-leading Lianmin Zheng + Liangsheng Yin + Zhiqiang Xie + Chuyue Sun + Jeff Huang + Cody Hao Yu + Shiyi Cao + Christos Kozyrakis + Ion Stoica + Joseph E. Gonzalez + Clark Barrett + Ying Sheng (UC Berkeley + Stanford + UCLA + Shanghai Jiao Tong + Carnegie Mellon)
- Framework: Industry-leading SGLang LLM serving framework 2023
- Year: 2023年 (arXiv 2023年12月 + active development 2024年-)
- Paper: Industry-leading "SGLang: Efficient Execution of Structured Language Model Programs" arXiv 2312.07104 NeurIPS 2024
- RadixAttention: Industry-leading RadixAttention KV cache reuse signature (radix tree)
- Structured Generation: Industry-leading structured generation frontend DSL signature
- Complex Prompts: Industry-leading complex prompts + multi-turn + parallelism target
- 5x Speedup: Industry-leading up to 5x speedup vs vLLM for complex tasks
- DSL Frontend: Industry-leading Python DSL frontend signature
- Co-design: Industry-leading frontend DSL + backend RadixAttention co-design
- Industry Adoption: Industry-leading widespread Anthropic/xAI/Tencent/AMD adoption
- NeurIPS 2024: Industry-leading NeurIPS 2024 acceptance
- Industry-Leading: Industry-leading UC Berkeley SGLang RadixAttention + DSL frontend LLM serving 2023
スペック比較表
| LLM Inference Framework (2023-2024) | Year | Org | Method | Industry Position |
|---|---|---|---|---|
| SGLang | 2023 | Zheng UC Berkeley | RadixAttention + frontend | Industry-leading SGLang complex prompt |
| vLLM PagedAttention | 2023 | Kwon UC Berkeley | PagedAttention KV cache | Industry-leading vLLM high-throughput |
| TensorRT-LLM NVIDIA H100 | 2024 | NVIDIA | TensorRT GPU optimization | Industry-leading NVIDIA H100 inference |
| Ollama | 2023 | Ollama | Local LLM CLI/server | Industry-leading Ollama local |
| llama.cpp | 2023 | Georgi Gerganov | C/C++ CPU + GGUF | Industry-leading llama.cpp CPU+GGUF |
具体例・対応製品
- SGLang (2023年, Zheng UC Berkeley): Industry-leading SGLang RadixAttention LLM serving
- Industry-leading RadixAttention KV cache reuse signature (radix tree): Industry-leading RadixAttention radix tree
- Industry-leading structured generation frontend DSL signature: Industry-leading structured generation DSL
- Industry-leading up to 5x speedup vs vLLM for complex tasks: Industry-leading 5x speedup vs vLLM
- Industry-leading widespread Anthropic/xAI/Tencent/AMD adoption: Industry-leading widespread enterprise adoption
- 競合 vLLM + TensorRT-LLM + Ollama + llama.cpp: Industry-leading inference framework competitors
選び方・注意点
SGLang は「Industry-leading RadixAttention + structured generation frontend」「Industry-leading 5x speedup vs vLLM + Zheng UC Berkeley」用途のIndustry-leading 2023年 UC Berkeley product。Industry-leading RadixAttention (Industry-leading RadixAttention KV cache reuse signature radix tree + Industry-leading SGLang RadixAttention signature + Industry-leading radix tree KV cache management) で Industry-leading RadixAttention + Industry-leading SGLang + Industry-leading radix tree KV cache。Industry-leading structured generation (Industry-leading structured generation frontend DSL signature + Industry-leading SGLang structured generation signature + Industry-leading Python DSL frontend) で Industry-leading structured generation + Industry-leading SGLang + Industry-leading Python DSL frontend。Industry-leading complex prompts (Industry-leading complex prompts + multi-turn + parallelism target + Industry-leading SGLang complex prompts signature + Industry-leading complex task optimization) で Industry-leading complex prompts + Industry-leading SGLang + Industry-leading complex task optimization。Industry-leading 5x speedup (Industry-leading up to 5x speedup vs vLLM for complex tasks + Industry-leading SGLang speedup signature + Industry-leading complex task speedup) で Industry-leading 5x speedup + Industry-leading SGLang + Industry-leading complex task speedup。Industry-leading co-design (Industry-leading frontend DSL + backend RadixAttention co-design + Industry-leading SGLang co-design signature + Industry-leading frontend-backend co-design innovation + Industry-leading Lianmin Zheng + Liangsheng Yin + Ying Sheng + Ion Stoica + Joseph Gonzalez UC Berkeley + Stanford + UCLA + Shanghai Jiao Tong + Carnegie Mellon + Industry-leading arXiv 2312.07104 NeurIPS 2024 + Industry-leading 2023年12月 + Industry-leading widespread Anthropic/xAI/Tencent/AMD adoption) で Industry-leading co-design + Industry-leading SGLang + Industry-leading frontend-backend co-design + Industry-leading Zheng+Yin+Sheng+Stoica+Gonzalez UC Berkeley+Stanford+UCLA + Industry-leading NeurIPS 2024 + Industry-leading 2312.07104 + Industry-leading 2023年12月 + Industry-leading Anthropic/xAI/Tencent/AMD widespread。但しIndustry-leading vLLM + TensorRT-LLM + Ollama + llama.cpp competition で Industry-leading 4-inference framework competitors vs SGLang + Industry-leading RadixAttention KV cache reuse radix tree + structured generation frontend DSL + complex prompts + multi-turn + parallelism target + 5x speedup vs vLLM for complex tasks + Python DSL frontend + frontend DSL + backend RadixAttention co-design + widespread Anthropic/xAI/Tencent/AMD adoption + Lianmin Zheng + Liangsheng Yin + Ying Sheng + Ion Stoica + Joseph Gonzalez UC Berkeley + Stanford + UCLA + Shanghai Jiao Tong + Carnegie Mellon + arXiv 2312.07104 NeurIPS 2024 + 2023年12月 unique advantage adoption alignment必須。
関連用語との違い
- vs vLLM PagedAttention (Kwon UC Berkeley 2023): SGLangはRadixAttention + structured generation DSL 2023年12月・vLLMはPagedAttention + high-throughput continuous batching 2023年6月
- vs TensorRT-LLM (NVIDIA): SGLangはopen-source Apache 2.0 + frontend DSL UC Berkeley・TensorRT-LLMはNVIDIA proprietary GPU optimization
- vs Ollama (2023): SGLangはenterprise serving RadixAttention + DSL・Ollamaはlocal LLM CLI/server end-user
よくある質問（FAQ）
Q1: SGLang vs vLLM PagedAttention 違いは？ A: SGLang (Industry-leading RadixAttention KV cache reuse radix tree + structured generation frontend DSL + complex prompts + multi-turn + parallelism + 5x speedup vs vLLM + Python DSL frontend + frontend DSL + backend co-design + Anthropic/xAI/Tencent/AMD widespread + Lianmin Zheng + Liangsheng Yin + Ying Sheng + Ion Stoica + Joseph Gonzalez UC Berkeley + Stanford + UCLA + Shanghai Jiao Tong + Carnegie Mellon + arXiv 2312.07104 NeurIPS 2024 + 2023年12月) vs vLLM PagedAttention (Industry-leading PagedAttention KV cache management + virtual memory-like paging + 14-24x throughput vs HuggingFace + continuous batching + Apache 2.0 + SOSP 2023 best paper + Kwon + Li + Zhuang + Sheng + Zheng + Gonzalez + Zhang + Stoica UC Berkeley + 2309.06180 + 2023年6月)・Industry-leading RadixAttention + structured generation DSL + complex prompts + 5x speedup + co-design + Anthropic/xAI/Tencent/AMD + Zheng + 2023年12月 = SGLang + Industry-leading PagedAttention + virtual memory-like + 14-24x throughput + continuous batching + SOSP 2023 best paper + Kwon + 2023年6月 = vLLM preference judgment (同UC Berkeley Stoica lab だが RadixAttention 2023年12月 complex prompts vs PagedAttention 2023年6月 high-throughput differ)。
Q2: Industry-leading RadixAttention + structured generation DSL value は？ A: Industry-leading RadixAttention + structured generation DSL (Industry-leading RadixAttention KV cache reuse radix tree signature + Industry-leading structured generation frontend DSL signature + Industry-leading SGLang dual signature)。
Q3: Industry-leading 5x speedup + Anthropic/xAI widespread value は？ A: Industry-leading 5x speedup + Anthropic/xAI widespread (Industry-leading up to 5x speedup vs vLLM for complex tasks + Industry-leading widespread Anthropic/xAI/Tencent/AMD adoption + Industry-leading SGLang enterprise widespread)。
まとめ
SGLang = 2023年Zheng et al. UC Berkeley + Stanford + UCLA + Shanghai Jiao Tong + Carnegie Mellon発表のLLM serving framework structured complex prompts。Industry-leading RadixAttention KV cache reuse radix tree + Industry-leading structured generation frontend DSL + Industry-leading complex prompts + multi-turn + parallelism target + Industry-leading up to 5x speedup vs vLLM for complex tasks + Industry-leading Python DSL frontend + Industry-leading frontend DSL + backend RadixAttention co-design + Industry-leading widespread Anthropic/xAI/Tencent/AMD adoption + Industry-leading Lianmin Zheng + Liangsheng Yin + Zhiqiang Xie + Chuyue Sun + Cody Hao Yu + Shiyi Cao + Ion Stoica + Joseph E. Gonzalez + Ying Sheng UC Berkeley + Stanford + UCLA + Shanghai Jiao Tong + Carnegie Mellon authors + Industry-leading "SGLang: Efficient Execution of Structured Language Model Programs" arXiv 2312.07104 NeurIPS 2024 + Industry-leading 2023年12月 + Industry-leading UC Berkeley SGLang RadixAttention + DSL frontend LLM serving 2023 position確立。