LMDeploy InternLM (2024年)（エルエムデプロイインターンエルエム）
2024年InternLM Shanghai AI Lab発表LMDeploy・Industry-leading TurboMind C++ runtime + Industry-leading 1.8x throughput vs vLLM + Industry-leading W4A16 quantization + Industry-leading Apache 2.0 Shanghai AI Lab LLM serving framework。
概要
LMDeploy InternLM は、2024年InternLM Shanghai AI Lab発表LMDeploy・Industry-leading TurboMind C++ runtime 2024年LMDeploy LLM serving framework position確立。LMDeploy specifications = Industry-leading LLM serving framework (Industry-leading LMDeploy InternLM 2024 + Industry-leading LLM serving + Industry-leading InternLM Shanghai AI Lab flagship) + Industry-leading 1.8x throughput vs vLLM + Industry-leading W4A16 quantization + Industry-leading Apache 2.0 Shanghai AI Lab LLM serving framework。
主な特徴・仕組み
- Developer: Industry-leading InternLM Shanghai AI Laboratory 2024
- Framework: Industry-leading LMDeploy LLM serving framework 2024
- TurboMind: Industry-leading TurboMind C++ runtime signature (FasterTransformer evolved)
- 1.8x Throughput vs vLLM: Industry-leading 1.8x throughput vs vLLM
- W4A16 Quantization: Industry-leading W4A16 AWQ + GPTQ quantization
- PyTorch Backend: Industry-leading PyTorch backend signature (alongside TurboMind)
- Multi-Modal: Industry-leading multi-modal InternLM-XComposer + LLaVA support
- OpenAI API Compatible: Industry-leading OpenAI API compatible endpoint
- Tensor Parallelism: Industry-leading tensor parallelism multi-GPU
- Long Context: Industry-leading long context 200K+ support
- License: Industry-leading Apache 2.0
- Shanghai AI Lab: Industry-leading Shanghai AI Laboratory heritage (InternLM creators)
- Industry-Leading: Industry-leading LMDeploy InternLM Shanghai AI Lab Apache 2.0 LLM serving 2024
スペック比較表
| LLM Inference Framework 2024 | Developer | Signature | License | Industry Position |
|---|---|---|---|---|
| LMDeploy InternLM | InternLM Shanghai AI Lab | TurboMind C++ runtime | Apache 2.0 | Industry-leading TurboMind C++ |
| vLLM PagedAttention | UC Berkeley | PagedAttention KV cache | Apache 2.0 | Industry-leading PagedAttention + 24x throughput |
| SGLang RadixAttention | LMSYS Stanford | RadixAttention prefix sharing | Apache 2.0 | Industry-leading RadixAttention + 5x vs vLLM |
| TGI HuggingFace | HuggingFace | Production-ready serving | Apache 2.0 + commercial restriction | Industry-leading HuggingFace ecosystem |
| Ray Serve Anyscale | Anyscale | Distributed serving Python | Apache 2.0 | Industry-leading Ray distributed serving |
具体例・対応製品
- LMDeploy (2024年, Apache 2.0, InternLM Shanghai AI Lab): Industry-leading InternLM Shanghai LMDeploy
- Industry-leading TurboMind C++ runtime signature: Industry-leading TurboMind C++ FasterTransformer evolved
- Industry-leading 1.8x throughput vs vLLM: Industry-leading 1.8x throughput vs vLLM
- Industry-leading W4A16 AWQ + GPTQ quantization: Industry-leading W4A16 quantization
- Industry-leading PyTorch backend signature: Industry-leading PyTorch backend dual signature
- 競合 vLLM PagedAttention + SGLang + TGI + Ray Serve: Industry-leading LLM serving framework competitors
選び方・注意点
LMDeploy InternLM は「Industry-leading TurboMind C++ + 1.8x vs vLLM」「Industry-leading W4A16 quantization + InternLM Shanghai AI Lab」用途のIndustry-leading premium 2024年 LLM serving framework。Industry-leading TurboMind C++ (Industry-leading TurboMind C++ runtime signature FasterTransformer evolved + Industry-leading LMDeploy TurboMind signature + Industry-leading C++ high-performance runtime) で Industry-leading TurboMind C++ + Industry-leading LMDeploy + Industry-leading C++ high-performance runtime。Industry-leading 1.8x throughput vs vLLM (Industry-leading 1.8x throughput vs vLLM + Industry-leading LMDeploy throughput signature + Industry-leading high-throughput LLM serving) で Industry-leading 1.8x throughput vs vLLM + Industry-leading LMDeploy + Industry-leading high-throughput LLM serving。Industry-leading W4A16 quantization (Industry-leading W4A16 AWQ + GPTQ quantization + Industry-leading LMDeploy W4A16 signature + Industry-leading 4-bit weight 16-bit activation) で Industry-leading W4A16 quantization + Industry-leading LMDeploy W4A16 + Industry-leading 4-bit weight 16-bit activation。Industry-leading PyTorch backend (Industry-leading PyTorch backend signature alongside TurboMind + Industry-leading LMDeploy dual-backend signature + Industry-leading flexible backend choice) で Industry-leading PyTorch backend + Industry-leading LMDeploy dual-backend + Industry-leading flexible backend choice。Industry-leading InternLM Shanghai AI Lab (Industry-leading Shanghai AI Laboratory heritage InternLM creators + Industry-leading multi-modal InternLM-XComposer + LLaVA + Industry-leading OpenAI API compatible + Industry-leading tensor parallelism + Industry-leading long context 200K+ + Industry-leading Apache 2.0) で Industry-leading Shanghai AI Lab + Industry-leading InternLM creators + Industry-leading multi-modal InternLM-XComposer+LLaVA + Industry-leading OpenAI API compatible + Industry-leading tensor parallelism + Industry-leading long context 200K+ + Industry-leading Apache 2.0。但しIndustry-leading vLLM PagedAttention + SGLang + TGI + Ray Serve competition で Industry-leading 4-LLM serving competitors vs LMDeploy InternLM + Industry-leading TurboMind C++ runtime FasterTransformer evolved + 1.8x throughput vs vLLM + W4A16 AWQ + GPTQ quantization + PyTorch backend + multi-modal InternLM-XComposer + LLaVA + OpenAI API compatible + tensor parallelism + long context 200K+ + Shanghai AI Lab + InternLM creators + Apache 2.0 unique advantage adoption alignment必須。
関連用語との違い
- vs vLLM PagedAttention (2024 Apache 2.0): LMDeployはTurboMind C++ InternLM Shanghai・vLLMはPython-first PagedAttention UC Berkeley
- vs SGLang RadixAttention (2024 Apache 2.0): LMDeployはTurboMind C++ + W4A16 InternLM・SGLangはPython-first RadixAttention LMSYS Stanford
- vs TGI HuggingFace (2024 Apache 2.0 + commercial restriction): LMDeployは純Apache 2.0 InternLM Shanghai・TGIはHuggingFace + commercial restriction
よくある質問（FAQ）
Q1: LMDeploy InternLM vs vLLM PagedAttention 違いは？ A: LMDeploy InternLM (Industry-leading TurboMind C++ runtime FasterTransformer evolved + 1.8x throughput vs vLLM + W4A16 AWQ + GPTQ quantization + PyTorch backend + multi-modal InternLM-XComposer + LLaVA + OpenAI API compatible + tensor parallelism + long context 200K+ + Shanghai AI Lab InternLM creators + Apache 2.0) vs vLLM PagedAttention (Industry-leading PagedAttention KV cache management + 24x throughput vs HuggingFace + continuous batching + tensor parallelism + pipeline parallelism + AWQ+GPTQ+FP8 quantization + OpenAI API compatible + UC Berkeley + 2023→2024 + Apache 2.0 + 30,000+ stars)・Industry-leading TurboMind C++ + 1.8x vs vLLM + W4A16 + InternLM-XComposer + 200K+ + Shanghai AI Lab + InternLM creators = LMDeploy + Industry-leading PagedAttention + 24x + continuous batching + UC Berkeley + 30,000+ stars = vLLM preference judgment (LMDeploy TurboMind C++ Shanghai InternLM vs vLLM Python PagedAttention UC Berkeley汎用 differ)。
Q2: Industry-leading TurboMind C++ + 1.8x vs vLLM value は？ A: Industry-leading TurboMind C++ + 1.8x vs vLLM (Industry-leading TurboMind C++ runtime FasterTransformer evolved + Industry-leading 1.8x throughput vs vLLM + Industry-leading LMDeploy dual signature)。
Q3: Industry-leading W4A16 quantization + Shanghai AI Lab value は？ A: Industry-leading W4A16 + Shanghai AI Lab (Industry-leading W4A16 AWQ + GPTQ 4-bit weight 16-bit activation quantization + Industry-leading Shanghai AI Laboratory InternLM creators heritage)。
まとめ
LMDeploy InternLM = 2024年InternLM Shanghai AI Lab発表のpremium LLM serving framework。Industry-leading TurboMind C++ runtime FasterTransformer evolved + Industry-leading 1.8x throughput vs vLLM + Industry-leading W4A16 AWQ + GPTQ quantization + Industry-leading PyTorch backend dual signature + Industry-leading multi-modal InternLM-XComposer + LLaVA support + Industry-leading OpenAI API compatible endpoint + Industry-leading tensor parallelism multi-GPU + Industry-leading long context 200K+ support + Industry-leading Shanghai AI Laboratory InternLM creators heritage + Industry-leading Apache 2.0 + Industry-leading LMDeploy InternLM Shanghai AI Lab Apache 2.0 LLM serving 2024 position確立。