SGLang

SGLang is an open-source inference framework for large language models, vision-language models, and diffusion models, optimized for agent tasks, large-scale serving, and RL rollouts. It aims for low-latency, high-throughput serving from a single GPU to distributed clusters. You install it with pip or Docker, start a server with a single command, and call it through an OpenAI-compatible endpoint. It is operated under LMSYS, a non-profit open-source organization, and states that it is used on more than 400,000 GPUs worldwide. The pricing page on the official site returned a 404 (not found).

• Prefill/decode disaggregation, speculative decoding, various parallelism methods, zero-overhead scheduler
• RadixAttention and prefix caching
• OpenAI-compatible endpoint
• Broad hardware support, including NVIDIA and AMD GPUs, TPU, Ascend NPU, Intel, and Apple Silicon
• Supports a variety of open models such as DeepSeek, Qwen, and Llama
• Built-in SGLang Diffusion (image and video generation)
• Beam search support

Open source (free)

Apache 2.0 오픈소스 추론 프레임워크(pip·Docker로 직접 설치), 가격 페이지는 404로 유료 요금제 없음

공식 페이지 확인 2026-10-14(산출 규칙 2026-10-15)

• One of 43 AI tools in the Inference & Compute field.
• Connected to 6 other tools — #6 in its field.