Open source inference frameworks SGLang and vLLM integrated DeepSeek’s newly released kernels and optimizations to match or exceed the performance of DeepSeek’s internal DeepInfer engine on the 552B parameter DeepSeek-V4.1-Flash model. SGLang reached up to 1.04 times DeepInfer’s prefill throughput on four Nvidia GB300 or B300 GPUs and up to 7.4% higher decode throughput across 64 GB300 chips after adding Mega kernels, SWA bounded replay and Engram optimizations. Meanwhile, vLLM matched DeepInfer’s decode curve across batch sizes 8 to 256 using half the expert parallelism (DEP32 versus EP64DP64) and hit 954,000 tokens per second per GPU on the SemiAnalysis AgentX benchmark across a GB300 NVL72 rack, 7.9 times its single node result.

Both updates supply the production serving infrastructure that DeepSeek omitted when it open sourced DeepInfer under an Apache 2.0 license on Oct 20. SemiAnalysis and the SGLang team noted that DeepInfer’s repository lacks a persistent KV cache system, prefill and decode disaggregation, and load balancing across replicas, which SGLang provides alongside an end to end reproduction guide through its RadixAttention, HiCache and Linker components.