The engine achieves generation speeds of 230 to 300 tokens per second and utilizes a persistent block KV cache with 100 PiB of SSD capacity. It employs "SWA Bounded Replay" to reconstruct attention states from the latest 128 tokens, reducing total prefill computation by nearly half. However, DeepSeek has not yet open-sourced the KV-cache management and load-balancing components because they remain tightly coupled with internal infrastructure.
Tuesday, October 20, 2026
Story
DeepSeek Open-Sources Production Engine Powering V4.1 Flash
AI15 hours ago
The company's in-house inference framework, DeepInfer, processes 155 trillion tokens per day from a single cluster while maintaining a 96% input cache hit rate. DeepSeek open-sourced the engine and the accompanying deepseek-recipe library to allow the community to replicate the production stack used for its V4.1 Flash model. The system supports Nvidia and Huawei Ascend hardware, utilizing separate stages for vision encoding, prefill, and decode to independently scale resources based on workload.
Looking for more context or analysis?
Tilt chat provides reliable, tailored insights, alerts, and automations.