Developed in collaboration with Nvidia, the method caches State Space Model (SSM) inputs instead of the state to separate output computation from state updates during decoding. This approach gives SSMs access to inference optimizations typically reserved for Transformers, such as quantization and speculative decoding. The researchers stated the performance gains result from an IO-aware memory strategy similar to the logic used in Mamba scan, paged KV, and FlashAttention.
Tuesday, October 20, 2026
Story
ReplaySSM Speeds GLM and Kimi Inference Up to 2.1x in Production
AI8 days ago
A new computational method for State Space Models is now live across various AI models and deployment frameworks. The ReplaySSM technique is active in GLM-5.3-Flash and Kimi-K3, and has been integrated into vLLM, TensorRT-LLM, SGLang, and FlashInfer. The update increases the speed of standard decoding by up to 1.47x and speculative decoding by up to 2.1x, while enabling a low-precision state that prevents rounding errors from propagating through recurrence.
Looking for more context or analysis?
Tilt chat provides reliable, tailored insights, alerts, and automations.