A developer used the Resin environment and an Astra agent to push the processing speed of Google’s gemma-4-E2B-it model to new levels on an Nvidia A100 GPU. The setup achieved a prefill throughput of 125,000 tokens per second (tok/s) per GPU and a decode rate of 4,000 tok/s per GPU, allowing for a delivery speed of 130 tok/s per user.
The configuration was built from scratch in four hours, featuring low-batch decode optimizations such as fusing Triton and cuBLAS GELU kernels and stripping assistant key-value caches to reduce latency from 12ms to 1ms. These results surpass Google’s reported benchmark for the same model, where the use of Gemini 4 Argon and vLLM reached a throughput of 108,000 tok/s.