The model has immediate support from the vLLM and SGLang inference frameworks for serving on private infrastructure. By applying MXFP4 and block-FP8 quantization, SGLang reduces the weights of the model from 534GB to 292GB, enabling the system to fit on 4 Hopper GPUs.
