A subsequent update extended these Metal kernels to cover all 26 weight formats supported by the Mac GPU, including BF16 and MXFP4. This expansion allows matrix multiplies to run up to 4.4x faster on M3 Ultra hardware when a model verifies multiple tokens simultaneously. The changes were contributed by developer qvac and merged by project maintainer ggerganov.
Tuesday, October 20, 2026
Story
llama.cpp Speeds Mac Speculative Decoding 3.4x for M3 Ultra
AI10 days ago
New GPU optimizations merged into the llama.cpp library significantly increase the efficiency of token generation on Apple hardware. These updates allow speculative decoding to run at 110 tok/s on an M3 Ultra compared to 32.1 tok/s for plain decoding, a 3.4x increase in speed that reverses a previous performance lag where the speculative method was slower than standard execution.
Looking for more context or analysis?
Tilt chat provides reliable, tailored insights, alerts, and automations.