Magnitude Runs Open Models 2x Faster Than Ollama

The software is optimized for AI agents like Pi and Codex, employing a quantized KV cache to reduce memory consumption by approximately half. It also features shared prefix caches, allowing multiple agents to run concurrently without degrading system performance. Future updates will include attention pruning to accelerate prefill and expert streaming from RAM or disk to support models that exceed a GPU's memory capacity.