What it is
AI developers and open-source infrastructure projects are competing to make advanced models easier to access, benchmark, serve and run locally.
Overview
Tencent launched Hy4 as a vision-capable API model with a plan to release weights later, while other labs added open agents and models. Thinking Machines released Inkling 2 Small, reported strong open-weight ARC-AGI-2 results and published its weights under Apache 2.0. New reporting adds llama.cpp and ggml v1.0.0, improving local execution of large mixture-of-experts models through caching and offloading.
How it could play out
Portfolio
Why it matters
The race affects developers, enterprises, researchers and buyers choosing between hosted APIs, open weights, local inference, privacy, cost and deployment flexibility.
Key details
• Inkling 2 Small has 594B total parameters and 17B active parameters. • Inkling 2 Small scored 81.7% on ARC-AGI-2 at $0.27 per task. • Tencent’s Hy4 supports a 1M token context window and vision. • llama.cpp v1.0.0 adds LRU caching, offloading, FP8 KV cache support and a redesigned web UI.
Tilt can alert you when its scenarios shift, or send you a recommendation when one crosses a threshold.
Stories in this saga
Portfolio
How it could play out
Saga, how it could play out
SagaOpen-weight AI model API and benchmark race
The new caching layer materially improves local and small-cluster inference for open-weight mixture-of-experts models. Developers, researchers, and some enterprises shift more experimentation away from API-only services toward local deployment, reinforcing the open-weights trend started by DeepSeek, Mistral, Kimi, and llama.cpp.
The cache improves some workloads but does not overcome memory, bandwidth, and quantization constraints for the largest open-weight MoE models. Adoption remains concentrated among researchers and hobbyists rather than mainstream enterprises.
Hyperscalers respond by packaging llama.cpp v1.0, FP8 support, and MoE caching into managed inference offerings. Instead of displacing cloud AI spending, the update becomes another optimization layer for hybrid and rented GPU deployments.
The prospect of powerful open-weight models running locally increases attention from regulators, copyright holders, and enterprise security teams. Near-term market effects are limited, but compliance requirements may shape how local MoE tools are distributed and audited.
Portfolio
Portfolio
