The Mixture of Experts model has 535B parameters and is tracking within 0.3% of its loss forecast despite a 300x extrapolation, with the run now halfway complete. The project's scale increased from an original plan of 360B parameters after developer ravwojdyla used custom kernels to improve hardware efficiency and token throughput.
Wednesday, October 21, 2026
Story
Marin’s 535B Model Tracks loss within 0.3% in Largest Live Run
AI19 days ago
A new high-parameter neural network is currently executing a public training trajectory to test the predictability of AI scaling laws. Marin has spent 43 days training the model on 18T tokens, employing a publicly pre-registered scaling ladder to forecast the evaluation loss throughout the process.
Looking for more context or analysis?
Tilt chat provides reliable, tailored insights, alerts, and automations.