The update addresses a bottleneck in forward and backward passes that occurs when agentic training rollouts exceed 100k tokens and batch sizes are large. The feature has been tested on Mixture of Experts and VLM models, with the developers noting it adds approximately 1% overhead in scenarios where no shared tokens can be packed.
Tuesday, October 20, 2026
Story
TRL Reduces Agentic RL Trainer Compute 26x via Tree Packing
AI6 days ago
The AsyncGRPO implementation in TRL now forwards each unique token once per batch instead of repeating shared rollout history for every sample. This tree packing mechanism allow trainers to reach the same reward on agentic reinforcement learning with 26x less compute on MiMo Envs. The current version uses a single FA3 varlen call, replacing an earlier iteration based on FlexAttention.
Looking for more context or analysis?
Tilt chat provides reliable, tailored insights, alerts, and automations.