Reinforcement learning has become one of the most important, and most expensive, stages of modern model building. Post-training with verifiable rewards is how a lot of reasoning and agent behavior gets taught. It is also where the compute bill quietly explodes, for a structural reason worth spelling out.
The multiplier is built in
Supervised training does roughly one unit of work per example. RL does not. For every single learning update, the model generates many rollouts, plays them out, scores them, and only then takes a step. The number of rollouts per update is often in the hundreds once you count across a batch of prompts. The generation phase, not the update, dominates the cost.
That structure is a multiplier, and multipliers are indifferent to which direction you push them. Any inefficiency in a single rollout is not paid once. It is paid on every rollout, on every update, across the entire run. The waste compounds.
Which means the savings compound too
The same math runs in reverse. A saving on the cost of a single rollout is also paid out on every rollout, every update, across the entire run. In a stage whose whole cost is a large number multiplied by a per-run figure, bringing down the per-run figure is the highest-leverage move available.
This is why RL and agent post-training are near the center of how we think about where efficiency matters most. The teams building reasoning models and training agents are running the loops where a small per-run improvement turns into a large bill or a large saving. The compounding is already there in the structure. The only question is which way it points.