The biggest AI labs have quietly revolutionized how they train frontier models, moving from human preference learning to outcome-based reinforcement learning that rewards models for getting verifiable results.

A Hugging Face analysis of public reports from major labs shows three clear patterns emerging in 2026. Every frontier lab now uses reinforcement learning. Most have moved beyond the original GRPO algorithm with custom modifications. And the newest systems train models in interactive environments rather than on single responses.

From preferences to execution

The shift began with OpenAI's o1 model in September 2024, which used "large-scale reinforcement learning" to teach reasoning through chain-of-thought processes. Performance "consistently improves with more reinforcement learning and with more time spent thinking," OpenAI said.

Two months later, AI2's Tulu 3 coined the term "Reinforcement Learning with Verifiable Rewards" (RLVR). Instead of training on human preferences, models now learn from executable checks — running code tests, solving math problems with numerical answers, or other verifiable outcomes.

DeepSeek-R1 demonstrated this approach in January 2025, showing that "reasoning abilities of LLMs can be incentivized through pure reinforcement learning." The model learned to solve problems by spending more time thinking, trained using the GRPO algorithm.

Meta's Llama 4 made online RL central to its pipeline, alternating between training and using the model to filter prompts by difficulty.

Executable rewards everywhere

Qwen3 describes training on "query-verifier pairs" where every prompt includes its own verification check. Xiaomi's MiMo-7B curated 130,000 verifiable math and programming problems, adding difficulty-sensitive code rewards to provide richer learning signals than simple pass-fail outcomes.

Mistral AI's Magistral limits itself to "problems with verifiable solutions" including mathematical problems with numerical answers and code problems with associated tests. Google's Gemma 3 uses "code execution feedback" and "ground-truth rewards for solving math problems."

Moonshot AI's Kimi K2 anchors even its model-based critics to verifiable rewards, using "on-policy rollouts generated from verifiable-reward prompts" to train evaluation models.

GLM-5 combines multiple approaches in a "hybrid reward system" integrating rule-based functions, outcome reward models, and generative reward models.

The pattern represents a fundamental shift from learning human preferences to learning from objective, executable outcomes — a change that appears to be driving the reasoning capabilities emerging in 2026's frontier models.