Qwen2.5-Math-7B posted a reported +20.10% accuracy gain on AIME2024, and self-supervised confidence training is what earned it. The method is called RLSC, short for Reinforcement Learning via Self-Confidence. And it uses the model’s own confidence in its answers as the reward signal, which the paper says eliminates the need for labels, preference models. And reward engineering (Confidence Is All You Need). The model grades its own homework, then trains on that grade. The authors call the setup “zero-label reinforcement learning at scale.” If you pay per token for reasoning models, that last phrase is the one to sit with.
Because a model that knows when it is finished stops burning compute on ceremony.
The Mechanic: Grade the Answer, Skip the Reasoning
Standard RLHF is a supply chain. You need labelers producing gold answers, a preference dataset, and a reward model you have to keep honest. RLSC deletes every one of those dependencies. The loop samples candidate responses, computes confidence scores for the sampled reasoning-and-answer pairs. And updates the model to minimize a confidence-based training objective.
The recipe on paper is almost comically lean. The authors ran it on Qwen2.5-Math-7B, which they describe as a small-scale model, using only the AIME2024 training set, for 4 epochs, with 8 samples generated per question. That is the entire apparatus.
The detail I keep coming back to is the masking. An assistant mask isolates the answer tokens, so the reasoning tokens are masked from the training loss entirely. The chain-of-thought never gets directly graded. The model learns that a reasoning path is good only insofar as it lands on an answer the model itself believes in, which is a cleaner signal than any rubric a contractor could write.
There is also a companion setup in the same paper, ConfSFT, which needs no gold answers and no external judges. No auxiliary datasets, no instruction tuning, no preference models. Confidence is the whole supervision.
The Numbers: Big, Narrow, Single-Source
The reported gains are +20.10% on AIME2024, +49.40% on MATH500, and +52.50% on AMC23.
Those are serious movements on hard math benchmarks.
Now the skeptic’s list, as I have read enough single-paper results to keep one handy.
These are math benchmarks on a math-tuned model.
The paper reports accuracy, not token counts.
So if you came here for proof that confidence training shortens reasoning, this specific paper does not show it. The efficiency case comes from the surrounding literature, which I will get to. And every number above is one paper’s report. So treat it as a claim worth tracking rather than a settled result.
Self-improvement itself is not new, either.
An earlier lineage that circulated on r/MachineLearning fine-tuned models on their own self-generated solutions and reported a 540-billion-parameter model moving from 74.4% to 82.1% on GSM8K, DROP from 78.2% to 83.0%. And OpenBookQA from 90.0% to 94.4% (source). SaySelf, a separate framework, trains models to produce fine-grained confidence estimates using supervised fine-tuning followed by reinforcement learning, with its training data built from multiple sampled reasoning chains per question. And reports reduced calibration error while holding task performance steady. What changed with RLSC is that confidence got promoted from a reporting feature to the reward signal itself.
Why Efficiency Is the Real Prize
Reasoning inference is the expensive kind, and the literature is blunt about it.
Reasoning-oriented inference means longer outputs, deeper model computation.
And considerably more computational and energy resources than a plain completion (paper).
The field’s existing fixes come in two buckets.
One bucket is inference-time early stopping, including training decision models to learn dynamic stopping policies. The other bucket is training that pushes for shorter chains outright, spanning supervised fine-tuning through reinforcement learning, with length penalties in RL cited as a standard example (overview).
Confidence training is a third bucket, and I think it is the right one.
The “Self-Training Big Language Models with Confident Reasoning” line of work uses confidence-guided supervision to monitor and control the reasoning process inside a single chain-of-thought trajectory (paper). Here is my read on the difference. A length penalty bribes the model to shut up, while a confidence signal teaches it to know when it is done. The first optimizes a proxy. And the second builds a skill. And skills transfer to routing, abstention, and budget control in ways a token cap never will.
What Small Operators Should Actually Do
I run a one-person automation shop, so I will translate. You are not going to fine-tune a math model, and you should not.
Three moves do transfer.
First, start logging confidence next to correctness in every eval you run.
When a reasoning model answers something checkable, record whether it was right and how confident it claimed to be.
That is calibration data, and it is the input to every routing decision you will ever make.
Second, audit your reasoning-token spend like a budget line, since it is one. Go task by task and ask what the long thinking actually changed in the output.
My default assumption is that most of those chains are insurance the task did not need.
And the audit is how you find out before you keep paying the premium.
Third, watch for calibrated confidence to arrive in the models you already call. SaySelf’s whole direction is models that express fine-grained confidence instead of a shrug. The day that lands in an API you use, it becomes a routing input overnight. And the shops already logging confidence will wire it up in an afternoon while everyone else starts from zero.
The Takeaway
The flashy result is a model gaining accuracy with no labels at all. The boring win is worth more to you: a model that says “I’m not sure” and hands off to a human or a cheaper model, instead of reasoning at length toward a confident mistake. Self-supervised confidence training is the research direction most likely to put that switch in your stack. And the eval logging you start today is what makes it useful the day it arrives.
If your reasoning-model spend keeps climbing while output quality sits flat, that is the kind of problem I fix for a living.
Mediascout builds and audits AI automation for small teams.
And a first look at where your tokens are going is a conversation, not a contract.
