investigation
The researchers confirm their claims in experiments: they post-train models such as Qwen and Llama on a task of solving systems of equations, where the final answer is 0 or 1 so about half the cases can be solved by guessing.
A investigation by החוקרים. Statements are as reported by the cited publishers — not verified by DecisionGateway.
החוקרים מאששים את הטענות גם בניסויים: הם עושים אימון המשך (Post-training) למודלים כמו Qwen ו-Llama על משימה שמטרתה לפתור מערכות משוואות.
Discussion
Related claims
The models trained on the easier examples are tested on problems harder than those they saw in post-training and still succeed much more than models trained directly on the test difficulty level, without receiving full solutions or feedback on intermediate steps.המחברים · investigationWhen the authors train the models only on problems at the test difficulty level, the models reach about 50% accuracy, almost like random guessing.המחברים · investigationThe researchers show that training leads the model to learn an efficient solution path, even though it receives no extra reward for choosing that path.החוקרים · investigationWhen the test-difficulty examples are replaced with a mixture of easier examples, Qwen jumps to 91.6% and Llama to 98.6%, while the test remains at exactly the same difficulty level.המחברים · investigationOn algorithmic tasks (CLRS-Text), ten such layers raised an 86M-parameter model from 55% to 84%, after regular training got stuck at 55% and even training twice as long did not move it.post author · statementThe researchers prove that in their synthetic framework, the training data must include enough simple examples (requiring few reasoning steps); with enough such examples the model learns to solve problems step by step, and without them it does not learn.החוקרים · investigation