statementSaid on 2026-10-25
Diffusion training is relatively simple regression — 'here is a noisy image (letter), guess the noise' — with no need for negative examples and no normalization constant to compute, which is why it is stable and scalable.
A statement by post author, said on 2026-10-25. Statements are as reported by the cited publishers — not verified by DecisionGateway.
האימון הוא רגרסיה פשוטה יחסית: "הנה תמונה (אות) מורעשת, נחש את הרעש". אין צורך בדוגמאות שליליות ואין קבוע נרמול לחשב, ולכן זה יציב ומתרחב.
Discussion
Related claims
Diffusion is almost an EBM: the direction diffusion learns is exactly the gradient-descent direction of some energy (in the transition from noisy data to clean), meaning diffusion directly learns the slope of the energy surface and skips the energy itself — the very trick that bypassed the difficulty of training EBMs.post author · statement'The end of diffusion' is hype; the better question is whether it is worth paying for more expensive training to get built-in verification and flexible compute at inference time — a real and interesting trade-off.מייק · statementThe post's author concludes that in supervised learning a distribution shift between train and test is usually a problem, while for RL training it is part of the solution.כותב הפוסט · statementThe models trained on the easier examples are tested on problems harder than those they saw in post-training and still succeed much more than models trained directly on the test difficulty level, without receiving full solutions or feedback on intermediate steps.המחברים · investigationWhen the same gradient keeps appearing across several checkpoints, the writers interpret it as "demand" that SGD fails to supply.the writers · statementLarge AI labs train models with MTP, but 80% of the gradient they backpropagate to the model is static noise that interferes with learning.post author · allegation