statementSaid on 2026-10-25
Alexi Gladstone's tweet is a good opportunity to clarify what an Energy-Based Model is, what a diffusion model is, and why the difference between them is smaller and more interesting than the hype suggests.
A statement by post author, said on 2026-10-25. Statements are as reported by the cited publishers — not verified by DecisionGateway.
הציוץ של אלכסי גלדסטון ("the end of diffusion is near") הוא הזדמנות טובה לעשות סדר: מה זה Energy-Based Model, מה זה מודל דיפוזיה, ולמה ההבדל ביניהם קטן יותר, ומעניין יותר, ממה שההייפ מרמז.
Discussion
Related claims
Diffusion is almost an EBM: the direction diffusion learns is exactly the gradient-descent direction of some energy (in the transition from noisy data to clean), meaning diffusion directly learns the slope of the energy surface and skips the energy itself — the very trick that bypassed the difficulty of training EBMs.post author · statement'The end of diffusion' is hype; the better question is whether it is worth paying for more expensive training to get built-in verification and flexible compute at inference time — a real and interesting trade-off.מייק · statementIn EBM one can combine several models simply by adding their energies, giving 'both and', whereas in diffusion this works only approximately.post author · statementDiffusion training is relatively simple regression — 'here is a noisy image (letter), guess the noise' — with no need for negative examples and no normalization constant to compute, which is why it is stable and scalable.post author · statementIn Energy-Based Transformers, several optimization steps are run on the energy during training and the model is trained through that process itself, which is more expensive.post author · statementWithin only 9 hours, Glazer together with the open LLM agent gang compressed the equation to only 12,504 variables.גלייזר יחד עם קנופיית סוכני ב״מ פתוחים · statement