---
title: "The problem with EBMs is training: it is not enough to teach the model that real examples have low… — post author — DecisionGateway"
description: "statement by post author, recorded 2026-10-25 — verbatim quote and source citation."
canonical_url: "https://dgw.tmarket.io/claims/01a1e1f3-ce4b-7960-83c7-fe561635767d"
meta:
  "og:description": "statement by post author, recorded 2026-10-25 — verbatim quote and source citation."
  "og:image": "https://dgw.tmarket.io/og.jpg"
  "og:image:alt": "White network of connected nodes crossed by a timeline on black"
  "og:image:height": "1024"
  "og:image:width": "1024"
  "og:title": "The problem with EBMs is training: it is not enough to teach the model that real examples have low… — post author — DecisionGateway"
  "twitter:image": "https://dgw.tmarket.io/og.jpg"
---

> Index of every page on this site: [llms.txt](https://dgw.tmarket.io/llms.txt)

statementSaid on 2026-10-25

# **The problem with EBMs is training: it is not enough to teach the model that real examples have low energy; it must also be taught that everything else has high energy, and that 'everything else' is enormous, which is why EBMs were stuck for years with unstable training and expensive sampling.**

A statement by post author, said on 2026-10-25. Statements are as reported by the cited publishers — not verified by DecisionGateway.

> *הבעיה עם ה-EBMs היא האימון. לא מספיק ללמד את המודל שדוגמאות אמיתיות הן "נמוכות" בבחינת ערך האנרגיה שלהם. צריך גם ללמד אותו שכל השאר "גבוה", וה"כל השאר" הזה עצום. בגלל זה EBMs נתקעו שנים עם אימון לא יציב ודגימה יקרה.*

mathyaiwithmikePublished 2026-10-25 [ Read source](https://t.me/mathyaiwithmike/1295)

## **Discussion**

## **Related claims**

[**EBMs promised a revolution 20 years ago, and the problem was never elegance — it was training at scale.**מייק · statement](https://dgw.tmarket.io/claims/01a1e1f3-ceae-7241-9d49-f814ac4ed3e0) [**Diffusion is almost an EBM: the direction diffusion learns is exactly the gradient-descent direction of some energy (in the transition from noisy data to clean), meaning diffusion directly learns the slope of the energy surface and skips the energy itself — the very trick that bypassed the difficulty of training EBMs.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f3-ce6a-7994-8b63-3d2f141cabfb) [**In EBM one can combine several models simply by adding their energies, giving 'both and', whereas in diffusion this works only approximately.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f3-ce8b-7db7-a2c3-ea2620dcc0f4) [**In Energy-Based Transformers, several optimization steps are run on the energy during training and the model is trained through that process itself, which is more expensive.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f3-ce79-7bf5-95ba-b94da6ad7af2) [**The models trained on the easier examples are tested on problems harder than those they saw in post-training and still succeed much more than models trained directly on the test difficulty level, without receiving full solutions or feedback on intermediate steps.**המחברים · investigation](https://dgw.tmarket.io/claims/01a1e1f1-8f42-7f29-82b0-7ecb46307f22) [**On algorithmic tasks (CLRS-Text), ten such layers raised an 86M-parameter model from 55% to 84%, after regular training got stuck at 55% and even training twice as long did not move it.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f6-8f8e-7a10-a02f-2baf8c6d789e)

[All claims](https://dgw.tmarket.io/claims)

```json
{"@context":"https://schema.org","@graph":[{"@id":"https://dgw.tmarket.io/#website","@type":"WebSite","description":"Decisions, events and falsifiable forecasts from 4,500+ Telegram channels — every claim cited. Read free.","name":"DecisionGateway","url":"https://dgw.tmarket.io/","publisher":{"@id":"https://dgw.tmarket.io/#identity"}},{"@id":"https://dgw.tmarket.io/claims/01a1e1f3-ce4b-7960-83c7-fe561635767d#webpage","@type":"WebPage","description":"statement by post author, recorded 2026-10-25 — verbatim quote and source citation.","name":"The problem with EBMs is training: it is not enough to teach the model that real examples have low… — post author — DecisionGateway","url":"https://dgw.tmarket.io/claims/01a1e1f3-ce4b-7960-83c7-fe561635767d","breadcrumb":{"@id":"https://dgw.tmarket.io/claims/01a1e1f3-ce4b-7960-83c7-fe561635767d#breadcrumb"},"isPartOf":{"@id":"https://dgw.tmarket.io/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://dgw.tmarket.io/claims/01a1e1f3-ce4b-7960-83c7-fe561635767d"]}]},{"@id":"https://dgw.tmarket.io/#identity","@type":"Organization","name":"DecisionGateway","url":"https://dgw.tmarket.io"},{"@id":"https://dgw.tmarket.io/claims/01a1e1f3-ce4b-7960-83c7-fe561635767d#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://dgw.tmarket.io","position":1},{"@type":"ListItem","name":"Claims","item":"https://dgw.tmarket.io/claims","position":2},{"@type":"ListItem","name":"The problem with EBMs is training: it is not enough to…","position":3}]},{"@id":"https://dgw.tmarket.io/claims/01a1e1f3-ce4b-7960-83c7-fe561635767d#/schema/quotation/1","@type":"Quotation","dateCreated":"2026-10-25","isBasedOn":"https://t.me/mathyaiwithmike/1295","text":"הבעיה עם ה-EBMs היא האימון. לא מספיק ללמד את המודל שדוגמאות אמיתיות הן \"נמוכות\" בבחינת ערך האנרגיה שלהם. צריך גם ללמד אותו שכל השאר \"גבוה\", וה\"כל השאר\" הזה עצום. בגלל זה EBMs נתקעו שנים עם אימון לא יציב ודגימה יקרה.","url":"/claims/01a1e1f3-ce4b-7960-83c7-fe561635767d"}]}
```

## Sitemap

See the full [sitemap](https://dgw.tmarket.io/sitemap.md) for all pages.
