---
title: "Large AI labs train models with MTP, but 80% of the gradient they backpropagate to the model is… — post author — DecisionGateway"
description: "allegation by post author, recorded 2026-11-05 — verbatim quote and source citation."
canonical_url: "https://dgw.tmarket.io/claims/01a1e1ee-c8cc-76f8-a0cf-fdc0d5c6fade"
meta:
  "og:description": "allegation by post author, recorded 2026-11-05 — verbatim quote and source citation."
  "og:image": "https://dgw.tmarket.io/og.jpg"
  "og:image:alt": "White network of connected nodes crossed by a timeline on black"
  "og:image:height": "1024"
  "og:image:width": "1024"
  "og:title": "Large AI labs train models with MTP, but 80% of the gradient they backpropagate to the model is… — post author — DecisionGateway"
  "twitter:image": "https://dgw.tmarket.io/og.jpg"
---

> Index of every page on this site: [llms.txt](https://dgw.tmarket.io/llms.txt)

allegationSaid on 2026-11-05

# **Large AI labs train models with MTP, but 80% of the gradient they backpropagate to the model is static noise that interferes with learning.**

A allegation by post author, said on 2026-11-05. Statements are as reported by the cited publishers — not verified by DecisionGateway.

> *מעבדות ה-AI הגדולות מאמנות מודלים עם MTP, אבל מסתבר ש-80% מהגראדיאנט שהן מפעפעו אחורה (בקפרופ) למודלל הוא בכלל רעש סטטי שמפריע למודל ללמוד.*

mathyaiwithmikePublished 2026-11-05 [ Read source](https://t.me/mathyaiwithmike/1302)

## **Discussion**

## **Related claims**

[**If you train models with MTP architecture, stop injecting noisy CE gradients and instead add the Soft-MTP line of code.**מייק · commitment](https://dgw.tmarket.io/claims/01a1e1ee-4898-7ced-9f43-9f192a44b50c) [**When decomposing the standard MTP gradient (trained with standard Cross-Entropy), two opposing forces are hidden inside: a useful self-distillation signal and a residual noise component.**post author · statement](https://dgw.tmarket.io/claims/01a1e1ee-c9b5-7ec8-958e-20cbda9fe423) [**MIT researchers discovered that despite everyone using MTP, no one noticed a built-in mathematical bug in the training process.**post author · investigation](https://dgw.tmarket.io/claims/01a1e1ee-c94e-7286-b56f-53a1d78ff717) [**In a regular network, if the model errs, the backprop algorithm stops everything, goes back from end to start and updates all the weights; in the brain there is no such central manager, and instead each synapse learns on its own from its local environment using STDP.**author · statement](https://dgw.tmarket.io/claims/01a1e1f1-9366-7832-923e-d2896e97aec1) [**On algorithmic tasks (CLRS-Text), ten such layers raised an 86M-parameter model from 55% to 84%, after regular training got stuck at 55% and even training twice as long did not move it.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f6-8f8e-7a10-a02f-2baf8c6d789e) [**When the same gradient keeps appearing across several checkpoints, the writers interpret it as "demand" that SGD fails to supply.**the writers · statement](https://dgw.tmarket.io/claims/01a1e1f6-8f5d-7c1b-85c7-28a8997d5012)

[All claims](https://dgw.tmarket.io/claims)

```json
{"@context":"https://schema.org","@graph":[{"@id":"https://dgw.tmarket.io/#website","@type":"WebSite","description":"Decisions, events and falsifiable forecasts from 4,500+ Telegram channels — every claim cited. Read free.","name":"DecisionGateway","url":"https://dgw.tmarket.io/","publisher":{"@id":"https://dgw.tmarket.io/#identity"}},{"@id":"https://dgw.tmarket.io/claims/01a1e1ee-c8cc-76f8-a0cf-fdc0d5c6fade#webpage","@type":"WebPage","description":"allegation by post author, recorded 2026-11-05 — verbatim quote and source citation.","name":"Large AI labs train models with MTP, but 80% of the gradient they backpropagate to the model is… — post author — DecisionGateway","url":"https://dgw.tmarket.io/claims/01a1e1ee-c8cc-76f8-a0cf-fdc0d5c6fade","breadcrumb":{"@id":"https://dgw.tmarket.io/claims/01a1e1ee-c8cc-76f8-a0cf-fdc0d5c6fade#breadcrumb"},"isPartOf":{"@id":"https://dgw.tmarket.io/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://dgw.tmarket.io/claims/01a1e1ee-c8cc-76f8-a0cf-fdc0d5c6fade"]}]},{"@id":"https://dgw.tmarket.io/#identity","@type":"Organization","name":"DecisionGateway","url":"https://dgw.tmarket.io"},{"@id":"https://dgw.tmarket.io/claims/01a1e1ee-c8cc-76f8-a0cf-fdc0d5c6fade#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://dgw.tmarket.io","position":1},{"@type":"ListItem","name":"Claims","item":"https://dgw.tmarket.io/claims","position":2},{"@type":"ListItem","name":"Large AI labs train models with MTP, but 80% of the…","position":3}]},{"@id":"https://dgw.tmarket.io/claims/01a1e1ee-c8cc-76f8-a0cf-fdc0d5c6fade#/schema/quotation/1","@type":"Quotation","dateCreated":"2026-11-05","isBasedOn":"https://t.me/mathyaiwithmike/1302","text":"מעבדות ה-AI הגדולות מאמנות מודלים עם MTP, אבל מסתבר ש-80% מהגראדיאנט שהן מפעפעו אחורה (בקפרופ) למודלל הוא בכלל רעש סטטי שמפריע למודל ללמוד.","url":"/claims/01a1e1ee-c8cc-76f8-a0cf-fdc0d5c6fade"}]}
```

## Sitemap

See the full [sitemap](https://dgw.tmarket.io/sitemap.md) for all pages.
