---
title: "The residual component is 4 times noisier than the first term, and is responsible for 80% of the… — post author — DecisionGateway"
description: "statement by post author, recorded 2026-11-05 — verbatim quote and source citation."
canonical_url: "https://dgw.tmarket.io/claims/01a1e1ee-c9e3-7b12-b55d-3721312de156"
meta:
  "og:description": "statement by post author, recorded 2026-11-05 — verbatim quote and source citation."
  "og:image": "https://dgw.tmarket.io/og.jpg"
  "og:image:alt": "White network of connected nodes crossed by a timeline on black"
  "og:image:height": "1024"
  "og:image:width": "1024"
  "og:title": "The residual component is 4 times noisier than the first term, and is responsible for 80% of the… — post author — DecisionGateway"
  "twitter:image": "https://dgw.tmarket.io/og.jpg"
---

> Index of every page on this site: [llms.txt](https://dgw.tmarket.io/llms.txt)

statementSaid on 2026-11-05

# **The residual component is 4 times noisier than the first term, and is responsible for 80% of the update strength flowing back to the shared trunk; as the model grows it only intensifies and pulls the weights in a direction that interferes with next-token prediction.**

A statement by post author, said on 2026-11-05. Statements are as reported by the cited publishers — not verified by DecisionGateway.

> *מגלים שרכיב השארית רועש פי 4 מהאיבר הראשון.*

mathyaiwithmikePublished 2026-11-05 [ Read source](https://t.me/mathyaiwithmike/1302)

## **Discussion**

## **Related claims**

[**A famous 2021 paper taught that without the heaviest components (MLP layers, about 65% of compute) the model's information would 'collapse' and become uniform, but the book proves mathematically this is simply not true: Residuals alone prevent the collapse, and the MLP is there to add computational flexibility (non-linearity), not to save the model from collapse.**the post author · statement](https://dgw.tmarket.io/claims/01a1e1eb-c829-7457-8739-f90d60023742) [**During the backward pass, the MTP residual noise and the NTP residual noise meet with opposite signs and cancel each other out, leaving only the clean self-distillation signal; the result is 0 added memory, 0 impact on token throughput, and full compatibility with Fused kernels.**post author · statement](https://dgw.tmarket.io/claims/01a1e1ee-ca92-7c91-9863-1727b4a888ac) [**Researchers discovered long ago that about 75% of attention heads in a trained model can be deleted and it will keep working almost the same, and the book shows the structural reason: at the end of each layer the model merges all units' insights into one blend, so the next layer receives a 'fruit shake' and cannot know which unit discovered what.**the post author · statement](https://dgw.tmarket.io/claims/01a1e1eb-c746-7456-8138-bfb85325f512) [**In the deep layers, when units become most focused and smart, the model creates a bottleneck that erases most of their information.**the post author · statement](https://dgw.tmarket.io/claims/01a1e1eb-c7a4-7515-afef-b631f7c32e38) [**The book proves that unifying weights inside the attention mechanism to save memory erases the model's built-in ability to understand direction, order and causality.**the post author · statement](https://dgw.tmarket.io/claims/01a1e1eb-c7e3-7928-919d-d78b93be7f2e) [**In a regular network, if the model errs, the backprop algorithm stops everything, goes back from end to start and updates all the weights; in the brain there is no such central manager, and instead each synapse learns on its own from its local environment using STDP.**author · statement](https://dgw.tmarket.io/claims/01a1e1f1-9366-7832-923e-d2896e97aec1)

[All claims](https://dgw.tmarket.io/claims)

```json
{"@context":"https://schema.org","@graph":[{"@id":"https://dgw.tmarket.io/#website","@type":"WebSite","description":"Decisions, events and falsifiable forecasts from 4,500+ Telegram channels — every claim cited. Read free.","name":"DecisionGateway","url":"https://dgw.tmarket.io/","publisher":{"@id":"https://dgw.tmarket.io/#identity"}},{"@id":"https://dgw.tmarket.io/claims/01a1e1ee-c9e3-7b12-b55d-3721312de156#webpage","@type":"WebPage","description":"statement by post author, recorded 2026-11-05 — verbatim quote and source citation.","name":"The residual component is 4 times noisier than the first term, and is responsible for 80% of the… — post author — DecisionGateway","url":"https://dgw.tmarket.io/claims/01a1e1ee-c9e3-7b12-b55d-3721312de156","breadcrumb":{"@id":"https://dgw.tmarket.io/claims/01a1e1ee-c9e3-7b12-b55d-3721312de156#breadcrumb"},"isPartOf":{"@id":"https://dgw.tmarket.io/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://dgw.tmarket.io/claims/01a1e1ee-c9e3-7b12-b55d-3721312de156"]}]},{"@id":"https://dgw.tmarket.io/#identity","@type":"Organization","name":"DecisionGateway","url":"https://dgw.tmarket.io"},{"@id":"https://dgw.tmarket.io/claims/01a1e1ee-c9e3-7b12-b55d-3721312de156#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://dgw.tmarket.io","position":1},{"@type":"ListItem","name":"Claims","item":"https://dgw.tmarket.io/claims","position":2},{"@type":"ListItem","name":"The residual component is 4 times noisier than the first…","position":3}]},{"@id":"https://dgw.tmarket.io/claims/01a1e1ee-c9e3-7b12-b55d-3721312de156#/schema/quotation/1","@type":"Quotation","dateCreated":"2026-11-05","isBasedOn":"https://t.me/mathyaiwithmike/1302","text":"מגלים שרכיב השארית רועש פי 4 מהאיבר הראשון.","url":"/claims/01a1e1ee-c9e3-7b12-b55d-3721312de156"}]}
```

## Sitemap

See the full [sitemap](https://dgw.tmarket.io/sitemap.md) for all pages.
