---
title: "In a linear layer, each new memory is simply added to the weights; retrieval returns the correct… — post author — DecisionGateway"
description: "statement by post author, recorded 2026-10-19 — verbatim quote and source citation."
canonical_url: "https://dgw.tmarket.io/claims/01a1e1f0-e5bf-76de-9edf-9889f747958d"
meta:
  "og:description": "statement by post author, recorded 2026-10-19 — verbatim quote and source citation."
  "og:image": "https://dgw.tmarket.io/og.jpg"
  "og:image:alt": "White network of connected nodes crossed by a timeline on black"
  "og:image:height": "1024"
  "og:image:width": "1024"
  "og:title": "In a linear layer, each new memory is simply added to the weights; retrieval returns the correct… — post author — DecisionGateway"
  "twitter:image": "https://dgw.tmarket.io/og.jpg"
---

> Index of every page on this site: [llms.txt](https://dgw.tmarket.io/llms.txt)

statementSaid on 2026-10-19

# **In a linear layer, each new memory is simply added to the weights; retrieval returns the correct value plus a bit of every other memory depending on key overlap.**

A statement by post author, said on 2026-10-19. Statements are as reported by the cited publishers — not verified by DecisionGateway.

> *כל זיכרון חדש פשוט מתווסף למשקולות, וזהו. בשליפה מקבלים את הערך הנכון, ועוד קצת מכל זיכרון אחר, בהתאם למידת החפיפה בין המפתחות.*

mathyaiwithmikePublished 2026-10-19 [ Read source](https://t.me/mathyaiwithmike/1289)

## **Discussion**

## **Related claims**

[**Adding nonlinearity that silences somewhat-similar keys and keeps only the truly similar ones yields an MLP: one layer holds the keys, the second the values, and the activation decides who gets to speak.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f0-e60e-744a-aaa7-f3c1a3218ffc) [**In attention the memories do not sit in the weights; every token in the context is a memory, and the store grows as the context lengthens.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f0-e6cc-744c-9e55-fb2059e3c9ca) [**Every LLM is essentially a huge memory store with an extremely sophisticated retrieval mechanism.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f0-e585-7eb6-b0bf-0e7b0d65f3b4) [**Each implanted layer is multiplied by a learned gate initialized to zero; at the moment of implantation the network does not change at all, and the gradient descent decides whether to open the gate.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f6-8f6a-7096-8c31-22b8c40596c3) [**The book proves that unifying weights inside the attention mechanism to save memory erases the model's built-in ability to understand direction, order and causality.**the post author · statement](https://dgw.tmarket.io/claims/01a1e1eb-c7e3-7928-919d-d78b93be7f2e) [**To know where to implant, the agent computes the loss gradient for each token according to its representation in each layer.**post author · statement](https://dgw.tmarket.io/claims/01a1e1f6-8f4f-7234-a827-16c47834e007)

[All claims](https://dgw.tmarket.io/claims)

```json
{"@context":"https://schema.org","@graph":[{"@id":"https://dgw.tmarket.io/#website","@type":"WebSite","description":"Decisions, events and falsifiable forecasts from 4,500+ Telegram channels — every claim cited. Read free.","name":"DecisionGateway","url":"https://dgw.tmarket.io/","publisher":{"@id":"https://dgw.tmarket.io/#identity"}},{"@id":"https://dgw.tmarket.io/claims/01a1e1f0-e5bf-76de-9edf-9889f747958d#webpage","@type":"WebPage","description":"statement by post author, recorded 2026-10-19 — verbatim quote and source citation.","name":"In a linear layer, each new memory is simply added to the weights; retrieval returns the correct… — post author — DecisionGateway","url":"https://dgw.tmarket.io/claims/01a1e1f0-e5bf-76de-9edf-9889f747958d","breadcrumb":{"@id":"https://dgw.tmarket.io/claims/01a1e1f0-e5bf-76de-9edf-9889f747958d#breadcrumb"},"isPartOf":{"@id":"https://dgw.tmarket.io/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://dgw.tmarket.io/claims/01a1e1f0-e5bf-76de-9edf-9889f747958d"]}]},{"@id":"https://dgw.tmarket.io/#identity","@type":"Organization","name":"DecisionGateway","url":"https://dgw.tmarket.io"},{"@id":"https://dgw.tmarket.io/claims/01a1e1f0-e5bf-76de-9edf-9889f747958d#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://dgw.tmarket.io","position":1},{"@type":"ListItem","name":"Claims","item":"https://dgw.tmarket.io/claims","position":2},{"@type":"ListItem","name":"In a linear layer, each new memory is simply added to the…","position":3}]},{"@id":"https://dgw.tmarket.io/claims/01a1e1f0-e5bf-76de-9edf-9889f747958d#/schema/quotation/1","@type":"Quotation","dateCreated":"2026-10-19","isBasedOn":"https://t.me/mathyaiwithmike/1289","text":"כל זיכרון חדש פשוט מתווסף למשקולות, וזהו. בשליפה מקבלים את הערך הנכון, ועוד קצת מכל זיכרון אחר, בהתאם למידת החפיפה בין המפתחות.","url":"/claims/01a1e1f0-e5bf-76de-9edf-9889f747958d"}]}
```

## Sitemap

See the full [sitemap](https://dgw.tmarket.io/sitemap.md) for all pages.
