statement

A famous 2021 paper taught that without the heaviest components (MLP layers, about 65% of compute) the model's information would 'collapse' and become uniform, but the book proves mathematically this is simply not true: Residuals alone prevent the collapse, and the MLP is there to add computational flexibility (non-linearity), not to save the model from collapse.

A statement by the post author. Statements are as reported by the cited publishers — not verified by DecisionGateway.

מאמר מפורסם מ-2021 לימד את כולנו שבלי הרכיבים הכי כבדים במודל (שכבות ה-MLP, שלוקחות כ-65% מכוח החישוב), המידע במודל "יקרוס" וכולו יהפוך לאחיד.
mathyaiwithmikePublished 2026-10-21 Read source

Discussion

0/2000

Related claims