statement
In the deep layers, when units become most focused and smart, the model creates a bottleneck that erases most of their information.
A statement by the post author. Statements are as reported by the cited publishers — not verified by DecisionGateway.
דווקא בשכבות העמוקות, כשהיחידות נהיות הכי ממוקדות וחכמות, המודל יוצר צוואר בקבוק שמוחק את רוב המידע שלהן.
Discussion
Related claims
Researchers discovered long ago that about 75% of attention heads in a trained model can be deleted and it will keep working almost the same, and the book shows the structural reason: at the end of each layer the model merges all units' insights into one blend, so the next layer receives a 'fruit shake' and cannot know which unit discovered what.the post author · statementThe book proves that unifying weights inside the attention mechanism to save memory erases the model's built-in ability to understand direction, order and causality.the post author · statementA famous 2021 paper taught that without the heaviest components (MLP layers, about 65% of compute) the model's information would 'collapse' and become uniform, but the book proves mathematically this is simply not true: Residuals alone prevent the collapse, and the MLP is there to add computational flexibility (non-linearity), not to save the model from collapse.the post author · statementThe residual component is 4 times noisier than the first term, and is responsible for 80% of the update strength flowing back to the shared trunk; as the model grows it only intensifies and pulls the weights in a direction that interferes with next-token prediction.post author · statementOn algorithmic tasks (CLRS-Text), ten such layers raised an 86M-parameter model from 55% to 84%, after regular training got stuck at 55% and even training twice as long did not move it.post author · statementThe same layer works better when implanted in the middle of training than when it is there from the first step.post author · statement