The Attention Mechanism in Large Language Models

3-Second Blueprint (English)

1. Static word embeddings assign one fixed coordinate to ambiguous words like 'apple', confusing the fruit with the tech brand.
2. Introduced in Transformers, attention uses matrix operations to pull word vectors toward relevant context words in space.
3. Well-separated context embeddings like 'orange' versus 'phone' shift each 'apple' to distinct coordinates to resolve meaning.