💡
Attention Spotlight
A pronoun's meaning depends entirely on context. Guess which word the highlighted pronoun refers to — exactly the job self-attention does inside every transformer, word by word.
🚀 Advanced
~2 min
Concept: Attention · Transformers
Sentence 1 of 6
Which word does the highlighted pronoun point to?
📌 The trick
Notice how changing just one word (big → small, feared → advocated) completely flips which word gets the most attention. That's context-dependence — the whole reason self-attention beats fixed word embeddings.