Attention and transformers

Words change meaning with context. Who is “it” in “The animal didn't cross the street because it was too tired”? And if we end with “…because it was too wide”? To understand language, each word needs to look at the others. That's exactly what attention does.