That ability helped make the Transformer, the architecture behind modern LLMs, practical to train at scale.
The same word can mean different things
“She reached the bank” leaves a question open. Add “of the river,” and bank means an edge of land. Add “to deposit a cheque,” and it means a financial institution. A language model cannot settle that question by looking at bank alone. It needs a representation of the word that changes with its context.
The two uses of bat below offer another case. The arrows are an illustration of possible information flow, not a record of what a particular model did.
What attention actually does
First, the model splits text into tokens—pieces that may be words or parts of words—and represents each as numbers. For every token, an attention layer makes three learned versions of those numbers. A query is what the current position uses to look for useful information. A key is what another position offers for comparison. A value is the information that position can contribute. They are three numerical representations, not questions and answers written in English.
The layer compares a query with the keys it is allowed to see and gives each comparison a score. It then turns the scores into weights that add up to one. Finally, it uses those weights to mix the values. The result carries information from other positions into the current one. A token can draw more from one neighbor and less from another; attention is a mixture, not a switch that picks a single word.
This worked example follows it in a short sentence. The thicker path is a teaching choice, not a claim about measured weights in a trained model. The key and value cards on each side come from the same source token; a key does not literally turn into a value.
Why this changed language models
Earlier recurrent networks processed a sequence step by step. To carry information from a distant word to a later one, it had to travel through the intervening steps. In a Transformer attention layer, a position can draw directly from any position it is allowed to see. That shorter route helps the model use distant context.
The original 2017 Transformer was designed for translation. It also showed a useful training property: the positions of a known sequence can be processed together within a layer, instead of waiting for a recurrent chain. Later, GPT used a decoder-only Transformer trained to predict the next token from earlier text. That became a key route to text-generating LLMs.
There is an important limit to “parallel.” During training, many positions in a known sequence can be worked on together. While writing, an autoregressive LLM must produce one new token before it can use that token to produce the next. The picture separates those two moments.
Attention is only part of an LLM. The model also needs information about token order, other neural-network layers, and learned weights throughout. Many attention operations, called heads, give it several learned ways to combine context; they are not assigned tidy jobs such as “the grammar head.”
The basic idea remains simple: compare what one position seeks with what other positions offer, then blend their information. That does not make the model a human reader or guarantee a correct answer. It gives the model a powerful way to use context as it predicts what comes next.