# Inside a Transformer: One Word's Journey to the Next Word > Follow one word through a language model, from token ID to the final pick: embeddings, a stack of layers, attention, feed-forward memory and probabilities. - Author: Tai Pham Phu - Published: 29 Oct 2026 - Tags: How AI Works, LLM, Transformers, Applied AI, Technology - URL: https://phamphutai.com/2026/10/29/inside-a-transformer/ In 2022, four researchers from MIT, Northeastern University and the Technion went looking for one fact inside an AI model. The model was GPT-J, an open model from the EleutherAI group, built along the same lines as GPT-3. The fact was simple: the Eiffel Tower is in Paris. They found where the model seemed to keep it. Then they changed a small set of the model's numbers. After that, the model said the Eiffel Tower is in Rome. It didn't just repeat the new answer to the exact question. It used it in other sentences about the tower too. Two years later, Anthropic found a pattern inside its Claude 3 Sonnet model that switched on for the Golden Gate Bridge, and turned it up. Asked how to spend ten dollars, the modified model suggested driving across the Golden Gate Bridge and paying the toll. Both experiments point to the same thing. A language model isn't a fog of numbers. It has parts, and the parts do different jobs. This post opens the box and follows one word all the way through it. --- ## Where we are in the series This is part 5 of How AI Works, the last stop of the journey. In [part 4](https://phamphutai.com/2026/10/28/attention-how-ai-knows-what-it-means/) we saw how attention lets the word "it" look back at "cat" and borrow its meaning. Here is everything so far: 1. **[How ChatGPT actually works](https://phamphutai.com/2026/10/25/how-chatgpt-actually-works/).** A language model does one thing: it predicts the next token, adds it to the text, and repeats. 2. **[What's a token](https://phamphutai.com/2026/10/26/what-is-a-token/).** Models read text as tokens, chunks of words, each with a number called a token ID. The context window is how many tokens the model can look at at once. 3. **[Embeddings](https://phamphutai.com/2026/10/27/embeddings-how-ai-turns-words-into-numbers/).** Each token ID is swapped for an embedding, a long list of numbers (a vector) that places the token on a map of meaning. 4. **[Attention](https://phamphutai.com/2026/10/28/attention-how-ai-knows-what-it-means/).** Each token looks at the other tokens and pulls in what's relevant, so "it" can work out that it means "the cat". Our running sentence is the same as always: **"The cat sat on the mat because it was tired."** For this post we stop it one word early. The model has read "The cat sat on the mat because it was" and has to produce the next word. We'll follow the last token, "was", from the bottom of the model to the top until "tired" comes out. --- ## The whole trip on one page Here is the whole journey. Read it from the bottom up, the direction the numbers travel.
READ FROM THE BOTTOM UP 6 · pick one token: "tired" softmax: scores become probabilities tired 40% · sleepy 15% · hungry 8% · … (illustrative) 5 · a score for every token in the vocabulary 50,257 scores in GPT-2, every single time final vector for "was" 3 · LAYER N 4b · feed-forward this word alone: recall, refine 4a · attention gather from earlier words 3 · LAYER 1 4b · feed-forward this word alone: recall, refine 4a · attention gather from earlier words + + + + the same kind of layer, repeated (12 in the smallest GPT-2, 96 in the largest GPT-3) notepad 2 · embedding (meaning) + position ("I'm token 9") one long list of numbers per token 1 · token IDs: 464 3797 3332 319 262 2603 780 340 [373] The · cat · sat · on · the · mat · because · it · was 7 · add "tired" to the text, run the trip again Token IDs from GPT-2's tokenizer. Probabilities are made up for illustration.
One token's trip. The thick line is the residual stream: each layer reads it, and attention and feed-forward each add their update back with a "+". Nothing is erased on the way up.
Seven steps. Let's take them one at a time. --- ## Step 1: words become token IDs The model never sees letters. As we saw in part 2, the text is first cut into tokens, and each token is looked up in a fixed list called the **vocabulary**. Its position in that list is its **token ID**. Using GPT-2's tokenizer (the same one GPT-3 used), our sentence becomes nine tokens: | Token | The | ␣cat | ␣sat | ␣on | ␣the | ␣mat | ␣because | ␣it | ␣was | |---|---|---|---|---|---|---|---|---|---| | ID | 464 | 3797 | 3332 | 319 | 262 | 2603 | 780 | 340 | 373 | The "␣" marks a space. In this tokenizer, " cat" with a space in front is a different token from "cat" without one. That vocabulary has 50,257 tokens in total. Newer tokenizers are bigger: OpenAI's tiktoken library includes one with about 100,000 tokens and one with about 200,000. The idea is the same. --- ## Step 2: IDs become embeddings, plus a sense of position Each token ID is swapped for its **embedding**: the list of numbers from part 3 that places the token on a map of meaning. In GPT-2's smallest version that list is 768 numbers long. In the largest GPT-3 it is 12,288. Words with related meanings, like "cat" and "dog", end up with similar lists. But there's a gap. Attention, from part 4, lets every token look at other tokens. It doesn't, on its own, know what order they came in. "Dog bites man" and "man bites dog" contain the same three tokens. Without extra help, the model would see the same ingredients and miss the story. So the model also gets **position information**. In the original 2017 Transformer paper, the authors added a position signal straight onto each embedding at the bottom of the stack: a pattern of numbers that says, in effect, "I'm token 1", "I'm token 2", and so on. Many newer models handle position differently. Meta's Llama models, for example, use a method called rotary position embeddings, which works the position into the attention step instead of adding it once at the start. Either way, the result is the same: each token now carries **what it is** and **where it sits**. From here on, I'll call this combined list the token's **notepad**. --- ## Step 3: a tall stack of layers, and a notepad that only grows The middle of the model is a stack of **layers**. They all have the same design, but each has its own learned numbers, so each ends up doing slightly different work. The smallest GPT-2 has 12 layers. The largest GPT-3, described in OpenAI's 2020 paper, has 96. Most companies don't publish these details for their newest models. Every token's notepad travels up through all of those layers. Here's the important part: **the layers don't rewrite the notepad. They add to it.** Each layer reads the notepad, works out an update, and adds that update to what was already there. The engineering name is a **residual connection**. A good way to remember it: *keep the original, add the update.* Researchers who study what's going on inside these models call the running notepad the **residual stream**. Imagine a document passed up a line of 96 people. Nobody may delete what the others wrote. Each person reads it and adds a few notes in the margin. By the top, the document holds the original text plus everything anyone found worth adding. Where the analogy breaks: the notes aren't words. They're numbers added onto the same list, so later layers see a blend, not tidy separate notes. There's one more small part in each layer, **normalisation**: a step that keeps the numbers on the notepad in a steady range, so they don't drift too large or too small as layers keep adding to them. Inside every layer, two other parts do the real work: the two boxes in the diagram. --- ## Step 4a: attention mixes in context The first box is **attention**, which you know from part 4. Our token "was" looks back at the tokens before it and pulls in what's relevant. In a GPT-style model, attention has one strict rule: a token can only look **backwards**. "was" can look at "The", "cat", "sat" and so on, but never at anything that comes after it. When the model is writing, there is nothing after it yet. The original paper calls this **masking**. For our sentence, the useful thing for "was" to pick up is what "it" refers to. In part 4 we saw "it" gather meaning from "cat". Now "was" can gather from "it", which by this layer already carries "cat-ness". After a few layers, the notepad for "was" holds something like: *the subject is a cat, something is being explained ("because"), a description of the cat's state comes next.* Each layer has many attention "heads" looking for different things. The point to remember: **attention moves information between tokens.** It's the only place in the layer where tokens see each other. --- ## Step 4b: the feed-forward part works on each token alone The second box is the **feed-forward network**. Once attention has gathered context, each token's notepad goes through the feed-forward part **on its own**, without looking at any other token. Mechanically, it's "multiply and add". The notepad is multiplied by one big table of learned numbers to make a much wider list (in GPT-3, four times wider), passed through a simple on/off-style filter, and multiplied by a second big table to shrink it back. The result is the update that gets added to the notepad. It's also where most of the model's numbers sit. In a GPT-3-style layer, attention has roughly four "model-width by model-width" tables of numbers and the feed-forward part has the equivalent of eight. So about two-thirds of the numbers inside the layers are in feed-forward parts, not in attention. What do all those numbers do? **Researchers think a lot of the model's stored knowledge lives here.** In a 2021 paper, Mor Geva and colleagues argued that feed-forward layers work like key–value memories. Each "key" watches for a pattern on the notepad. When the pattern shows up, its "value" pushes the notepad towards certain next words. The Eiffel Tower study from the opening found the same kind of thing from another direction. The fact seemed to be retrieved by feed-forward parts in the middle layers, while the model was processing the last token of "Eiffel Tower", and attention then carried it forward to where the answer was needed. Editing a small set of numbers in one of those feed-forward parts moved the tower to Rome. So, in our sentence: once "was" knows it's describing a cat's state, feed-forward detectors that respond to *"describing why an animal did something restful"* may push the notepad towards words like "tired", "sleepy" or "lazy". That's an illustration, not a measured result, but it's the kind of thing researchers observe. Two honest warnings. First, "knowledge lives in feed-forward layers" is a useful summary of research, not a settled law. Knowledge is spread out, attention plays a part, and edits like the Eiffel Tower one can spill over into related facts. Second, a "memory" here is not a database row. There's no line that says `Eiffel Tower → Paris`. It's a pattern spread across many numbers, which is exactly why models can recall things confidently and still be wrong. Then the next layer runs, attention again, feed-forward again, all the way up the stack. --- ## Watching the guess form Here's a lovely side effect of "keep the original, add the update". Because the notepad keeps the same shape all the way up, you can stop it halfway and ask: *if the model had to answer now, what would it say?* A researcher writing as nostalgebraist tried this on GPT-2 in 2020 and called it the **logit lens**. Early layers give vague guesses. As you move up, the guesses get sharper, until the top layers settle on the final answer. For our sentence, you might see something like "the" or "a" near the bottom, then descriptive words in the middle, then "tired" winning near the top. (That sequence is illustrative. The real pattern varies by model and sentence.) --- ## Step 5: a score for every token, then probabilities After the last layer, a final normalisation tidies the notepad for "was". Now it has to become an answer. The model multiplies this final vector by one more big table. Out comes **one score for every token in the vocabulary**. In GPT-2, that's 50,257 scores. Every single time the model writes one token. A higher score means "this would fit well here". Scores can be any size, positive or negative, so the model passes them through a function called **softmax**, which turns them into **probabilities**: positive numbers that add up to 100%. The highest score gets the biggest share, and small differences in score become big differences in share. For "The cat sat on the mat because it was", the result could look something like this (made-up numbers, for illustration): | Next token | Probability | |---|---| | ␣tired | 40% | | ␣sleepy | 15% | | ␣hungry | 8% | | ␣lazy | 5% | | … 50,000+ others | the rest, most of them close to 0% | So what the model actually produces is not a word. It's a **probability for every possible next token**. --- ## Step 6: pick one Now one token is chosen from that list. The simplest way is to take the top one. Most chat products don't always do that. They pick at random, weighted by those probabilities, so a likely word usually wins but not always. That's a big reason the same question can get different answers. How adventurous that pick is can be tuned with settings such as temperature. That gets its own part later in the series. For today, the model picks "tired". --- ## Step 7: add it, and run the whole trip again The model has produced **one token**. To write the next one, "tired" is added to the text, and the trip runs again: the new token gets its embedding and position, climbs every layer, gathers from all the tokens before it, and comes out as another 50,257 scores. Then another pick. Then again. A 500-token answer means 500 trips up the stack. In practice, models don't redo all the work for the earlier tokens on each trip. The parts of attention that earlier tokens offer to later ones are saved and reused, a trick called a **KV cache** (short for key–value cache). So each new trip mainly computes the new token, while still looking back at everything before it. That cache is part of why long conversations use more memory. And because every new token is still a trip, AI companies [charge per token](https://phamphutai.com/2026/10/14/hidden-costs-of-ai-tools/) for output. This loop is the "predict the next token, add it, repeat" idea from [part 1](https://phamphutai.com/2026/10/25/how-chatgpt-actually-works/), now with the inside filled in. --- ## GPT-style vs the original 2017 design The Transformer was introduced in the 2017 Google paper "Attention Is All You Need" (Vaswani and colleagues). It was built for **translation**, and it had two stacks: - An **encoder** that reads the whole input sentence, say in English. Its tokens can look in both directions, because the full sentence is already there. - A **decoder** that writes the output, say in German, one token at a time. Its tokens can only look backwards at what it has written so far, plus an extra attention step that looks across at the encoder's understanding of the English. Each stack had 6 layers, and every layer had the same pattern you've just seen: attention, feed-forward, "keep the original, add the update", normalisation.
2017: ENCODER-DECODER GPT-STYLE: DECODER-ONLY Encoder looks both ways Decoder looks back only reads "The cat is tired." whole input "Die Katze ist" output so far "müde" Decoder only looks back only "…because it was" the text so far "tired"
The 2017 Transformer used two stacks for translation. GPT-style models keep only the decoder side and treat everything, question and answer alike, as one text to continue.
OpenAI's first GPT paper in 2018 kept only the **decoder** side, without the extra look-across step, and trained it to predict the next token of ordinary text. GPT-2 and GPT-3 followed the same plan, with a few changes, such as moving the normalisation to the start of each block. This **decoder-only** design is what the journey in this post describes. Openly published chat models like Meta's Llama use the same basic shape. Most companies don't publish the full design of their newest models, but next-token prediction with a stack like this is the common starting point. Your question and the model's answer are just one long text, and the model keeps continuing it. Encoders didn't disappear. Encoder-only models like Google's BERT (2018) are still used for jobs like search and sorting text. --- ## What this means for you Three practical things follow from all this. 1. **Facts are patterns, not records.** What the model "knows" is spread across numbers in its layers, mostly learned during training. It can be out of date or confidently wrong, as plenty of [real AI fails](https://phamphutai.com/2026/10/23/ai-fails-every-business-should-know/) show. Check anything important, or use a tool that searches real sources and shows you where the answer came from. 2. **Every token is a trip.** Long answers take longer and cost more, because each token runs through the whole stack. If you pay for AI by usage, asking for shorter answers is a real saving. 3. **Every run is a fresh draw.** Because the last step picks from probabilities, asking again can give a different answer. If an answer matters, ask twice or ask for the reasoning, and build systems that [expect the model to be occasionally wrong](https://phamphutai.com/2026/10/14/treat-the-llm-like-an-unreliable-api/). --- ## What we covered in this series Five parts, one sentence, one journey: - Text is cut into **tokens**, and each token gets a **token ID** (part 2). - Each ID becomes an **embedding**, a vector on a map of meaning (part 3), plus **position** information. - The vector climbs a stack of **layers**. In each one, **attention** mixes in context from earlier tokens (part 4), and the **feed-forward** part processes each token on its own, where researchers think much of the stored knowledge lives. - **Residual connections** keep the original and add the update, and **normalisation** keeps the numbers steady. - At the top, the final vector becomes a **score for every token**, softmax turns the scores into **probabilities**, one token is picked, and the whole thing repeats. That's **next-token prediction** (part 1), seen from the inside. That's the machine. What we haven't covered is where its numbers come from, and how a raw next-token predictor becomes a helpful assistant. The series continues with exactly that: how training works, fine-tuning and learning from human feedback, temperature and how the pick is made, why bigger models got better (scaling), and the newer reasoning models that "think" before they answer. If you'd like to see the training side in code first, I wrote about [fine-tuning a small GPT-2 model](https://phamphutai.com/2025/06/14/generative-ai-large-language-model-i-fine-tunning-gpt-model/) back in 2025. Next: How AI Training Works, and why it costs millions. --- *Sources: Vaswani et al., "Attention Is All You Need" (NeurIPS 2017, arXiv 1706.03762); Radford et al., "Improving Language Understanding by Generative Pre-Training" (OpenAI, 2018); Radford et al., "Language Models are Unsupervised Multitask Learners" (OpenAI, 2019, GPT-2); Brown et al., "Language Models are Few-Shot Learners" (2020, GPT-3, Table 2.1); OpenAI tiktoken library (r50k_base, cl100k_base and o200k_base encodings), checked October 2026; Su et al., "RoFormer: Enhanced Transformer with Rotary Position Embedding" (2021); Touvron et al., "LLaMA: Open and Efficient Foundation Language Models" (Meta, 2023); Geva et al., "Transformer Feed-Forward Layers Are Key-Value Memories" (EMNLP 2021); Meng, Bau, Andonian and Belinkov, "Locating and Editing Factual Associations in GPT" (NeurIPS 2022, rome.baulab.info); Anthropic, "Golden Gate Claude" (23 May 2024) and "Mapping the Mind of a Large Language Model" (21 May 2024); nostalgebraist, "interpreting GPT: the logit lens" (LessWrong, 2020); Devlin et al., "BERT" (2018).* *Related: [What an "AI Agent" Actually Is, Without the Hype](https://phamphutai.com/2026/10/14/what-an-ai-agent-actually-is/), [Treat the LLM Like an Unreliable API](https://phamphutai.com/2026/10/14/treat-the-llm-like-an-unreliable-api/), and [The Rise of Generative AI: Simplifying Learning](https://phamphutai.com/2025/06/10/generative-ai-chapter-1-what-is-gen-ai/).*