How ChatGPT Actually Works, Explained Simply

Type this into ChatGPT: “The capital of France is”.

It answers “Paris”. It feels like it looked the answer up. It didn’t. There is no table of countries and capitals inside it, and unless the app is set up to search the web, it doesn’t go online either.

What it did was smaller and stranger. It looked at your words and worked out which word was most likely to come next. “Paris” won by a mile. So it wrote “Paris”.

That’s the whole trick, repeated over and over. And once you see it, a lot of things about AI chat tools stop being mysterious: why the answer appears bit by bit, why the same question can get two different answers, why it writes so smoothly, and why it can be completely wrong while sounding completely sure.

This is part 1 of a five-part series called How AI Works, for people who use ChatGPT, Claude or Gemini every day and want to know what is going on inside. No maths beyond “a list of numbers”, “multiply and add” and “a score”. It’s also the written companion to my video on the same topic, with more room to explain.

The sentence we’ll follow

Through this series we’ll follow one sentence through a language model:

The cat sat on the mat because it was tired.

It’s a boring sentence on purpose. Every part of the series zooms in on one step of what happens to it inside the model. In this part we stay zoomed out and look at the whole machine.

The big idea: it predicts the next word

Open a message on your phone and type “See you”. Above the keyboard, your phone suggests the next word: soon, tomorrow, later. It has seen a lot of messages, and those words usually come next.

ChatGPT does the same job at a completely different scale. The technical name for what’s inside it is a large language model, or LLM. Large because it’s enormous. Language because it works with text. Model because it’s a model of how language usually goes.

Here’s what happens when you send a message. Let’s say the text so far is:

The cat sat on the mat because it was

The model reads all of it and produces a probability for every possible next piece: a score between 0 and 1 for how likely that piece is to come next. The scores add up to 1. Something like this (I made these numbers up to show the idea; a real model gives scores for tens of thousands of options):

Then the app picks one. It adds that word to the text and runs the whole thing again, now asking what comes after “…because it was tired”. Then again. And again. Word by word, until the model produces a special “I’m done” marker or hits a length limit.

This is called next-token prediction, and it’s the one thing a language model does. Everything else in this post, and in the series, is about how it does that one thing well.

Why the answer appears bit by bit

When an answer streams onto your screen like someone typing, that isn’t a visual effect. It really is being written one piece at a time, each piece depending on everything before it. The app shows you each piece as soon as it exists.

Why the same question gets different answers

Notice step 5 in the diagram says “usually a likely one”. Chat apps don’t always pick the single top-scoring word. They pick at random, weighted by the scores: “tired” most of the time, “hungry” sometimes, “warm” now and then. Developers can turn this randomness up or down with a setting usually called temperature. Low temperature means almost always the top choice; high temperature means more variety.

A bit of randomness makes writing sound more natural and less repetitive. It’s also why you can ask the same question twice and get two different answers. One small different pick early on sends the rest of the answer down a different path.

Tokens, briefly

I’ve been saying “word”, but models actually work in tokens: pieces of text that are often a whole word, sometimes part of one, sometimes a space plus a word or a bit of punctuation. Each token has a number, its token ID, and the model only ever sees those numbers. Part 2 of this series is all about tokens and why they explain the limits and prices of AI tools. For now, “word” is close enough.

How it learned: one guessing game, played an enormous number of times

So where do those probabilities come from? Nobody wrote a rule that says “after ‘because it was’, ‘tired’ is likely”.

The model learned them by reading, and by playing one game over and over.

First, the company collects a huge amount of text: books, websites, articles, code, and more. Far more than a person could read in many lifetimes.

Then the game. Take a piece of that text. Hide the next token. Ask the model to give its probabilities. Check how much probability it gave to the token that actually came next. Then nudge the model slightly so that next time it would give the right answer a bit more. Guess, check, adjust. Repeat billions of times, across the whole pile of text.

This first, long phase is called pre-training. It’s the “P” in GPT: Generative Pre-trained Transformer. “Generative” because it generates text. “Transformer” is the name of the design inside the model, introduced by Google researchers in a 2017 paper called Attention Is All You Need. We’ll open it up in part 5.

Something surprising comes out of the game

Here’s the part that still surprises researchers. To get really good at guessing the next token in a recipe, a contract, a news article or a joke, the model has to pick up a lot more than word order. It picks up grammar. Facts that appear often. Styles of writing. Patterns that look like reasoning. Nobody typed those rules in. They came out of the guessing game, because they help with the guessing.

An analogy: imagine someone who has read every cookbook ever written but has never tasted food. Ask them for a new recipe and they can write one that sounds delicious and follows all the patterns of a real recipe. Often it will actually work. But they have no tongue to check it with. Their skill is knowing what recipes look like, not knowing what tastes good.

Where the analogy breaks: the cookbook reader at least knows which recipes they’ve read. The model doesn’t keep the books. It can’t point back to “page 41 of the book I read”. That matters a lot, and we’ll come back to it.

The reading stops at a date

The text is collected up to some point in time, called the training cutoff. The model knows nothing about what happened after that unless you tell it, or the app looks it up for it. That’s why a model can be confidently out of date about last month’s news or today’s prices.

What the model actually is: a huge set of learned numbers

Here’s the part people find hardest to picture. When I say “the model”, what is it, physically?

It’s a very long list of numbers. These numbers are called parameters (or weights). The model works by taking your tokens, turning them into numbers, and then doing a very large amount of multiply and add with its parameters. Layer after layer. At the end, out comes a score for every possible next token.

Think of the parameters as dials. Each “adjust” step in the training game turns millions of dials by a tiny amount. After training, the dials are set, and those settings are everything the model learned. There’s no separate place where the facts are kept. No text files, no list of capitals. Just the dial settings.

How many dials? OpenAI’s GPT-3, published in 2020, had 175 billion parameters. Many companies no longer publish the size of their newest models, so I won’t guess at today’s numbers. It’s enough to know it’s in the billions, and that training a large model takes months on thousands of specialised chips. That’s why only a handful of companies build these from scratch.

Where the “dials” picture breaks: no single dial means “Paris” or “cat”. What the model knows is spread across huge numbers of them working together, which is part of why it’s hard to look inside a model and say exactly why it gave a particular answer.

If you want to see what adjusting a small model looks like in practice, I wrote about fine-tuning a small GPT-2 model myself in 2025.

From base model to assistant

A model that has only been through pre-training is called a base model. It’s a brilliant text continuer, but it isn’t an assistant yet.

Ask a base model “What’s the capital of France?” and it might reply “What’s the capital of Spain? What’s the capital of Italy?”, because on the internet that question often appears in a list of quiz questions. It’s continuing the text, not answering you.

So there’s a second phase of training. People write examples of good answers to many kinds of requests, and the model is trained further on those (fine-tuning). Then people compare several of the model’s answers to the same prompt and rank them from best to worst, and the model is trained to produce more answers like the highly ranked ones (reinforcement learning from human feedback, or RLHF). OpenAI described this recipe in its 2022 InstructGPT paper, and said ChatGPT was trained with the same methods. One striking result from that paper: people preferred the answers of a fine-tuned 1.3-billion-parameter model over the original 175-billion-parameter GPT-3, despite it being about 100 times smaller. The second phase matters a lot. Modern assistants use more elaborate versions of this step, which a later series will cover in detail.

Underneath the friendly assistant, though, the same machine is running: the same loop of predicting the next token. Fine-tuning changes which continuations are likely (helpful answers instead of more quiz questions). It doesn’t change what the model is.

Why it can be confidently wrong

Put the pieces together and one consequence follows.

The model produces likely-sounding text. Usually, likely text and true text are the same thing: after “The capital of France is”, the likely next word is the true one, because that sentence appears correctly all over the training text. But when the truth is rare, obscure, recent or not in the training text at all, the model doesn’t stop. There’s no “I don’t know” alarm built into the loop. It keeps producing whatever an answer would most plausibly look like.

That’s what people call a hallucination: fluent, confident text that isn’t true.

The most famous example is a real court case. In 2023, in Mata v. Avianca in New York, a lawyer used ChatGPT to research a brief, and the brief, filed by his colleague, cited six earlier court decisions that didn’t exist. When the lawyer asked ChatGPT whether one of the cases was real, it said yes. On June 22, 2023, the judge sanctioned the two lawyers and their firm, with a $5,000 penalty. (The video version says “two lawyers used ChatGPT”; to be precise, one did the research with it, and both were sanctioned.)

ChatGPT wasn’t lying. Lying means knowing the truth and saying something else. It was doing exactly what it was built to do: produce text that looks like a good answer. A list of legal citations has a very recognisable shape, and it produced that shape.

Why does it sound so sure? Partly because the second training phase rewards answers that sound like good, confident answers. And researchers at OpenAI argued in a 2025 paper, Why Language Models Hallucinate, that the way models are trained and tested rewards guessing over saying “I’m not sure”, much like a multiple-choice exam where a blank answer scores zero but a guess might score a point. So models learn that a confident guess is the better bet.

The practical rule is simple: sounding right is not the same as being right. Check anything that matters. I’ve collected some real business examples of what happens when nobody checks in 5 AI Fails Every Business Should Know.

What it is not

Two misunderstandings cause most of the trouble people have with these tools.

It’s not a database

A database stores records and gives you back exactly what was put in. Ask for customer 1042 and you get customer 1042, or “not found”.

A language model stores nothing like that. It has dial settings that make some continuations more likely than others. When it gets a fact right, it’s because the right continuation was the likely one, not because it retrieved a record. That’s why it can be right about famous facts and wrong about less famous ones in exactly the same confident tone. And it’s why “not found” doesn’t exist for it.

It’s not searching, unless a tool is used

The base loop never goes online. It only sees the text in front of it.

Some apps add tools on top. ChatGPT, for example, has a search feature (OpenAI launched it in October 2024) where the app runs a web search, pastes the results into the conversation, and the model writes its answer from them, with links. Other assistants have similar features. The model is still doing next-token prediction. The difference is that the app has put fresh, relevant text in front of it first. That helps a lot with accuracy, and it lets you check the sources, but the model can still misread or misquote what it found.

When a model calls tools and acts on the results in steps, you’re getting close to what people call an agent. I explain the difference in What an “AI Agent” Actually Is, Without the Hype.

It only knows what’s in front of it

One more consequence of “it only sees the text in front of it”. The model has no memory of you between conversations, apart from whatever notes the app saves and pastes back in. Within a conversation, it re-reads the whole chat each time it predicts the next token. But there’s a limit to how much text it can take in at once, called the context window, and part 2 explains it.

That’s also why giving it more detail works so well. The more of the right text you put in front of it, the more the likely continuation matches what you want. That’s the whole idea behind the 5-part prompt recipe.

The whole picture in five lines

• A language model does one thing: given some text, it gives a probability for every possible next token.
• An app turns that into an answer by picking a token, adding it, and repeating.
• The probabilities come from billions of learned numbers, set by playing “guess the next token” across a huge amount of text.
• A second round of training with human-written examples and rankings turns that text continuer into a helpful assistant.
• It produces likely text, not checked facts. It’s not a database, and it doesn’t search unless the app gives it a tool to do so.

None of this makes it less useful. It’s an extraordinary writer, summariser and first-draft machine. It just helps to know what you’re holding: a very, very good autocomplete, with you as the editor.

What’s next in the series

Over the next four parts we’ll follow “The cat sat on the mat because it was tired.” through the model, one step at a time:

• Part 2, What’s a Token, and Why AI Has Limits: how the sentence gets chopped into pieces, and why that sets the limits and prices of AI tools.
• Part 3, How AI Turns Words Into Numbers: Embeddings Explained: how each piece becomes a list of numbers that captures its meaning.
• Part 4, How AI Knows What “It” Means: Attention Explained: how the model works out that “it” refers to the cat, not the mat.
• Part 5, Inside a Transformer: One Word’s Journey to the Next Word: all the steps together, layer by layer, until “tired” comes out the other end.

Next: What’s a Token, and Why AI Has Limits.

Sources: OpenAI, “Introducing ChatGPT”, Nov 30, 2022 (launch date; trained with RLHF using the same methods as InstructGPT); Sam Altman on X, Dec 5, 2022 (1 million users); Reuters, “ChatGPT sets record for fastest-growing user base - analyst note”, Feb 1, 2023 (UBS estimate of 100 million monthly users); Radford et al., “Improving Language Understanding by Generative Pre-Training”, OpenAI, 2018; Vaswani et al., “Attention Is All You Need”, 2017; Brown et al., “Language Models are Few-Shot Learners”, 2020 (GPT-3, 175 billion parameters); Ouyang et al., “Training language models to follow instructions with human feedback”, 2022 (InstructGPT); Kalai et al., “Why Language Models Hallucinate”, 2025; Mata v. Avianca, Inc., No. 22-cv-1461 (S.D.N.Y. June 22, 2023), opinion and order on sanctions, and Bloomberg Law coverage; OpenAI, “Introducing ChatGPT search”, Oct 31, 2024; OpenAI API reference, temperature parameter. All checked October 2026. Probabilities in the example and diagram are illustrative.

Related: The Rise of Generative AI: Simplifying Learning, and Treat the LLM Like an Unreliable API.