Transformers
2 essays on this topic.
Inside a Transformer: One Word's Journey to the Next Word
Follow one word through a language model, from token ID to the final pick: embeddings, a stack of layers, attention, feed-forward memory and probabilities.
How AI Knows What "It" Means: Attention Explained
How does AI know what "it" refers to? A plain-English guide to attention: queries, keys and values, softmax, heads, masking and why long context costs more.
