Index / Writing / Page 3
Notes on building things.
Business, startups, product, software architecture and applied AI. 105 posts. Subscribe via RSS.
Page 3
25–36 of 105Inside a Transformer: One Word's Journey to the Next Word
Follow one word through a language model, from token ID to the final pick: embeddings, a stack of layers, attention, feed-forward memory and probabilities.

Write It, Then Prove It: AI That Checks Its Own Work With Math
AI paired with proof checkers like Lean: Olympiad medals, Fermat's Last Theorem, verified code, and the catch: a proof only proves what the spec says.

How AI Knows What "It" Means: Attention Explained
How does AI know what "it" refers to? A plain-English guide to attention: queries, keys and values, softmax, heads, masking and why long context costs more.

How AI Turns Words Into Numbers: Embeddings Explained
A token ID is just a label. An embedding is a learned list of numbers that puts each word on a map of meaning. Here's how it works, and where it shows up.

What's a Token, and Why AI Has Limits
AI doesn't read letters or words. It reads tokens. A plain-English guide to tokens, token IDs, context windows, per-token pricing and the strawberry problem.

How ChatGPT Actually Works, Explained Simply
ChatGPT, Claude and Gemini predict the next word, one piece at a time. A plain-English guide to how they learn, why they sound confident when wrong, and what they are not.

How Many Levels of AI Are There? Every Ladder Compared
ANI to ASI, the four types, OpenAI's five levels, DeepMind's grid, agent autonomy and safety levels: six AI ladders compared, and where AI sits in October 2026.

Watching JEPA Write Code Live: What It Got Right, and the Check It Fooled
RIAI Lab Notes #5: a live test of the JEPA code writer. Fast, readable, no LLM, and correct for its examples. But the examples didn't say what was meant.

Next: Teaching an Agent to Imagine Code With JEPA
RIAI Lab Notes #4: why exact-value prediction hit a wall, how a JEPA plans code in vector space, and first results: 10 of 12 when grounded.

Why Most Small Service Businesses Charge Too Little
Small service owners price by fear and by competitors, not by cost. A worked example of the real hourly rate, and how to raise prices without drama.

Writing Its Own Code: From a Teacher Model to Code Built by Search
RIAI Lab Notes #3: moving the 'write the code' step from Claude Haiku into the agent, with program search, execution traces and honest gaps.

A Terminal That Learns What Commands Do
RIAI Lab Notes #2: an agent that learns what sort, grep and its own Python tools do by experimenting, then plans in its imagination.
