RIAI: Toward a Terminal Agent That Writes Its Own Skills

“Woke up in the spring of year 1 as one of the first two humans, knowing nothing but how to pick things up, eat berries and drink.”

That line was written by one of my programs, not by me. It’s the first entry in a biography the simulation keeps for one family line, generation after generation, for 150 simulated years. By the end of that run, the descendants of those two people had invented ropes, bows, wheels, metal, writing and a library. They had also forgotten how to make an axe at least once, because everyone who knew died before teaching anyone.

That simulation is one small part of a research project I call RIAI. This post is the first in a series of lab notes about it: what I’m building, the rules I set for myself, and what actually happened when I ran it. The wins are in here, and so are the failures. There are quite a few failures.

The question

Every experiment in RIAI asks the same thing, in different settings:

Can something that starts out knowing nothing work out a world by itself, through curiosity, memory and skills it builds itself, and then pass that knowledge on?

The title of the very first prototype says it in four words: Born Knowing Nothing.

Most AI systems today learn the other way. A large language model starts with a huge amount of human knowledge poured in during training, then gets shaped into an assistant. That works remarkably well. But it leaves an open question: what does it take to discover knowledge rather than absorb it? What does a mind need in order to get better on its own: to invent, remember, reuse, and teach?

RIAI is my workbench for that question. It’s small: it runs on a laptop and a sandboxed virtual machine. The aim is to find out, honestly, which ideas actually help a mind improve itself, and which just sound like they should.

Three rules I hold myself to

It’s easy to fool yourself with this kind of project. A demo looks clever, the agent seems to be “thinking”, and you stop measuring. So from early on, every experiment follows three rules.

1. The mind sees only what a player would see. In the game experiments, the agent gets pixels, any text the game shows, and a reward. It presses numbered buttons whose meaning it isn’t told. The code says it plainly: anything else the world knows, like achievement names, “is only for the dashboard, never for the mind”. In the computer experiments, the agent sees the screen and can only press mouse buttons and keys.

2. Every result races a baseline at equal steps. The benchmark file describes itself as “an honest ruler for every agent”, and its random player is “the floor every agent must beat”. If a clever new idea can’t beat pressing random buttons for the same number of steps, it doesn’t count.

3. The agent never grades its own homework. On the virtual machine, results are checked by the host through a separate channel the AI never gets to use. When the agent says “done”, something else confirms it. That turned out to matter a lot, as you’ll see.

These rules make the numbers smaller and less exciting. They also make them real.

Where I want this to go

All of these experiments point at one goal: an agent that does everything from the terminal, and develops itself by writing its own Python code.

The terminal is the right home for that. Almost everything a computer can do is reachable from a command line: files, programs, data, the network, other tools. An agent that truly understands the terminal doesn’t need a special integration for each task. It needs to know what commands do, and how to make new ones.

The second half is the important part. When the agent meets something it can’t do yet, it shouldn’t just fail or ask for help. It should write a small Python program for it, test it, and keep it as a new skill. The next time a similar job comes up, the skill is already there. Skills get built on top of skills.

What I expect, and what I’m testing for, is that this compounds. Every new skill makes the next problem a little easier, so the agent keeps developing, more and more, instead of hitting a ceiling set by whoever built it.

That’s the ambition. The rest of this post is the honest version of how far it has got.

The four stages

The project moved through four settings, each harder and closer to the real world than the last.

The four stages of RIAI so far, with one headline result each. Every number comes from the run reports.

Stage 1: Simulated people who invent, teach and forget

The early prototypes were artificial life: creatures with genes that mutate, species that split apart, cells that evolve with no rewards at all. They taught me a humbling lesson early. One run logged “first animals” and “life conquers the land” in its early years, then ended with an average body of about one cell and zero animals. Evolution found the simplest thing that worked and stayed there.

The current version, Discovery, is about people. They’re born knowing only instincts. The world hides a tree of 41 inventions in 8 layers (axe, then logs, then campfire, then cooking; clay tablet and charcoal, then writing, then library), and nobody knows a single recipe. Skills are built from skills: you can only learn to make something once you know what it’s made from. Curious people experiment. They teach their families, learn by watching, and once writing exists, they can read at a library, so knowledge can outlive the person who found it. Traits like curiosity and learning speed are inherited with mutation.

In one 150-year run: 113 discoveries, 82 of them brand-new items the world generated on its own. The campfire didn’t appear until year 43. Writing arrived in year 56 and the first library in year 57. Before that library existed, the biography recorded skills being lost and rediscovered 29 times. After it, newborns learned dozens of skills by reading. Some things never happened at all: no house, no book, no school, no mill.

Stage 2: One mind, any game

Next I wanted a single mind that could play games it had never seen, from pixels alone. The design is inspired by Go-Explore, a research approach that cracked famously hard exploration games by remembering interesting places and going back to them. My version remembers each screen as a tiny colour sketch, returns to rare or promising moments, and turns whatever led to an event into a reusable skill. Skills can contain other skills.

It reached 13 of 22 achievements in Crafter, a 2D Minecraft-like benchmark, and solved 7 of 10 MiniGrid puzzles.

The clearest result came from a game generator that invents endless new games, with random layouts, colours and rules: keys and doors, switches, lava, enemies, teleporters. Three versions of the mind played the same 130 games. The typical number of steps to solve a game was:

1,285 for the plain mind, starting fresh each game
326 for a mind that perceives objects, starting fresh
94 for a mind that perceives objects and keeps its experience between games

Seeing objects made it about 4 times faster. Keeping experience made it another 3.5 times faster on top. That’s the core idea of self-improvement in its simplest form: getting better at new things because of what you learned on old ones.

Stage 3: A real computer

Games are clean. Computers aren’t. So I built an isolated Linux virtual machine with no shared folders, kept offline unless an experiment needs the network, where the agent can only see the screen and press mouse buttons and keys, and the host checks the results.

Here the experiments split into two kinds. Some use a language model as a teacher, and some try to do without one. A Claude model driving the desktop finished 5 of 5 simple tasks. Curiosity alone, with no language at all, explored 73 different screens but completed 0 of 5 tasks. A cheaper model, Claude Haiku, built a library of 15 reusable skills for itself, and its own judgement of “did that work?” matched the hidden host check 10 out of 10 times. A small neural network of my own, taught by Haiku only when it was unsure, slowly took over more of the work.

Stage 4: A terminal that learns what commands do (the main line)

This is where the project is heading, and where most of the recent work is. Instead of being told what sort or grep do, the agent experiments. For each command it keeps a list of possible effects and crosses off every one that disagrees with what it observes, including its failures. Curiosity sends it to practise the commands it’s least sure about. Then it plans in its imagination before acting for real.

By the latest run it understood 35 of 39 commands, predicted their effects with 98% accuracy, and passed a 20-task exam, including small coding tasks, with zero calls to a language model. When no command could do a job, it got a new Python tool written and added it as a new command. I want to be precise about that: Haiku wrote those tools, not the agent. The one place where code is built with no LLM at all is a separate coder that searches over small building blocks, guided by a world model it learned by running those blocks. It solved 13 of 13 short tasks.

Where the self-writing part stands today

Here’s the honest status of the core idea, the agent writing its own code to grow its skills:

It already grows its own command set. When nothing it knows can do a job, it gets a Python tool written, tests it in the sandbox, and registers it as a new command. Then it learns what that command does by trying it, like any other command. Eleven tools have been added this way, things like averaging a CSV column or finding the most common word in a file. On the harder exam, new coding tasks went from 0 of 4 solved in one run to 4 of 4 in the next. A later method, writing each tool one small step at a time and testing every step on real sample input, passed 13 of 15 host-checked tool tasks.
But today, a language model writes that code. Claude Haiku writes the tools, and when it translates a task it also flags that a new tool is needed. The agent gets each tool tested, adds it as a command and learns to use it, but it doesn’t write the code itself yet.
The first code with no LLM at all. A separate coder builds programs by searching over small building blocks, guided by a world model it learned by running those blocks tens of thousands of times. It solved 13 of 13 short tasks, with no language model writing a line.
The next step is a code world model. I’ve collected line-by-line traces of 61,611 programs running in the sandbox, nearly a million traced lines, including the agent’s own tools. The aim is to train a small model that predicts exactly what each line of code will do. With that, the agent can imagine and check its own programs before running them, and write more of its code itself.

So the loop exists: find a gap, get a skill written, test it, keep it, use it. Moving the “write it” step from a teacher model into the agent itself is the main work ahead.

Many ways to improve yourself

Looking across the stages, RIAI ended up testing a surprising number of different ways a mind can get better on its own:

Skills built from skills, so knowledge stacks layer on layer
Curiosity: going back to rare and surprising moments, and practising what you’re least sure of
Experience carried across tasks: what buttons do, which kinds of things are dangerous
Culture: teaching, watching, and writing things down so knowledge outlives the learner
Evolution: inheriting traits like curiosity, with mutation
World models: learning to predict what an action will do, then planning in imagination
Teacher and student: a strong model teaches a small one only when the small one is unsure
A self-made skill library that keeps skills which work and replaces them when a shorter way turns up
Growing new abilities: adding tools as new commands, then learning what they do by trying them

Some of these helped a lot. Some did nothing. A few made things worse.

The failures are the point

I’ll write a full post on what didn’t work, because I think it’s the most useful part. A preview:

A planner that imagined several steps ahead scored 9.0 achievements in Crafter. The plain mind without it scored 9.3 at the same number of steps.
Twenty agents pooling their knowledge as a “culture” did worse than twenty agents alone: 6.8 against 7.1.
On MiniGrid puzzles in the benchmark, random button-pressing solved more puzzles than any of my minds.
A subgoal planner meant to help the student on the computer took it from 2 of 5 tasks down to 0 of 5.

Every one of those was an idea that sounded smart. The equal-steps rule caught them. Without it, I’d have written a much more exciting and much less true blog post.

How this fits the bigger picture

If you’ve read my series The Road to AGI, RIAI sits among what I called the other bets: open-ended learning, curiosity, evolution and culture, rather than simply training a bigger model on more text. It also borrows heavily from ideas in world models: learn how the world responds, then think before you act.

RIAI isn’t a claim about a path to anything grand. It’s a lab notebook: small experiments, measured honestly, about what “learning on its own” really requires.

What’s next in this series

The next lab notes go deeper, starting with the main line:

A terminal that learns what commands do, by experimenting
Writing its own tools: from a teacher model to code built by search
Training a code world model from 61,611 program traces
My failure log: when smarter ideas made it worse
The earlier stages: 150 years of simulated invention, and one mind for any game

Key ideas

The goal is a terminal agent that develops itself: it does everything from the command line, and writes Python code to build new skills, so it keeps growing.
RIAI asks one question in many settings: can a mind that starts from nothing work out a world by itself and keep what it learns?
Three rules keep it honest: the mind sees only what a player sees, every result races a baseline at equal steps, and a separate checker confirms success.
The clearest win so far is transfer: keeping experience between new games cut typical solve time from 326 steps to 94.
Culture matters in simulation: before writing and libraries, skills were lost and rediscovered 29 times.
The self-writing loop already runs, but a teacher model writes the code today. Search-built code (13/13) and a code world model trained on 61,611 traces are the path to the agent writing its own.
Many clever ideas failed the baseline test, and those failures are the most useful results.