How an AI writes its next word
Know what to build. Then build it.
A little sapling named Yagrumo shows you how an AI writes, one word at a time. And what happens when it doesn't know the answer.
Text is split into pieces, often whole words and sometimes parts of words. Each piece maps to a number the model can work with.
Each token becomes a long list of numbers: a position in a space of meaning. Similar meanings sit close together.
Every token weighs how much every other token matters to it. Dozens of layers repeat this, refining what the sentence means.
The model scores every word it knows and turns the scores into probabilities. It picks one, adds it, and repeats.
When the answer isn't in what it learned, a model can still produce a fluent, confident guess. Fluent is not the same as true.
Before answering, the system looks up facts in your own data and hands them to the model, so the answer can cite where it came from.