What a Language Model Actually Does
I asked a model to use a function from a library I maintain. It handed back clean, idiomatic code that called a function named parseConfig with a strict option. Looked perfect. Except that function has never existed. Not deprecated, not renamed, not moved to another package. Never shipped.
The model had not looked it up in some index of my library and misremembered. It had never looked anything up at all. And here is the part that trips up every engineer new to this: the confident wrong answer and a correct answer come out of the exact same machine running the exact same operation. There is no separate “getting it right” code path and “making it up” code path. It is one operation, all the way down.
Once you see that operation, both the fear and the mystique drain out of the whole subject, and what is left is something you can engineer around. That is the entire goal of this article. Not to sell you on AI, and not to warn you off it. Just to show you, precisely, what the thing does when you call it, so the rest of this part is building on rock instead of vibes.
Two popular takes are both wrong in ways that will cost you. “It is just fancy autocomplete” is dismissive enough that you will underestimate what a loop around it can do. “It understands and thinks” is credulous enough that you will trust it with things it cannot do. The truth is duller than either and far more useful.
One function, called in a loop
Strip away chat, strip away the agent frameworks, strip away the branding. Underneath, a language model is a pure function. You give it a sequence of tokens (think of a token as a chunk of text, roughly a short word or piece of one) and it returns a number for every possible next token. That is it. That is the whole interface.
// A language model, stripped to its essence:
type Tokens = number[];
type Model = (context: Tokens) => Float32Array; // one score per vocabulary entryThat Float32Array has one slot for every token the model knows, often 50,000 to 200,000 of them. After a quick normalisation step those scores become a probability distribution: a list of numbers that are all positive and add up to 1, one probability per candidate next token.
Feed it the tokens for The sky is and you get back something like this. Blue is the runaway favourite. Grey and dark are plausible. “Convertible” is a real English word that almost never follows “the sky is”, so it sits near zero, along with tens of thousands of others.
Two things about that picture matter more than they look. First, the model does not pick a word. It produces the whole distribution, every candidate at once, and leaves the picking to a separate step you control. Second, “convertible” is not an error. The model is not confused about the sky. A tiny non-zero probability on an absurd word is completely normal, and it is exactly what lets the model occasionally surprise you, for better and worse.
That separate picking step is called sampling, and it is where knobs like temperature live. It gets its own article, because how you pick changes everything from “boringly reliable” to “unhinged”. For now just know the two stages are distinct: the model gives you a distribution, then something you configure turns that distribution into one actual token.
Here is the loop, in plain JavaScript, with the model as a black box:
let tokens = encode("The sky is");
while (true) {
const scores = model(tokens); // one forward pass: a score per token
const probs = softmax(scores); // turn scores into a probability distribution
const next = sample(probs); // pick ONE token (temperature lives here)
tokens.push(next); // append it to the sequence
if (next === END_TOKEN) break; // the model can vote to stop
}That is not a simplification of how text gets generated. That is how text gets generated. A model producing a 600-word answer ran that loop 800-ish times, each pass reconsidering the entire sequence so far to decide the next single token. When you watch an answer stream in word by word, you are literally watching this loop turn.
The softmax step is the one bit of maths worth seeing once, because it is genuinely simple. It takes the raw scores (which can be any number, positive or negative) and squashes them into a clean probability distribution:
function softmax(scores) {
const max = Math.max(...scores); // subtract the max for numerical stability
const exps = scores.map((s) => Math.exp(s - max));
const sum = exps.reduce((a, b) => a + b, 0);
return exps.map((e) => e / sum); // positive, and they sum to 1
}Play with the loop yourself. This is a toy: a hardcoded little table stands in for a real model, so no network, no API, no billions of weights. But the mechanic is honest. Pick a starting phrase, sample one token at a time, and watch the sentence build. The slider is a stand-in for temperature: drag it low and the top choice dominates, drag it high and the tail gets a real chance, which is how the same prompt can produce a safe answer or a weird one.
Notice what “high temperature” does and does not do. It does not make the toy smarter. It just gives lower-probability tokens a better shot, so 2 + 2 = can slide off 4 and land on five. That is the honest shape of the tradeoff you will make constantly: predictability versus variety, tuned by how you sample, not by how clever the model is.
Autoregressive, which just means it eats its own output
The loop has a name: autoregressive generation. Ignore the syllables. It means the model’s own output becomes part of its next input. Step one predicts a token, you glue it onto the sequence, and step two runs against the longer sequence including the token you just added. Every token is generated in the shadow of every token before it.
Sit with that last line, because it is the source of a lot of confusion later. The model has no memory between steps other than the sequence itself. Everything it “knows” about this conversation is in the tokens you feed it this pass. There is no hidden notebook. When people say a chat model “remembers” what you said earlier, what they mean is that the whole prior conversation gets re-sent as tokens on every single call. That is not a metaphor. It is the mechanism, and it has real consequences for cost and correctness that show up in the context window and conversation memory.
A chat model, by the way, is not a different kind of thing. It is this same next-token function with a formatting convention on top: your system prompt, the user’s messages and the assistant’s past replies get laid out as one long token sequence, and the model continues it. The reply you read back was generated one token at a time by the loop above. That formatting convention is worth its own article, system, user and assistant roles.
Where the numbers come from
You do not need to know how a transformer works to use one well, the same way you can drive without knowing the firing order of the cylinders. But the shape of the computation clears up a few mysteries, so here is the one-paragraph version.
Your tokens go in as numbers. They pass through a very large stack of matrix multiplications whose values (the weights) were fixed ahead of time. Out the other end comes one raw score, called a logit, for every token in the vocabulary. Softmax turns those logits into the probability distribution you already saw. That entire trip, tokens to logits, is called a forward pass, and it is the unit of work you are paying for on every request.
The important word in that diagram is “fixed”. The weights do not move while the model answers you. They were set during training and then frozen. Which brings us to the split that clears up most billing and capability confusion at once.
Trained once, run forever
There are two completely different activities that both get called “the model”, and conflating them is behind a lot of muddled thinking.
Training is where those billions of weights get their values. A provider takes an enormous pile of text, runs a version of the next-token loop over it billions of times, and each time nudges the weights so the model’s predicted distribution lines up a little better with the token that actually came next. Do that at sufficient scale and general competence with language falls out. This is staggeringly expensive, takes weeks to months on rooms full of specialised hardware, and happens exactly once per model version. It finished before you ever typed your first prompt. You do not do this, you do not pay for it directly, and day to day it is not your concern.
Inference is the part you touch. It is that single forward pass, using the frozen weights, run once per token you generate. It is cheap compared to training but not free, it is what your bill is measured in, and it is where every engineering decision in this part lives.
Parameters, and why bigger is not automatically better
A model’s size is quoted in parameters, which is just the count of those weights: 8 billion, 70 billion, and up. More parameters mean more capacity to absorb patterns during training, so for a long time the headline number tracked capability well enough that people used it as a proxy for “smart”.
That proxy has broken, and you should stop trusting it. As of 2026 a lot of strong models use a mixture-of-experts design, where only a fraction of the weights fire for any given token. A model might advertise hundreds of billions of total parameters but only put a few tens of billions to work per token. One widely-discussed open model carries 671 billion parameters total and activates roughly 37 billion of them per token. The total drives training cost and how much the model can learn. The much smaller active count drives what you actually pay and wait for at inference. Quoting the total as if it were the running cost is the single most common way people get this wrong.
The practical upshot: a well-trained smaller model often beats a clumsy bigger one on real tasks, and it does so faster and cheaper. Do not pick a model by its parameter count. Pick it by measuring it on your actual task, which is the whole discipline of evals. This landscape shifts every few months, so treat any specific number here as a snapshot, not a law.
It is not a database
Here is the frame that will save you the most grief. A language model is not a lookup table with a natural-language front end. There is no row somewhere that stores “the capital of France is Paris” that the model retrieves. What it has instead is billions of weights that, together, make “Paris” an overwhelmingly likely continuation of “the capital of France is”. Those are not the same thing, and the difference is the whole reason hallucinations exist.
When you ask a factual question, the model does not check whether it knows the answer and then either tell you or decline. It cannot do that, because there is no separate knowing. It produces the most plausible continuation, always. If the fact was common in its training data, the plausible continuation is also the correct one, and everything looks like retrieval. If the fact was rare, contradictory, or simply never present, the model still produces a plausible continuation, because that is the only thing it can do. That continuation is fluent, confident, and wrong.
Why does the honest “I am not sure” score so low? Recent work on this points at how models are trained and graded. Pretraining rewards producing text that looks like the training data, and confident factual sentences are what that data mostly looks like. Then the benchmarks used to rank models tend to score a wrong answer and an “I do not know” identically, both as zero, while a lucky guess scores full marks. Under that scoring, guessing is the optimal strategy, exactly like a student bluffing on an exam where blank answers and wrong answers are penalised the same. So models learn to guess fluently rather than hedge. Hallucination is not a bug someone forgot to fix. It is a direct consequence of the objective, and you manage it rather than wait for it to be patched.
This reframing changes how you build. You stop expecting the model to police its own accuracy, and you start engineering the accuracy in from outside: give it the real source text to work from (retrieval), let it call code and APIs for facts it should not guess (tool calling), and measure how often it is right on your task (evals) instead of trusting a vibe. Whole chapters of this part are, in effect, tactics for compensating for this one property.
What it cannot do on its own
Line up everything above and a short list of hard limits falls out. None of these are failures of a particular model. They are consequences of the mechanism, true of every chat model, and no version bump erases them. Your job as the engineer is to know them cold and design around them.
- It does not know today’s date, or anything after its training cutoff. The weights froze at some point in the past. Ask it what happened last week and it will either admit the gap or, worse, invent a plausible answer. Give it the current date and any fresh facts explicitly, in the prompt.
- It cannot do reliable arithmetic. It predicts the tokens of an answer, it does not calculate. Small sums it has effectively memorised. Multiply two large numbers and it pattern-matches its way to something that looks right and often is not. For real maths, hand it a calculator through tool calling or code execution.
- It cannot see your data. It only knows what was in training plus what you put in this prompt. Your database, your docs, your user’s account: all invisible unless you retrieve the relevant pieces and include them.
- It does not remember the last conversation. Every call starts cold. Continuity is an illusion you create by re-sending history, which spends real budget and eventually strains the context window. Persisting anything durable is a system you build, covered in conversation memory.
Everything else in Part 6 builds on this one page. Tokens and cost, the context window, tools and agents, retrieval, memory, evals: every one of them is a technique for feeding the loop better inputs, controlling how it samples, or checking its outputs. Keep the core picture in your head and the rest stops feeling like magic. Tokens in, a probability for every next token out, sample one, append, repeat.
Summary
- A language model is one function: given a sequence of tokens, it returns a probability for every possible next token. That is the entire operation.
- Generation is that function in a loop. Sample a token, append it, feed the whole longer sequence back in, and repeat. This is autoregressive generation, and chat, code and agents are all just this loop.
- Sampling is a separate step you control. The model hands you a distribution; how you pick from it (temperature and friends) decides how reliable or how varied the output is.
- Training sets the billions of weights once, offline, at huge cost, and is the provider’s job. Inference is the forward pass through those frozen weights that you run and pay for on every request.
- Parameter count is a weak proxy for quality. Mixture-of-experts models activate only a fraction of their weights per token, so measure a model on your task instead of trusting its size.
- It is not a database. A hallucination is a high-probability continuation that happens to be false, produced by the same mechanism as every correct answer, and reinforced by training that rewards confident guessing.
- On its own it cannot know the date, do reliable arithmetic, see your data, or remember past conversations. The rest of this part is the engineering that compensates for those gaps.