My LibraryBuy PDF
Part 7/The Pipeline

TurboFan: The Optimising Tier

Write a + b and you have written one of the fussiest operations in the language. Before it can add anything, the engine has to ask: are these two numbers? Two strings to glue together? Is one of them an object that needs converting to a primitive first, which can run your code in the middle of an addition? A BigInt, which refuses to mix with a plain number? The interpreter asks all of that every single time the line runs.

Now profile a tight numeric loop that does nothing but add. In the machine code V8 is actually running, that same + has collapsed into a single CPU instruction. No questions, no branches, no boxing. The engine did not find a clever shortcut through the specification. It placed a bet.

This lesson is about the tier that places those bets. When a function gets genuinely hot, V8 hands it to TurboFan, the top of the four-tier pipeline, and TurboFan produces the fastest code the engine can make. It does that by assuming your program keeps doing what it has been doing. Understanding exactly what it assumes is the whole game, because those assumptions are the reason some code is quick and some code falls off a cliff the moment its data changes shape.

What optimising actually buys

Start with the add, because it is the clearest case. The interpreter’s handler for + is generic: one piece of code that has to be correct for every possible pair of operands. So it branches on the types every time, works with values that are boxed and tagged so it can tell a number from a pointer, and allocates a fresh box for the numeric result. It is a small decision tree, walked on every add.

TurboFan has something the interpreter never had while it was writing that handler: a record of what actually showed up here. The feedback the interpreter gathered says this add only ever saw numbers. So TurboFan stops writing code for the general case and writes code for that case. Check once that the inputs really are numbers, then a single machine add on raw, unboxed values.

the same a + b, two waysgeneric: the interpreter1 both small integers?2 else doubles? unbox both3 else a string? concatenate4 else object? ToPrimitive5 box the numeric resultdecided again on every addspecialised: TurboFanguard: are they numbers?cheap, often hoisted out of the loopone machine addoperands unboxed, kept in registersno per-operation type decisionsthe one bet already covered them
The same a + b, two ways. The interpreter walks a type decision on every add; optimised code guards once, then does one machine add.

That is the shape of every optimisation in this tier. Replace “figure out what to do, then do it” with “do the specific thing we already know is needed”. Property access gets the same treatment: instead of a generic lookup by name, TurboFan emits a quick check that the object still has the shape it always had, then a direct load from a fixed offset, the way a struct field works in C. The shapes and inline caches that make that possible are their own lessons. The pattern is identical to the add: a guard, then the fast direct thing.

Speculation: betting the future looks like the past

Notice what the feedback actually is. It is a record of the past. It says “every time we ran this, we saw numbers”, and TurboFan treats that as “we will keep seeing numbers”. That is a guess, not a proof. Nothing in JavaScript stops you from calling a number-crunching function with a string an hour into the process.

This is the word that defines the whole tier and the next two lessons: speculation. TurboFan optimises on the assumption that the future looks like the past, which is what lets it throw away all the general-case handling and emit tight code. But an assumption that might be wrong has to be checked, or the fast code would silently compute nonsense the first time reality diverged. So every specialised path is fronted by a cheap guard. While the guard passes, you run the fast code. When it fails, the fast code cannot handle the new case, so V8 discards it and drops back to the interpreter, which handles everything correctly. That fallback is deoptimisation, a lesson of its own.

a specialised path is a bet, and a bet is guardedfeedback: this + has only ever seen numbersa record of the past, not a promiseguardstill numbers?holdsbreaks (rare)specialised machine codeone add, no type checksfast, while the bet holdsdeoptimisethrow out the fast coderesume in the interpretera wrong guess is only slower, never incorrect
A specialised path is a bet, and a bet is guarded. While the values stay numbers the fast code runs; when they change, V8 deoptimises back to the interpreter.

The safety here is the important part, and it is easy to miss. Aggressive optimisation sounds dangerous, like it might trade correctness for speed. It does not. A broken assumption never yields a wrong answer, only a slower one, because the guard catches the divergence and hands control back to code that is always correct. That guarantee is what lets TurboFan be as reckless as it is with the common case.

Inlining: the big multiplier

Type specialisation is the base win. Inlining is the one that multiplies everything else, and if you only remember one TurboFan technique, remember this one.

Inlining pastes a called function’s body directly into its caller, so the call disappears. The obvious payoff is removing the call itself: setting up a stack frame, passing arguments, jumping, and returning all cost something, and for a tiny function called constantly that overhead can dwarf the actual work. But that is the smaller half of the story.

The bigger half is what inlining exposes. A function call is an opaque wall to the optimiser. It cannot see across it, so it has to assume the worst about what happens on the other side. Paste the body in and the wall is gone: now the caller’s values and the callee’s code sit in one place, and every other optimisation can reach across the old boundary.

function square(x) {
  return x * x;
}

// two calls to a tiny function
function lengthSquared(a, b) {
  return square(a) + square(b);
}
// what TurboFan effectively works with after inlining square
// (illustrative, not a literal dump)
function lengthSquared(a, b) {
  return a * a + b * b;
}

Once square is inlined, lengthSquared is a single arithmetic expression. TurboFan can specialise all of it on the number feedback at once, keep every intermediate value unboxed in a register, and emit a short straight line of machine code with no calls in it at all. That is the combination that makes idiomatic JavaScript fast: small functions, getters, helper methods, the way you actually like to write code. Inlining flattens the pile of tiny functions, and type feedback specialises the flattened result.

inlining: paste the body in, delete the callbeforelengthSquared(a, b)return square(a) + square(b);square(x)return x * x;call overheadframe, jump, returninlineafterlengthSquared(a, b), square gonereturn a * a + b * b;call removednow optimised as one expression
Inlining pastes the callee's body into the caller. The call overhead is gone, and the merged code becomes one expression the optimiser can work on as a whole.

Inlining is speculative too. TurboFan inlines the function it saw being called at that site, guarded so that if a different function turns up later, the fast code deopts. And it is bounded. The engine cannot inline everything, or the compiled code would explode in size and the compile would take forever, so it uses heuristics: small, hot callees get inlined, large or rarely-called ones do not. You do not control those heuristics directly, but you feed them by keeping call sites consistent, which is the same thing you do for every other optimisation here.

The optimisations you get once the door is open

With types pinned down and functions inlined, TurboFan can run the classic compiler optimisations that any textbook would recognise. None of them are exotic. What is interesting is that JavaScript normally cannot do them at all, because it never knows enough about the types until the feedback and the guards let it pretend it does.

constant folding
60 * 60 * 24
86400
compute what is knowable at compile time, once
strength reduction
n * 8
n << 3
swap a costly op for a cheaper equal one
redundancy elimination
a.x + a.x
t = a.x; t + t
read the field once, reuse the value
loop-invariant hoisting
loop: y = a*b every pass
y = a*b once, then loop
lift work that does not change out of the loop
Once speculation pins the types down, standard compiler optimisations apply: fold constants, cheapen operations, remove repeated work, and lift invariant work out of loops.

There are more of them. TurboFan removes dead code that can never run, and it eliminates checks it can prove are unnecessary: if it has already established that an array index is in range, it drops the redundant bounds check on the next access, and if it can prove an integer add cannot overflow, it skips the overflow handling. Each one is small. Stacked on top of inlined, type-specialised code running in a hot loop a few million times, they add up to the difference between “interpreted JavaScript” and “close to what you would get from a static language”.

The thing to hold onto is the dependency. All of these rest on the speculation underneath. Constant folding across an inlined function only works because the inline was allowed; the direct field load only works because a shape guard proved the field is where TurboFan thinks it is. Pull the type stability out and the whole stack collapses back to generic dispatch.

What goes in, what comes out, and what it costs

TurboFan’s inputs are the same two things the interpreter produced: the bytecode, which is the engine’s shared source of truth for what the function does, and the type feedback gathered while that bytecode ran, which is the record of what it did with. Out comes optimised machine code specialised for this one function and the types it saw. That is the trade in one sentence.

The catch is the price. Producing that code is genuinely expensive. TurboFan does deep analysis, builds and rewrites an intermediate representation of the whole function, and runs pass after pass over it, all of which takes real time and real memory. To keep that from freezing your program, V8 runs the compile on a background thread while the current, slower code keeps executing on the main thread; when the optimised version is ready, the engine swaps it in. And any of it can be thrown away in an instant if a guard fails.

what goes in, what comes out, what it costsbytecodethe shared source of truthtype feedbackwhat the interpreter sawTurboFanspecialise, inline, fold, hoistslow and memory-hungryruns on a background threadoptimisedmachine codedeopt discards it and falls back to the interpreterthe price is exactly why only the hottest code climbs this high
TurboFan reads bytecode plus type feedback and emits specialised machine code, at a real cost in time and memory, on a background thread. A failed guard deoptimises and discards it.

That cost is the entire reason the cheaper tiers exist. If optimising were free, V8 would optimise everything up front and skip the interpreter. It is not free, so the engine spends the effort only where it pays back: code hot enough that the machine-code speedup, multiplied by the number of times it runs, beats the compile bill. A function called three times is not worth a TurboFan compile and never gets one.

You can watch the specialisation pay off below. Both lanes sum the same array to the same total. The generic lane does the interpreter’s work per element, a small pile of steps for every add; the specialised lane does the one step TurboFan’s guard bought it. Step through and watch the work counters diverge. The step counts are a stylised simulation, not a measurement of real V8:

interactiveGeneric dispatch vs specialised add: count the work

Every engine speculates

The name TurboFan is V8’s. The idea is not. Once you accept that JavaScript can only be made fast by guessing at the types and guarding the guess, every serious engine ends up building an optimising compiler shaped like this one.

What this means for your code

You never call TurboFan and there is no annotation that asks for it. What you actually control is whether its bets are easy to keep. That is the entire practical takeaway, and it comes down to being predictable.

  • Give it stable types. A function called with numbers, then strings, then objects, gives its feedback nothing consistent to specialise on, and it stays generic or keeps deopting. The same function fed one steady type is the one that gets the single-instruction add. Consistency beats cleverness here, every time.
  • Give it stable shapes. Build your objects the same way each time so they share a hidden class, and property access can become a direct field load instead of a lookup. Objects assembled ad hoc, with properties added in different orders, defeat that. This is the subject of hidden classes and inline caches.
  • Do not hand-optimise against the tier. Tricks like x | 0 scattered everywhere, or manually inlining functions to “help”, mostly produce uglier code that a future engine makes pointless. Small, single-purpose functions with predictable arguments are exactly what inlining and specialisation reward. Write those and get out of the way.
  • Measure the warm state. The fast machine code appears only after a function heats up, and a bad assumption can deopt you back down mid-run, so a naive benchmark can measure a tier your users never hit. Benchmarking traps is the lesson on getting that right.

Summary

  • For the hottest code, V8 compiles with TurboFan, the top-tier optimising compiler, which produces the fastest machine code the engine can make.
  • Optimising means specialising: instead of the interpreter’s generic “check the types, then dispatch” on every operation, TurboFan emits code for the exact types the feedback recorded, so a + b becomes a single machine add behind one cheap check.
  • The defining idea is speculation: it optimises on the assumption the future looks like the past. Every specialised path is a bet fronted by a guard, and when the guard fails the code is discarded and execution falls back to the interpreter via deoptimisation. A wrong guess is only ever slower, never incorrect.
  • Inlining is the multiplier: pasting a small callee’s body into its caller removes call overhead and, more importantly, exposes the merged code so every other optimisation can reach across the old boundary. Inlining plus type feedback is what makes idiomatic, small-function JavaScript fast.
  • Once types are pinned and functions inlined, standard optimisations apply: constant folding, strength reduction, redundancy elimination, loop-invariant hoisting, and dropping bounds and overflow checks it can prove unnecessary.
  • Optimising is expensive in time and memory, which is why it runs on a background thread and only for hot code, and why the cheaper tiers exist at all.
  • Tier names and internals are V8-specific and evolving (the IR is being rebuilt as this is written), but every major engine speculates on feedback, specialises, inlines, and guards.

What this means for your code: you cannot request optimisation, only make it easy to keep. Feed TurboFan stable types and stable shapes and it rewards you with tight specialised code; feed it chaos and it stays generic or bails out.