Inline Caches: Mono-, Poly-, Megamorphic
Here is a loop that reads one property, a hundred thousand times:
let total = 0;
for (const p of points) {
total += p.x;
}Hidden classes told you what p.x really is. Not a search for a property named x, but a shape check and a read from a fixed slot: is p the shape I expect, then read slot 0. Good. Read that again though, because there is a hole in it. “Read slot 0” is the answer. Where does the answer come from? Somebody has to work out that x lives at slot 0 for this shape, and if the engine worked that out on every one of those hundred thousand turns of the loop, it would hand back most of what shapes just bought you.
So it does the work once. The first time this exact line runs, the engine does the full lookup, learns that for p’s shape the property x sits at slot 0, and writes that fact down right on the line of code. Every turn after the first reads the note instead of redoing the lookup. That note is an inline cache, and it is the piece that turns “shapes make fast access possible” into “this loop is actually fast”.
The note lives on the line, not on the object
The first execution of total += p.x is the expensive one. The cache is empty (V8 calls that state uninitialized), so the engine runs the full property-lookup algorithm the language specifies, gets back “shape S, offset 0”, and stores that pair in the slot the interpreter reserves for this spot. Every later execution walks a much shorter path: compare the incoming object’s shape against the cached one, and if it matches, read the cached offset. One pointer comparison and one indexed load.
The word to hold onto is site. The cache is attached to a location in your code, the exact place where .x is read, not to the object p and not to the property name x. That distinction is small to say and easy to skip past, and it is the whole reason this lesson exists. Get it and most inline-cache behaviour stops being mysterious.
Why it is called “inline”
The name is older than V8, and it is literal. Inline caching was invented for Smalltalk in 1984, and the original trick was to patch the machine code at the call site. When the lookup resolved an address, the engine overwrote the placeholder in the compiled instruction with the real one. The cache lived inside the code, one line of it, hence inline.
Modern V8 does not patch machine code at each read. The cached shape and offset live in a per-function side table called the feedback vector, one slot per site, and the interpreter reads and writes that slot as it runs. The mechanism moved off the instruction and into a data structure. The name stuck anyway, because everyone already knew it.
That [3] in the bytecode is the same slot index the interpreter lesson pointed at on an Add. It is the address of this site’s cache. The polymorphic extension, where one cache can remember several shapes instead of one, came a few years later from the Self language. That is where the mono, poly and mega vocabulary starts.
One property, two sites, two caches
Because the cache belongs to the site, two reads of the same property in two different places are two separate caches that know nothing about each other.
function getX(p) {
return p.x; // site 1: its own cache
}
function leftEdge(p) {
return p.x - p.width / 2; // site 2: a different cache that also reads .x
}Both functions read .x. They do not share what they learn. If getX is only ever called with one shape, its cache stays monomorphic and fast forever, no matter what leftEdge sees. And if leftEdge gets fed a chaotic mix of shapes, its cache degrades on its own without touching getX. Same property name, same objects even, two independent verdicts.
This is why “is my code monomorphic?” is not quite the right question. There is no single verdict for a function. Each property access, each method call, each operator inside it has its own little cache with its own history. A function can be fast at one read and slow at the next line.
When one cache holds more than one shape
You have met the words monomorphic, polymorphic and megamorphic in type feedback and speculation. There they described how much variety an operation has seen. Here they have a concrete, physical meaning you can almost point at: how many entries this one cache is holding, and what it does when you overflow it.
- Monomorphic is one entry. The cache holds a single (shape, offset) pair. Every access is one comparison and one load. This is as good as it gets.
- Polymorphic is a few entries. The cache holds a short list, say (A, 0), (B, 1), (C, 0), and each access walks that list top to bottom until a shape matches. Three shapes means up to three comparisons. Still cheap, still specialised, just more checking than one.
- Megamorphic is the giving-up state. Past a small limit (currently around four shapes in V8, an internal number that has changed before), the engine decides a per-site list is not worth maintaining. It stops tracking shapes here and routes the read through a large shared hash table that any megamorphic site can use. Generic, and slow, but bounded.
The middle state is the one people misjudge, in both directions. Polymorphism is not a failure. A short scan of two or three shapes is genuinely cheap, and code that handles a couple of related shapes is doing exactly what the polymorphic cache was built for. The cliff is the last step, poly to mega, where the engine stops keeping a tidy per-site list and everyone falls through to the generic path.
A heterogeneous list is the classic way to go mega
Here is the failure in ordinary code, and it is worth recognising because it is common. A function reads one property in a loop. If every object in the loop has the same shape, the read stays monomorphic and the loop flies:
function sumX(arr) {
let t = 0;
for (const p of arr) t += p.x; // one site; this is where it is won or lost
return t;
}
// one factory, one shape, so p.x sees a single shape
const uniform = Array.from({ length: 100000 }, (_, i) => ({ x: i, y: i }));
sumX(uniform); // fast: the read stays monomorphicNow feed the same function a list of objects that are all “points” to you but were each built a little differently. To the engine they are different shapes, and the single read inside sumX sees all of them:
const zoo = [
{ x: 1, y: 1 },
{ y: 2, x: 2 }, // different order → different shape
{ x: 3, y: 3, z: 3 }, // extra field → different shape
{ x: 4 }, // missing field → different shape
legacyPoint(5), // built by other code → different shape again
];
sumX(zoo); // that one p.x read goes polymorphic, then megamorphicThe function is character-for-character the same. Nothing about the arithmetic changed. The read went slow because the inputs to that one site got varied, and the full walk of why identical code has opposite fates lives in the type-feedback lesson. The mechanism you now hold is the cache behind it: the read at t += p.x filled its cache past the limit and dropped to the generic table, and it drags every element through that table, even the { x, y } objects that were shape A the whole time.
The fix is never to touch sumX. It is upstream, where the objects are built: give the things you treat as one kind one shape, and the read stays monomorphic for free.
Feel a cache fill up
Feed one read site objects of shapes you choose. Watch its cache gain rows, watch each access scan those rows to find a match, and watch it tip over into the megamorphic global-table state once the variety gets too high. The offsets and the four-shape limit here are a simulation of the behaviour, not a live measurement of V8.
Feed it the same shape twice and the second read is a hit on row one, the cheapest thing there is. Feed it four different shapes and each access has to scan further. Feed it a fifth and the whole per-site list evaporates. That scan-then-give-up shape is the entire cost model in miniature.
Method calls and prototype reads cache too
Property reads are the easy case to picture, but the same machinery sits behind method calls, and there the payoff is bigger because the lookup it skips is longer. When you call user.save(), save almost never lives on user itself. It lives on the prototype, and finding it means walking the prototype chain until it turns up.
class User {
constructor(name) { this.name = name; }
save() { /* ... */ }
}
const u = new User("Ada");
u.save(); // first call walks the chain to find save; the site caches the resultAn inline cache collapses that walk. The first call finds save on User.prototype, and the site records something like: for a receiver of shape U, the method to call is this exact function on the prototype. The next call with a shape-U receiver checks the shape and jumps straight to the cached target. No chain walk.
There is a subtlety in that guard. The cache is betting on two things staying put: the receiver’s shape, and the fact that save is still the same function on the prototype. If someone reassigns User.prototype.save or swaps the prototype out from under the object, that bet is off, so the engine tracks prototype changes and invalidates the cached method when they happen. This is one reason mutating prototypes at runtime, a trick some older code loved, quietly defeats these caches. It is also why calling many methods on one steady class of object is fast: every call site settles into a monomorphic cache and the chain walks disappear.
What the cache tells the optimiser
Everything so far is true in the interpreter alone, no compiler needed. But the caches are also the evidence the optimising compilers read when a function gets hot. A site whose cache is monomorphic is telling them, in the clearest terms they understand, that this read has only ever seen one shape. That is the green light to compile a specialised version: bake in a shape check as a guard, and follow it with a hard-coded offset load, no cache lookup at all.
A polymorphic site with two or three entries can still be compiled well, as a small chain of guards. A megamorphic site gives the optimiser nothing to bet on, so it emits the generic path there too. And if a guard ever fails at runtime, because a shape the optimiser never saw finally shows up, the specialised code cannot handle it and the function falls back, which is deoptimisation and its own lesson. The cache is the thread connecting all of it: it captures the speed in the interpreter, and it hands the optimiser the facts to bet on.
The honest calibration
This is worth the same caution as shapes: it explains a real hotspot, and it is a waste of attention almost everywhere else.
It matters when you create one kind of object in bulk and read its properties or call its methods in a hot loop. Particles, parser nodes, grid rows, vectors, entities in a game step. There, keeping a read monomorphic instead of megamorphic is a direct-load versus a global-table probe, times millions, and it is genuinely the difference. The good news is you buy it with the thing you should be doing anyway: build those objects consistently so they share a shape.
It does not matter for a config object read a handful of times, an options bag, a request handler that runs once per request, or the ninety-odd percent of code that is not in a tight loop. A polymorphic or even megamorphic site that runs occasionally costs you nothing you can measure. And some sites are megamorphic by design and should stay that way: a logger, a serialiser, a deep-clone, anything generic over every object in your app touches many shapes on purpose. There is nothing to fix there.
Summary
- An inline cache records the (shape, offset) of a property right at the spot in your code where it is read, so after the first access the engine does a shape check and a direct slot read instead of a full lookup. That check-and-read is the fast path hidden classes make possible.
- The cache belongs to the call site, not the object or the property name. The same
.xin two functions is two independent caches, and each can be a different state. - The states are the size of one cache: monomorphic is one entry (one check), polymorphic is a short list scanned in turn, megamorphic is the give-up state that routes through a shared generic table. The costly step is poly to mega, and a megamorphic site does not heal itself.
- Feeding one read a heterogeneous list of differently-shaped objects is the everyday way to drive it megamorphic. The fix is upstream: build objects of one kind the same way so the read stays monomorphic.
- Method calls and prototype reads cache too, guarded by the receiver’s shape and the stability of the prototype, which is why steady classes are fast and runtime prototype mutation is not.
- A monomorphic cache is the optimiser’s green light to specialise; the cached shape becomes the guard, and a failed guard deoptimises. The name is a 1984 Smalltalk relic; the concept lives in every engine.
What this means for your code: you never manage inline caches, you only decide what each site sees. Build the objects you make in bulk consistently, call methods on steady kinds of object, and the caches under your hot reads stay monomorphic without you writing a single line aimed at them.