Start learning
Part 3/Regular Expressions

Sticky flag "y", searching at position

Most regexp searches ask “is this pattern anywhere in the string?”. The sticky flag pattern:y asks a sharper question: “is this pattern right here, at this exact position?”. That distinction sounds small, but it’s the difference between a scanner that works and one that quietly matches the wrong thing.

The task that needs it

A classic job for regular expressions is lexical analysis: take some text — source code, an HTML document, a config file — and break it into its structural pieces. HTML has tags and attributes. JavaScript has keywords, identifiers, operators, string literals. Building a full tokenizer is its own field with its own algorithms, so we won’t go deep. But it rests on one small, repeated operation: read whatever token sits at a known position.

Here’s the miniature version. You have a line of code and you know a variable name begins at index 4:

let str = 'set itemPrice = 999';
//         0123456789...
//             ^ variable name starts here, at index 4

You want to pull itemPrice out, using the pattern pattern:\w+ for “one or more word characters”. (Real identifiers need a fussier pattern, but that’s not the point here.) The obvious tools both miss:

  • str.match(/\w+/) returns only the first match in the whole string — that’s set, not what you asked for.
  • str.match(/\w+/g) returns every word: set, itemPrice, 999. You’d then have to figure out which one started at index 4.

Neither lets you say “search at position 4”. To get there, you need regexp.exec and an understanding of lastIndex.

How exec walks a string with the g flag

The method regexp.exec(str) behaves differently depending on the flags.

With no pattern:g and no pattern:y, exec finds the first match and stops. It’s equivalent to str.match(regexp) without pattern:g.

Add the pattern:g flag and it turns into a stateful stepper. Each call:

  1. starts searching at the index held in regexp.lastIndex (initially 0),
  2. on success, moves lastIndex to the position right after the match,
  3. on failure, resets lastIndex back to 0.

So lastIndex is both an input (where to begin) and an output (where the next call resumes). Repeated calls hand you matches one at a time.

set␣itemPrice
call 1  start lastIndex=0 → match “set” → lastIndex=3
call 2  start lastIndex=3 → match “itemPrice” → lastIndex=13
call 3  start lastIndex=13 → null → lastIndex resets to 0
With the g flag, each exec call reads lastIndex, matches, then writes lastIndex to just past the match. The final failing call resets it to 0.

Here’s that in code:

let str = 'set itemPrice'; // find every word in this string
let regexp = /\w+/g;

alert(regexp.lastIndex); // 0 (starts here)

let word1 = regexp.exec(str);
alert(word1[0]);         // set (first word)
alert(regexp.lastIndex); // 3 (just after "set")

let word2 = regexp.exec(str);
alert(word2[0]);         // itemPrice (second word)
alert(regexp.lastIndex); // 13 (just after "itemPrice")

let word3 = regexp.exec(str);
alert(word3);            // null (nothing left)
alert(regexp.lastIndex); // 0 (reset on failure)

Because a null return ends the sequence, you can drain every match with a while loop — the assignment inside the condition doubles as the exit test:

let str = 'set itemPrice';
let regexp = /\w+/g;

let result;

while (result = regexp.exec(str)) {
  alert( `Found ${result[0]} at position ${result.index}` );
  // Found set at position 0
  // Found itemPrice at position 4
}

This is a lower-level alternative to str.matchAll. You trade convenience for control: you can inspect and even rewrite lastIndex between iterations.

Setting lastIndex by hand

Since lastIndex is where exec begins, you can write to it yourself and search from a chosen offset. Back to the original task — start at index 4:

let str = 'set itemPrice = 999';

let regexp = /\w+/g; // without the "g" flag, lastIndex is ignored

regexp.lastIndex = 4;

let word = regexp.exec(str);
alert(word); // itemPrice

That works. exec began at 4, found varName, and returned it. Note the comment: lastIndex only matters when pattern:g (or pattern:y) is present. A plain /\w+/ ignores it completely and always starts from zero.

Why “starting from” isn’t good enough

There’s a catch hiding in that success. With pattern:g, exec starts at lastIndex and then keeps scanning forward until it finds a match or runs out of string. If nothing matches exactly at lastIndex, it happily reports the next match further along.

Point it at index 3, which is the space between let and varName:

let str = 'set itemPrice = 999';

let regexp = /\w+/g;

// begin the search at position 3
regexp.lastIndex = 3;

let word = regexp.exec(str);
// there's no word AT 3, so it scanned ahead and matched at 4
alert(word[0]);    // itemPrice
alert(word.index); // 4

You asked about position 3. There’s a space there, no word. But pattern:g slid forward and handed you the match at 4 anyway. For a tokenizer that’s a bug: “what token is at index 3?” must answer “nothing”, not “the next token somewhere down the line”.

lastIndex = 3, flag g
set␣ite…→ matches at 4 ✅
lastIndex = 3, flag y
set␣ite…→ null (space at 3) ❌
g scans forward from lastIndex until it finds a match. y checks only lastIndex, and fails if the pattern does not start exactly there.

The sticky flag: match exactly here or fail

The pattern:y flag forces regexp.exec to test the pattern only at lastIndex. No scanning forward. If the pattern doesn’t begin precisely at that index, the result is null.

Same setup, sticky flag instead of global:

let str = 'set itemPrice = 999';

let regexp = /\w+/y;

regexp.lastIndex = 3;
alert( regexp.exec(str) ); // null (position 3 is a space, not a word)

regexp.lastIndex = 4;
alert( regexp.exec(str) ); // itemPrice (a word does start at 4)

At 3, pattern:/\w+/y refuses to match — where pattern:g would have jumped to 4. At 4, it matches. That “exactly here” behavior is precisely what a position-aware scanner needs.

Try it directly. Drag the slider to point lastIndex anywhere in the string, then flip the flag. Watch how pattern:g slides forward to the next word while pattern:y insists on a word starting exactly under the caret.

interactiveg scans forward, y stays put

The performance payoff

Sticky matching is also faster in a common bad case. Picture a long string with no match at all. A pattern:g search from some offset has to grind all the way to the end of the text before it can conclude “nothing here” — every position gets tried. A pattern:y search checks one position and gives up immediately.

ycheck pos→ fail, stop
gtrytrytrytry… all the way …try→ fail
On a no-match string, y tests a single position; g must try every remaining position before failing.

In lexical analysis you run enormous numbers of these “what’s at this position?” checks, most of which fail so you can try the next token type. Making each check touch exactly one position, rather than the rest of the file, is the difference between a snappy parser and a sluggish one. So pattern:y gives you both correctness (no accidental look-ahead) and speed.

Here is that last idea made real: a tiny tokenizer. Each token type is a sticky regexp, all sharing one lastIndex. Step through and watch the caret walk the input, trying each type at the current spot and advancing only past whatever matched.

interactiveA sticky-regexp tokenizer, one step at a time