My LibraryBuy PDF
Part 6/Model Foundations

Calling a Model From JavaScript

A side project. One HTML file, a text box, and about forty lines of JavaScript that called a model straight from the browser. It worked on the first try, which turned out to be the dangerous part. Eleven hours later the provider sent an email: the account had spent a little over 2,900 dollars and had been throttled for abuse.

The code was fine. The architecture was the bug. The key sat in the JavaScript bundle, which every visitor downloads, so anyone who opened the network tab could read it and use it. A scraper found it before a single real user did.

Here is the thing that makes this mistake so easy. Calling a model really is just an HTTP request. It looks like any other fetch you would fire from the browser, so it is tempting to fire it from the browser. Do not. By the end of this you will know exactly what that request is, what comes back, and why the key belongs on a server you own.

Under the SDK, it is one POST

Provider SDKs are pleasant. They give you autocomplete, typed responses, retries, and a tidy function call. They can also make the whole thing feel like magic, and magic is hard to debug at 3am. So strip the SDK away for a second.

A chat model call is an HTTP POST. The body is JSON. It carries three things that matter: a list of messages, the name of the model you want, and a few knobs. Here is the entire request, no library involved:

// This runs on a SERVER. Note where the key comes from.
const res = await fetch("https://api.your-provider.com/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": `Bearer ${process.env.MODEL_API_KEY}`,
  },
  body: JSON.stringify({
    model: "your-model-name",
    messages: [
      { role: "system", content: "You are a terse assistant." },
      { role: "user", content: "What is a bearer token?" },
    ],
    max_tokens: 300,
    temperature: 0.2,
  }),
});

const data = await res.json();
console.log(data.choices[0].message.content);

That is a real call. If you have read how an HTTP server actually works, none of it is new: a method, a URL, headers, a JSON body, a JSON response. The Authorization header carries your credential. The JSON body carries the conversation. Everything else on this page is detail on those two lines.

The request is a list of messages

The heart of the body is messages, an array of objects. Each object has a role and some content. The model reads the whole array top to bottom as one conversation and predicts what the assistant would say next.

{
  "model": "your-model-name",
  "messages": [
    { "role": "system", "content": "You are a terse assistant." },
    { "role": "user", "content": "What is a bearer token?" },
    { "role": "assistant", "content": "A credential you present to prove access." },
    { "role": "user", "content": "Is it safe to keep one in the browser?" }
  ],
  "max_tokens": 300,
  "temperature": 0.2
}

Three roles cover almost everything. system sets standing instructions and tone. user is a human turn. assistant is a turn the model produced earlier, which you replay so the conversation stays coherent. The model has no memory between calls, so that array is the entire truth it gets. The full treatment of roles, and the bug where you forget to append the assistant’s reply, lives in messages and roles.

The knobs are optional and provider-specific in their names. Two you will touch constantly: a cap on how many tokens the answer may use (max_tokens here, max_output_tokens elsewhere), and temperature, which controls how much the sampling gambles. Low temperature for anything you will parse, higher for anything you want to feel creative. Those tradeoffs get their own article in sampling and temperature.

what you sendwhat comes backmessages (in order)system: you are terseuser: what is a bearer tokenassistant: earlier replyuser: is it safe in the browsermodel: your-model-namemax_tokens: 300temperature: 0.2themodelone assistant messageassistant:no, keep it on aserver you controlusage24 in, 58 out tokensfinish_reasonstopthe wire shape varies by provider; the pieces do not
Messages go in as an ordered list of role and content, plus a model name and a couple of knobs. One assistant message comes back, with token usage and a stop reason bolted on.

What comes back

One assistant message. That is the payload you actually wanted. Wrapped around it is metadata that most tutorials skip and that production code depends on.

The exact JSON differs by vendor, and this is the one place the provider-agnostic story cracks a little, so look at both common shapes. One family nests the reply under a choices array:

{
  "choices": [
    {
      "message": { "role": "assistant", "content": "No, keep it on a server." },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 24, "completion_tokens": 58, "total_tokens": 82 }
}

Another family puts the reply at the top level, splits content into an array of blocks, and renames the fields:

{
  "role": "assistant",
  "content": [{ "type": "text", "text": "No, keep it on a server." }],
  "stop_reason": "end_turn",
  "usage": { "input_tokens": 24, "output_tokens": 58 }
}

Different keys, same three ideas: the message, how many tokens it cost, and why the model stopped. Learn to see past the field names to those three things and you can read any provider’s response.

Usage: the number on your bill

The usage block is the token count for that one call, split into input and output. This is not trivia. You pay per token, your rate limit is measured in tokens per minute, and the context window is a token budget. Log these numbers from day one. When someone asks why the bill tripled, usage summed per user or per feature is the answer, and if you did not record it you are guessing. The mechanics of what a token even is are covered in tokens, and why they cost you.

Finish reason: why it stopped

The stop reason tells you whether you got a complete answer or a truncated one. The values differ by provider but map to the same handful of causes:

  • Natural stop (stop, end_turn). The model decided it was done. This is the happy path.
  • Hit the token cap (length, max_tokens). Your answer was cut off mid-sentence because it ran into max_tokens. Raise the cap or ask for less.
  • Wanted a tool (tool_calls, tool_use). The model is asking you to run a function, not answering. Covered later in the course.
  • Filtered (content_filter). The safety layer blocked it.

The bug this prevents is subtle. If you only read content and ignore the stop reason, a truncated answer looks exactly like a complete one. Your code happily parses half a JSON object and falls over somewhere downstream, far from the real cause. Check the reason.

The same call, through an SDK

Now put the SDK back. Here is the identical request through a typical toolkit, the kind that speaks to many providers behind one interface:

import { generateText } from "ai";

const { text, usage, finishReason } = await generateText({
  model: "your-provider/your-model",
  messages: [
    { role: "system", content: "You are a terse assistant." },
    { role: "user", content: "What is a bearer token?" },
  ],
});

Same messages array. Same idea. What did the SDK do for you? It set the Authorization header from an environment variable by convention, chose the endpoint, serialized the body, parsed the response, and, crucially, normalized it. It reached into choices[0].message.content (or the block array) and handed you a flat text string. It renamed prompt_tokens and input_tokens to one agreed shape. It gave finishReason a single set of values regardless of vendor. It probably retried a couple of transient failures on your behalf too.

That normalization is the real reason to use an SDK, not the autocomplete. Switching providers becomes a one-line change instead of a rewrite. But it is a wrapper, not a different thing. When it misbehaves, you drop back down to the POST you saw above and everything is legible again.

Your API key is a bearer credential

Back to the 2,900 dollar email. The word in Authorization: Bearer <key> is doing real work. A bearer credential means whoever bears it, whoever holds it, can use it. There is no second factor, no per-user check, no device binding. The key is the identity. It bills to you and it authorizes everything your account can do.

So the rule has no exceptions worth the risk:

Your API key lives on a server you control. Never in the browser, never in client-side JavaScript, never in a mobile app binary, never in a public repository, never in a URL.

Anything shipped to a client can be read by the client. Minification is not encryption. A locked-down CORS policy does not help, because the attacker is not using a browser, they are replaying your key with curl. Environment variables in a frontend build (the NEXT_PUBLIC_ or VITE_ kind) get inlined into the bundle and are just as exposed. If a key ever touches a device you do not own, treat it as burned and rotate it.

the mistake: the browser holds the key and calls the model directlybrowser (your JS)API key baked inmodel APIbills your accountanyone can read the bundlethe shape that works: your server holds the key and sits in the middlebrowserno keyyour serverholds the keymodel APIsends a requestadds key, auth, limitsreply flows backthe key never reaches a machine you do not control
Top: the key rides in the browser bundle, so anyone can read it and spend your money. Bottom: your server holds the key and stands between the browser and the model.

The correct topology is one hop longer and worth every millisecond: browser to your backend to the model. The browser talks to your own server. Your server holds the key and makes the model call. The reply comes back the same way.

Where the key actually lives

Keep it out of the codebase entirely. In development, load it from a .env file that is listed in .gitignore and read it through process.env.MODEL_API_KEY. In production, use the secret store your platform gives you (a secrets manager, encrypted environment variables, whatever your host calls it) rather than pasting the value into a dashboard field that ends up in build logs. Scope keys to the smallest permission set the provider allows, give each environment its own key so you can revoke one without downing everything, and rotate on any hint of a leak. Provider dashboards and secret scanners in your git host will often catch a committed key before an attacker does, but do not build on that safety net.

What the server in the middle actually does

“Put it behind a server” sounds like pure overhead, one extra network hop for nothing. It is the opposite. That server is where every control you need actually lives, and none of them are possible from the browser.

browseryour useryour serverauthenticate the callerenforce rate limits and quotasattach the secret keylog tokens, latency, costmodel APIholds nothingevery one of these is impossible from a browser that talks to the model directly
The server is not a passthrough. Holding the key is one of four jobs it does that the browser cannot.
  • Authenticate the caller. The model call has no idea who your user is. Your server does. It checks the session or token before spending a cent, so a stranger cannot drive your account. This is your own auth, unrelated to the model’s key. See authorization.
  • Rate-limit and cap spend. One buggy client in a retry loop, or one abusive user, can run up a bill fast. Per-user limits and a hard daily ceiling live here. Read rate limiting, and since model calls are slow and concurrent, server concurrency matters too.
  • Hold the key. Obviously. It attaches the credential the browser never sees.
  • Log everything. Prompt, token usage, latency, cost, and which user made the call. When you debug a weird answer or a cost spike next month, this log is the only record you have. Put it into your observability from the start.

A thin endpoint that does all four is maybe thirty lines. It is the smallest amount of backend that turns a toy into something you can put in front of real users.

The errors you will actually hit

Happy-path demos never show these, and then your first real traffic is nothing but these. Sort them into two piles: transient failures you retry, and mistakes you fix. Retrying a mistake just wastes calls and money.

the error, what caused it, and what to do about it401bad or missing API keyfix the credential429rate limit or quota hitback off and retry400malformed body or bad model namefix your codecontextlimitprompt plus reply exceed the windowtrim the inputcontentfilterblocked by the safety layerhandle gracefullyretrying a 400 or 401 wastes calls; only transient failures earn a retry
Two piles. A 429 or a 5xx is worth retrying with backoff. A 400 or 401 is a bug in your code or input, and retrying it changes nothing.

401 Unauthorized. The key is wrong, expired, revoked, or not being sent. Do not retry it, the answer will not change. In practice this is usually an env var that did not load, so the header went out as Bearer undefined.

429 Too Many Requests. You crossed a limit: requests per minute, tokens per minute, or a spending cap. This one you do retry, but politely. Read the Retry-After header if the provider sends one, and back off exponentially with jitter so a fleet of clients does not all retry in lockstep. This is the same discipline as any upstream call, covered in retries and backoff.

400 Bad Request. The body was malformed: a wrong model name, a role the API does not recognize, a message with no content, a broken tool schema. The response body almost always says exactly what is wrong, so read it before you guess. Fix the code. Validate the request you build the same way you validate input at any API boundary.

Context length exceeded. Your prompt plus the room reserved for the answer add up to more than the model can hold. It usually arrives as a 400 with a specific code. The fix is not a bigger model, it is fewer tokens: trim history, shrink retrieved text, summarize. This is the whole subject of the context window.

Content filter. The provider’s safety layer blocked the input or the output. You will see it as an error or as a content_filter stop reason. Handle it like a real outcome your users will hit, with a clear message, not an unhandled exception.

A first cut at the routing logic is short, and mirrors how you would treat status codes and errors anywhere else:

if (res.ok) return res.json();

if (res.status === 429 || res.status >= 500) {
  // transient: back off, honour Retry-After, retry a few times
  throw new RetryableError(res.status);
}

// 400, 401, 403: your fault. Log the body, surface it, do not retry.
const body = await res.text();
throw new FatalError(`${res.status}: ${body}`);

One request, two ways to receive it

There is a fork in the road you meet immediately, and it is the last thing to settle here. The same POST can return its answer two ways.

Synchronous. You send the request and wait. The whole answer generates on the provider’s side, then arrives in one response. Simple to code, simple to reason about. The catch is the wait: the model produces the reply token by token internally, and you sit through all of it before you see a single word. For a long answer that is several seconds of a blank screen.

Streaming. You send the same request with a flag that asks for a stream, and the server sends each token as it is produced. The first words appear almost immediately and the rest flow in. Nothing about the model changes, only the delivery. The response is a long-lived event stream instead of one JSON blob.

synchronous: wait for the whole answer, then it appearsgenerating (you see nothing)full answer at oncetime to first word equals time to last wordstreaming: the same POST, tokens arrive as they are producedNo,keepiton aserverfirst word in a moment; the user reads while the rest generates
Same request, two deliveries. Synchronous waits for the last token before showing the first. Streaming shows tokens as they are made.

Streaming feels dramatically faster even though the total time is about the same, because the metric users feel is time to first word, not time to last. It also costs you real engineering: flushing each chunk, parsing partial data, letting a user cancel mid-answer so you stop paying, and coping with a connection that dies halfway. That is a whole article on its own, so it gets one. See streaming responses. Start synchronous, ship it, add streaming when the wait becomes the thing people complain about.

See the shape with no network

Enough words. Here is a mock chat client. There is no model and no network anywhere in it. A fake endpoint in the page returns a canned reply, and the panels on the right show the exact messages array that gets sent and the JSON that comes back, so you can watch the shape without spending a token or exposing a key.

Send a message and read the request. Then flip the key to invalid and send again to see the 401 shape your error handling has to deal with.

interactiveA mock chat client: watch the request and response shapes

Notice two things while you play with it. The messages array grows every turn, because the model keeps no memory and you resend the whole conversation each time. And the request shape is identical whether the response is a 200 with an assistant message or a 401 with an error. Your code has to be ready for both.

Summary

  • Under every SDK, a chat model call is one HTTP POST: a messages array, a model name, and a few parameters go up as JSON, and an assistant message comes back.
  • messages is an ordered list of role-and-content objects. The model has no memory between calls, so that array is the whole conversation it sees, and it grows every turn.
  • The response wraps the reply in metadata. usage is your token bill and belongs in your logs. The finish reason tells you whether the answer is complete or truncated, so check it.
  • Field names differ by provider (nested under choices or top level, prompt_tokens or input_tokens, stop or end_turn). The three ideas underneath are the same everywhere.
  • An SDK is a wrapper over that POST. It sets the auth header, picks the endpoint, and normalizes the response shape across providers. Useful, but it does not change any rule below it.
  • The API key is a bearer credential that bills to you. It lives on a server you control. Never the browser, never client-side JavaScript, never a public repo, never a URL.
  • The topology that works is browser to your backend to the model. Your server holds the key and adds the things a browser cannot: auth, rate limits, spend caps, and logging.
  • Keep the key out of the codebase: a git-ignored .env in development, a real secret store in production, scoped and rotatable keys.
  • Sort errors into retry and fix. A 429 or 5xx earns a backed-off retry; a 400 or 401 is your bug and retrying wastes calls. Context-length and content-filter errors are outcomes to handle, not crashes.
  • Synchronous returns the whole answer after the wait; streaming sends tokens as they are made and feels far faster. Start synchronous, add streaming when the wait starts to hurt.