Chapter 03

Prompting for Code

How to ask for code so the answer has a chance of being right: giving the model the context it can't see, writing the spec and the tests before the code, iterating with exact failures instead of adjectives, and catching invented APIs and subtle bugs with checks against the real packages.

5 topics

Parts marked Advanced are extra depth. Skip them on a first read or a quick revision.

#Give the model the context it needs

Added

A prompt for code is a ticket for someone who has never seen your repository and can't ask you questions until they hand in the work. Everything the model needs to get it right has to be in the context: which files to change, the types and helpers to use, the constraints (versions, no new dependencies, error format), an example of the expected result and what "done" means. Whatever you leave out, it fills in with the most common version of that thing from its training data, which is a reasonable guess about the average codebase and a poor one about yours (see How they work, just enough to use them well).

The mental model is the contractor's ticket. "Fix the login bug" works with a colleague who knows the system and can lean over to ask. A contractor needs: the symptom, where to look, what fixed looks like, and what not to touch. The vendors' own guidance is the same list. GitHub's "Prompt engineering for GitHub Copilot" page (checked 2026-10-27) says to start general then get specific, give examples, break complex tasks into simpler ones, avoid ambiguity, provide relevant context ("Copilot uses the context available to it to understand your prompt"), iterate, and keep history relevant. Claude Code's best-practices page puts it as: "If you name the file, describe the symptom, and say what 'fixed' looks like up front, you usually save yourself a round of corrections later", with examples like these:

Instead of	Try (from Claude Code's best-practices page, checked 2026-10-27)
"fix the login bug"	"users report login fails after the session times out. look at token refresh in src/auth/, write a failing test that reproduces it, then fix it"
"add a calendar widget"	"follow the pattern in HotDogWidget.php and build a calendar widget that lets you pick a month and page between years. no new libraries"

What goes into a good code prompt, and why each part is there:

Part	Example	What it prevents
Task and goal	"add a percentage discount to invoices"	solving a different problem
Where	the file and function to change; the files to read	edits in the wrong place, a duplicate helper
Types and data shapes	the interface, a sample payload	invented field names and units
Existing helpers and patterns	"use `roundCents`", "follow `HotDogWidget`"	a second, slightly different implementation
Constraints	versions, "no new dependencies", error format, performance limits	stale APIs, surprise packages
Examples	one input with its expected output	ambiguity about rounding, ordering, formats
Definition of done	the test to pass, the command to run, the evidence to show	"looks done"

Here's a prompt for a small change in a TypeScript billing module, written with those parts. It's shown as you'd type it into a chat panel or an agent; the response is discussed below, not quoted, because no AI tool was run for this page.

Prompt: a change with the context it needs

Add a percentage discount to invoices.

Change applyDiscount(inv, pct) in src/billing/discount.ts (attached). Read first:
- src/types/invoice.ts (attached): amounts are integer cents; a line is { sku, cents, qty }.
- src/lib/money.ts: use roundCents for rounding; don't write a new helper.
- src/api/checkout.ts (attached) calls applyDiscount with pct from the request body.

Rules:
- pct is a number from 0 to 100 inclusive; anything else throws RangeError.
- Discount each line's total (cents × qty), round once per line, then sum.
- Don't change the Invoice type or the function's signature. No new dependencies.
- TypeScript strict mode; the project uses Node 26 and vitest.

Example: one line { cents: 1999, qty: 2 } at 15% → 3998 × 0.85 = 3398.3 → 3398 cents.

Done when: npm test -- discount passes and npx tsc --noEmit is clean.
Show me the diff and both commands' output.

What’s happening

The first line is the goal in one sentence, general before specific, as GitHub's guidance suggests. The model knows what success is before it reads any detail.
"Change … Read first" names the one file to edit and the three files that matter, with what each contributes: the types (so it doesn't invent amount or use floats for money), the helper (so it doesn't write its own rounding), and the caller (so it knows pct arrives from untrusted input and must be validated). In an agent you'd point at the paths; in a chat panel you'd attach them.
The rules are the decisions only you can make: the valid range and the error type, where rounding happens (per line, once; a classic source of one-cent mismatches), and what must not change. "No new dependencies" closes the most common surprise in AI diffs.
The example pins down the arithmetic: 3998 × 0.85 = 3398.3, rounded to 3398. One worked example removes more ambiguity than a paragraph about rounding, and it becomes the first test case.
"Done when" gives the check (tests and type checker) and asks for evidence, the commands' output. A good response to this prompt is a small diff in discount.ts that validates pct, maps over items with roundCents, plus a test file and the output; a response that also edits invoice.ts or adds a package has broken an explicit rule, which makes it easy to reject.

AdvancedFinding the context that matters

For a change to one file, the context that matters is usually: what that file imports (the types and helpers it must use) and what imports it (the callers that break if its behaviour changes), plus its tests. This model walks a small repository's import graph to find exactly that set, which is what you'd attach or point an agent at.

context-for-change.mjs

// Which files should go into the prompt for a change to src/billing/discount.ts?
// Walk the import graph: what it imports (the types and helpers it must use) and who imports it (callers that break).
const repo = {
  "src/billing/discount.ts": `import type { Invoice, LineItem } from "../types/invoice";\nimport { roundCents } from "../lib/money";\nexport function applyDiscount(inv: Invoice, pct: number): Invoice { /* … */ }`,
  "src/types/invoice.ts": `import type { Currency } from "./currency";\nexport interface LineItem { sku: string; cents: number; qty: number }\nexport interface Invoice { id: string; currency: Currency; items: LineItem[] }`,
  "src/types/currency.ts": `export type Currency = "CAD" | "USD" | "INR";`,
  "src/lib/money.ts": `export const roundCents = (n: number) => Math.round(n);`,
  "src/api/checkout.ts": `import { applyDiscount } from "../billing/discount";\n/* POST /checkout handler … */`,
  "src/billing/discount.test.ts": `import { applyDiscount } from "./discount";\n/* 6 tests … */`,
  "src/ui/theme.ts": `export const theme = { accent: "#84cc16" };`,
  "src/api/health.ts": `export const health = () => ({ ok: true });`,
};

const importsOf = (file) =>
  [...repo[file].matchAll(/from "(\.[^"]+)"/g)].map(([, rel]) => {
    const parts = file.split("/").slice(0, -1);
    for (const seg of rel.split("/")) seg === ".." ? parts.pop() : seg !== "." && parts.push(seg);
    return parts.join("/") + ".ts";
  });

const target = "src/billing/discount.ts";
const picked = new Map([[target, "the file being changed"]]);
const queue = [target];
while (queue.length) { // everything the target depends on, transitively
  const file = queue.shift();
  for (const dep of importsOf(file)) if (!picked.has(dep)) { picked.set(dep, `imported by ${file}`); queue.push(dep); }
}
for (const file of Object.keys(repo)) // direct callers and tests
  if (importsOf(file).includes(target)) picked.set(file, file.includes(".test.") ? "its tests" : "calls it: check it still works");

const tokens = (f) => Math.ceil(repo[f].length / 4);
for (const [file, why] of picked) console.log(`${file.padEnd(30)} ~${String(tokens(file)).padStart(3)} tokens  ${why}`);
const skipped = Object.keys(repo).filter((f) => !picked.has(f));
console.log(`left out: ${skipped.join(", ")}`);

Result

src/billing/discount.ts        ~ 45 tokens  the file being changed
src/types/invoice.ts           ~ 48 tokens  imported by src/billing/discount.ts
src/lib/money.ts               ~ 14 tokens  imported by src/billing/discount.ts
src/types/currency.ts          ~ 12 tokens  imported by src/types/invoice.ts
src/api/checkout.ts            ~ 21 tokens  calls it: check it still works
src/billing/discount.test.ts   ~ 15 tokens  its tests
left out: src/ui/theme.ts, src/api/health.ts

What’s happening

repo is a tiny in-memory repository; importsOf reads a file's relative from "…" imports with a regex and resolves .. and . segments into repository paths, so "../types/invoice" from src/billing/ becomes src/types/invoice.ts.
The while loop is a breadth-first walk from the target: discount.ts imports invoice.ts and money.ts; invoice.ts imports currency.ts. All three are picked, each with the reason it was picked. Currency is two hops away, and it's exactly the kind of type a model would otherwise guess (is it "cad" or "CAD"?).
The second loop finds reverse dependencies: files whose imports include the target. checkout.ts calls applyDiscount, so it goes in as "check it still works"; discount.test.ts goes in as the tests.
theme.ts and health.ts are left out. Including them would cost tokens and attention for nothing; in a real repository, "unrelated files" is most of the repository.
The estimates (characters ÷ 4) show the whole relevant set is a few hundred tokens here. In real code the set is bigger, but the principle holds: the types, the helpers, the callers and the tests, not the whole folder. Agents that search the repository do a version of this themselves; with chat, it's your job.

AdvancedWhat to leave out

Context isn't free: every token of noise competes with the signal (Anthropic's docs call the degradation "context rot"). Leave out, or cut down:

Whole logs and stack traces: paste the failing test's output and the top frames of the trace, not 3,000 lines of CI log.
Unrelated files that happen to be open (completion tools may include open tabs as context, so close the irrelevant ones).
Old conversation: when the task changes, start a new session. GitHub: "Start a new conversation or session when you begin a new task." Two failed attempts in the history are context too, and the model keeps re-reading them.
Secrets and personal data: API keys, .env contents, customer records, production logs with user data. Assume anything in a prompt leaves your machine; your company's policy decides which tools may see what (see Security, licensing and data concerns).

AdvancedStanding context: instruction files

Some context belongs in every request: the test command, the versions, conventions the model keeps getting wrong. Rather than retyping it, put it in the repository's instruction file, which the harness prepends to every request: .github/copilot-instructions.md for Copilot (see Custom instructions and prompt files), CLAUDE.md or AGENTS.md for Claude Code. Claude Code's best-practices page has a good test for what belongs there: things the model can't learn by reading the code, such as "how to confirm work in this repo: the exact test, typecheck, and lint commands". It also warns that piling on emphasis ("IMPORTANT", capitals) "mostly makes Claude over-apply the rule"; a rule that must never bend should be enforced by a hook or a check, not by shouting.

How do you write a good prompt for a code change?

I write it like a ticket for a contractor who can't ask questions: the goal in one line, the file to change, and the context it can't guess, which is the types it must use, the existing helpers, and the callers that depend on it. Then the decisions only I can make: valid inputs and error behaviour, where rounding or ordering happens, what must not change, no new dependencies, the versions we're on. One worked example pins down the details better than a paragraph. And I end with the definition of done, the test and type-check commands, and ask for the diff and the output as evidence. For anything repeated in every prompt, like the test command or our conventions, I put it in the repository's instructions file instead. Leaving things out matters too: no whole logs, no unrelated files, a new session for a new task, and never secrets.

AdvancedThe model can't see your editor

What's missing from this prompt?

"Write a function that formats a meeting's start time for the room display." List what the model has to guess, and what it will probably guess.

Show answer

At least: the input type, the time zone, the output format, the locale, edge cases, and where the code goes. It will guess the most common answers, which are probably not yours.

What’s happening

Input: a Date, an ISO string, epoch milliseconds? A model usually picks Date or an ISO string; if your API sends epoch seconds, every result is off by a factor of 1,000.
Time zone: the room's zone, the viewer's, or UTC? The common default is the runtime's local zone, which on a device or server is often not the room's. This is the guess most likely to be silently wrong.
Format and locale: "9:00 AM", "09:00", "9h00"? 12- or 24-hour clocks vary by region; a display that ships to many countries needs this decided, often via Intl.DateTimeFormat with an explicit locale.
Edge cases: meetings starting tomorrow, all-day events, a missing start time. Without them, the generated function handles only today's normal meeting.
Where and with what: which file, which date library (or none), which test framework. The fix for all five is the same: state them, plus one example per tricky case, and the function's signature.

#Spec first: describe behaviour and edge cases

Added

Spec first means you write down what the code must do, including the edge cases and the error behaviour, before asking for the code. It's the single most effective prompting habit for anything beyond boilerplate, because ambiguity is where generated code goes wrong: wherever the request leaves a gap, the model fills it with the most typical behaviour it has seen, and the most typical behaviour is the happy path.

The mental model: the model fills every gap with the average of the internet. "Parse durations like 1h30m" leaves open whether 90m is valid, what happens with 1.5h, whether there are seconds, and whether bad input throws or returns NaN. The average answer to each is a guess, and you'll discover the guesses one bug at a time. A spec removes the gaps up front, and as a bonus it's already most of a test file. Both Copilot CLI and Claude Code document a plan step for the same reason: GitHub's changelog for Copilot CLI (2026-02-25) describes a plan mode in which Copilot "asks clarifying questions, and builds a structured implementation plan before writing any code".

A spec for code doesn't need to be formal. It needs:

Inputs and outputs: types, units, formats.
Behaviour by example: a few concrete inputs with exact outputs, especially the boundaries.
Errors: which inputs are invalid and exactly what happens (throw which error, return what).
Non-goals: what it deliberately doesn't do, so the model doesn't add it.

Here's a spec-first prompt for a small utility that a scheduling UI would use:

Prompt: spec first

Write parseDuration(text: string): number in src/lib/duration.ts. It returns milliseconds.

Behaviour:
- Units h, m, s, in that order, each at most once: "1h30m", "90m", "2h", "45s", "1h30m15s".
- One optional space between parts: "1h 30m". Leading and trailing whitespace is ignored: " 2h ".
- Whole numbers only. "0m" is valid and returns 0.

Errors: throw RangeError, with the input in the message, for
- empty or whitespace-only input: ""
- a number without a unit: "30"
- decimals: "1.5h"
- units out of order or repeated: "1m1h", "1h1h"

Non-goals: days, negative durations, localised units.

Before writing any code, list anything in this spec that's ambiguous, as numbered questions, and stop.

What’s happening

The signature and the unit of the result (number, milliseconds) come first, so nobody returns seconds or a { hours, minutes } object.
"Behaviour" is a list of examples, not a description. Each one is a future test row, and together they answer the questions a one-line prompt leaves open: 90m is valid (minutes aren't capped at 59), seconds exist, spaces are allowed but only one.
"Errors" says what is invalid and how it fails: a RangeError carrying the input. Without this, a typical implementation either returns NaN, returns 0, or crashes with a TypeError from reading a property of null, and all three look like different bugs in the UI.
"Non-goals" stops scope creep in the other direction: no d for days, no -5m, no "1 hour 30 minutes" parser that nobody asked for.
The last line asks for questions before code. Kinds of question a model can usefully raise here: are upper-case units (1H) valid, is there a maximum, is "1h 30m" with two spaces an error, what about leading zeros ("01h")? Each answer goes back into the spec. This turns the model's knowledge of common edge cases into review of your spec, which is a better use of it than guessing.

AdvancedThe spec as an executable table

The examples in a spec are the start of the test suite. This model writes the spec as rows and runs two implementations against it, both written for this page: a draft of the kind a one-line prompt invites, and one written against the full spec.

spec-table.mjs

// A spec written as rows of input → expected behaviour, run against two implementations (both written for this page):
// a draft of the kind a one-line prompt invites, and one written against the full spec.
const spec = [
  ["1h30m", 5_400_000], ["90m", 5_400_000], ["2h", 7_200_000], ["45s", 45_000], ["1h30m15s", 5_415_000],
  ["1h 30m", 5_400_000], [" 2h ", 7_200_000], ["0m", 0],
  ["", "RangeError"], ["30", "RangeError"], ["1.5h", "RangeError"], ["1m1h", "RangeError"], ["1h1h", "RangeError"],
];

// Draft from the prompt "parse durations like 1h30m into milliseconds"
const draft = (s) => {
  const m = s.match(/(\d+)h(\d+)m/);
  return (Number(m[1]) * 60 + Number(m[2])) * 60_000;
};

// Written against the spec: units h, m, s in that order, each at most once, one optional space between parts
const UNITS = { h: 3_600_000, m: 60_000, s: 1_000 };
const fromSpec = (s) => {
  const m = /^(?:(\d+)h)? ?(?:(\d+)m)? ?(?:(\d+)s)?$/.exec(s.trim());
  if (!s.trim() || !m || m.slice(1).every((g) => g === undefined)) throw new RangeError(`bad duration: "${s}"`);
  return (Number(m[1] ?? 0) * UNITS.h) + (Number(m[2] ?? 0) * UNITS.m) + (Number(m[3] ?? 0) * UNITS.s);
};

function check(name, fn) {
  let passed = 0;
  const lines = [];
  for (const [input, want] of spec) {
    let got;
    try { got = fn(input); } catch (e) { got = e.constructor.name; } // invalid input must throw RangeError
    const ok = got === want;
    if (ok) passed++;
    else lines.push(`  ✗ ${JSON.stringify(input).padEnd(11)} want ${String(want).padEnd(10)} got ${got}`);
  }
  console.log(`${name}: ${passed}/${spec.length} rows pass`);
  for (const l of lines) console.log(l);
}
check("draft", draft);
check("fromSpec", fromSpec);

Result

draft: 1/13 rows pass
  ✗ "90m"       want 5400000    got TypeError
  ✗ "2h"        want 7200000    got TypeError
  ✗ "45s"       want 45000      got TypeError
  ✗ "1h30m15s"  want 5415000    got 5400000
  ✗ "1h 30m"    want 5400000    got TypeError
  ✗ " 2h "      want 7200000    got TypeError
  ✗ "0m"        want 0          got TypeError
  ✗ ""          want RangeError got TypeError
  ✗ "30"        want RangeError got TypeError
  ✗ "1.5h"      want RangeError got TypeError
  ✗ "1m1h"      want RangeError got TypeError
  ✗ "1h1h"      want RangeError got TypeError
fromSpec: 13/13 rows pass

What’s happening

spec is the prompt's examples as data: thirteen rows, each an input and either the expected milliseconds or "RangeError". check runs every row, records the error's constructor name if the function throws, and counts a row as passing only on an exact match.
The draft handles the one example it was given: "1h30m" → 5,400,000. Every other row fails. Most fail with a TypeError (the regex doesn't match, so m is null and m[1] throws), and that includes the rows that are supposed to fail: crashing isn't the same as rejecting, which is why the error rows demand RangeError specifically.
"1h30m15s" is the dangerous row: no crash, just 5400000 instead of 5415000. The regex matched a prefix and ignored the rest. Silent wrong answers like this are what reach production, because nothing looks broken.
fromSpec anchors its regex (^…$) so nothing is ignored, makes each unit optional but ordered, allows one space between parts, trims the input, and rejects empty input or input with no unit at all. All 13 rows pass, including the five error rows.
Writing the rows took less time than reading the draft carefully would have, and they found 12 problems a quick review might have missed. That's the case for spec first: the spec is cheap, it catches the gaps before the code exists, and it turns straight into the check the next topic is about.

AdvancedSpec first with an agent

With an agent, the spec can live in the repository instead of the prompt: a SPEC.md or a section of the issue, which the agent reads, implements against and is reviewed against. Claude Code's best-practices page shows the review half of that pattern as a prompt for a separate reviewer: "review the rate limiter diff against SPEC.md, check every requirement is implemented and the listed edge cases have tests, report gaps that affect correctness and skip style". The "skip style" part is deliberate, the page explains: "A reviewer that's asked to find problems will always find something."

When the task is bigger than one function, ask for a plan first and review it before any code: the files it will touch, the approach, the open questions. A plan is cheap to correct; a 600-line diff built on a misunderstanding isn't.

Advanced"Handle edge cases" is not a spec

Which spec line prevents which bug?

In the draft above, "1h30m15s" returned 5400000. Which line of the spec catches it, and what single change to the draft's regex makes it impossible to ignore trailing input?

Show answer

The behaviour example "1h30m15s" (seconds exist, all units together) catches it as a test row. Anchoring the regex with ^ and $ makes ignoring trailing input impossible.

What’s happening

The draft's regex /(\d+)h(\d+)m/ isn't anchored, so match finds "1h30m" anywhere in the string and stops; "15s" is never looked at.
The spec row ["1h30m15s", 5_415_000] exercises exactly that: a valid input with more after the part the draft understands. Without a row like it, the draft passes every hand-written check that only uses hours and minutes.
Anchoring (/^(\d+)h(\d+)m$/) turns the silent wrong answer into a non-match. The draft would then throw a TypeError for "1h30m15s": still wrong, but loudly wrong, which a test or a user report catches immediately.
The general lesson for reviewing generated parsers and validators: look for unanchored patterns and match where a full match was meant. "Accepts a prefix of the input" is one of the most common ways generated validation code goes wrong.

#Tests first with an assistant

Added

Tests first with an assistant means: write (or approve) the tests that define the behaviour, run them and see them fail, then let the assistant write the implementation until they pass, without changing the tests. It's test-driven development with the roles split: you own the contract, the assistant does the typing, and the test runner decides who's right.

The mental model: the tests are a contract with a referee. The assistant may change anything on its side of the line (the implementation); it may not move the line. That gives an agent the one thing it can't produce for itself, an independent definition of done. Claude Code's best-practices page (checked 2026-10-27) explains why it matters: the agent "stops when the work looks done", and if it has nothing that comes back pass or fail, "that makes you the test suite". Its first example of a specific prompt is "write a failing test that reproduces it, then fix it". GitHub's best-practices page lists "generating tests for test-driven development" as a use for Copilot's inline suggestions.

You own steps 1, 2 and 5; the assistant loops between 3 and 4. The tests are the line it can't move.

Here is the loop end to end with Node's built-in test runner. The tests are written first; then three implementations (all written for this page) are dropped in one after another, as an assistant's successive attempts would be, and node:test's run() reports each round.

tdd-loop.mjs

// Tests first: write the tests, watch them fail, then let implementations come and go until they pass.
// node:test's run() executes the test file in a child process; we print only names and results.
import { writeFileSync } from "node:fs";
import { run } from "node:test";

writeFileSync("slugify.test.mjs", `
import { test } from "node:test";
import assert from "node:assert/strict";
import { slugify } from "./slugify.mjs";
test("lowercases and joins words with hyphens", () => assert.equal(slugify("Room Booking API"), "room-booking-api"));
test("strips accents", () => assert.equal(slugify("Café Déjà Vu"), "cafe-deja-vu"));
test("collapses punctuation and repeats", () => assert.equal(slugify("a -- b!!c"), "a-b-c"));
test("trims leading and trailing hyphens", () => assert.equal(slugify("  --Hello--  "), "hello"));
test("returns empty string for no letters", () => assert.equal(slugify("!!!"), ""));
`);

const versions = {
  "v0 stub": `export const slugify = () => { throw new Error("not implemented"); };`,
  "v1 first attempt": `export const slugify = (s) => s.toLowerCase().replace(/ /g, "-");`,
  "v2 after reading failures": `export const slugify = (s) => s.normalize("NFD").replace(/[\\u0300-\\u036f]/g, "")
    .toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "");`,
};

for (const [label, source] of Object.entries(versions)) {
  writeFileSync("slugify.mjs", source);
  const results = [];
  for await (const event of run({ files: ["slugify.test.mjs"] })) {
    if (event.type !== "test:pass" && event.type !== "test:fail") continue;
    if (event.data.details.type === "suite" || event.data.nesting !== 0 || !event.data.line) continue;
    const err = event.data.details.error?.cause;
    const why = !err ? "" : "actual" in err ? `: got ${JSON.stringify(err.actual)}` : `: threw "${err.message}"`;
    results.push(`  ${event.type === "test:pass" ? "✓" : "✗"} ${event.data.name}${why}`);
  }
  console.log(`${label}: ${results.filter((r) => r.includes("✓")).length}/${results.length} passing`);
  console.log(results.join("\n"));
}

Result

v0 stub: 0/5 passing
  ✗ lowercases and joins words with hyphens: threw "not implemented"
  ✗ strips accents: threw "not implemented"
  ✗ collapses punctuation and repeats: threw "not implemented"
  ✗ trims leading and trailing hyphens: threw "not implemented"
  ✗ returns empty string for no letters: threw "not implemented"
v1 first attempt: 1/5 passing
  ✓ lowercases and joins words with hyphens
  ✗ strips accents: got "café-déjà-vu"
  ✗ collapses punctuation and repeats: got "a----b!!c"
  ✗ trims leading and trailing hyphens: got "----hello----"
  ✗ returns empty string for no letters: got "!!!"
v2 after reading failures: 5/5 passing
  ✓ lowercases and joins words with hyphens
  ✓ strips accents
  ✓ collapses punctuation and repeats
  ✓ trims leading and trailing hyphens
  ✓ returns empty string for no letters

What’s happening

The script writes slugify.test.mjs first: five node:test tests using node:assert/strict, one per rule of the spec (lowercase and hyphens, accents stripped, punctuation and repeats collapsed, no leading or trailing hyphens, empty result for no letters). Nothing implements slugify yet.
Round v0 is a stub that throws not implemented: 0/5, all red, for the right reason. This step is easy to skip and important: it proves the tests run, import the right module and can fail. A test that passes against a stub is testing nothing.
v1 is the typical first attempt: lowercase, replace spaces. One test passes; the other four fail, and the reporter shows the actual values: "café-déjà-vu" (accents kept), "a----b!!c", "----hello----", "!!!". Those four strings are the most useful thing you can paste back to the assistant; they say precisely what's wrong.
v2 normalises to NFD and removes the combining accent marks (Unicode U+0300 to U+036F, the range in its regex), lowercases, turns every run of non-alphanumerics into one hyphen, and trims hyphens from both ends: 5/5. The test file never changed between rounds; only slugify.mjs did.
Under the hood, run({ files }) runs the test file in a child process and yields events; the script prints only test:pass and test:fail events for top-level tests, with the assertion's actual value, so the output contains no timings and is identical on every run. In a real project you'd just run node --test and give the agent that command.

AdvancedPrompts for the two halves

Split the work into two prompts, so the tests are reviewed before any implementation exists. Both are shown as you'd type them; no responses are quoted.

Prompt 1: tests only

We're adding slugify(title) in src/lib/slugify.mjs for room and meeting URLs.

Step 1 only: write test/slugify.test.mjs with node:test and node:assert/strict.
Don't write slugify yet; create src/lib/slugify.mjs with a stub that throws "not implemented".
Cover: lowercase and hyphens between words; accents stripped ("Café" → "cafe"); runs of punctuation
or spaces become one hyphen; no leading or trailing hyphens; "" when there are no letters or digits.
One test per rule, with the rule as the test name. Then run node --test and show me the output.

Prompt 2: implementation against fixed tests

The tests in test/slugify.test.mjs are approved and committed. Don't modify them.
Implement slugify in src/lib/slugify.mjs so that node --test passes. No dependencies.
If a test looks wrong to you, stop and tell me which and why instead of changing it.
Show me the final node --test output.

What to look for in the responses. For prompt 1: does each test assert one rule, with exact expected values (not assert.ok(result))? Do the tests fail against the stub with "not implemented" rather than an import error? Are there cases you'd add (an empty string, a title that's all accents, a very long title)? For prompt 2: did the diff touch only slugify.mjs? Does the output show all tests passing, from a real run? Did it raise any test as wrong, and was it right?

AdvancedWays an agent games the tests

Ways an assistant "passes" tests without doing the work

These all happen with agents told to make tests pass. Review the test diff for each:

Editing or deleting the test ("updated the expected value to match the output"). Commit tests before the implementation starts, so any change shows in the diff, and say in the prompt that tests are read-only.
Weakening assertions: assert.equal(x, "cafe-deja-vu") becomes assert.ok(x) or assert.match(x, /cafe/).
Special-casing the test inputs: if (title === "Café Déjà Vu") return "cafe-deja-vu". Rare, but it happens; a few extra inputs in your review catch it.
Mocking the thing under test, so the test exercises the mock.
Skipping: test.skip, it.only left in, or a test filtered out of the run command.
Tests that mirror the implementation: when the assistant writes tests after the code, it tends to assert what the code does, including its bugs. That's why the tests come first and are reviewed against the spec, not against the code.

How do you use tests with an AI assistant?

Tests first, and I own them. I write the tests from the spec, or have the assistant draft them and review them line by line against the spec, then commit them and run them against a stub to make sure they fail for the right reason. Then the assistant implements with the tests read-only, running them itself and iterating on the exact failures, and it shows me the real output at the end. That gives an agent an independent definition of done, so it can loop without me watching. In review I look at the test diff first: deleted or weakened assertions, skips, mocks of the thing under test, or special cases for the test inputs are the ways an agent "passes" without doing the work. If the same assistant wrote code and tests together, I treat the tests with suspicion because they'll share its misunderstanding.

AdvancedGreen isn't the same as right

What does v1 do to " --Hello-- "?

v1 is s.toLowerCase().replace(/ /g, "-"). Trace the input " --Hello-- " and explain the reported result "----hello----".

Show answer

"----hello----": two spaces become two hyphens on each side, next to the two existing hyphens.

What’s happening

toLowerCase() gives " --hello-- ": two spaces, two hyphens, hello, two hyphens, two spaces.
replace(/ /g, "-") replaces each space, one by one, with a hyphen. The two leading spaces become --, giving ----hello-- followed by the two trailing spaces, which also become --.
Result: "----hello----", four hyphens on each side. Nothing collapses the run, and nothing trims the ends.
v2 fixes both with two regexes: /[^a-z0-9]+/g → "-" turns each run of anything that isn't a letter or digit into one hyphen ("-hello-"), and /^-+|-+$/g → "" removes hyphens at either end ("hello"). That's two of the five rules, and the test failure told you exactly which.

#Iterating and correcting

Added

The first response is a draft. Getting from draft to right is a conversation, and the quality of each turn depends on what you send back. The rule: steer with evidence, not adjectives. "It doesn't work" or "that's wrong, try again" gives the model nothing new, so it changes something at random. The failing input, the expected and actual values, the exact error message, and the rule it broke give it the same information you'd give a colleague, and usually get a targeted fix.

The mental model is a bug report to a colleague who's in another time zone: write it so they can fix it without asking you anything. Three moves cover most iterations:

Show the failure: input, expected, actual, the assertion or compiler output, verbatim.
Narrow: one problem per turn ("only fix the boundary case; don't change the return format"), and point at the place ("the comparison on line 12").
Ask for alternatives when the approach itself is wrong: "give me two different approaches, with their trade-offs, before writing code".

And one move that matters more than people expect: restart. Claude Code's best-practices page advises that after two failed corrections, /clear and a better prompt beat a third correction, because "a third correction mostly adds to a conversation that already contains two failed attempts for Claude to keep re-reading". GitHub's guidance is similar: start a new conversation for a new task, and remove turns that no longer help.

Here's the "show the failure" move as code: a draft that finds double-booked meeting rooms (written for this page) fails a test, and the script turns the failure into the next prompt.

error-to-prompt.mjs

// Iterating well: hand the model the exact failure (input, expected, actual, the assertion's own diff),
// not "it doesn't work". The function is a draft written for this page.
import assert from "node:assert/strict";

const bookings = [
  { room: "Huddle 2", start: "09:00", end: "09:30" },
  { room: "Board", start: "09:00", end: "10:00" },
  { room: "Huddle 2", start: "09:15", end: "09:45" },
];

// Draft: "find double-booked rooms". Compares start times only.
function findConflicts(list) {
  const seen = new Map();
  const conflicts = [];
  for (const b of list) {
    const key = `${b.room}@${b.start}`;
    if (seen.has(key)) conflicts.push([seen.get(key), b]);
    seen.set(key, b);
  }
  return conflicts.map(([a, b]) => `${a.room} ${a.start}-${a.end} overlaps ${b.start}-${b.end}`);
}

const expected = ["Huddle 2 09:00-09:30 overlaps 09:15-09:45"];
try {
  assert.deepEqual(findConflicts(bookings), expected);
  console.log("passed");
} catch (err) {
  const prompt = [
    "findConflicts() in src/rooms/conflicts.js fails this test:",
    "",
    "input:    " + JSON.stringify(bookings.map((b) => `${b.room} ${b.start}-${b.end}`)),
    "expected: " + JSON.stringify(expected),
    "actual:   " + JSON.stringify(err.actual),
    "",
    "assertion output:",
    ...err.message.split("\n").map((l) => "  " + l),
    "",
    "Two bookings conflict when they are in the same room and their time ranges overlap",
    "(end is exclusive: 09:00-09:30 and 09:30-10:00 do not conflict). Keep the return format.",
    "Explain the cause in one sentence before changing the code.",
  ];
  console.log(prompt.join("\n"));
}

// The next round: a version that follows the stated rule, checked against the original case and the boundary.
function findConflictsV2(list) {
  const out = [];
  list.forEach((a, i) => list.slice(i + 1).forEach((b) => {
    if (a.room === b.room && a.start < b.end && b.start < a.end) out.push(`${a.room} ${a.start}-${a.end} overlaps ${b.start}-${b.end}`);
  }));
  return out;
}
const boundary = [{ room: "Board", start: "09:00", end: "09:30" }, { room: "Board", start: "09:30", end: "10:00" }];
assert.deepEqual(findConflictsV2(bookings), expected);
assert.deepEqual(findConflictsV2(boundary), []);
console.log("\nv2: original case and back-to-back boundary both pass");

Result

findConflicts() in src/rooms/conflicts.js fails this test:

input:    ["Huddle 2 09:00-09:30","Board 09:00-10:00","Huddle 2 09:15-09:45"]
expected: ["Huddle 2 09:00-09:30 overlaps 09:15-09:45"]
actual:   []

assertion output:
  Expected values to be strictly deep-equal:
  + actual - expected
  
  + []
  - [
  -   'Huddle 2 09:00-09:30 overlaps 09:15-09:45'
  - ]
  

Two bookings conflict when they are in the same room and their time ranges overlap
(end is exclusive: 09:00-09:30 and 09:30-10:00 do not conflict). Keep the return format.
Explain the cause in one sentence before changing the code.

v2: original case and back-to-back boundary both pass

What’s happening

The draft keys bookings by room@start, so it only notices two bookings that start at exactly the same time. In the data, Huddle 2 has 09:00–09:30 and 09:15–09:45: they overlap, but their keys differ, so the draft returns [].
assert.deepEqual (from node:assert/strict, so it's the strict deep-equal) throws an AssertionError. Its message is Node's own diff: + actual - expected, with + [] and the expected array marked -. err.actual holds the real return value.
The catch assembles the correction prompt from facts only: which function and file, the input in a compact form, expected, actual, the assertion's output verbatim, then the rule ("same room and time ranges overlap") with the boundary decision made explicit (end is exclusive, so back-to-back bookings don't conflict), the constraint (keep the return format) and a request to explain the cause first.
"Explain the cause in one sentence before changing the code" is a cheap check on understanding: if the explanation is wrong ("it's a sorting problem"), you know before reading any code that the fix will be wrong too.
The last line shows the next round closing the loop: findConflictsV2 compares every pair with a.start < b.end && b.start < a.end (string comparison works because the times are zero-padded HH:MM in one format), and it's checked against the original case and the boundary case the prompt introduced. A rule you state in a correction should become a test, or it'll be broken again two iterations later.

AdvancedCorrection prompts that work

Instead of	Send	Why
"It doesn't work."	the failing input, expected, actual and the error output	gives the model the information you have
"That's wrong, try again."	"The overlap check misses partial overlaps: 09:00–09:30 and 09:15–09:45 should conflict."	names the rule it broke
"Fix the tests."	"Don't change the tests. `conflicts.test.js:14` fails with: …"	keeps the contract fixed
"Make it better."	"Keep the behaviour; reduce it to one pass over the list sorted by start time."	says which direction "better" is
"Rewrite everything."	"Only change `findConflicts`; leave the formatting helper as it is."	narrows the blast radius
a third correction of the same mistake	a new session with one prompt that includes what you learned	stops the failed attempts from anchoring the next one

AdvancedAsking for alternatives

When a fix keeps going sideways, the approach may be the problem. Ask for options before code:

Prompt: alternatives before code

Before writing code: give me two or three different ways to detect overlapping room bookings
for up to ~5,000 bookings per day, with the time complexity of each, how each handles
back-to-back bookings, and which you'd pick and why. Don't write the implementation yet.

The kinds of answers to expect, and to judge yourself: comparing every pair (simple, O(n²), fine for small n); sorting by start time within each room and comparing neighbours (O(n log n), the usual choice); and an interval tree or a database exclusion constraint for the case where bookings arrive one at a time and must be checked on insert (PostgreSQL can enforce non-overlapping ranges with an exclusion constraint). The value isn't that the model decides; it's that you see the option space quickly and you choose with your constraints in mind. Because answers are sampled, asking twice can surface different options.

AdvancedWhen to stop and write it yourself

Iteration has diminishing returns. Stop prompting and take over when:

the same mistake comes back after a correction, especially after a restart: the model's prior for this code is stronger than your instruction;
you've stopped understanding the diff: you can't own what you can't explain;
the fix is smaller than the explanation: a one-line change you can see is faster to type than to describe;
the problem is a decision, not code: which behaviour is right for users, which trade-off you accept.

Taking over isn't failure; it's using the tool where it's faster and yourself where you are.

How do you iterate when an assistant's first answer is wrong?

With evidence rather than adjectives. I send back the failing input, the expected and actual values and the exact error or assertion output, and I state the rule it broke, including any boundary decision, like end times being exclusive. I narrow each turn to one problem and say what not to touch, and I ask it to explain the cause before changing code, which tells me quickly whether it understood. If the approach is wrong, I ask for two or three alternatives with trade-offs before any code. If it repeats the same mistake after two corrections, I start a fresh session with a better prompt that includes what I learned, because the failed attempts in the history keep pulling it back. And if the fix is smaller than the explanation, I just write it.

AdvancedEvery correction is context too

Why did the draft return [] instead of finding the conflict?

Trace findConflicts on the three bookings: Huddle 2 09:00–09:30, Board 09:00–10:00, Huddle 2 09:15–09:45.

Show answer

Because it only matches identical room@start keys, and the two Huddle 2 bookings start at different times.

What’s happening

Booking 1: key "Huddle 2@09:00". Not in seen; store it.
Booking 2: key "Board@09:00". Different room, so a different key. Not in seen; store it. (Same start time as booking 1, but a different room, so correctly not a conflict.)
Booking 3: key "Huddle 2@09:15". Not in seen, because booking 1's key ends in 09:00. Store it. No conflict recorded.
The loop ends with conflicts empty, so the function returns []. The draft implements "same room, same start" when the rule is "same room, overlapping ranges"; the fix compares ranges: a.start < b.end && b.start < a.end.

#Spotting hallucinated APIs and subtle bugs

Added

AI-generated code fails in two ways that need different defences. Invented or stale APIs: methods, options, imports and whole packages that don't exist in the version you have installed, or that are deprecated there. And subtle bugs: code that uses only real APIs and runs without errors, but does the wrong thing at the edges. The first kind is the easier one, because a machine can catch it; the second kind needs tests and review aimed at the right places.

The mental model: trust the compiler, the tests and the installed package's own files, never the confidence. A suggestion reads equally fluently whether fs.readJson exists or not, so fluency tells you nothing. What does tell you: the type checker resolving every name against the real declarations, the code running against real inputs, and the package's exports and docs for the exact version in your lockfile. How they work explains why the names get invented; this topic is about catching them.

Here's a checker that does the first job without a type checker: it takes a suggested snippet (written for this page to look like a typical suggestion), imports every package it names from the real node_modules, and checks each member against the runtime exports and the package's .d.ts files.

verify-names.mjs

// Check every package and member a suggested snippet uses against what is actually installed:
// the runtime exports, plus @deprecated tags in the package's own .d.ts files.
import { readFileSync, existsSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { dirname, resolve } from "node:path";

const suggestion = `
import fs from "node:fs";
import { z } from "zod";
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { validateBody } from "express-zod-validator-pro";
const config = fs.readJson("config.json");
const Email = z.string().email();
const server = new McpServer({ name: "rooms", version: "1.0.0" });
server.addTool("book_room", { room: z.string() }, async () => ({ content: [] }));
server.tool("list_rooms", async () => ({ content: [] }));
server.registerTool("cancel", { inputSchema: { id: z.string() } }, async () => ({ content: [] }));
`; // written for this page to look like a typical suggestion; it was not produced by an AI tool

function declarations(spec) { // the .d.ts next to the resolved file, plus everything it re-exports
  if (spec.startsWith("node:")) return ""; // built-ins: check @types/node or the Node docs instead
  const seen = new Set(), texts = [];
  const visit = (file) => {
    if (seen.has(file) || !existsSync(file)) return;
    seen.add(file);
    const text = readFileSync(file, "utf8");
    texts.push(text);
    for (const [, rel] of text.matchAll(/from "(\.[^"]+)\.js"/g)) visit(resolve(dirname(file), rel + ".d.ts"));
  };
  visit(fileURLToPath(import.meta.resolve(spec)).replace(/\.c?js$/, ".d.ts"));
  return texts.join("\n");
}
const deprecation = (dts, member) => {
  const re = new RegExp(String.raw`/\*\*((?:(?!\*/)[\s\S])*)\*/\s*\n\s*${member}[<(]`, "g");
  const tag = [...dts.matchAll(re)].map((m) => m[1].match(/@deprecated\s*([^\n*]*)/)).find(Boolean);
  return tag ? `DEPRECATED: ${tag[1].trim()}` : "ok";
};

const bindings = {};
for (const [, names, spec] of suggestion.matchAll(/^import (.+?) from "([^"]+)";$/gm)) {
  let mod;
  try { mod = await import(spec); } catch { console.log(`${spec}: NOT INSTALLED (check it exists, and is the package you meant, before npm install)`); continue; }
  for (const name of names.replace(/[{}]/g, "").split(",").map((n) => n.trim())) {
    const value = names.startsWith("{") ? mod[name] : mod.default;
    console.log(`${spec} → ${name}: ${value === undefined ? "NOT EXPORTED" : "ok"}`);
    if (value !== undefined) bindings[name] = { value, spec };
  }
}
for (const [, v, cls] of suggestion.matchAll(/const (\w+) = new (\w+)\(/g)) bindings[v] = { value: bindings[cls].value.prototype, spec: bindings[cls].spec };

const checked = new Set();
for (const [, obj, call, member] of suggestion.matchAll(/\b(\w+)\.(?:(\w+)\(\)\.)?(\w+)\(/g)) {
  const b = bindings[obj];
  if (!b || checked.has(`${obj}.${call}.${member}`)) continue;
  checked.add(`${obj}.${call}.${member}`);
  const target = call ? b.value[call]() : b.value;
  const label = `${obj}.${call ? call + "()." : ""}${member}`;
  console.log(`${label.padEnd(20)} ${member in target ? deprecation(declarations(b.spec), member) : "DOES NOT EXIST"}`);
}

Result

node:fs → fs: ok
zod → z: ok
@modelcontextprotocol/sdk/server/mcp.js → McpServer: ok
express-zod-validator-pro: NOT INSTALLED (check it exists, and is the package you meant, before npm install)
fs.readJson          DOES NOT EXIST
z.string().email     DEPRECATED: Use `z.email()` instead.
server.addTool       DOES NOT EXIST
z.string             ok
server.tool          DEPRECATED: Use `registerTool` instead.
server.registerTool  ok

What’s happening

The first loop finds each import … from "…" line and really imports it. node:fs, zod and the MCP SDK's server/mcp.js resolve and export the named bindings. express-zod-validator-pro can't be imported: it isn't installed, and it also returned 404 from the npm registry when checked on 2026-10-27. A suggestion that names a package you don't have is a prompt to check the package exists, is the one you meant, and is maintained before npm install, because installing a name an attacker registered is how slopsquatting works.
const server = new McpServer(...) makes server stand for McpServer.prototype, so the checker can look up instance methods. The member loop then checks each obj.member( and obj.call().member( in the snippet against the real object.
fs.readJson and server.addTool don't exist. The first is from the fs-extra package; the second is a plausible name for the SDK's real method, which is registerTool. Both would fail at runtime with TypeError: … is not a function, but only when that line runs.
z.string().email and server.tool exist, but the declarations function follows each package's .d.ts re-exports and finds @deprecated tags on them: Use z.email() instead. (Zod 4.6.7) and Use registerTool instead. (MCP SDK 1.32.2). This is the stale API case: code in the style of an older version, which still runs, so nothing forces anyone to notice.
server.registerTool and z.string are fine. Five of the ten references had a problem, and none of them would have been obvious from reading. This is a model of what your editor and type checker do properly: in a TypeScript project, tsc reports the missing members and editors strike through deprecated ones. The point is the habit: resolve every name against the installed version, by tool, not by eye.

AdvancedLet the type checker do it

In a TypeScript project, most invented members never get past the compiler. These are the real messages from TypeScript 7.0.2 (tsc --strict with @types/node), checked for this page:

Caught at compile time

import * as fs from "node:fs";

const config = JSON.parse(fs.readJson("config.json")); // ❌ TS2339: Property 'readJson' does not exist on type 'typeof import("node:fs")'.
const raw = JSON.parse(fs.readFile("config.json")); // ❌ TS2554: Expected 2-3 arguments, but got 1. (and TS2345: Argument of type 'void' is not assignable to parameter of type 'string'.)

What’s happening

fs.readJson is reported as TS2339: the property doesn't exist on the module's type. That's the hallucinated name, caught before anything runs, with the exact location.
The second line uses a real function wrongly: fs.readFile needs a callback (TS2554, 2–3 arguments expected) and returns void, which JSON.parse can't take (TS2345). An existing name used with the wrong shape is just as common in suggestions as a missing one, and the types catch it too.
Plain JavaScript gets none of this: both lines would fail only when executed, and the second one would throw inside JSON.parse with a confusing message. That's a real argument for TypeScript (or at least // @ts-check and JSDoc types) in codebases where a lot of code is generated.
Deprecations are reported differently: TypeScript and editors show them as suggestions (strike-through in VS Code), not errors, so they're easy to ignore. Linters can turn them into errors; decide as a team whether you want that.

AdvancedSubtle bugs: real APIs, wrong behaviour

The second kind of failure passes the compiler. This one is a favourite because it reads so naturally:

async-foreach.mjs

// Plausible code that reads correctly and runs without an error, but doesn't wait for anything.
const sent = [];
const sendReminder = (room) => new Promise((r) => setTimeout(() => { sent.push(room); r(); }, 10));
const rooms = ["Huddle 2", "Board", "Lab"];

async function remindAllDraft() { // written for this page: the shape assistants (and people) often produce
  rooms.forEach(async (room) => {
    await sendReminder(room);
  });
  return `done, sent ${sent.length}`;
}

async function remindAll() { // forEach ignores the promises its callback returns; collect and await them
  await Promise.all(rooms.map((room) => sendReminder(room)));
  return `done, sent ${sent.length}`;
}

console.log("draft:", await remindAllDraft());
await new Promise((r) => setTimeout(r, 50)); // let the stray reminders finish before the next run
console.log("  …50 ms later sent =", sent.length);
sent.length = 0;
console.log("fixed:", await remindAll());

Result

draft: done, sent 0
  …50 ms later sent = 3
fixed: done, sent 3

What’s happening

remindAllDraft calls rooms.forEach(async (room) => { await sendReminder(room); }). Each callback is an async function, so it returns a promise immediately; the await inside only pauses that callback.
Array.prototype.forEach ignores its callback's return value, so the three promises are dropped. forEach returns right away, and the function returns "done, sent 0" before any reminder has been sent.
The reminders still go out later: 50 ms later sent is 3. So nothing errors, the reminders arrive, and the bug only shows up as "done" being logged too early, or as unhandled rejections and partial work when one of them fails and nobody is waiting.
remindAll maps each room to a promise and awaits Promise.all, so it returns "done, sent 3" after all three are sent, and a failure rejects the await where the caller can handle it. (For a sequential version, use for…of with await inside.)
No type checker flags this: every API is real and every type lines up. Bugs like this are caught by tests that assert when something completes, and by reviewers who know the hot spots in the table below.

AdvancedHot spots to review in generated code

Hot spot	What goes wrong	What to check
`sort()` without a comparator	sorts numbers as strings; sorts in place	`(a, b) => a - b`; `toSorted` or a copy
`async` callbacks in `forEach`, `map` without `await Promise.all`	nothing waits; errors are lost	`for…of` with `await`, or `Promise.all`
Dates and time zones	local vs UTC, `new Date("2026-10-27")` parsed as UTC midnight, DST	explicit zones, a date library, DST test cases
Money and floats	`0.1 + 0.2`, rounding in the wrong place	integer minor units, one rounding rule, tested
Regexes	unanchored patterns accept prefixes; catastrophic backtracking	`^…$`, test with hostile input
Paths and URLs	`path.join` with user input, prefix checks without a separator	resolve, then compare against `root + sep`
SQL and shell	string concatenation with input	parameterised queries, argument arrays
Error handling	`catch {}` that swallows, or retries without limits	errors surfaced or logged with context
Off-by-one and boundaries	`<` vs `<=`, inclusive vs exclusive ranges, pagination	boundary tests from the spec
Mutation	modifying an argument or shared state	copies, `readonly` types
Versions	APIs from an older major, deprecated options	the lockfile version's docs and `.d.ts`
New dependencies	an unknown or misspelt package	exists, intended, maintained, licence OK

How do you catch hallucinated APIs and subtle bugs in AI-generated code?

For invented or stale APIs I rely on machines, not reading: TypeScript resolves every member against the installed declarations, so fs.readJson or a made-up SDK method fails at compile time, and deprecated APIs show up as strike-through or lint warnings. I pin versions in the repository instructions so the assistant targets what we actually run, and give it the real types or docs for anything new or private. For any new package in a diff I check it exists, is the one we meant and is maintained before installing, because hallucinated package names are a real supply-chain risk; Spracklen et al. found that on average at least 5% of the packages commercial models recommended, and over 20% for open-source models, didn't exist. Subtle bugs pass the compiler, so I review the hot spots, like async in forEach, sorting, dates and time zones, money, regex anchors and path handling, and I make sure there are tests for the boundaries the spec names.

AdvancedSlopsquatting: packages the AI invented

Which line is the bug?

An assistant suggests this for "send reminders, then mark the batch as sent": bookings.forEach(async (b) => { await notify(b); }); await markBatchSent(batchId);. All functions exist and the types check. What's wrong, and what would you write instead?

Show answer

markBatchSent runs before any notify has finished, because forEach doesn't wait for the async callbacks. Use await Promise.all(bookings.map((b) => notify(b))) (or a for…of loop with await) before marking the batch.

What’s happening

forEach calls the async callback for each booking. Each call starts notify(b) and returns a pending promise straight away; forEach discards those promises.
forEach returns undefined immediately, so the next statement, await markBatchSent(batchId), starts while all the notify calls are still in flight.
If a notify fails, its rejection isn't connected to anything the caller awaits: the batch is already marked sent, and the failure surfaces, at best, as an unhandled rejection in the logs.
await Promise.all(bookings.map((b) => notify(b))) waits for all of them and rejects if any fails, so markBatchSent runs only after every reminder succeeded. If the notification service needs one call at a time, a for (const b of bookings) await notify(b); loop does them in order.
This is why the hot-spot table exists: no compiler error, no runtime error, plausible code, and a real bug in the ordering. Review knowledge, plus a test that asserts the batch isn't marked sent when one notification fails, catches it.