AI-Assisted Development Today

The loop is the same everywhere; what changes is what the tools can reach and who's watching:

• Editor agents (Copilot agent mode, Cursor's agent, the Claude Code IDE extensions) edit your working copy while you watch the diff appear. Tools are the editor's own (read, search, edit, run a terminal command), plus MCP servers you configure. You're usually present, so approvals are cheap.
• Terminal agents (Claude Code, Copilot CLI) have the shell: they can run your build, your test runner, git, curl. That makes them powerful for "make it pass" loops and dangerous without permission rules, which is why both products document approval modes. GitHub's changelog for Copilot CLI (2026-02-25) describes a plan mode in which Copilot "asks clarifying questions, and builds a structured implementation plan before writing any code", and an autopilot mode that keeps "iterating without stopping for approval".
• Cloud agents (Copilot cloud agent, Claude Code on the web) run on the vendor's machines in a fresh checkout, so you aren't watching. Their output is a branch or a pull request, and review becomes the only checkpoint. GitHub's changelog entry of 2026-04-01 ("Research, plan, and code with Copilot cloud agent", the one that announced the new name) says the cloud agent can now work on a branch without opening a pull request, or produce a plan for you to review before it writes code.

The choice is mostly about feedback: work you want to steer turn by turn belongs in the editor; a well-specified task with a test that defines "done" can go to a terminal or cloud agent.

Completions look like magic because the request is invisible. GitHub described what goes into Copilot's completion prompts in "How GitHub Copilot is getting better at understanding your code" (Johan Rosenkilde, GitHub blog, May 2023, updated February 2024): prompts are "compilations of IDE code and relevant context", built from the code before the cursor (the prefix), the code after it (the suffix, which "fill-in-the-middle" training lets the model use), and snippets from other open files ("neighboring tabs"), chosen to fit a small budget; at the time its fast models could process "about 6,000 characters at a time". This model assembles a request the same way. It's the idea from that article, not Copilot's actual algorithm, and today's limits are different.

// How a completion request can be assembled: the code before the cursor (prefix), the code after it (suffix),
// and snippets from other open files that share identifiers with the code near the cursor. This follows the
// ideas in GitHub's 2023 write-up on Copilot's prompts; it isn't Copilot's actual algorithm.
const editing = `import { Room } from "./types";
export function freeRooms(rooms: Room[], at: Date): Room[] {
  return rooms.filter((r) => ▮);
}
export function roomLabel(r: Room) { return r.name + " (" + r.capacity + ")"; }`;

const openTabs = {
  "types.ts": "export interface Room { id: string; name: string; capacity: number; bookings: Booking[] }\nexport interface Booking { start: Date; end: Date }",
  "booking.ts": "export function overlaps(b: Booking, at: Date) { return b.start <= at && at < b.end; }",
  "theme.ts": "export const colors = { primary: '#84cc16', surface: '#0f1a04' };",
};

const BUDGET = 6_000; // characters; the 2023 article says its fast models handled "about 6,000 characters"
const KEYWORDS = new Set(["import", "export", "from", "function", "return", "const", "interface", "string", "number"]);
const words = (s) => new Set(s.match(/[A-Za-z_]\w+/g).filter((w) => !KEYWORDS.has(w)));
const [prefix, suffix] = editing.split("▮");
const nearCursor = words(prefix.slice(-200) + suffix.slice(0, 50));

const scored = Object.entries(openTabs).map(([file, text]) => {
  const w = words(text);
  const shared = [...w].filter((x) => nearCursor.has(x));
  return { file, text, shared, score: shared.length / new Set([...w, ...nearCursor]).size };
}).sort((a, b) => b.score - a.score);

let prompt = "";
for (const s of scored) {
  const keep = s.score > 0.05; // a deliberately low bar, as the article reports worked best
  console.log(`${s.file.padEnd(11)} score ${s.score.toFixed(2)} shared [${s.shared.join(", ")}] ${keep ? "→ included" : "→ skipped"}`);
  if (keep) prompt += `// from ${s.file}\n${s.text.replace(/^/gm, "// ")}\n`;
}
prompt = `${prompt}<PRE>${prefix}<SUF>${suffix}<MID>`;
console.log(`prompt: ${prompt.length} of ${BUDGET} characters; the model generates the missing middle after <MID>`);
console.log(prompt);

booking.ts  score 0.17 shared [at, Date] → included
types.ts    score 0.13 shared [Room, Date] → included
theme.ts    score 0.00 shared [] → skipped
prompt: 495 of 6000 characters; the model generates the missing middle after <MID>
// from booking.ts
// export function overlaps(b: Booking, at: Date) { return b.start <= at && at < b.end; }
// from types.ts
// export interface Room { id: string; name: string; capacity: number; bookings: Booking[] }
// export interface Booking { start: Date; end: Date }
<PRE>import { Room } from "./types";
export function freeRooms(rooms: Room[], at: Date): Room[] {
  return rooms.filter((r) => <SUF>);
}
export function roomLabel(r: Room) { return r.name + " (" + r.capacity + ")"; }<MID>

The model is the same few model families across many products. The harness is where products differ, and it's what you configure as a team:

That's also why "which model is best" is the wrong first question for a team: the same model with better context, better instructions and a test it can run gives better results. The Prompting for Code chapter is about the context half of that.

Underneath every chat panel and agent is an API request that contains the whole conversation. This is the shape of a request to Anthropic's Messages API at the agent's third turn, written from the API reference (checked 2026-10-27); it wasn't sent anywhere. Other vendors' APIs differ in field names, not in the idea.

{
  "model": "claude-sonnet-5-5",
  "max_tokens": 4000,
  "system": "You are a coding agent. Repository instructions: run `npm test` to check your work…",
  "tools": [
    { "name": "read_file", "description": "Read a file in the repository",
      "input_schema": { "type": "object", "properties": { "path": { "type": "string" } }, "required": ["path"] } },
    { "name": "run_tests", "description": "Run the test suite and return its output",
      "input_schema": { "type": "object", "properties": {} } }
  ],
  "messages": [
    { "role": "user", "content": "The cart test fails. Fix it." },
    { "role": "assistant", "content": [
      { "type": "tool_use", "id": "toolu_01", "name": "read_file", "input": { "path": "src/cart.js" } } ] },
    { "role": "user", "content": [
      { "type": "tool_result", "tool_use_id": "toolu_01",
        "content": "export const total = (items) => items.reduce((sum, i) => sum + i.price, 0);" } ] }
  ]
}

Three things about tokens matter day to day:

• Limits and prices are in tokens, not characters or lines. Context window sizes, output limits and API bills are all counted in tokens. The "about four characters" rule is fine for estimates; for exact numbers use the vendor's counter (Anthropic's docs point to a token-counting API; OpenAI's to its Tokenizer tool and the tiktoken library).
• Code and non-English text tokenise differently from English prose. Identifiers split into pieces (readFileSync is likely several tokens), long runs of punctuation and minified code are expensive, and many languages other than English use more tokens per word. Treat the rule of thumb as a rule of thumb.
• A name is generated piece by piece. Because a long identifier is several tokens chosen one after another, a model can start a real name and finish it with a plausible but nonexistent suffix (readJson + Sync). That's one more reason invented names look so convincing: every piece was individually likely.

Context windows are large now (Anthropic's docs list 1M tokens for its current models and 200k for some older ones, checked 2026-10-27), but agents fill them fast, because whole files, tool output and every previous turn ride along. The budget is shared by everything, including the answer, and the harness has to decide what goes in. This model packs one request by priority with the characters-÷-4 estimate, no tokenizer needed. The window is deliberately small (32k) so the numbers are readable.

// Tokenizer-free context budgeting: estimate tokens as characters / 4 (a rule of thumb for English and code;
// real tokenizers differ, so use the vendor's token counter when it matters) and pack by priority.
const WINDOW = 32_000; // a small window keeps the numbers readable; current models offer 128k to 1M
const RESERVED_FOR_OUTPUT = 4_000;
const est = (chars) => Math.ceil(chars / 4);

function pack(items) {
  let used = 0;
  const budget = WINDOW - RESERVED_FOR_OUTPUT;
  const rows = [];
  for (const item of [...items].sort((a, b) => a.priority - b.priority)) {
    const tokens = est(item.chars);
    const fits = used + tokens <= budget;
    if (fits) used += tokens;
    rows.push(`${fits ? "  in " : "  OUT"} ${item.name.padEnd(34)} ${String(tokens).padStart(6)}`);
  }
  rows.push(`  used ${used} of ${budget} tokens (${WINDOW} window - ${RESERVED_FOR_OUTPUT} reserved for the answer)`);
  return rows.join("\n");
}

const base = [
  { name: "system prompt + tool definitions", chars: 14_000, priority: 0 },
  { name: "repo instructions file", chars: 3_200, priority: 1 },
  { name: "the task (your message)", chars: 600, priority: 2 },
  { name: "file being edited: billing.ts", chars: 18_000, priority: 3 },
  { name: "types it imports: invoice.ts", chars: 5_600, priority: 5 },
  { name: "its test: billing.test.ts", chars: 9_000, priority: 6 },
  { name: "older chat turns", chars: 22_000, priority: 7 },
];
const bigLog = { name: "pasted CI log (whole file)", chars: 72_000, priority: 4 };
const smallLog = { name: "pasted CI log (failing test)", chars: 1_400, priority: 4 };

console.log("A: the whole CI log pasted in");
console.log(pack([...base, bigLog]));
console.log("B: only the failing test's output pasted");
console.log(pack([...base, smallLog]));

A: the whole CI log pasted in
  in  system prompt + tool definitions     3500
  in  repo instructions file                800
  in  the task (your message)               150
  in  file being edited: billing.ts        4500
  in  pasted CI log (whole file)          18000
  OUT types it imports: invoice.ts         1400
  OUT its test: billing.test.ts            2250
  OUT older chat turns                     5500
  used 26950 of 28000 tokens (32000 window - 4000 reserved for the answer)
B: only the failing test's output pasted
  in  system prompt + tool definitions     3500
  in  repo instructions file                800
  in  the task (your message)               150
  in  file being edited: billing.ts        4500
  in  pasted CI log (failing test)          350
  in  types it imports: invoice.ts         1400
  in  its test: billing.test.ts            2250
  in  older chat turns                     5500
  used 18450 of 28000 tokens (32000 window - 4000 reserved for the answer)

Fitting in the window isn't the same as being used well. Two findings worth naming:

• Context rot. Anthropic's context-window docs (checked 2026-10-27) say it directly: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot", so "curating what's in context [is] just as important as how much space is available."
• Lost in the middle. Liu et al., "Lost in the Middle: How Language Models Use Long Contexts" (arXiv 2307.03172, July 2023; published in TACL), found models did best when the relevant information was at the start or end of a long input, and significantly worse when it was in the middle, even for models built for long contexts. Models have improved since; the direction of the advice hasn't changed. GitHub's prompt-engineering page for Copilot (checked 2026-10-27) says the same thing in product terms: "Copilot gives extra weight to instructions at the beginning and end of a prompt."

So the practical rules are: include what the task needs (the types, the test, the exact error), leave out what it doesn't (the whole log, unrelated files), and start a fresh session for a new task so the old one's history isn't competing for attention. GitHub's page lists "Start a new conversation or session when you begin a new task", and Claude Code's best-practices page recommends /clear between unrelated tasks "because a focused context gets you better answers and costs less."

Put the four facts together and invented APIs are what you'd expect, not a rare glitch:

• Training mixes versions. The model saw Zod 3 code for years and Zod 4 for much less time, Express 4 far more than Express 5, early MCP SDK examples more than current ones. It writes the average of what it saw. In the Zod 4.6 installed for this course, z.string().email() still works but its declaration file marks it @deprecated Use z.email() instead; in the MCP TypeScript SDK 1.32.2, server.tool(...) is marked @deprecated Use registerTool instead. Older-style code from an assistant is common, and often still runs, which makes it easy to miss.
• Neighbouring libraries bleed together. fs.readJson (fs-extra), axios-style options passed to fetch, a Lodash function called on a native array. Each is right somewhere.
• Private APIs are invisible. Your company's SDK, a device's REST API, an internal helper: unless it's in the context, the model guesses from names, and the guess is shaped like a common public API.
• Package names too. Spracklen et al., "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs" (arXiv 2406.10279, June 2024, revised March 2025, USENIX Security 2025), generated 576,000 code samples with 16 models and found that on average at least 5.2% of the packages recommended by commercial models, and 21.7% of those recommended by open-source models, didn't exist: 205,474 unique hallucinated names. An attacker who registers such a name gets installed by anyone who trusts the suggestion (a practice sometimes called "slopsquatting").

The defence isn't a better prompt alone; it's a check in the loop: the compiler or type checker, the tests, a look at the installed package's exports and docs. Spotting hallucinated APIs and subtle bugs builds a checker that does exactly that against the real node_modules.

Because each token is sampled, two runs of the same prompt can produce different code, and a "regenerate" button is just another draw. That has practical consequences:

• One good answer isn't proof the prompt is good, and one bad answer isn't proof it's bad. If a team prompt file matters, try it several times.
• Editor tools rarely let you set temperature or other sampling settings; they pick them for you. Don't plan on "set temperature to 0" as a reliability strategy, and remember the model above: low temperature only makes the most likely answer more likely, right or wrong.
• Variety is useful on purpose. Asking for "three different approaches, with trade-offs" uses the same mechanism to explore alternatives (see Iterating and correcting).

Subtle logic: a daily meeting across a DST change

Time is the classic "short, plausible, wrong" domain. A booking system for meeting rooms repeats a 09:00 meeting daily; the obvious implementation adds 24 hours. In Toronto, daylight saving time ends on Sunday 1 November 2026.

// "Repeat this 09:00 meeting daily": adding 24 hours vs adding a calendar day, across the end of daylight
// saving time in Toronto (Sunday 2026-11-01, when clocks go from 02:00 back to 01:00).
const TZ = "America/Toronto";
const fmt = new Intl.DateTimeFormat("en-CA", {
  timeZone: TZ, weekday: "short", month: "short", day: "numeric", hour: "2-digit", minute: "2-digit", hour12: false,
});
const offsetMinutes = (d) => { // the zone's UTC offset at instant d, in minutes
  const p = Object.fromEntries(new Intl.DateTimeFormat("en-CA", { timeZone: TZ, hourCycle: "h23",
    year: "numeric", month: "2-digit", day: "2-digit", hour: "2-digit", minute: "2-digit" })
    .formatToParts(d).map((x) => [x.type, x.value]));
  return (Date.UTC(p.year, p.month - 1, p.day, p.hour, p.minute) - d.getTime()) / 60000;
};

const first = new Date("2026-10-30T13:00:00Z"); // Friday 09:00 in Toronto (EDT, UTC-4)

// Plausible version: a day is 24 hours of milliseconds
const plusHours = (d, n) => new Date(d.getTime() + n * 24 * 3600 * 1000);

// Calendar version: keep the local wall-clock time, then convert back with that day's offset
function plusCalendarDays(d, n) {
  const guess = new Date(d.getTime() + n * 24 * 3600 * 1000);
  return new Date(guess.getTime() + (offsetMinutes(d) - offsetMinutes(guess)) * 60000);
}

for (let n = 0; n <= 3; n++) {
  console.log(`day ${n}: +24h ${fmt.format(plusHours(first, n)).padEnd(22)} calendar ${fmt.format(plusCalendarDays(first, n))}`);
}

day 0: +24h Fri, Oct 30, 09:00     calendar Fri, Oct 30, 09:00
day 1: +24h Sat, Oct 31, 09:00     calendar Sat, Oct 31, 09:00
day 2: +24h Sun, Nov 1, 08:00      calendar Sun, Nov 1, 09:00
day 3: +24h Mon, Nov 2, 08:00      calendar Mon, Nov 2, 09:00

• Boilerplate and repetition: the fifth route handler, DTOs from a schema, mapping code, config files, migrations that follow a pattern already in the repo.
• Tests for code that exists: a first set of cases, including edge cases you might not list, if you review them (assistants also write tests that assert the current bug; see Tests first with an assistant). GitHub's best-practices page lists "Writing tests and repetitive code" first among what Copilot is good at.
• Mechanical refactors with a safety net: renames across files, extracting a function, converting callbacks to async/await, when tests and the type checker will catch mistakes.
• Explaining unfamiliar code or errors: a starting hypothesis about what a function or a stack trace means, faster than reading cold. Check the parts you'll rely on.
• One-off scripts, regexes, shell pipelines, SQL: small, verifiable by running them on a sample.
• Translating between languages or frameworks you know on both sides (Angular to React, a Python script to Node), where you can judge the result.

• Subtle logic: time zones and DST, money and rounding, concurrency and ordering, retries and idempotency, caching and invalidation. The code is short and plausible; the bug is an invariant the model didn't know.
• Security-sensitive code: auth, input handling, file paths, SQL, crypto, secrets. The studies above, and the model above, are why.
• Code you can't evaluate: an unfamiliar language, framework or domain. "Looks fine" from someone who can't tell is the most dangerous review there is.
• Version-sensitive and private APIs: new library majors, your internal SDK, a device's REST API. The model writes the API it expects to exist.
• Large, cross-cutting design decisions: an assistant will happily produce an architecture; it can't know your team's constraints, history and operating costs unless you tell it, and its confident structure can anchor a discussion that should have stayed open.
• When there's no check: no tests, no types, no way to run it. Then every generated line is only as good as your reading of it.

The single most useful habit with agents is to ask for evidence and read it: the command that ran, its exit code and its output; the screenshot next to the mockup; the before-and-after query plan. "I've fixed it and all tests pass" is a claim. node --test output showing # pass 42 # fail 0, on the branch you're about to merge, is evidence. Claude Code's docs make the same point: reading the output "is faster than rerunning the check yourself", and for a session you weren't watching "it's also the only evidence you have".

Practical rules that follow:

• Define done before you start: a test to turn green, a command that must exit 0, a page that must match a mockup. An agent with a check can loop until it passes; without one, it stops when the work looks done.
• Keep changes small enough to review. A 1,500-line AI-generated diff gets skimmed, and skimmed diffs are where the path-traversal bug ships. Ask for one concern per change.
• Review the tests separately from the code. If the same assistant wrote both, a wrong understanding shows up in both, and they agree with each other.
• Watch for scope creep: new dependencies, reformatted files, "while I was there" refactors, deleted tests. None of them were in the ticket.
• Be able to explain every line. If you can't explain why a line is there, you can't own it; ask, test or rewrite it.

Before reading an agent's diff line by line, check that its shape matches the ticket: which files changed, whether tests were deleted, whether dependencies or CI changed. It takes seconds and catches the scope creep agents are prone to. The diff summary below is made up for the example, in git diff --numstat format.

// Before reading an agent's diff line by line, check its shape against the ticket.
// The diff summary is made up for this page (git diff --numstat style: added, deleted, path).
const ticket = { title: "Return 409 when a room is double-booked", scope: ["src/rooms/", "test/rooms/"] };
const numstat = `
42	3	src/rooms/bookings.js
18	0	test/rooms/bookings.test.js
0	27	test/rooms/conflicts.test.js
2	0	package.json
311	4	package-lock.json
6	6	.github/workflows/ci.yml
120	118	src/lib/format.js`;
const packageJsonDiff = `+    "date-fns-tz-lite": "^0.3.1",`; // the added dependency line

const flags = [];
for (const line of numstat.trim().split("\n")) {
  const [added, deleted, path] = line.split("\t");
  const inScope = ticket.scope.some((s) => path.startsWith(s));
  if (path.startsWith("test/") && Number(added) === 0 && Number(deleted) > 0) flags.push(`${path}: ${deleted} test lines deleted, none added`);
  else if (path === "package.json") flags.push(`${path}: new dependency ${packageJsonDiff.match(/"([^"]+)"/)[1]} (not in the ticket)`);
  else if (path.endsWith("-lock.json")) flags.push(`${path}: lockfile changed (${added} lines): follows the new dependency`);
  else if (path.startsWith(".github/")) flags.push(`${path}: CI config changed`);
  else if (!inScope && Math.abs(Number(added) - Number(deleted)) < 10 && Number(added) > 50) flags.push(`${path}: ${added}+/${deleted}- outside scope, looks like a reformat`);
  else if (!inScope) flags.push(`${path}: outside the ticket's scope`);
}
console.log(`ticket: ${ticket.title}`);
console.log(flags.length ? flags.map((f) => "  ⚑ " + f).join("\n") : "  shape matches the ticket");

ticket: Return 409 when a room is double-booked
  ⚑ test/rooms/conflicts.test.js: 27 test lines deleted, none added
  ⚑ package.json: new dependency date-fns-tz-lite (not in the ticket)
  ⚑ package-lock.json: lockfile changed (311 lines): follows the new dependency
  ⚑ .github/workflows/ci.yml: CI config changed
  ⚑ src/lib/format.js: 120+/118- outside scope, looks like a reformat

Some responsibilities stay with people no matter how good the tools get, and an interviewer for a lead role will listen for them:

• Correctness and security of what ships: the reviewer and the author are accountable, as they were before.
• Licensing and data: what code and data you paste into which tool, and whether generated code may match licensed public code. That's a team policy question (see Security, licensing and data concerns).
• Design and trade-offs: the assistant can list options; deciding which constraints matter for your system is the job.
• Understanding: when the 2 a.m. incident comes, the person on call needs to know how the code works, whoever typed it.