How AI coding actually works

Almost every frustration you'll have with these tools comes from one misunderstanding: people expect the AI to be a very fast, slightly dim employee who remembers everything. It isn't. It's something stranger, and once you see the actual shape of it, the tool stops feeling random and starts feeling predictable.

What the model actually is

A large language model is a system that takes a pile of text and predicts what text should come next. That's genuinely all it does. It has read an enormous amount of code — public repositories, documentation, tutorials, forum answers — and has become extremely good at producing text that looks like the correct continuation.

Two consequences fall straight out of that, and they explain most of the tool's behaviour:

• It's brilliant at common things and shaky at rare things. A login form in a popular framework? It has seen a hundred thousand of them. Your industry's peculiar invoicing rules? It has seen none, and will confidently invent something reasonable-looking.
• It doesn't know what it doesn't know. There's no internal "I'm unsure" flag it can consult. A guess and a certainty come out in the same confident tone. This is the single most important thing to internalise.

Context: the only concept you truly must understand

The model has no memory between sessions and no ability to see anything you haven't shown it. Everything it knows about your project in a given moment is in its context window — a working area holding your conversation so far, plus whatever files the tool has read.

Think of it as a desk. Big, but finite. Everything the AI can see must be on the desk. If a file isn't on the desk, the AI does not know it exists, and will happily write a second version of something you already have.

Tokens, and why sessions cost money

Text is chopped into tokens — roughly ¾ of a word each. You're billed (or rate-limited) on tokens in and tokens out. The bit that surprises people: every time you send a message, the entire conversation so far goes with it. Message fifty in a session costs far more than message one, because it's carrying everything before it.

This is why "one long session for the whole day" is both worse quality and more expensive than several focused ones. You'll find the practical cost controls in Security and spend.

What makes these "agents" rather than chatbots

ChatGPT in a browser can write code and hand it to you. Claude Code and Cursor's agent mode do something meaningfully different: they can act on your computer. They read your actual files, write changes to disk, run commands, read the errors that come back, and try again.

That loop — act, observe, correct — is why they can complete real work. It's also why they can do real damage, and why the guide spends a whole page on version control before you build anything.

1. You describe what you want.
2. The agent decides what it needs to know and reads some files.
3. It edits files, or creates new ones.
4. It runs something — the app, a test, a build — and reads the output.
5. If that output contains an error, it goes back to step 2 with the error on the desk.

Your job sits at step 1 and after step 5: give it a target it can actually hit, then check what it hit. Everything in the middle is its business, not yours.

Why it sometimes goes very wrong

Four failure modes cover nearly everything you'll hit. Recognising them by name is most of the cure.

1. The doom loop

Something's broken. The AI fixes it. A different thing breaks. It fixes that. The first thing breaks again. Each "fix" is a plausible local patch that doesn't address the real cause, and the mess compounds. Cure: stop, undo everything back to the last working state, start a new session, and describe the original problem rather than the current pile of symptoms.

2. Confident invention

It uses a setting or function that doesn't exist, because something very like it does exist elsewhere. Cure: ask it to check the actual documentation or the actual installed version, and always run the code.

3. Scope creep

You asked for a button; you got a button, a new design system, three new dependencies and a refactor of two unrelated files. Cure: say what's out of bounds. "Only change Header.tsx. Don't add libraries. Don't touch styling elsewhere." Then review what changed before you accept it.

4. The plausible shell

The nastiest one. You get a beautiful settings page with every toggle in place — and none of the toggles are wired to anything. It looks finished. Cure: never trust a screenshot, including your own. Click the thing. Refresh the page. Check the data actually changed.

What this means for how you work

You're not a coder now. You're something closer to a demanding product manager with an unusually fast, unusually literal team that has amnesia. The skills that make you good at this are:

• Specificity. Being able to say exactly what you want, including what you don't want.
• Verification. Checking the work rather than accepting the summary of the work.
• Decomposition. Turning "build my app" into thirty tasks that each fit on a desk.
• Knowing when to throw away. Undoing twenty minutes of bad work costs twenty minutes. Nursing it costs days.

I'm a non-technical founder building [one-sentence description of your app] using AI coding tools.

Based on how large language models actually work, tell me:

1. Which parts of this app you'd expect to generate reliably first time, and why (be specific about why they're well represented in training data).
2. Which parts you'd expect to get subtly wrong, and what the wrongness would look like — not "it might have bugs" but "it will probably handle X in a way that seems fine until Y".
3. Which parts you cannot know without reading the actual documentation for the libraries involved, and should therefore look up rather than recall.
4. Three specific things I should manually test that I probably wouldn't think to test.

Be concrete and a bit pessimistic.

For the next task I give you, narrate your work as you go in a way that teaches me:

- Before reading any file, say what you expect to find in it and why.
- After reading, say whether you were right, and what surprised you.
- Before each edit, say in one sentence what you're changing and what could break as a result.
- After running anything, quote the actual output, not a summary of it.

Then do this task: [small task]

I'm learning to supervise AI coding work, so the narration matters more to me than speed.