Old bugs announced themselves. Segfaults, stack traces, obviously-wrong output — the machine told you, rudely, that you’d made a mistake.
The characteristic failure of generated code is different. It compiles. It passes the happy-path test. It reads better than what you’d have written at 6pm on a Thursday. And it’s wrong in a way you’ll discover three weeks later, in production, at scale.
We call it plausible wrongness, and building an immune system against it is one of the core skills we train. A few patterns from our review rubric:
The confident edge case
Generated code handles the case you described and silently invents behavior for the cases you didn’t. The fix is a habit, not a tool: before accepting any generated function, write down what should happen at the boundaries — empty input, duplicate input, concurrent calls — then read the code against your list. Reading without a list is how plausibility wins.
The imported assumption
The model has seen ten thousand versions of your problem, and it will quietly import the most common context’s assumptions into your context: a default timezone, an unbounded cache, pagination that assumes small data. The question to ask in review: “what codebase does this code think it lives in?”
The test that tests nothing
Ask a model for tests and you’ll often get beautiful scaffolding that asserts the code does what the code does. Tautology with green checkmarks. Our rule: the test author (human or machine) must be given the spec, never the implementation, and at least one test must encode a fact about the domain the implementation could plausibly get wrong.
Why this is good news
Every one of these defenses is teachable, reviewable, and durable across tool generations. Prompting tricks expire with every model release; verification discipline compounds. That’s why our Waypoint rubrics grade “AI collaboration quality” as its own line — knowing when the machine is lying to you is the new debugging, and it’s very much a human skill.