You have given the same exercise for three years. A small React component, an hour, a shared editor. It used to sort people. The strong ones finished early and talked about edge cases. The weak ones got stuck on state. This year everyone finishes, the code looks oddly alike, and you can't remember the last time the exercise changed a decision.
Last Tuesday Felix Rieseberg, an engineering manager at Anthropic, wrote that he had stopped using it. "This month, I laid the traditional coding interview to rest." On X he put it shorter: the coding interview is dead. The replies split the way they always do. Jaana Dogan, a principal engineer at Google, said she'd rather watch how people use coding agents. ZipRecruiter's economist told The Register that companies aren't abandoning skills tests, they're locking the room.
I think both camps are describing the same thing, and neither has named it. The coding interview didn't die. It saturated, the way benchmarks saturate, and people who build tests have known for decades what to do when that happens.
Three versions in two years
The best record of this is also from Anthropic. In January, Tristan Hume wrote up the take-home his performance engineering team had used since early 2024. Candidates optimise code for a simulated accelerator. He spent two weeks designing it so it would "capture the demands of the role," and it did. About a thousand candidates took it, and it "helped us hire most of our current performance engineering team."
Then a pre-release Claude Opus 4 sat it and, in his words, "came up with a more optimized solution than almost all humans did within the 4-hour limit." He cut the time to two hours, removed the multicore part, and leaned on clever insight over volume. Version two lasted until a pre-release Opus 4.5, which matched the best human score by the two-hour mark. Version three, the current one, is a set of puzzles on a tiny, artificial instruction set, chosen to be, in his words, "sufficiently out of distribution."
The first version lasted about sixteen months. The second, about six. Nobody knows how long the third gets.
The line in his post that stayed with me is about version three. "I'm still sad to have given up the realism and varied depth of the original. But realism may be a luxury we no longer have." That's a test designer admitting he traded away the thing that made the test predictive, because the realistic version could no longer tell a strong engineer from a strong engineer with a model.
That's saturation, and it has a name
In my post on evals I wrote that an item every model passes carries no information about which model to pick. Swap models for candidates and nothing changes. An exercise that everyone solves, because everyone has the same assistant, has zero discrimination. It still costs an hour of two people's time. It still produces a score. The score just doesn't separate anyone.
Rieseberg says it himself, more honestly than most. Solving small problems quickly "was a weak but useful predictor." That matches the research. Paul Sackett's 2022 revision of the hiring literature puts work samples at a correlation of .33 with later job performance. Weak, useful, and better than most of what hiring runs on. Then a tool arrived that every candidate can use and that solves the sample. A .33 item that everyone passes is a 0 item.
This is the same story as benchmark contamination, which I wrote about last month under its older name. A test works while the people taking it can't see the answers. Once they can, the scores go up and the information goes out.
Three ways out
Everyone arguing this week is picking one of three exits. Two of them are old moves with new clothes.
Lock the room. Proctored and in-person assessments. ZipRecruiter says 70% of information-sector employers have moved that way to deter cheating. Sundar Pichai has said Google will keep at least one in-person round. It works, in the narrow sense that the score is the candidate's again. But it now measures how someone codes without the tool they'll use every day. You get a clean measurement of a skill the job uses less each quarter.
Drop the test. This is Rieseberg's move, and he's candid that he hasn't found the replacement. For now he is "scanning resumes and portfolios for little prototypes that exhibit taste, empathy, and principled thinking." I like the instinct and distrust the method. Taste judged by one person, with no rubric written in advance, is an unstructured interview conducted on a GitHub profile. Sackett puts the unstructured interview at .19. And "hunger beats tenure," his headline, is the kind of criterion The Register pointed out has landed tech companies in age discrimination cases.
The coding exercise lived in the work-sample row. A portfolio read for taste, without a rubric, drops to the highlighted one.
Test the job as it is now. Hume's version three allows AI. Google is piloting a code-comprehension round with Gemini switched on. Meta and Canva run AI-assisted rounds. Dogan wants to watch people drive an agent. This is the only exit that measures the current job, and it's harder than it sounds, because "uses the agent well" isn't a score until somebody writes down what well means.
The job is mostly reading now
What does the current job look like? I wrote last week that the expensive part of AI-generated work is reading it. The model types. The engineer decides whether what it typed is true, whether it fits the fifty thousand lines already there, and what to throw away. That decision is the job. So that's what the interview should sample.
Here is what I'd build. Give the candidate a change an agent wrote against a realistic codebase, a few hundred lines with a handful of defects planted in it. Leave the agent available. Ask for a review. Score what they catch, what they flag that wasn't wrong, what they'd delete, and how they'd prove the fix works.
There's a nice property hiding in there. An ICML 2025 study found that on one leaderboard, when two models are both wrong, they agree on the wrong answer 60% of the time. Models share blind spots. Plant defects of that kind, the ones a second model waves through, and you get items the assistant can't saturate for the candidate. The test stays realistic and stays hard, for the same reason the job does.
The left exit measures yesterday's job cleanly. The middle one stops measuring. Only the right one samples the work as it's done now.
I haven't validated this. Nobody has, at scale, and anyone who tells you their AI-era interview is proven is selling it. Hume reports that version three's scores "correlate well with the caliber of candidates' past work," which is the right check, and he calls it early. That's the honest state of the field.
What I would change this quarter
If you run a technical loop, four things. None of them needs a vendor.
Check whether your exercise still discriminates. Pull pass rates for the last two years. If nearly everyone passes now, the round is dead weight, and you already have the data to prove it.
Pick an exit per round, out loud. AI banned in this round, required in that one, and the candidate told in writing which is which. A round where nobody said leaves the score meaning one thing for the candidate who used a model and another for the one who didn't.
Write the rubric before the first candidate. Whatever replaces the exercise, the questions and what a strong answer contains get written in advance and asked of everyone. That single habit is the gap between .19 and .42.
Close the loop. Log the interview score and look again at six months. If the round doesn't predict who did well, change it before the next hiring wave, not after.
Closing
The exercise was never the point. It was an item, and items wear out. The question for the next hiring round isn't whether to keep interviewing engineers. It's what you'll ask them that the model can't answer for them, and how many months that question will last.