Felix Rieseberg, an engineering manager at Anthropic, said on October 27, 2026 that he has removed traditional coding interviews from his team's hiring process. Anthropic's performance engineering team has also repeatedly changed its take-home test to keep pace with AI progress. The second version was matched by Claude Opus 4.5 to the best human candidate's score within months of its introduction. As Claude improves, timed coding tests are becoming a harder way to identify top talent.
The coding test has not been abolished across all of Anthropic, however. While Rieseberg's team has dropped it, the performance engineering team has introduced a new assignment that allows AI use. The test is designed to be hard for AI to solve, but it has shifted from code optimization close to real work to puzzles with unusual constraints. Resisting AI while still measuring the skills the job actually requires is proving difficult.
Only some teams dropped coding interviews; company-wide policy is unchanged
Rieseberg, who has been at Anthropic since 2025 and has led development of Claude.ai, Cowork, and Claude Code Desktop, explained his reasoning in an October 27 blog post, Hunger beats tenure: Advice for junior engineers. He wrote that the ability to grasp a problem quickly and arrive at an elegant solution used to be a "weak but useful" predictor of how well a candidate would perform. Now, he concluded, Claude produces better results than people even on small React exercises, and test results are less connected to the abilities of engineers who deliver major results.
In a post on X, Rieseberg stated that, at least on his own team at Anthropic, coding interviews had stopped being an effective evaluation method, so he removed them from the process in October. He has not yet settled on a replacement. For now he reviews résumés and portfolios and values small prototypes that convey design sense, empathy, and principled judgment. He also said he does not expect traditional coding interviews to return.
Meanwhile, Anthropic's guidance for candidates asks applicants not to use AI tools such as Claude during take-home assignments and live interviews unless told otherwise. The take-home test designed by Tristan Hume of the performance engineering team is an exception that explicitly permits AI use. In other words, Anthropic itself runs hiring processes side by side that measure skills without AI, evaluate the ability to work with AI, and dispense with coding tests entirely.
The second take-home version was cracked by Claude within months
Tristan Hume, who leads Anthropic's performance optimization team, described how the take-home test was repeatedly rebuilt in Designing AI-resistant technical evaluations, published January 21, 2026. The assignment asks candidates to optimize a program running on a simulated accelerator to reduce the number of clock cycles it needs. Fewer cycles earn a higher rating, and the test was meant to measure the optimization skills real performance engineering work demands.
Hume designed the first test over about two weeks in November 2023 and began using it in early 2024. Roughly 1,000 candidates took it over the next year and a half, and it was used in hiring for the performance engineering team. But by May 2025, Claude 3.7 Sonnet had reached a level at which, for more than 50% of test-takers, handing the whole task to Claude Code produced a better result.
| Date | Event |
|---|---|
| November 2023 | Hume designs the take-home test in two weeks. Time limit: 4 hours |
| Early 2024 | Version 1 goes into use. About 1,000 candidates take it over the next year and a half |
| May 2025 | A pre-release Claude Opus 4 outperforms nearly all candidates within 4 hours; the test changes to a 2-hour Version 2 |
| May 22, 2025 | Claude Opus 4 released |
| Before November 24, 2025 | A pre-release Claude Opus 4.5 matches the best human score within 2 hours. The exact test date was not disclosed |
| November 24, 2025 | Claude Opus 4.5 released |
| January 21, 2026 | Hume publishes how the test changed. Version 3 moves to a puzzle format |
| October 27, 2026 | Rieseberg announces he has dropped coding interviews on his own team |
On the first test, a pre-release Claude Opus 4 outperformed nearly all human candidates within the time limit, though there was still room to evaluate the strongest ones. Hume therefore built a second version that starts from the point where Opus 4 began to struggle and removes multicore processing. He also cut the time limit from four hours to two, mainly because scheduling test dates was difficult and was delaying hiring by several weeks.
The second version worked at first, but according to Hume it could separate the strongest candidates from Claude's abilities for only "a few months." When he tried a pre-release Claude Opus 4.5, it matched the best human score within two hours. The top human candidate had also made heavy use of Claude 4.
The challenge Hume has made public starts from the original optimization task like Version 1, but it uses the same instruction set and single-core configuration as Version 2, so cycle counts are comparable. On it, Opus 4.5 running in a normal Claude Code session recorded 1,790 cycles, roughly equal to the best human score in two hours. In a dedicated evaluation environment with more inference-time compute, Opus 4.5 improved to 1,579 cycles in two hours, surpassing the best human score. The result Anthropic cited in the Opus 4.5 announcement, that it outperformed all past candidates within the two-hour limit, corresponds to this condition.
Version 1 was used for just over a year, from early 2024 to May 2025, whereas Version 2 stayed effective for only a few months. But Version 2 had its time limit halved and its starting point and structure changed, so the two cannot be compared under exactly the same conditions. Opus 4 and Opus 4.5 were also released about six months apart, yet both cracked the test as pre-release models, so the gap between public releases cannot be treated as the period Version 2 remained valid.
Still, it is clear that gains in model performance forced the coding test to be rebuilt in short order. Hume took the point where Opus 4 struggled as the starting point for Version 2, but the next-generation Opus 4.5 got past it. Opus 4.5 initially stopped, judging that further improvement was impossible because of memory bandwidth limits, but once Hume told it the cycle count that was achievable, it found a new optimization technique. It then continued debugging and tuning, and improved its score with additional compute time. A test designed around a specific model can lose its ability to evaluate quickly once later models improve.
Version 3 abandons work-like tasks for an unusual puzzle format
After the second version was cracked by Claude, some colleagues proposed banning AI use in the take-home test. Hume, however, thought a ban would be hard to enforce and that there should be a way to evaluate human ability even in an environment where AI can be used as it is in real work. The goal was to assess not only the ability to write programs without AI but also whether candidates can make sound decisions while using it.
Hume first tried another code optimization problem, based on real work: transposing data between TPU registers while avoiding memory bank conflicts. But Opus 4.5 found an efficient solution he had not anticipated, and even after he modified the task so that technique was unavailable, it reached the optimal solution when given more inference compute. Optimization techniques known from real work may also be in a model's training data, so simply raising the difficulty did not solve the problem.
For the third version, Hume therefore adopted programming puzzles with unusual constraints, similar to Zachtronics games. The instruction set is cut down drastically, and candidates must solve each problem with as few instructions as possible. When Hume had Opus 4.5 try the first medium-difficulty puzzle he created, the model could not reach a correct answer. He then added more puzzles and, through verification by colleagues, confirmed that humans can outperform Claude.
Version 3 also provides no visualization features or debugging tools from the start. Candidates must decide whether to build the tools they need themselves or have a coding AI build them, and that decision is itself part of the evaluation. According to Hume, early results showed a good correlation between test performance and the level of candidates' past work. Composing the test from multiple independent puzzles may also make it less dependent on aptitude for any single problem.
These changes came at a cost, though. Hume explains that while the first test resembled real work, the new one imitates unfamiliar work. He also acknowledged that, to make the test harder for AI to solve, he gave up realism and the diversity and depth of optimization that the original task offered. This is one of the challenges coding tests face as they try to adapt to AI progress, though it does not mean every hiring test must lose its relevance to real work.
Coding tests are changing in method more than being abolished
Anthropic's experience does not mean coding tests themselves have stopped working. Version 3, designed by Hume, is at least in early results able to evaluate candidates' abilities. And when the time limit is removed, the best human solutions far exceed what Claude produced using large amounts of compute.
On the public challenge, Opus 4.5 scored 1,487 cycles after 11.5 hours in the dedicated evaluation environment, and improved further to 1,363 cycles in an improved environment. Even so, according to Hume, that still falls short of the best human solutions submitted in the past. The result that Claude matched the top humans was therefore confirmed mainly on selection tasks with limits of two or four hours. For complex optimization work with ample time, human experts retain room to come out ahead.
Rieseberg's reason for dropping coding interviews is also not just the time limit or the speed of AI's answers. He believes that strong performance on conventional coding problems no longer indicates whether someone can achieve major results in real work. While Hume is trying to preserve evaluation ability by changing the difficulty and format of the task, Rieseberg is moving away from conventional tests and placing weight on prototypes people have actually built and on design decisions.
Other major technology companies are also reviewing how AI is treated in hiring. In a June 2025 announcement, Canva said it would require candidates for backend, machine learning, and frontend engineering roles to use AI tools in technical interviews. At Meta, according to internal materials reported by 404 Media in July 2025, a trial was planned that would allow some coding candidates to use AI during interviews. Google CEO Sundar Pichai, meanwhile, said on Lex Fridman's podcast that being able to write good code with AI is a strength, but that he intends to hold at least one in-person interview to confirm fundamental ability.
Nor is there a broad movement among companies to scrap technical tests. Nicole Bachaud, a labor economist at job site ZipRecruiter, described to The Register the results of an October 2026 survey of 1,500 employers. It found that 62.5% of companies in the information sector impose a formal skills test at the early application stage, and 70% have moved to proctored or in-person tests. Many companies are changing how tests are administered, rather than dropping them, to prevent cheating with AI.
Coding tests are also used in engineer hiring in Japan. In a survey by Will of Tech of 522 IT engineers with job-change experience, conducted June 27–30, 2024, 38.1% said they had been asked to take a coding test during hiring, rising to 53.1% among people in their 20s. The survey did not clarify whether AI use was allowed during the tests or whether they were supervised. When AI use is permitted in an unsupervised take-home assignment, it may become hard to tell AI's ability from the candidate's, as Anthropic experienced.
Will Version 3 hold up against later Claude models?
The key question for the third version introduced by Anthropic's performance engineering team is whether it can keep evaluating candidates against later Claude models. After the release of Opus 4.5 in November 2025, Anthropic continued improving its models, announcing Opus 4.6 on February 5, 2026. As far as can be confirmed, however, neither Anthropic's model announcements nor Hume's article list the scores of later models on the Version 3 task. The lack of published results alone does not show that the test has yet to be cracked.
Version 1 was used for just over a year, while Version 2 was matched by Claude to the best candidate within months. If later models reach the top human score within the time limit on Version 3 as well, Anthropic will have to keep updating its tests quickly to match AI progress. Conversely, if Version 3 stays effective for longer than Version 1 did, it becomes more likely that changing the task design can keep pace with AI performance gains.
Hume is offering the original take-home test as an untimed public challenge. He is inviting solutions that beat the 1,487 cycles Claude Opus 4.5 recorded at its release, and encouraging strong performers to apply to Anthropic.
AI progress does not make coding tests immediately unnecessary, but writing code in limited time is no longer enough to identify excellent engineers. Even within Anthropic, different approaches coexist: dropping the test, changing to a task that permits AI, and the conventional selection process without AI. Going forward, how to evaluate problem-solving with AI, design judgment, and results in real work will be a key question in engineer hiring.