In an October 2026 post on X, AI researcher Andrej Karpathy said the gap in how people perceive AI capabilities is wider than ever.
Back in April, he had described the debate as two groups with different views of AI talking past each other. Now, he says, people fall into four tiers based on their experience with AI, and the more advanced the systems someone uses, the fewer such people there are.
Someone who occasionally uses a free AI chatbot and someone who hands large-scale software development to multiple AI agents imagine very different capabilities when they hear the same word, "AI."
Karpathy also warns that human verification and understanding are starting to fall behind the speed at which AI produces results.
Looking at a publicly documented large-scale development experiment with AI agents, and at the flood of AI-generated math papers, helps explain why this perception gap exists and what it will take to turn AI output into genuinely useful knowledge.
AI experience splits into four tiers, and only about 5,000 people have touched the frontier
In his October post on X, Karpathy sorted the world's population into four tiers by AI experience.
The largest group is roughly 6 billion people who have barely touched a large language model (LLM). At the other end, he estimates that only about 5,000 people have access to the most advanced internal systems.
These figures are Karpathy's own rough estimates, not statistics from a survey of actual usage.
| AI experience | Karpathy's rough estimate | Perception and use of AI |
|---|---|---|
| People who have barely used LLMs | About 6 billion | Little to no direct experience with AI |
| People who occasionally use free AI chat | About 1–2 billion | Use it for everyday tasks such as search assistance and writing |
| People who use frontier models for math or programming | About 20 million | Have delegated complex work to AI agents that once took weeks or months |
| People with access to frontier internal systems | About 5,000 | Have seen long-running collaboration among many AI agents and applications to scientific and mathematical research |
The tiers do not reflect differences in users' knowledge or ability. They reflect which AI systems people use and how complex the work is that they hand over.
Because the figures are not based on rigorous surveys, they also cannot simply be added up to calculate global AI adoption.
According to Karpathy, most people outside math and programming, including executives and other professionals, sit in the second tier.
Executives, for example, spend much of their day in conversations with people. Even if they understand AI progress as knowledge, they may have few chances to experience its capabilities firsthand.
Software developers, by contrast, can hand AI agents tasks like building an app or porting a program, which once took weeks, and compare the results with their own experience.
The two groups judge AI differently not merely because they understand it differently, but because the work they delegate to it is different.
Karpathy adds that inside frontier AI companies, larger-scale work is under way using systems that are not publicly available.
He says thousands of AI agents are collaborating over weeks on large software projects, discovering unpatched vulnerabilities, and conducting scientific and mathematical research.
In his view, the results people outside these companies have seen are only a glimpse of what the most advanced systems can do.
That said, this is Karpathy's own assessment. Neither the figure of about 5,000 people nor the capabilities of the internal systems have been confirmed in independent comparative tests.
Three reasons perceptions of AI diverge
Karpathy cites three factors behind the differing perceptions: the scale of the task, differences in capability across fields, and differences in the systems people can access.
The first is the scale of the work given to AI and how long it works on it.
Generating a paragraph-length answer to a short question does not fully draw out a frontier model's abilities. Differences become clear only when the model is given large, complex tasks and left to work on them for a long time.
For example, generating natural-sounding text is a very different skill from carrying a large software project spanning many files through to completion.
The latter requires continuously handling many steps, including managing progress, finding and fixing errors, running tests, and integrating changes.
Comparing answers to single questions makes it hard to assess the ability to carry out such long-running work.
The second factor is that AI capability varies widely by field.
Karpathy explains that AI capabilities are improving especially fast in areas that are fully digital, where correctness can be verified, and where economic value is high.
Programming and mathematics are the prime examples.
In programming, running generated code or passing tests gives a relatively clear judgment of whether the result is correct. In mathematics, correctness can sometimes be confirmed by checking answers or verifying formal proofs.
These traits suit "reinforcement learning from verifiable rewards."
This is a training method in which a model's outputs are evaluated and correct results are rewarded, improving its abilities. In fields where right and wrong can be judged objectively, it is easier to design the evaluation criteria used for training.
Karpathy also believes that AI companies concentrating training compute on commercially valuable fields contributes to the uneven distribution of capabilities.
However, rewarding correct answers while training a model is a different process from running tests and iterating on fixes during actual development work.
The former is a mechanism for improving the model itself; the latter is a way of using a trained model to get work done. In both, being able to confirm that results are correct is essential.
The third factor is differences in access to AI systems.
Free models, frontier models available on paid plans, and research systems used inside AI companies differ in the capabilities, computing resources, and execution environments available.
Even with the same model, results change depending on the work assigned and the mechanism used to run the agents.
In other words, AI capability cannot be compared simply as free versus paid.
The gap Karpathy points to arises from differences in purpose, execution environment, and verification methods, on top of differences in the models' own performance.
16 AI agents build a C compiler, and humans play a key role
A notable example of large-scale development with AI agents is an experiment published on February 5, 2026, by Anthropic researcher Nicholas Carlini.
On Anthropic's engineering blog, Carlini reported the results of running 16 Claude agents in parallel to build a C compiler in Rust.
A compiler is software that converts programs written by humans into machine code and other forms a computer can execute.
The experiment used Claude Opus 4.6 and ran about 2,000 Claude Code sessions over roughly two weeks.
The result was a C compiler of about 100,000 lines of code that could build Linux 6.9 for three architectures: x86, ARM, and RISC-V.
API usage costs came to about $20,000.
On its face, this looks like AI developing sophisticated software with almost no human help.
What matters most in Carlini's report, however, is how humans designed the mechanisms that made long-running development by AI agents possible.
According to him, much of the effort during development went not into writing the compiler's code directly but into building an environment in which the AI could keep working autonomously.
Specifically, that meant designing tests, building the execution environment, recording progress, detecting errors, and feeding results back to the AI.
AI agents can work autonomously on a given task. But if they cannot correctly judge whether their work succeeded, they may keep heading in the wrong direction.
That made it necessary to prepare suitable test environments and define clearly what counted as success and failure.
The problem showed up most clearly when the agents took on building the Linux kernel.
At first, when the agents were fixing different bugs across multiple independent tests, parallel work by the 16 agents was effective.
But once they began tackling the single large task of building the Linux kernel, the situation changed.
Multiple agents ran into the same bug and tried the same fix, overwriting one another's changes.
Adding more agents can create confusion rather than efficiency if the work cannot be divided appropriately.
Carlini therefore used GCC, an existing compiler, as a reference for confirming correct behavior.
First, most of the Linux kernel is compiled with GCC, and only some files are handled by the Claude-built compiler.
If a problem occurs, the set of files handled by the Claude compiler is narrowed further to pinpoint the cause of the bug.
This let each agent work on problems in different files, making parallel work possible.
In other words, it was not only the AI model's capability that made large-scale development work. It was also important that humans organized the work into divisible pieces and designed a mechanism to verify the results.
Of course, the finished compiler still has limitations.
GCC was used to generate the 16-bit code needed to boot Linux on x86, and GCC-family tools were used for the assembler and linker in the demonstration.
The efficiency of the generated code also fell short of GCC with optimizations disabled, he said.
Carlini himself acknowledges that it is not at the stage of being a drop-in replacement for a practical compiler, and that the quality of the generated Rust code does not match that of skilled developers.
Still, the fact that 16 AI agents produced a compiler of about 100,000 lines in around two weeks is a result that shows the potential of long-running autonomous development.
Note, however, that the roughly $20,000 figure is API usage cost, not the total development cost including human environment design and verification. It cannot be concluded from this result that the cost of ordinary software development could be cut in the same way.
Nor does this February 2026 experiment directly prove the capabilities of the frontier internal systems Karpathy mentioned in October.
AI publishes 722 math manuscripts: can verification and correction keep up?
The problem arising between AI-generated output and human verification is also surfacing in mathematical research.
On October 6, 2026, OpenAI published mathematical research results generated by its most advanced internal AI model on GitHub.
The release comprised 722 math manuscripts, containing results and proofs related to open problems across a wide range of mathematical fields.
For many of the proofs, the mathematical claims and arguments were also written in a formal language called Lean so that they can be checked by computer.
Lean is a system for writing mathematical proofs according to strict rules so that their logical correctness can be checked mechanically.
But the publication of 722 manuscripts does not mean all of them are confirmed as correct new discoveries.
In fact, an update history dated October 7, the day after publication, reports that some manuscripts were withdrawn or corrected.
According to the update history of OpenAI's math research repository, the main changes are as follows.
| Date | What was announced | Points to note |
|---|---|---|
| October 6 | Published 722 math manuscripts, with Lean formalization of many proofs | Correctness and novelty have not been confirmed for all published results |
| Withdrawal record dated October 7 | Withdrew 3 manuscripts | A sign error found in one manuscript also affected the arguments in two dependent manuscripts |
| Correction record, same date | Corrected another 14 manuscripts | Repaired proofs, corrected claims and assumptions, and fixed references |
| Additional update, same date | Updated references and version dates in 13 more manuscripts | These update references to corrected related manuscripts and do not indicate 13 independent proof errors |
These are the dates recorded in the update history, not necessarily the dates on which each problem was first discovered.
In the three withdrawn manuscripts, a sign error in one of them invalidated a key part of its proof.
Because two other manuscripts relied on that argument, they were affected as well, bringing the total withdrawn to three.
It is worth noting that finding an error in a proof is different from finding that the mathematical statement being proved is itself false.
The withdrawals mean that the arguments as presented do not correctly establish the claims.
The same update history also says that 300 of the 719 main results, about 42%, have been formalized in Lean.
This is the share of results covered by machine verification; it does not mean the remaining 58% or so are wrong.
Meanwhile, more formalized proofs do not eliminate the need for verification and understanding by mathematicians.
Machine checking can confirm whether a proof written in a formal language follows a given logical system.
But questions such as how significant a result is mathematically, how it relates to existing research, and what new research it might lead to require human understanding and judgment.
It is also important to check that the formalized claims and assumptions match what the original research intended.
This parallels the need to distinguish between the range covered by passing tests and practical quality in the AI-built compiler.
Even if AI can produce large volumes of results, it is hard to trust and use them unless it is clear what has been verified and what has not yet been confirmed.
OpenAI has said it will improve how it presents mathematical explanations, citations, and results, and will support workshops and research meetings to help researchers deepen their understanding of the results.
The company explained that the average compute used to generate one mathematical result is equivalent to about three hours of reasoning in ChatGPT Pro.
However, this is a rough measure of the compute AI used to generate a result, not the time humans need to understand and verify its content.
The speed at which AI generates results and the speed at which humans can verify them and make them usable are separate problems.
Measuring AI capability means asking what was achieved and how far it was verified
Karpathy's four tiers, together with the large-scale development and math research examples that have actually been published, point to something important about evaluating AI capability.
Rather than simply comparing what AI has become able to do, we need to check under what conditions, and to what extent, results were achieved.
Experience writing text with a free AI chatbot is not enough to understand what long-running software development by multiple agents can do.
Conversely, even when a large-scale development succeeds, that does not mean the result is of practical quality.
For a compiler, besides the fact that it could build Linux, it is necessary to check the efficiency of the generated code, dependence on external tools, and unsupported features.
For math research, what matters is not only the number of published manuscripts but also how much of each proof is formalized, the history of corrections and withdrawals, and the status of verification by other researchers.
In short, evaluating AI results requires clarifying the criteria used to judge success and the verification methods, along with the type of model and the scale of the work.
There is no need to take Karpathy's estimate of about 5,000 people as a rigorous statistic.
What matters is his point that perceptions of AI capability differ greatly between people who directly use the most advanced AI systems and everyone else.
Narrowing this gap requires publishing the results produced by frontier AI in a form that outsiders can verify.
That means disclosing not only successful outcomes but also the conditions under which they were achieved, what could not be done, and how far correctness has been confirmed.
Other developers need to be able to verify, fix, and use code generated by AI agents. Researchers need to be able to understand proofs presented by AI and build on them in later work.
Only through such processes do AI-generated results move beyond being the experience of a few researchers and developers and become knowledge and technology that can be widely shared.
As AI progress accelerates, what will matter more and more is not just the ability to produce results. The mechanisms that let people understand, verify, and confidently use those results must develop just as much.