On November 2, 2026, SemiAnalysis released GDPx, a new benchmark for evaluating the practical capabilities of AI agents. According to the official site, version 1.0 consists of 206 tasks across 11 domains, developed with the cooperation of 14 companies. In the overall ranking, Claude Opus 5.5 took first place with a score of 12.0, followed by Gemini 4 Carbon at 10.9 and GPT-6 Astra at 9.6.
GDPx evaluates the ability to find the necessary materials, operate tools, and complete the requested deliverables. However, the top score of 12.0 cannot be read as "AI can automate 12% of general work." A closer look at how the evaluation targets were chosen and how scoring works shows what this ranking can tell us, and what must be verified separately before real-world adoption.
206 tasks that evaluate everything from finding materials to completing deliverables
The 11 domains covered by GDPx were not sampled to represent jobs across the whole economy. SemiAnalysis selected areas where it judged that AI would have a greater impact over the next one to two years. They include chip design, finance, law, cybersecurity, biology, materials science, quantitative trading, hardware design, and software engineering. The benchmark also evaluates "recursive self-improvement," in which AI improves the development process of next-generation AI models, and "alignment," which measures whether an AI can achieve its goals while adhering to behavioral constraints.
SemiAnalysis cites OpenAI's GDPval benchmark as a point of comparison. It argues that a format giving a short instruction and attached materials cannot adequately evaluate the information-gathering burden involved in real work. GDPx therefore builds the process of locating the necessary information into the evaluation. This, however, is a design choice made by SemiAnalysis; it has not been demonstrated that the evaluation methods of other benchmarks are inferior.
In the finance domain, for example, evaluation environments were built from actual transactions, with a total of about 4,000 anonymized files prepared. Each task requires finding the necessary information in materials stored in the folders of the parties to a deal. The materials science domain uses a simulated laboratory, and hardware-related tasks also use tools such as Siemens NX and AWS FPGA. The benchmark tests not just the ability to answer questions but the ability to carry out multiple tasks in environments close to real work.
The standard time limit for each task is eight hours, with some tasks allowing less. This is the upper limit for a single attempt, not the average time required. Models with a dedicated agent execution environment use tools such as Claude Code, Codex, and Antigravity, while models without one use OpenCode. The results therefore reflect not only the performance of the AI models themselves but also differences in the execution environments that handle tool operation and task progression. It would not be appropriate to treat the overall ranking as the result of comparing all models in an identical environment.
What does the top overall "12%" mean?
To understand GDPx's overall score, it is necessary to check how scoring and aggregation work. SemiAnalysis uses both pass/fail grading, which determines whether a task succeeded, and partial credit, which evaluates how far the requirements were met. It also runs three trials for each task and reports pass@1 based on those results. pass@1 measures the rate of success on a single attempt, and differs from pass@3, where success in any one of three attempts counts.
In aggregating the overall score, all 11 domains are weighted equally. Where multiple partner companies participate in the same domain, each company is also weighted equally. In other words, GDPx's overall score does not reflect the economic size, number of workers, or actual working hours of each domain. It should be noted that it is not an indicator that directly represents the economic impact of AI on a particular occupation or industry.
The table below organizes what can and cannot be judged from each measurement condition, based on the official GDPx v1.0 methodology and the published overall scores. It is neither a newly calculated performance metric nor the result of independent third-party replication.
| Measurement condition | What can be judged | What cannot be judged |
|---|---|---|
| 11 deliberately selected domains | Evaluation results on difficult tasks in the target domains | Automation rates across the economy or all job functions |
| Equal weighting of domains and partner companies | An overall evaluation based on the published aggregation rules | Economic impact reflecting market size or working hours |
| Combined pass/fail and partial credit | Task completion status and the degree to which requirements were partially met | A simple accuracy rate from grading all 206 tasks by a single standard |
| pass@1 based on three trials | The rate of success on a single attempt | Performance if success in any one of three attempts counted |
| Effects of execution environment and safety classifiers | Results that include the environment used and safety constraints | Standalone model performance, or rankings with safety constraints lifted |
The published explanations do not allow us to fully confirm the details of how partial credit by domain is calculated and reflected in the overall score. For that reason, the overall score cannot be converted into a simple accuracy rate, such as how many of the 206 tasks were answered correctly. Likewise, it would be wrong to infer from the top result of 12.0 that "AI fails in 88% of general work." What GDPx measures is only the capability of AI agents on the selected tasks and under the stated evaluation conditions.
Finance and law evaluations show a gap between partial credit and practical usability
In the finance domain, applying strict pass/fail grading resulted in a 0% success rate for every model. SemiAnalysis therefore also includes average partial credit in its domain-by-domain comparison charts. However, a 0% success rate does not mean the AI failed to handle any part of the tasks correctly. It is necessary to distinguish between whether the final deliverable met the passing standard and how far requirements were achieved at intermediate stages.
One published task, called "Bellwood-QoE," involves modifying an existing spreadsheet file for quality-of-earnings analysis. Among the 25 selected trials, none fully accomplished the task. However, the best-performing trial met 8 of 11 checks. Note that the figure of 25 refers to the number of trials, not to 25 of the 206 tasks having been evaluated.
In the law domain as well, partial credit is presented because strict pass/fail criteria make the tasks too difficult. In a task to revise contract documents accompanying a Series B funding round, none of the 20 total trials by four models produced a revised draft usable in practice. Failures included cases in which the model misjudged whether a client's email or the signed term sheet should take priority, and cases in which consistency across multiple documents was not maintained.
These results show that correctly revising part of a document and producing a completed deliverable that can be used in practice as is are separate problems. A high partial credit score does not necessarily mean human review and correction can be skipped. This matters especially in work like legal documents, where problems arise if the document as a whole loses consistency even when individual statements are accurate. At the same time, the limited sample of published trials does not allow us to estimate the failure rate of AI across legal work as a whole.
Adoption decisions require checking domain-level performance and safety constraints
According to SemiAnalysis, the overall scores for Claude Opus 5.5 and GPT-6 Astra include the effects of constraints from safety classifiers. The impact is said to be particularly large in three domains: cybersecurity, biology, and alignment, and the company has also published aggregated results that exclude them. It is necessary to distinguish between cases in which a request was refused for safety reasons and cases in which a task was not achieved because of insufficient capability. However, not all low scores are necessarily due to safety refusals, and it is not possible to infer what the rankings would be if the constraints were lifted.
The best-performing model also differs by domain. In SemiAnalysis's evaluation, Claude Opus 5.5 led in quantitative trading, hardware, and chip design. Gemini 4 Carbon, meanwhile, led in biology and software engineering, and GPT-6 Astra in recursive self-improvement and materials science. This shows that the overall ranking alone cannot tell us which model suits a particular task. These are, however, evaluation results under conditions set by SemiAnalysis, not conclusions confirmed by independent replication.
Unite.AI has also covered the ranking at launch, along with costs and average partial credit by domain, but it did not present its own performance measurements. Companies considering actual adoption need to examine in detail the evaluation results for domains close to their own work, along with the scoring conditions. In addition, because GDPx includes non-public tasks and data, it remains unclear whether third parties can reproduce all 206 tasks under the same conditions.
To make adoption decisions, it is effective to run small-scale trials using the materials a company actually handles and the deliverables it actually needs, after clarifying the success criteria and the acceptable range of failure. Safety policies and operating costs, as well as steps that require human review, should be included in the evaluation. SemiAnalysis plans to release new versions of GDPx every three to six months. Going forward, an important basis for judging practicality will be not only growth in overall scores but also how far AI agents can complete deliverables on tasks close to real-world work.