On October 28, Google announced a new AI model, Gemini 4 Carbon. The company positions it as its most capable model to date and says it delivers top-tier performance, particularly in cybersecurity and in long, complex tasks. It also took first place on the composite index published by the independent AI evaluation firm Artificial Analysis, with a score of 59. But the gap to second place, Claude Opus 5.5, is just one point. In the results Google itself published, large margins appear only in a limited set of evaluations, such as GraphWalks, which measures long-context understanding, and ExploitGym, which tests whether vulnerabilities can be exploited. After the introductory pricing period ends, its list price will match Opus 5.5, and no timeline has been given for a general-purpose API release. Under what conditions does Carbon's advantage hold, and where are its limits?
Top overall score of 59, but only 1 point ahead of second place, and first in 4 of 9 evaluations
In the "Intelligence Index v5.0" published by Artificial Analysis as of October 29, Gemini 4 Carbon (reasoning setting: High) ranks first with 59 points. Claude Opus 5.5 (Max) follows with 58, and the same Opus 5.5 at the Xhigh setting scores 57. In the index values Google presented, GPT-6 Astra scores 57 and the previous model, Gemini 4 Argon, scores 50. The top three scores fall within a two-point range, and Carbon's lead over second place is a single point. Carbon is first on the composite index, but it has not pulled far ahead of other companies' top models.
The index combines the results of nine benchmarks. A large win in one evaluation can be offset by a weaker showing in another, which narrows the gap in the overall score, so a one-point difference alone says nothing about each model's strengths and weaknesses. Differences in reasoning settings also matter when making comparisons. High, Max, and Xhigh adjust how much compute is used for reasoning, and the same model can score differently depending on the setting. Indeed, Opus 5.5 is listed separately at 58 (Max) and 57 (Xhigh), while Carbon's 59 is its result at the High setting.
Looking at the nine evaluations that make up the index individually, the leaders as of October 29 and Carbon's results are as follows.
| Evaluation | Leader as of Oct. 29 | Gemini 4 Carbon (High) result |
|---|---|---|
| AA-Omniscience v1.1 | Gemini 4 Carbon (index 54) | First. Second-place Opus 5.5 (Max) scores 48 |
| HLE-Diamond | Gemini 4 Carbon (69.1%) | First. Second-place Opus 5.5 (Max) scores 67.9% |
| AutomationBench-AA | Gemini 4 Carbon (77.8%) | First. Second-place Argon (High) scores 77.5% |
| Terminal-Bench-Science 0.1 | Gemini 4 Carbon (64.8%) | First. Second-place GPT-6 Astra (Max) scores 63.3% |
| Terminal-Bench 4.0 | Kimi K3.1 Pro (67.2%) | Second, at 65.2% |
| FrontierCode 2.0 | Claude Opus 5.5 (Max, 46.4%) | Not first |
| CritPt v2 | GPT-6 Astra (High) | Not first |
| AA-Briefcase v1.1 | Several models rank above Carbon | 1609 Elo. Below Grok 4.7 (Xhigh) at 1638 and Muse Wonder (Max) at 1659 |
| GDP.pdf | A model other than Carbon | Not first |
Gemini 4 Carbon thus took first place in 4 of the 9 evaluations, and in three of those the margin over second place is small: 1.2 points on HLE-Diamond, 0.3 points on AutomationBench-AA, and 1.5 points on Terminal-Bench-Science 0.1. Even the largest gap, on AA-Omniscience, was 6 points on the index.
In the remaining five, other companies' models lead. On Terminal-Bench 4.0, Kimi K3.1 Pro scored 67.2%, putting Carbon second, 2.0 points behind. On FrontierCode 2.0, Claude Opus 5.5 (Max) took first place, though its reasoning setting differs from Carbon's. On CritPt v2, GPT-6 Astra at the same High setting came first, so matching the reasoning setting does not give Carbon an edge.
On AA-Briefcase, Carbon improved by 126 points from the previous model Argon (1483 Elo) to reach 1609 Elo, but still fell short of models such as Grok 4.7, Muse Wonder, and Kimi K3.1 Pro. Elo is a relative rating calculated from head-to-head evaluation results, so a large improvement does not necessarily mean Carbon surpassed other companies' models. Note also that the CritPt v2 result used in the composite index is limited to the "Hard Subset," which differs from the overall score shown on the evaluation page. For this reason, this article does not compare specific CritPt v2 scores.
Other independent evaluations also show the top models running neck and neck. In the "Vals Index v2.2" published by Vals AI, Carbon (High) recorded 70.36%, ranking second among 29 models. First-place Claude Sonnet 5.5 scored 70.85% and third-place Opus 5.5 scored 70.18%, so the gaps to Carbon are just 0.49 and 0.18 points respectively. Meanwhile, on Vals AI's legal research benchmark, "Legal Research Bench," Carbon ranked first among 87 models. Across composite indexes from different evaluators, the differences among top models are extremely small, and rankings could easily shuffle as new models are released.
In Google's comparison table, large gaps on GraphWalks and ExploitGym, but with limited comparison targets
Google DeepMind's model introduction page lists results for 17 benchmarks. The comparison covers four models: Gemini 4 Argon, GPT-6 Astra, Claude Opus 5.5, and Claude Fable 5.1. Carbon recorded the highest score in 15 of the 17. It missed first place in two: on FrontierSWE 2, Carbon scored 61.2% against Astra's 65.5% and Opus 5.5's 62.3%, and on the OSWorld-2.1 Offline subset, it scored 81.5% against Opus 5.5's 87.2%.
The widest gap in Google's comparison appears in GraphWalks, which measures long-context understanding. This benchmark evaluates the ability to follow graph connections scattered across a huge context of 256K to 1M tokens using breadth-first search (BFS). Scoring uses an F1 score that accounts for both missed correct answers and incorrect answers. In Google's published figures, Carbon reached 96.2%, ahead of Argon's 84.2%, Astra's 71.8%, Opus 5.5's 66.8%, and Fable 5.1's 65.0%. The gap to Opus 5.5 is 29.4 points, and even the gap to the previous model Argon is 12.0 points, indicating a substantial improvement in tracking relationships among information spread across a long context.
Google also published three cybersecurity-related benchmarks: "SEC-bench Pro," which measures the ability to discover vulnerabilities; "ExploitGym," which tests whether discovered vulnerabilities are actually exploitable; and "CWE-bench," which measures the ability to write patches that fix vulnerabilities. According to Google, these evaluations were run with the guardrails that prevent harmful output disabled. On SEC-bench Pro, Carbon scored 88.2%, ahead of Argon's 83.4% and Astra's 85.4%. On CWE-bench, Carbon recorded 71.0%, ahead of Argon and Astra at 68.0%, Opus 5.5 at 67.0%, and Fable 5.1 at 58.0%. For vulnerability discovery and fixing, however, the margin over the runner-up is roughly 3 points.
In contrast, Carbon's advantage stands out on ExploitGym. Carbon scored 54.8%, more than 12 points above Argon's 42.0% and Astra's 42.4%, and 20.3 points above Opus 5.5's 34.5%. Of the three cybersecurity evaluation stages, the biggest gap arose in the step that checks whether a vulnerability can actually be exploited. Google adds that in internal evaluations, the share of secure code generated more than doubled compared with Argon, and in an evaluation by the Chrome team, Carbon outperformed another company's large frontier model, whose name was withheld, by more than 10 times at generating high-quality fix patches. However, what metric the latter "10 times" compares has not been disclosed. All of these are evaluations run by Google itself, which should be kept in mind.
Importantly, GraphWalks and the three cybersecurity evaluations are not among the nine items that make up Artificial Analysis's Intelligence Index. In other words, Carbon, which was only one point ahead of second place on the composite index, widens its lead considerably in certain specific areas in Google's published results. These results are useful for understanding Carbon's strengths, but they cannot be treated on par with an overall evaluation by an independent third party.
Caution is also needed about which models Google chose for comparison. On Terminal-Bench 4.0, for example, Carbon's 65.2% is the highest value in Google's table, but in Artificial Analysis's overall ranking, Kimi K3.1 Pro leads with 67.2%. Carbon's 65.2% itself is the same in both; the difference is that Kimi K3.1 Pro is not included in Google's table. Likewise, Grok 4.7 and Muse Wonder, which beat Carbon on AA-Briefcase, are not among the comparison targets. The result of "first in 15 of 17 items" therefore holds only against the four models Google selected.
Improved from Argon in about two weeks, but some metrics got worse
Gemini 4 Argon was announced on September 30 through the "Fairwind Program" for cyber defenders, and on October 13 it became broadly available through the Gemini API (AI Studio) and the Gemini app's AI Pro, Ultra, and other plans. The "two weeks ago" Google mentions in its Carbon announcement refers to this October 13 general release. According to Artificial Analysis, Carbon raised the composite index by 9 points, from 50 to 59, while keeping usage costs at roughly the same level as Argon. However, Intelligence Index v5.0 is a revised version with added evaluation items, so its overall scores cannot be directly compared with those of the older version.
Improvements show up across multiple benchmarks. On Artificial Analysis's AA-Briefcase, the score rose 126 points, from Argon (High)'s 1483 Elo to Carbon (High)'s 1609 Elo. In Google's published comparison table too, the Terminal-Bench 4.0 score improved by 8.1 points, from Argon's 57.1% to Carbon's 65.2%. Beyond the composite index, individual evaluations also confirm results above the previous model.
Not every metric improved, however. On Artificial Analysis's knowledge evaluation "AA-Omniscience," Carbon's accuracy rose to 58%, 10 points above Argon, but its hallucination rate also worsened from 4% to 11%. This hallucination rate refers to the share of questions on which the model gave a wrong answer when it should have abstained, and it differs from the AA-Omniscience index (−100 to 100) listed in the table above. Still, Carbon's 11% is the second-lowest among all models; the lowest is Argon's 4%. Opus 5.5 (Max) and GPT-6 Astra are both at 50%, so Carbon's hallucination rate remains low compared with competing models.
Given how the evaluation works, accuracy and the hallucination rate can easily rise at the same time. For example, if a model starts actively answering questions it previously declined, correct answers increase, but so can wrong ones. However, Artificial Analysis merely reports the changes in both metrics and does not say that such a behavioral change actually occurred in Carbon. The cause requires further verification.
Usage costs have barely changed from the previous model. Artificial Analysis's per-task cost, calculated at list prices, was $7.68 for Argon and $7.76 for Carbon. Google also presents several results on safety improvements: in Gray Swan's indirect prompt injection evaluation, the attack success rate fell from Argon's 0.7% to 0.0% for Carbon. Indirect prompt injection is an attack that embeds malicious instructions in web pages, emails, and the like to steer the behavior of an AI agent that reads them. For Terminal-Bench 4.0, Google also reports that the task success rate through illegitimate means was 0.0% and that the rate of illegitimate attempts themselves was 0.4%.
List price equals Claude Opus 5.5, but per-task costs still differ
Gemini 4 Carbon's introductory price is set at $2 per million input tokens and $10 per million output tokens, the same as Argon. A 95% discount applies to cached input tokens. However, in a footnote to its announcement, Google states that after the introductory period ends, prices will rise to $4 for input and $20 for output. Google has not said how long the introductory price will apply, and Artificial Analysis treats it as a 50% discount with no end date set.
For the previous model Argon, by contrast, the pricing change date is known. The Gemini API pricing page updated on October 27 states that the input price for the flagship reasoning model, apparently Argon, will be $2 per million tokens through November 30 and $4 from December 1. The output price will rise from $10 to $20, and the cached input price from $0.10 to $0.20. However, Carbon is not yet listed on this pricing page, so Argon's pricing change date cannot be assumed to apply to Carbon.
Comparing the pricing of the main models gives the following.
| Model | Input (per 1M tokens) | Output (per 1M tokens) | Introductory price |
|---|---|---|---|
| Gemini 4 Carbon (after intro period) | $4 | $20 | $2 input, $10 output (end date not announced) |
| Gemini 4 Argon (from Dec. 1) | $4 | $20 | $2 input, $10 output (through Nov. 30) |
| Claude Opus 5.5 | $4 | $20 | None |
| GPT-6 Astra | $10 | $50 | None |
At list prices after the introductory period, Gemini 4 Carbon and Claude Opus 5.5 are level at $4 per million input tokens and $20 for output. Compared with GPT-6 Astra's $10 input and $50 output, Carbon's unit price is 40% on both input and output. According to Anthropic's official materials, Opus 5.5 supports a context of up to 1 million tokens and output of up to 128,000 tokens at this price.
However, the same per-token price does not necessarily mean the same cost for actually processing tasks, because API charges are determined by multiplying the unit price by actual token consumption. Artificial Analysis's per-task cost for the Intelligence Index, calculated at list prices, was $7.76 for Carbon, $10.04 for Opus 5.5 (Max), and $6.25 for Astra (Max). Under these conditions, Carbon is about 23% cheaper than Opus 5.5 but about 24% more expensive than Astra. Carbon's output averaged 123,000 tokens per task, about 29% fewer than Opus 5.5 (Max)'s 173,000.
In other words, by per-token price alone Astra is the most expensive and Carbon and Opus 5.5 are level. But by per-task cost, the order from cheapest is Astra, Carbon, Opus 5.5, making Astra the lowest-cost option. This shows that even with a high unit price, total cost can be kept down if the number of tokens needed is small. Note, though, that this comparison used different reasoning settings: High for Carbon and Max for Opus 5.5 and Astra, and changing the setting would change token consumption. Artificial Analysis's cost comparisons are also all based on list prices, not on costs with Carbon's introductory discount applied.
No general-purpose API yet; three conditions that could shape future assessments
According to Google, Gemini 4 Carbon is being provided to some cyber defenders starting the week of the announcement and will roll out in stages to trusted partners through the Fairwind Program. More than 650 partners participate in the program, and only security teams can use it. Access also requires phishing-resistant multi-factor authentication. Google says it intends to expand availability to developers, enterprises, and general users as early as possible, but has not announced specific dates. Carbon has also taken part in a voluntary evaluation that the U.S. government conducts on AI models before their official release.
Artificial Analysis's model page for Carbon likewise states that there is "No publicly available API." The previous model Argon took 13 days to go from its limited release on September 30 to general availability on October 13, but Carbon will not necessarily expand on the same schedule. For developers in Japan too, there is currently no way to integrate Carbon into their own applications or workflows via API; Argon is what they can try right away. Even Argon's introductory pricing ends on November 30, with a price increase to $4 for input and $20 for output scheduled from December 1.
There are three main conditions under which the assessment of Carbon's competitiveness could change.
First, Google could announce the start date of the general-purpose API and the end of the introductory pricing. At present, Carbon is said to cost half as much as Opus 5.5 during the introductory period and the same afterward, but what price general users will actually pay is not settled. Once the terms are known, comparisons can include real adoption costs as well as performance.
Second, third parties could try to reproduce the results Google published for GraphWalks and the cybersecurity evaluations. The 29.4-point gap on GraphWalks and the 20.3-point gap on ExploitGym are key figures supporting Carbon's advantage over other companies' models. If independent evaluations confirm similar gaps, Carbon's strengths in long-context understanding and in verifying exploitability of vulnerabilities would be on firmer ground.
Third, results could change in the five of Artificial Analysis's nine evaluations where Carbon missed first place: FrontierCode 2.0, CritPt v2, Terminal-Bench 4.0, AA-Briefcase, and GDP.pdf. Re-evaluations after general release or new models from other companies could shift the rankings, and with them Carbon's current position at the top of the composite index.
What can be confirmed at this point is that Gemini 4 Carbon took first place on Artificial Analysis's Intelligence Index v5.0 at the High setting, one point ahead of second place. The larger gaps reported on GraphWalks and ExploitGym, meanwhile, rest on comparison results published by Google. It is too early to conclude that Carbon has clearly pulled ahead of competitors in overall performance; independent evaluations after the general-purpose API opens, and verification of actual usage costs, will be important.