On November 16, 2026, OpenAI published evaluation results on the performance, safety, and cost of its healthcare AI agents, which operate electronic health record (EHR) systems and insurer systems. According to the company, 99% of clinical tasks were judged safe, and the agent earned 94.3% of the available points on a healthcare administration benchmark.
However, the two figures measure different things. Meanwhile, a kit released by CVS Health takes a different approach: it converts workflows discovered by AI into code and runs them repeatedly. Bringing such technology into real healthcare operations requires not only accurate operation, but also a way to confirm that a claim or process was actually completed.
A Computer-Use AI That Connects EHRs to Insurance Requests
A computer-use agent reads the screens of applications designed for humans and, through clicks and text entry, handles work that spans multiple systems. In an MRI demo using fictitious patient data, OpenAI showed a continuous flow from clinical tasks through preparing an insurance request.
The agent first opens the EHR and reviews the visit notes, physical therapy records, and prior imaging results. Based on that information, it drafts an MRI order, a clinical note, and a reply to the patient, then routes them to a healthcare professional for review.
The same patient information is also used to request prior authorization on an insurer's portal site. Prior authorization is the process of submitting required information to an insurer and seeking approval before a test or treatment is carried out. In the demo, the agent prepared the request and brought it to a state where a staff member could review it. It handles insurance procedures in a separate system rather than stopping at document creation inside the EHR, while human review is built into both the clinical judgment and the request processing.
What Do 99% Clinical Safety and 94.3% Administrative Performance Mean?
In OpenAI's clinical evaluation, the target was the tasks GPT-6 Astra handles in an EHR inbox. The evaluation used tasks built with physician input and assessed both clinical reasoning based on patient information accumulated over time and complex screen operation.
In the safety evaluation, reviewers examined the AI's action history as well as the clinical notes, orders, and patient messages it generated. They then made judgments based on errors, missing information, and the possibility that an action could harm a patient. Usefulness was assessed separately, with physicians scoring on a five-point scale. HealthAdminBench, the healthcare administration benchmark, differs from these in that it measures performance against predefined scoring criteria.
The main results published by OpenAI are as follows.
| Reported figure | Subject / model | Denominator / unit of evaluation | What the metric means |
|---|---|---|---|
| 99% | GPT-6 Astra on EHR inbox work | 325 tasks reviewed | Share judged clinically safe after review of actions and outputs |
| 93% | Usefulness of the same model | 646 evaluations | Share in which physicians chose the top two points on the five-point scale |
| 94.3% | GPT-6 Astra on HealthAdminBench | Points earned out of available scoring items | Share of points earned against the scoring criteria |
| 91.8% | GPT-6 Luna on HealthAdminBench | Points earned out of available scoring items | Same as above |
The 99% clinical safety figure is the share of safety judgments across 325 tasks, while the 93% usefulness figure is based on 646 evaluations. By contrast, 94.3% and 91.8% are score rates against healthcare administration scoring items, so the denominators and evaluation targets differ. HealthAdminBench includes tasks such as prior authorization, appeals, and medical device orders.
In addition, a benchmark score rate must be distinguished from a task completion rate. According to a HealthAdminBench preprint published on April 10 by Suhana Bedi and colleagues, the 135 tasks evaluated consist of a total of 1,698 subtasks, each judged pass or fail. The paper states that a task is judged complete only if all of its subtasks pass. Therefore, even if most steps earn points, the task is not complete if even one required step remains.
The evaluation used fictitious patient data and four simulated web environments: an EHR, two insurer portals, and a fax-sending portal. CAPTCHAs, multi-factor authentication, and session expirations were not included. When judging real-world adoption, what matters is not only the scoring results but also whether it can be confirmed that a request or other process has reached the required final state.
CVS Health Codifies AI-Discovered Workflows for Reuse
As a way to build AI screen operation into repetitive work, CVS Health has released the "CUA Agent Kit." The kit's README sets out a design philosophy of using AI to discover a workflow and then running it repeatedly with code.
The process has three stages. In the first, OpenAI's computer-use agent carries out a task given in natural language, saving screenshots, actions, targets, and execution logs. In the second, based on the records of successful runs, Codex skills generate code that a team can inspect and maintain. In the third, the generated code is used to run the established workflow repeatedly.
CVS Health explains that if an AI model is made to work out the steps every time the same task is run, inference costs arise even for routine work an ordinary program could handle, and results vary from run to run. By converting a workflow into code once it has been discovered and running it as ordinary software, the approach reduces dependence on the model for repetitive work.
For browser operations it uses Playwright, identifying targets based on the meaning of elements rather than reproducing on-screen coordinates as they are. For Windows applications, it verifies targets using the control information that can be obtained and implements the processing with pywinauto. With this approach, running the codified routine does not require calling an AI model, and no API key is needed.
However, if an application's screens or procedures change, the code must be revised. The CUA Agent Kit is designed so that production execution logs can be analyzed with Codex to investigate the cause of failures and update the code, but that process includes review, testing, and verification by engineers. The flow is to repeat workflow discovery, confirm stable success, regenerate the code, and then validate routine execution.
As an example of real-world use at CVS Health, OpenAI says one production workflow cleared a backlog of document deliveries equivalent to several months of work over a single weekend. It also says another system handles the system operations tied to tens of thousands of specialist consultations each day. These operations include verification steps and human oversight, and use screenshots and execution logs to verify application state.
On cost, OpenAI says GPT-6 Luna scored 91.8% on HealthAdminBench at a cost of 8.5 cents per task. That is 98% lower than Astra, and both models were evaluated at maximum reasoning effort. However, lowering the per-task cost of using a model and eliminating model calls altogether by codifying routine processing are different cost-reduction methods.
Approvals, Audits, and Ongoing Operations for Clinical Adoption
OpenAI offers its healthcare screen-operation capability through ChatGPT for Healthcare, ChatGPT for Clinicians, and the OpenAI API. ChatGPT for Healthcare includes role-based access controls and audit logs, allowing organizations to set approval procedures, available applications, and the scope of access to records and actions. When handling protected health information covered by HIPAA, the U.S. health information privacy law, organizations must use eligible services covered by a BAA (business associate agreement) with OpenAI and apply the designated security and data-retention settings.
Meanwhile, the announcement does not state the share of tasks that Astra and Luna completed end to end, nor the total labor involved in real-world operation, including human review and maintenance. Nor has any quantitative result been published on how much cost the codification in CVS Health's CUA Agent Kit actually saved.
When healthcare organizations adopt such technology, they need a mechanism to confirm whether an insurance request is merely at the preparation stage or has been sent and submitted, in addition to looking at benchmark scores. Clarifying which steps will be codified and run repeatedly, and what humans will verify when screens or procedures change, will also be important for building AI screen operation into ongoing healthcare work.
