Inside an ultra-cold cryostat, microwave pulses fly back and forth to control superconducting circuits. Even the slightest drift in their amplitude or frequency—caused by minute thermal fluctuations—can cause the quantum states held within to collapse catastrophically. For future useful quantum algorithms that must run continuously for days on end, how to correct this "drift" while the system is actively operating stands as a fundamental hurdle that goes far beyond mere engineering concerns.
The Contradiction Between Computation and Calibration in Ever-Larger Quantum Hardware
Quantum computers are inherently analog machines, and compared to digital classical computers, they possess an extremely fragile nature. Quantum error correction (QEC) protocols are the means by which this fragility is overcome. QEC provides a mechanism that converts analog state-change errors into binary detection signals—"error" or "no error"—and pulls the logical quantum state back to its correct original form.
For QEC to achieve a practical logical error rate (LER), a fundamental precondition must be met: the physical gate error rate must be kept well below a threshold (roughly in the range of $10^{-3}$ to $10^{-2}$). The process of tuning a system into this precise state is called "calibration." Decades of accumulated research have established physics-based calibration methods that sequentially adjust individual control parameters using directed acyclic graphs. As major players like IBM push forward with roadmaps to scale hardware from hundreds to thousands of physical qubits, this calibration technology has supported the industry's rapid evolution.
However, as an inevitable consequence of analog control, even a system that has been perfectly calibrated will degrade in performance over time as it is exposed to environmental fluctuations and temperature changes in the instrumentation. The conventional solution used in past experiments has been to halt the entire QEC process and recalibrate the system. This approach—completely separating computation from calibration—works for today's small-scale experiments. But as qubit counts scale up from tens of thousands to millions, and the number of parameters requiring control explodes, this method clearly reaches its limits. When considering algorithms that will eventually need to run continuously for days or even months, computation cannot be paused midway. Unless a method is established for continuing calibration without stopping computation, fault-tolerant quantum computing will remain unattainable.
Turning the Error Detection Signal into a "Teacher" for Reinforcement Learning
The QEC algorithm itself held the key to solving this dilemma. Errors caused by imperfect calibration manifest on the QEC circuit as measurable syndromes, exactly the same way as errors caused by noise.
The Google Quantum AI research team turned this error detection process into a direct solution. Rather than simply consuming detected errors as "data for correction," they built a framework that repurposes this data as a learning signal to train an artificial intelligence agent.
This system is built on reinforcement learning (RL). The memory of a classical controller stores the control parameters used to convert the QEC circuit into physical waveforms. During computation execution, the RL agent intentionally applies extremely small perturbations (fluctuations) to all control parameters simultaneously.
These tiny changes cause subtle shifts in the probability of error detection events occurring. The purpose of the learning algorithm is to untangle the causal relationship—which parameters, moved in which direction, reduce the error detection rate. This mechanism continuously chases the optimal parameter distribution as it adapts to an ever-changing environment.
Discovering Locality to Untangle Tens of Thousands of Control Parameters
Here a mathematical wall arises. What we ultimately want to optimize is the logical error rate ( ), but in multi-qubit systems such as surface codes, setting directly as the objective function is not practical.
The performance of a surface code is described by the scaling model , where $d$ is the code distance and represents the suppression factor of the physical error rate. As distance $d$ increases, shrinks exponentially. Attempting to accurately observe this vanishingly small error rate would require an astronomical number of QEC cycles, far too many to allow for real-time optimization.
Imagine a vast concert hall where tens of thousands of performers (quantum gates) are playing, and you're trying to pinpoint who is out of tune using just a single microphone (the system-wide error rate)—an almost impossible task. If instead you place small microphones every few performers (local detectors), you can instantly identify discordant notes and correct just that section. The factor graph method adopted by the research team corresponds precisely to building this kind of network of local microphones.
The research team defined the average occurrence rate $C$ of error detection events within a specific spacetime region of the QEC circuit as a new surrogate objective function. A proportional relationship holds between the gradient of and the gradient of $C$ : . This equation means that moving parameters in the direction that minimizes the local error detection rate $C$ automatically minimizes the overall logical error rate as well.
By linking a specific detector's signal only to the control parameters of nearby gates, the dependencies between parameters become extremely sparse. Simulation results confirmed that even when simultaneously optimizing approximately 40,000 control parameters in a distance-15 surface code, the convergence speed of learning does not depend on system size.
| Comparison Item | Conventional Physics-Based Calibration | This Study's Reinforcement Learning Framework |
|---|---|---|
| Execution Timing | Performed with computation completely halted | Runs in real time, in parallel with quantum error correction |
| Optimization Target | Sequential adjustment of a small number of parameters (directed acyclic graph) | Global optimization of thousands to tens of thousands of parameters (leveraging locality) |
| Response to Environmental Drift | Performance degrades over time since settings are fixed in advance | Autonomously tracks and corrects parameters in response to drift during operation |
| Hardware Requirements | Requires dedicated calibration circuits and additional measurement resources | No additional resources needed, since existing error detection signals are repurposed |
The Willow Processor Surpasses the Limits of Skilled Engineers
In real-hardware experiments using Google's next-generation superconducting processor "Willow," this RL framework delivered results that exceeded expectations.
The experiments were conducted using distance-5 and distance-7 surface codes, as well as a distance-5 color code. First, the processor was calibrated to its absolute limit using conventional physics-based approaches combined with manual tuning by skilled engineers. Starting from that state, additional fine-tuning via reinforcement learning further suppressed the logical error rate by approximately 20%.
The precision achieved reached record levels—the best ever recorded for any physical qubit modality. Using the neural network decoder "AlphaQubit2" on the distance-7 surface code, the team recorded an average logical error rate per cycle of $7.72(9) \times 10^{-4}$ . For the distance-5 color code, they achieved a figure of $8.19(14) \times 10^{-3}$ . These results can be understood as the model-free reinforcement learning approach, which views the system holistically, cleaning up the accumulation of small errors arising from simplified physical models and unknown device physics.
The reinforcement learning agent also possesses the ability to track and adapt to environmental fluctuations. In experiments where deliberate drift (temporal variation) was injected into parameters such as microwave pulse amplitude and frequency, conventional systems with fixed control policies saw their error rates progressively worsen. In contrast, the RL agent autonomously learned to track the fluctuations and continued correcting parameters accordingly. It reduced the average logical error rate by 24% and suppressed variance in the distribution, boosting stability by a factor of 2.4. When combined with coordinated adjustment of decoder-side parameters, the stability improvement reached as much as 3.5-fold.
Adaptability That Transcends Modality Barriers, and Remaining Physical Hurdles
The significance of this technology lies in the fact that it does not depend on any specific hardware such as superconducting circuits. All that is required are two things: an "error detection signal" and "adjustable control parameters." This gives it the versatility to be directly applied to other physical modalities, such as neutral atom architectures that couple spatially separated qubits, or ion trap systems. Regardless of hardware type, it could serve as a universal software foundation for controlling large-scale fault-tolerant quantum computers.
The new paradigm in which quantum computers learn from their own errors and continue calibrating without ever stopping computation shows an extremely promising path toward operating large-scale fault-tolerant systems. That said, there remain hurdles to clear before reaching fully autonomous operation.
The biggest dilemma is the trade-off between the "parameter exploration (intentional fluctuation)" needed for learning and "maintaining an optimal state." In this experiment, since the quantum state was reinitialized each time a short logical algorithm was repeatedly executed, the temporary increase in errors caused by exploration was not a problem. In an environment where long computations must run continuously without stopping, the act of perturbing parameters itself would adversely affect the computation. Simulations confirmed that as long as the drift speed is sufficiently slow, the benefits of learning-based optimization outweigh the drawbacks of exploration noise.
For abrupt fluctuations that exceed the response speed of the learning algorithm—such as correlated drift caused by high-energy particles (like cosmic rays) striking the superconducting circuit—the system would be destroyed before learning can catch up. Against such sudden disturbances, rather than relying solely on a software-based approach through reinforcement learning, it remains necessary to continue employing physical shielding and mitigation measures at the hardware level. The era of human engineers constantly standing by to adjust instruments will eventually come to an end. Toward realizing an autonomous quantum computer that keeps running while self-healing from operational noise, the optimization of system control has entered a new domain.