A research team from Meta AI and the University of Illinois Urbana-Champaign (UIUC) has unveiled "EvoHarness-RL," a training method that teaches AI agents not just how to use external tools but when to decide to use them. AI agents rely on execution infrastructure that records the current state, remembers completed work, and carries past failures forward into future attempts. Until now, however, humans have determined when to consult and when to update that infrastructure, relying on prompts or fixed rules.
According to a preprint published on arXiv on August 5, 2026, the team used Qwen3-8B and recorded a 96.9% average success rate on ALFWorld's in-distribution validation set of 140 "seen" tasks. A standard ReAct setup scored 47.9%, a gap of 49.0 percentage points.
However, this result comes from a single benchmark involving text-based household tasks. What matters as much as the headline number is the shift from hand-written rules to a learnable policy for dividing labor between the agent and its external harness.
Compressing the external harness into four actions
What the paper calls the "harness" is a general term for the mechanisms that support execution outside the model itself. Beyond tools and memory, this includes state tracking and execution logs. The pathway that feeds verification results back to the model is also part of the harness. EvoHarness-RL organizes these functions—which vary by environment—into three domains: Belief, Progress, and Experience (BPE).
Belief maintains "what is currently true in the environment," such as the location and state of objects. Progress records subgoals as completed, incomplete, or stalled. Experience accumulates procedures, common failure patterns, and exploration priorities that can be reused across different attempts.
Rather than unconditionally loading BPE, the policy model chooses among four meta-actions. track queries Belief, commit writes progress status to Progress. recall retrieves relevant knowledge from Experience, and note leaves new insights to be consolidated later. Regular environment actions and harness actions sit side by side as candidate moves, and both consume one step.
This design creates an opportunity cost for accessing external state. Searching memory every time doesn't necessarily make things safer—it delays the actual task at hand. Conversely, skipping too many references causes the agent to lose track of state and repeat the same exploration. What EvoHarness-RL learns is precisely this boundary.
The ALFWorld implementation is surprisingly simple. Belief is updated by a rule-based parser that reads actions and observations, and Experience is retrieved through keyword overlap. Progress holds up to 8 subgoals, while Experience is capped at 80 entries per category across 4 categories. Rarely used entries are evicted using an LFU (least frequently used) policy. While BPE represents a general-purpose division of roles, the internals were not left entirely to a general-purpose model.
SFT teaches the mechanics, GRPO refines the timing
Training proceeds in two stages. In the initial supervised fine-tuning (SFT) phase, the team used Claude Opus as a teacher to run 500 ALFWorld training games, keeping 87 successful trajectories. From these, they generated 1,153 next-action conversation pairs to teach Qwen3-8B the format of both environment actions and the four meta-actions.
The adopted trajectories averaged 26.5 turns per episode, with the teacher invoking the harness 405 times in total—about 18% of all turns. SFT is the stage that carefully establishes the scaffolding for using external state.
In the subsequent GRPO stage, the reward combined a success-based core with bonuses for shorter procedures and action diversity, along with penalties for repetition and format violations. Because efficiency bonuses only apply when a task is actually solved, simply cutting harness calls does not increase the reward on its own. Training ran for 150 epochs, with each episode capped at 70 steps, using 8 NVIDIA H200 GPUs.
Both the policy being updated here and the model interacting with ALFWorld at runtime are Qwen3-8B. Claude Opus, meanwhile, served not only as the SFT teacher but also took on the role of consolidating Experience at the boundary of each epoch. It would therefore be inaccurate to say the system was "completed by the 8B model alone." While the execution policy is 8B, a larger model was involved in preparing the training data and maintaining the experience store.
From 47.9% to 96.9%, and 86.6% on unseen tasks
On the 140 seen tasks, the average success rate was 47.9% for standard Qwen3-8B ReAct, and 56.4% for EvoHarness-Base, which simply added BPE at prompt time. After SFT, this rose to 68.6%, and EvoHarness-RL after GRPO reached 96.9%.
| Qwen3-8B Configuration | ALFWorld Seen (In-Distribution) Average Success Rate |
|---|---|
| ReAct | 47.9% |
| EvoHarness-Base (inference-time BPE) | 56.4% |
| EvoHarness-SFT | 68.6% |
| EvoHarness-RL | 96.9% |
In the comparison presented in the paper, existing trainable methods scored 80.2% for SkillOS and 89.9% for SkillRL. However, the table mixes values the authors ran themselves with values cited from existing papers. SkillRL uses a different Qwen-family base model, so this is not an apples-to-apples comparison with matched compute resources or harness conditions.
The official ALFWorld repository distinguishes valid_seen as in-distribution and valid_unseen as out-of-distribution validation data, containing 140 and 134 examples respectively. On the out-of-distribution unseen set, EvoHarness-RL scored 86.6% versus ReAct's 50.0%. Notably, inference-time BPE alone reached 77.6%, while SFT alone dropped to 69.4%. The authors suggest that SFT may simply imitate the teacher's harness usage patterns learned from seen trajectories, without optimizing for whether access is actually worthwhile out-of-distribution. The improvement after GRPO is consistent with this explanation, though since the entire policy is updated together, this isn't a number that isolates the effect of access decisions alone.
Ablation experiments removing individual BPE components measure the role each domain plays in inference-time harness use with a frozen model. With a frozen Qwen3-8B in an inference-time configuration, full BPE scored 56.4%, but dropped to 50.0% without Belief, 50.7% without Progress, and 48.6% without Experience. At least in this environment, there is value in maintaining three separate types of memory—present, in-progress, and past—rather than a single unified memory function.
From frequent external references to roughly one selective access
During training, the frequency of harness use did not move in the same direction as the success rate. In the early stages of GRPO, BPE was used frequently, but as training progressed, usage sharply declined and stabilized at roughly once per episode. The research team interprets this as the policy internalizing routine usage patterns while consulting external state only when genuinely necessary—a phenomenon they call "harness annealing."
The four actions did not decline uniformly. recall, which retrieves past exploration procedures, remained relatively persistent, while commit and note approached near-zero usage. track fell somewhere in between. The paper explains that in ALFWorld, Experience holds particular value because household procedures and prior knowledge about object locations are easily reusable. The relative importance of Belief in visual environments, or Progress in software development, may increase, but the paper has not yet tested these scenarios.
The Experience store itself is not static either. Early in training, general procedures and task-specific procedures grew, along with accumulated failure examples and exploration priorities. Later, duplicates were consolidated, unused entries were discarded, and frequently referenced entries were retained. The authors call this shift "harness evolution." The policy changes "when to use" the harness, while the external store changes "what it returns." Running both within the same training loop is the core innovation of EvoHarness-RL.
The next test lies beyond text-based household environments
96.9% is an impressive number, but it does not mean 96.9% reliability has been achieved for real-world, long-duration tasks. ALFWorld is an environment where text instructions guide an agent to find, wash, heat, and place objects in designated locations. Software repository dependencies, changing web interfaces, and enterprise systems involving permissions all present different types of state and different costs of failure.
The paper does not report confidence intervals or variance across multiple seeds for its main results, and no link to the original authors' public code could be found. Because Experience consolidation relies on Claude Opus, reproducing this work requires matching not only Qwen3-8B's training but also the behavior of both the teacher model and the consolidation model.
Even so, this work clearly adds a new dimension of judgment to the ongoing race to simply "make external memory bigger." Long-running agents need to maintain records and decide which state to externalize. Within a limited action budget, when to read that state back can also be trained. What remains to be verified is whether this division of roles—BPE—generalizes to other domains. After building state representations suited to each domain, whether success rates can rise while external access decreases will be the true test of generality.