On August 27, 2026, a research team from Google Research and Virginia Tech published "WikiSkill" on arXiv, a system that accumulates AI agents' successes and failures to update execution procedures. The name suggests a mechanism where an agent consults a shared wiki while working. However, in the experiments' default configuration, the execution agent that solves tasks never reads the wiki. Rather than directly expanding a model's memory, the wiki sits within a development pipeline that turns execution logs into reusable procedures.
That boundary is the core of WikiSkill. Model parameters are never updated; even failed skill modifications are recorded, and only procedures that improve validation scores are passed to the next execution round. The research team reports that WikiSkill outperformed existing skill-evolution methods across five benchmarks and five models, but the paper is an arXiv v1 preprint whose peer-review status cannot be confirmed. It's important to separate what the numbers suggest from the conditions that haven't yet been tested for real-world deployment.
The Wiki Is an Editorial Layer for Skills, Not a Memory Store
WikiSkill's workspace is divided into three layers: Raw, Wiki, and Skill. The Raw layer stores execution traces—reasoning, tool calls, results, and final answers—unmodified. The Wiki layer organizes recurring failures and successful strategies from those traces into Markdown pattern pages, accumulating change history and their impact on skills. The Skill layer holds the procedure documents actually used for the next task.
The inspiration comes from the "LLM Wiki" concept that Andrej Karpathy published in April 2026. The original idea was to continuously reflect a document's contradictions and connections into a Markdown wiki, rather than re-searching and reassembling documents for every question. WikiSkill moves this idea into agent development, adding a layer that converts knowledge into execution procedures and a mechanism to judge whether changes should be adopted.
However, simply opening the wiki to the execution agent doesn't necessarily help. In an ablation study across four benchmarks using Gemini-3.5-Flash, disabling the execution agent's wiki access while giving the skill proposer persistent wiki access raised the average score from 48.7% to 63.7%—a 15.0-point gain. But when the execution agent was also given wiki access, the score dropped 2.8 points to 60.9%.
The research team suggests that when the execution agent pulls solutions directly from the wiki, it becomes harder to obtain traces that reveal what the current skills are missing. This is the authors' hypothesis, not a directly measured causal result. Still, the separation of access rights is clear: the wiki supports diagnosing and fixing procedures more than it supports directly answering tasks.
A Four-Stage Loop That Preserves Failed Updates
The evolution loop proceeds in four stages. First, the Inference Agent executes training tasks using the current skills. Next, the Wiki Maintainer reads the successful and failed traces and reflects the causes and effective strategies into the wiki. The Skill Proposer then selects and reads necessary records from the wiki's index, past change effects, and this round's success/failure list, proposing exactly one skill creation or modification per cycle. Finally, the candidate skill is tested on a separate validation set, and it's adopted only if it exceeds the previous best score.
If not adopted, the Skill layer reverts to its previous version. But the wiki is never rolled back. The proposal's changes, validation score, and adoption decision are appended to skill-impact.md, so future proposers don't repeat the same failure. The design makes execution procedures reversible while accumulating the learning history that led to them.
Nor does the Wiki Maintainer ingest logs without limit. Each iteration selects up to 8 items—at most 5 failures and 3 successes. Each log entry is capped at 15,000 characters. The proposer doesn't receive a large, fixed batch of logs all at once; instead, it uses tools to select the necessary pattern pages or raw traces as needed. The wiki functions not as a warehouse for dumping long histories wholesale into a prompt, but as an intermediate representation where diagnostic results can be searched.
This mechanism differs from "continual learning," in which a model autonomously learns anything on its own. Here, there are training, validation, and test splits with known correct answers, and an evaluation function that can judge improvement. What WikiSkill automates is the process of editing experience into procedures under these conditions, and reverting changes that make things worse.
A 9B Model Plus Skills Outperformed a 27B Model Without Skills
The evaluation targets are LiveMath for math reasoning, SealQA for web search, SpreadsheetBench for spreadsheet tasks, OfficeQA for long-document QA, and ALFWorld for interactive environments. Five models were tested: Qwen at 4B, 9B, and 27B parameters, plus Gemma-4-31B-It and Gemini-3.5-Flash. The paper compares WikiSkill against three methods—Trace2Skill, EvoSkill, and SkillOpt—as well as a no-skill baseline, reporting averages from three independent runs of the entire pipeline.
Within the Qwen series, the average score improvement from no-skill to WikiSkill was 12.3 points for the 4B model, 17.5 points for the 9B model, and 23.9 points for the 27B model. Larger models gained more from discovering and using procedures. However, model size alone doesn't determine the outcome. When Qwen-3.5-9B was given skills it had evolved itself, it achieved an average of 47.4%, surpassing the 39.4% recorded by Qwen-3.6-27B without any skills.
Furthermore, on ALFWorld, Qwen-3.5-9B scored 63.4% using skills it created itself, but jumped to 70.2% when using skills transplanted from Qwen-3.6-27B. While skills from other models don't always perform better, this demonstrates that the ability to discover procedures and the ability to execute them are distinct capabilities. This could point toward a configuration where a high-performance model serves as the skill developer, distributing procedures to cheaper models.
However, an average "win" shouldn't be confused with winning every single cell. WikiSkill scored 28.5% on OfficeQA with Qwen-3.5-4B, below the no-skill baseline's 30.2%. The paper's statistical test uses a 1,000-iteration bootstrap method with resampling of test items, checking for significance at the 5% level. Depending on the task-model combination, evolved procedures don't always reliably enable stable long-document retrieval.
The Stronger the Numbers, the More Important the Fine Print
The first constraint is that skill retrieval and invocation were never tested. To avoid mixing selection errors into the evaluation, the researchers directly inserted the full text of all valid skills into the system prompt at execution time. Whether an appropriate skill can be found among many and fit within limited context remains a separate, unresolved problem.
The adoption judgment also has variability. The validation sets across the five benchmarks range from 10 to 40 items, and the paper itself acknowledges noise from such small validation sets. While three independent runs and bootstrap testing on test results reduce variance in final values, they don't guarantee that every adoption decision within each iteration is correct. Additionally, since changes that don't strictly exceed the previous best score are rejected, changes that might help improvements later on could be discarded even if their immediate score is the same.
Extended operational periods raise other issues. The wiki's pattern pages, change logs, and history of rejected changes keep growing, with no automatic organization or pruning mechanism. Operations spanning hundreds of steps or tasks lasting hours haven't been evaluated either; the system isn't designed to update procedures mid-execution during a single long run.
Computational cost also requires careful reading. The paper estimates the number of optimization LLM calls per iteration—combining one Wiki Maintainer call and the Skill Proposer's ReAct processing—at "1 plus roughly 10 to 20" when training data is processed in batch. This count doesn't scale with the size of the training set, but the cost of executing each task to generate traces is separate. This doesn't mean end-to-end computational cost or pricing remains constant.
What WikiSkill proposes isn't simply adding vague "long-term memory" to AI agents, but rather managing experience by separating it into evidence, knowledge, and execution procedures. Judging its practical value will require testing whether the system can correctly select skills without direct injection, whether it can maintain improvement while keeping the wiki organized, and whether the failure reduction achieved justifies the computational cost of the entire pipeline.