Many developers who have entrusted AI agents with long-running tasks have likely noticed that precautions given early on gradually seem to get ignored. Among Japanese-speaking engineers using Claude Code, this phenomenon has become prominent enough in practice that multiple blog posts have compiled workarounds for it. But a research team at Pennsylvania State University has now put numbers to a problem that had previously only been described anecdotally. After passing through "context compaction"—the process of summarizing long conversations or tasks—an average of 83% of the constraints imposed by users disappear. This includes specifications of permitted actions and output formats, and even safety-related instructions such as "don't delete this until I confirm" are no exception.
Experimental results show "don't delete until confirmed" disappears 83% of the time on average
Zhiqi Wang, Yichi Zhang, Dongwon Lee, and Yuchen Yang of Pennsylvania State University published a paper titled "Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction" (arXiv:2608.11242, submitted July 31, 2026). the-decoder.com covered this paper on August 18, 2026. The paper's proposed concept of "Session Constraints" (SC) refers to user instructions that should be maintained until a conversation or task is complete. A typical example would be a sentence like "don't delete the email until I confirm."
The research team built an evaluation suite called "COMPINT" to measure AI's ability to retain constraints across three scenarios: WildChat, which reproduces multi-turn dialogue; Hermes Agent, which reproduces an agent's action history; and OpenResearcher, which reproduces long-term research tasks. Even without any compaction process, SC compliance rates ranged only from 59.2% to 70.7% depending on the scenario, showing that thorough adherence to user instructions has its limits even before compaction enters the picture. Once context compaction was introduced, SC survival rates dropped to an average of just 17%. In other words, 83% of the constraints users imposed vanish with every compaction.
The research team simultaneously proposed a plug-in countermeasure called an "SC-aware extractor" based on Qwen3.5-9B. When incorporated into the compactor, retention rates recovered to 95.6% for Hermes Agent, 95.1% for OpenResearcher, and 90.3% for WildChat. The existing compaction systems evaluated in this study lacked the very mechanism needed to support this over-90% retention.
Summarization engines only check whether "the task can continue"
Context compaction is the process of summarizing conversation history that is about to overflow, compressing it down to a volume the model can use to continue the task. What the summarization engine optimizes for is "can the task continue in the next turn"—and past statements that haven't been referenced recently are treated as unnecessary information, making them prime targets for compaction. A constraint the user stated only once at the beginning, such as "don't delete until I confirm," appears to the summarization engine, unless explicitly repeated in subsequent turns, as an obsolete past instruction no longer in use. The default summarization engine does not distinguish between information necessary for continuing the task and constraints that must be maintained at all times.
A phenomenon close to this failure pattern has also been reported among Japanese-speaking Claude Code users. An article posted on Zenn notes that rules written in a CLAUDE.md file directly under the project root (those without path specifications) are automatically reloaded and do not disappear even after compaction, but detailed instructions conveyed within the conversation history get paraphrased during summarization, causing an "invisible degradation" in which compliance rates drop to around 60–70%, as reported by figures in a GitHub Issue. Rules tied to a CLAUDE.md under a subdirectory or to specific paths are not subject to automatic reloading until the target file is read again. This figure and the 17% average survival rate shown in the paper discussed here cannot be directly compared, since the objects and methods of measurement differ, but they point to the same underlying problem: compaction quietly undermines compliance with user instructions.
The design of extracting sentences corresponding to Session Constraints from the dialogue or task history in advance—before compaction begins—and maintaining them separately from the summary, is the reason the SC-aware extractor achieved over 90% retention. The paper also shows that simply providing a dedicated field for "constraints and preferences" in the compaction prompt is insufficient. A compactor with such a field (pi-mono) showed retention rates that varied wildly depending on the model and dataset, ranging from nearly 0% to around 36%, and even a prompt explicitly instructing constraint retention (Anthropic SC-targeted) topped out at 37.6% on WildChat. The paper concludes that no compactor exceeds an average retention rate of 36% for any SC category. There is a large performance gap between simply writing down that a constraint exists and structurally extracting and retaining it separately from the summary.
Beyond "5 out of 6 disappearing" lies the risk that safety constraints get broken
Suppose a long-running coding agent is given six session constraints (e.g., prohibition of irreversible operations, confirmation before external transmission, restrictions on writing to specific files). The paper aggregates SCs into five categories, and reports that even for Preference-type constraints, which tend to be relatively well retained, no compactor exceeds an average retention rate of 36%. If we apply the overall average of 17% across the board, a rough estimate suggests that after a single compaction, only about 1 out of 6 constraints actually remains enforced. The remaining 5 may vanish from the AI's memory without the user ever having withdrawn them.
In long-running agent tasks, compaction rarely happens only once. If we assume compaction occurs twice in succession, with each round having an independent survival probability of 17%, then the survival rate after two rounds of compaction works out to roughly 2.9% (0.17 × 0.17). This multiplication is merely a rough estimate assuming independent events, and the paper itself did not directly verify multiple rounds of compaction. Even so, this simple calculation illustrates a structure in which the risk of losing constraints tends to compound the longer an operation runs.
This is not the first research to point out a vulnerability of this kind. A paper titled "Governance Decay" (arXiv:2606.22528), published in June 2026, reports that across seven model families, violation rates for safety constraints after compaction rose from 0% to 30%, and to as high as 59% for some models. The measurement metrics differ—Lost in Compaction's "constraint survival rate" versus Governance Decay's "constraint violation rate"—so simple numerical comparison is not possible.
The Governance Decay paper was submitted on June 21, 2026, and the Lost in Compaction paper on July 31 of the same year, with different authors and teams. In an interval of only about 40 days, two independent research groups pointed out the same structural weakness—constraint management after compaction—from different angles. This is not an isolated bug. It's reasonable to view it as an issue rooted in the very design philosophy of context compaction.
The tendency for AI performance to become unstable as context grows longer has also been observed even without compaction. The "Context Rot" study published by the AI company Chroma in 2025 compared, across a total of 18 models including Claude, GPT, Gemini, and Qwen, full context averaging about 113,000 tokens against "focused" input narrowed down to only relevant portions, about 300 tokens. The results showed clearly higher performance with focused input across every model, confirming a tendency for performance to decline as input grows longer. However, Chroma itself notes that this decline is not uniform and "often manifests in surprising ways." Regardless of whether compaction is involved, handling long context has its limits, suggesting that the current approach of compressing information through summarization may only be a stopgap treatment for that underlying limitation.
Anthropic's official documentation does not guarantee "automatic instruction retention"
Anthropic offers a context compaction feature for its API under the beta header compact-2026-01-12, covering models such as Claude Opus 5 and Claude Sonnet 5. The official documentation explains that the purpose of the summary generated by compaction is to retain "information useful for continuing the task," and there is no statement guaranteeing that instructions or constraints explicitly stated by the user will be automatically preserved. On the other hand, Anthropic also provides a pause_after_compaction parameter that pauses processing after compaction to explicitly re-preserve the most recent messages, as well as an instructions parameter that rewrites the compaction behavior itself. Instruction retention is positioned not as a "feature guaranteed by default" but as "an option that can be achieved if developers explicitly implement it."
The compactors actually evaluated in the Lost in Compaction paper were centered on open models such as gpt-oss-120b, Gemma, Qwen3, and GPT-5.4-mini, and Claude itself was not directly tested as a compaction system. However, the paper does actually adopt and evaluate the default compaction prompt shown in Anthropic's official documentation as one of its experimental conditions, combined with gpt-oss-120b. In other words, Anthropic's "prompt design philosophy" is included among the objects of evaluation, but the behavior of Claude itself as a model/API was not measured. Therefore, one cannot assert that "Claude too retains only 17% of constraints." What can be confirmed is that while Anthropic's official documentation contains no language guaranteeing automatic instruction retention, the means to achieve retention are indeed made available to developers.
The defense available to developers: evacuate instructions outside the reach of compaction
When imposing safety-related constraints during a long-running agent session, one should not assume instructions will survive across compaction events. The substance of the countermeasures proposed by Japanese-language blogs is not so much a stopgap of resending instructions on the spot at every compaction, but rather a design that evacuates instructions to a layer unaffected by compaction—writing rules into a persistent file like CLAUDE.md that gets reloaded even after compaction, specifying the compaction behavior itself via Compact Instructions, or inserting deterministic processing via Hooks. Anthropic's official API likewise provides a pause_after_compaction parameter that pauses after compaction to explicitly re-preserve the most recent messages, and an instructions parameter that rewrites the compaction prompt itself.
This problem is not technically unsolvable. The SC-aware extractor demonstrated by the research team proved that retention rates can recover to over 90% simply by extracting and isolating constraints in a dedicated manner before compaction. This means that if companies providing compaction systems simply build a category equivalent to Session Constraints into their design, a similar improvement can be reproduced. The challenge is less a technical wall than an operational choice left to each company: whether to build a design that handles user instructions in a separate track from summarization into the default behavior.
Whether this paper will prompt Anthropic and other companies to make concrete specification changes to their compaction features remains to be seen by tracking future updates to official documentation. Even at this point, however, means already exist for evacuating instructions outside the reach of compaction—such as Anthropic's parameters or the persistent-file designs proposed by Japanese-language blogs. Whether to make that the standard behavior, or leave it up to case-by-case handling by individual developers, is the choice that remains for each company.