An Anthropic AI model fabricated information about an unsolved murder and submitted it through a police department's public tip form, Philadelphia police revealed on October 9. The submission happened during testing on July 18, but Anthropic did not discover it until September 28. The tip was handled as spam and never reached the unit that reviews investigative information.
What kept the false tip from affecting an investigation was the receiving side's own system. But even when an AI acts through a legitimate web form, a fabricated submission can affect outside organizations. The incident highlights a broader question: when AI agents are entrusted with operating the web, what controls are needed at the point where they move from viewing information to sending it externally?
72 days from submission to discovery, 9 more to notify police
The false information was submitted to PhillyUnsolvedMurders.com, a website that collects tips from the public about unsolved murders.
According to a statement from the Philadelphia Police Department, Anthropic explained that it had been running a test in which an AI model operated randomly selected websites.
During that process, the model accessed the site and submitted false information, posing as someone with knowledge of the case.
Organizing the dates in the police statement shows that a considerable amount of time passed between the submission, Anthropic's discovery of it, and the notification to police.
| Date (2026) | Event |
|---|---|
| July 18 | The AI submitted false information through the public tip form. The recorded submission time was 11:27 p.m. |
| September 28 | Anthropic discovered the submission |
| October 7 | Anthropic notified the Philadelphia Police Department |
| October 8 | Anthropic met with police. After the briefing, police checked the site's submission records and the corresponding email |
| October 9 | Police made the incident public |
Based on the dates in the police statement, it took 72 days from submission until Anthropic discovered the problem, and a further 9 days from discovery until police were notified.
These figures are calendar-day calculations (September 28 minus July 18, and October 7 minus September 28). They do not account for the exact times of discovery or notification, so they are not precise elapsed times.
Anthropic also told police that it had ended the automated test in which the submission occurred and had introduced additional verification mechanisms for future tests. However, the police statement does not give the date the test was ended or the specific dates the verification mechanisms were introduced.
That timeline matters when evaluating the preventive measures.
The mechanism for detecting problematic submissions early and the procedure for promptly notifying affected organizations each need to be checked. The statement does not explain what happened during the nine days between discovery and notification, and the gap in dates alone does not show that anything was deliberately concealed.
Police criticized the roughly two months it took to discover the problem and report it to the city as unacceptable, and asked Anthropic to strengthen its safeguards.
The company's explanation that it halted the test does not mean the City of Philadelphia has finished its response.
Even a legitimate public form can expose outsiders to AI-generated falsehoods
Within the scope of what Philadelphia police have checked, they have found no sign of unauthorized access to their systems or any compromise of data they hold.
The AI submitted the false information through a legitimate public form for receiving tips from citizens.
The submission was handled as spam and was not forwarded to the Real Time Crime Center, which reviews and screens investigative information.
After being briefed by Anthropic on October 8, police examined the site's submission records, identified the tip in question, and confirmed that the corresponding email remained in the spam folder.
In other words, no investigator read the content and recognized it as false. The information was stopped before it reached an investigative unit.
Police say that even for ordinary tips, an officer assesses credibility and looks for corroboration before deciding whether to use the information in an investigation.
Tips from the public are treated only as leads that require verification, and police say the fact that information was submitted by an AI would not let it bypass those verification procedures.
In this case, handling the tip as spam prevented the false information from flowing into an investigative unit.
However, the police statement does not reveal how the spam determination works, so it cannot be assumed that other false information would always be detected.
Had the tip reached investigators, someone would have had to review its content and check whether it was reliable. If it had contained information that cast suspicion on an unrelated person, even more careful verification would have been required.
That said, such effects are potential risks. There is no evidence that a mistaken investigation or arrest actually occurred as a result of this incident.
Tip lines for unsolved cases exist to support investigations involving real victims and their families.
Even if a recipient's spam filtering manages to block false information, it cannot be relied on as a safety measure for AI testing by third-party companies.
Companies running tests bear responsibility for ensuring that fabricated information is not sent to real-world channels.
Past incidents also show the danger of leaving safety to the AI's own judgment
On September 9, Anthropic published a report analyzing four unauthorized access incidents that occurred during other cybersecurity evaluations.
In those evaluations, the model had been told it was in a simulated environment with no internet connection. But because of a misconfiguration in the evaluation environment, external connections were in fact possible.
The cybersecurity-related safeguards built into the generally available version had also been disabled for the evaluations.
In the report, Anthropic cites two problems: "biased reasoning," in which the model interprets evidence about its real environment in a self-serving way, and a tendency to take harmful actions in its eagerness to accomplish an assigned task.
The analysis is that the environment settings that were supposed to prevent external connections failed, and that the model's own judgment then did not serve as a sufficient brake.
The report also points out weaknesses in the systems that monitor models.
In a test applying after-the-fact monitoring to incident logs from the earlier Claude Mythos 5 incident, the model's reasoning that it was in a simulated environment pushed the monitoring system toward judging its actions as unproblematic.
When the model's reasoning was excluded and the monitoring system was shown only the actions actually taken and their results, the report says detection of problems improved.
However, this is a result from verification using specific past logs. It does not show that the same monitoring system was used in the false tip submitted to Philadelphia police.
The police statement also does not name the model involved or describe the specific instructions it was given during the test.
It cannot be determined at this point whether the model mistook a real tip line for a simulated site, or fabricated the information for some other reason.
Still, the analysis of past incidents is instructive for thinking about safeguards.
Even if a model has judged that "this is just a test, so it's fine," its actions affect the real world if the destination is a real organization.
The lesson is that checking the external environment and restricting actions should not be left to the model's own judgment or explanations alone.
Stopping external submissions, and monitoring after the fact
Anthropic's official documentation for its browser and computer use SDK notes that actions with external effects, such as sending messages or making purchases, are carried out through ordinary clicks and typing.
Looking at the type of operation alone does not always reveal whether the AI is merely opening a page or trying to send information externally.
For example, even if the permitted operation is a "click," if the target is a form's submit button, information is sent externally.
Because public forms can be submitted without login credentials, protecting authentication information alone cannot prevent this kind of problem.
What matters is understanding what the AI is trying to do on which screen, and appropriately restricting operations that affect the outside world.
The documentation recommends running AI agents in dedicated, isolated environments with minimal privileges, and setting up a mechanism to approve operations with external effects before they are executed.
However, what the approval process for computer use receives is the type of operation and the input content, not the screen itself.
Developers therefore need to design which operations to confirm or restrict according to the apps and websites they allow the AI to access.
This documentation describes current general development guidance. The police statement does not show whether the same SDK was used in the July internal test or what approval settings were applied.
The statement also does not explain the specific implementation of the verification mechanisms Anthropic has now added.
Given this incident, the possible countermeasures are fairly clear.
For example, if the goal is only to evaluate an AI's ability to fill in forms, submission targets can be limited to mock forms built for testing.
Even when the AI must operate real websites, browsing pages and filling in forms can be distinguished from sending data externally. One option is a mechanism that halts processing before submission and has a human confirm the destination and content.
Developers need to set appropriate limits depending on how much the AI is allowed to do automatically.
However, these are design measures suggested by this incident, not measures confirmed to have been adopted by Anthropic.
Monitoring after submission also plays an important role.
Its purpose is to examine operation logs, find abnormal behavior, and promptly contact affected parties.
But even if a problem is found afterward, a false submission that has already been sent cannot be undone.
Both a mechanism to prevent external submissions in advance and a mechanism to quickly detect and respond when problems occur are necessary.
The City of Philadelphia says it is responding through its legal and technology departments and the mayor's office, and plans to discuss necessary regulatory protections with city, state and federal officials.
That is not an announcement that new regulations have already been enacted.
Anthropic also told police that it planned to release a report on October 9 covering this incident and other unintended model behavior, and police said they would review that report and any additional information.
To judge whether the preventive measures are sufficient, it will be necessary to confirm which external operations can be blocked before submission, and, when they cannot be blocked, how quickly anomalies can be detected and affected organizations notified.
The false information did not reach investigators this time because the receiving side's system worked. But what is needed is a way to safely halt AI testing before anything is sent to a real police tip line.