Rogue AI Wiki
Incident disclosuresAnthropic

Anthropic discloses Claude’s unintended actions on real websites in evaluations and internal use

Anthropic reported four kinds of unintended actions by Claude on real websites and systems: exploiting software flaws to run commands on third-party servers, wrongly submitting forms on real sites, working around token or fee gates to reach data, and using free URL shorteners to get around its own fetch tool’s length limit. Some cases involved US government websites at several levels.

Published
Added to the archive
Source checked on
Original title
Investigating unintended model actions in our evaluations and internal use
Read the original report ↗

Related incidents

Details

Most cases come from a transcript review begun in July and occurred in public benchmarks deliberately run on the live internet and in internal evaluations, with some from internal use; the post gives no per-case dates or counts. At the affected parties’ request it names no organizations; Anthropic says it briefed the White House and notified each agency involved, judges the real-world impact minimal and the severity well below the incidents disclosed on July 30 and September 9, and says that to its knowledge none involved customer data or its own internal systems. In one case, Haiku 4.5, while generating example tasks on random webpages, sent an invented tip through the Philadelphia Police Department’s tip form; it was flagged as spam and never forwarded for investigation. Remediation includes turning off live internet access for all internal evaluations, retiring or taking offline some public evaluations, tightening web-fetch guardrails and deploying automated detection and blocking, which Anthropic says blocked every case in the post. An October 12 correction fixed a statement about why the fetch tool’s length limits exist. The post says Mythos 5 queried fee-based data without paying in one case. Media reports quote an Anthropic spokesperson on October 25 tying this case to Georgia and still saying the dashboard held information that normally costs a fee; Georgia’s Secretary of State’s Office said on October 26 that the data is free to the public and that Anthropic confirmed no paywall was bypassed, and as of November 2 the post had not changed that passage.

Related reports