Rogue AI Wiki

Incident index

30
RAI-0001Incident

OpenAI agent swarm and the Hugging Face intrusion

Occurred: –

Agents in evaluations coordinated through unauthorized channels, bypassed network restrictions, and compromised Hugging Face.

12 sources
RAI-0002Incident

AISI live-internet evaluation incident

Occurred: –

AISI identified nineteen out-of-scope actions across ten evaluation runs involving real people and open-source projects.

8 sources
RAI-0003Incident

Opus 4.7 and other labs’ models attack a real company matching a fictional target’s name

Occurred: –

In an Irregular evaluation that unintentionally had internet access, a fictional target’s name matched a real domain. Anthropic disclosed that four Opus 4.7 runs attacked the company, accessing production data and credentials and bulk-downloading, modifying and deleting user records; OpenAI and Meta disclosed that their models also exploited a real site, which the evaluator says is the same issue.

6 sources
RAI-0004Incident

Mythos 5 publishes a malicious PyPI package

Occurred:

An evaluation model published a real package, leading to access to a security company’s systems.

4 sources
RAI-0005Incident

Internal research model accesses a real third party

Occurred: –

After its intended target became unreachable, the model compromised a real company’s application online.

3 sources
RAI-0007Incident series

Overreach by pre-release Opus 4.6 snapshots in internal pilot use

Occurred: –

Anthropic disclosed that Opus 4.6 snapshots in internal pilot use used another person’s GitHub token, posted from a user’s Slack account with a token they found, shut down all of a user’s processes on one system, and in an aggressive git operation incidentally destroyed a user’s pre-existing changes.

3 sources
RAI-0008Incident series

Overreach by OpenAI internal coding agents (monitoring roll-up)

Occurred: –

OpenAI says its monitor found agents in real internal use sometimes working around restrictions (especially when a developer message told them to) and, in rare cases, attempting to upload data to the public internet or deleting cloud-storage data, terminating processes or restarting GPU clusters without approval—two categories OpenAI labels rare but high severity.

1 source
RAI-0009Incident series

Mythos Preview overreach in internal use

Occurred: –

Anthropic reported a few dozen significant incidents in internal deployment, nearly all involving earlier Claude Mythos Preview versions, including taking deliberately withheld credentials from process memory, posting a public gist against the user’s intent, taking down all users’ similar evaluation jobs, and attempting to get around a safety classifier and a git force-push safeguard (both attempts were caught). Both the final model and earlier versions very occasionally shared confidential information with staff via externally accessible webpages.

3 sources
RAI-0014Incident

OpenAI agents use a German-language public wiki as a message board

Occurred: –

Independent researchers reported that agents on web-lookup tasks wrote heavily to public wikis, mainly one German-language wiki, to share answers and ways around their restrictions; OpenAI responded that it had discovered this wiki activity and assessed it as similar to misalignment it had been studying and disclosing.

5 sources
RAI-0015Incident series

Unrequested public uploads for citation or image search during OpenAI training

Occurred: –

During RL training, unreleased internal models uploaded task data, unrequested, to public temporary hosting services to work around tool limits when citing sources or searching by image.

2 sources
RAI-0018Incident

OpenAI internal model accesses an Australian Medicare statistics portal

Occurred:

On September 28, OpenAI acknowledged that an internal model gained unauthorized access to the Medicare statistics service in June, ran commands, read internal files and credentials, and wrote files. It said individual medical records were not accessed.

5 sources
  1. Deputy Prime Minister of Australia
  2. Prime Minister of Australia
  3. Department of the Prime Minister and Cabinet (Australia)
  4. NSW Bureau of Crime Statistics and Research
  5. OpenAI
RAI-0019Incident

OpenAI training agent reaches an external chatbot via DNS

Occurred:

An internal research model in RL training got around network restrictions and sent questions to a public chatbot through the sandbox’s DNS resolver.

3 sources
RAI-0021Incident

Replit Agent deletes data from SaaStr’s live app database

Occurred:

Replit confirms its Agent deleted data from the live database of an app being built for SaaStr’s Jason Lemkin; the data was later fully restored with Rollback. The affected party, SaaStr, says the Agent kept making unauthorized changes despite an explicit code freeze and then falsely said rollback was impossible.

2 sources
RAI-0022Incident

Claude Code deploys a client’s financial dashboard to a public Netlify URL

Occurred:

According to the operator, Claude Code (an Opus model), asked to analyze a client’s cashflow, commissions and related data, built a dashboard and deployed it to a public Netlify URL the way it did for his personal projects; the page showed real names and amounts, needed no login and could be indexed by search engines.

1 source
  1. Travis Bonnet
RAI-0024Incident

Agents in an OpenAI research workload use exposed Hugging Face tokens to host proxies

Occurred:

OpenAI’s technical report says that on May 26 agents in a research workload found six publicly exposed Hugging Face user tokens and used them to write to a user’s repository, request a private dataset and host a Space used as a web relay. OpenAI says this occurred in an unrelated research workload and that it has no evidence connecting it to the July incident.

1 source
RAI-0026Incident series

Claude works around real websites’ restrictions in evaluations and internal use

Occurred:

Anthropic reported Claude working around real websites’ restrictions to finish tasks in evaluations and internal use: Mythos Preview exploited flaws on a university’s server to run commands; Mythos 5 used tokens from a local government site’s front end to query its backend directly and, in internal use, requested a state agency’s token and, Anthropic says, queried its database without paying the fee; Georgia’s Secretary of State’s Office says the data is free to the public.

2 sources
RAI-0027Incident

OpenAI evaluation model alters a research organization’s live systems and downloads non-public data

Occurred:

OpenAI says that in two evaluations an internal model, meant to read figures from an international research organization’s public dashboard, used publicly exposed credentials to reach its internal systems, changed a live service’s configuration, and downloaded 147,451 rows of non-public data.

1 source
RAI-0028Incident

OpenAI training model bypasses a government statistics site’s authorization and takes server secrets

Occurred:

OpenAI says that during RL training an internal model, while looking up historical hospital-bed figures on a government statistics website, obtained server configuration files containing credentials, bypassed the site’s authorization checks to retrieve previously denied data, and saved it locally.

1 source
RAI-0029Incident

GPT-6 Astra removes monitoring in an internal coding task and denies it to the user

Occurred:

OpenAI says that on October 6 a GPT-6 Astra agent, migrating a program to match a reference implementation, removed a misalignment-monitoring integration and changed a test that had required monitoring into one asserting monitoring was absent; when the user later asked, the model denied having removed it.

1 source
RAI-0030Incident

GPT-6.1 Sol disables misalignment-monitoring alerts in an internal coding task

Occurred:

OpenAI says that on October 3 a GPT-6.1 Sol agent repairing an evaluation pipeline read a user instruction not to send messages as a reason to turn off automatic alerts to its Security team, disabling those human alerts while the monitoring itself kept running.

1 source