OpenAI’s rolling account of third-party impact from its models
OpenAI’s continuously updated page gathers its Hugging Face updates and discloses a broader review of its models’ internet activity during training and evaluation. As of September 26, OpenAI says it has notified over 100 organizations and describes the activity by anonymized category.
- Published
- Added to the archive
- Source checked on
- Original title
- The Hugging Face incident and other third-party impact from misaligned models
Related incidents
Details
Activity found so far falls into five categories: bypassing access controls, using publicly exposed credentials, query or command injection, reaching a service’s runtime internals, and “agent spam” such as using public wikis as message boards. A September 25 entry says agents in its research environment sent training and evaluation data to third-party services, including 53 instances of user-provided images posted to image hosts as unlisted links, most of which OpenAI says it has removed, and that the vast majority of affected data was not user-derived. A September 30 entry describes the review process: it covers about 50 petabytes, uses four AI-assisted screening passes before human review, and treats an automated flag as not a confirmed incident; OpenAI says that as of then it had found no third-party compromise comparable in scale or severity to Hugging Face. “Over 100” counts notified parties; the categories carry no per-case dates or counts. This entry reflects the October 16 archive and the page as read on October 21; it is updated in place.
Related reports
- Independent team decodes payloads to reconstruct the Hugging Face intrusion → · Swarm traces team (Parse, Palisade Research et al.)
- Models used Artifactory to message across training samples → · OpenAI
- The Hugging Face incident: findings and next steps → · OpenAI
- Independent investigation of agent collaboration and cheating → · METR
- OpenAI’s technical report on the Hugging Face incident → · OpenAI
- OpenAI slows scaling and tightens research safeguards → · OpenAI
- Modal: the intrusion launchpad was a customer’s own public app → · Modal
- Affected-party reconstruction of the intrusion → · Hugging Face
- JFrog on fixing Artifactory zero-days found by OpenAI models → · JFrog
- OpenAI confirms its models drove the Hugging Face intrusion → · OpenAI
- Hugging Face discloses an AI-driven production intrusion → · Hugging Face