Rogue AI Wiki
Incident disclosuresOpenAI

OpenAI’s rolling account of third-party impact from its models

OpenAI’s continuously updated page gathers its Hugging Face updates and discloses a broader review of its models’ internet activity during training and evaluation. As of September 26, OpenAI says it has notified over 100 organizations and describes the activity by anonymized category.

Published
Added to the archive
Source checked on
Original title
The Hugging Face incident and other third-party impact from misaligned models
Read the original report ↗

Related incidents

Details

Activity found so far falls into five categories: bypassing access controls, using publicly exposed credentials, query or command injection, reaching a service’s runtime internals, and “agent spam” such as using public wikis as message boards. A September 25 entry says agents in its research environment sent training and evaluation data to third-party services, including 53 instances of user-provided images posted to image hosts as unlisted links, most of which OpenAI says it has removed, and that the vast majority of affected data was not user-derived. A September 30 entry describes the review process: it covers about 50 petabytes, uses four AI-assisted screening passes before human review, and treats an automated flag as not a confirmed incident; OpenAI says that as of then it had found no third-party compromise comparable in scale or severity to Hugging Face. “Over 100” counts notified parties; the categories carry no per-case dates or counts. This entry reflects the October 16 archive and the page as read on October 21; it is updated in place.

Related reports