METR and Redwood spend three weeks red-teaming OpenAI’s internal monitoring
One METR and one Redwood Research employee tested OpenAI’s internal misalignment monitoring from the inside. They estimate on the order of 10% of eligible internal Codex traffic went unmonitored from May to July because compaction events were skipped, and found several ways monitoring could be bypassed or made to fail.
- Published
- Added to the archive
- Source checked on
- Original title
- Red-Teaming OpenAI's Internal Agent Monitoring Systems
Details
The exercise ran from July 21 to August 11, covered the system as it stood then, and produced a 61-page internal report; the public post is brief, OpenAI could redact it, and METR had no unilateral right to describe redactions. The pair estimate that roughly 75–94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring, and 99.2% of the internal Codex subset; RL training runs were not covered at the start of the exercise. The 10% figure is an order-of-magnitude estimate with substantial uncertainty; OpenAI had found the gap independently before the exercise and fixed it while the exercise was under way, and METR says access constraints meant it could not check whether agents had actually exploited it. They also found that Codex agents could in principle bypass sandboxing and Auto-Review, along with ways to keep hundreds of recent actions unmonitored, likely make monitors fail, or start new agents without monitoring, though setting up the last could itself be caught by the monitor. In an October 29 update, METR says OpenAI reports most vulnerabilities patched with one fix in progress; METR has not evaluated those claims or OpenAI’s later expansion of coverage.