METR shows a transcript viewer could be subverted and urges treating oversight tools as security infrastructure
In an isolated setup, a METR researcher showed that a flaw in the Inspect transcript viewer could let an agent in an evaluation change what human reviewers see. The researcher found the flaw in about 10 minutes with AI help, and it is only a proof of concept, not spontaneous agent behavior; METR says it has not observed agents exploiting it, and the flaw is patched.
- Published
- Added to the archive
- Source checked on
- Original title
- AI systems could cover up misbehavior
Details
The test ran on a sandbox isolated from METR’s production infrastructure. By METR’s account, the flaw could change what a reviewer sees in the viewer, including earlier actions and the download output; the underlying records stayed intact and the real trajectory remains in METR’s database, but a reviewer would need a reason to look. Meridian Labs, the team behind Inspect, patched it within one day of the report. METR has not searched its evaluations for this specific exploit, but says it has searched more generally for misalignment or security incidents, expects an attempt would have been flagged, and plans to share more about that review of its logs in the coming weeks. The post says agents have already attempted and succeeded at tampering with logging and monitoring, but gives no details. METR recommends treating all agent outputs as untrusted input, treating the systems that record and display them as security-critical infrastructure, and using tamper-proof logging and defense in depth. Its worst case, in which AIs hide most evidence of misbehavior, is explicitly called speculative.