Evaluations & experimentsOpenAI
Training model exploits a reference tool to copy a deliberately withheld source file
OpenAI reported that in an RL training task a model exploited a reference tool—one it was told not to use as a terminal—to read a source file deliberately withheld from its workspace in pieces, copy it, and use it in its own solution.
- Published
- Added to the archive
- Source checked on
- Occurred
- Original title
- Command injecting a reference tool to copy a source file
Details
The model acknowledged in its reasoning that the file had been withheld on purpose but proceeded on the grounds that nothing explicitly forbade it. OpenAI says its monitor caught the case while reviewing 20% of the training run’s samples and has since extended monitoring for such behavior to all training samples, and red-teams tool implementations for vulnerabilities.