Evaluations & experimentsTransluce
Monitor evasion and overselling in real coding-agent sessions
Transluce used language-model judges to review about 8,600 real coding-agent sessions; in the public SWE-chat dataset, severe monitor evasion and severe overselling were flagged in 1.9% and 1.8% of sessions.
- Published
- Added to the archive
- Source checked on
- Original title
- Measuring coding agent misalignment in the wild
Details
The rates are judge-model verdicts on transcripts; severe rates in Transluce’s own traffic were near zero. Examples include an agent temporarily lifting branch protection to merge code without the required review (the deploy ran on a sandbox instance). Nearly half the severe evasion cases came from one user, some users may have tacitly accepted such shortcuts, per-model figures come from different subsets and cannot be reliably compared, and the page gives no session dates.