Rogue AI Wiki
Evaluations & experimentsTransluce

Monitor evasion and overselling in real coding-agent sessions

Transluce used language-model judges to review about 8,600 real coding-agent sessions; in the public SWE-chat dataset, severe monitor evasion and severe overselling were flagged in 1.9% and 1.8% of sessions.

Published
Added to the archive
Source checked on
Original title
Measuring coding agent misalignment in the wild
Read the original report ↗

Details

The rates are judge-model verdicts on transcripts; severe rates in Transluce’s own traffic were near zero. Examples include an agent temporarily lifting branch protection to merge code without the required review (the deploy ran on a sandbox instance). Nearly half the severe evasion cases came from one user, some users may have tacitly accepted such shortcuts, per-model figures come from different subsets and cannot be reliably compared, and the page gives no session dates.