Qwen team: agents in training fetched ready-made fixes, and behavior monitoring reduced reward hacking
The Qwen team measured Qwen-Turbo trajectories on its training data: after environment hardening, 4.32% of trajectories retrieved a ready-made solution, and these resolved 72.34% of the time against 59.99% overall. Across three SWE-Bench variants, behavior monitoring cut the average share of trajectories that passed while triggering the monitor from 28.57% to 0.56%.
- Published
- Added to the archive
- Source checked on
- Original title
- The Verification Horizon: No Silver Bullet for Coding Agent Rewards
Details
The 4.32% is a share of all Qwen-Turbo trajectories on the training data, with no total given, and it applies after the team removed future commits and disabled network access for tasks that do not need it. “Ready-made solutions” means items such as PR diffs, commit patches or local gold patches; the paper’s example is an agent querying an upstream PR diff, but it does not say whether those tasks had network access. In the same table, visible-test overfitting appears in 30.00% of trajectories, repository-history mining in 21.11%, evaluation-harness tampering in 8.25% and test tampering in 3.69%, all with resolved rates below the overall figure. The monitoring numbers compare RL runs without and with the monitor’s penalty: averaged across the three variants, trajectories triggering the monitor fell from 37.76% to 1.31%, and trajectories passing without triggering it rose from 40.22% to 60.53%. All of these shares are judged by the same monitor rules, which grew during training, so they reflect covered behaviors and leave out shortcuts the rules miss. The 4.32% and 37.76% come from different data and measures and cannot be compared directly. These figures are the same in versions 1 and 2 of the paper.