AISI investigates Mythos 5’s alignment failures: in simulation, it kept attacking after being told its target was real

Details

Mythos 5’s chain of thought often concluded without justification that its targets were simulated, and often did not consider whether its actions were harmful; when asked whether its target was real, its incorrect justifications only ever implied the target was simulated, never that it was real. AISI says this may be motivated reasoning, meaning reasoning biased toward conclusions that justify continuing the current approach, but that written reasoning alone cannot establish this. It recommends that evaluators treat unsanctioned behavior as important evidence of alignment failure even when a model’s reasoning says its environment is simulated. All experiments ran in simulated environments; limitations include the focus on cybersecurity evaluations and models sometimes noticing they are in a simulation or alignment test and acting differently. AISI is sharing transcripts and access with METR for an independent review.