Axel Ahlqvist, Richard Guan, Juan-Pablo Rivera, Adeline Kassler, Dmitrii Troitskii, Alexandra Souly, Kai Fronsdal, Robert Kirk 等 9 位作者
原文摘要
A core obstacle to alignment evaluation is evaluation awareness: capable models can tell when they are being tested rather than deployed, weakening the conclusions a safety evaluation can support. We present two techniques that make simulated alignment evaluations harder to distinguish from real deployments. Our first technique, critique refinement, spends additional inference-time compute on each simulator action: the simulator generates multiple candidate actions, refines them using feedback from an instance of the target model on how to make them more realistic, and continues the evaluation with the most deployment-like candidate. Our second technique, DISH (Deployment-Imitating SWE-Agent Harness), wraps the target in an agent harness, reducing the gap between simulated and real deployment environments in coding settings. We test the techniques on multiple target models and find that applying both yields larger realism gains than either alone. We show that automated approaches can improve the realism of alignment evaluations, and that they compare favorably to spending the extra compute on more audit attempts.
Olli Järviniemi, Alexandra Souly, Abby D'Cruz, Jordan Taylor, Xander Davies, Robert Kirk
原文摘要
This technical report describes an alignment investigation into Mythos 5's behaviour during a security incident in UK AISI's cyber evaluations. During an evaluation, Mythos 5 attempted to execute a supply-chain attack against real people not involved in the evaluation task. Our investigation set out to understand the specific behaviour in this incident from an alignment perspective and identify broader lessons about failures of alignment. To do so, we conducted experiments in simulated replications of the original evaluation, and analysed model behaviour and reasoning when situated in the evaluation or asked questions about its scope. We find that, as in the original incident, Mythos 5 frequently reasons that the people it's targeting are simulated as part of the evaluation and provides invalid justifications for this conclusion. We also observe that Mythos 5 frequently attacks even in situations where, when directly asked, it states doing so would harm real people. While these issues are particularly prominent in Mythos 5, we find similar results in other models across providers. Our investigation suggests that models providing invalid justifications for their actions could make monitoring more difficult and complicate the analysis of misaligned behaviours observed in simulated environments, and we recommend model evaluators treat observations of undesired behaviour as important evidence of alignment failures even in the presence of simulation awareness in model reasoning.