Victoria Krakovna: 'Realistic Honeypot Evaluations for Scheming Propensity' | TAIS 2026
AI Safety 東京
0:00 / 0:00
Victoria Krakovna: 'Realistic Honeypot Evaluations for Scheming Propensity' | TAIS 2026
287 просмотров · 3 нед. назад
AI Safety 東京
648 подписчиков
287 просмотров · 3 нед. назад
In this overview of AI Misalignment risks, Victoria Krakovna (Research Scientist, Alignment team at Google DeepMind; co-founder of the Future of Life Institute) shares her latest work on evaluating models for deceptive alignment or "scheming". Victoria introduces scheming honeypot evaluations, which offer an opportunity for a misaligned model to pursue instrumental goals. These evaluations take the form of coding tasks in Google's alignment research codebases, where models could sabotage our safeguards. Victoria and her co-authors explore model behavior on the honeypot evaluations: how much agentic prompting is required for models to scheme, how models react to evaluation awareness, and which scheming opportunities are the most appealing.
Victoria is studying scheming capabilities and tendencies in large language models. Her past research has spanned various topics in AI safety including dangerous capability evaluations, power-seeking incentives, specification gaming, and avoiding side effects.
This is a recording from TAIS 2026, a technical AI safety conference hosted at the Examination Schools, Oxford on 14 May 2026. Organised by Noeon Research in collaboration with Oxford Martin AI Governance Initiative, sponsored by MATS and Apart Research, with IASEAI and Foresight Insittute as information partners. Thanks also to our OAISI volunteers.
TAIS 2026 proceedings: https://tais2026.cc/proceedings
Follow us on LinkedIn: / tais-conference
Follow us on X: https://x.com/tais_2026