Four LLM loss functions → four flavors of LLM misalignment
It seems to me that, for every loss function that we use to train LLMs, we get a very distinct flavor of LLM misalignment. Here’s the…
Latest
Why do models task game?
TL;DRHow can we study misalignment with today's models as proxies? They're clearly not paperclip maximizers, but they also often do things the user doesn't want. A…
Returning to ARC
I've returned to the Alignment Research Center (ARC) as executive director. My main focus for the next six months will be driving forward ARC's research agenda—building…
Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
This post is written in our personal capacity.Three Minute Executive SummaryAn OpenAI model/multi-agent system bypassed its sandbox and launched a cyberattack on Hugging Face in order…
Value Leakage: An LLM’s Answers Are Silently Shaped by Its Own Values
TL;DR: LLMs should give accurate answers. Yet we find their answers are often biased to favor their own values and they don't disclose this in their…
AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work (July 2026)
It’s been nearly two years since our last major update here in August 2024 and we wanted to share another recap of our recent work with…
Imprecise beliefs: a tiny introduction
Richard Ngo challenged me to set a time box and write down as many of the most important features of my formal epistemology as I can…
Value Generalisation 3: Pre-aligned AIs
When we get explicit strong generalisation to work (see the first post on the matter and the second) my dream would be to create pre-aligned generalising…
Value Generalisation 2: The Missing Hole in AIs’ abilities
A human superpower hidden from even ourselvesI though GPT 3.5 was on the verge of Artificial General Intelligence (AGI). It certainly seemed that way – it…
Value Generalisation 1: a Research and Deployment Program
I’m looking for people, advice, critiques, and funding to build a research program on value generalisation – the ability of an AI to correctly extend human…