Toward A Public Science of Model Behavior
This is a linkpost for our essay "Toward A Public Science of Model Behavior" on transluce.org. The full text is reproduced below.Today’s AI systems frequently behave…
Latest
AI Safety Policy Needs to train Legal Practitioners
I completed my law degree at a working-class London university. In my first year, I was 18 years old, and I was often the youngest person…
What would it take for AI to discover penicillin?
At the end of May, I attended the SciFM26 conference hosted at UChicago which had many speakers discussing agentic science and the future of AI+Science. This…
Natural Language Autoencoders are summarizers, but do they have to be?
TL;DR: NLAs are an interesting idea. They do reconstruct the activations well, they can kinda audit model organisms for hidden objectives. But there are hints that…
Where Do LLM Values Come From?
This work was done as part of the MATS 8.1 Program.0: TL;DRLLMs learn "values": general considerations (e.g. "playfulness & humor", "mental health sensitivity") that influence their…
Selective Optimism: a critique of AI 2040
Some context for this post: I’ve been working part-time as a consultant for the AI Futures Project over the last year. Most of the work I’ve…
Rogue ASI Can't Stay Aligned to Itself
In a world where rogue ASI can form a singleton, could it really widely deploy an agent fleet across the world(let alone onto other stellar bodies)…
Criticism against "unembedded FDT" doesn't apply to FDT
TL;DR: Some criticisms aimed at FDT are actually aimed at a self-contradictory "unembedded FDT," and are therefore irrelevant to any refutation of FDT.Functional Decision Theory (FDT)…
Your Prompt-Injection Defense Metric Might Be Lying to You
Indirect Prompt Injection at present day, is one of the main reasons for agentic failures deployed in personal systems as well as enterprise grade applications/systems. The…
When is misalignment just a bug?
Cross-posted from The Foretellix CTO Blog. Introduction and epistemic status: This is the first post in a planned series, “Alignment as a verification problem”. I co-originated…