SFF is very suboptimal
I recently served as a recommender in SFF's annual funding round (grants will be decided and announced in September). I'm deeply grateful for SFF's funders, and…
Latest
Bounding eval awareness of ~human-level AI across the safe-to-dangerous shift
In our last post, we argued that measuring evaluation awareness is fundamentally challenging because of the safe-to-dangerous distributional shift: we cannot directly measure the evaluation awareness…
Sub-agent delegation chaining
Epistemic status: pretty confident in the validity of the core proposal, not that confident in specific implementation detailsTL;DR: we should cryptographically verify that sub-agent instances/sessions are…
New funding opportunity on digital minds from Longview
This is a linkpost for Request for Proposals: Research and Applied Work on Digital Minds.The text below was written in the first person by Zach Freitas-Groff,…
Lightning Jazz
“One thing I like about jazz, kid, is that I don’t know what’s going to happen next.” — attributed to Bix Beiderbecke.I hate lectures. I refuse…
Tie training can make DPO/RLHF-trained AIs generalize better
This post covers our recent ICML paper: Spurious Correlation Learning in Preference Optimization: Mechanisms, Consequences, and Mitigation via Tie Training.TL;DROur theorems and experiments suggest that DPO…
Tips on leveraging AI for empirical research
Tl;dr I've been thinking hard and experimenting with how to best take advantage of AI for my work. I've developed some practices, mostly through trial and…
Desiderata for functional welfare experiments on LLMs
TLDRLLMs appear to have functional welfare: coherent sets of behaviour that track how well things are going relative to their goals. Improving model functional welfare matters…
Qualia-based emotion steering makes LLMs attribute conscious states to themselves
Summary results/takeawaysI steer Qwen3-32B and 235B along qualia-related emotion directions (blissful, tormented, terrified, serene, etc.) by adding an emotion vector to the residual stream at varying…
Computation enables Action: Exploding the Simulation Fallacy
In April, Tyler Cowen linked (without comment) to a paper titled "Why AI can simulate but not instantiate consciousness". I've been paying attention to AI and…