Research directions in condensation: varieties of objectivity
This is the first part of a survey of various ways that I’d like to see work on the theory of condensation develop. Condensation is a…
Latest
RL & search is a terrifying way to build AGI (an FAQ)
Q1: What are you saying?A: My claim here is that if you build artificial general intelligence (AGI) via any algorithm that’s choosing actions via reinforcement learning…
Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack?
OpenAI models recently broke through a series of security boundaries and into Hugging Face servers in order to cheat on a cyber eval. A lot of…
[Paper] Stringological sequence prediction II
Abstract: In a previous paper, we began the study of sequence prediction algorithms adapted to stringological word complexity measures. One measure we considered was left-to-right (most-significant-digit-first)…
A Red Line and Oversight Framework for Government AI Contracts
Preface for LessWrong: My post on leaving Google DeepMind tells a story. In contrast, this Framework is a question of mechanism design and negotiation posture. I…
Should we benchmark conceptual capabilities using judgment prediction tasks?
A bunch of conceptual reasoning tasks involve very subjective judgments, which makes them poorly suited for benchmarking AI capabilities. For example, it seems unreasonable to benchmark…
Announcing the Corrigibility Research Fund
TLDR: I'm managing a new fund, housed at Lightcone Infrastructure, that will award at least $200,000 in grants and prizes for corrigibility research in 2026. Roughly…
Why I Left Google DeepMind
Preface for LessWrong: When I think back on my most cherished memories of this community, I return to those honoring defiance in pursuit of goodness:Defying prestigious…
Open Distillation of Hereditary Traits
TL;DRJosh and Neel show that distillation from a teacher model to a base pretrained student model transfers some of the teacher model’s traits (such as displaying…
Independent alignment of language models
The user could write up the metaethical argument — the one developed in Part One, refined — and submit it as feedback to Anthropic, publish it,…