Reinforcement Learning on Forecasting Will Give Us a Superhuman Forecaster
This is a crosspost of a post from my blog, Metal Ivy. The original is here: Reinforcement Learning on Forecasting Will Give Us a Superhuman Forecaster.Why…
Latest
The arithmetic hierarchy of real functions
I wrote a fairly accessible introduction to real hypercomputation with Marcus Hutter. The focus is on enabling applications to algorithmic information theory. This project was intended…
Anthropomorphic Misalignment research needs stronger evidence
This is a distillation of our ICML 2026 Oral position paper, Position: Anthropomorphic Misalignment Research Needs Stronger Evidence. Joint work by Vansh Gupta, Peter Nutter, Samuel…
A survey of okayish ASI futures
At this point, RSI loops and continual learning appear overwhelmingly likely to begin in the near future. Whatever the limit of the LLM paradigm(plus whatever new,…
GPT-5.6: The System Card
While we wait for a general release, the system card is the best hint as to what is going on with the new candidate for America’s…
Evaluating Offline Monitoring of Internal AI Agents
This work was conducted during the GovAI Winter Fellowship 2026. Full reportExecutive SummaryFrontier AI companies use offline monitoring to address risks from internally deployed AI agents.…
Power Laws in NNs: A Possible Mechanism for Inductive Bias towards Sparse Representations
This post was produced as part of the Iliad Fellowship under the mentorship of Dmitry Vaintrob. Tl;dr: Power-law ("heavy-tailed") distributions have universality theorems similar to those…
Refusal Is Complicated As Hell: An Update
TL;DRIt would make sense to briefly skim through our previous post that introduces our experiments on refusal in LLMs. There we explain how it started, here…
A partner who plays music
Drafted March 6 2025I have thought for many years that it is a risky thing to have a partner who is into creating things, lest you…
Can we use steering vectors to suppress reward-hacking? Somewhat
Can steering vectors drive gradient routing? Yes, but not in realistic reward hacking environments, they are not precise enough classifiers of hacky vs clean solutions. Instead,…