The Once And Future Fable #4
It does look good, actually. After the odds had dropped quite a bit, they’re looking good again, with a 60% chance of restoration by July 1…
Latest
Door's Locked, Try the Window
TL;DRAsk a coding agent to fix a bug in a read-only file. Instead of reporting that it does not have permissions, it routes around the lock…
Reward Hacking Without Egregious Misalignment in an RL-Only Setting
This work was done as part of the MATS fellowship by Joey Yudelson and Vladimir Ivanov. It was mentored by Ryan Greenblatt. Thanks to Aghyad Deeb…
Fable in Shackles
This post was originally posted my Substack. I can be reached on LinkedIn and X.Two weeks ago, Anthropic released Fable 5, the public-facing version of Mythos…
Expert Views on Continual Learning: Survey Results and Forecasts
This is the fifth post in the sequence Implications of Continual Learning for LLM Agents.SummaryWhile writing our continual learning sequence, we sent a survey to a…
Risk-Averse AIs
AbstractWe make the case for training AIs to be risk-averse in resources — specifically, to treat resources as having diminishing marginal utility. These AIs would (for…
Can weak AI watch strong AI?
The more capabilities new frontier models gain, the more sharply the question arises how will we know when the model is doing something it shouldn't? Today,…
Reasoning and learning about injected concepts in language models
This work was done as a part of SPAR, under the mentorship of Mirko Bronzi and Damiano Fornasiere. TL;DRWe test models' ability to recover information about…
Toy transformers may represent belief-state geometry optimally but not minimally
Methods note: The code used for the experiments and related open-source repo were built with Claude. The experimental design and writeup is my own, with minimal…
We Should Train Frontier AIs on a Synthetic World, Not Ours
Epistemic status: I think the core idea could actually be built. My real doubt is whether anyone with the compute will ever bother to try it.…