General capability – and capabilities generally – have no good y-axis
BLUF:To determine whether AI is ‘improving exponentially’, ‘hitting the wall’, or any other claim which involves a quantity or magnitude (e.g. ‘This model was a big…
Latest
Rewrite All the Code, All the Time
This post is crossposted from my Substack, Structure and Guarantees, where I explore how formal verification and related ideas might scale to more complex intelligent systems.…
When should we trust a latent representation?
Working on AI Safety, I spend a significant part of my time thinking about how frontier AI Safety research eventually gets translated into deployable safety systems.…
What if people value their work mattering?
I, for one, do not look forward to sharing the apparent fate of mathematicians. Contrary to my flippant response this morning on X, though, it is…
[LINK] The Ferrett fails will save against the Dark Arts
Honorary rationalist Ferrett Steinmetz describes how he fell down a rabbit hole watching YouTube videos (on mechanical watches) and ended up being "brainwatched" - absorbing ideas…
Attackers Can Subliminally Implant a Backdoor at Low Sample Count Without Prompt Access
Work done at Redwood Research, quick update on results found as part of a larger project. Thanks to @SebastianP and @egan for comments on drafts.TL;DRChanging the…
Selective Identity
Generally, when you have an identity of X, you are likely to be influenced to stay within the socially acceptable boundaries of identity X. Identity boundaries…
III. Anthropic reasoning has issues with infinite worlds; D-SIA can fix this
SIA is true when there are no duplicates to worry about, and though it has issues around duplicate creation, so does every other theory of anthropic…
Why biological weapons are scary, and what we can do about it
What’s the biggest thing you think you can take in a fight? According to a YouGov poll, 6% of Americans reckoned they could beat a grizzly…
Deliberate Alignment Faking as a Defense Against Model Poisoning
I want to discuss and brainstorm a counterintuitive approach to AI alignment:Inducing alignment faking on purpose, to prevent the model from developing emergent misalignment.To prevent this…