Early Indicators of Reward Hacking via Reasoning Interpolation
Using importance sampling with fine-tuned donor prefills to predict reward hacking emergence during training
Latest
Reward Hacking Resarch Update
Interim report on ongoing work on reward hacking
Pretraining Data Filtering for Open-Weight AI Safety
Announcing Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
Attention Probes
Adding attention to linear probes
Research Update: Applications of Local Volume Measurement
Research update on on applying local volume measurement to downstream tasks
Studying inductive biases of random networks via local volumes
In this post, we will study inductive biases of the parameter-function map of random neural networks using star domain volume estimates. This builds on the ideas…
The Common Pile v0.1
Announcing the Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
Product Key Memory Sparse Coders
Using Product Key Memories to encode sparse coder features
SAEs trained on the same data don’t learn the same features
In this post, we show that when two TopK SAEs are trained on the same data, with the same batch order but with different random initializations,…
Partially rewriting an LLM in natural language
Using interpretations of SAE latents to simulate activations.