Repurposing Protein Folding Models for Generation with Latent Diffusion
PLAID is a multimodal generative model that simultaneously generates protein 1D sequence and 3D structure, by learning the latent space of protein folding models. The awarding…
Open Source 52 stories
Scaling Up Reinforcement Learning for Traffic Smoothing: A 100-AV Highway Deployment
Training Diffusion Models with Reinforcement Learning We deployed 100 reinforcement learning (RL)-controlled cars into rush-hour highway traffic to smooth congestion and reduce fuel consumption for everyone.…
SAEs trained on the same data don’t learn the same features
In this post, we show that when two TopK SAEs are trained on the same data, with the same batch order but with different random initializations,…
Partially rewriting an LLM in natural language
Using interpretations of SAE latents to simulate activations.
We’re moving our blog!
We’re excited to announce that our blog is moving to its new home! From now on, all our new blog posts will be published directly on…
Third-party evaluation to identify risks in LLMs’ training data
An overview of the minetester and preliminary work
Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback
Much of the recent advancements in large language models (LLMs) have been powered by human feedback, usually in the form of preference datasets. Think of preferences…
Applying Theory of Mind: Can AI Understand and Predict Human Behavior?
“Theory of Mind” (ToM) is the ability to understand that others have their own thoughts and beliefs, even when they differ from ours — a skill…
Ai2 at COP 16: Harnessing AI and Conservation Tech to Protect Our Planet
Empowering conservation efforts through innovative technologies and global collaborationA vessel captured by NASA’s Landsat 8. Skylight’s computer vision models leverage this imagery to identify suspicious behavior,…
Mechanistic Anomaly Detection Research Update 2
Interim report on ongoing work on mechanistic anomaly detection
RLHF and RLAIF in GPT-NeoX
GPT-NeoX now supports post-training thanks to a collaboration with SynthLabs.
Investigating Pretraining Dynamics and Stability with OLMo Checkpoints
A central goal of the OLMo project is to use our experience to contribute to an open science of LM pretraining to provide a foundation for…
The Practitioner's Guide to the Maximal Update Parameterization
Exploring the implementation details of muTransfer
Ai2 at DEF CON 32
By Technical Program Manager Christopher FiorelliAt Ai2, we’re continually looking for ways to expand the impact of AI research. Through collaboration with DSRI, Ai2 was invited…
OLMoE: An open, small, and state-of-the-art mixture-of-experts model
We’re introducing OLMoE, jointly developed with Contextual AI, which is the first mixture-of-experts model to join the OLMo family. OLMoE brings two important aspects to the…
Digital Socrates: Evaluating LLMs through Explanation Critiques
Blog written by Yuling GuLooking for an interpretable explanation evaluation tool that can automatically characterize the explanation capabilities of modern LLMs? Meet Digital Socrates at ACL…
Open research is the key to unlocking safer AI
The last few years of AI development have shown the power and potential of generative AI. Naturally, these leaps in machine intelligence have opened existential questions…
Latest and greatest: Ai2’s release notes
Along with our rebrand, we’re excited to debut a new release note process. Because we’re making regular updates and new asset roll-outs in our open ecosystem…
Mechanistic Anomaly Detection Research Update
Interim report on ongoing work on mechanistic anomaly detection
Open Source Automated Interpretability for Sparse Autoencoder Features
Building and evaluating an open-source pipeline for auto-interpretability
Experiments in Weak-to-Strong Generalization
Writing up results from a recent project
Free Form Least-Squares Concept Erasure Without Oracle Concept Labels
Achieving even more surgical edits than LEACE without concept labels at inference time.
VINC-S: Closed-form Optionally-supervised Knowledge Elicitation with Paraphrase Invariance
Writing up results from a project from Spring 2023
Yi-34B, Llama 2, and common practices in LLM training: a fact check of the New York Times
Setting the record straight regarding Yi-34B and Llama 2.