Observations from two CLI game reimplementation runs with Opus 4.6
Summary: Opus 4.6 can, with a simple agent scaffold, create mostly-playable but somewhat broken CLI versions of Slay the Spire and Balatro1. Intro Last weekend I…
Latest
We are Changing our Developer Productivity Experiment Design
METR previously published a paper which found the use of AI tools caused a 20% slowdown in completing tasks among experienced open-source developers, using data from…
Five lessons from having helped run an AI-Biology RCT
.show-xxl { display: none; @media (min-width: 1471px) { display: block; } } Evidence-based AI policy is important but hard. We need more in-depth studies – which…
How We Protect Confidential Information
.content .tab-pane blockquote { padding: 0.5rem 1rem; margin: 0.5rem 0; } .content .tab-pane blockquote p { font-size: 1.15rem; margin-bottom: 0.75rem; } .custom-padding-table-selector + table * {…
Analyzing coding agent transcripts to upper bound productivity gains from AI agents
Introduction Human uplift studies like the one we did in 2025 are becoming more expensive as working without AI becomes increasingly costly. In this post, I…
Measuring Time Horizon using Claude Code and Codex
Most of METR’s time horizon measurements are done using two scaffolds: Triframe and ReAct1. People sometimes see that we use these two scaffolds and feel skeptical…
A simpler AI timelines model predicts 99% AI R&D automation in ~2032
In this post, I describe a simple model for forecasting when AI will automate AI development. It is based on the AI Futures model, but more…
Regulación de seguridad de IA de frontera: una referencia para el personal de laboratorios
Ver como PDF Los desarrolladores de IA de frontera como OpenAI, Google, Anthropic, xAI y otros tienen obligaciones de seguridad y protección bajo la SB 53…
Frontier AI safety regulations: A reference for lab staff
View as PDF Frontier AI developers such as OpenAI, Google, Anthropic, xAI, and others are governed by safety and security obligations under California’s SB 53, New…
前沿 AI 安全法规：AI 公司员工参考指南
查看英文 PDF 版 OpenAI、Google、Anthropic、xAI 等前沿 AI 开发者，以及部分中国 AI 开发者，已经需要遵守多项安全与安保义务。主要来源包括加州 SB 53、纽约 RAISE 法案，以及欧盟《人工智能法》中有关前沿 AI 的条款。相关法规是否适用，不取决于公司注册地，而取决于模型在哪里部署、公司在当地如何开展业务。这些要求涵盖事件报告、模型评估、安全与安保缓解措施、内部治理和举报人保护。本文只梳理关键条款，不能替代官方法律文本。 法律 适用对象 风险 义务 时间表 加州 SB 53 使用 >10^26 FLOPs…