smevals – a small eval suite for evaluating models, prompts, and harnesses
smevals - a small eval suite for evaluating models, prompts, and harnesses I've been working with Jesse Vincent's Prime Radiant applied AI research lab building out…
Latest
Slack Emoji Maker
Tool: Slack Emoji Maker I wanted to create a new Slack emoji, and their tool recommends a square that's 128x128 and has a transparent background... so…
datasette-agent 0.4a0
Release: datasette-agent 0.4a0 New await context.browser_task() mechanism allowing agent tools to run code directly in the user's browser. #33 This is an exciting new capability: it…
Advancing the price-performance frontier with GPT‑5.6
Advancing the price-performance frontier with GPT‑5.6 Huge price drop from OpenAI today: GPT-5.6 Terra got a 20% reduction, and GPT-5.6 Luna got a massive 80% drop.…
Investigating three real-world incidents in our cybersecurity evaluations
Investigating three real-world incidents in our cybersecurity evaluations It happened again! This is turning into something of a pattern. Last week OpenAI accidentally exploited Hugging Face…
llm 0.32rc2
Release: llm 0.32rc2 Hot on the heels of RC1, this fixes a dependency issue and also adds two neat new features: The default model for users…
Quoting Bruce Schneier
The writing assignments I give my students are gym tasks, not work tasks. I ask them to write policy memos not because the world needs more…
llm-chat-completions-server 0.1a0
Release: llm-chat-completions-server 0.1a0 A key goal of the new content-addressable logs in LLM 0.32rc1 was being able to support OpenAI Chat Completion style requests where each…
llm 0.32rc1
Release: llm 0.32rc1 This RC for LLM 0.32 finishes the work that started in LLM 0.32a0 - it adds a new schema design that does a…
Quoting D. Richard Hipp
Years ago, we didn’t have SQL. There were people whose job was to generate software that would query large data sets. Their job title was COBOL…