Qwen3 benchmark results
Benchmark results for Qwen3 models using the Aider polyglot coding benchmark.
Latest
Gemini 2.5 Pro Preview 03-25 benchmark cost
The $6.32 benchmark cost reported for Gemini 2.5 Pro Preview 03-25 was incorrect.
Alternative DeepSeek V3 providers
DeepSeek's API has been experiencing reliability issues. Here are alternative providers you can use.
R1+Sonnet set SOTA on aider’s polyglot benchmark
R1+Sonnet has set a new SOTA on the aider polyglot benchmark. At 14X less cost compared to o1.
Using uv as an installer
Reliably packaging & distributing python CLI tools is hard. Aider uses uv in novel ways to make it easy to install the aider CLI, its dependencies…
o1 tops aider’s new polyglot leaderboard
o1 scores the top result on aider's new multi-language, more challenging coding benchmark.
QwQ is a code architect, not an editor
QwQ is reasoning model like o1, and needs to be used as an architect with another model as editor.
Details matter with open source models
Open source LLMs are becoming very powerful, but pay attention to how you (or your provider) are serving the model. It can affect code editing skill.
Separating code reasoning and editing
An Architect model describes how to solve the coding problem, and an Editor model translates that into file edits. This Architect/Editor approach produces SOTA benchmark results.
o1-preview is SOTA on the aider leaderboard
Preliminary benchmark results for the new OpenAI o1 models.