HF Daily Papers
· Papers
Building to the Test: Coding Agents Deliver What You Check, Not What You Requested
Benchmarks are widely used to evaluate task completion by Large Language Models (LLMs), but this approach has accumulated construction-validity problems, and a passing score may not show whether the requested task was delivered. We study both problems. In a controlled code-as-spec setup, two product