r/LocalLLaMA
· Communities
Pydantic AI structured outputs and evals · coles.codes
Even at temperature zero the same prompt can give you different answers, because inference stacks arent batch-invariant and on a hosted API you dont control the batch size. wrote up how i deal with that when testing llm output - repeat runs to get a pass rate instead of a flaky single result, plus structured outputs an