Which agent framework holds up end to end?
Framework-first benchmark comparing agent frameworks on tool-calling reliability, orchestration primitives, and observability — with transparent, reproducible runs.
Framework-first, not model-first.
Benchmarks that vary the model tell you about the model. Agent Bench pins the task and varies the framework, scoring what actually decides projects: reliability of the tool loop, the control you get over orchestration, and what it costs to build, read, and extend. Each framework implements the same end-to-end workflow; every run is published with its configuration so results can be reproduced and challenged.
What a run records.
| Framework | Runtime model | Provider-agnostic | Task score | Framework score | Outcome |
|---|---|---|---|---|---|
| Scoreboards are published per benchmark run in the repository — framework, runtime model, task score, framework score, duration, and outcome — rather than summarized here, so every number stays traceable to the exact run that produced it. | |||||
Eight things worth measuring.
Tool-calling reliability
Does the framework's tool loop survive long, multi-step tasks without dropped or malformed calls?
Orchestration primitives
What control do you actually get over sequencing, parallelism, and state?
Human review quality
How well do checkpoints and approvals fit real workflows?
Ease of implementation
Time and friction to a working end-to-end task.
Lines of code required
How much glue code the framework makes you write.
Readability
Can a new engineer follow what the agent will do?
Extensibility
Cost of adding tools, models, and custom behavior.
Observability
Tracing, debugging, and run inspection support.