Web Automation Microbench
Head-to-head benchmarks of 33 browser automation tools on one identical task, with a second real-work ranking that reorders them.
What it is
One fixed job, one independent pass test, and 33 browser automation tools scored on the same terms. The project publishes two leaderboards that deliberately disagree: fast-path speed and cost on a controlled microbenchmark, and real-work capability on 11 harvested browser tasks. The fastest tool on the toy task is among the weakest on real work.
A shared Python benchlib owns the task text, timer, token accounting and result schema, so each tool is a small adapter and nothing else. Pass or fail is decided by separate code reading the page and the app state, never the agent's own claim. Failed runs are kept, screened at two reps and promoted to five so a single lucky rep cannot inflate reliability.