Explore

Looking for our schedule?

Next benchmark run

public

AI Gateway — Anthropic public release

Fri, Sep 4 · 9:00 AM

Benchmark summaries

Latest results

Next benchmark Friday5h 30m 12s

Sandboxes

Composite score · Higher is better

Composite score · Higher is better

Composite score · Higher is better

Composite score · Higher is better

OpenRouter

89.7

Cloudflare AI Gateway

What We Benchmark

Five suites covering the infrastructure AI products actually run on

Time to interactive

Sandboxes

Cold-start latency and reliability across sandbox providers, measured in a 100-sandbox burst.

View benchmarks

Clone → install → typecheck

Real-world workload

A full dax-style developer loop inside a fresh sandbox — system packages, Bun, a pinned opencode clone, install and typecheck — timed phase by phase.

View benchmarks

Latency and throughput

Storage

Upload and download of a 10 MB object against each provider's bucket, with each operation timed independently.

View benchmarks

Actions per second

Browsers

Per-action throughput inside a live headless session — 10 sequential actions across 100 sessions, in stealth mode at 1920×1080.

View benchmarks

Cold E2E and TTFT

AI gateways

Connection-phase latency, time to first token, and generation throughput for each gateway, scored against a direct-to-Anthropic baseline.

View benchmarks

Methodology

How the Numbers Are Made

Every benchmark is reproducible: open code, fixed workloads, results published in full

One workload, every provider

The same script, payload, and model hit every provider, addressed the way its own API expects. Nobody gets a tuned path.

Repeated on a schedule

Every suite runs many iterations per provider, automated in GitHub Actions on a recurring schedule, so results reflect steady state rather than one lucky request.

Scored on the tail, not the best case

Each composite score blends median and tail latency against a fixed ceiling, then multiplies by success rate — providers that fail often lose real points.