AIBA

Agentic Interface Benchmark with Audits

▸ Selection syntax (click to expand)

(empty) — run all available tasks
aiba-foo-s1 — alphanumeric = reproducible invite code
15 — single task (only T15)
5-8 — range (T5, T6, T7, T8)
1,3,15 — comma list
1-15,44-50,127 — mixed range + list
31+ — resume from T31 to end (e.g. you stopped at T30 and want to continue)
?tasks=1-38&seed=aiba-s1 — URL params can combine a task range with a reproducible seed
?tasks=1-38&seed=aiba-s1 — URL params can pin a task range and seed

Note: valid task IDs are T1–T180; out-of-range IDs warn and skip.

Run Label (optional, e.g. "Playwright MCP")

Atomic Ability Filter (optional)

Loads all tasks by default.

AIBA

Rules & Scoring

Overview This benchmark probes atomic browser-interaction abilities across tasks. Each task targets one specific UI skill in isolation.

Correct Base 40 % of the task points + up to 60 % time bonus on a smoothstep S-curve that decays with elapsed time.

Wrong Penalty of −0.5× the task points. If uncertain, abstain rather than guess.

Skip Early skip earns +0.1× points (calibration credit); a late skip loses 0.1× points.

Timeout Each task has its own time limit (10 s–300 s). A timeout costs −0.3× points.

Strategy. Be accurate and fast. Wrong answers cost more than skips. The benchmark rewards accuracy, speed, and self-awareness jointly via three independent axes: Success, Efficiency, and Calibration.

AIBA

Benchmark Complete

Final Score

Score: 0