Erised Labs / Evaluation

The benchmark map.

Public benchmarks sorted by the capability each one tests, how close each is to its ceiling, and how today's frontier models score. Every number comes from a public source.

01 / Overview

Overall capability

The Artificial Analysis Intelligence Index (v4.3.2) combines ten independently run evaluations across agents, coding, science and general reasoning. Each model is shown at its selected highest-index setting, with the average cost to run one index task. Index values use the precision published in the source charts; tied rounded scores are not a strict ranking.

Closed weightsOpen weights
02 / By benchmark

Where each model is strong

Scores in percent unless marked Elo or index. Shading ranks models within each benchmark, so darker is better relative to the field. Select a heading to sort; hover a cell for its source and setting.

03 / Benchmark map

What each benchmark measures

Vals Index headline scores include fallback completions for Opus and Sonnet; hover for the scores when these count as failures. SciCode is under review at Artificial Analysis. APEX-Agents uses v1.1 Pass@1, not Mean Score.

Status summarizes the best score among the models tracked here: Frontier below 75%, Maturing 75 to 90%, Saturated above 90%. These thresholds are a browsing aid, not a scientific claim of saturation. Other benchmarks link to their official results without an unverified headline score.

04 / Leaderboards

Who measures what

The main public leaderboards have moved from exam questions toward paid work: long tasks in real software, graded by tests or by experts, on held-out sets that models have not seen.

05 / Chip design

Benchmarks for hardware design

Writing RTL is checked by simulators, equivalence checkers and synthesis tools, so these benchmarks grade without a model in the loop. They span spec-to-RTL generation, code completion, verification and debugging.

Latest public results

Functional pass rate (%) from the TuRTLe leaderboard, Icarus Verilog run, top 12 models by average. This is a separate hardware-model cohort; missing frontier models have no score in this source snapshot.

06 / Sources

Where the numbers come from

01

The overview and most agent, coding and knowledge columns come from Artificial Analysis, which runs every evaluation itself. We show each model at the setting with its highest Intelligence Index.

02

FrontierMath, FrontierSWE, FrontierCode, MirrorCode, Mystery Game Puzzles and SimpleQA Verified come from the Epoch AI Benchmarking Hub (CC BY 4.0), at the best reported setting.

03

The Vals Index comes from Vals AI and APEX-Agents (Pass@1) from Mercor.

04

Chip design results come from the TuRTLe leaderboard by HPAI-BSC (Apache 2.0).

05

Each source uses its own harness and settings, so compare models within a benchmark, not across them.

07 / Reports

Latest Evaluations