# EvalMap · AI benchmark sources and licences Source page: https://evalmap.ai/licenses/ # Sources and licences Every number on the atlas comes from one of the sources below and is shown under its own name. Nothing is averaged or rescaled, and the coverage figures say how many of the 88 configurations each source actually reaches. Built 2026-11-06; [what changed](https://evalmap.ai/changelog/) lists every change this site has been through. The list of [model releases](https://evalmap.ai/releases/) is not a source of numbers; each of its rows links to where it came from. [← back to the atlas](https://evalmap.ai/) ## Artificial Analysis - Licence: Proprietary · taken through the AA Data API (Pro), with attribution · [site terms](https://artificialanalysis.ai/terms-of-use) · [data platform terms](https://artificialanalysiscdn.com/legal/ProDataPlatformTerms.pdf) - Source: [https://artificialanalysis.ai/](https://artificialanalysis.ai/) - As of: 2026-11-04 - Metrics and coverage: tb4 88/88, critpt 84/88, speed 88/88, steps 21/88, iq 88/88, knowAll 88/88, knowSwe 88/88, accuracy 88/88, price 88/88 - What it measures: One independent harness runs every model, so all 88 configurations are directly comparable on its evaluations. An untimed configuration is shown at an explicitly marked visual zero. Terminal-Bench here is AA’s own 198-trial run, not the official board. Since 6 October 2026 the snapshot comes from AA’s Data API on a Pro key; its speed is the median across providers on a 10,000-token prompt. Artificial Analysis publishes two sets of terms. The site Terms of Use bind every visitor and permit personal, non-commercial access only. The Data Platform Terms bind paying subscribers, EvalMap among them, and do permit public charts with attribution, but never a structured, tabular or machine-readable reproduction of individual values. Neither covers what this page does; the numbers are credited to Artificial Analysis as the API requires, and published on the owner’s decision. ## Terminal-Bench 4.0 official board - Licence: Apache-2.0 · [terms](https://github.com/harbor-framework/terminal-bench/blob/main/LICENSE) - Source: [https://www.tbench.ai/leaderboard](https://www.tbench.ai/leaderboard) - As of: 2026-09-02 - Pinned commit: b90196f7105235f869729f14aa90ed7a02bb865b - Metrics and coverage: tb4 4/88 - What it measures: Vendor-submitted runs on each model’s native agent (Claude Code, Codex, Grok Build, mini-SWE-agent) over 330 trials, with a published 95% interval. Only the submitted configurations are covered. ## DeepSWE v1.1 - Licence: Apache-2.0 · [terms](https://github.com/datacurve-ai/deep-swe/blob/main/LICENSE) - Source: [https://deepswe.datacurve.ai/](https://deepswe.datacurve.ai/) - As of: 2026-09-22 - Metrics and coverage: deepswe 11/88 - What it measures: 113 original long-horizon engineering tasks, pass@1 over four whole-benchmark passes on mini-SWE-agent, with a run-to-run 95% interval. Timeouts and context-window failures count as failures. ## EvalMapBench 6.7 - Licence: Private data · only these aggregates are published · [this page](https://evalmap.ai/licenses/) - Source: [https://evalmap.ai/evalmapbench/](https://evalmap.ai/evalmapbench/) - As of: 2026-11-06 - Metrics and coverage: evalmapbench 20/88 - What it measures: Measured here, and the one source on this page nobody can reproduce from outside. EvalMapBench 6.7 is forty-three maintenance tickets on two small private Python services of one developer, each ticket written tersely, the way a colleague writes it: the feature, not the checklist. Everything else the change must respect is stated somewhere in the repository — in CONTRIBUTING, in the docs or a decision record, in the code, or only as a regularity the surrounding code follows — and every check records where. A hidden verifier probes the changed repository as a black box: core checks for the feature itself, detail checks for the obligations the ticket left unsaid, regression checks for behaviour nothing mentioned; no judge model. The score is the share of detail checks met over three attempts on every task, twenty minutes each, beyond a baseline: every detail check the weakest configuration measured, gpt-oss-120b (high), met in any attempt is set aside (222 of 506), and only the remaining 284 count. The runs go through one open harness for every model (Harbor’s terminus-2), so the spread between rows is the model and its requested reasoning effort, named in each row. The reference solution passes every check and the untouched repository fails every core and detail check, so the scale runs from 0 to 100 by construction. The 95% interval is over the 129 trials and includes run-to-run spread. Nothing in the tasks can have been trained on and nothing in them can be shown. Earlier tracks (the v3 SWE share, the knowledge index, Blitz v5 and v5.1) are retired and no longer published. ## Epoch AI - Licence: CC BY 4.0 · [terms](https://epoch.ai/benchmarks/use-this-data) - Source: [https://epoch.ai/benchmarks](https://epoch.ai/benchmarks) - As of: 2026-10-18 - Metrics and coverage: eci 60/88, gpqa 15/88, aime 17/88, fmath 15/88, simpleqa 14/88 - What it measures: Five feeds from one publisher: the Capabilities Index plus four benchmarks Epoch runs itself — GPQA Diamond, OTIS Mock AIME, FrontierMath tiers 1-3 and SimpleQA Verified — each with the standard error of that run and the effort level it was run at. The index itself: a fitted latent-capability scale, not a percentage: anchored so Claude 3.5 Sonnet = 130 and GPT-5 = 150, unbounded above. Intervals are Epoch’s 90% bootstrap over 500 resamples, narrower in kind than the 95% intervals the benchmark boards publish. Epoch scores a model once, not once per reasoning effort. ## LMArena leaderboard - Licence: CC BY 4.0 · [terms](https://creativecommons.org/licenses/by/4.0/) - Source: [https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset](https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset) - As of: 2026-10-08 to 2026-10-14 - Metrics and coverage: arena 26/88, arenaCode 26/88, arenaMath 24/88, arenaExpert 26/88, arenaWeb 27/88, arenaFact 26/88, arenaAgent 24/88, arenaTools 24/88, arenaBash 21/88, arenaSteer 24/88 - What it measures: Ten leaderboards of two kinds. Six are Bradley-Terry ratings from blind paired votes, and measure what people prefer rather than whether a task passed: the style-controlled text leaderboard, coding prompts only, maths prompts, expert prompts, votes scored for factuality, and the WebDev arena where two apps are built from one prompt. Four come from the agent arena, its headline score and three of its five signals (invented tools, bash recovery, steerability): each is an IPS estimate of how much a session’s outcome changes when the model runs the agent, from causal tracing of live sessions. Every value carries its 95% bounds and the number of votes, sessions or signal observations behind it; observation counts are not comparable across the signal boards, which record from under one to about fifty-five per session. EvalMap rounds the values, gives each interval as a half-width and matches the entries to configurations; nothing else is changed. ## models.dev catalogue - Licence: MIT · [terms](https://github.com/anomalyco/models.dev/blob/dev/LICENSE) - Source: [https://models.dev/](https://models.dev/) - As of: 2026-10-18 - Metrics and coverage: ctx 74/88, price 72/88 - What it measures: An open catalogue of what providers publish: list price, context window and the capability flags, across 222 providers. The price shown is the model owner’s own listing where there is one, because the same model is also resold and given away; the full spread travels with the value. It agrees with Artificial Analysis on the output price of most configurations, which is the point of carrying both. ## LiteLLM model prices - Licence: MIT · [terms](https://github.com/BerriAI/litellm/blob/main/LICENSE) - Source: [https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) - As of: 2026-10-18 - Metrics and coverage: ctx 67/88, price 65/88 - What it measures: Cheapest listed output price across every provider LiteLLM knows serves the model. A provider count and the full spread travel with each value, because one model can be resold at very different rates. ## Vals AI - Licence: No licence and no terms published · republished with attribution · [their methodology](https://www.vals.ai/methodology) - Source: [https://www.vals.ai/benchmarks](https://www.vals.ai/benchmarks) - As of: 2026-09-01 to 2026-10-16 - Metrics and coverage: lawResearch 29/88, lawAgent 24/88, lawBench 16/88, tax 14/88, medCode 23/88, finAgent 29/88 - What it measures: Six boards in domains nothing else on this page reaches: legal research, legal agent work, open legal reasoning, tax, medical billing codes and the work of a financial analyst. Vals runs every evaluation itself, on its own harness, and publishes the standard error and the reasoning effort of each run. Four of the six are graded against held-out sets it does not release, so those values cannot be reproduced from outside; the board says which. The benchmarks are not all theirs: the legal agent board is Harvey’s, with Harvey’s own scoring, LegalBench is Stanford CodeX’s dataset, the research rubrics were written by practising firms, and the medical coding set is Protege’s. Their cost and latency are per task rather than per million tokens or per second, so neither is carried here. Vals publishes no terms of use, no licence and no robots exclusion — checked path by path on the fetch date recorded in each extract — which is silence rather than permission, so these values carry attribution as a matter of practice. ## BridgeBench - Licence: No licence and no terms published · republished with attribution · [their method](https://www.bridgebench.ai/blog/how-the-leaderboard-works) - Source: [https://www.bridgebench.ai/leaderboard](https://www.bridgebench.ai/leaderboard) - As of: 2026-10-15 - Metrics and coverage: bbOverall 51/88, bbReasoning 51/88, bbFront 51/88, bbBack 51/88, bbOneShot 51/88, bbSecurity 51/88, bbTrust 51/88, bbEfficiency 51/88, bbSpeed 51/88, bbCost 51/88 - What it measures: A leaderboard of 30 coding models, each rated on nine axes — reasoning, front end, back end, one-shot completeness, security, trustworthiness, efficiency, speed and cost — plus an overall that is their plain mean, all on 0 to 1000 with higher better, Cost included. These are editorial ratings, not pass rates: in BridgeBench’s own words a fixed conversion of its comparative assessments, combining judgement with the evidence it has, in whole points of a 0-10 scale. The prompts, tasks and rubrics are kept private, so no value can be reproduced from outside. It rates a model once, and says a rating can combine evidence from different configurations, so every effort variant of a model here carries the same number, and its Speed and Cost are judgements of time and spending to a useful result, not tokens per second or a price. BridgeBench is run by BridgeMind, which sells an agent app and sells sponsorship on the leaderboard page. It publishes no terms of use and no licence, and its robots.txt allows the leaderboard: silence rather than permission, so these values carry attribution as a matter of practice. ## EvalMap measurement - Licence: Measured here; every run is kept with the site’s sources · [this page](https://evalmap.ai/licenses/) - Source: [https://evalmap.ai/licenses/](https://evalmap.ai/licenses/) - As of: 2026-09-22 - Metrics and coverage: - What it measures: Output speed EvalMap measured itself where Artificial Analysis has not yet timed a configuration. On 22 September, the day they reached AA, that was all seventeen new rows, the five Claude Opus 5.5 and the twelve GPT-6 Sol and GPT-6 Luna configurations. As AA publishes its own timings, they replace these values; only configurations still missing AA speed use this source. Five public explainer prompts per reasoning effort, sent through each maker’s own CLI with no tools — Claude Code for Opus, Codex for Sol and Luna; the speed is the output tokens the API reports, thinking included, over the time from the first streamed chunk to the last, which is how Artificial Analysis defines it, and the value is the median of the five. It is one evening on one client, not AA’s 72-hour median, so read it as close, not as equal. ## No speed data - Licence: Not applicable · no measurement · [visual placeholder](https://evalmap.ai/#method) - Source: [https://evalmap.ai/#method](https://evalmap.ai/#method) - As of: 2026-11-04 - Metrics and coverage: - What it measures: A visual zero for a model whose speed has not been published. Not a measurement; excluded from Pareto comparisons and the price cloud. ## Why this exists, and what it is not EvalMap is a reading tool: it gathers published measurements alongside its own clearly identified speed measurements and [EvalMapBench](https://evalmap.ai/evalmapbench/) public aggregates, and computes the Pareto frontier for the chosen axes and sources; it recommends nothing. EvalMapBench’s tasks and raw run records remain private. The jeep bench’s scores stay in the jeep gallery and off the chart. Nothing here earns from the data — no advertising, no affiliate links, no sponsorship, no paid placement, nothing for sale. No source pays to appear and none can pay to move. Nothing that identifies a visitor is collected: no cookies, no accounts, no third-party scripts or fonts, and choices are kept in the address bar so a link can be shared. Cloudflare hosts the files, and its cookieless page counter is deliberately left on so the readership is known; it reports totals only — pages, referrers, country, browser — and identifies nobody. A source owner who wants values removed or corrected only has to ask. ## Contact Questions about a value, corrections and requests from a source owner: [support@evalmap.ai](mailto:support@evalmap.ai). ## Non-endorsement Artificial Analysis, Terminal-Bench 4.0 official board, DeepSWE v1.1, EvalMapBench 6.7, Epoch AI, LMArena leaderboard, models.dev catalogue, LiteLLM model prices, Vals AI, BridgeBench, EvalMap measurement, No speed data do not endorse EvalMap, are not affiliated with it and have not reviewed it. Their names and marks belong to them and are used here only to say where a value came from. Benchmark questions and answers remain the property of their creators, and externally sourced results keep their original terms.