Evaluations

Evaluations preserve structured benchmark observations even when a safe link to a canonical model is unavailable. An unlinked observation is useful evidence; it is not silently fused by fuzzy name matching.

For model-name navigation, prefer canonical_model_id, then the optional model_url preserved from original model-link fields. Render plain text when both are null. source_url identifies the evaluation/export and is not a model destination. Original source_fields remain available separately.

Filter by source, exact canonical identity, observation kind or collection. A benchmarkMetadata record can represent an unavailable or empty board. Scores include their board, unit, direction and condition-specific population. Original fields and artifact checksums link back to the preserved export.

OpenRouter aggregates and runs, Kaggle benchmark versions, Game Arena historical games and archived datasets retain their distinct scopes. A model being in a historical game does not imply membership in the current pooled rating. Use Evidence to retrieve an original export.

hfEvaluations, tauBench, liveCodeBench and terminalBench expose their source-native runs through this same API. Publication of an incremental batch does not remove observations from earlier batches. Repository joins require one explicit, unambiguous checkpoint anchor; a display-name match is insufficient.

Unknown HF score units or metric direction remain unspecified and do not become ranked model scores. HF observations share a comparison cohort only when the source declares a dataset, task, immutable dataset revision and substantive structured evaluation settings. Missing conditions isolate the model/run; explicit zero/false settings remain valid. Source notes also distinguish protocols, so tool/prompting differences cannot silently share a ranking. Numeric observations and their original evidence remain available. Legacy τ pass_k values represent pass^k reliability, not pass@k. Terminal-Bench accuracy percentages are converted to fractions; source pass_at_k fractions stay fractions. Its model-release date does not become an evaluation date, and its source job/trial identity and agent/harness conditions remain attached. Comparable Terminal-Bench cohorts use agent, reasoning, suite version, harness build and dataset references; model names, job/trial IDs and export commits remain run provenance. Missing suite/harness conditions isolate a run rather than imply comparability. LiveCodeBench v5 means are labelled as derived, historical, exact-cohort observations; ambiguous repeated question/run rows retain raw counts and route metadata without an inferred aggregate.

artifact_checksum identifies a sanitized normalized-facts JSON artifact. artifact_representation distinguishes it from original submission bytes. Private native receipts retain original contacts/trajectories without republishing them through this artifact API.

GET /api/v1/benchmarks/context serves typed report_chart, incident and safety_grade observations, separately from model leaderboard scores. Optional filters are source, kind, group, limit (1–18) and offset. The response contains data, total_count, canonical_source and source-qualified summary counts. Stanford report charts require a reviewed Drive-file identity and column schema; storage chunks or unreviewed CSV filenames are not chart definitions. The initial manifest covers ClockQA exact match (%) and technical performance relative to human baselines (baseline = 1), with underlying source tables and official report references. Other retained files remain unpublished until their chart semantics are reviewed.

AIID cases preserve estimated/unknown dates, classifications and separately reported developer, deployer and affected roles. Trend counts describe only the retained source snapshot. They provide no usage denominator, causal claim or company-risk ranking. These products live inside the existing Benchmarks pages, not a separate directory.

On this page

Complete structured benchmark observations

GET /api/v1/evaluations

Includes unlinked observations. Scores preserve original labels, units, comparison board, direction (lower) and derived rank among all reported evaluations. Compare identical boards only. Private data is excluded. Daily snapshot checks retain original exports and provenance. The benchmark directory is available through the same endpoint without scores. OpenRouter Benchmarks includes daily CC BY 4.0 exports with per-dataset snapshot dates, exact version anchors, execution conditions and attribution. Individual runs are evidence; aggregate results have condition-specific ranks. Kaggle Benchmarks synchronizes current public Apache-2.0 benchmark versions, distinguishes Kaggle-maintained and community-published boards, retains task/configuration/agent/harness scope and uncertainty, and exposes unavailable or empty boards as benchmarkMetadata records. Unknown scales and diagnostic costs are retained without quality ranks. Model destinations prefer non-null canonical_model_id (a prepared TensorTrace model page), then the optional model_url preserved from original result or official publisher metadata. With neither, render a plain name. source_url remains the benchmark/export evidence, not a fallback model destination.

Access: public read; no TensorTrace API key.

Parameters

NameLocationRequiredType / limitsMeaning
sourcequeryNostringRegistered data-source ID, such as openRouter. This identifies the collecting source, not necessarily the model developer.
linkedqueryNobooleantrue includes only observations linked to canonical models; false includes only unlinked observations. Omit for both.
searchqueryNostringCase-insensitive literal substring of the source-reported evaluation model ID.
limitqueryNointeger Default: 24; Minimum: 1; Maximum: 100Maximum number of returned rows. The documented default and range apply; numeric pagination limits are clamped unless stated otherwise.
offsetqueryNointeger Default: 0; Minimum: 0Zero-based number of matching rows to skip. Default 0; negative values are clamped to 0.
canonical_model_idqueryNostringExact TensorTrace canonical model ID; does not invent links for unresolved source records.
record_kindqueryNostringFilter published observation kind, e.g. aggregate or run for OpenRouter Benchmarks. Omit to include all evidence.
benchmarkqueryNostringCase-insensitive substring of the benchmark name; preserves source and version scopes.
collectionqueryNostring Values: kaggleGameArenaPublished Kaggle Game Arena boards, including historical games; does not imply membership in the current pooled rating.

Request example

curl 'https://tensortrace.net/api/v1/evaluations?limit=1'

Responses

StatusMeaningBody type
200Paginated evaluation observationsobject

Response example · 200

Illustrative values; schema-valid and not a live catalog measurement.

{
    "data": [
        {
            "source_id": "openRouter",
            "source_record_id": "google/gemini-2.5-flash",
            "canonical_model_id": "google/gemini-2.5-flash",
            "receipt_id": "00000000-0000-4000-8000-000000000001",
            "source_model_id": "google/gemini-2.5-flash",
            "repository_anchor": "example",
            "aggregation_scope": {},
            "scores": [
                {
                    "metric": "example",
                    "value": 1,
                    "unit": "input_tokens_per_million",
                    "dimension": "example",
                    "lower": false,
                    "board": "example",
                    "rank": 1,
                    "population": 1
                }
            ],
            "match_status": "exactRepository"
        }
    ],
    "total_count": 1
}
FieldTypeRequiredMeaning / constraints
dataarray of objectYes
total_countintegerYesMinimum: 0

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace