# Benchmarks

## Current baseline policy

- Active baseline reference: [BASELINE_RESULTS.md](./BASELINE_RESULTS.md) (`2026-03-11`)
- Baseline artifacts: [2026-03-11 summary](./runs/2026-03-11/summary.json)
- Verification evidence: [VERIFY_2026-03-11.log](./VERIFY_2026-03-11.log)
- Future cycle policy: compare subsequent target runs to baseline `2026-03-11` unless explicitly re-anchored.

## Delta artifacts
- [DELTA_2026-03-11.md](./DELTA_2026-03-11.md)
- [DELTA_2026-03-11.json](./DELTA_2026-03-11.json)

# AI Research Benchmarks

## Run artifacts layout

Canonical benchmark artifact locations are defined in `ARTIFACT_CONTRACT.md` and implemented by
`benchmarks/artifact_contract.sh`.

- Run directory root: `${ARTIFACT_RUNS_DIR}` (defaults to `benchmarks/runs`)
- Per-run summary: `${ARTIFACT_RUNS_DIR}/<RUN_ID>/${ARTIFACT_SUMMARY_NAME}`
- Per-run logs:
  - `${ARTIFACT_RUNS_DIR}/<RUN_ID>/*.stdout.log`
  - `${ARTIFACT_RUNS_DIR}/<RUN_ID>/*.stderr.log`
- Current run symlink: `${ARTIFACT_LATEST_LINK}` (defaults to `benchmarks/runs/latest`)
- Latest run pointer file: `${ARTIFACT_LATEST_RUN_FILE}` (defaults to `benchmarks/runs/LATEST_RUN.txt`)
- Latest summary copy: `${ARTIFACT_LATEST_SUMMARY}` (defaults to `benchmarks/runs/latest_summary.json`)

The contract also provides helper functions:
- `artifact_run_dir <RUN_ID>`
- `artifact_summary_path <RUN_ID>`

The `run_benchmarks.sh` helper also accepts an optional second argument for an alternate output directory (legacy compatibility).

## Required `summary.json` schema

`summary.json` includes:

- `generated_at`
- `run_id`
- `summary`
- `benchmarks`

Each item in `benchmarks` must contain at least these fields:

- `name`
- `status`
- `pass`
- `runtime_sec`
- `metric`

Additional metadata fields may also be present.