# AlphaEvolve Proxy Loop

This fork is the clean room for a faster meta-loop.

It keeps the existing 300-second truth run, but demotes it to a promotion check.
The main search happens in a much faster proxy regime.

## Why This Exists

The current repo is good at one thing:

- fixed-budget trunk comparisons on the TinyStories byte-level proxy

It is bad at another thing:

- rapidly exploring a large action surface of gates, partitions, and routing ideas

The attack surface already exists in [salience_replacement_matrix.md](./salience_replacement_matrix.md).
What was missing was a search loop that treats that matrix as a move space instead of as a handwritten to-do list.

## AlphaEvolve Mapping

This is an AlphaEvolve-style search loop translated into experimental research:

- proposal model: Codex CLI on GPT-5.4 with `high` reasoning
- evaluator: the local trainer plus `score_run.py`
- archive: proxy and truth best-run JSONs plus TSV ledgers
- selector: keep/discard policy from the evaluator, not chat memory

- State:
  - current best proxy checkpoint/config
  - current best truth checkpoint/config
  - experiment history
  - attack-matrix priors
- Action:
  - one isolated intervention from the attack matrix
  - or one explicit pair interaction when the matrix says pairing is justified
- Policy prior:
  - attack-matrix priority order
  - recent local win-rate by part family
  - human idea boosts from `human_ideas.md`
- Fast rollout:
  - `uv run train.py --regime proxy_fast`
- Value estimate:
  - reduced-budget `val_bpb`
  - runtime / steps / VRAM
  - optimizer telemetry such as AdamW and Muon gate statistics
- Promotion:
  - only top proxy keepers get replayed under the 300-second truth regime
- Tree update:
  - winning branches raise the prior for adjacent ideas
  - repeated clean losses suppress the family

## Regimes

There are now two intended regimes in `train.py`:

- `truth`
  - existing 300-second trunk benchmark
  - use this to decide what is actually worth keeping
- `proxy_fast`
  - smaller model and much shorter eval
  - use this to search broadly and cheaply

`proxy_fast` is not the source of truth.
It is a ranking surrogate.

## Recommended Search Discipline

1. Start from the current best truth config.
2. Choose one attack-matrix action.
3. Run it in `proxy_fast`.
4. Log the result to a separate proxy ledger.
5. Promote only the strongest proxy results to `truth`.
6. Update local priors:
   - upweight families with repeated wins
   - downweight families with repeated clean losses

## What Counts As A Good Proxy

A proxy loop is useful if it preserves:

- optimizer family interactions
- routing / gate saturation behavior
- obvious stability failures
- broad sign of direction

It does not need to preserve exact truth-run ranking every time.
It only needs to produce a shortlist that is materially better than random search under the 300-second regime.

## Commands

Smoke:

```powershell
uv run train.py --regime proxy_fast --smoke-test
```

Fast proxy:

```powershell
uv run train.py --regime proxy_fast > proxy.log 2>&1
```

Overnight autonomous proxy search:

```powershell
uv run python overnight_alpha_loop.py --truth-cadence 6
```

Detached overnight launch:

```powershell
powershell -ExecutionPolicy Bypass -File .\start_overnight_alpha_loop.ps1
```

Detached stop:

```powershell
powershell -ExecutionPolicy Bypass -File .\stop_overnight_alpha_loop.ps1
```

Truth run:

```powershell
uv run train.py --regime truth > run.log 2>&1
```

Score against a regime-matched reference:

```powershell
uv run score_run.py proxy.log --reference proxy_best_run.json --mode runtime
uv run score_run.py run.log --reference current_best_run.json --mode trunk
```

## Suggested Proxy Ledger

Keep this separate from `results.tsv`.

Suggested columns:

```text
commit	regime	val_bpb	memory_gb	status	part	class	description
```

The truth ledger should stay conservative.
The proxy ledger should be broad and cheap.

The overnight runner writes:

- `proxy_results.tsv`
- `proxy_search_state.json`
- `proxy_runs/`
- optionally `truth_promotions.tsv`, `truth_override_best.json`, and `truth_runs/` when truth promotion is enabled

## Immediate Search Policy

Use Wave 1 from the attack matrix as the initial action prior:

1. AdamW second-moment rules
2. Group partitioning
3. Value embedding optimizer
4. LM head optimizer
5. Scalar optimizer
6. Muon salience gate tuning

Do not spend the fast loop on broad architecture churn first.
Use the fast loop to map the optimizer/control surface before escalating.

## Paper Seed: OOD Sparsity

Also treat the paper below as an explicit experiment family:

- Mingyu Jin et al., "Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs"
- arXiv: [2603.03415](https://arxiv.org/abs/2603.03415)

What is relevant here:

- harder / more OOD inputs produced sparser last hidden states
- the paper treats sparsity as an adaptive signal, not just an incidental artifact
- they use that signal to drive curriculum decisions

Translate that into this repo as three concrete action families:

1. Instrumentation:
   - measure last-layer hidden-state sparsity in the fast proxy loop
   - correlate it with `val_bpb`, optimizer gates, and failure modes
2. Gating:
   - use hidden-state sparsity as an input to VE gates, residual routing, or optimizer gates
3. Curriculum / weighting:
   - use sparsity as a sequence-level or token-level weight for training order or loss emphasis

This should be treated as a first-class branch of the attack matrix, not as a side note.