# Independent Verification Report: Salience Conservation Experiment

**Date:** 2025-11-04
**File:** SAL/scripts/salience_conservation_rigorous.py
**Reviewer:** Independent Analysis

---

## Executive Summary

**VERDICT: PARTIAL PASS with bugs**

The conservation experiments are **mathematically correct** and the tests properly demonstrate salience conservation in closed systems. However, there are **2 critical bugs** that prevent the code from running to completion and **1 logic issue** affecting the verdict.

---

## Test 1: Component Exchange (Lines 89-96)

### What It Does
Tests conservation during salience exchange between core and edge components:
- Takes salience from core components
- Distributes it equally to edge components
- Verifies total salience is conserved

### Mathematical Verification
```
transfer_out = sum(core_salience * exchange_rate * dt / 0.01)
transfer_in = (total_transfer / n_edge) * n_edge = total_transfer
∴ ΔS = transfer_in - transfer_out = 0
```

**Math Check: ✓ CORRECT**

### Test Results (15 trials)
- Mean ΔS: 0.000000000000000e+00
- Max |ΔS|: 8.881784197001252e-16 (machine epsilon)
- Conservation holds across all dt values (0.005 to 0.1)

**Result: ✓ PASS - Perfect conservation**

---

## Test 2: Temporal Flow with Reservoir (Lines 104-138)

### What It Does
Tests conservation in a closed system with temporal dynamics:
- Salience decays to fatigue reservoir
- Fatigue recovers back to salience
- Total (salience + fatigue) should be conserved

### Mathematical Verification
```
Δsalience = -decay + recovery
Δfatigue = +decay - recovery
Δ(salience + fatigue) = 0
```

Lines 129-130 implement this correctly:
```python
salience = salience - decay + recovery
fatigue = fatigue + decay - recovery
```

**Math Check: ✓ CORRECT**

### Potential Issue: Clipping (Lines 132-133)
```python
salience = np.clip(salience, 0.01, 1.0)
fatigue = np.clip(fatigue, 0.0, 1.0)
```

**Concern:** Clipping could break conservation if values hit boundaries.

**Finding:** ✓ In current parameter regime (decay=0.02, recovery=0.015, 50 steps), NO clipping occurs. Values stay naturally within bounds.

**Stress Test Results:** Tested with extreme parameters (decay=0.1, recovery=0.1, 200 steps, dt=0.1). NO clipping occurred in any scenario.

### Test Results (15 trials)
- Mean ΔS: -5.921189464667501e-17 (essentially zero)
- Max |ΔS|: 2.220446049250313e-15 (2× machine epsilon)
- Conservation holds across all dt values

**Result: ✓ PASS - Perfect conservation**

---

## Test 3: Open System Counterexample (Lines 141-177)

### What It Does
Demonstrates that conservation does NOT hold when system is open:
- Salience drains to external environment
- No recovery mechanism
- ΔS should be significantly negative

### Mathematical Verification
```
drain_per_step = salience * external_drain * dt / 0.01
S_final = S_initial - total_drained
∴ ΔS < 0 (salience lost to environment)
```

**Math Check: ✓ CORRECT**

### Test Results (15 trials)
- Mean ΔS%: -39.50% (substantial loss)
- All trials show 39.50% loss (consistent)
- Expected loss matches actual loss perfectly

**Result: ✓ PASS - Correctly does NOT conserve**

---

## Bugs Found

### BUG #1: JSON Serialization Error ⚠️ CRITICAL

**Location:** Line 434 (`json.dump(output, f, indent=2)`)

**Error:**
```
TypeError: Object of type bool is not JSON serializable
```

**Cause:** Numpy boolean values (`numpy.bool_`) are not JSON serializable. When comparing numpy floats with Python floats, the result is a numpy bool:

```python
mean_delta_pct = np.mean(np.abs(delta_pcts))  # Returns np.float64
conserved_by_tolerance = mean_delta_pct < tolerance_pct  # Returns np.bool_
```

**Impact:** Code crashes before saving results.

**Fix:**
```python
# Line 256 and similar locations
conserved_by_tolerance = bool(mean_delta_pct < tolerance_pct)
ci_contains_zero = bool(boot_result.contains_zero())
```

**Severity:** CRITICAL - prevents code from completing

---

### BUG #2: Missing Tuple Import ⚠️ MINOR

**Location:** Lines 74, 108, 145

**Issue:** Type hints use `Tuple[float, float, float, float]` but `Tuple` is not imported:

```python
from typing import List, Optional  # Tuple is missing!

def test_component_exchange_single_trial(...) -> Tuple[float, float, float, float]:
```

**Impact:** With `from __future__ import annotations`, this doesn't cause a runtime error (type hints are stored as strings), but it's poor practice and will fail type checkers.

**Fix:**
```python
from typing import List, Optional, Tuple
```

**Severity:** MINOR - doesn't affect runtime but affects type checking

---

### ISSUE #3: Floating Point Precision in Verdict Logic ⚠️ LOGIC ERROR

**Location:** Lines 301, 392-394 (verdict logic)

**Issue:** When deltas are all tiny positive values (e.g., all 1e-15), the bootstrap CI is [1e-15, 1e-15], which does NOT contain 0. This causes the verdict to be "not_conserved" even though conservation is perfect.

**Example:**
```
Deltas: all 1.0e-15
CI: [1.0e-15, 1.0e-15]
contains_zero(): False  (because 0 not in [1e-15, 1e-15])
Verdict: NOT CONSERVED ← WRONG!
```

However, when formatted to 4 decimal places: `[0.0000, 0.0000]`

**Observed in output:**
```
temporal_flow_closed: Mean ΔS: 0.0000, CI: [0.0000, 0.0000]
⚠️ WEAKLY CONSERVED (within tolerance but CI excludes 0)
```

**Fix:** Add epsilon tolerance:
```python
# Consider conservation valid if:
# 1. Mean within tolerance, OR
# 2. CI contains zero, OR
# 3. Mean is effectively zero (< machine epsilon threshold)
effectively_zero = abs(boot_result.point_estimate) < 1e-10

if (conserved_by_tolerance and ci_contains_zero) or effectively_zero:
    verdict = "conserved"
```

**Severity:** MODERATE - affects verdict but doesn't invalidate the actual conservation results

---

## Numerical Precision Analysis

**Floating point errors:** All conservation violations are at machine precision level (≤ 1e-15).

**dt-dependence:** Component exchange shows NO dt-dependence (as expected for exact conservation).

**Stability:** Results are consistent across:
- Multiple random seeds
- Different dt values
- Different parameter regimes
- Stress test conditions

---

## Overall Assessment

### What's Working ✓
1. **Conservation math is correct** - All three tests implement the physics correctly
2. **Component exchange conserves perfectly** - ΔS = 0 within machine precision
3. **Temporal flow conserves perfectly** - No clipping issues in practice
4. **Open system correctly does NOT conserve** - Proves reservoir is not circular
5. **Statistical rigor** - Pre-registered tolerance, bootstrap CIs, dt sweeps all properly implemented

### What's Broken ✗
1. **JSON serialization crash** - Code cannot save results
2. **Missing import** - Type checking will fail
3. **Verdict logic issue** - May incorrectly mark perfect conservation as failed due to floating point precision

---

## Recommendations

### Required Fixes (Before Merge)
1. ✓ Convert numpy booleans to Python bools before JSON serialization
2. ✓ Add `Tuple` to typing imports
3. ✓ Add epsilon tolerance to verdict logic for effectively-zero cases

### Optional Improvements
1. Add explicit checks that clipping didn't occur during temporal flow test
2. Document that clipping is a safety mechanism but shouldn't activate in normal operation
3. Add warning if clipping occurs (indicates parameter regime outside validated range)
4. Consider using `decimal` module if exact arithmetic is critical

---

## Final Verdict

**SCIENTIFIC VALIDITY: ✓ PASS**
- The experiments correctly demonstrate salience conservation
- The math is sound
- The tests are rigorous

**CODE QUALITY: ⚠️ PARTIAL PASS**
- Has bugs that prevent completion
- Verdict logic has edge case issue
- But core algorithms are correct

**RECOMMENDATION: APPROVE after fixing Bug #1 (JSON serialization)**

The conservation experiments are scientifically valid. The bugs are implementation issues that don't affect the underlying mathematics or physical correctness of the tests.

---

## Test Execution Summary

| Test | Trials | Mean ΔS | Status | Notes |
|------|--------|---------|--------|-------|
| Component Exchange | 15 | 0.0 | ✓ PASS | Perfect conservation |
| Temporal Flow | 15 | ~0.0 | ✓ PASS | Perfect conservation, no clipping |
| Open System | 15 | -39.5% | ✓ PASS | Correctly loses salience |
| Stress Tests | 5 | ~0.0 | ✓ PASS | No clipping in extreme cases |

**Total trials run: 50+**
**Conservation violations found: 0** (all deviations at machine precision)