# Sensor sweep data

The article uses the physical model `X = y aᵀ + √v Z`, with Gaussian targets/noise and `‖a‖²=p`. Population squared error is `(w·a−1)²+v‖w‖²`. Targets are not normalized.

## Current Quick overview

Both learned rules now use the same 1,000 racks at each of 153 sensor counts across the four existing presets: 153,000 paired draws. Thick curves are medians; both bands are the 25th–75th percentiles of population fresh error across racks. These are descriptive intervals, not confidence intervals. The ordinary Gaussian label energies remain unnormalized. Within each rack, labels persist across counts while sensor responses and measurement noise refresh; both rules use exactly the same measurements at each count.

`generate-overview.py` generates `../../assets/sensor-overview-data.js` and `overview-manifest.json`. It uses master seed 2026102901, named SHA256/PCG64 streams, and NumPy linear quantiles at full precision. It retains every declared draw with no filtering, replacement, conditioning on label energy, or seed search. All four presets retain their original width ranges and dense counts near interpolation. The optional dashed green curve is the analytic correlator mean conditional on label energy equal to n; it is a different statistic from either sampled median.

```sh
OPENBLAS_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 python3 blog/data/sensor-fixtures/generate-overview.py --raw-output /tmp/sensor-overview-paired.npz
```

Run from the repository root. The optional NumPy archive contains all 153,000 paired scores and stays outside the website. The manifest records its hash, the plotted data hash, the generator hash, and 36 independent full-vector checks: 12 direct square solves against production SVD at interpolation, and 24 SVD checks against production Gram solves above interpolation and at the largest count. The raw array for each preset has shape `(sensor counts, 1000 racks, 2 rules)`, with minimum norm followed by correlator; recomputing q25/q50/q75 along the rack axis reproduces all six plotted arrays.

## Historical reproduction

Use Python 3 and NumPy. Run the scripts in a copy of this directory to leave the supplied artifacts intact.

```sh
python3 generate-base.py
python3 generate.py
```

`base.json` preserves the original 100-rack sweeps and training losses. `generate-base.py` reproduces all 776 printed values with tolerance 1.1e-6 relative + 2e-9 absolute, including the 52-candidate signed-ridge policy. Its Gaussian noise law is rotation invariant, so ±1 response vectors and normalized Gaussian response vectors give the same distribution for these estimators. `base-provenance.json` identifies the lecture archive, revision, hashes and source-specified seeds.

`fixtures.json` preserves the dense interpolation medians/IQR (360 racks per width) and correlator q10/q90 whiskers (1,000 racks per width). `generate.py` records master seed 2026100404, explicit SHA256-named PCG64 streams, grids, NumPy version and hashes in `manifest.json`. Each rack shares unnormalized Gaussian labels across widths; response/noise streams depend on width. All 51,360 draws are retained in `samples.npz`; no filtering, resampling, seed search or winsorizing. Quantiles use linear interpolation and seven significant digits for the final plotted values.

These historical dense/whisker artifacts used separate reproducible samples, and the base arrays preserve the original lecture samples. They are unchanged for archival reproduction; the current Quick overview uses the paired fixture above.

Part 5 now uses separate [dense ridge summaries](RIDGE.md): every integer width through 100, nested racks and continuously tuned analytic risks over the full positive branch and the negative branch before the first Gram pole. The base signed-ridge grid remains historical reproduction data for the original lecture plots.