# Physical sensor story fixtures

`story.json` is the canonical sensor walkthrough archive. Ordinary reading
loads only `story-opening.json`: the three minimum-norm fresh-error medians
and middle-half bands through 50,000 sensors, one shared fresh-oracle curve,
and the catalog of available presets. Every plotted count and stored value
is retained at full precision. Per-range bounds reserve room for the hidden
layers without downloading their curves.

Complete summaries load from `story-presets/<preset>.json` when another rule,
training scores, or another setting is selected; the five saved paths load
separately from `story-racks/<preset>.json` for individual-rack comparisons.
The opening therefore does not wait for the 5 MB archive.

`hero.json` is a compact subset for the SGD lab and the saved hero's SGD
endpoints. `story-mechanism.json` supplies the mechanism vectors only when that
drawer opens. All these views retain the canonical values without rounding or
resampling. The old ridge fixtures, preset IDs and output schema are unchanged.

The published site ships only the files the browser reads: these loading
subsets, `mirror-gaussian.json` and `mirror.json` (described below). The canonical `story.json`
archive and the scripts and tests named in the commands below are not part of
the published site; the commands document how the shipped files were produced
and checked in the source tree.

Generate the canonical archive and all its loading subsets with:

```sh
OPENBLAS_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 python3 blog/data/sensor-fixtures/generate-story.py
python3 blog/data/sensor-fixtures/split_story.py
```

Recreate the loading subsets from an existing archive, without NumPy or any
new sampling, with:

```sh
python3 blog/data/sensor-fixtures/split_story.py
```

The split views carry the canonical archive's data checksum. The opening
retains the visible median, lower-quartile and upper-quartile values at every
count. Identical fresh-oracle arrays share one copy. The complete summaries,
saved paths and mechanism are exact copies of their canonical objects. The browser
shares concurrent requests, caches successful pieces, retries failed pieces,
and keeps the currently displayed comparison until a requested view is ready.

The version is `sensor-story-v1`; master seed is `2026100501`. Each rack and
stream gets the first eight little-endian bytes of SHA256 of
`sensor-story-v1:master:preset:index:stream`, interpreted as a NumPy PCG64 seed.
Labels, sensor noise, orders, fresh noise and fresh labels have separate
streams. Adding draws or changing the noise block size preserves the existing
racks. The generator retains every declared draw without conditioning label
energy, rejecting outliers, replacing racks or searching for favorable seeds.
The JavaScript sampler has its own `physical-js-v1-...` identities and uses the
existing SGD iid sampler; equal numeric seeds do not make a JS draw a Python
fixture.

The physical model is `X = y aᵀ + sqrt(v) Z`, with iid standard-Gaussian `y,Z`,
`a=(1,...,1)` and `||a||²=p`. Training and fresh labels are the physical target,
not the clean first coordinate of the existing SGD iid model. All sensor
prefixes within a rack retain their labels and earlier sensor columns.

The nine added presets are `n1s025`, `n1s1`, `n1s4`, `n5s1`, `n5s25`, `n20s1`,
`n25s25`, `n100s25`, and `n100s500`. The first seven retain 300 independent
racks each; the last two retain 100. The width grid contains every integer
1–200, then 97 logarithmically spaced widths through 50,000, including
1,000, 3,000, 10,000 and 50,000. Public data contains only quantiles and five
fixed realization paths, plus the two canonical objects.

For `w=Xᵀα`, the shared scores are:

```text
G = X Xᵀ; b = X a; q = yᵀy
test = (bᵀα − 1)² + v αᵀGα
train = ||Gα − y||² / q
trainRaw = ||Gα − y||² / n
```

The correlator is `α=y/(p q)`. Minimum-norm least squares uses the actual thin
SVD when `p<=n`, reusing Part 5's physical-risk helper; wider racks use the
equivalent Cholesky/dual calculation. Training error is exactly zero at and
above interpolation. There is no hidden ridge shift. Oracle weights are
`a/(p+v)`, outside the sample span in general, and are scored directly.

Each new ensemble rule stores median, quartiles and 10/90 quantiles for test,
normalized training and raw training error. Training is normalized **per draw
before aggregation**. Oracle test error and its normalized-training median
are exactly `t=v/(p+v)`. The normalized residual and label vectors have an
exchangeable joint Gaussian law, giving this median even for one example.
Realized oracle training scores and all training bands remain sampled;
individual oracle points need not lie on the train=test diagonal. Its
normalized-training mean is `t²+n/(n−2)t(1−t)` for `n>2` and is infinite for
`n<=2`; the displayed summaries are medians.

`story-v1-n100s500-rack-0` is the hero. It exports full-precision labels, final
Gram matrix, response projection, label energy, and three independently seeded
canonical epoch orders. `story-v1-n1s1-rack-0` supplies the single-example
mechanism, including actual calibration and independent fresh noise through
10,000 sensors, a fresh physical target, and running products/squares.
`hero.json` also carries the hero's prefix score curves, matching realization
zero in the full story file.

The first complete generation took 29.69 seconds with peak resident memory
133.7 MiB on this host. Noise is streamed in blocks of at most 1,024 sensors;
no full 50,000 × racks × n archive is allocated or saved. `story.json` is about
5.05 MB and `hero.json` about 242 KB before compression. Provenance contains
the actual generation time/memory, source/helper SHA256 values, numerical
tolerance and data checksums. Canonical arrays and plotted values retain
float64 precision.

Actual hero scores differ from ensemble medians:

| Rule | Hero fresh error | Hero normalized training error |
|---|---:|---:|
| Minimum-norm interpolator | 0.62757975 | 0 |
| Correlator | 0.04758639 | 14.14443858 |
| Oracle | 0.00990099 | 0.00689275 |
| Best predictor in the span | 0.04561842 | 12.62491231 |

The physical learning-rate coordinate is `c=η p(1+v)`. Canonical-order SGD has:

| c | First-pass fresh / training | Third-pass fresh / training |
|---:|---:|---:|
| 0.5 | 0.770164 / 0.171093 | 0.650961 / 0.005232 |
| 1 | 0.593156 / 0.016795 | 0.627180 / 0.000002373 |
| 2 | 0.353317 / 0.999387 | 0.368919 / 0.995182 |
| 10 | 0.071607 / 21.869828 | 625.162605 / 188445.251949 |
| 30 | 0.758886 / 268.409427 | 46072021.297805 / 16508277188.627089 |

The new model's “best first pass” searches the selected draw over `c∈[0.1,30]`
using a logarithmic basin scan and local refinement. It is a population-scored
comparison in this declared range, not a rate chosen from observed training
loss. Saved default orders are unchanged across rate comparisons; a new order
seed changes later passes while retaining the canonical first pass.

Verification:

```sh
node --test blog/tests/sgd-sensor-model.test.cjs
OPENBLAS_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 python3 blog/tests/sensor-story-numerics.test.py
```

Checks compare dual SGD with independent explicit-feature updates, actual
physical benchmark predictors, low-rank least squares, fresh-draw Gaussian
moments, raw pre-update timing, seed/order identity, canonical regeneration
with a different block size, exact oracle medians, stored mechanism vectors,
checksums, and hero endpoints against independently implemented NumPy updates.

## Physical mirror of the saved hero

`mirror.json` is a compact view of that same hero, used by the optional
100-object mirror in `why-the-span.html`. `generate-mirror.py` reconstructs
all 50,000 sensor columns from the hero's saved PCG64 noise seed and labels,
verifies its full Gram matrix and response projection against `hero.json`,
then computes `a_mirror = 2 Xᵀ (XXᵀ)⁻¹ X a − a`.
It checks both response norms and each object's full residual noise energy
before publishing the first 24 coordinates. No draw or coordinate search is
used. The resulting response norm squared is 50,000 and the cosine between
the two responses is approximately −0.581581.

The mirror drawer opens on a second rack, `mirror-gaussian.json`, so that the
mirrored response looks like the same kind of draw as the original.
`generate-mirror-gaussian.py` keeps the hero's labels and reconstructed noise
`Z`, draws `g ~ N(0, I)` in all 50,000 coordinates from PCG64 seed `20261019`,
and sets `a = g sqrt(50,000)/||g||`, so `||a||² = 50,000` exactly. It forms
`X = y aᵀ + sqrt(500) Z`, verifies the saved rack's Gram matrix from the same
noise, and applies the same reflection and checks. Its cosine between the two
responses is approximately −0.575085. The drawer's “structured original”
button loads `mirror.json`, the saved hero with equal responses.

```sh
python3 blog/data/sensor-fixtures/generate-mirror.py
python3 blog/data/sensor-fixtures/generate-mirror-gaussian.py
node --test blog/tests/sensor-mirror.test.cjs blog/tests/span-disclosures.test.cjs
```

Each 61 KB mirror fixture loads only when it is first needed: the Gaussian
rack when the drawer opens, the structured rack when its button is pressed. The browser
computes selected-row noise energy and the likelihood ratio from Gram
diagonals, response projections, response norms and the unchanged labels;
the display retains the original readings while its hidden decomposition
changes. All-record likelihood is conditional on the known reference values.