# Sample thresholds for the physical sensor model

`thresholds.json` is copied without numerical edits from the talk's archived
frame data, which is not published. SHA-256:
`cb3fe5c2e1c70307a42763c7f47f23147def832f8e40e9341070bb92311ace49`.

The physical model is `X = y aᵀ + sqrt(variance) Z`, with independent standard
Gaussian labels and noise, `||a||² = p = 50000`, and `variance = 500`. Predictions
target the latent quantity `Y`; measurement noise along the response direction
still leaves uncertainty about that target.
Population test error is `(aᵀw−1)² + variance ||w||²`. Training error is divided
by the realized zero-predictor error, `||Xw−y||² / ||y||²`.

The best-in-span predictor is an oracle benchmark that uses the true response
to choose the risk-minimizing predictor for analysis; the interpolator is the minimum-norm exact fit.
They are computed on the same calibration draw. The corresponding stylized
comparison has `gamma = 1+p/variance = 101`, `d0 = p−1 = 49999`, orthogonal
nuisance rows, and a **clean signal-score target**. Matching the covariance
does not make its target identical to the physical target. The physical model
has the oracle floor `variance/(p+variance) = 0.009900990099…`; the stylized
clean-score target does not.

## Reproduction

The archived `thresholds.json`, its generator `generate-thresholds.py`, the
reference module `sensor-threshold-models.js` and the test named at the end of
this note are not part of the published site; the commands below document how
the archive was produced and checked in the source tree. The live widget's
files in `assets/` (`sample-threshold-models.js`, `sample-threshold-view.js`
and `sample-threshold.css`) are published.

With Python 3 and NumPy:

```sh
python3 blog/data/sensor-fixtures/generate-thresholds.py /tmp/thresholds.json
python3 blog/data/sensor-fixtures/generate-thresholds.py --check
```

The generator uses NumPy's default PCG64 stream, seed **20261006**. It draws
**40,000 independent calibration sets at each of 193 integer sample counts**
(a rounded, deduplicated 240-point logarithmic grid from 1 through 49,999).
Each `opt`, `interp`, or `train` value is a Monte Carlo median; the matching
`Lo` and `Hi` values are the 25th and 75th percentiles. No draws are selected
or removed. The extra `train_at_opt` and `test_at_opt` values use **200,000
additional draws at n=50**. Quantiles use NumPy's default linear interpolation.
The fixture retains all generated double-precision values.
Last-bit differences in matrix algebra may occur across NumPy/BLAS builds;
reproduction should compare numerical values rather than require identical
JSON bytes. The archived fixture's hash above identifies the archived values.

The exact sampler reduces the physical model to two-dimensional Wishart
matrices. The generator's docstring gives the conditional law; `--check`
compares its medians with independently generated explicit sensor matrices.
This is an exact finite-dimensional sampling law, with finite Monte Carlo
error in the reported quantiles.

The stored crossings of test error 0.1 use log-log interpolation between
adjacent saved medians: **49.5225** and **1141.3399**. The talk's counts
**about 50** and **about 1,142** are their ceilings. These are approximate
physical thresholds, not exact integer crossings established by a search
over every sample count. At n=50, the separately simulated median training
error of the best-in-span predictor is **80.7059**.

The stylized formulas give exact integer thresholds **45** and **1,088**;
its best-in-span training error at n=45 is **96.4744**. Its curves are
calculated directly in `sensor-threshold-models.js`.

## Archived marginal sampler

The independent reference module `sensor-threshold-models.js` uses a seeded Gaussian/chi-square sampler with the
same exact Wishart reduction, expressed through triangular factors for
numerical stability. A fresh draw samples one calibration set **at the
currently selected n**. Changing n uses its correct marginal law, not a
claim of adding rows to the same fixed calibration set. Fresh markers must
not be joined into an apparent nested-data trajectory. The model returns an
`overflow` flag for nonfinite arithmetic rather than presenting it as a
finite clipped score.

## Shared live widget

Both Core and Sensors now use `assets/sample-threshold-models.js`,
`assets/sample-threshold-view.js`, and `assets/sample-threshold.css`.
The archived fixture above is an independent numerical check, not the plotted
data source. Sensors defaults to physical medians at γ=101, d₀=49999, n=50;
Core retains its stylized reading default. Model buttons preserve parameters
and use fixed plot axes. Saved URLs use `threshold.mode=sensor`, `iid`, or
`stylized` on either page.

The common eigencoordinate representation is x=(√γ s,z), with independent
standard Gaussian s and nuisance coordinates z. The iid clean-score target
is s; the physical target is √(1−1/γ)s+e/√γ, with independent Gaussian e.
This corresponds to p=d₀+1 and σ²=p/(γ−1), with γ=1 handled as the no-signal
limit. Test errors use the population-zero=1 ruler. **Live training error is
raw mean squared error**, not the archived ratio to each draw's label energy.
At n=50 the live estimated training median is about 79.36, versus the archived
normalized median about 80.71; those are different statistics.

In One realization, Gaussian Schur-complement updates give exact sufficient
statistics for prefixes of one growing dataset. The two random modes share
features and clean scores; physical targets add an independent noise stream.
Moving n or γ retains the draw. Fresh draw changes its seed and selects One
realization; changing d₀ rebuilds the experiment using the same seed.

Best-in-span sensor medians and 10–90% bands are exact affine transforms of
the clean-score beta law. Physical interpolation and training summaries use
2,048 deterministic Monte Carlo samples from the exact two-dimensional law.
Their bands contain the middle 80%, consistently with the shared iid widget.
The physical median interpolation crossing is labeled approximate (≈1,141
at target0.1); the larger archived fixture gives ≈1,142. Realization crossings
scan prefixes for the first target hit, allowing interpolation error to rise
again near full rank. The physical oracle floor is 1/γ.

`blog/tests/sample-threshold-sensor-models.test.cjs` checks the shared formulas
against independent dense-matrix predictions, compares nested distributions
with the archived marginal sampler, and verifies old iid seeds are preserved.