SlimeCodec NormMAP — measurement ledger and verification record
Development history, measurement procedure and primary sources, split out from the product page (NormH.264). Every number here is checked against the raw benchmark logs, and anything we have not verified is stated as not verified. Last updated: 2026-07-27.
(1) Reduction against the source file (the −74.3% on the product page): how much smaller the output is than the input. This depends heavily on the input bitrate — a bloated master shrinks far more. It maps directly to your bandwidth bill, but it is not a measure of codec efficiency.
(2) Efficiency at equal quality (iso-VMAF BD-rate): the bits needed to reach the same quality, compared against standard x264. This is the codec's actual merit, and every number in the ledger below is of this second kind.
1. Iso-VMAF BD-rate ledger Confirmed
Negative = NormMAP reaches the same quality with fewer bits (a win). BD-rate is a cubic fit over VMAF → log10(rate).
1-1. VOD shipping band (VMAF 93–96), 5 materials — this is what the headline rests on
A dense ladder confined to the quality band you actually ship in. The opponent is x264 at aq-mode 3 with psy-rd 1.00,0.15 (a standard VOD configuration), NormMAP runs at strength 0.10, and both arms go through the same binary at the same preset. These figures are more trustworthy than ones spanning a wide quality range, because they do not carry the chord-interpolation error that afflicts wide-range fits.
| Material | Type | BD-rate (band) | BD-rate (band3, re-run) |
|---|---|---|---|
| Our own ProRes 4K master | real camera, 4K24 | −5.53% | −6.60% |
| crowd_run | 1080p50, high motion | −5.68% | −6.13% |
| park_joy | 1080p50, high motion | −4.32% | −4.97% |
| sintel trailer | 1080p24, CG | −4.08% | −2.69% |
| dinner | 1080p30, interior | −2.61% | −3.62% |
| Median (n = 5) | −4.32% | −4.97% | |
All five materials win, the spread is −2.6 to −6.6%, and the median is −4–5%. Two independent ladder runs land in the same place. This is where the product page's "−4–5% at equal quality" comes from.
But it is one material. The five-material median is −6.7%, which misses the −10% line we had registered in advance. So we do not advertise "over −10% across the board" — the verdict is that the gain depends on the content. That is precisely why the pre-flight check in §2 sits at the centre of the product.
1-2. Low-bitrate band (720p, 3G-class) — independent reproduction, and the gate evidence
A measurement with every condition swapped out: 720p (Lanczos downscale), 0.3–1.5 Mbps, against x264 at aq-mode 1, VMAF built-in HD model vmaf_v0.6.1. Different material, different band, different opponent — and the win lands on the same 4–5%.
| Material | Condition | BD-rate | Iso-VMAF range | Verdict |
|---|---|---|---|---|
| sintel (high g2) | 720p aq1 clean | approx. −5% | 85–94 (watchable) | Win, on material that has headroom at the high-quality end. A conservative figure — a CRF-floor clamp in the harness penalised the Norm arm, so the true value is at least this good |
| sintel (high g2) | 720p 3 s aq1 (separate ladder) | −4.08% | 60–93 | Win. Independent reproduction of the module-1 floor |
| water8 (live-action 4K) | 720p aq1, gate off | approx. +4% | 45–81 | Loss. Downscaling to 720p consumed the headroom |
| water8 (live-action 4K) | 720p aq1, g2 gate on | +0.1% | 40–82 | Break-even. Direct evidence that the gate floors the loss at zero |
| 4k-nature | 4K equal-size control (~8 MB) | Loss | x264 VMAF 70 (8.1 MB) vs Norm VMAF 69 (7.8 MB) | At equal size plain x264 is slightly ahead. There is no Norm-specific equal-quality win on this material |
| park_joy (g2 = 0) | 720p aq1 | −5.0% | 12–50 (below watchable) | Floor case, excluded. Not used for operational judgement |
∴ Exactly one claim is confirmed: with the gate enabled, NormMAP needs 4–5% fewer bits at equal quality when the material has headroom, breaks even when it does not, and does not go negative. The reason we put that number on the banner is that two entirely different setups — the VOD shipping band (§1-1) and the low-bitrate band (§1-2) — land on the same 4–5%.
A claim we disproved: "a flat −10%" does not hold. In both the VOD band and the 3G band the win is around 5% and never crossed the threshold we had registered in advance. Reduction is content-dependent.
_vod_bench/ as band_results.tsv, band3_results.tsv, results.tsv and lever2_results.tsv, and running bdrate_recheck.py re-derives every row above (no encoding and no VMAF run required). The losing rows and the outlying points are still in there — we do not delete them.
2. The g2 pre-flight check — deciding before you encode
NormMAP's gain is content-dependent. We therefore compute a g2 statistic from the input (the proportion of low d-values and of high-d edges) and decide before encoding whether the material is a fit. In ledger terms: sintel (high g2) reads as "will work"; water8 after downscaling reads as "will not".
3. The gate — why a miss does not cost you
Where the g2 check reads "will not work", NormMAP passes the material through unchanged (the same path as plain x264). That is what row 4 of the ledger demonstrates: water8, which cost +4.0% (i.e. grew by 4%) with the gate off, lands at +0.1% with the gate on.
For a purchasing decision the relevant property is the asymmetry — a hit reduces your bill, a miss leaves it where it was — and that single row is its evidence.
4. Why module 2 was frozen Frozen
The second module originally claimed −10.03%. Re-measured for its marginal contribution over module 1 (the floor), the contribution turned out to be zero or negative. Three arms measured under identical intermediate conditions:
| Arm | vs x264 | vs module 1 (marginal) | Verdict |
|---|---|---|---|
| Module 1 floor only | −4.08% | — | This is the real figure |
| 1 + 2, initial settings | −3.56% | +0.50% | Zero marginal contribution (slightly worse) |
| 1 + 2, aggressive settings | +3.98% | +7.83% | Clearly worse. The −10.03% claim did not reproduce |
Module 2 is therefore frozen as an off-by-default switch and removed from the product claims. The code remains, but nothing routes through it by default.
5. VMAF underestimates NormMAP (work in progress)
The −4–5% above is measured entirely with VMAF as the yardstick. This week we established that VMAF structurally understates what NormMAP does:
- VMAF evaluates luma only, by construction. Differences arising in the chroma domain therefore cannot register in the score at all. That is a property of the metric, not a measurement of ours.
- Correction (2026-07-27): this section previously stated "measured on chroma the separation is 5.13×". We withdraw that. The 5.13× is a motion-detection hit rate on a single already-compressed clip (city3s), not a measure of how many bits can be saved at equal quality. The source document itself lists an unresolved confound — the figure may be a side effect of AV1's chroma quantisation — and our own measurement protocol records it as one of that week's errors. It should never have been cited as supporting evidence.
- In our public NormAV1 switching demo, output scoring VMAF 54.57 — numerically "collapsed" — is one that most viewers cannot pick out against the original. VMAF's collapse and visible degradation diverge.
∴ the ledger's −4–5% is a lower bound, and we expect the figure to be larger when measured against viewing quality rather than VMAF. That expectation, however, has not yet been measured — see §6.
5-2. We located why the metrics lose — by measurement (2026-07-28)
§5 said VMAF scores luma only, so differences in chroma cannot reach the score. That is true of VMAF, but it does not explain our case. We measured chroma directly and NormMAP has the larger error there too (below). The cause is not the colour space. It is weighting by area.
On a customer's own 4K60 street footage we compared two files of the same size (NormMAP 5,074,511 B against plain x264 5,486,521 B — NormMAP is 7.5% smaller). Luma MAE, lower is closer to the master:
| Region (reference σ) | NormMAP | plain x264 | difference |
|---|---|---|---|
| white sedan (61.0) | 6.851 | 8.273 | −1.422 |
| dark SUV (49.8) | 7.403 | 7.504 | −0.102 |
| building and people (71.7) | 7.762 | 8.436 | −0.673 |
| empty road (18.7) | 1.626 | 1.241 | +0.385 |
NormMAP is closer to the master on all three detailed regions and loses only on the empty road. That is NormMAP behaving as designed: it accepts more total error in order to move error off the subject.
Score the same two files as whole-frame averages and all ten metrics select plain x264 (luma MAE 4.022 / 3.614, U 1.023 / 0.926, V 1.152 / 1.044, luma PSNR 31.140 / 31.796, U 43.620 / 44.121, V 42.496 / 43.017, VMAF 68.815 / 72.049, grid-locked step 0.2405 / 0.1915). Flat road and sky are most of the area of a 4K city frame, so a small loss over that majority dilutes a large win over a small minority until it disappears. An eye does not weight by area. It goes to the vehicle.
Against us: no whole-frame metric can ever be made to endorse this product by tuning, because it deducts precisely what NormMAP deliberately gave up. The BD-rate figures in §1 rest on a metric of that same family.
For us: losing on a whole-frame metric is not evidence against the product. A comparison offered on PSNR or VMAF grounds is offered on a metric that cannot observe the mechanism in principle.
What should be used instead: a region-weighted endpoint with the regions declared in advance, or paired human judgement. We have neither in publishable form yet. A randomised blind set (8 pairs; material, bitrate, crop position and A/B assignment all drawn, seed recorded) has been generated and not yet judged.
5-3. Peak bitrate (for transmission use, with limits stated)
From packet sizes on the same two files, at equal total size:
| measure | NormMAP | plain x264 | difference |
|---|---|---|---|
| mean | 4,051 kbps | 4,380 kbps | −7.5% |
| frame maximum | 120,202 | 140,703 | −14.6% |
| frame p99 | 11,394 | 13,733 | −17.0% |
| one-second window peak | 6,006 kbps | 6,225 kbps | −3.5% |
| peak-to-mean ratio | 1.48× | 1.42× | slightly worse |
Instantaneous frame peaks fall by 15–17%, but over the one-second sustained window a transmission link is sized on, the advantage is 3.5%. NormMAP shifts the whole rate curve down rather than flattening it — the burstiness ratio is in fact marginally worse. "Peak bandwidth is strongly suppressed" is not supported at the one-second window. Whether frame-level peak reduction helps depends on the decoder buffer, which we have not measured. One clip.
5-4. We disqualified our own automated judge (2026-07-29)
§5-2 concluded that whole-frame metrics cannot observe this mechanism and that paired human judgement is required. As a stopgap until human trials, we had an AI model judge blind pairs. The outcome first: the judge itself was disqualified and both rounds were discarded.
Protocol. Material (7 masters), target bitrate, Norm strength, crop position, frame number and A/B assignment were all drawn at random with the seed recorded. Both arms are matched to the same size (worst mismatch 2.1%). The answer key is sealed and opened only after the verdicts are written to disk.
| Round | Condition | Result |
|---|---|---|
| 1 (8 pairs) | unrestricted draw | the AI chose plain x264 in 7 of 8 |
| 2 (8 pairs) | Norm arm confined to CRF≤20, strength 0.35–0.50 (the corner where we had registered a prediction in advance that Norm would win) | Norm won 2 of 7. The registered prediction (≥5 of 8) failed |
We could have stopped there and reported that Norm lost. We did not, because we checked how the verdicts were reached.
Across the 15 decided pairs the AI chose slot B 13 times. For a fair two-way choice that is two-tailed p = 0.0074. It is not chance.
In round 2 the Norm arm sat in slot A for four pairs, and choosing B in all of them selected plain x264 automatically. Position and arm are confounded, so "judged the image" cannot be separated from "picked the second one shown."
Therefore both rounds are discarded as measurements of which arm is better. Not only round 1 (1 of 8), which went against us, but round 2 as well — the one we could have quoted. Keeping only the favourable half would be selection, not measurement.
Round 1 drew 0.22–0.49 and round 2 drew 0.36–0.50. The product default is 0.15, and the measurements behind the −4–5% in §1 used 0.10 and 0.20. The ranges barely overlap.
This is a limitation, not an excuse — the reason for discarding both rounds (the judge's position bias) holds regardless of range, and we have no evidence that the range difference explains the outcome. A third round at the shipping range (strength 0.06–0.20) is running under the same protocol and has not been judged. It will be published here whichever way it goes.
What this does not show. It does not show that Norm is worse. It shows that our judge was unusable. Equally, it is not evidence that Norm is better. A bias is unusable in both directions; that is what makes it a bias.
What survives. Sixteen pairs of stimuli are encoded, and the seeds, sealed keys and images are reusable. The design fix is settled: present each pair twice with the slots swapped and reject any judge whose answer flips. This AI judge fails that check. Our publication conditions in §6 (blind A/B with multiple human subjects) gain one more measured reason not to be relaxed.
One observation alongside. Agreement between VMAF and the AI verdicts was 50% in round 1 and 86% in round 2. A metric that moves that far between two rounds of one protocol is not tracking a stable quantity — though given the confound above, neither agreement figure means much on its own.
6. What we have not verified Unverified
For the reasons in §5 we expect −20–25% when quality is judged by eye rather than by VMAF. At present this is a hypothesis, not a measurement. The blind-test stimuli (reference plus 8 clips and a sealed key) are prepared, but the assessment has not been run.
The conditions for publishing it are fixed in advance: (1) a blind A/B with multiple subjects, not one; (2) run on a high-g2 material where Norm wins at equal size (4k-nature does not qualify — see §1); (3) a second stage that shrinks the Norm output until the eye detects a difference and compares that point against x264's. When all three hold, the figure and the full procedure go on this page. Independent corroboration with external subjects (a BT.500-style procedure) is the step after that.
- "Half the CDN bill" and "−42%" — withdrawn. The figure came from 4k-nature: "x264 14.3 MB (VMAF 80) vs Norm 8.2 MB (VMAF 69) = −42%". That is a same-CRF comparison — Norm is smaller because its quality is 11 points lower, not because it delivered the same quality for fewer bits. The equal-size control (row 5 of §1) puts x264 slightly ahead. Wherever those figures survive in older material, they are void.
- Compression-history bonus — under investigation. Feeding already-compressed material appears to widen module 1's gain (−1.86 / −2.79 / −3.39% at intermediate CRF 12/24/32), but we cannot yet rule out a geometric artifact, so it is not part of any claim.
- A known defect affects ProRes master input; a fix is pending.
7. Measurement conditions (for reproduction)
| §1-1 VOD shipping band | 1080p24–50 plus 4K24 (our ProRes master), dense ladder confined to VMAF 93–96. Opponent: x264 at aq-mode 3 with psy-rd 1.00,0.15 |
|---|---|
| §1-2 Low-bitrate band | 720p (Lanczos downscale), 0.3–1.5 Mbps ladder. Opponent: x264 at aq-mode 1 (same aq as the NormMAP arm) |
| NormMAP setting | strength 0.10 (§1-1 also reports strength 0.20) |
| Quality metric | VMAF: built-in HD model vmaf_v0.6.1 for 1080p material, the 4K model for 4K material |
| BD-rate | Cubic fit over VMAF → log10(rate) |
| Arm parity | Both arms run through the same binary at the same preset |
| Aggregation rule | Read per material; do not collapse to a median. The band below VMAF ~50 is not used for operational judgement. Wide-range BD-rate carries a chord-interpolation error, so the banner uses the shipping-band figures (§1-1) |
| Recomputation | _vod_bench/bdrate_recheck.py (re-derives every row from the frozen ladders) |
| Software versions | ffmpeg 6.1.1 / libvmaf 42276bc |
| Build configuration | Trial tier (1 thread, preset veryslow, TRIAL watermark on) — please read the note below |
Every measurement above was made with a Trial-tier build (1 thread, preset veryslow, TRIAL watermark on). The paid shipping configuration is 4–8 threads, preset medium, no watermark — a different thing. What we know about the direction of each difference:
- The watermark is burned into both arms, x264 and NormMAP alike. A common fixed bit cost W on both arms pulls the ratio toward 1, so the measured difference necessarily shrinks. The published −4–5% is therefore biased toward zero, and the true value without the watermark lies on the larger side.
- The direction of preset veryslow → medium is unknown. We predict NormMAP gains more at medium, on the reasoning that veryslow's mb-tree and mode decisions pre-empt some of NormMAP's work — but that is a prediction, not a measurement.
These figures come from a pre-registered measurement protocol. They shift when the material or the configuration (binary build, preset, thread count, watermark) changes. We recommend verifying on your own footage — free trial.