Skip to content

Compression levels: Light, Medium, High and Max - #1009

Merged
bonninr merged 5 commits into
devfrom
feature/light-compression
Nov 9, 2026
Merged

bonninr merged 5 commits into
devfrom
feature/light-compression

Conversation

@bonninr

@bonninr bonninr commented Nov 9, 2026

Copy link
Copy Markdown
Owner

Follows #447.

Four levels of lossless compression in memory, chosen beside "Lossless compression in memory" in Organ settings > Loading.

Level Tools Ratio Decode, ns a frame
Light packed blocks, fixed predictor of order 0-2 4.78x 1.56
Medium split Rice, fixed predictor of order 3, vectorised width check 5.60x 2.87
High Medium + a cycle stage: the sample one pipe period back, lag in half frames 7.30x 2.9
Max High + a linear predictor of 8, 16 or 32 taps per block 8.64x 5.98

Figures: 41 Lemmer near samples held at 16 bits, AVX2, decode and store, against 4.5 ns a frame for 0.9.0's coding. Encoding at Max takes 3.1 s for the set, High 2.1 s.

  • High and Max apply to samples held at 16 bits. A sample held deeper is coded at Medium, and the menu greys the two items unless the resident format is 16-bit.
  • 24-bit samples: Light 1.90x, Medium 2.03x.
  • Settings: .mporgan key compress 1 Medium, 2 Light, 3 High, 4 Max. The sample cache tag carries the level. Command line: --compress-samples light|medium|high|max.
  • Max format: predictor byte bit 7, order in bits 0-5, then the shift, the int16 coefficients and 32 frames of int16 history. The encoder uses it only where the block and its history fit 16 bits. The decoder runs eight values at a time: the older values by multiply-add, the two newest runs held in registers, both channels interleaved.

Tests

  • functional.codec.split-rice-max: Max on resonant noise beats High by more than 10%, decodes exactly with every kernel, refuses damaged headers, and codes a full-scale opposite-channel stereo pair exactly.
  • Every level in the kernels test, the High tests, and test_coded_samples.
  • Fast loop 224 run, 0 failed; Windows mp_tests --no-perf 518 run, 0 failed.
  • In the program: the demo organ loaded at 16-bit Max (log: 976 samples held compressed, 32 MB, compressed=max).
  • In the program: the GrandOrgue demo at 16-bit, every stop drawn, short.mid played and the app's own output recorded. Plain, High and Max takes are bit-identical, 914,400 frames each. Resident memory: plain 85.8 MB, High 36.0 MB, Max 32.8 MB.

Screenshots

Resident format "As recorded": High and Maximum greyed.

Compression levels at As recorded

Resident format 16-bit: all four levels.

Compression levels at 16-bit

A second block kind in the split Rice stream: the residuals of a predictor
of order 0 to 2 packed at a fixed width per run of 64, with no unary part.
Chosen per organ as compress 2, --compress-samples light, or the level box
beside the compression setting. The width check after each block runs four
values at a time, which makes every coded block cheaper to decode.
The level replaces the light on/off flag through the library, the
processor, the organ settings file (compress 1 medium, 2 light, 3 high,
4 max), the session choice, the sample cache tag (|z1 to |z4, medium
keeps |z1) and --compress-samples (light, medium, high, max; on stays
medium). Settings lists the four levels; High and Max are offered for
16-bit storage, and a 24-bit sample asked for at either is coded medium.

The encoder tries all four stereo modes the decoder reads: left and
right, left and side, side and right, mid and side.

AVX2 kernels for the packed unpack, the prefix sums and the width check,
and a vector store of decoded values into a voice's window.
A channel may hold its residuals less a share of the residual one pipe
period back, from frame lag of the block on, so every block still decodes
alone. The predictor byte carries the gain (2, 3 or 4 quarters) in bits 4
and 5, and a u16 lag follows; a decoder without the stage refuses the
block. The encoder finds the period once per sample from the
autocorrelation of its middle, and tries a frame either side of it with
each gain and predictor order, ranked by an estimate.

On 41 samples of a 24-bit set held at 16 bits: 7.07x against the
recording, against 5.60x for Medium, decoded at the same speed.
Bit 6 of the predictor byte takes the point between two residuals one
period back, ((e[i - lag] + e[i - lag - 1]) * g >> 3), for a pipe whose
period falls between whole frames. The encoder tries the lags in half
frames.

On 41 samples of a 24-bit set held at 16 bits: 7.30x against the
recording (7.07x with whole frames only), decoded at Medium's speed.
A channel may predict each value from the 8, 16 or 32 before it, with
coefficients fitted to the block by least squares and held at 16 bits
with a shift. Bit 7 of the predictor byte marks it. The decoder takes
what the values before a run of eight give as a vector dot product and
the rest one value after another, both channels side by side.

On 41 samples of a 24-bit set held at 16 bits: 8.64x against the
recording (High 7.30x), 5.98 ns a frame to decode (Medium 2.87).
@bonninr
bonninr merged commit 2bec303 into dev Nov 9, 2026
11 checks passed
@bonninr
bonninr deleted the feature/light-compression branch November 9, 2026 13:56
bonninr added a commit that referenced this pull request Nov 10, 2026
The ARM builds get 75 minutes for the profile training step under emulation; every other build keeps 40.

For #1009
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants