FIX spectral indices to use band values as read, with radiometric scaling - #52
Merged
aradfarahani merged 1 commit intoAug 21, 2026
Merged
Conversation
…ling
All six index calculators obtained their inputs from
FileHandler.get_normalized_bands(), which applies a min-max rescale
independently to each band. Because every band received a different affine
transform, this altered the relationships between bands - and those
relationships are the entire physical content of a band ratio.
Three consequences, all reproducible on the bundled example:
* Published thresholds did not apply. NDVI mean was 0.3351 against 0.5103
for the standard definition, and the fraction above 0.3 read 55.64%
instead of 59.88%.
* A pixel's value depended on how much of the image was loaded, since the
rescale used the loaded extent's own extrema. The same pixel gave 0.8373
on the full scene and 0.8692 on a 400x400 crop.
* Inter-band relationships inverted outright. AFRI's correlation with NDVI
ran +0.71 on the values as stored and -0.69 after rescaling, against a
source paper reporting the two as nearly identical under clear sky.
Add FileHandler.get_bands(), returning the arrays as read, and switch the six
indices to it. Min-max normalization is untouched for the enhancement, HSV,
PCA and SVM modules, where rescaling is appropriate - and for SVM it is
required, since an RBF kernel needs comparable feature scales.
Removing the rescale is necessary but not sufficient: it leaves the indices
computing on digital numbers, and two of them carry constants defined in
reflectance units. SAVI's soil adjustment L = 0.5 and AFRI's 0.66/0.50
coefficients contribute nothing when added to a DN in the thousands, so the
result is silently not the cited index. Normalized differences (NDVI, NDWI,
UI, BSI) are invariant to a common gain and remain valid on raw DN.
Add scale_factor/offset to every index, applied as rho = DN * s + o, with
published values for the common analysis-ready products exposed as
RADIOMETRIC_PRESETS (Landsat C2 L2, Sentinel-2 L2A before and after
processing baseline 04.00). SAVI and AFRI warn when handed values far outside
the reflectance range.
NDVI now reproduces a rasterio-computed reference exactly, and is identical
between a full scene and a crop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #51 — review #50 and #51 first; this PR targets
hotfix/afri-formula.Problem
All six index calculators read from
FileHandler.get_normalized_bands():applied independently to each band. Each band therefore receives a different affine transform, which changes the relationships between bands — and those relationships are the entire physical content of a spectral index.
(NIR' - Red') / (NIR' + Red')where NIR' and Red' were rescaled by different factors is simply not NDVI.Reproduced on
example/data/:The last two are the ones I find most telling. Karnieli et al. report AFRI and NDVI as "almost identical" under clear sky, and bare ground must oppose canopy. Both relationships held on the stored values and inverted under the rescale — the index was not merely rescaled, it was reporting the opposite of the physical truth. Those are the two indices whose formulas #51 just corrected; they only start behaving correctly here.
The extent dependence is the reproducibility failure: loading a crop changed the answer for pixels the crop shared with the full scene.
Changes
1. Indices compute on values as read. New
FileHandler.get_bands()returns the arrays already held inself.bands, mirroring the existing accessor pattern. The six calculators switch to it;self.normalized_bandsbecomesself.source_bands.Min–max normalization is untouched for the enhancement, HSV, PCA and SVM modules, where rescaling is appropriate — and for SVM it is genuinely required, since an RBF kernel needs comparable feature scales.
2. Radiometric scaling, because removing the rescale is not sufficient on its own. It leaves the indices computing on digital numbers, and two of them carry constants defined in reflectance units:
L = 0.5Adding
L = 0.5to a DN in the thousands contributes nothing, so SAVI on unscaled input is silently not SAVI. Every index now acceptsscale_factor/offset, applied asρ = DN × s + o, with published values for the common analysis-ready products shipped asRADIOMETRIC_PRESETS:Covering Landsat C2 L2 (
2.75e-5,−0.2), Sentinel-2 L2A (1e-4,0), and Sentinel-2 L2A baseline ≥ 04.00 (1e-4,−0.1, for theBOA_ADD_OFFSETintroduced in that baseline). SAVI and AFRI emit aUserWarningwhen handed values far outside the reflectance range, so the silent-nonsense case becomes loud.Behaviour change
Index values change for every user — deliberately, and documented prominently in
docs/spectral-indices.md. The previous values were scene-dependent and not the cited indices, so preserving them would preserve the defect. Defaults arescale_factor=1.0, offset=0.0, so input already in reflectance needs nothing passed.Tests
347 pass (334 on #51, 13 new).
tests/tools/spectral_indices/radiometry_test.pycovers:The six existing index test fixtures were updated from
get_normalized_bands/normalized_bandstoget_bands/source_bands.