Access binning information

[lmoureaux]

For corrections based on bins (e.g. muon/egamma scale factors, jet energy scale, ...), precision analyses need to decorrelate the statistical uncertainty component between bins. However, a single stat variation of the correction is typically provided, merging all bins together.

Since correctionlib doesn't provide binning information, the bin boundaries currently have to be hard-coded separately by the analyses. This is very likely to go out-of-sync.

A possible solution in the current framework is to ask for stat_bin1... stat_bin50 variations in the JSON, but I think it's not realistic to ask all the POGs to do this for the benefit of a handful of analyses.

It would therefore be great if we could inspect corrections to retrieve a binning or bin index.

[nsmith-]

Is the programmatic access to the json via the schema not viable? You can alway load e.g. with

import gzip
import correctionlib.schemav2 as cs

cs.IGNORE_FLOAT_INF = True

with gzip.open("correction.json.gz", "rb") as fin:
    cset = cs.CorrectionSet.model_validate_json(fin.read())


for corr in cset.corrections:
    if isinstance(corr.data, cs.Binning):
        print(f"Correction '{corr.name}' is a binning with edges: {corr.data.edges}")
    elif isinstance(corr.data, cs.MultiBinning):
        print(f"Correction '{corr.name}' is a multi-binning with axes:")
        for axis, edges in zip(corr.data.inputs, corr.data.edges):
            print(f"  - axis {axis} with edges: {edges}")

That said, depending on how the POG set up the correction, the binning nodes may be further down in the tree. I think this starts to get into the general topic of encoding systematics, and how they related to https://cms-analysis.docs.cern.ch/guidelines/uncertainty_digest/. This relates also to #4.

One option may be to improve the input stats as shown by the rich printout:

The same walk algorithm that determines the range could also record any edges:

correctionlib/src/correctionlib/schemav2.py

Lines 256 to 266 in 5b54e55

	def summarize(
	self, nodecount: dict[str, int], inputstats: dict[str, _SummaryInfo]
	) -> None:
	nodecount["Binning"] += 1
	inputstats[self.input].overflow &= self.flow != "error"
	low = float(self.edges[0]) if isinstance(self.edges, list) else self.edges.low
	high = (
	float(self.edges[-1]) if isinstance(self.edges, list) else self.edges.high
	)
	inputstats[self.input].min = min(inputstats[self.input].min, low)
	inputstats[self.input].max = max(inputstats[self.input].max, high)

Transforms prevent us from inspecting these edges, however.

[lmoureaux]

This sounds useful, but I'm using the C++ API.

[lmoureaux]

So my basic idea was to assign an index to each value that can be returned from a correction, and add a way to query this index. This means that in a 10-bins correction with 5 categories one would have 50 indices. Only 10 of them would really matter, but they would at least be unique. This also works trivially with transforms.

*Maybe not for each value, but for each terminal node in the tree (value and formula nodes)

[nsmith-]

You might have the same requirement as in #5 (comment) where they found a workaround by using a default category node to keep the number of nodes under control.