Skip to content

feat(seed): expand industrial process catalogue with per-process subcategories - #609

Open
mrivas00 wants to merge 7 commits into
mainfrom
feat/mati/industrial-processes-per-process
Open

feat(seed): expand industrial process catalogue with per-process subcategories#609
mrivas00 wants to merge 7 commits into
mainfrom
feat/mati/industrial-processes-per-process

Conversation

@mrivas00

@mrivas00 mrivas00 commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Context

Two reviewers flagged the same gap in Alcance 1: the industrial-process catalogue only covers cement, glass, steel and zinc, so a company running any other process has nowhere to declare it.

Gabriela: si la empresa realiza procesos industriales distintos a acero, fundición de metales, cemento o vidrio (por ejemplo industria química), no hay dónde incorporarlo.

Gianluca: en emisiones directas, ¿no es muy restrictivo solo acero, cinc y vidrio?

They are right, and the standards say why. The current set of four is an arbitrary subset of the IPCC IPPU tree: it includes glass and zinc, which the GHG Protocol's own sector table does not list, and omits lime, aluminium, chemicals and pulp & paper, which it does (GHG Protocol Corporate Standard, Appendix D "Industry Sectors and Scopes", pp. 92–93).

Nothing here needs code. The catalogue is seed data plus the maintainer screens, and no migration has ever touched methodology content — the one data migration in this repo, 20260819120000_unify_custom_factors, relabels a source string on captured inventory lines.

Two options, and why there are two PRs

Neither standard requires a subcategory per material. The GHG Protocol's own Alcance 1 taxonomy has a single "process emissions" bucket (ch. 6, p. 41), and subdividing by source type is explicitly optional information (ch. 9, p. 63). The IPCC, meanwhile, structures IPPU as a two-level hierarchy — branch (2A mineral, 2B chemical, 2C metal, 2H other) then leaf (2A1 cement, 2A2 lime, …), with an explicit Other (please specify) leaf on every branch (2006 GL Vol. 3, Fig. 1.1, p. 1.5).

So the real question is only where the boundary between subcategory and dimension falls. Both options cover the same IPPU leaves.

Option A — IPPU branches Option B — one subcategory per process (this PR)
Branch feat/mati/industrial-processes-ippu-branches feat/mati/industrial-processes-per-process
Subcategories in Alcance 1 11 → 11 11 → 18
…of which industrial 4 → 4 4 → 11
Shape subcategory = branch, ProcesoMaterial o método cascade subcategory = process, one flat dimension
Existing four subcategories replaced untouched
Dimension values / seeded IPCC factors 79 / 41 66 / 33
Adding a process later a dimension value in the maintainer UI, no release new subcategory + explanation + release
Reporting granularity per branch (2A / 2B / 2C / 2H) per process
Reduction-plan initiatives per branch — cement and glass share a bucket per process, exact
Explanation guides 4 11
Installed deployments the four existing subcategories need rework via the maintainer screens purely additive

The costs are real on both sides and neither is free:

  • A collapses the per-process breakdown in reports (carbon_inventory_subtotals_view aggregates by subcategory) and blunts initiative targeting, because getSuggestedReductionPlan filters only by subcategoryId. ReductionPlanInitiative already has dimensionValue1Id / dimensionValue2Id columns, so scoping initiatives to the process later is a query change, not a schema change.
  • B keeps that granularity but grows Alcance 1 to 18 subcategories, 11 of them industrial. useSubcategoryPreselectionData lists every subcategory of the methodology (recommended ones merely sort first), so every user — a bakery, a law firm — scrolls past all of them.

Recommendation: A, and this PR is the alternative. A matches the IPCC hierarchy, keeps the subcategory count flat, and turns "there is nowhere to put my process" into a maintainer-screen edit rather than a recurring release. This PR is the right call if the priority is not touching data that already exists in RD/demo — it is purely additive, so no existing line, factor or initiative changes shape.

Worth naming its long-term cost honestly: it leaves mixed levels of the IPPU tree side by side as siblings. Cemento (leaf 2A1) sits next to Química (branch 2B) and Cerámica y otros carbonatos (leaf 2A4) — a distinction that is invisible to a user and permanent once inventories reference it.

These two PRs are mutually exclusive — merge one and close the other. Option A: #608

What this PR does

Keeps Cemento, Vidrio, Acero and Cinc exactly as they are and adds seven subcategories:

Subcategory IPPU Dimension Factors
Procesos industriales - Cal 2A2 Tipo de cal 3
Procesos industriales - Aluminio 2C3 Tecnología de celda 2
Procesos industriales - Química 2B Proceso (16 values) none
Procesos industriales - Ferroaleaciones y otros metales 2C2, 2C4, 2C5 Producto o proceso (15 values) 13
Procesos industriales - Papel y celulosa 2H1 Proceso none
Procesos industriales - Cerámica y otros carbonatos 2A4 Proceso none
Procesos industriales - Otros 2D, 2E, 2G, 2H2 Proceso none
  • Open-ended dimensions end in a bare Otro, matching the convention established on feat/mati/machinery-catalog-otro-option (the dimension name supplies the context, and the capture dropdown sorts that exact literal last). Closed taxonomies — lime types, aluminium cell technology — get none, since Otro carries no methodological meaning there.
  • Adds reduction-plan initiatives and one explanation guide per new subcategory.
  • Allowed measurement units on the seven are g, kg and ton, matching every other mass-denominated subcategory in the methodology.

Emission factors

All values are IPCC 2006 Guidelines, Volume 3 (IPPU) Tier 1 defaults, read from the published PDFs, in kg/ton:

Process Source Values
Cal Table 2.4 high-calcium 750 · dolomitic 770 · hydraulic 590
Aluminio Table 4.10 Prebake 1600 · Søderberg 1700
Ferroaleaciones Table 4.5 2500 · 3600 · 4000 · 4800 · 1300 · 1500 · 1400 · 5000 · 1300 / 1600
Plomo generic factors by furnace ISF 590 · DS 250 · secondary 200 · default 520

Química, Papel y celulosa, Cerámica y otros carbonatos and Otros seed no defaults, deliberately. The IPCC values there are either per-gas (N₂O for nitric and adipic acid) or depend on the carbonate content of the specific feedstock, so a single seeded kgCO₂e figure would be misleading. Those require a custom factor, and each explanation walks through deriving it.

Dolomitic lime uses 0.77, the value IPCC Table 2.4 recommends for developing countries, rather than 0.86.

Ferrochromium is offered twice. Table 4.5 lists it as 1.3 (1.6 with sinter plant) — sintering chrome ore releases CO₂ of its own — so seeding only 1.3 understated a plant with its own sinter line by 23%. Split into Ferrocromo (sin planta de sínter) 1300 and Ferrocromo (con planta de sínter) 1600, with the guide explaining which applies. Every other value in this PR was re-read against the published IPCC PDFs and matched its table exactly; the audit of the four pre-existing subcategories, and the four corrections it produced, is in #619

⚠️ Aluminium seeds only the anode-consumption CO₂ factor, which is all Table 4.10 covers. PFC emissions from the anode effect are excluded. The explanation states this and instructs a separate line, because the seeder cannot express a per-gas split — EmissionFactor.gasDetails exists and is validated against the total, but seedEmissionFactors hardcodes it to {}.

Factor normalization moved out

The kg/ton normalization of the four existing subcategories now lives in its own PR cut from main, and this PR is based on that branch: #619

It is a behaviour change — steel and zinc process factors were understated by 1000×, and cement and glass now resolve in tonnes rather than kilograms — and it is the one part of this work that is not purely additive. It also applies regardless of which catalogue shape is chosen, which is why it was split out: it fixes main on its own. Read it first; the reasoning and the impact on existing inventories are spelled out there.

Installed deployments get it through a migration

Purely additive to the seed data, but on a database that already holds the catalogue the seed cannot install it: seedEmissionFactors runs createMany({ skipDuplicates: true }) over every factor, and skipDuplicates deduplicates nothing here — the partial unique index covers (subcategory, dimension_value_1, dimension_value_2, source), one of those dimension columns is NULL on most factors, and Postgres treats NULLs as distinct in a unique index. A reseed would duplicate every existing factor and then abort on its own row-count check, so RD and demo would never receive these subcategories.

20260824130000_add_industrial_process_subcategories installs them: subcategory rows with their explanation guides, allowed units, dimensions and values, the IPCC factors in both kg/ton and kg/kg, and the reduction-plan initiatives. The SQL is generated from tools/seed/src/data/base rather than hand-copied, so it cannot drift from the seed. Guarded by NOT EXISTS throughout, and on an empty database — an integration-test container — the category lookup matches nothing and every statement inserts zero rows.

Verified by replaying it against a seeded database inside a transaction, on top of the correction migration from #619: the resulting 70 factors and 70 dimension values across the 11 industrial subcategories match the base dataset exactly, and applying both migrations twice leaves the same state.

Verification

  • pnpm format:check, pnpm lint, pnpm type-check — all clean.
  • Both datasets parse against the real FullMethodologyDataSchema: 30 subcategories, 11 industrial, 66 dimension values, 33 factors, identical in base and testing.
  • Checked against the constraints that only bite at seed time or in the editor: unique (subcategory, code) and (subcategory, position) on dimensions, unique (dimension, value) on values, unique (subcategory, dv1, dv2, source) on factors, every factor's rate-unit denominator present in the subcategory's allowed units, every parentValue and factor reference resolving, and one explanation file per subcategory.
  • Not run: the seed itself against a live database.

Out of scope, found on the way

  1. The factor seed is not idempotent. emission_factor_unique_subcategory_dims_source covers (subcategory, dimension_value_1, dimension_value_2, source), and one of those dimension columns is NULL on most rows — Postgres treats NULLs as distinct in a unique index, so createMany({ skipDuplicates: true }) deduplicates nothing. Running the seed twice duplicates those factors and then trips its own row-count check. (An earlier revision of this section claimed 25 of 255 factors were being dropped by that index; they are not — the seeded database holds all of them, counted directly.) Adding the rate unit to the index and making the NULL columns non-distinct would fix both.
  2. gasDetails is never seeded. The column supports a per-gas breakdown and is validated against the total, but the seeder writes {}. Populating it would let aluminium carry its CO₂/PFC split and satisfy the per-gas reporting both ISO 14064-1 and the GHG Protocol require.
  3. No new sector recommendations. Chemical subsectors still do not recommend the chemical process subcategory. Deliberately left out so the two PRs differ only in structure; worth a follow-up so the feature is discoverable rather than merely present.
  4. Overlap with refactor/mati/remove-factor-propio-source. That branch relabels the custom factor source to Otro. The explanation guides here say "Factor propio", matching main today — whichever lands second needs a pass over the new guides.

CI

22 of 24 checks pass. Audit and Trivy Image Scan (api) fail on deepmerge-ts 7.1.5 (CVE-2026-40345), a transitive dependency of @prisma/client. Pre-existing on main and unrelated to seed data — already addressed on fix/mati/deepmerge-ts-cve.

@mrivas00 mrivas00 added type: feature new user-facing capability or enhancement area: database packages/database (Prisma schema / migrations) labels Aug 19, 2026
@mrivas00
mrivas00 force-pushed the feat/mati/industrial-processes-per-process branch from 6b6dc8e to 07922f7 Compare August 24, 2026 14:36
@mrivas00
mrivas00 changed the base branch from main to fix/mati/industrial-process-factor-units August 24, 2026 15:14
@mrivas00
mrivas00 force-pushed the feat/mati/industrial-processes-per-process branch 4 times, most recently from 64b2fcd to 1b07c95 Compare August 24, 2026 18:42
Base automatically changed from fix/mati/industrial-process-factor-units to main August 25, 2026 11:46
@mrivas00
mrivas00 force-pushed the feat/mati/industrial-processes-per-process branch from 63bc3a9 to 450f510 Compare August 25, 2026 13:32
…ategories

The Alcance 1 catalogue exposed four industrial processes as sibling
subcategories (Cemento, Vidrio, Acero, Cinc). That set is an arbitrary subset of
the IPCC IPPU tree: it includes glass and zinc, which the GHG Protocol's own
sector table does not list, and omits lime, aluminium, chemicals and pulp &
paper, which it does. A company doing anything else - chemicals being the case
that surfaced this - had nowhere to declare it.

Keeps the four existing subcategories untouched and adds one per missing IPPU
leaf, plus a residual:

  Procesos industriales - Cal                             (2A2)
  Procesos industriales - Aluminio                        (2C3)
  Procesos industriales - Quimica                         (2B)
  Procesos industriales - Ferroaleaciones y otros metales (2C2, 2C4, 2C5)
  Procesos industriales - Papel y celulosa                (2H1)
  Procesos industriales - Ceramica y otros carbonatos     (2A4)
  Procesos industriales - Otros                           (2D, 2E, 2G, 2H2)

Coverage goes from 4 processes to 31. Alcance 1 grows from 11 subcategories to
18, of which 11 are industrial processes; the preselection screen lists every
subcategory of the methodology, so all users scroll past them.

Emission factors are IPCC 2006 Vol.3 Tier 1 defaults in kg/ton: lime Table 2.4,
aluminium Table 4.10, ferroalloys Table 4.5, lead from its generic factor table.
Quimica, Papel y celulosa, Ceramica and Otros seed no defaults on purpose - the
IPCC values there are either per-gas (N2O for nitric and adipic acid) or depend
on the carbonate content of the specific feedstock, so a single seeded kgCO2e
figure would be misleading. Those require "Factor Propio", and each explanation
walks through deriving it.

Aluminium seeds only the anode-consumption CO2 factor, which is what Table 4.10
covers; the explanation states that PFC emissions are excluded and must be
declared in a separate line, since the seed has no way to express a per-gas
breakdown (EmissionFactor.gasDetails exists but the seeder hardcodes it to {}).

Adds reduction-plan initiatives and one explanation guide per new subcategory.
…ries

The open-ended dimensions added for the expanded industrial catalogue named
their residual value after the process ("Otro proceso quimico", "Otro uso de
carbonatos", ...). The repo convention is a bare "Otro": the dimension name
already supplies the context, and the capture dropdown sorts that exact literal
last via EMISSION_FACTOR_DIMENSION_OTHER_VALUE, so a longer name would land
alphabetically in the middle of the catalogue instead of at the end.

Only factor-less values are renamed. Closed taxonomies (lime types, aluminium
cell technology, steel route) keep no escape hatch, and values that own a real
emission factor keep their own name.
The expanded industrial process explanations were written against the old
factor source selector and told users to pick "Factor propio", an option
that no longer exists after the custom factor sources were unified.

Where the note also sits next to the residual "Otro" process value, the
field name is spelled out ("Otro" como "Fuente factor") so the two are not
read as the same selector. Generic prose about using "un factor propio" is
left as-is: it names the concept, not the option.
Table 4.5 of the IPCC 2006 Guidelines, Volume 3, gives ferrochromium as
"1.3 (1.6 with sinter plant)" — the sintering of chrome ore releases CO2 of
its own. Only the lower figure was seeded, so a plant with its own sinter
line understated the process by 23%.

Split into two options, "sin planta de sínter" (1300) and "con planta de
sínter" (1600), with the guide explaining which to pick.
The seven new subcategories allowed only kilograms and tonnes, which made them
the only mass-denominated subcategories in the methodology without grams: every
subcategory that offers kilograms offers grams as well.

Nobody will declare clinker or ferroalloys in grams, but leaving the unit out
buys nothing — the editor resolves factors by exact rate-unit denominator, so
an unused unit costs a dropdown row and no more — while an inconsistent unit
list is a real surprise for whoever edits this catalogue next.
Same treatment the four existing subcategories get in the base branch: Cal,
Aluminio and Ferroaleaciones now seed each IPCC default twice, in kg/ton and
in kg/kg, so a declaration in tonnes and one in kilograms both resolve to the
library factor instead of falling through to a custom one.

The subcategories that deliberately seed no defaults — Química, Papel y
celulosa, Cerámica y otros carbonatos, Otros — are unaffected.
…egories

The catalogue lives in seed data, but the seed cannot carry an addition to an
installed deployment. seedEmissionFactors runs createMany({ skipDuplicates:
true }) over every factor in the dataset, and skipDuplicates cannot deduplicate
these rows: the partial unique index covers (subcategory, dimension_value_1,
dimension_value_2, source), one of those dimension columns is NULL on most
factors, and Postgres treats NULLs as distinct in a unique index. Re-running
the seed duplicates every existing factor and then aborts on its own row-count
check. So RD and demo would never receive these seven subcategories.

Adds them here: the subcategory rows with their explanation guides, allowed
units, dimensions and values, the IPCC factors in both kg/ton and kg/kg, and
the reduction-plan initiatives.

The SQL is generated from tools/seed/src/data/base — methodologies.json,
initiatives.json and explanations/subcategories — so it installs the same
catalogue as the seed rather than a hand-copy that can drift from it.

Purely additive and guarded by NOT EXISTS throughout. On an empty database the
category lookup matches nothing and every statement inserts zero rows, so an
integration-test container is unaffected.

Verified by replaying it against the running seeded database inside a
transaction, on top of the correction migration: the resulting 70 factors and
70 dimension values across the 11 industrial subcategories match the base
dataset exactly, and applying both migrations twice leaves the same state.
@mrivas00
mrivas00 force-pushed the feat/mati/industrial-processes-per-process branch from 450f510 to 4fb0aff Compare August 25, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: database packages/database (Prisma schema / migrations) type: feature new user-facing capability or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant