Problem
pysimlin hand-maintains a mirror of the Rust json::Compat schema (src/simlin-engine/src/json.rs), and it is out of sync -- both the dataclass and the cattrs structure hook silently drop compat fields on every patch/edit round-trip.
The canonical Rust Compat (src/simlin-engine/src/json.rs:78) now has 10 fields:
active_initial, non_negative, can_be_module_input, is_public, data_source, and (new on the conveyor-engine branch) conveyor, leakage, spreadflow, queue, overflow.
pysimlin declares only 4:
src/pysimlin/simlin/json_types.py:35 -- the Compat dataclass declares only active_initial / non_negative / can_be_module_input / is_public. It is missing data_source (a pre-existing lag) and the five new conveyor / leakage / spreadflow / queue / overflow fields.
src/pysimlin/simlin/json_converter.py:490 -- structure_compat hand-reads only those same four fields, so it actively drops data_source today and would drop conveyor / leakage / spreadflow / queue / overflow.
The module header at src/pysimlin/simlin/json_types.py:2-3 claims "These types match the Rust JSON types in src/simlin-engine/src/json.rs" -- but they do not.
Separately, ArrayedEquation (src/pysimlin/simlin/json_types.py:55) is missing has_except_default, which exists in the Rust struct (json.rs:212) and in the TypeScript mirror (src/engine/src/json-types.ts:145 hasExceptDefault).
Why it matters (correctness -- active data loss)
Any pysimlin edit/patch that round-trips a variable through this converter demotes it:
- A conveyor or queue stock loses its
conveyor / queue marker and is silently demoted to a plain stock.
- A conveyor leakage / spread / queue-overflow flow loses its
leakage / spreadflow / overflow marker.
data_source (external-data reference) is dropped today, before the conveyor work even landed.
This is silent and irreversible on a routine edit. Same bug class as PR #810 / #811 (fidelity fields dropped on edit round-trips), but for pysimlin's Compat mirror rather than core's view/variable path.
Components affected
src/pysimlin/simlin/json_types.py -- the Compat dataclass (line 35) and ArrayedEquation (line 55).
src/pysimlin/simlin/json_converter.py -- structure_compat (line 490) plus the unstructure hooks and per-variable structure paths that build Compat (there are several Compat(...) / conv.structure(..., Compat) call sites around lines 544-704).
Suggested fix
- Extend the
Compat dataclass with data_source, conveyor, leakage, spreadflow, queue, overflow.
- Add
Conveyor / Leakage / SpreadFlow / Queue dataclasses mirroring json.rs (note SpreadFlow is adjacently tagged {type, distribution} in JSON; check the exact tagging against the Rust serde attributes).
- Update the cattrs structure/unstructure hooks (
structure_compat and the Compat-omit-when-default unstructure logic near line 132) to cover the full field set.
- Add
has_except_default to ArrayedEquation.
- Add tests (the pysimlin gate is mypy strict + ruff + pytest); a JSON round-trip test asserting a conveyor/queue stock and leak/spread/overflow flow survive a structure/unstructure cycle would pin the regression.
Context
Discovered while fixing a json::Compat serialization drop on the conveyor-engine branch. The data_source drop is pre-existing (predates the conveyor work), and the five new fields are the conveyor/queue additions; both are follow-ups, not part of the conveyor-engine branch's regression fix. Distinct from #811 (core view-element fidelity) and tech-debt item #58 (TS Dimension type).
Problem
pysimlin hand-maintains a mirror of the Rust
json::Compatschema (src/simlin-engine/src/json.rs), and it is out of sync -- both the dataclass and the cattrs structure hook silently drop compat fields on every patch/edit round-trip.The canonical Rust
Compat(src/simlin-engine/src/json.rs:78) now has 10 fields:active_initial,non_negative,can_be_module_input,is_public,data_source, and (new on theconveyor-enginebranch)conveyor,leakage,spreadflow,queue,overflow.pysimlin declares only 4:
src/pysimlin/simlin/json_types.py:35-- theCompatdataclass declares onlyactive_initial/non_negative/can_be_module_input/is_public. It is missingdata_source(a pre-existing lag) and the five newconveyor/leakage/spreadflow/queue/overflowfields.src/pysimlin/simlin/json_converter.py:490--structure_compathand-reads only those same four fields, so it actively dropsdata_sourcetoday and would dropconveyor/leakage/spreadflow/queue/overflow.The module header at
src/pysimlin/simlin/json_types.py:2-3claims "These types match the Rust JSON types in src/simlin-engine/src/json.rs" -- but they do not.Separately,
ArrayedEquation(src/pysimlin/simlin/json_types.py:55) is missinghas_except_default, which exists in the Rust struct (json.rs:212) and in the TypeScript mirror (src/engine/src/json-types.ts:145hasExceptDefault).Why it matters (correctness -- active data loss)
Any pysimlin edit/patch that round-trips a variable through this converter demotes it:
conveyor/queuemarker and is silently demoted to a plain stock.leakage/spreadflow/overflowmarker.data_source(external-data reference) is dropped today, before the conveyor work even landed.This is silent and irreversible on a routine edit. Same bug class as PR #810 / #811 (fidelity fields dropped on edit round-trips), but for pysimlin's
Compatmirror rather than core's view/variable path.Components affected
src/pysimlin/simlin/json_types.py-- theCompatdataclass (line 35) andArrayedEquation(line 55).src/pysimlin/simlin/json_converter.py--structure_compat(line 490) plus the unstructure hooks and per-variable structure paths that buildCompat(there are severalCompat(...)/conv.structure(..., Compat)call sites around lines 544-704).Suggested fix
Compatdataclass withdata_source,conveyor,leakage,spreadflow,queue,overflow.Conveyor/Leakage/SpreadFlow/Queuedataclasses mirroringjson.rs(noteSpreadFlowis adjacently tagged{type, distribution}in JSON; check the exact tagging against the Rust serde attributes).structure_compatand theCompat-omit-when-default unstructure logic near line 132) to cover the full field set.has_except_defaulttoArrayedEquation.Context
Discovered while fixing a
json::Compatserialization drop on theconveyor-enginebranch. Thedata_sourcedrop is pre-existing (predates the conveyor work), and the five new fields are the conveyor/queue additions; both are follow-ups, not part of the conveyor-engine branch's regression fix. Distinct from #811 (core view-element fidelity) and tech-debt item #58 (TSDimensiontype).