Skip to content

core datamodel.ts Compat conversion is lossy (carries only activeInitial); confirm whether conveyor/queue markers must reach core + web editor #883

Description

@bpowers

Problem

src/core/datamodel.ts converts between the engine JSON schema and core's own datamodel types, and its variable types carry only activeInitial out of the Rust json::Compat structure (src/simlin-engine/src/json.rs:78).

  • The variable structs declare a single readonly activeInitial: string | undefined (e.g. datamodel.ts:241, 420, 443, 464).
  • The from-JSON paths read only json.compat?.activeInitial (e.g. datamodel.ts:554, 642, 727).
  • The to-JSON paths reconstruct only compat: { activeInitial } (e.g. datamodel.ts:300-301, 601-605, 690-694, 769-773).

So any variable edited through core's datamodel representation already drops non_negative / data_source / is_public / can_be_module_input, and now would additionally drop the conveyor-engine additions conveyor / queue / leakage / spreadflow / overflow. If a conveyor or queue model is ever edited through this path, the conveyor/queue stock markers and the leak/spread/overflow flow markers are stripped.

Why it matters

This may be intentional -- core's datamodel is an abstraction, not a faithful mirror of the engine JSON, and the deployed save paths may round-trip full fidelity through a different (protobuf) path rather than through core's datamodel. But whether that assumption still holds for conveyor/queue models is unverified, and the loss is silent.

The point of filing is to force a decision, not to assume the fix:

  • Does conveyor/queue support need to reach core's datamodel + the web editor (in which case these fields must be round-tripped, mirroring the variable-compat work in PR Pre-deploy hardening: fix silent data-loss + undo regression, pin pnpm, add canary deploy #810)?
  • Or does core deliberately round-trip full fidelity through the protobuf path, so the datamodel-level loss is a non-issue for real save flows (in which case a code comment documenting the safe paths, and the deliberate limitation, is the right resolution)?

Same bug class as #811 (view-element fidelity dropped on view edits) and tech-debt item #58 (TS Dimension type loses parent/mappings/size/mapsTo), both of which are the "core datamodel.ts is lossy vs the engine JSON" family -- but this one is the variable-level Compat fields specifically.

Components affected

  • src/core/datamodel.ts -- variable from/to-JSON conversions and the readonly activeInitial fields (lines ~241/300/420/443/464/554/601/642/690/727/769).
  • Downstream: the web editor, if conveyor/queue models are meant to be editable there.

Suggested resolution

First confirm the intended fidelity contract for core's datamodel (protobuf path vs JSON path). Then either:

  • extend core's variable types + the from/to-JSON conversions to round-trip the full Compat field set (non_negative, data_source, is_public, can_be_module_input, conveyor, queue, leakage, spreadflow, overflow), mirroring PR Pre-deploy hardening: fix silent data-loss + undo regression, pin pnpm, add canary deploy #810's variable-compat approach, with an end-to-end test that survives a datamodel round-trip; or
  • document the deliberate limitation and the safe (non-lossy) save path in a code comment at the conversion site.

Context

Discovered while fixing a json::Compat serialization drop on the conveyor-engine branch. This is a pre-existing gap (the non-activeInitial fields were already dropped before the conveyor work) and is a follow-up, not part of the conveyor-engine branch's regression fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions