Repository navigation
CALM Hub seed: the AI governance control requirements and architecture are not valid CALM #3469
Description
Bug Report
Some of the CALM Hub seed data is not valid against any CALM version. CALM Hub does not validate these documents when it seeds them, so the seed succeeds. But the hub serves documents that fail validation against the schema they declare. I found this in #3468 (step 6 of #3240). The documents came from #2392.
| Seed data | Problem |
|---|---|
The 16 control requirements in calm-hub/mongo/controls/ai-governance/ |
Each one has id and summary. control-requirement.json requires control-id, name and description. |
The ai-governance-v2 architecture mcp-api-pipeline, in calm-hub/mongo/init-mongo.js and calm-hub/nitrite/init-nitrite.sh |
4 nodes and 3 relationships have controls as an array. CALM requires an object keyed by control id, whose requirements hold requirement-url. |
control-requirement.json is the same in every CALM version, from the 2025-03 draft to 1.3. So no $schema value makes the control requirements valid. On main they declare the 2025-03 draft; #3468 changes them to 1.3.
Steps to Reproduce:
- Validate the object under
requirement["1-0-0"]incalm-hub/mongo/controls/ai-governance/003-air-op-004-hallucination-and-inaccurate-outputs.jsonagainst its$schema. - Validate the
mcp-api-pipelinearchitecture ininit-nitrite.shagainst its$schema.
Expected Result:
Both are valid.
Actual Result:
- Control requirement:
must have required property 'control-id'andmust have required property 'description'. - Architecture:
/nodes/N/controls must be object(4 nodes) and/relationships/N/controls must be object(3 relationships).
ajv gives the same errors for CALM 1.0 to 1.3 and for the 2025-03 draft.
Environment:
main at b544734. ajv (JSON Schema 2020-12) with the meta-schemas from @finos/calm-schema.
Additional Context:
Possible fix:
- In the control requirements, rename
idtocontrol-idandsummarytodescription. The schema allows the other properties. - In the architecture, change each
controlsarray to the CALM shape, withrequirement-urlset to the CALM Hub control requirement. First check that the CALM Hub UI does not read the array shape. - Add a test that validates each seed document against its
$schema, so that this does not happen again.
Activity
lui01212 commentedon Oct 27, 2026
I'd like to take this. Plan: first a test that validates every seed document against its
$schemawith ajv and the@finos/calm-schemameta-schemas, which fails on main for the 16 ai-governance control requirements and formcp-api-pipeline. Then renameidtocontrol-idandsummarytodescriptionin those 16 files, and change thecontrolsarrays ininit-mongo.jsandinit-nitrite.shto the CALM object shape, withrequirement-urlpointing at each control requirement's hub URL. Before that I'll check that calm-hub-ui doesn't read the array shape. For the test's home I was thinking of a vitest file inshared/, since it already has ajv and calm-schema; happy to put it elsewhere if you prefer. I'd base it on #3513 if that lands first. Does that scope work, and okay to assign me?rocketstack-matt commentedon Oct 27, 2026
Thanks for volunteering @lui01212 - assigned. Please make sure you have signed the CLA for us to be able to accept your contribution.
fix(calm-hub): make the AI governance seed documents valid CALM