Preserve operating content across governed revisions

docs/architecture.md

@@ -896,3 +896,11 @@ Approval authorizes an exact proposed revision. Activation and effective dates d
 A first recorded approval does not establish that a channel or control failed before that date. Reporting diagnostics describe missing authorization evidence with an exact Program and interval. When a valid baseline resolves that limitation wholly before selected coverage, validation retains it as `info`, with no current action. A resolved limitation wholly after an earlier examination is retained as subsequent information. Overlapping or unresolved evidence limitations remain actionable. Actual invalid changes, missing approvals and stale proof retain their normal checks.
 
 The selected Audit's agreed range or as-of date takes precedence over a management candidate period. `validateWorkspace` accepts optional `programId`, `auditId` and `coverage` context; shared workflow and packet checks use the same classification. Cached information is restored to its original diagnostic before assessing a different Program or examination interval. Global structural errors remain visible. Validation counts report `information` separately from errors and warnings.
+
+Same-ID revisions follow the same rule. Returning governed content to draft or approving its next revision leaves the last valid committed operating edition in force until its actual cutover. Shared scheduling, readiness, workforce acknowledgments and examination coverage read that edition's own JSON, approval, activation and Markdown from Git. They do not attach current proposed text or expanded scope to an older approval. Actual retirement and invalid implemented revisions stop fallback. Historical queries cannot borrow descendants of their selected commit.
+
+Evidence packets retain current source proposals in the normal record and content export. When a different committed edition still operates at the selected period end, `operatingBaselines` identifies its commit, lifecycle dates and exported files under `operating-baselines/`. Those exact JSON and Markdown files join the packet checksum manifest; the existing history export retains intervening revisions.
+
+A content revision does not restart an unchanged adopted schedule. Its Obligation Rule keeps its anchor, recorded completions and still-open earlier windows. Shared planning and occurrence validation use the first actual relevant governing operation together with the rule's own cutover. Initial content activation and real rule amendments retain their boundaries; this does not waive gaps in period authorization or evidence.
+
+Owner retention applicability uses the same operating schedule edition. Source Coverage, existing applications and new application previews use its exact rule and procedure, while present service and source fingerprints remain current. A proposed period or disposal change does not stale an application before cutover. The browser displays the approved values being applied; actual activation of a changed rule requires reassessing an old application when its binding no longer matches.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "filegrc-monorepo",
-  "version": "0.21.8",
+  "version": "0.21.9",
   "private": true,
   "workspaces": [
     "packages/filegrc",

packages/create-filegrc/package.json

@@ -1,6 +1,6 @@
 {
   "name": "create-filegrc",
-  "version": "0.21.8",
+  "version": "0.21.9",
   "description": "Create a filegrc workspace for a SOC 2 program",
   "license": "MIT",
   "repository": {

packages/create-filegrc/src/index.js

@@ -524,13 +524,13 @@ async function runCombinedSetup(target, input) {
 async function writeMinimalLockfile(target, name, versionRange) {
   const lock = {
     name,
-    version: "0.21.8",
+    version: "0.21.9",
     lockfileVersion: 3,
     requires: true,
     packages: {
       "": {
         name,
-        version: "0.21.8",
+        version: "0.21.9",
         dependencies: { filegrc: versionRange }
       }
     }

packages/create-filegrc/template/package.json

@@ -1,6 +1,6 @@
 {
   "name": "{{project_name}}",
-  "version": "0.21.8",
+  "version": "0.21.9",
   "private": true,
   "description": "filegrc workspace for a SOC 2 program",
   "type": "module",

packages/filegrc/package.json

@@ -1,6 +1,6 @@
 {
   "name": "filegrc",
-  "version": "0.21.8",
+  "version": "0.21.9",
   "description": "Zero-dependency Git-native GRC engine",
   "license": "MIT",
   "repository": {

packages/filegrc/src/audit-preparation.js

@@ -98,7 +98,7 @@ export async function assessAuditPreparation(input, options = {}) {
         await documentsStage(loaded, audit, byId, calendarAsOf),
         evidenceStage(audit, records, byId, loaded.model),
         populationsStage(audit, records, byId, loaded.model),
-        occurrenceContinuityStage(audit, records, loaded.model, programReadiness.asOf, loaded.root)
+        occurrenceContinuityStage(audit, records, loaded.model, programReadiness.asOf, loaded.root, loaded)
       ]
     : [];
   stages.push(fieldworkStage(audit, fieldworkSections, programReadiness));
@@ -956,7 +956,7 @@ function fieldworkStage(audit, sections, programReadiness) {
   );
 }
 
-function occurrenceContinuityStage(audit, records, model, asOf, root) {
+function occurrenceContinuityStage(audit, records, model, asOf, root, loaded) {
   if (audit.auditKind !== "soc-2-type-2") {
     return stage(
       "occurrences",
@@ -991,7 +991,7 @@ function occurrenceContinuityStage(audit, records, model, asOf, root) {
     || record.controlIds.some((id) => selectedControlIds.has(id))
   ));
   const plan = planObligations(periodResources, {
-    root,
+    root, loaded,
     programId: audit.programId,
     from: periodStart,
     asOf: periodThrough,

packages/filegrc/src/cli.js

@@ -450,7 +450,7 @@ export async function runCli(argv = process.argv.slice(2)) {
   }
   if (command === "obligations") {
     const loaded = await loadWorkspace(root);
-    const result = planObligations(loaded.resources, { root: loaded.root,
+    const result = planObligations(loaded.resources, { root: loaded.root, loaded,
       programId: flags.program,
       asOf: flags["as-of"] ?? currentCalendarDate(loaded.workspace.timezone),
       from: flags.from,

packages/filegrc/src/collection-revision.js

@@ -1,16 +1,15 @@
+import { governedMarkdownSource } from "./governed-content-baseline.js";
 import { scheduleSetWithoutExcludedWithdrawals } from "./schedule-set.js";
 import { resourceIndex } from "./resource-index.js";
 import { calculationMemo } from "./calculation-cache.js";
 import { createHash } from "node:crypto";
-import { readFileSync } from "node:fs";
 import { modelSupports } from "../model/index.js";
 import {
   authoritativeSourceRevisionValue,
   collectionRevisionInputs,
   collectionScopeRevisionFacts,
   scopedCollectionRecords
 } from "./collection-scope.js";
-import { resolveDataPath } from "./paths.js";
 import { getFileAtRevision } from "./git.js";
 import { reviewHistoricalWorkspace, reviewHistoryCommits, reviewHistoryContext } from "./historical-workspace.js";
 import { resolveProgram } from "./program.js";
@@ -267,7 +266,12 @@ function calculateCollectionRevisionUncached(loaded, resourceType, options, lega
             ? loaded.historicalFiles
               ? loaded.historicalFiles.get(`data/${path}`) ?? null
               : getFileAtRevision(loaded.root, options.historicalCommit, `data/${path}`)
-            : loaded.historicalFiles ? loaded.historicalFiles.get(`data/${path}`) ?? null : loaded.historicalCommit ? getFileAtRevision(loaded.root, loaded.historicalCommit, `data/${path}`) : readFileSync(resolveDataPath(loaded.root, path), "utf8");
+            : (() => {
+              const entry = loaded.entries?.find((item) => item.record === record);
+              if (entry?.reviewFiles instanceof Map) return entry.reviewFiles.get(`data/${path}`) ?? null;
+              if (entry?.reviewCommit) return getFileAtRevision(loaded.root, entry.reviewCommit, `data/${path}`);
+              return governedMarkdownSource(loaded, record, path);
+            })();
           if (content === null) return [];
           return [{ path, revision: createHash("sha256").update(content).digest("hex") }];
         } catch (error) {

packages/filegrc/src/evidence-packet.js

@@ -1,4 +1,4 @@
-import { governedContentCoversPeriod } from "./governed-content-baseline.js";
+import { governedContentBaseline, governedContentSource, governedMarkdownSource, registerGovernedLookup, governedContentCoversPeriod } from "./governed-content-baseline.js";
 import { createHash } from "node:crypto";
 import { createReadStream } from "node:fs";
 import { copyFile, mkdir, readFile, readdir, readlink, rm, writeFile } from "node:fs/promises";
@@ -69,7 +69,7 @@ export async function prepareEvidencePacket(input, options = {}) {
   if (!validation.ok) throw new Error(`The workspace has ${validation.counts.errors} validation ${validation.counts.errors === 1 ? "error" : "errors"}. Fix them before generating evidence.`);
   const { loaded } = validation;
   const records = loaded.resources;
-  const byId = new Map(records.map((record) => [record.id, record]));
+  const byId = registerGovernedLookup(new Map(records.map((record) => [record.id, record])), loaded);
   const entriesById = new Map(loaded.entries.map((entry) => [entry.record.id, entry]));
   const audit = options.auditId ? byId.get(options.auditId) : null;
   if (options.auditId && audit?.type !== "audit") throw new Error(`Audit "${options.auditId}" was not found.`);
@@ -100,7 +100,7 @@ export async function prepareEvidencePacket(input, options = {}) {
       byId.get(id)?.type === "evidence" ? byId.get(id).sourceResourceIds || [] : []
     ))
   );
-  const plan = planObligations(records, { root: loaded.root,
+  const plan = planObligations(records, { root: loaded.root, loaded,
     programId: program?.id || options.programId || audit?.programId,
     asOf: end,
     from: start,
@@ -344,6 +344,32 @@ export async function prepareEvidencePacket(input, options = {}) {
       };
     })
     .sort((a, b) => a.type.localeCompare(b.type) || a.title.localeCompare(b.title));
+  // Keep editable proposals in the source export and identify the exact
+  // committed edition still operating at the selected coverage end.
+  const operatingBaselines = [...selectedIds].flatMap((id) => {
+    const current = byId.get(id);
+    const baseline = governedContentBaseline(current, byId, end, loaded.model);
+    if (!baseline || baseline === current) return [];
+    const source = governedContentSource(loaded, baseline);
+    if (!source.commit || !source.entry) return [];
+    return [{ resourceId: id, baselineId: baseline.id, type: baseline.type,
+      commit: source.commit, approvedOn: baseline.approvedOn, effectiveOn: baseline.effectiveOn,
+      ...(baseline.activatedOn ? { activatedOn: baseline.activatedOn } : {}),
+      recordPath: `operating-baselines/${id}/${source.commit}/record.json`,
+      relatedRecords: baseline.type === "document" && baseline.documentKind === "schedule"
+        ? source.loaded.resources.filter((record) => record.type === "retention-schedule-item" && record.scheduleDocumentId === baseline.id
+          || record.type === "collection-review" && record.resourceType === "retention-schedule-item"
+            && (record.populationResourceIds || []).some((rowId) => source.loaded.resources.some((row) => row.id === rowId && row.scheduleDocumentId === baseline.id)))
+          .map((record) => ({ id: record.id, type: record.type,
+            recordPath: `operating-baselines/${id}/${source.commit}/records/${record.type}/${record.id}.json`,
+            contentPaths: markdownEntries(source.loaded.model, record)
+              .filter(({ path, required }) => required || governedMarkdownSource(source.loaded, record, path) !== null)
+              .map(({ path }) => ({ sourcePath: path,
+              exportedPath: `operating-baselines/${id}/${source.commit}/content/${path}` })) })) : [],
+      contentPaths: markdownEntries(source.loaded.model, baseline).map(({ path }) => ({
+        sourcePath: path, exportedPath: `operating-baselines/${id}/${source.commit}/content/${path}`
+      })) }];
+  });
   await assertLoadedEntriesCurrent(loaded);
   const dataDigest = await dataTreeDigest(loaded.root);
   await assertLoadedEntriesCurrent(loaded);
@@ -526,7 +552,8 @@ export async function prepareEvidencePacket(input, options = {}) {
     managementPreparation,
     controlCoverage,
     gaps,
-    records: packetRecords
+    records: packetRecords,
+    operatingBaselines
   };
   preparedPacketValidations.set(packet, {
     validation,
@@ -767,7 +794,7 @@ export async function writeEvidencePacket(input, packet, options = {}) {
   if (!validation.ok) throw new Error(`The workspace has ${validation.counts.errors} validation ${validation.counts.errors === 1 ? "error" : "errors"}. Fix them before writing evidence.`);
   await assertPacketSourceState(packet, validation.loaded);
   const entriesById = new Map(validation.loaded.entries.map((entry) => [entry.record.id, entry]));
-  const byId = new Map(validation.loaded.resources.map((record) => [record.id, record]));
+  const byId = registerGovernedLookup(new Map(validation.loaded.resources.map((record) => [record.id, record])), validation.loaded);
   const files = [];
   let outputCreated = false;
   try {
@@ -818,6 +845,27 @@ export async function writeEvidencePacket(input, packet, options = {}) {
         }
       }
     }
+    for (const item of packet.operatingBaselines || []) {
+      const baseline = governedContentBaseline(byId.get(item.resourceId), byId, packet.period.end, validation.loaded.model);
+      const source = governedContentSource(validation.loaded, baseline);
+      if (!baseline || source.commit !== item.commit || !source.entry) throw new Error(`Operating baseline for ${item.resourceId} changed after packet preparation.`);
+      await writePacketFile(output, item.recordPath, source.entry.source || `${JSON.stringify(baseline, null, 2)}\n`, files);
+      for (const related of item.relatedRecords || []) {
+        const entry = source.loaded.entries.find(({ record }) => record.id === related.id && record.type === related.type);
+        if (!entry) throw new Error(`Operating schedule record ${related.id} is unavailable.`);
+        await writePacketFile(output, related.recordPath, entry.source || `${JSON.stringify(entry.record, null, 2)}\n`, files);
+        for (const content of related.contentPaths) {
+          const text = governedMarkdownSource(source.loaded, entry.record, content.sourcePath);
+          if (text === null) throw new Error(`Operating schedule content for ${related.id} is unavailable.`);
+          await writePacketFile(output, content.exportedPath, text, files);
+        }
+      }
+      for (const content of item.contentPaths) {
+        const text = governedMarkdownSource(validation.loaded, baseline, content.sourcePath);
+        if (text === null) throw new Error(`Operating content for ${item.resourceId} is unavailable.`);
+        await writePacketFile(output, content.exportedPath, text, files);
+      }
+    }
     for (const item of packet.evidence) {
       const record = byId.get(item.id);
       for (const path of record?.filePaths || []) {
@@ -972,6 +1020,7 @@ function packetHandlingMarkdown(packet) {
     "",
     "Transfer this directory through the auditor's approved encrypted channel. Do not email an unencrypted packet. Give access only to the engagement team and retain or remove exported copies under the organization's evidence-retention rules.",
     "",
+    ...((packet.operatingBaselines || []).length ? ["Current source records can include pending proposals. `manifest.json` identifies the exact committed editions still operating at period end in `operatingBaselines`; their JSON and Markdown are in `operating-baselines/`. Use those editions with the full history when assessing operation.", ""] : []),
     "After transfer, enter the packet directory and run `shasum -a 256 -c SHA256SUMS` or `sha256sum -c SHA256SUMS`. filegrc does not sign or encrypt the packet because those operations require organization-controlled keys and transfer-system choices.",
     ""
   ].join("\n");
@@ -2221,6 +2270,7 @@ function packetMarkdown(packet) {
       ? `Open \`index.html\` for the auditor-oriented index. \`control-matrix.csv\` cross-references criteria, Controls, filegrc Evidence, Evidence Artifacts, and tests.${modelSupports(packet.dataModelVersion, "governed-document-activation") ? " `document-lifecycle-index.csv` lists each engagement Document's approval and activation facts and exact Markdown revisions." : ""} \`source-component-index.csv\` identifies the Components used to produce Evidence. \`evidence-artifact-index.csv\` lists material that must be delivered or accessed outside this packet. For Type 2, \`population-index.csv\` records management's population reconciliation and fixed source exports. FileGRC records, governed Markdown, fixed attachments, and committed historical versions are included in their respective directories.`
       : "Open `index.html` for the auditor-oriented index. `control-matrix.csv` cross-references criteria, controls, filegrc Evidence, External Evidence, and tests. `source-system-index.csv` identifies the systems of record used to produce External Evidence. `external-evidence-index.csv` lists material that must be delivered or accessed outside this packet. For Type 2, `population-index.csv` records management's population reconciliation and fixed source exports. filegrc records, governed Markdown, fixed attachments, and committed historical versions are included in their respective directories.",
     "",
+    ...((packet.operatingBaselines || []).length ? ["Current source records can include pending proposals. `manifest.json` identifies the exact committed editions still operating at period end in `operatingBaselines`; their JSON and Markdown are in `operating-baselines/`. Use those editions with the full history when assessing operation.", ""] : []),
     "After transfer, enter the packet directory and run `shasum -a 256 -c SHA256SUMS` or `sha256sum -c SHA256SUMS`. The checksum file covers every other packet file.",
     ""
   ];