feat(knowledge): Phase 3 — absorb, audit and single-source the D365FO knowledge canon - #156
Merged
Merged
Conversation
…it generates Phase 3.2 of docs/KNOWLEDGE_AUDIT_PLAN.md. Generated code has been gated fail-closed for a while (`validate references`, `validate xpp`, the build), but the knowledge shipped to the model was gated by nothing — the asymmetry upstream d365fo-mcp-server closed with apiSymbols/exampleValidation, and whose own eval README flags this repo's skill corpus as never having had that treatment. Two halves, both offline-capable: - `KnowledgeRefExtractor` pulls every named AOT element out of skills/_source (static call, extends/implements, new, attribute, intrinsic, declaration), conservatively: markdown links, `<Slot>` placeholders, container literals, XML/JSON fences and the `My…` placeholder convention are all excluded, so an unresolved reference is a defect rather than prose noise. - `KnowledgeAudit` resolves those against the real symbol index through a new `IKnowledgeSymbolLookup` (MetadataRepository now answers over 22 named AOT collections, not the 9 the reference resolver needed), reporting unknown type, unknown member and wrong casing. - `KnowledgeExamples` routes every ```xpp / ```xml block through the same offline BP validator `validate xpp` runs; intentional wrong-vs-right demos are pinned, and a pin that stops firing fails too. `d365fo knowledge audit [--capture|--verify]` runs both. With a full standard index it resolves live; otherwise every reference must be covered by eval/knowledge-audit.snapshot.json, captured here against a real PackagesLocalDirectory, so CI (which has no index) can still refuse an un-audited knowledge edit. Exceptions live as reviewed data in eval/knowledge-audit.allow.json — kernel types that exist only in the AOS binaries, plus the one name the corpus mentions in order to say it does not exist. What the first run found, and this commit fixes: - `SysRunnable` exists in no AOT. `MigrationScriptScaffolder` emitted `extends SysRunnable` and the corpus taught it. A D365FO runnable class is a plain class with a static `main(Args)` — verified against a live index and against AppTroubleshootingCore's own runnable classes. Scaffolder, topic, eval case title and golden all corrected; scaffolder regression tests added. - `FmParameters` in the table/number-sequence examples collides with FleetManagement's real `FMParameters` (casing finding). Renamed, with a note to check the name is free first. - A `[ExtensionOf]` example class not ending `_Extension` (COC003), and three prose placeholders written as if they were real element names. CI gains a `knowledge-audit` job running the snapshot half. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 3.1 + 3.1b of docs/KNOWLEDGE_AUDIT_PLAN.md (finding R1). Upstream d365fo-mcp-server carries 3,400 lines of curated X++/D365FO knowledge covering ~44 domains this repo's corpus did not touch. Ported into the single-source skill corpus rather than forked as a second store, so `d365fo knowledge`, the Copilot/Anthropic/d365fo-cli skill variants and the embedded resource all still have exactly one place to correct a fact. 16 new topics (19 -> 35), grouped by domain rather than one-per-entry: posting-and-financials, ssrs-report-authoring, security-modeling, integration-dmf-dualwrite, runtime-frameworks, inventory-and-warehouse, xpp-data-access-apis, forms-and-navigation, transactions-and-concurrency, performance-and-caching, xpp-runtime-types, number-sequence-patterns, workflow-authoring, testing-and-quality, analytics-and-er, build-error-triage plus new sections on existing topics: retryable/async batch (sysoperation-batch-patterns) and table inheritance (table-scaffolding). Every topic went through `d365fo knowledge audit` against the live symbol index before landing: 267 references resolve or are reviewed exceptions, and all 61 code examples pass the offline BP validator. That gate is why the corpus states `SrsReportParameterAttribute` (not SRS…), `<DefaultAggregate>` (not AggregateFunction), `SysGlobalTelemetry` (not SysTelemetry) and `AxMenuElementSubMenu`/`<SubMenu>` — and it caught two casing defects in this change before review did. Every CLI invocation shown was checked against the real `--help` rather than assumed. The allowlist gains the kernel/.NET names that exist only in the AOS binaries, and a `prefixes` section for the corpus's own Fleet example namespace, each with a written reason. 3.1b: d365foErrorHelp.ts's error catalog becomes 11 new XppcFixHints rules (missing `next`, overlayering, metadata deserialization, BPUpgradeCodeToday, nested loops, tts imbalance, UpdateConflict, missing forUpdate, CLRError, number-sequence setup, missing field) plus the build-error-triage topic. Two new tests keep the scoring honest — UpdateConflict must outrank the generic tts rule, the field rule must outrank the generic identifier rule — and a third asserts every rule's knowledge pointer resolves to a topic that actually exists. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 3.3 of docs/KNOWLEDGE_AUDIT_PLAN.md (audit finding K1). The same rules were written out three times — in `agent-prompt`, across the MCP surface, and in the shipped skill files — so a correction landed in one and silently missed the rest. Each rule block now has exactly one home: the topic that explains it, fenced by `<!-- canon:<id> -->` … `<!-- /canon -->`. Eight blocks (core, queries, coc, classes, statements, bp, aot-xml-safety, never-auto) across six topics. The plan called for a generated side-artifact plus a drift check. Two of the three consumers do better than that: `RuleCanon` reads the blocks out of the corpus that is *already* embedded in the assembly, so `agent-prompt` and the new MCP `initialize` instructions compose from the source text at runtime and drift is not merely detected, it cannot be represented. Only the third consumer is a file on disk — skills/d365fo-cli/SKILL.md — and there the generated regions are written by emit-skills.py, with CI's drift job widened to cover it. Also here: - MCP `initialize` gains the spec's `instructions` field. Clients surface it once before any tool call, so the canon is paid for once per session rather than re-stated in every tool description. - SKILL.md's hand-written "Key rules" list keeps only the rules about driving the CLI; the X++ rules it duplicated now come from the canon. Its reference table was listing 19 of 35 topics — it is generated from a new `covers:` frontmatter field, so a new topic cannot be invisible to the skill again. - The agent prompt's query and CoC sections point at `d365fo knowledge get <topic>` for the worked examples the canon omits. Tests cover all three consumers, plus the failure mode that motivated the design: `RuleCanon.Require` on an undeclared id throws with the list of real ids rather than emitting a silently empty rule section. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…his repo Phase 3.4 of docs/KNOWLEDGE_AUDIT_PLAN.md (audit finding K2). The Anthropic skill variant has been emitted for a while but had no installer — the documented path was a `cp -r`, which never prunes, so a renamed or retired topic kept feeding an agent guidance the corpus had dropped. `scripts/Install-D365FoClaudeSkills.ps1` mirrors the Copilot installer: it regenerates `skills/anthropic/` when empty (pwsh / Windows PowerShell / python, whichever the machine has), installs one `.claude/skills/<topic>/SKILL.md` per topic, and prunes topics that no longer exist upstream — while leaving unrelated skills in the same folder alone, which the smoke test covers explicitly. This repo's own `.claude/skills/d365fo-knowledge/` is a router rather than a second copy of the corpus: it says where the source of truth is, which files are generated, how to run `knowledge audit` before calling a knowledge edit done, and that the rule canon is edited in the topic that owns it. Duplicating 35 topics into the repo a second time would have created exactly the drift Phase 3 removed. Also updates docs/KNOWLEDGE_AUDIT_PLAN.md with what Phase 3 actually did, including the two places the implementation deviated from the plan and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes Phase 3 of
docs/KNOWLEDGE_AUDIT_PLAN.md— findings R1 (curated knowledge base + its audit), K1 (rule canon written out three times) and K2 (skills emitted but never installable).Four commits, each independently reviewable.
3.2 — gate the corpus itself, not just the code it generates
Generated code has been gated fail-closed for a while (
validate references,validate xpp, the build). The knowledge shipped to the model was gated by nothing — the asymmetry upstreamd365fo-mcp-serverclosed withapiSymbols/exampleValidation, and whose own eval README flags this repo's corpus as never having had that treatment.KnowledgeRefExtractorpulls every named AOT element out ofskills/_source(static call,extends,new, attribute, intrinsic, declaration). Conservative by construction: markdown links,<Slot>placeholders, container literals, XML/JSON fences and theMy…placeholder convention are excluded, so an unresolved reference is a defect rather than prose noise.KnowledgeAuditresolves them through a newIKnowledgeSymbolLookup;MetadataRepositorynow answers over 22 named AOT collections instead of the 9 the reference resolver needed — knowledge prose names menu items, reports, services and security elements as freely as it names classes.KnowledgeExamplesroutes every X++ / AOT-XML fence through the same offline BP validatorvalidate xppruns. Intentional wrong-vs-right demos are pinned, and a pin that stops firing fails too.d365fo knowledge audit [--capture|--verify]runs both halves. With a full standard index it resolves live; otherwise every reference must be covered byeval/knowledge-audit.snapshot.json, captured here against a realPackagesLocalDirectory, so CI (which has no index) can still refuse an un-audited edit. Exceptions are reviewed data ineval/knowledge-audit.allow.json— kernel types that exist only in the AOS binaries, plus the one name the corpus mentions in order to say it does not exist.What the first run found:
SysRunnableexists in no AOT.MigrationScriptScaffolderemittedextends SysRunnableand the corpus taught it.main(Args)— verified against a live index and against AppTroubleshootingCore's own runnable classes. Scaffolder, topic, eval case and golden corrected; scaffolder regression tests added.FmParametersin the number-sequence examples collides with FleetManagement's realFMParameters(casing finding)[ExtensionOf]example class not ending_Extension(COC003)3.1 + 3.1b — absorb the predecessor's 63-entry knowledge base
Upstream carries ~3,400 lines of curated X++/D365FO knowledge covering ~44 domains this corpus did not touch. Ported into the single-source corpus rather than forked as a second store: 16 new topics (19 → 35), grouped by domain rather than one per entry.
posting-and-financials·ssrs-report-authoring·security-modeling·integration-dmf-dualwrite·runtime-frameworks·inventory-and-warehouse·xpp-data-access-apis·forms-and-navigation·transactions-and-concurrency·performance-and-caching·xpp-runtime-types·number-sequence-patterns·workflow-authoring·testing-and-quality·analytics-and-er·build-error-triagePlus new sections on existing topics: retryable/async batch (
sysoperation-batch-patterns) and table inheritance (table-scaffolding).Every topic went through
knowledge auditagainst the live index before landing. That gate is why the corpus statesSrsReportParameterAttribute(notSRS…),<DefaultAggregate>(notAggregateFunction),SysGlobalTelemetry(notSysTelemetry) andAxMenuElementSubMenu/<SubMenu>— and it caught two casing defects in this change before review did. Every CLI invocation shown was checked against the real--helprather than assumed.3.1b:
d365foErrorHelp.ts's catalog becomes 11 newXppcFixHintsrules — missingnext, overlayering, metadata deserialization,BPUpgradeCodeToday, nested loops, tts imbalance,UpdateConflict, missingforUpdate,CLRError, number-sequence setup, missing field — plus thebuild-error-triagetopic. Two tests keep the scoring honest (UpdateConflictmust outrank the generic tts rule; the field rule must outrank the generic identifier rule) and a third asserts every rule's knowledge pointer resolves to a topic that actually exists.3.3 — single-source the rule canon
The same rules were written out three times, so a correction landed in one place and silently missed the rest. Each block now has exactly one home: the topic that explains it, fenced by
<!-- canon:<id> -->. Eight blocks across six topics.Deviation from the plan, and an improvement on it. The plan called for a generated side-artifact plus a drift check. Two of the three consumers do better:
RuleCanonreads the blocks out of the corpus that is already embedded in the assembly, soagent-promptand the new MCPinitializeinstructions compose from the source text at runtime — drift is not merely detected, it cannot be represented. Onlyskills/d365fo-cli/SKILL.mdis a file on disk; its regions are generated byemit-skills.py, with CI's drift job widened to cover it.The long MCP tool descriptions turned out not to carry the X++ canon — they are tool-usage text. The third consumer became the spec's
instructionsfield oninitialize, which is where clients want the rules anyway: paid once per session instead of restated per tool.Also: SKILL.md's reference table was listing 19 of 35 topics. It is now generated from a new
covers:frontmatter field, so a new topic cannot be invisible to the skill again.3.4 — make the skills installable, and load them here
The Anthropic variant has been emitted for a while but had no installer — the documented path was a
cp -r, which never prunes, so a renamed topic kept feeding an agent guidance the corpus had dropped.scripts/Install-D365FoClaudeSkills.ps1mirrors the Copilot installer: regenerates when empty (pwsh / Windows PowerShell / python, whichever the machine has), installs one.claude/skills/<topic>/SKILL.mdper topic, and prunes retired topics while leaving unrelated skills in the same folder alone — covered by an explicit smoke test.This repo's own
.claude/skills/d365fo-knowledge/is a router, not a second copy: where the source of truth is, which files are generated, how to runknowledge auditbefore calling a knowledge edit done, and that the canon is edited in the topic that owns it. Duplicating 35 topics into the repo again would have recreated exactly the drift this phase removes.Verification
dotnet test— 939 passing, 0 failing (+29 new)d365fo knowledge audit— 269 references, 61 examples, 0 defects; snapshot captured against a livePackagesLocalDirectoryd365fo eval run --all— 51/51 golden-cleanpython scripts/emit-skills.py— idempotent, no driftInstall-D365FoClaudeSkills.ps1— 35 skills installed; prune removes a retired topic and keeps an unrelated oned365fo knowledge searchlands on the new domains (posting, feature management, dual-write change tracking, SYS10028, on-hand, aggregate measurements)CI gains a
knowledge-auditjob running the snapshot half; theskillsdrift job now coversskills/d365fo-cli/as a whole.Not in scope
Phase 1.4 and Phase 2 remain open — this phase was independent of them by design (see the plan's sequencing note).
🤖 Generated with Claude Code