Skip to content

feat(knowledge): Phase 3 — absorb, audit and single-source the D365FO knowledge canon - #156

Merged
dynamics365ninja merged 4 commits into
mainfrom
feat/knowledge-phase-3
Aug 6, 2026
Merged

feat(knowledge): Phase 3 — absorb, audit and single-source the D365FO knowledge canon#156
dynamics365ninja merged 4 commits into
mainfrom
feat/knowledge-phase-3

Conversation

@dynamics365ninja

Copy link
Copy Markdown
Owner

Closes Phase 3 of docs/KNOWLEDGE_AUDIT_PLAN.md — findings R1 (curated knowledge base + its audit), K1 (rule canon written out three times) and K2 (skills emitted but never installable).

Four commits, each independently reviewable.


3.2 — gate the corpus itself, not just the code it generates

Generated code has been gated fail-closed for a while (validate references, validate xpp, the build). The knowledge shipped to the model was gated by nothing — the asymmetry upstream d365fo-mcp-server closed with apiSymbols / exampleValidation, and whose own eval README flags this repo's corpus as never having had that treatment.

  • KnowledgeRefExtractor pulls every named AOT element out of skills/_source (static call, extends, new, attribute, intrinsic, declaration). Conservative by construction: markdown links, <Slot> placeholders, container literals, XML/JSON fences and the My… placeholder convention are excluded, so an unresolved reference is a defect rather than prose noise.
  • KnowledgeAudit resolves them through a new IKnowledgeSymbolLookup; MetadataRepository now answers over 22 named AOT collections instead of the 9 the reference resolver needed — knowledge prose names menu items, reports, services and security elements as freely as it names classes.
  • KnowledgeExamples routes every X++ / AOT-XML fence through the same offline BP validator validate xpp runs. Intentional wrong-vs-right demos are pinned, and a pin that stops firing fails too.

d365fo knowledge audit [--capture|--verify] runs both halves. With a full standard index it resolves live; otherwise every reference must be covered by eval/knowledge-audit.snapshot.json, captured here against a real PackagesLocalDirectory, so CI (which has no index) can still refuse an un-audited edit. Exceptions are reviewed data in eval/knowledge-audit.allow.json — kernel types that exist only in the AOS binaries, plus the one name the corpus mentions in order to say it does not exist.

What the first run found:

Defect Fix
SysRunnable exists in no AOT. MigrationScriptScaffolder emitted extends SysRunnable and the corpus taught it. A D365FO runnable class is a plain class with a static main(Args) — verified against a live index and against AppTroubleshootingCore's own runnable classes. Scaffolder, topic, eval case and golden corrected; scaffolder regression tests added.
FmParameters in the number-sequence examples collides with FleetManagement's real FMParameters (casing finding) renamed, with a note to check the name is free first
An [ExtensionOf] example class not ending _Extension (COC003) renamed
Three prose placeholders written as if they were real element names wrapped as slots

3.1 + 3.1b — absorb the predecessor's 63-entry knowledge base

Upstream carries ~3,400 lines of curated X++/D365FO knowledge covering ~44 domains this corpus did not touch. Ported into the single-source corpus rather than forked as a second store: 16 new topics (19 → 35), grouped by domain rather than one per entry.

posting-and-financials · ssrs-report-authoring · security-modeling · integration-dmf-dualwrite · runtime-frameworks · inventory-and-warehouse · xpp-data-access-apis · forms-and-navigation · transactions-and-concurrency · performance-and-caching · xpp-runtime-types · number-sequence-patterns · workflow-authoring · testing-and-quality · analytics-and-er · build-error-triage

Plus new sections on existing topics: retryable/async batch (sysoperation-batch-patterns) and table inheritance (table-scaffolding).

Every topic went through knowledge audit against the live index before landing. That gate is why the corpus states SrsReportParameterAttribute (not SRS…), <DefaultAggregate> (not AggregateFunction), SysGlobalTelemetry (not SysTelemetry) and AxMenuElementSubMenu/<SubMenu> — and it caught two casing defects in this change before review did. Every CLI invocation shown was checked against the real --help rather than assumed.

3.1b: d365foErrorHelp.ts's catalog becomes 11 new XppcFixHints rules — missing next, overlayering, metadata deserialization, BPUpgradeCodeToday, nested loops, tts imbalance, UpdateConflict, missing forUpdate, CLRError, number-sequence setup, missing field — plus the build-error-triage topic. Two tests keep the scoring honest (UpdateConflict must outrank the generic tts rule; the field rule must outrank the generic identifier rule) and a third asserts every rule's knowledge pointer resolves to a topic that actually exists.

3.3 — single-source the rule canon

The same rules were written out three times, so a correction landed in one place and silently missed the rest. Each block now has exactly one home: the topic that explains it, fenced by <!-- canon:<id> -->. Eight blocks across six topics.

Deviation from the plan, and an improvement on it. The plan called for a generated side-artifact plus a drift check. Two of the three consumers do better: RuleCanon reads the blocks out of the corpus that is already embedded in the assembly, so agent-prompt and the new MCP initialize instructions compose from the source text at runtime — drift is not merely detected, it cannot be represented. Only skills/d365fo-cli/SKILL.md is a file on disk; its regions are generated by emit-skills.py, with CI's drift job widened to cover it.

The long MCP tool descriptions turned out not to carry the X++ canon — they are tool-usage text. The third consumer became the spec's instructions field on initialize, which is where clients want the rules anyway: paid once per session instead of restated per tool.

Also: SKILL.md's reference table was listing 19 of 35 topics. It is now generated from a new covers: frontmatter field, so a new topic cannot be invisible to the skill again.

3.4 — make the skills installable, and load them here

The Anthropic variant has been emitted for a while but had no installer — the documented path was a cp -r, which never prunes, so a renamed topic kept feeding an agent guidance the corpus had dropped.

scripts/Install-D365FoClaudeSkills.ps1 mirrors the Copilot installer: regenerates when empty (pwsh / Windows PowerShell / python, whichever the machine has), installs one .claude/skills/<topic>/SKILL.md per topic, and prunes retired topics while leaving unrelated skills in the same folder alone — covered by an explicit smoke test.

This repo's own .claude/skills/d365fo-knowledge/ is a router, not a second copy: where the source of truth is, which files are generated, how to run knowledge audit before calling a knowledge edit done, and that the canon is edited in the topic that owns it. Duplicating 35 topics into the repo again would have recreated exactly the drift this phase removes.


Verification

  • dotnet test939 passing, 0 failing (+29 new)
  • d365fo knowledge audit — 269 references, 61 examples, 0 defects; snapshot captured against a live PackagesLocalDirectory
  • d365fo eval run --all51/51 golden-clean
  • python scripts/emit-skills.py — idempotent, no drift
  • Install-D365FoClaudeSkills.ps1 — 35 skills installed; prune removes a retired topic and keeps an unrelated one
  • d365fo knowledge search lands on the new domains (posting, feature management, dual-write change tracking, SYS10028, on-hand, aggregate measurements)

CI gains a knowledge-audit job running the snapshot half; the skills drift job now covers skills/d365fo-cli/ as a whole.

Not in scope

Phase 1.4 and Phase 2 remain open — this phase was independent of them by design (see the plan's sequencing note).

🤖 Generated with Claude Code

dynamics365ninja and others added 4 commits August 6, 2026 05:40
…it generates

Phase 3.2 of docs/KNOWLEDGE_AUDIT_PLAN.md. Generated code has been gated
fail-closed for a while (`validate references`, `validate xpp`, the build), but
the knowledge shipped to the model was gated by nothing — the asymmetry upstream
d365fo-mcp-server closed with apiSymbols/exampleValidation, and whose own eval
README flags this repo's skill corpus as never having had that treatment.

Two halves, both offline-capable:

- `KnowledgeRefExtractor` pulls every named AOT element out of skills/_source
  (static call, extends/implements, new, attribute, intrinsic, declaration),
  conservatively: markdown links, `<Slot>` placeholders, container literals,
  XML/JSON fences and the `My…` placeholder convention are all excluded, so an
  unresolved reference is a defect rather than prose noise.
- `KnowledgeAudit` resolves those against the real symbol index through a new
  `IKnowledgeSymbolLookup` (MetadataRepository now answers over 22 named AOT
  collections, not the 9 the reference resolver needed), reporting unknown type,
  unknown member and wrong casing.
- `KnowledgeExamples` routes every ```xpp / ```xml block through the same
  offline BP validator `validate xpp` runs; intentional wrong-vs-right demos are
  pinned, and a pin that stops firing fails too.

`d365fo knowledge audit [--capture|--verify]` runs both. With a full standard
index it resolves live; otherwise every reference must be covered by
eval/knowledge-audit.snapshot.json, captured here against a real
PackagesLocalDirectory, so CI (which has no index) can still refuse an
un-audited knowledge edit. Exceptions live as reviewed data in
eval/knowledge-audit.allow.json — kernel types that exist only in the AOS
binaries, plus the one name the corpus mentions in order to say it does not
exist.

What the first run found, and this commit fixes:

- `SysRunnable` exists in no AOT. `MigrationScriptScaffolder` emitted
  `extends SysRunnable` and the corpus taught it. A D365FO runnable class is a
  plain class with a static `main(Args)` — verified against a live index and
  against AppTroubleshootingCore's own runnable classes. Scaffolder, topic,
  eval case title and golden all corrected; scaffolder regression tests added.
- `FmParameters` in the table/number-sequence examples collides with
  FleetManagement's real `FMParameters` (casing finding). Renamed, with a note
  to check the name is free first.
- A `[ExtensionOf]` example class not ending `_Extension` (COC003), and three
  prose placeholders written as if they were real element names.

CI gains a `knowledge-audit` job running the snapshot half.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 3.1 + 3.1b of docs/KNOWLEDGE_AUDIT_PLAN.md (finding R1). Upstream
d365fo-mcp-server carries 3,400 lines of curated X++/D365FO knowledge covering
~44 domains this repo's corpus did not touch. Ported into the single-source
skill corpus rather than forked as a second store, so `d365fo knowledge`, the
Copilot/Anthropic/d365fo-cli skill variants and the embedded resource all still
have exactly one place to correct a fact.

16 new topics (19 -> 35), grouped by domain rather than one-per-entry:

  posting-and-financials, ssrs-report-authoring, security-modeling,
  integration-dmf-dualwrite, runtime-frameworks, inventory-and-warehouse,
  xpp-data-access-apis, forms-and-navigation, transactions-and-concurrency,
  performance-and-caching, xpp-runtime-types, number-sequence-patterns,
  workflow-authoring, testing-and-quality, analytics-and-er,
  build-error-triage

plus new sections on existing topics: retryable/async batch
(sysoperation-batch-patterns) and table inheritance (table-scaffolding).

Every topic went through `d365fo knowledge audit` against the live symbol index
before landing: 267 references resolve or are reviewed exceptions, and all 61
code examples pass the offline BP validator. That gate is why the corpus states
`SrsReportParameterAttribute` (not SRS…), `<DefaultAggregate>` (not
AggregateFunction), `SysGlobalTelemetry` (not SysTelemetry) and
`AxMenuElementSubMenu`/`<SubMenu>` — and it caught two casing defects in this
change before review did. Every CLI invocation shown was checked against the
real `--help` rather than assumed.

The allowlist gains the kernel/.NET names that exist only in the AOS binaries,
and a `prefixes` section for the corpus's own Fleet example namespace, each with
a written reason.

3.1b: d365foErrorHelp.ts's error catalog becomes 11 new XppcFixHints rules
(missing `next`, overlayering, metadata deserialization, BPUpgradeCodeToday,
nested loops, tts imbalance, UpdateConflict, missing forUpdate, CLRError,
number-sequence setup, missing field) plus the build-error-triage topic. Two new
tests keep the scoring honest — UpdateConflict must outrank the generic tts rule,
the field rule must outrank the generic identifier rule — and a third asserts
every rule's knowledge pointer resolves to a topic that actually exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 3.3 of docs/KNOWLEDGE_AUDIT_PLAN.md (audit finding K1). The same rules were
written out three times — in `agent-prompt`, across the MCP surface, and in the
shipped skill files — so a correction landed in one and silently missed the rest.

Each rule block now has exactly one home: the topic that explains it, fenced by
`<!-- canon:<id> -->` … `<!-- /canon -->`. Eight blocks (core, queries, coc,
classes, statements, bp, aot-xml-safety, never-auto) across six topics.

The plan called for a generated side-artifact plus a drift check. Two of the
three consumers do better than that: `RuleCanon` reads the blocks out of the
corpus that is *already* embedded in the assembly, so `agent-prompt` and the new
MCP `initialize` instructions compose from the source text at runtime and drift
is not merely detected, it cannot be represented. Only the third consumer is a
file on disk — skills/d365fo-cli/SKILL.md — and there the generated regions are
written by emit-skills.py, with CI's drift job widened to cover it.

Also here:

- MCP `initialize` gains the spec's `instructions` field. Clients surface it once
  before any tool call, so the canon is paid for once per session rather than
  re-stated in every tool description.
- SKILL.md's hand-written "Key rules" list keeps only the rules about driving the
  CLI; the X++ rules it duplicated now come from the canon. Its reference table
  was listing 19 of 35 topics — it is generated from a new `covers:` frontmatter
  field, so a new topic cannot be invisible to the skill again.
- The agent prompt's query and CoC sections point at
  `d365fo knowledge get <topic>` for the worked examples the canon omits.

Tests cover all three consumers, plus the failure mode that motivated the design:
`RuleCanon.Require` on an undeclared id throws with the list of real ids rather
than emitting a silently empty rule section.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…his repo

Phase 3.4 of docs/KNOWLEDGE_AUDIT_PLAN.md (audit finding K2). The Anthropic
skill variant has been emitted for a while but had no installer — the documented
path was a `cp -r`, which never prunes, so a renamed or retired topic kept
feeding an agent guidance the corpus had dropped.

`scripts/Install-D365FoClaudeSkills.ps1` mirrors the Copilot installer: it
regenerates `skills/anthropic/` when empty (pwsh / Windows PowerShell / python,
whichever the machine has), installs one `.claude/skills/<topic>/SKILL.md` per
topic, and prunes topics that no longer exist upstream — while leaving unrelated
skills in the same folder alone, which the smoke test covers explicitly.

This repo's own `.claude/skills/d365fo-knowledge/` is a router rather than a
second copy of the corpus: it says where the source of truth is, which files are
generated, how to run `knowledge audit` before calling a knowledge edit done, and
that the rule canon is edited in the topic that owns it. Duplicating 35 topics
into the repo a second time would have created exactly the drift Phase 3 removed.

Also updates docs/KNOWLEDGE_AUDIT_PLAN.md with what Phase 3 actually did, including
the two places the implementation deviated from the plan and why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dynamics365ninja
dynamics365ninja merged commit 8e76e23 into main Aug 6, 2026
5 checks passed
@dynamics365ninja
dynamics365ninja deleted the feat/knowledge-phase-3 branch August 6, 2026 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant