[RFC]: Madar Core Reset — narrow TypeScript semantic-path engine

[mohanagy]

Status

Accepted on 2026-07-19. Scope and baseline is authorized after governance PR #579 merges; later phases remain gated.

This issue is the governing source of truth for the Madar Core Reset. The reset is a controlled replacement with mandatory deletion, not an additive refactor and not a permanent V1/V2 split.

Accepted decision

Approved: a narrow pre-1.0 reset that turns Madar into a local TypeScript/JavaScript semantic-path engine for coding agents.

The reset keeps a small set of proven safety/workspace primitives, replaces the graph/index/retrieval center, moves evaluation tooling out of the shipped runtime, and deletes or defers unrelated product surfaces.

Why this reset is necessary

The current architecture accumulated compatibility paths and product surfaces faster than they were retired:

• Production source is approximately 96,766 TypeScript lines across 181 modules.
• The in-memory graph keys an edge only by source and target, so parallel relationship kinds between the same nodes overwrite one another.
• Default auto generation runs SPI and legacy extraction over the same supported JS/TS files, then removes the SPI edge/hyperedge projection while retaining legacy topology.
• Production retrieval contains repository- and benchmark-shaped rules.
• Retrieval, comparison, installation, context governance, proof, federation, time travel, media ingestion, multiple installers, and evaluation tooling share one shipped package.
• Untuned production validation has not yet established a repeatable correctness, token, or latency win, and five live design partners do not yet exist.

Continuing to patch these paths would add another layer without correcting the substrate.

Product contract

Initial user

A senior TypeScript engineer or technical lead who:

• uses Claude Code or Codex several days per week;
• works in a TypeScript/JavaScript repository with several hundred source files or multiple apps/packages;
• asks questions that cross routes, services, queues, persistence, notifications, or public read models;
• values local source processing.

One job to be done

When I ask a coding agent to explain a cross-layer runtime flow in an unfamiliar TypeScript repository, return the smallest complete evidence path on the first pass so the agent can answer correctly without broad repository exploration.

Initial product hypothesis

Madar is a local TypeScript semantic-path engine that gives coding agents the smallest complete cross-layer execution path before broad search.

This is a hypothesis until blinded comparison and external reuse prove it.

Explicit non-users and non-jobs

The reset is not initially for:

• small repositories or single-file questions;
• Python/Go/polyglot-first teams;
• runtime debugging that requires logs or distributed traces;
• hosted knowledge bases;
• general PR review or security analysis;
• documentation search;
• enterprise administration.

Scope disposition

Keep and simplify

Preserve proven behavior, not necessarily current file boundaries:

• atomic artifact writes;
• sensitive-file and unsafe-path exclusions;
• source-path normalization;
• repository/worktree root discovery;
• external per-worktree artifact isolation;
• useful provenance concepts;
• framework-detection algorithms that can emit canonical TypeScript graph facts.

Rebuild

1. Graph schema

   • Always directed.
   • True multigraph with stable edge identities.
   • Preserve parallel relationships.
   • Store relationship kind, evidence location, provenance, and confidence where measurable.

2. Canonical TypeScript/JavaScript index

   • One TypeScript Program-based engine.
   • SPI writes the canonical schema directly.
   • Generic framework adapters for the supported TypeScript frameworks.
   • No SPI-to-legacy projection.

3. Incremental index

   • Correct add/change/delete/rename behavior.
   • Incremental output must equal a clean regeneration from the same repository state.

4. Generic retrieval

   • Exact file/symbol anchors.
   • Lexical ranking.
   • Typed directional traversal.
   • Minimal coherent evidence-path selection.
   • Directly stated missing targets.
   • No repository-specific paths, symbols, answer phrases, or hidden proof profiles.

5. Thin delivery

   • Small lazy-loaded CLI.
   • Small MCP tool surface.
   • Claude Code and Codex first-class during validation.
   • Generic MCP configuration remains possible for other clients.

Move outside the shipped runtime

Move under a development-only evaluation boundary and exclude from dist/npm:

• benchmark runners and corpora;
• compare/review-compare;
• prompt runners;
• proof reports and share-safe receipt production;
• real-repository evaluation fixtures;
• generated evaluation outputs.

The evaluation capability stays in the repository but production cannot import it.

Delete or defer

After replacement gates pass, remove:

• the legacy extraction engine and its framework extractors;
• Go/Python/generic Tree-sitter extraction;
• PDF/document/image/audio/video ingestion;
• SPI projector and extraction modes;
• context sessions, recovery, governance, resolution, delta, task planners, and global confidence machinery;
• feature maps, risk maps, implementation packs/checklists, relevant-files and generalized PR-impact wrappers;
• Neo4j, federation, time travel, graph-diff products, HTML/community visualization from the core;
• semantic embeddings and optional model dependencies;
• HTTP graph serving;
• routing profiles, prompt hooks, generated skills, and non-priority installers;
• generated benchmark artifacts currently included in npm.

Target architecture

Workspace scanner
      |
TypeScript Program indexer + framework adapters
      |
Directed typed multigraph + incremental store
      |
Generic evidence-path query
      |
Thin MCP and CLI adapters

Accepted dependency boundaries:

adapters -> application use cases -> domain contracts

The domain must not import filesystem, MCP, CLI, TypeScript compiler, or evaluation code.

Public surface target

MCP

No more than five tools:

• retrieve
• get_node
• get_neighbors
• graph_status
• shortest_path only if held-out evaluation proves it is needed

CLI

A narrow lazy-loaded surface:

• generate
• query
• status
• doctor
• install
• internal mcp entrypoint

Preserve existing command names only when their meaning remains valid.

Compatibility and release policy

• main / 0.32.x becomes maintenance-only for genuine P0 and security fixes.
• Work lands through a temporary protected core-reset integration branch.
• There will be no permanent core-v1 or core-v2 directories.
• No runtime mode or fallback will retain the old core.
• Existing 0.32.x releases remain pinnable through npm and Git history.
• New graph artifacts use a new schema and require regeneration.
• Compatibility is documented through migration notes, not permanent adapters.
• Publish 0.40.0-beta.n under the npm next tag only after one complete vertical slice passes.
• Publish 0.40.0 as latest only after all technical and business gates pass.

Dependency-ordered roadmap

1. Freeze scope and record baselines.
2. Establish held-out graph and answer fixtures.
3. Replace the graph with directed multigraph schema v2.
4. Make the TypeScript indexer canonical and delete legacy extraction.
5. Implement correct incremental refresh.
6. Replace retrieval and delete the context/governance stack.
7. Replace CLI/MCP and remove extra transports/integrations.
8. Move evaluation tooling outside runtime and reduce the npm package.
9. Run blinded native vs Graphify vs Madar capability evaluation.
10. Run external design-partner validation.
11. Release, pivot, or stop from the evidence.

Technical release gates

Graph

• Parallel edge types between the same nodes survive.
• Edge direction is preserved.
• Node and edge IDs are stable and deterministic.
• Evidence locations and provenance survive serialization.
• File deletion/rename leaves no stale graph facts.

Indexing

Manually labelled fixtures must cover:

• ESM and CJS imports;
• barrel re-exports and aliases;
• TypeScript path mappings and project references;
• calls, classes, interfaces, and types;
• Express, NestJS, Next.js, tRPC, Prisma, React Router, Fastify, and Hono boundaries.

Initial target gates:

• at least 95% import/re-export recall;
• at least 90% call/framework-edge recall;
• precision reported independently;
• no repository-specific production logic.

Retrieval

• Held-out real repositories only.
• At least 80% one-call evidence completeness.
• Typical result no more than 12 files, 25 snippets, and 4,000 input tokens.
• No global low-confidence label standing in for explicit missing evidence.
• Correctness no worse than the best comparator.
• At least 25-30% lower median total provider input than native search.
• At least 20% lower median end-to-end time.
• At least 75% of successful questions use one Madar call and no more than two focused verification reads.

Engineering

• Zero unused-local and unused-parameter diagnostics.
• Zero dependency-cycle and architecture-boundary violations.
• Production runtime is net-negative in LOC throughout the reset.
• Target shipped core: 25,000-35,000 production LOC and fewer than 80 source modules.
• Target npm package: fewer than 150 files and 1.5 MB unpacked.
• Target madar --version: below 100 ms and 80 MB RSS.
• MCP handshake and tool listing: below one second cold.
• Warm retrieval p95: below 500 ms on an approximately 15,000-node graph.

Targets are calibrated against the frozen baseline before implementation and may be changed only through an explicit RFC amendment.

Business validation gates

Discovery

Within the first 14 days:

• interview 15 qualified users;
• at least 10 report this problem weekly;
• at least 8 rate it 4/5 or 5/5 in pain;
• at least 5 provide real tasks/evidence;
• at least 5 commit to a four-week trial.

Activation and retention

• Four of five partners complete setup.
• Median time to first useful answer is below 15 minutes.
• At least three of five retain Madar and use it during three trial weeks.
• Broad-search restart after a Madar response is below 25%.
• Incorrect confident paths are below 5%.
• At least two teams request continued access and are willing to fund a pilot, or three credible buyers sign purchase-intent letters.

Downloads, stars, registry presence, and graph-generation counts are not retention evidence.

Anti-drift rules

Every implementation issue and PR must state:

• the user outcome;
• the RFC requirement;
• files/components added or modified;
• old files/components deleted;
• net production LOC delta;
• tests and held-out evidence;
• exit gate;
• non-goals.

A replacement is incomplete until the predecessor, obsolete tests, imports, flags, docs, and dependencies are deleted.

Additional rules:

• No new production feature without evidence from at least three independent target users.
• No repository-specific retrieval rules.
• No runtime imports from evaluation tooling.
• No parallel extractor or permanent compatibility layer.
• No stable release per intermediate phase.
• No issue closes merely because code merged; its deletion and evidence gates must pass.
• Any scope expansion requires an RFC amendment before code.
• Review the removal manifest and scorecard at every phase boundary.

Accepted open issue disposition

• P0: Cross-layer retrieval returns UI noise and drives agents into broad fallback #565 remains a real acceptance failure and contributes held-out retrieval evidence.
• P0: Simplify retrieval to one bounded phase-complete evidence pass #574 must not be implemented as another patch over the existing retrieval stack; its intended outcome is absorbed into the generic retrieval replacement.
• P2: madar --help omits the supported generate --spi option #571 becomes obsolete when extraction modes are removed; do not invest in documenting a mode scheduled for deletion.
• P0: Codex installer writes project MCP config that Codex CLI never loads #567 is folded into the thin Codex delivery work and must still be fixed before beta activation tests.

These issues change only through the dependency-ordered phase that owns each one; acceptance does not make later phases ready.

Tracking

• Milestone: v0.40.0 — Core Reset
• Project: Madar Roadmap
• Governing documentation: docs: establish Core Reset governance #579, versioned design, removal manifest, and scorecard.
• Weekly RFC comments will report completed work, deleted code, metrics, business evidence, drift decisions, and the next gate.

Acceptance checklist

• Product contract and explicit non-goals accepted.
• Keep/rebuild/move/delete disposition accepted.
• No permanent V1/V2 or compatibility fallback accepted.
• Target architecture accepted.
• Dependency order accepted.
• Technical gates accepted or amended.
• Business gates accepted or amended.
• Existing open-issue disposition accepted.
• Removal manifest and scorecard reviewed.
• Implementation explicitly authorized after this checklist is complete.