Implementation Plan: Complete GnuCash Cognitive Accounting

[drzo]

Implementation Plan: Complete GnuCash Cognitive Accounting

Guiding principles

• Prefer a correct, testable simulated cognitive core that always builds, with optional real OpenCog/ggml backends behind clean adapters.
• Integrate into existing GnuCash engine lifecycle (book open, transaction commit, account tree changes) before UI polish.
• Treat docs/demos as secondary until the engine layer compiles, tests green, and produces auditable accounting value (validation, attention ranking, prediction bounds).
• Keep changes surgical inside libgnucash/engine first; GUI/report work only after stable APIs.

---

Phase 0 — Make it build (foundation unblocker)

Goal: Clean compile of gnc-engine with cognitive sources on a stock Linux build (no OpenCog/ggml required).

Work

1. Resolve all merge remnants in gnc-cognitive-accounting.cpp:
   • Choose one coherent GncCognitiveAtomSpace layout (single struct with always-present maps + optional OpenCog handles pointer).
   • Ensure create_hierarchy_link, fund pools, truth values, and constructors exist on all build paths.
   • Fix MOSES discover path so strategy_atom is always defined before OpenCog/ASMOSES branches.
   • Fix URE validity function: remove dead code after early return and unify one algorithm path.
2. Fix gnc-cognitive-scheme.cpp:
   • Single valid raw-string Scheme bootstrap.
   • Remove orphaned Scheme fragments outside the string.
3. Unify messaging types:
   • One GncCognitiveMessage (prefer the module-enum hub in comms, or rename accounting’s string-based message to GncCognitiveAtomMessage).
   • Move C++-only APIs (std::vector receive) out of extern "C" or wrap with opaque C handles.
4. CMake hygiene:
   • Link GGML_LIBRARIES / include dirs when HAVE_GGML.
   • Ensure config.h / compile defs propagate consistently to engine + tests.
   • Add optional demo targets only if they build standalone without polluting core.
5. Fix tests to match defined semantics (e.g. balanced tx → confidence in [0.7, 1.0], not exact 1.0).
6. Gate CI: replace test-cognitive-accounting.sh presence checks with real ctest targets.

Exit criteria: ninja gnc-engine test-cognitive-accounting test-tensor-network succeeds without OpenCog/ggml.

---

Phase 1 — Solid simulated AtomSpace core

Goal: A deterministic in-process hypergraph that correctly mirrors Chart of Accounts and transactions.

Work

1. Atom model
   • Explicit atom records: handle, type, name, outgoing set, incoming set, SimpleTV (strength/confidence), attention (STI/LTI/VLTI/wage/rent).
   • Link integrity (Evaluation/Inheritance/Member/Implication) with real outgoing handles, not name-only strings.
2. Account mapping
   • gnc_account_to_atomspace: ConceptNode per account; InheritanceLink to type + parent; EvaluationLink for balance snapshots.
   • Invalidate/rebuild on rename, reparent, delete (QOF events).
3. Transaction mapping
   • Execution/Evaluation structures for splits (account, amount, value, commodity).
   • Double-entry as Implication/And patterns with TV derived from imbalance magnitude.
4. Persistence boundary (design now, implement light)
   • Session-scoped AtomSpace is OK initially.
   • Cognitive account flags already use KVP — keep that; document non-persistence of STI/LTI unless later serialized.

Exit criteria: Unit tests for hierarchy links, idempotent account atoms, balanced vs imbalanced TVs, and atom graph walk APIs.

---

Phase 2 — Engine lifecycle integration

Goal: Cognitive layer runs automatically when users do normal accounting, not only from demos.

Work

1. Init/shutdown hooks in engine/book lifecycle (gnc_engine_init / book open-close), guarded by feature flag (e.g. GNC_FEATURE_COGNITIVE_ACCOUNTING or runtime pref).
2. QOF event listeners:
   • Transaction commit → PLN validate + ECAN update per affected account + optional adaptive learning.
   • Account create/edit/destroy → AtomSpace sync.
3. Public query API for GUI/reports:
   • Last validation score for a transaction
   • Attention ranking of accounts
   • Predicted balance + uncertainty interval
4. Threading policy: start single-threaded on commit path; defer heavy MOSES/clustering to idle/async worker later.
5. Performance budget: O(splits) on commit; avoid full-book scans on every edit.

Exit criteria: Opening a book + posting transactions updates cognitive state; shutdown is leak-free under ASAN/tests.

---

Phase 3 — Real algorithms behind the OpenCog-shaped API

Complete each subsystem as useful accounting behavior, still simulated unless optional libs present.

3A. PLN validation & proofs

• Formalize TV combination (product/revision) and document formula.
• Double-entry: imbalance relative to magnitude → strength; split count, age, account reliability → confidence.
• N-entry: party/account diversity constraints.
• Trial balance proof: walk subtree, assert Σ debits = Σ credits; emit proof atom + structured report object (not just a handle).
• P&L proof: income − expense over period with commodity awareness.
• Multi-currency: convert via pricedb before validation.

3B. ECAN attention

• Global STI/LTI funds with wage on activity, rent on holdings, decay tick (time-based, not only on events).
• Hebbian-style association: co-occurring accounts in splits boost mutual attention.
• API: top-K attended accounts; “stale high-balance” detection.
• Optional OpenCog AttentionBank adapter when HAVE_OPENCOG_ATTENTION.

3C. MOSES strategy discovery

• Replace placeholder “return same transaction”:
  • Featureize historical txs (split count, type pairs, day-of-week, amount buckets).
  • Evolutionary or greedy search over categorization/balancing heuristics.
  • Output: ranked rule set (atoms + human-readable Scheme/JSON), not silent const_cast.
• Optimize path: suggest missing imbalance split account candidates (never mutate committed txs in place without edit API).

3D. URE prediction

• Historical series per account (balance over time or net flows).
• Simple models first: mean drift + variance → prediction ± bounds.
• Return structure with point estimate, low/high, confidence (extend API beyond bare gnc_numeric if needed).
• Seasonal hooks later (monthly bills).

3E. Emergence / distributed cognition

• Define measurable patterns (recurring transfer pairs, anomalous spikes, attention cascades).
• Wire comms hub so PLN validation results publish to ECAN/MOSES consumers with typed payloads (atoms or small structs), not gpointer voids.

Exit criteria: Golden-file tests on a sample book; proofs produce checkable numeric equality; predictions beat naive “current balance” on held-out windows for at least one synthetic series.

---

Phase 4 — Tensor network completion (Memory / Task / AI / Autonomy)

Goal: Turn the four-node architecture into a real pipeline over financial tensors.

Work

1. Feature schema (versioned):
   • Transactions: normalized amount, log-magnitude, day, dow, account-type one-hots, commodity id, imbalance flag, split count.
   • Accounts: type, depth, child count, balance z-score, STI/LTI, reconcile age.
2. Memory node: ring buffer / windowed tensor store; optional mmap cache for large books.
3. Task node: DAG of jobs (encode → cluster → attention → insights) with priorities from ECAN.
4. AI node / Cogfluence:
   • Replace sin/cos toys with real clustering (k-means or hierarchical on standardized features).
   • Pattern discovery: cluster centroids + silhouette; anomaly scores.
   • Insights as structured key/values + AtomSpace Concept/Evaluation links.
5. Autonomy node: adjust clustering k, attention decay, and task priorities from recent fitness (validation accuracy, user dismissals later).
6. ggml backend: when HAVE_GGML, build compute graphs for distance/clustering kernels; keep CPU fallback identical numerically within tolerance.
7. Bridge tensor insights → AtomSpace (AI node outputs become atoms; Autonomy modulates ECAN funds).

Exit criteria: End-to-end test: N transactions → clusters → insights table → attention redistribution; ggml and fallback agree within ε.

---

Phase 5 — Scheme / Guile integration (GnuCash-native)

Goal: Use GnuCash’s existing Guile stack rather than OpenCog Scheme-only.

Work

1. Expose cognitive C API via SWIG/Guile modules ((gnucash cognitive)).
2. Scheme helpers for reports: validation summary, attention top accounts, predicted balances.
3. Keep OpenCog SchemeEval path only when AtomSpace is real; otherwise evaluate through Guile.
4. Hypergraph pattern strings become export format + optional BindLink-like DSL, not fake eval.

Exit criteria: Guile REPL can init cognitive core, map a book, and print validation/attention without C demos.

---

Phase 6 — Product surfaces (UI, import, reports)

Goal: Users can see and act on cognitive output.

Work

1. Preferences: enable cognitive engine; attention decay; clustering window.
2. Register/transaction UI: validation badge/tooltip (confidence + reason).
3. Account tree: optional attention heat or sort-by-attention.
4. Reports (HTML/Guile):
   • Cognitive Trial Balance / P&L proof report
   • Attention & activity report
   • Anomaly/cluster report
5. Import pipeline (CSV/OFX/QIF): post-import batch validate + suggest categorizations from MOSES patterns.
6. Python bindings examples updated for cognitive queries (optional).

Exit criteria: Manual UX path on sample book without running C++ demos.

---

Phase 7 — Optional real OpenCog / distributed backends

Goal: Adapters, not ifdefs sprinkled through business logic.

Work

1. CognitiveBackend interface: SimulatedBackend (default), OpenCogBackend, (later) RemoteCogServerBackend.
2. Map Gnc atom types ↔ OpenCog types carefully; own handle table.
3. PLN/URE/MOSES call real modules only inside backend; simulated remains reference implementation for tests.
4. CogServer: optional out-of-process AtomSpace for heavy analysis; never block GUI commit path.
5. Packaging: document deps; CI matrix job with -DWITH_OPENCOG=ON when images exist.

Exit criteria: Same public API passes tests on Simulated and OpenCog backends for a core subset.

---

Phase 8 — Hardening, security, performance, docs

Work

1. Fuzz/property tests: random balanced/unbalanced txs; attention fund conservation invariants.
2. Memory: no growth on repeated open/close; message queues bounded.
3. Security: no system/eval of untrusted Scheme from book data; sanitize names in generated Scheme strings.
4. Multi-commodity and business features (invoices) cognitive mapping — phase after personal accounting path is solid.
5. Rewrite docs to match reality:
   • Architecture (what works / optional)
   • Developer guide (build flags, adding rules)
   • User guide (features + limitations)
6. Demote IMPLEMENTATION_REPORT.md claims; track completion with a real checklist in repo or issues.
7. Benchmarks on large books (10k–100k txs).

---

Recommended delivery milestones

Milestone	Outcome
M0	Clean build + green cognitive/tensor unit tests (no external cognitive deps)
M1	AtomSpace mirrors CoA + txs; QOF hooks live behind feature flag
M2	PLN validation + trial balance/P&L proofs with numeric checks
M3	ECAN ranking + URE forecast with uncertainty
M4	Tensor pipeline with real clustering + insights → atoms
M5	Guile/report surface for validation & attention
M6	Import assists + register UI badges
M7	OpenCog backend adapter (optional CI)
M8	Performance, docs truth-up, release-quality flag

---

Suggested workstream order (dependency graph)

1. Stabilize compile & types (Phase 0)
2. Atom model + tests (Phase 1)
3. Lifecycle hooks (Phase 2)
4. Parallelize after M1:
   • PLN/proofs (3A)
   • ECAN (3B)
   • Tensor features/clustering (Phase 4 core)
5. URE + MOSES (3C/3D) using validation fitness from 3A
6. Guile/reports (Phase 5–6)
7. OpenCog adapter (Phase 7)
8. Hardening (Phase 8)

---

Explicit non-goals for early milestones

• Full AGI / “emergent consciousness” messaging without measurable metrics
• Mutating user transactions silently
• Requiring OpenCog or ggml for basic cognitive features
• Blockchain or multi-agent distributed ledgers
• GPU path before CPU correctness

---

Immediate next implementation slice (when coding starts)

1. Repair gnc-cognitive-accounting.cpp / gnc-cognitive-scheme.cpp / message type clash.
2. Introduce unified GncCognitiveAtom record + always-on simulated backend.
3. Align unit tests with TV semantics and fund accounting invariants.
4. Add QOF commit hook behind feature flag calling PLN + ECAN.
5. Replace Cogfluence sin/cos with k-means on a documented feature vector.

That slice turns the project from “documented scaffold” into a shippable cognitive validation layer others can extend.