Skip to content

Mixed schema_version: replay fixtures and handler tests for every event version #34

Open

Description

@abrcrmb

Task

The indexer only handles schema_version: 1 today. Add replay fixtures and tests that prove it does the right thing with more than one version per event type — and that an unknown version is treated as a gap it stops on, never silently skipped.

Context

  • services/indexer/src/handlers/index.ts:19-27 maps all seven event types, each to { 1: handler }. handlerFor() looks up HANDLERS[type][schemaVersion] and throws UnsupportedEventError for anything else.
  • services/indexer/src/ingest/decode.ts:67 reads schema_version off the event; AGENTS.md §8.2 requires handling multiple versions per type.
  • Fixtures already exist at services/indexer/test/fixtures/events/testnet-2026-10-06.json, replayed by test/ingest.test.ts:110 (twice = identical DB) and :128 (cursor resume).
  • README.md:9 records that mixed-version handling is the open part of M2-10.

Before you start: mandatory context review

Kinlock moves other people's money, so the docs are binding constraints, not background reading.
They record decisions this task must not silently reopen. Read them in this order and tick each box:

  • docs/ARCHITECTURE_ESSENTIALS.md — start here, it is short (AGENTS.md §1)
  • docs/ARCHITECTURE.md — only the sections named in Context above
  • docs/PRD.md — if this task touches user-facing behaviour or scope
  • AGENTS.md §3 (hard rules), §4 (deferred list), §8 for this repo, §12 (roadmap rule)
  • ROADMAP.md — the row(s) listed under Related roadmap rows, including their Depends on

In your first PR comment, name the hard-rule numbers or doc sections that constrain this task. That
line is how we know the review happened.

Then:

  • If code and docs disagree, or two docs disagree: stop and say so on this issue. Do not pick one
    silently, and do not "fix" either side without approval (AGENTS.md §1).
  • If the task looks like it needs anything from the deferred list (§4), a change to a hard rule (§3),
    or a new dependency (§2): raise it here first. Those need a human; deciding them is not your job.
  • If a row you are told to finish is BLOCKED or depends on a row that is not DONE, say so here
    instead of starting.

Why we need this

  • Contract events are append-only and versioned; the day LockCreated ships v2, an indexer that drops v1 rows or crashes on v2 rewrites people's financial history.
  • Silent skipping is the failure mode the spec forbids: "Never skip a gap silently: stop and alert" (AGENTS.md §8.2). That behaviour needs a test, not a comment.
  • M2-10 and M2-14 are P0 rows gating the testnet pilot.

Scope

In scope

  • services/indexer/test/fixtures/events/** — new fixture files
  • services/indexer/test/*.test.ts — new cases (mostly ingest.test.ts, api.test.ts)
  • services/indexer/src/handlers/** — only to register a version you are testing, with no change to v1 behaviour

Do not touch (security-sensitive or generated; changing these needs a human first — AGENTS.md §2, docs/project_structure.md §8)

  • packages/sdk/src/links.ts, hash.ts, receipts.ts, preflight.ts — claim-link privacy, ref hashing, receipt verification, preflight gating
  • services/indexer/src/ingest/** — cursor, gap detection and decode; money-relevant
  • services/indexer/src/db/migrations/** and db/schema.ts — migrations are forward-only and reviewed
  • bindings/** (generated from the contract), package.json and pnpm-lock.yaml (no new dependencies without approval)
  • .github/workflows/**

Suggested implementation

  1. Read ARCHITECTURE_ESSENTIALS.md, then docs/ARCHITECTURE.md §5 (off-chain design) and §4's event section, then handlers/index.ts and ingest/decode.ts.
  2. Add a v1/v2 fixture pair for at least two event types, where v2 adds a field and keeps v1's meaning. Do not invent a v2: model it as a hypothetical forward-compatible shape and say so in the PR, or use a real schema_version difference you can point to in the contract.
  3. Test three things: (a) a stream mixing v1 and v2 rows persists both correctly, (b) schema_version 99 makes the run stop and surface an alert rather than skip, (c) replaying the mixed stream twice yields an identical database (M2-14's existing standard).
  4. Keep chain_events append-only in every assertion — no update, no delete, ever.
  5. If you conclude the current UnsupportedEventError path already satisfies (b), prove it with a test and say so. Do not change ingest/ to make a test nicer.

A different approach is usually welcome — post it here before building and we will answer. This is a
suggestion, not a spec.

Edge cases and traps

  • Unknown is not only 99: schema_version 0, a negative, and a missing field must all land on the same stop path.
  • v1 and v2 rows for the same lock interleaved in one stream: both must persist, and any derived state must not double-count the lock.
  • Replay the mixed stream twice and assert identical DB state the way test/ingest.test.ts:110 does for the single-version fixture.
  • chain_events is append-only (§8.2). A fixture that repairs a row by UPDATE or DELETE looks like a passing test and is a hard-rule violation.
  • Do not invent a v2 as fact. Either point at a real version difference in the contract or label the v2 fixture as a hypothetical forward-compatible shape in the PR.

Done means

  • Mixed-version fixtures exist and are replayed by a test
  • A test proves an unknown schema_version stops the run and does not skip silently
  • Replay-twice on the mixed stream yields a byte-identical DB state (assert the same way ingest.test.ts:110 does)
  • pnpm test green; v1 behaviour unchanged — show it with a diff of the v1 handlers
  • If M2-10/M2-14 are now fully met, flip them; otherwise leave IN PROGRESS and add a row for what is left

Tests

Do not weaken an existing fixture or expectation to accommodate yours. If a test fails in a way you cannot explain, post the failure here — it may be a real gap in M2-10.

Documentation

README.md's indexer section and the handler map comment if the version story changed. No new public API.

Verify before you open the PR

pnpm install
pnpm lint
pnpm typecheck
pnpm test
pnpm build

Run every command above and paste the real output in the PR. We do not accept "should pass" —
AGENTS.md §6: "You actually ran the commands."

Opening the pull request

  • Assignment is required before you start. Comment on this issue and wait to be assigned. Do not
    open a PR against an issue someone already holds — if it has been quiet, ask us to release it first.
  • Closes #34 in the PR description, so the issue and the PR link automatically.
  • Branch feat/…, fix/…, docs/…, test/… or chore/…. Commit subject example:
    test(indexer): mixed schema_version fixtures and stop-on-unknown coverage
  • Update ROADMAP.md in the same PR: M2-10/M2-14 status, Changelog entry, Last updated.
  • Follow .github/PULL_REQUEST_TEMPLATE.md (AGENTS.md §7). In "Hard rules / invariants touched",
    give numbers, not prose.
  • Small and focused: one logical change. Do not mix a refactor into it.
  • Do not add the security-sensitive label — that is the maintainer's flag for contract-logic PRs.
  • Do not reformat or touch files outside Scope. Unrelated diff noise costs review time and can
    break the roadmap merge.
  • Never commit to main; open a PR. Never force-push.
  • Evidence for the PR: the real output of every command under Verify before you open the PR, plus what you could not run and why.

How we review

Two maintainers: one reviews correctness against the hard rules and the roadmap row's Done when, the
other reviews diff scope and whether the tests were actually run. Order is hard-rule correctness → tests
really executed → diff stays inside Scope → style last. First response on a PR: 2 business days.

What fails review fastest: a change outside Scope, a deleted or weakened assertion, a claim with no
command output behind it, and a ROADMAP.md with no Changelog entry (AGENTS.md §12).
For this issue, look first at that the v1 handlers are unchanged (git diff on handlers/index.ts) and that the unknown-version test asserts a stop plus an alert, not a silent skip.

Tier and effort

tier/medium · roughly 1–2 days. Wave complexity tag: Medium = 150 points — the published mapping is Trivial 100 · Medium 150 · High 200
(Drips' maintainer guide); the cycle's
own values are confirmed on drips.network/wave/stellar before
the Wave opens. Kinlock's tier rules and what each tier means for review:
docs/wave-issue-format.md.

Related roadmap rows

M2-10 — Indexer handlers per event type and schema_version (IN PROGRESS)
M2-14 — Indexer tests: fixture replay, idempotency, cursor recovery, mixed versions (IN PROGRESS)

Both rows are already IN PROGRESS with idempotency and cursor-recovery coverage in place; this task lands the mixed-version part and should leave both rows IN PROGRESS unless it completes them fully.

Questions

Comment on this issue — a maintainer answers within 2 business days. Say "I'll take this" so we can flag it on
the Wave board and stop two people doing the same work. If you get half way and stall, post what you
have as a draft PR rather than disappearing; a blocked draft is more useful to us than silence.

Activity

  1. added
    waveScoped for Drips Wave contributors
    area:indexerOwnership: kinlock-sdk indexer
    tier/mediumWave tier: standard feature or fix work, a few hours, tests included
    on Oct 8, 2026
  2. jdjioe5-cpu commented on Oct 17, 2026

    @jdjioe5-cpu

    I'll take this. Starting work on M2-10/M2-14 mixed schema_version fixtures, stop-on-unknown behavior, and deterministic idempotency replay.

  3. jdjioe5-cpu commented on Oct 17, 2026

    @jdjioe5-cpu

    I have completed the task and submitted pull request #43:
    #43

    Implementation Summary

    1. Forward-Compatible v2 Handlers:
      • In services/indexer/src/handlers/lock-created.ts and released.ts, exported lockCreatedV2 and releasedV2 handler stubs.
      • In services/indexer/src/handlers/index.ts, mapped LockCreated: { 1: lockCreatedV1, 2: lockCreatedV2 } and Released: { 1: releasedV1, 2: releasedV2 } without modifying any existing v1 behavior.
    2. Mixed Schema Version Fixtures:
      • Synthesized services/indexer/test/fixtures/events/mixed-versions.json (15 events) interleaving v1 and v2 events, specifically covering Lock 3 created under v2 and released across v2 (tranche 0) and v1 (tranche 1).
      • Exported mixedFixture and mixedFixtureLocks in services/indexer/test/helpers/fixtures.ts.
    3. Comprehensive Ingest Test Coverage:
      • Verified mixed event streaming resolves Lock 3 to Completed without double-counting, accurately tracking per-tranche released flags.
      • Proved replay idempotency across varying page sizes (1, 4, 100).
      • Enforced Hard Rule 8.2 stop-on-unknown: verified schema_version 0 and 99 halt ingest immediately with UnsupportedEventError, and missing schema_version triggers DecodeError with atomic batch rollback.
    4. Roadmap & Housekeeping:
      • Updated ROADMAP.md row statuses for M2-10 and M2-14 to DONE, refreshed progress snapshots (Phase 3 at 76%, Overall at 41%), and logged Section 17 Changelog.
      • Zero modifications to packages/sdk/src/, migrations, or database schema.

    Local Verification Evidence

    • pnpm test: 15 test files passed, 171 tests passed (100% green).
    • pnpm typecheck: Clean (0 errors across packages).
    • pnpm lint: Biome check clean (74 files checked, 0 errors).
    • pnpm build: Both packages built cleanly.
  4. added
    drips-waveDrips Wave bounty task for Stellar ecosystem contributors
    and removed
    waveScoped for Drips Wave contributors
    drips-waveDrips Wave bounty task for Stellar ecosystem contributors
    on Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:indexerOwnership: kinlock-sdk indexertier/mediumWave tier: standard feature or fix work, a few hours, tests included

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions