Skip to content

Default ledger is empty and publication fails with missing schema_meta on bestia #140

Closed

Description

@lukemaj

What: Observer publication on bestia fails because the default ledger is zero bytes and has no schema tables.
Why: The populated ledger seen earlier in this task is no longer present at the default path; the responsible writer and cause are unknown.
So what: Investigate the empty-ledger transition and recover the default reporting path; AgentsMD #257 uses an isolated native-session report meanwhile.

Outcome and contribution to the Objective

Preserve durable native records and reproducible local reporting, with missing coverage disclosed rather than counted as zero.

Evidence

Agent Observer 1.4.0 completed a selected-source Codex sync during toolboxmd/agentsmd#257 delivery. A later publish --task toolboxmd/agentsmd#251 --repo toolboxmd/agentsmd --pr 257 printed:

no such table: schema_meta
Reports and lookups require an existing ledger; run agent-observer sync first.

Read-only Path.stat() and SQLite queries confirmed that ~/.local/state/agent-observer/observer.db is zero bytes, sqlite_master contains zero tables, and no schema_meta table exists. AGENT_OBSERVER_DB is unset. A nonempty observer.db.schema-2.bak remains; it was not restored, migrated or modified. Cause and data recoverability are unverified; this does not establish corruption or permanent loss.

Primary sources checked: installed CLI launcher, --version and --help, db.py default-path code, CLI error handling, file metadata and read-only schema, README ownership/publication sections, and open Issues searched for schema_meta and empty-ledger/database terms.

A separate temporary ledger initialized with the current CLI imported the existing native source successfully. Its explicitly session-scoped publication updated toolboxmd/agentsmd#257 (comment) without modifying the default ledger or inventing task ownership.

Acceptance criteria and proof

  • Identify the actual transition/writer using direct evidence.
  • Preserve existing recovery evidence and establish an authorized recovery path.
  • Add a deterministic regression for the confirmed cause when practical.
  • Verify default reporting against a populated ledger with honest coverage limits.

Non-goals and blockers

Cause is unknown. Filing this observation does not authorize an in-place restore, destructive backup replacement, deployment, installation or guessed historical attribution. Preserve ledger/sidecar and backup evidence before recovery writes.

Elon record

  • Requirements and who asked: The user's standing AgentsMD rule requires filing discovered problems in their owning repository; requested Observer publication exposed this failure.
  • Deleted: Unapproved default-ledger restoration, manual ownership repair and claims of permanent data loss.
  • Bottleneck: The default ledger has no schema, although native evidence still imports into a separate ledger.
  • Checked myself: Ran publication, inspected installed CLI version/path and primary error handling, checked file size/schema read-only, and verified isolated import and session publication.

Activity

  1. lukemaj commented on Oct 19, 2026

    @lukemaj
    ContributorAuthor

    Cause found (2026-10-19, Warsaw time): the zero-byte ledger is not an Observer defect. During the approved 1.4.2 install on bestia, the 1.4.2 sync process died at 18:59 with SIGSEGV; the kernel journal ties that PID to Oops: invalid opcode in btrfs_readahead (kernel 7.2.5-3-omarchy). The installer's rollback then started copying the verified pre-upgrade backup (~/.local/state/agent-observer/backup-2026-10-19-pre-1.4.2/observer.db, 700 MB, integrity ok) over the live ledger; that copy is stuck in uninterruptible kernel I/O, which left the live file at zero bytes. The sync timer is disabled so nothing writes meanwhile. Machine-level recovery is tracked in lukemaj/promachos#27. Leave this Issue open only if a real Observer change is needed after recovery (for example a sync that refuses to write into a zero-byte ledger).

  2. lukemaj commented on Oct 20, 2026

    @lukemaj
    ContributorAuthor

    Resolved on bestia, 2026-10-20 09:25 Warsaw. The empty ledger came from the 2026-10-19 host crash (btrfs oops during the 1.4.2 sync, then a rollback copy stuck in uninterruptible I/O; lukemaj/promachos#27), not from Agent Observer code. After the host recovered, the live ledger has integrity ok and a complete schema_meta. With 1.5.0 installed, two service syncs passed (19114 sessions), and agent-observer publish --task toolboxmd/agentsmd#257 --repo toolboxmd/agentsmd --pr 257 --dry-run exits 0 with a summary instead of the schema_meta error. Details: runbook bestia/CHANGES.md (9c6e960).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions