Skip to content

test(loop): cover db/csv-import - #30

Merged
drakulavich merged 1 commit into
mainfrom
loop/test-gap-filler-20260801
Aug 1, 2026
Merged

test(loop): cover db/csv-import#30
drakulavich merged 1 commit into
mainfrom
loop/test-gap-filler-20260801

Conversation

@drakulavich

Copy link
Copy Markdown
Owner

Why this module

Weekly test-gap-filler run. src/db/csv-import.ts was the top-priority uncovered module in the loop queue: it has real parsing/mapping logic (semicolon-delimited CSV → sqlite for 10 Oura tables), several error branches, and no dedicated test file. Tests only — no production code was changed.

How it's tested

importFromCSV reads from a directory hard-coded at module-load time from process.env.HOME:

<HOME>/Documents/OpenClaw/projects/oura-ring/data/App Data

The test points HOME at a temp sandbox before importing the module (the CSV_DIR const is captured once, on first import), seeds real semicolon-delimited CSV files per test, and drives them into a fresh bun:sqlite in-memory database seeded with the real schema. HOME is restored in afterAll so nothing leaks into other test files. Boundaries are mocked, module internals are not.

Branches covered

  • Error branch: missing source directory → throws CSV directory not found.
  • Empty / missing files: directory present but no CSV files → all tables empty, daily_sleep: 0 rowsCSV import complete. still logged.
  • Parsing: header→column mapping; header-only file (< 2 lines) → 0 rows; blank/whitespace-only lines skipped; a row with fewer values than headers fills trailing columns with NULL (vals[i] ?? '').
  • num() null-guard: empty and non-numeric cells → NULL (not 0 / NaN); valid numbers pass through.
  • str(): empty cell → NULL.
  • Upsert: duplicate id in the same file → INSERT OR REPLACE keeps the last row, while the per-table log still reports the raw parsed row count.
  • spo2 JSON blob: average extracted from {"average": …}; swallowed JSON.parse on malformed blob → NULL; missing average key → NULL.
  • heartrate: day derived from timestamp.slice(0,10); INSERT OR IGNORE dedups on the (timestamp, source) unique index.

Findings (characterization — pinned, not fixed)

Per loop discipline, suspected issues are pinned against current behaviour with marker comments rather than fixed:

  1. CRLF line endings silently drop the last column. parseCSV splits on '\n' only and never strips '\r', so with a CRLF file the last header becomes "timestamp\r" and the insert (which reads key "timestamp") stores NULL. Non-terminal columns are fine. A real Oura export saved with Windows line endings would lose whatever column is last per file. Suggested fix for a separate PR: text.split(/\r?\n/).
  2. Empty workout.labelNULL here, but '' in the API path. csv-import.ts runs the label through str() (→ NULL), whereas import.ts coerces a null label to '' (asserted in import.test.ts). Same logically-empty label, two different stored values depending on the importer. Inconsistency, not clearly a bug.

Verification

  • bun test — whole suite green: 148 pass / 0 fail (was 133; +15 new).
  • bunx tsc --noEmit — clean.
  • bun install --frozen-lockfile — in sync (exit 0).

State file docs/loops/test-gap-filler-state.md updated in this PR (covered list, queue promotes commands/sync.ts to next, findings recorded).

Do not merge — a human merges.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TZnRjDeAggmffDp1dVP2an


Generated by Claude Code

Add characterization tests for importFromCSV (local Oura App-Data CSV
import into sqlite). Tests only; no production code changed.

Covers directory-missing throw, empty/missing files, header->column
mapping, num()/str() null-guards, header-only and blank-line handling,
short-row trailing NULLs, INSERT OR REPLACE upsert, spo2 JSON average
extraction plus its malformed/missing-key NULL branches, and heartrate
day derivation with INSERT OR IGNORE dedup on (timestamp, source).

Two behaviours pinned as findings (not fixed): CRLF line endings drop
the last column, and an empty workout label stores NULL here vs '' in
the API-sync path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZnRjDeAggmffDp1dVP2an
@drakulavich
drakulavich marked this pull request as ready for review August 1, 2026 13:57
@greptile-apps

greptile-apps Bot commented Aug 1, 2026

Copy link
Copy Markdown

Greptile Summary

This tests-only PR adds dedicated coverage for CSV parsing and database import behavior, while documenting two pre-existing characterization findings.

  • Adds fifteen tests covering missing and empty inputs, value conversion, row mapping, duplicate handling, SpO2 parsing, heartrate derivation, and known CSV edge cases.
  • Updates the test-gap-filler state and advances the next module in the coverage queue.

Confidence Score: 5/5

The PR appears safe to merge because it changes only tests and loop documentation, with no actionable regression in the added coverage.

The new tests exercise existing CSV-import behavior using fixed local fixtures, restore the modified environment after the suite, and do not alter production behavior.

Important Files Changed

Filename Overview
src/db/csv-import.test.ts Adds focused integration-style tests around importFromCSV using isolated temporary CSV fixtures and in-memory SQLite databases; no actionable defect was found.
docs/loops/test-gap-filler-state.md Records the new coverage, advances the queue, and clearly documents the two intentionally pinned pre-existing behaviors.

Reviews (1): Last reviewed commit: "test(loop): cover db/csv-import" | Re-trigger Greptile

@drakulavich
drakulavich merged commit 6583169 into main Aug 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants