Add PostgreSQL database foundation with config, connection, and health check#46
Add PostgreSQL database foundation with config, connection, and health check#46promisszn wants to merge 2 commits into
Conversation
…igrations) Adds the PostgreSQL-backed persistence foundation for the indexer in the `@fundable-indexer/common` package, covering issues Fundable-Protocol#20-Fundable-Protocol#23: Fundable-Protocol#20 Database tooling - Add exact-pinned deps: drizzle-orm 0.45.2, postgres 3.4.9, zod 3.25.67, drizzle-kit 0.31.10 (dev); update bun.lock - Add drizzle.config.ts (postgresql, schema entrypoint, ./migrations out) - Add initial migrations/ directory with Drizzle journal - Add db:generate / db:migrate scripts (package + root indexer:db:* ) - Document the migration workflow in indexer/README.md Fundable-Protocol#21 Config loader + validation - Add zod-based loadConfig in common/src/config; validates required INDEXER_DATABASE_URL, parses numeric INDEXER_PORT / POLL_INTERVAL_MS / START_LEDGER, aggregates problems into a clear ConfigValidationError Fundable-Protocol#22 Connection factory - Add createDbClient in common/src/db wrapping postgres.js + Drizzle, reading from validated config, with an injectable sql factory for tests Fundable-Protocol#23 Health check - Add checkDbHealth running `select 1`, returning typed healthy/unhealthy results without throwing Also scope root ESLint away from the indexer workspace (it is linted by Biome) and ignore generated migration metadata in Biome. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat(indexer): database foundation — config, connection factory, health check, migrations
|
@promisszn Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
📝 WalkthroughWalkthroughAdded validated indexer environment loading, a shared PostgreSQL client and health probe, Drizzle migration configuration and scripts, and workspace linting/documentation updates for the indexer package. ChangesIndexer workspace foundation
Sequence Diagram(s)sequenceDiagram
participant IndexerStartup
participant loadConfig
participant createDbClient
participant checkDbHealth
participant Sql as postgres.js Sql
IndexerStartup->>loadConfig: read env vars
loadConfig-->>IndexerStartup: IndexerConfig
IndexerStartup->>createDbClient: databaseUrl
createDbClient->>Sql: postgres(url, maxConnections)
createDbClient->>createDbClient: drizzle(sql)
IndexerStartup->>checkDbHealth: sql
checkDbHealth->>Sql: select 1
Sql-->>checkDbHealth: success or error
checkDbHealth-->>IndexerStartup: DbHealth
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install failed. For unrecoverable errors, disable the tool in CodeRabbit configuration. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@indexer/common/drizzle.config.ts`:
- Around line 18-20: The Drizzle config is silently defaulting
`dbCredentials.url` to an empty string when `INDEXER_DATABASE_URL` is missing,
which should instead fail immediately. Update the config in `drizzle.config.ts`
to validate `process.env.INDEXER_DATABASE_URL` up front and throw a clear local
error if it is absent, matching the required-URL behavior used by `loadConfig`;
keep the fix centered on the `dbCredentials` setup so `db:generate` and
`db:migrate` fail fast with a clear message.
In `@indexer/common/src/config/env.ts`:
- Around line 24-31: The positive integer parser in positiveIntFromString
currently accepts 0, so tighten the schema to reject zero. Update the zod chain
in env.ts by either adding a value check after the safe-integer refine or
splitting this into a non-negative parser plus a separate strictly-positive
schema, and then use the strictly-positive version for PORT and
POLL_INTERVAL_MS.
In `@indexer/common/src/db/health.ts`:
- Around line 20-24: `checkDbHealth` currently awaits `sql\`select 1\`` without
any deadline, so a stalled database can hang the probe instead of failing fast.
Add a bounded timeout path in `checkDbHealth` (or enforce it via
`createDbClient`/postgres.js config) so the health check returns unhealthy when
the query exceeds the limit, and make sure the timeout behavior is covered by
tests for the `checkDbHealth` and `createDbClient` flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: adc5c05a-7d7c-48f7-8a8e-c27a219f4493
⛔ Files ignored due to path filters (1)
bun.lockis excluded by!**/*.lock
📒 Files selected for processing (18)
eslint.config.mjsindexer/README.mdindexer/biome.jsoncindexer/common/drizzle.config.tsindexer/common/migrations/README.mdindexer/common/migrations/meta/_journal.jsonindexer/common/package.jsonindexer/common/src/config/env.test.tsindexer/common/src/config/env.tsindexer/common/src/config/index.tsindexer/common/src/db/client.test.tsindexer/common/src/db/client.tsindexer/common/src/db/health.test.tsindexer/common/src/db/health.tsindexer/common/src/db/index.tsindexer/common/src/db/schema.tsindexer/common/src/index.tspackage.json
| dbCredentials: { | ||
| url: process.env.INDEXER_DATABASE_URL ?? "", | ||
| }, |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Fail fast when INDEXER_DATABASE_URL is missing.
Falling back to "" here bypasses the same required-URL contract enforced by loadConfig and turns a simple misconfiguration into a downstream Drizzle CLI failure. Throwing locally gives db:generate/db:migrate a much clearer error path.
Suggested fix
+const databaseUrl = process.env.INDEXER_DATABASE_URL;
+if (!databaseUrl) {
+ throw new Error("INDEXER_DATABASE_URL is required for Drizzle Kit commands");
+}
+
export default defineConfig({
dialect: "postgresql",
schema: "./src/db/schema.ts",
out: "./migrations",
dbCredentials: {
- url: process.env.INDEXER_DATABASE_URL ?? "",
+ url: databaseUrl,
},
strict: true,
verbose: true,
});🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@indexer/common/drizzle.config.ts` around lines 18 - 20, The Drizzle config is
silently defaulting `dbCredentials.url` to an empty string when
`INDEXER_DATABASE_URL` is missing, which should instead fail immediately. Update
the config in `drizzle.config.ts` to validate `process.env.INDEXER_DATABASE_URL`
up front and throw a clear local error if it is absent, matching the
required-URL behavior used by `loadConfig`; keep the fix centered on the
`dbCredentials` setup so `db:generate` and `db:migrate` fail fast with a clear
message.
| /** A positive integer parsed from an environment string. */ | ||
| const positiveIntFromString = z | ||
| .string() | ||
| .trim() | ||
| .min(1, "must not be empty") | ||
| .regex(/^\d+$/, "must be a positive integer") | ||
| .transform((value) => Number.parseInt(value, 10)) | ||
| .refine((value) => Number.isSafeInteger(value), "must be a safe integer"); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
positiveIntFromString allows 0, which is not a positive integer.
The regex /^\d+$/ and subsequent parseInt accept "0". Zero is not a positive integer, and it is semantically invalid for PORT (ephemeral bind) and POLL_INTERVAL_MS (busy-loop). Either rename this to nonNegativeIntFromString and add a separate strictly-positive schema for PORT/POLL_INTERVAL_MS, or add .refine((v) => v > 0, "must be a positive integer") after the safe-integer check.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@indexer/common/src/config/env.ts` around lines 24 - 31, The positive integer
parser in positiveIntFromString currently accepts 0, so tighten the schema to
reject zero. Update the zod chain in env.ts by either adding a value check after
the safe-integer refine or splitting this into a non-negative parser plus a
separate strictly-positive schema, and then use the strictly-positive version
for PORT and POLL_INTERVAL_MS.
| export async function checkDbHealth(sql: Sql): Promise<DbHealth> { | ||
| const start = Date.now(); | ||
| try { | ||
| await sql`select 1`; | ||
| return { healthy: true, latencyMs: Date.now() - start }; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
Does postgres.js v3.4.9 support per-query timeout or AbortSignal cancellation for tagged-template queries, and what is the recommended way to bound a simple sql`select 1` health probe?
💡 Result:
As of version 3.4.9, postgres.js does not provide native support for per-query AbortSignal cancellation or a per-query timeout option for tagged-template queries [1][2][3]. The timeout functionality in the library is primarily designed for connection-level lifecycle management (e.g., idle_timeout, max_lifetime, and connect_timeout) [1][4]. For a simple health probe like sqlselect 1``, the recommended approach is to execute the query within a try-catch block [5]. Since the library manages connections seamlessly, a failed health probe will throw an error if the database is unreachable or the connection cannot be established, which is the standard way to detect health status [5]. If you require a hard timeout for your health check to prevent the process from hanging, you should wrap the query execution in a Promise that races against a standard setTimeout: const healthCheck = async (sql) => { const timeout = new Promise((_, reject) => setTimeout( => reject(new Error('Health check timed out')), 5000)); return Promise.race([ sqlselect 1, timeout ]); }; This pattern ensures that your application does not wait indefinitely if the database becomes unresponsive [5][6]. Note that while the query promise might "time out" from the perspective of your application logic, the underlying database driver may still be managing the connection, so ensure your overall pool configuration (such as idle_timeout) is tuned for your environment's needs [1][4][7].
Citations:
- 1: https://github.com/porsager/postgres
- 2: https://github.com/porsager/postgres/blob/master/README.md
- 3: https://npmx.dev/package/postgres/v/%5E3.4.9
- 4: https://registry.npmjs.org/postgres
- 5: The status field must be present porsager/postgres#515
- 6: [Question] - Best way to check pool health? brianc/node-postgres#3208
- 7: Document global var / setting
idle_timeoutsolutions for long-running development processes eg. Next.js porsager/postgres#119
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the health check and postgres client setup
git ls-files | rg '(^|/)indexer/common/src/db/health\.ts$|(^|/)postgres|(^|/)db/|(^|/)sql|(^|/)health'
echo '--- health.ts ---'
cat -n indexer/common/src/db/health.ts
echo '--- postgres-related files ---'
git ls-files | rg 'postgres|sql' | sed -n '1,120p'Repository: Fundable-Protocol/Backend
Length of output: 1705
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Search for any timeout/cancellation handling around postgres.js usage
rg -n --hidden --glob '!**/node_modules/**' --glob '!**/dist/**' --glob '!**/build/**' \
'connect_timeout|idle_timeout|max_lifetime|statement_timeout|AbortSignal|timeout|Promise\.race|select 1|postgres\(' \
indexer common . 2>/dev/null | sed -n '1,220p'Repository: Fundable-Protocol/Backend
Length of output: 809
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- client.ts ---'
sed -n '1,120p' indexer/common/src/db/client.ts
echo '--- health.test.ts ---'
sed -n '1,220p' indexer/common/src/db/health.test.ts
echo '--- docker-compose.yml (health-related section) ---'
sed -n '1,80p' docker-compose.ymlRepository: Fundable-Protocol/Backend
Length of output: 4527
Bound the health probe with a timeout. createDbClient() only sets max on postgres.js, and checkDbHealth() awaits sql\select 1`` directly, so a stalled connection can leave the probe stuck instead of returning unhealthy. Add a bounded timeout path here (or enforce one in the client config) and cover the failure mode in tests.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@indexer/common/src/db/health.ts` around lines 20 - 24, `checkDbHealth`
currently awaits `sql\`select 1\`` without any deadline, so a stalled database
can hang the probe instead of failing fast. Add a bounded timeout path in
`checkDbHealth` (or enforce it via `createDbClient`/postgres.js config) so the
health check returns unhealthy when the query exceeds the limit, and make sure
the timeout behavior is covered by tests for the `checkDbHealth` and
`createDbClient` flow.
1 participant
Summary
Adds the PostgreSQL-backed persistence foundation for the indexer in the
@fundable-indexer/commonpackage. This is tooling + infrastructure only — no domain tables are defined yet (that's deferred to later scoped issues).Implements issues #20, #21, #22, #23.
#20 — Database tooling
drizzle-orm@0.45.2,postgres@3.4.9,zod@3.25.67,drizzle-kit@0.31.10(dev);bun.lockupdatedindexer/common/drizzle.config.ts(postgresql dialect,src/db/schema.tsentrypoint,./migrationsout dir)migrations/directory with Drizzle journaldb:generate/db:migrate(incommon) andindexer:db:generate/indexer:db:migrate(root)indexer/README.md#21 — Config loader + validation
loadConfig()incommon/src/config— zod-validated, accepts an env record for testabilityINDEXER_DATABASE_URL; parses numericINDEXER_PORT/POLL_INTERVAL_MS/START_LEDGER; defaultsINDEXER_LOG_LEVELConfigValidationError; treats blank values as unset#22 — Connection factory
createDbClient()incommon/src/db— wraps postgres.js + Drizzle, reads from validated config, lazy (no socket until first query), injectablesqlfactory for tests,close()for graceful shutdown#23 — Health check
checkDbHealth()runsselect 1and returns a typedhealthy/unhealthyresult (with latency + error) without throwingIncidental
indexer/**(the workspace is linted by Biome) — prevents the two linters' differing global assumptions from conflictingmigrations/metadataVerification
All six gate commands pass:
bun run type-checkbun run testbun run lintbun run indexer:type-checkbun run indexer:testbun run indexer:lintbun run indexer:db:generatealso verified — reports "no schema changes" (expected, no tables yet).Closes #20
Closes #21
Closes #22
Closes #23
Summary by CodeRabbit
New Features
Documentation