Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion contracts/fixtures/AggregateGreeksSchema.json
Original file line number Diff line number Diff line change
@@ -1 +1,34 @@
{"delta":0.5,"gamma":0.01,"theta":-0.02,"vega":0.1}
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "AggregateGreeks",
"description": "Vault-level aggregate Greeks computed from open written positions via the shared pricing library. Consumed by the delta-hedging policy to decide whether exposure has drifted outside the configured band.",
"type": "object",
"required": ["delta", "gamma", "theta", "vega"],
"properties": {
"delta": {
"type": "number",
"description": "Aggregate portfolio delta (net directional exposure)."
},
"gamma": {
"type": "number",
"description": "Aggregate portfolio gamma (delta sensitivity to underlying moves)."
},
"theta": {
"type": "number",
"description": "Aggregate portfolio theta (time decay)."
},
"vega": {
"type": "number",
"description": "Aggregate portfolio vega (volatility sensitivity)."
}
},
"additionalProperties": false,
"examples": [
{
"delta": 0.5,
"gamma": 0.01,
"theta": -0.02,
"vega": 0.1
}
]
}
66 changes: 66 additions & 0 deletions docs/LEDGER.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,69 @@ a hardware-backed signing path with no seed phrase in the browser.
- Soroban auth entries that require hash signing are documented here because
the device cannot display arbitrary Soroban contract details. Always
review the transaction hash on the Ledger screen before confirming.

## Vault Data Warehouse & Analytics ETL

The vault analytics warehouse is a separate, read-only analytical store. It
is populated by scheduled ETL jobs that extract from the vault event indexer,
the historical epoch ledger, and the vault registry, then transform into
modeled marts. Dashboards and reporting exports query the warehouse — never
the operational database.

### Schemas

| Table | Grain | Key columns |
| --- | --- | --- |
| `raw_events` | one row per indexer event | `event_id`, `ledger`, `cursor`, `contract`, `topic`, `payload`, `ingested_at` |
| `vault_registry_snapshots` | vault × snapshot time | `vault_id`, `snapshot_at`, `strategy`, `strike_bucket`, `status` |
| `epoch_facts` | one row per epoch | `vault_id`, `epoch_id`, `opened_at`, `closed_at`, `nav_open`, `nav_close`, `premium_collected`, `fees`, `realized_pnl` |
| `flow_facts` | one row per deposit/withdrawal/queue event | `flow_id`, `vault_id`, `epoch_id`, `kind` (`deposit`/`withdrawal`/`queue_enter`/`queue_exit`), `amount`, `shares`, `occurred_at` |
| `share_price_series` | vault × timestamp | `vault_id`, `observed_at`, `share_price` |
| `redemption_queue_depth` | vault × timestamp | `vault_id`, `observed_at`, `depth`, `pending_shares` |

### ETL & Watermarking

- Jobs are **idempotent and replayable**. Each run reads the persisted
watermark (`etl_watermarks.job`, `etl_watermarks.cursor`) and processes
events strictly after it, upserting by natural key (`event_id`, `epoch_id`,
`flow_id`) so replays never double count.
- The watermark advances only after a successful transform + data-quality
gate. A failed gate leaves the watermark untouched so the next run retries
the same window.
- **Backfills and restatements**: pass an explicit `--from-cursor` to replay
a historical window. Admin epoch corrections are accepted as restatements —
the affected `epoch_facts` row is replaced (not appended) and downstream
marts are recomputed for that epoch.
- **Reorgs / cursor regressions**: if the indexer cursor moves backwards
(chain restart), the job rewinds the watermark to the earliest affected
ledger and reprocesses; upserts make this safe.

### Data-Quality Gates

Each transform run asserts, and **fails loudly** (non-zero exit, no mart
writes) when any check fails:

- **Row counts**: raw events consumed > 0 for a non-empty window; no epoch
with zero flows when NAV moved.
- **Monotonic NAV timestamps**: `nav_close` timestamps strictly increase per
vault; out-of-order snapshots abort the run.
- **Flow-to-epoch reconciliation**: sum of `flow_facts.amount` per epoch
reconciles against the on-chain ledger totals for that epoch within
tolerance; mismatches abort rather than writing bad aggregates.

### Backfill Runbook

1. Identify the target window (cursor range or epoch range).
2. Run `etl --job <name> --from-cursor <cursor>` against a staging warehouse.
3. Inspect data-quality gate output; fix source data or transform if it fails.
4. Re-run against production; the watermark advances only on success.
5. For restatements, replay the affected epoch and confirm `epoch_facts`
and dependent marts reflect the corrected values.

### Access & PII

- Dashboards and exports use a **read-only** warehouse role; ETL uses a
separate write role. Credentials live in the job runner's secret store,
never in the app runtime.
- No depositor-identifying fields are stored beyond what the data retention
policy permits; flows are keyed by opaque `flow_id`.
28 changes: 28 additions & 0 deletions src/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ const parsed = parseEnv({
NEXT_PUBLIC_CONTRACT_TIMELOCK_MAINNET: process.env.NEXT_PUBLIC_CONTRACT_TIMELOCK_MAINNET,
NEXT_PUBLIC_CONTRACT_DEPLOYER_TESTNET: process.env.NEXT_PUBLIC_CONTRACT_DEPLOYER_TESTNET,
NEXT_PUBLIC_CONTRACT_DEPLOYER_MAINNET: process.env.NEXT_PUBLIC_CONTRACT_DEPLOYER_MAINNET,
OTEL_EXPORTER_OTLP_ENDPOINT: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
OTEL_SERVICE_NAME: process.env.OTEL_SERVICE_NAME,
OTEL_TRACES_SAMPLER: process.env.OTEL_TRACES_SAMPLER,
OTEL_TRACES_SAMPLER_ARG: process.env.OTEL_TRACES_SAMPLER_ARG,
SOROBAN_RPC_URLS: process.env.SOROBAN_RPC_URLS,
SOROBAN_RPC_MAX_CONCURRENCY: process.env.SOROBAN_RPC_MAX_CONCURRENCY,
SOROBAN_RPC_MAX_ATTEMPTS: process.env.SOROBAN_RPC_MAX_ATTEMPTS,
Expand Down Expand Up @@ -101,6 +105,30 @@ export function contractEnvVarName(key: string): string {
return `NEXT_PUBLIC_CONTRACT_${key.toUpperCase()}_${stellarNetworkSuffix}`;
}

/**
* Distributed tracing configuration (issue #299).
*
* Traces are exported to the existing observability backend via OTLP. The
* sampler is parent-based with a ratio so that error traces are never dropped:
* the tail-sampling guarantee is enforced by the collector, while the head
* sampler here keeps the hot read path cheap. `OTEL_TRACES_SAMPLER_ARG` is the
* ratio for non-error spans; error spans are always recorded.
*/
export const tracingConfig = {
/** OTLP collector endpoint; unset disables export (spans still created locally). */
exporterEndpoint: parsed.OTEL_EXPORTER_OTLP_ENDPOINT,
/** Service name reported to the trace backend. */
serviceName: parsed.OTEL_SERVICE_NAME ?? "vault-frontend",
/** Head sampler strategy; defaults to parent-based ratio sampling. */
sampler: parsed.OTEL_TRACES_SAMPLER ?? "parentbased_traceidratio",
/** Ratio for non-error spans (0–1). Error traces are always sampled. */
samplerArg: Number.parseFloat(parsed.OTEL_TRACES_SAMPLER_ARG ?? "0.1"),
/** Header carrying the correlation ID across BFF → backend → indexer hops. */
correlationHeader: "x-correlation-id",
/** Maximum accepted length of an inbound correlation ID (bytes). */
maxCorrelationIdLength: 128,
} as const;

/**
* Soroban RPC resilience configuration (issue #282).
*
Expand Down
Loading