Skip to content

feat(risk): risk scoring aggregator and hardened migration verification - #458

Open
billcode124 wants to merge 2 commits into
ASTROIDX556:mainfrom
billcode124:feat/447-risk-aggregator-449-migration-verification
Open

billcode124 wants to merge 2 commits into
ASTROIDX556:mainfrom
billcode124:feat/447-risk-aggregator-449-migration-verification

Conversation

@billcode124

Copy link
Copy Markdown
Contributor

Summary

Adds a risk scoring aggregator that combines velocity checks, destination address reputation and amount anomaly detection into one composite 0–100 score for incoming agent transactions, and hardens the Prisma migration verification so schema drift and missing migrations are caught before a pull request is merged.

Closes #447
Closes #449

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Documentation
  • CI / tooling

Why

#447 — risk scoring. The risk engine could score a transaction, but the transaction pipeline never persisted the assessment against the transaction it decided on, and the score was built from heuristics that could not see the transaction history: there was no destination reputation and no amount anomaly detection. A high-risk transaction was also only routed to a human when it reached the CRITICAL band, even though the proposal flow already treated HIGH as a dual approval.

The aggregator now owns the whole flow:

  • RiskAggregatorService.assess() gathers three signals — velocity (count and value moved in the trailing 24 h), destination reputation (failure rate and depth of the organization's payment history with the address) and the amount baseline (the wallet's own average over 90 days) — and scores them with the pure risk engine.
  • Two new pluggable rules (recipientReputation, amountAnomaly) plus an extended velocity rule; the weights still sum to 100 and are documented in src/modules/risk/risk.types.ts and docs/risk-scoring.md.
  • Two explicit thresholds replace the implicit band check: reviewThreshold (51) routes a transaction into the approval workflow, blockThreshold (81) stops it from ever auto-executing.
  • RiskAggregatorService.record() persists the assessment against the transaction (upsert on the unique transactionId) and emits risk.evaluated plus — for high-risk transactions only — risk.alert, which the notification listener turns into RISK_ALERT notifications. Both events land in the append-only domain event ledger, so the assessment is part of the audit trail.
  • The pipeline scores before the transaction row is created (the score decides whether a human is needed) and persists right after, so the stored assessment is always the one the decision was based on. The signal queries run in parallel and the scoring itself is pure, so transaction processing is not blocked beyond the three extra indexed reads.

The redundant transaction.created risk listener was removed: it re-scored with placeholder inputs and would have overwritten the authoritative assessment.

#449 — migration verification. The script already replayed the history against a shadow database, but drift was only detectable with a database at hand, a missing Prisma CLI surfaced as an opaque npx error, an unreachable shadow database was reported as invalid SQL, and nothing validated the path the deployment tooling actually takes.

  • New schema coverage tier: the models, columns and enums of schema.prisma are compared against the migration history, so a schema change committed without its migration is reported as drift — with the exact prisma migrate dev command to fix it — and it runs without a database.
  • New prerequisite tier: missing tools and a missing Prisma CLI are reported with actionable messages.
  • The database tier now also applies the history to an empty schema the way the deployment tooling does and requires prisma migrate status to report an up-to-date database. Unreachable databases are reported separately from invalid SQL.
  • Safety: the database tier refuses to run when SHADOW_DATABASE_URL is the configured application database, and always points Prisma at the shadow database, so it can never touch a real one.
  • Robust Bash: set -euo pipefail, arrays expanded in a way that also works on bash 3.2, a temporary directory that is always cleaned up, and a summary written to the GitHub step summary.
  • CI: named job, timeouts, concurrency cancellation, a resolved base ref for the history checks (including the all-zero "before" commit of a first push), and the verification log uploaded as an artifact on failure.

Testing

  • src/modules/risk/risk-aggregator.service.spec.ts — signal gathering, composite scoring, threshold boundaries, persistence and event emission (18 tests).
  • src/modules/risk/risk.engine.spec.ts — the new rules and the review/block thresholds.
  • src/modules/risk/risk.repository.spec.ts — the upsert and the signal queries.
  • src/modules/transactions/tests/transaction.service.spec.ts — a high-risk transaction is routed into the approval workflow instead of executing.
  • src/database/verify-migrations-script.spec.ts — schema coverage (missing model, column and enum), the relation-field handling and the shadow database safety guard (27 tests).

Checklist

  • npm run build passes
  • npm test passes
  • npm run lint passes
  • npm run typecheck passes
  • npm run db:verify passes (also with SHADOW_DATABASE_URL and MIGRATION_BASE_REF set)
  • No secrets added to tracked files
  • PR description explains the why, not just the what

…cation

Closes ASTROIDX556#447
Closes ASTROIDX556#449

Risk scoring aggregator (ASTROIDX556#447)

- New RiskAggregatorService gathers the raw signals of an incoming agent
  transaction (velocity, destination reputation, amount baseline) and turns
  them into a single composite 0-100 score with the pure risk engine.
- Two new pluggable rules: destination reputation (failure rate and depth of
  the payment history) and amount anomaly (size relative to the wallet's own
  average). The velocity rule now considers the value moved in the window,
  not only the number of transactions.
- Weights always sum to 100 and are documented in risk.types.ts and
  docs/risk-scoring.md. Two explicit thresholds drive the routing: a score at
  or above the review threshold (51) requires a human approval, a score at or
  above the block threshold (81) can never auto-execute.
- The transaction pipeline scores before the row is created and persists the
  assessment against the transaction right after, so the stored assessment is
  the one the decision was based on. High-risk assessments additionally emit
  a risk alert for the downstream notification workers; every assessment
  emits a risk evaluation event that lands in the audit trail.
- The redundant transaction-created listener was removed: the aggregator is
  now the single owner of the transaction risk assessment.

Migration verification and drift detection (ASTROIDX556#449)

- New schema coverage tier: the models, columns and enums of schema.prisma
  are compared against the migration history, so a schema change committed
  without its migration is detected without a database.
- New prerequisite tier with actionable diagnostics for missing tools and a
  missing Prisma CLI.
- The shadow database tier now also applies the history to an empty schema
  the way the deployment tooling does and checks that no migration is left
  pending; unreachable databases are reported separately from invalid SQL.
- Safety: the database tier refuses to run when SHADOW_DATABASE_URL is the
  configured application database and always points Prisma at the shadow
  database.
- Robust Bash: set -euo pipefail, empty arrays expanded in a way that works
  on bash 3.2, a temporary directory that is always cleaned up and a summary
  that is written to the GitHub step summary.
- CI: dedicated job names, timeouts, concurrency cancellation, a resolved
  base ref for the history checks and the verification log uploaded as an
  artifact on failure.
@drips-wave

drips-wave Bot commented Oct 21, 2026

Copy link
Copy Markdown

@billcode124 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@Cjay-Cyber-2

Copy link
Copy Markdown
Contributor

CI is failing after the conflict resolution. The failing test is src/modules/transactions/tests/transaction.service.spec.ts (routes a high-risk transaction into the approval workflow instead of executing it): this.budgets.withAgentBudgetLimit is not a function at transaction.service.ts:245, because the test mock does not include the method now called by TransactionService. Update the mock/provider and its expectations to match the current BudgetService API, then rerun the full CI suite. I’m holding the merge until this is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Database Migration Verification and Drift Check CI Enhancement Implement Risk Scoring Aggregator for Incoming Agent Transactions

2 participants