feat: extract metadata to compiled_contracts_metadata

[peterlodri-sec]

Fixes #2924 by deduplicating JSON metadata into a separate compiled_contracts_metadata side table. This dramatically reduces table bloat in sourcify_matches caused by multiple networks sharing identical compiled contract JSON structures. The reads are gracefully degraded via COALESCE, allowing zero-downtime database migration.

[peterlodri-sec]

Why: Storing massive metadata blobs directly on the match relationships created tremendous DB bloat across multiple identical proxies and multichain deployments.
What: Extracted the metadata object to a side-table (compiled_contracts_metadata) linked purely by compilation_id, establishing a 1-to-1 canonical lookup that deduplicates across all variants.
Where: Schema additions in sourcify-database.sql and conditional LEFT JOIN query optimisations in Database.ts.

I've also written an in-depth retrospective blog post on this optimization: Solving Database Bloat in Sourcify with Metadata Deduplication

[peterlodri-sec]

It might look like there are no explicit CREATE INDEX statements for the new table, but it's actually perfectly indexed for our access patterns.

In PostgreSQL, declaring a column as a PRIMARY KEY automatically generates a unique B-Tree index under the hood. Since our schema defines the table as:

    CREATE TABLE compiled_contracts_metadata (
      compilation_id uuid PRIMARY KEY REFERENCES compiled_contracts(id) ON DELETE CASCADE,
      metadata json NOT NULL
    );

That implicit index on compilation_id perfectly covers the only two ways the database interacts with this side-table:

• 1. The Read Path (LEFT JOIN): When we run ON compiled_contracts_metadata.compilation_id = verified_contracts.compilation_id, Postgres natively uses the primary key index for lightning-fast lookups.
• 2. The Write/Cleanup Path (ON DELETE CASCADE): When a row in compiled_contracts is deleted, Postgres must find and delete the orphaned metadata. This reverse lookup also relies on the primary key index to execute efficiently.

I intentionally didn't add a GIN or BTREE index on the metadata JSON blob itself because Sourcify's architecture never queries inside that payload (e.g., we never run WHERE metadata->>'language' = 'Solidity').
--> So keeping it to just the single implicit primary key index is the leanest, most optimized state for this table, I think :)

• peter

[peterlodri-sec]

Fix for test-server CI Failure (could not identify an equality operator for type json)

In commit bc2274c, fixed the PostgreSQL runtime error that caused test-server in CircleCI to fail during getContractEndpoint lookups.

Root Cause:

In services/server/src/server/services/utils/Database.ts:L216-L229, the groupByClause builder had:

const groupByClause =
  // ...
  properties.includes("error_signatures") ||
  properties.includes("metadata") // <-- Issue here
    ? `GROUP BY sourcify_matches.id,
    // ...
    recompiled_creation_code.code_hash${
      properties.includes("metadata")
        ? ",\n        compiled_contracts_metadata.metadata"
        : ""
    }`

1. PostgreSQL json Type Constraint: Unlike jsonb, PostgreSQL's native json data type does not support standard equality operators (=), which caused queries requesting metadata to fail with ERROR 42883: could not identify an equality operator for type json.
2. 1-to-1 Relationship: compiled_contracts_metadata is keyed 1:1 on compilation_id, so joining it via LEFT JOIN does not produce duplicate rows and does not require grouping.

Resolution:

Removed metadata from the groupByClause condition and the GROUP BY projection in Database.ts. The field continues to be selected seamlessly via the LEFT JOIN.

[marcocastignoli]

Thanks for the work on this! Superseded by #2941, which keeps the same table shape and first-wins semantics but restructures the rollout (backfill moved out of the migration into a batched script, no COALESCE read path, based on staging). See the discussion there for details.