Equality-delete scans fail after dropping the delete key column

[mattfaltyn]

Description

Scanning a format-v2 table fails after an equality-delete key column is dropped from the current schema, even when the scan selects only surviving columns.

Equality delete files store stable table field IDs, and Iceberg retains historical schemas after a field is dropped. The read path currently resolves equality field IDs only through metadata.CurrentSchema(), so a live delete file becomes unreadable as soon as its key is absent from that schema.

Expected behavior

The scan should resolve the equality field ID from schema history, apply the delete, and return only the requested current-schema columns.

This matches the existing rewrite invariant: validateRewriteEqualityFieldIDs accepts equality fields retained only in schema history, and TestReplaceFilesWithDeleteFilesAllowsDroppedEqualityField permits rewriting such a delete file.

Actual behavior

Consuming the Arrow record iterator fails with:

equality delete field ID 1 not found in table schema for .../data/delete.parquet

The failure occurs before equality-delete filtering can run.

Reproduction

1. Create a format-v2 table with id: long (field ID 1) and data: string.
2. Append rows (1, "keep") and (2, "delete").
3. Commit an equality-delete file keyed by field ID 1 that deletes id = 2.
4. Confirm the scan returns only (1, "keep").
5. Drop id with UpdateSchema(...).DeleteColumn([]string{"id"}) and commit.
6. Scan only data and consume the record iterator.

The iterator returns the error above instead of one row containing "keep".

I reproduced this twice on current main at 0bf8092c8586baa279a5378afddb678baafed4a8 with Go 1.26.0 on macOS arm64. The same current-schema lookup is present in v0.6.0.

Root cause

readEqualityDeleteFile calls tableSchema.FindColumnName(fid), while arrowScanner supplies only metadata.CurrentSchema(). Unlike rewrite validation, this path cannot fall back to the table's historical schemas.

Suggested fix

Resolve each equality field ID from current or historical table schemas for physical delete evaluation while preserving the current schema and requested projection for returned records. Add an end-to-end regression test that drops the key and scans a surviving column.

[mattfaltyn]

I'd like to work on this.