Apache Iceberg version
main (development)
Please describe the bug 馃悶
Right now, dataFile.convertAvroValueToIcebergType asserts v.(int64) unchecked on every time and timestamp arm. Per the Avro spec, twmb/avro ignores a logical type invalid for its underlying Avro type and decodes the raw primitive - but the parsed schema still reports the logicalType.
Reading such a manifest panics. Partition() returns no error, so callers cannot recover.
panic: interface conversion: interface {} is int32, not int64
iceberg-go.(*dataFile).convertAvroValueToIcebergType(manifest.go:2316)
iceberg-go.(*dataFile).Partition(manifest.go:2370)
Separately, time-millis and timestamp-millis are returned as milliseconds, though iceberg.Time and iceberg.Timestamp count microseconds : 1000脳 too small, silently corrupting partition filtering.
Reachable only via manifests from foreign writers.
Status
Currently short on bandwidth to raise a PR for this fix. Contributions are welcome.
Apache Iceberg version
main (development)
Please describe the bug 馃悶
Right now,
dataFile.convertAvroValueToIcebergTypeassertsv.(int64)unchecked on every time and timestamp arm. Per the Avro spec,twmb/avroignores a logical type invalid for its underlying Avro type and decodes the raw primitive - but the parsed schema still reports thelogicalType.Reading such a manifest panics.
Partition()returns no error, so callers cannot recover.panic: interface conversion: interface {} is int32, not int64 iceberg-go.(*dataFile).convertAvroValueToIcebergType(manifest.go:2316) iceberg-go.(*dataFile).Partition(manifest.go:2370)Separately,
time-millisandtimestamp-millisare returned as milliseconds, thoughiceberg.Timeandiceberg.Timestampcount microseconds : 1000脳 too small, silently corrupting partition filtering.Reachable only via manifests from foreign writers.
Status
Currently short on bandwidth to raise a PR for this fix. Contributions are welcome.