Skip to content

Manifest: Partition logical-type decoding panics and mis-scales milliseconds#1847

Description

@badalprasadsingh

Apache Iceberg version

main (development)

Please describe the bug 馃悶

Right now, dataFile.convertAvroValueToIcebergType asserts v.(int64) unchecked on every time and timestamp arm. Per the Avro spec, twmb/avro ignores a logical type invalid for its underlying Avro type and decodes the raw primitive - but the parsed schema still reports the logicalType.

Reading such a manifest panics. Partition() returns no error, so callers cannot recover.

panic: interface conversion: interface {} is int32, not int64
	iceberg-go.(*dataFile).convertAvroValueToIcebergType(manifest.go:2316)
	iceberg-go.(*dataFile).Partition(manifest.go:2370)

Separately, time-millis and timestamp-millis are returned as milliseconds, though iceberg.Time and iceberg.Timestamp count microseconds : 1000脳 too small, silently corrupting partition filtering.

Reachable only via manifests from foreign writers.

Status

Currently short on bandwidth to raise a PR for this fix. Contributions are welcome.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions