Apache Iceberg version
main (development)
Please describe the bug 馃悶
Problem
io/gocloud bundles the S3 (s3.go), GCS (gcs.go), and Azure Blob (azure.go) FileIO implementations into a single Go package, wired up together via register.go's init() which compiles and links all three cloud SDKs, even if the consuming application only ever talks to one of them.
In practice this means a service that only needs S3 (or an S3-compatible store like Cloudflare R2/MinIO) ends up pulling in the full AWS SDK v2, the full Azure SDK (azcore, azidentity, storage/azblob, go-autorest, microsoft-authentication-library-for-go), and the full GCP stack (cloud.google.com/go, /iam, /monitoring, /storage, GCP-specific OpenTelemetry detectors) none of which it will ever call.
Impact
- Larger dependency surface to audit, license-check, and keep patched.
- Binary size and build time overhead with no functional benefit.
- Risk of version conflicts with an application's existing AWS/GCP/Azure SDK dependencies pinned for unrelated reasons (e.g. we already carry AWS SDK v1 elsewhere, and this adds v2 alongside it, purely as a side effect of needing the S3 FileIO backend).
Request
Could the cloud backends in io/gocloud be split so a consumer can select only what they need? A few possible approaches;
- Separate into subpackages e.g. io/gocloud/s3, io/gocloud/gcs, io/gocloud/azure each with its own init()/registration, so an app can import _ ".../io/gocloud/s3" without touching the other two. This also naturally lets Go's module graph pruning avoid pulling in the unused SDKs' transitive deps.
- Build tags on the existing files (s3.go behind //go:build iceberg_s3, etc.) as a lower-effort alternative if a full package split isn't desirable.
Apache Iceberg version
main (development)
Please describe the bug 馃悶
Problem
io/gocloud bundles the S3 (s3.go), GCS (gcs.go), and Azure Blob (azure.go) FileIO implementations into a single Go package, wired up together via register.go's init() which compiles and links all three cloud SDKs, even if the consuming application only ever talks to one of them.
In practice this means a service that only needs S3 (or an S3-compatible store like Cloudflare R2/MinIO) ends up pulling in the full AWS SDK v2, the full Azure SDK (azcore, azidentity, storage/azblob, go-autorest, microsoft-authentication-library-for-go), and the full GCP stack (cloud.google.com/go, /iam, /monitoring, /storage, GCP-specific OpenTelemetry detectors) none of which it will ever call.
Impact
Request
Could the cloud backends in io/gocloud be split so a consumer can select only what they need? A few possible approaches;