fix(source/aws_s3): paginate ListObjects to load >1000 migrations

[EylonLevy]

What

source/aws_s3 loads migrations with a single, non-paginated ListObjects call:

output, err := s.s3client.ListObjects(&s3.ListObjectsInput{ Bucket, Prefix, Delimiter })
for _, object := range output.Contents { ... } // no IsTruncated / pagination

The S3 ListObjects API returns at most 1000 keys per response and sets IsTruncated=true when more exist. The driver never checks IsTruncated and never continues past the first page, so any bucket/prefix holding more than 1000 objects has its remaining keys silently dropped from the migration listing.

Why it matters

A migrations directory with ~500 migrations (an .up.sql + .down.sql per version = ~1000 files) crosses the 1000-key boundary. Once it does, the highest-numbered migration files are the ones truncated away (they sort last lexicographically). The migrator then never sees the newest migrations: Up() returns ErrNoChange and callers get a false "database is up to date" while migrations silently fail to apply.

Fix

Replace the single ListObjects call with ListObjectsPages, accumulating Contents from every page. ListObjectsPages is already part of the s3iface.S3API interface the driver depends on, so there is no interface or signature change. The diff is minimal — the per-object parsing loop is unchanged, just moved into the page callback, and an append error is propagated out after pagination completes.

Test

Added TestLoadMigrationsPaginates, which registers 300 versions (600 objects) behind a fake S3 client that serves them in 50-object pages, then asserts every version — including the highest-numbered one that falls well beyond the first page — is loaded and readable. The fake gains a ListObjectsPages implementation (reusing its existing ListObjects filtering) and a pageSize knob to force the multi-page path. Before this fix the driver would have only seen the first page.

$ go build ./source/aws_s3/... && go vet ./source/aws_s3/... && go test ./source/aws_s3/...
ok  github.com/golang-migrate/migrate/v4/source/aws_s3

golangci-lint run ./source/aws_s3/... reports 0 issues.

[coveralls]

coverage: 54.439% (+0.03%) from 54.412% — EylonLevy:fix/aws-s3-list-pagination into golang-migrate:master

[EylonLevy]

@Fontinalis Hey, can you please take a look? This is an issue for a big project using go migrate