Skip to content

fix(lib-storage): strip file-level ContentMD5 from per-part UploadPart commands (#4321) - #7994

Open
mohanrajvenkatesan23-04 wants to merge 1 commit into
aws:mainfrom
mohanrajvenkatesan23-04:fix/issue-4321-lib-storage-content-md5-multipart
Open

fix(lib-storage): strip file-level ContentMD5 from per-part UploadPart commands (#4321)#7994
mohanrajvenkatesan23-04 wants to merge 1 commit into
aws:mainfrom
mohanrajvenkatesan23-04:fix/issue-4321-lib-storage-content-md5-multipart

Conversation

@mohanrajvenkatesan23-04

Copy link
Copy Markdown

Issue

Fixes #4321lib-storage Upload with a user-supplied ContentMD5 and a body larger than 5 MB fails with MalformedXML from S3.

Description

When Upload enters the multipart path, it spreads ...this.params into both UploadPartCommand (around line 311 of Upload.ts) and CompleteMultipartUploadCommand (around line 410). That leaks the file-level ContentMD5 — a single-object MD5 of the entire body — into every per-part command, and S3 rejects the multipart upload with MalformedXML. Maintainer @yenfryherrerafeliz identified the offending location in the issue thread.

This change destructures ContentMD5 out of the spread before constructing the per-part and completion commands:

const { ContentMD5: _fileLevelContentMD5, ...partParams } = this.params;
new UploadPartCommand({ ...partParams, /* per-part fields */ });

The same treatment is applied to CompleteMultipartUploadCommand. The single-part PutObjectCommand path is intentionally unchanged — ContentMD5 is still passed through there because it is valid for a single-object PUT.

Relationship with #7990: This fix is complementary to (and does not conflict with) PR #7990, which fail-fasts on flexible-checksum fields (ChecksumSHA256, ChecksumSHA1, ChecksumCRC32, ChecksumCRC32C, ChecksumCRC64NVME) for multipart uploads. ContentMD5 is the legacy MD5 header — different field, different semantics — so this PR transparently strips it from per-part commands while #7990's strict validation continues to surface clear errors for the flexible-checksum case. Both can land in either order without merge conflicts.

Files changed

  • lib/lib-storage/src/Upload.ts — destructure-and-spread in two command constructions
  • lib/lib-storage/src/Upload.spec.ts — three regression tests

Testing

Added a new describe(\"ContentMD5 handling on multipart uploads (issue #4321)\") block in lib/lib-storage/src/Upload.spec.ts with three tests:

  1. should strip file-level ContentMD5 from UploadPartCommand on multipart uploads — multipart upload with a 34-byte body and mocked MIN_PART_SIZE of 24 bytes; asserts UploadPartCommand is called twice and neither call carries a ContentMD5 property; CompleteMultipartUploadCommand also has none.
  2. should preserve other user params (e.g. Bucket, Key, Metadata) on UploadPartCommand — verifies the destructure only strips ContentMD5, not other fields like Bucket / Key / Metadata / PartNumber / UploadId.
  3. should preserve ContentMD5 on single-part PutObjectCommand uploads (issue #4321) — small body, single-part path; asserts PutObjectCommand IS called with ContentMD5 (single-part path is unaffected).

Local test status (transparency): local Vitest execution is currently blocked on this Windows workstation by a pre-existing workspace-build issue (@aws-sdk/client-s3 dist-*/ outputs not present from Yarn install). tsc --noEmit filtered to lib/lib-storage/ shows only the same pre-existing baseline errors caused by those missing workspace deps — no new errors on the changed lines. CI will validate full test execution end-to-end.

Checklist


generated by AI tools, and reviewed by Mohanraj Venkatesan

…t commands (aws#4321)

When Upload enters the multipart path, spreading `...this.params` into
UploadPartCommand and CompleteMultipartUploadCommand leaks the
file-level ContentMD5 (a single-object MD5 of the entire body) into
every per-part command, causing S3 to reject the upload with
MalformedXML. Destructure ContentMD5 out of the spread before
constructing the part and completion commands. The single-part
PutObjectCommand path is unchanged, since ContentMD5 is valid there.

Fixes aws#4321
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant