fix(lib-storage): strip file-level ContentMD5 from per-part UploadPart commands (#4321) - #7994
Open
mohanrajvenkatesan23-04 wants to merge 1 commit into
Conversation
…t commands (aws#4321) When Upload enters the multipart path, spreading `...this.params` into UploadPartCommand and CompleteMultipartUploadCommand leaks the file-level ContentMD5 (a single-object MD5 of the entire body) into every per-part command, causing S3 to reject the upload with MalformedXML. Destructure ContentMD5 out of the spread before constructing the part and completion commands. The single-part PutObjectCommand path is unchanged, since ContentMD5 is valid there. Fixes aws#4321
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
Fixes #4321 —
lib-storageUploadwith a user-suppliedContentMD5and a body larger than 5 MB fails withMalformedXMLfrom S3.Description
When
Uploadenters the multipart path, it spreads...this.paramsinto bothUploadPartCommand(around line 311 ofUpload.ts) andCompleteMultipartUploadCommand(around line 410). That leaks the file-levelContentMD5— a single-object MD5 of the entire body — into every per-part command, and S3 rejects the multipart upload withMalformedXML. Maintainer @yenfryherrerafeliz identified the offending location in the issue thread.This change destructures
ContentMD5out of the spread before constructing the per-part and completion commands:The same treatment is applied to
CompleteMultipartUploadCommand. The single-partPutObjectCommandpath is intentionally unchanged —ContentMD5is still passed through there because it is valid for a single-object PUT.Relationship with #7990: This fix is complementary to (and does not conflict with) PR #7990, which fail-fasts on flexible-checksum fields (
ChecksumSHA256,ChecksumSHA1,ChecksumCRC32,ChecksumCRC32C,ChecksumCRC64NVME) for multipart uploads.ContentMD5is the legacy MD5 header — different field, different semantics — so this PR transparently strips it from per-part commands while #7990's strict validation continues to surface clear errors for the flexible-checksum case. Both can land in either order without merge conflicts.Files changed
lib/lib-storage/src/Upload.ts— destructure-and-spread in two command constructionslib/lib-storage/src/Upload.spec.ts— three regression testsTesting
Added a new
describe(\"ContentMD5 handling on multipart uploads (issue #4321)\")block inlib/lib-storage/src/Upload.spec.tswith three tests:should strip file-level ContentMD5 from UploadPartCommand on multipart uploads— multipart upload with a 34-byte body and mockedMIN_PART_SIZEof 24 bytes; assertsUploadPartCommandis called twice and neither call carries aContentMD5property;CompleteMultipartUploadCommandalso has none.should preserve other user params (e.g. Bucket, Key, Metadata) on UploadPartCommand— verifies the destructure only stripsContentMD5, not other fields likeBucket/Key/Metadata/PartNumber/UploadId.should preserve ContentMD5 on single-part PutObjectCommand uploads (issue #4321)— small body, single-part path; assertsPutObjectCommandIS called withContentMD5(single-part path is unaffected).Local test status (transparency): local Vitest execution is currently blocked on this Windows workstation by a pre-existing workspace-build issue (
@aws-sdk/client-s3dist-*/outputs not present from Yarn install).tsc --noEmitfiltered tolib/lib-storage/shows only the same pre-existing baseline errors caused by those missing workspace deps — no new errors on the changed lines. CI will validate full test execution end-to-end.Checklist
The XML you provided was not well-formed or did not validate against our published schema#4321)generated by AI tools, and reviewed by Mohanraj Venkatesan