Multi part uploads from s3cmd broken

I’m facing comparability issues on Cloudflare R2. The behavior on multi part uploads seems to deviate from the spec and breaks tools such as s3cmd which expect a md5 hash in the etag header.

DEBUG: Canonical Request:
PUT
/my-file.tar.zst
partNumber=1&uploadId=[redacted]
content-length:15728640
host:[redacted].r2.cloudflarestorage.com
x-amz-content-sha256:22c5bf1bd95afe12f8cd6e13ae5db4299a9defcb6df2cfc69285488e2deb5c09
x-amz-date:20220818T052008Z

content-length;host;x-amz-content-sha256;x-amz-date
22c5bf1bd95afe12f8cd6e13ae5db4299a9defcb6df2cfc69285488e2deb5c09
----------------------
DEBUG: signature-v4 headers: {'content-length': '15728640', 'x-amz-date': '20220818T052008Z', 'Authorization': '[redacted]', 'x-amz-content-sha256': '22c5bf1bd95afe12f8cd6e13ae5db4299a9defcb6df2cfc69285488e2deb5c09'}
DEBUG: get_hostname([redacted]): [redacted].r2.cloudflarestorage.com
DEBUG: ConnMan.get(): re-using connection: https://[redacted].r2.cloudflarestorage.com#6
DEBUG: format_uri(): /my-file.tar.zst?partNumber=1&uploadId=[redacted]
    65536 of 15728640     0% in    0s     4.97 MB/sDEBUG: ConnMan.put(): connection put back to pool (https://[redacted].r2.cloudflarestorage.com#7)
DEBUG: Response:
{'data': b'',
 'headers': {'cf-ray': '73c832d5dd7e15cb-EWR',
             'connection': 'keep-alive',
             'content-length': '0',
             'date': 'Thu, 18 Aug 2022 05:20:13 GMT',
             'etag': '"ABwcsNIEIx/3TXD+37wkhu1YIh8AgUg/++I5bsBm9MiQotlGsTOkpQhTeRkj/p5IFx2PSa/ouG94ghv+Mniyltsnj6QDUb9omfJfRLd0hJVqTPReu9NfKcBp0Z9NTBHcwf83xI3u49eLDXsDH9rS/EDF9ALqJ6Y6HmUCfB4g6bwZSeAgly77Amaqib1kkH+uta/NcIfe1ot1he0iaLC5ZIwruHOrG+F5gsZkmJ1qZXpWrYLBVUhyFPZ6Yo1LlKjSJw=="',
             'expect-ct': 'max-age=604800, '
                          'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
             'server': 'cloudflare',
             'vary': 'Accept-Encoding'},
 'reason': 'OK',
 'size': 15728640,
 'status': 200}
 15728640 of 15728640   100% in    4s     3.34 MB/s  done
DEBUG: MD5 sums: computed=e7df577f795e45df5535f558c9931973, received=ABwcsNIEIx/3TXD+37wkhu1YIh8AgUg/++I5bsBm9MiQotlGsTOkpQhTeRkj/p5IFx2PSa/ouG94ghv+Mniyltsnj6QDUb9omfJfRLd0hJVqTPReu9NfKcBp0Z9NTBHcwf83xI3u49eLDXsDH9rS/EDF9ALqJ6Y6HmUCfB4g6bwZSeAgly77Amaqib1kkH+uta/NcIfe1ot1he0iaLC5ZIwruHOrG+F5gsZkmJ1qZXpWrYLBVUhyFPZ6Yo1LlKjSJw==
WARNING: MD5 Sums don't match!
WARNING: Too many failures. Giving up on 'my-file.tar.zst'

I made an issue in GH but can’t link to it here

You’ll need to disable this functionality since R2 doesn’t return a hash as the eTag in multipart uploads currently.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.