An error occurred (InternalError) when calling the PutObject operation

For Workers & Pages, what is the name of the domain?

Issue is not related to “Workers & Pages”. It is with R2.

What is the error number?

No Error number

What is the error message?

An error occurred (InternalError) when calling the PutObject operation (reached max retries: 2): We encountered an internal error. Please try again.

What is the issue or error you’re encountering

I’m unable to upload files using AWS S3 CLI

What steps have you taken to resolve the issue?

Everything was working fine until a few days ago. Today, I noticed my backups are not uploaded. Upon checking, I encountered the said error.
So, I created a new API token and configured it in my machine using “aws configure” command, but I’m still getting the same error.

What are the steps to reproduce the issue?

Create a new API Token:

Choose “Object Read & Write”
Specify buckets
TTL - Forever
Client IP Address Filtering - Didn’t even touch this field. Both Include and Exclude text-input fields are left blank.
I clicked on “Create API token”. The next page showed me “Access key ID” and “Secret Access Key”

In my ubuntu, opened up a new terminal and executed “aws configure”.
Copy and pasted “Access Key ID”
Copy and pasted “Secret Access Key”
For region, typed “auto”
For output, typed “json”.
Pressed enter.

Then, tried uploading a file like,
aws s3 cp test.txt s3://testbucket --endpoint-url https://.r2.cloudflarestorage.com

2 Likes

Check this Reddit - Dive into anything

If you running in GH actions something like below should do:

      - name: Install AWS CLI 2
        run: |
          curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-2.22.35.zip" -o "awscliv2.zip"
          unzip awscliv2.zip
          sudo ./aws/install --update
1 Like

R2 has recently pushed out more support for CRC32 checksums which was affecting the aws s3 cp operation. Your command should now be working without issues.

Because the CLI defaults to the NVME variant, you must supply --checksum-algorithm CRC32 as R2 doesn’t yet support the NVME variant.

No, it’s not resolved. I am also running into this issue when trying to use AWS CLI and calling aws s3 cp. What is the ETA for a solution?

I checked just now and got the same error.

I’m running into the same issue as well.

I have a small file and a large file to upload:

Same on the public Jenkins infrastructure. As per [trusted.ci.jenkins.io] Crawler fails to publish new tools metadata due to an S3 <-> Cloudflare R2 error `s3:PutObject NotImplemented: STREAMING-UNSIGNED-PAYLOAD-TRAILER not implemented` · Issue #4500 · jenkins-infra/helpdesk · GitHub, the problem is NOT fixed unless using < 2.23.x version of the aws CLI:

export AWS_DEFAULT_REGION=auto
aws configure set default.s3.max_concurrent_requests 2
aws configure set default.s3.multipart_threshold "50MB"
aws s3 sync ./updates/ s3://"${BUCKET_NAME}"/updates/ \
        --no-progress \
        --no-follow-symlinks \
        --endpoint-url "${BUCKET_ENDPOINT_URL}"
  • Using 2.22.35 works with no problem (repeatable and consistent)
  • Any tentative with [2.23.x] (up to x=15 as per our latest test) fails (consistently and repeatable)

Is there a specific configuration we’re not using causing this behavior?

1 Like

Does anyone care to fix this? AWS S3 CLI with R2 is completely unusable.

what’s going on here? R2 upload is still broken. using aws s3 sync

An error occurred (InternalError) when calling the PutObject operation (reached max retries: 2): We encountered an internal error. Please try again.

ok. there’s a fix / workaround. was hidden in earlier reply.

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-2.22.35.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install --update
rm -rf aws/ awscliv2.zip
echo "FINISHED Installing AWS CLI v2.22.35"

tldr: issue is latest version of aws cli has compatibility issues with r2. downgrade to 2.22.35 to avoid issues.

1 Like

Hi folks. R2 added CRC32 support. The AWS CLI changed recently and defaulted their client to send CRC64-NVME which we do not support. Most if not all official SDKs default to CRC32. The workaround when using the cli tool is to add --checksum-algorithm CRC32.

Apologies for any inconvenience, we will be sure to improve the response messages you get back when using the CLI to make this more apparent.

Support for CRC32C and CRC64NVME will be added shortly.

1 Like

That failure is out of date and should no longer appear with current usage of R2 as R2 supports STREAMING-UNSIGNED-PAYLOAD-TRAILER headers now. The issues are unrelated.

The current issue is that the CLI behavior is sending NVME checksums for CRC64. Support for CRC64NVME is not yet released, so you need to attach an explicit --checksum-algorithm CRC32 parameter to the cli call.

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.