R2 storage: unable to upload using pre-signed url, getting Invalid Argument Authorization

@KianNH

Hey all. I am using boto3 to generate pre-signed URL. I am following this example here: boto3 s3-presigned-urls (can’t post link)

with open('secrets.json') as f:
        secrets = json.load(f)
        account_id = secrets['account_id']
        access_key_id = secrets['access_key_id']
        secret_access_key = secrets['secret_access_key']
            
    endpoint = f'https://{account_id}.r2.cloudflarestorage.com'

    cl = boto3.client(
        's3',
        aws_access_key_id=access_key_id,
        aws_secret_access_key=secret_access_key,
        endpoint_url=endpoint,
        config=Config(
            s3={'addressing_style': 'path'},
            retries=dict( max_attempts=3 ),
        ),
    )
    
    key = sys.argv[1]
    filename = sys.argv[2]

    response = cl.generate_presigned_post('orbiter-mods', key)

This returns a response like so:

{'url': 'https://<account_id>.r2.cloudflarestorage.com/orbiter-mods', 'fields': {'key': 'foo/test1.txt', 'AWSAccessKeyId': '<access_key_id>', 'policy': '<policy>', 'signature': '<signature>'}}

Posting to the URL is not working the way it’s described in the boto3 docs. I can post to it using postman only if Auth is set to AWS Signature and if I specify the access/secret keys.

I have tried to
Asked my initial question here but was advised to create a new thread:

Maybe boto3 isn’t fully compatible with the r2 API because the presigned url format in the example sdk’s for php/js are different than what boto3 is generating.

Are you sure you should be using generate_presigned_post and not generate_presigned_url?

I think generate_presigned_post should work, but no matter. I was able to generate a URL and put a file there! Like this:

with open('secrets.json') as f:
        secrets = json.load(f)
        account_id = secrets['account_id']
        access_key_id = secrets['access_key_id']
        secret_access_key = secrets['secret_access_key']
            
    endpoint = f'https://{account_id}.r2.cloudflarestorage.com'
    
    cl = boto3.client(
        's3',
        aws_access_key_id=access_key_id,
        aws_secret_access_key=secret_access_key,
        endpoint_url=endpoint,
        config=Config(
            s3={'addressing_style': 'path'},
            signature_version='s3v4',
            retries=dict( max_attempts=3 ),
        ),
    )
    
    key = sys.argv[1]
    filename = sys.argv[2]

    response = cl.generate_presigned_url(
        ClientMethod = 'put_object',
        Params = {
            'Bucket': 'orbiter-mods',
            'Key': key,
        },
        ExpiresIn=3600
    )

    # get the length of filename file
    file_size = os.path.getsize(filename)

    with open(filename, "rb") as f:
        with tqdm(total=file_size, unit="B", unit_scale=True, unit_divisor=1024) as t:
            wrapped_file = CallbackIOWrapper(t.update, f, "read")
            requests.put(response, data=wrapped_file)

I am now trying to use this URL to put a file from the browser directly to the bucket, and I am not having much luck.

I have a form like this:

<form action="https://<account_id>.r2.cloudflarestorage.com/orbiter-mods/foo/test1.JPG?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=<credential>&X-Amz-Date=20220704T163733Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature>" method="put" enctype="multipart/form-data">
      <input type="hidden" name="X-Amz-Algorithm" value="AWS4-HMAC-SHA256" />
      <input type="hidden" name="X-Amz-Credential" value="<crednetial>" />
      <input type="hidden" name="X-Amz-Date" value="20220704T165618Z" />
      <input type="hidden" name="X-Amz-Expires" value="3600" />
      <input type="hidden" name="X-Amz-SignedHeaders" value="host" />
      <input type="hidden" name="X-Amz-Signature" value="<signature>" />
    File:
      <input type="file"   name="foo/test1.JPG" /> <br />
      <input type="submit" name="submit" value="Upload to Amazon S3" />
    </form>

Submitting gives me

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your secret access key and signing method. </Message>
</Error>

@KianNH

Anyway just want to say that I know R2 is in beta right now, and my experience with it has been absolutely amazing. I have never used Cloudflare or any of its services personally before, and R2 with its no egress costs is such a perfect fit for my current personal project that I had to try it, and boy has it been worth it.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.