Why is Cf-Cache-Status BYPASS in this specific case

200706991 · March 6, 2022, 1:13pm

I set a Cache Everything rule on MinIO, following is some debug info:

mc: <DEBUG> HEAD /course/ HTTP/1.1
Host: ***
User-Agent: MinIO (windows; amd64) minio-go/v7.0.11 mc/RELEASE.2021-06-13T17-48-22Z
Authorization: ***
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220306T083848Z

mc: <DEBUG> HTTP/1.1 403 Forbidden
Content-Length: 372
Cf-Cache-Status: BYPASS
Cf-Ray: 6e79c6fbbcb791b1-SIN
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Sun, 06 Mar 2022 08:38:49 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=gj6GJFSlaZqew1T%2FA36AgvEqP%2B5mk%2FIaEQAGkT26K5XrIoGhW3K04jnRJE7GIkmMKbL7OaIDk%2FmoIf6eVxqHku1ouMnzfAwNGL74nva6kAKAfSbo%2FynP3KJsyTsf5Pg%3D"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Request-Id: 16D9BDE4C65D52DD
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

The response I get after turn off Cache Everything:

mc: <DEBUG> HEAD /course/ HTTP/1.1
Host: ***
User-Agent: MinIO (windows; amd64) minio-go/v7.0.11 mc/RELEASE.2021-06-13T17-48-22Z
Authorization: ***
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220306T131206Z

mc: <DEBUG> HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Credentials: true
Cf-Cache-Status: DYNAMIC
Cf-Ray: 6e7b57580ec64ba4-SIN
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Sun, 06 Mar 2022 13:12:07 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=IlBNnsYqdQg%2FVE0MuxxK23AkvPMQDNIWjwBUfjxgcirjoEu5SgUVIiJLw7L5vUsWKAJaTDgJaj5yygi5VJrh75MU7fLzPevCn8eXxdyazklLw9Crnszq%2FNG78dUnMro%3D"}],"group":"cf-nel","max_age":604800}      
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Request-Id: 16D9CCCEE9696AE1
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
Content-Length: 0

I have two questions:

Why is the status BYPASS?
If I turn off Cache Everything, I’ll get 200 OK, I don’t know why I failed after turning on Cache Everything.

Thank you all!

sdayman · March 6, 2022, 2:39pm

I believe this thread addresses that:

I think the root question is why do you get a 403 with Cache Everything. Maybe your server logs can point you in the right direction.

200706991 · March 6, 2022, 3:53pm

Thank you. I found more clues after checking the log, in the log:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><BucketName>course</BucketName><Resource>/course/</Resource><RequestId>16D9D532CBA40512</RequestId><HostId>7bbeea80-374e-4359-aa15-ab35b3733dde</HostId></Error>

Do you have any clue what things CF does might cause this?

fritex · March 6, 2022, 3:57pm

Might below articles help? I am not familiar much with AWS, but:

Solve - SignatureDoesNotMatch Error in AWS CLI | bobbyhadz
https://forums.aws.amazon.com/thread.jspa?threadID=234825
SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details · Issue #86 · aws/aws-sdk-js · GitHub

Some trailingslash issue here:

SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details · Issue #86 · aws/aws-sdk-js · GitHub

Wrong keys used here:

javascript - SignatureDoesNotMatch error from S3 putObject - Stack Overflow

200706991 · March 6, 2022, 2:03pm

I just set a Cache Level Bypass Page Rule for *mydomain.com/*, but when I request my *.mp3 file, it returns cf-cache-status: DYNAMIC. Shouldn’t it be BYPASS?

sdayman · March 6, 2022, 2:28pm

Good question. I can see how the wording leads to come confusion.

The Page Rule to “Bypass” is more like an “Ignore Cache” setting. BYPASS means the origin said to not cache the resource.

https://developers.cloudflare.com/cache/about/default-cache-behavior/#cloudflare-cache-responses

200706991 · March 6, 2022, 3:34pm

In my other post

200706991:

mc: <DEBUG> HEAD /course/ HTTP/1.1
Host: ***
User-Agent: MinIO (windows; amd64) minio-go/v7.0.11 mc/RELEASE.2021-06-13T17-48-22Z
Authorization: ***
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220306T083848Z

mc: <DEBUG> HTTP/1.1 403 Forbidden
Content-Length: 372
Cf-Cache-Status: BYPASS
Cf-Ray: 6e79c6fbbcb791b1-SIN
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Sun, 06 Mar 2022 08:38:49 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=gj6GJFSlaZqew1T%2FA36AgvEqP%2B5mk%2FIaEQAGkT26K5XrIoGhW3K04jnRJE7GIkmMKbL7OaIDk%2FmoIf6eVxqHku1ouMnzfAwNGL74nva6kAKAfSbo%2FynP3KJsyTsf5Pg%3D"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Request-Id: 16D9BDE4C65D52DD
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

The Cf-Cache-Status is BYPASS. And we can see that there is no Cache-Control header in response and I also haven’t enabled Origin Cache Control, so why is it BYPASS?

BYPASS The origin server instructed Cloudflare to bypass cache via a Cache-Control header set to no-cache , private , or max-age=0 even though Cloudflare originally preferred to cache the asset. BYPASS is returned when enabling Origin Cache-Control. Cloudflare also sets BYPASS when your origin web server sends cookies in the response header.

fritex · March 6, 2022, 3:41pm

Why 403 in first place?
Due to the HEAD, or rather wrong auth credentials, or rather Security Level “I am under an attack”, or does this event appear in the Firewall tab → Overview as “challenged” or “blocked”?
Bot Fight Mode is enabled?

By default XML filetype is not cached, but from where is 403 coming?:

https://developers.cloudflare.com/cache/about/default-cache-behavior/#default-cached-file-extensions

Is it the same if you change the user-agent?

Is it maybe the JSON, rather than XML request/response?

.mp3 is cached by default since:

But a bit odd if it doesn’t get BYPASS when you set your Page Rule like that.

Have you tried individually and manually to Purge the specific URL at Caching → Configuration → Custom Purge from Cloudflare dashboard?

Is the filesize large?

200706991 · March 6, 2022, 4:14pm

I just find some very very very weird thing. On my PC, when I output log, it shows a HEAD request with 403 error. But on backend, there is no HEAD requests at all, only GET requests.

My guess is Cloudflare turned my HEAD request into GET request, which causes the wrong signature (You change the request method, the signature will change as well).

Do you have any clue, why Cloudflare do this when Cache Everything turned on?

fritex · March 6, 2022, 4:19pm

Hm, may I ask have you tried using a different app to send a request?
For example using Postman to test it out?

I am not familiar how this should happen at all, neither haven’t heard anything which would indicate Cloudflare is doing something “in between” the requests to change or modify the original one being sent to some other type, of it if that is somehow possible.
Might have to think for any idea, or wait for another reply from someone else

A helpful a bit to clearify few things:

Nevertheless:

Might be this one?:

Other articles:

https://developers.cloudflare.com/fundamentals/get-started/http-request-headers/

Or it might have to be something with content-length as far as it’s the .mp3 resource?

200706991 · March 6, 2022, 4:22pm

Nope, too complicated to handcraft a request like this. I’m not familiar with the specific signature method they use. But I’m pretty sure it’s Cloudflare that turned the HEAD method to GET, which caused this problem.

200706991 · March 6, 2022, 4:30pm

In front end debug (mc --debug):

mc: <DEBUG> HEAD /course/ HTTP/1.1
Host: **
User-Agent: MinIO (windows; amd64) minio-go/v7.0.11 mc/RELEASE.2021-06-13T17-48-22Z
Authorization: ***
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220306T162505Z

mc: <DEBUG> HTTP/1.1 403 Forbidden
Content-Length: 372
Access-Control-Allow-Credentials: true
Cf-Cache-Status: BYPASS
Cf-Ray: 6e7c72087a3e91b9-SIN
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Sun, 06 Mar 2022 16:25:06 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=CBcniI1Rl0Yg%2BNOmvhRjuZCYSXN2vxzwWMJoiZtKPxR84nxwFuTz4ggYfW4JQqx2fy9CG2eg5g7dllBj5Awxq6nvhU1JPX62f5OrIONx1XzK4iKXfuocO%2By%2Be%2BlmHsw%3D"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Request-Id: 16D9D756D7CECF0A
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

In backend debug (mc admin trace -a -v):

s3-2:9000 GET /course/
s3-2:9000 Proto: HTTP/1.1
s3-2:9000 Host: ***
s3-2:9000 Cf-Visitor: {"scheme":"https"}
s3-2:9000 User-Agent: MinIO (windows; amd64) minio-go/v7.0.11 mc/RELEASE.2021-06-13T17-48-22Z
s3-2:9000 X-Real-Ip: 108.162.219.189
s3-2:9000 X-Forwarded-Port: 443
s3-2:9000 X-Forwarded-Server: reverse-proxy-1-2
s3-2:9000 Accept-Encoding: gzip
s3-2:9000 Cdn-Loop: cloudflare
s3-2:9000 Cf-Ipcountry: HK
s3-2:9000 Content-Length: 0
s3-2:9000 X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
s3-2:9000 Authorization: ***
s3-2:9000 Cf-Connecting-Ip: 210.0.158.172
s3-2:9000 X-Forwarded-Host: s3.awwpan.com
s3-2:9000 Cf-Ray: 6e7c7205dcc391b9-EWR
s3-2:9000 X-Amz-Date: 20220306T162505Z
s3-2:9000 X-Forwarded-For: 108.162.219.189
s3-2:9000 X-Forwarded-Proto: https
s3-2:9000 
s3-2:9000 [RESPONSE] [2022-03-06T16:25:06:000] [ Duration 148µs  ↑ 247 B  ↓ 695 B ]
s3-2:9000 403 Forbidden
s3-2:9000 Content-Length: 372
s3-2:9000 Server: MinIO
s3-2:9000 Vary: OriginAccept-Encoding
s3-2:9000 X-Amz-Request-Id: 16D9D756C1DE73A5
s3-2:9000 X-Xss-Protection: 1; mode=block
s3-2:9000 Accept-Ranges: bytes
s3-2:9000 Content-Type: application/xml
s3-2:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
s3-2:9000 X-Content-Type-Options: nosniff
s3-2:9000 Content-Security-Policy: block-all-mixed-content
s3-2:9000 <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><BucketName>course</BucketName><Resource>/course/</Resource><RequestId>16D9D756C1DE73A5</RequestId><HostId>7bbeea80-374e-4359-aa15-ab35b3733dde</HostId></Error>
s3-2:9000

system · March 21, 2022, 4:31pm

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.