Automatic Signed Exchanges (SXGs) Beta Launch

sdayman · October 12, 2021, 5:27pm

You’re currently using Siteground name servers. First you’ll have to completely turn off the Siteground integration in your SG account.

I don’t know what will happen to your zone here. It might go “inactive” and wait for you to update name servers. It would tell you which ones. Or you may have to delete the zone from your account here, then re-add it. Again, I’m not sure what happens at this end in your situation.

firat · October 12, 2021, 5:53pm

@nsgoyat @eva2000 @desmondgrey

Regarding setting a cache period for SXGs: we are testing a way of doing this and I will get back to you as soon as I have something.

firat · October 12, 2021, 5:55pm

Hi junellabanag1, please have a look at the reply below.

firat · October 12, 2021, 5:56pm

Hi everyone. I have more details regarding headers and the SXGs. This will be also added to the first post in this thread:

Signed exchanges can strip out certain cookies and headers, and create problems with dynamic content. The following hop-by-hop and other uncached headers will be stripped during the signed exchange generation:

Hop-by-hop header fields listed in the Connection header field (Section 6.1 of [RFC7230]).
Header fields listed in the no-cache response directive in the Cache-Control header field (Section 5.2.2.2 of [RFC7234]).
Header fields defined as hop-by-hop: Connection, Keep-Alive, Proxy-Connection Trailer, Transfer-Encoding, Upgrade

Since Cloudflare cannot be sure whether a signed exchange does or does not include private information, a signed exchange will not be generated in the presence of the following headers:

Authentication-Control, Authentication-Info, Clear-Site-Data, Optional-WWW-Authenticate, Proxy-Authenticate, Proxy-Authentication-Info, Public-Key-Pins, Sec-WebSocket-Accept, Set-Cookie, Set-Cookie2, SetProfile, Strict-Transport-Security, Vary, WWW-Authenticate

eva2000 · October 12, 2021, 6:09pm

Cheers

That’s interesting so HSTS and Vary response headers would exclude signed exchange generation!

junellabanag1 · October 12, 2021, 6:13pm

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload header is enabled and can confirm that SXGs is working on my domain.

eva2000 · October 12, 2021, 6:17pm

Yeah noticed that too, origin with HSTS seems to still generate the SXG for me too.

firat · October 12, 2021, 7:12pm

@junellabanag1 @eva2000 that’s interesting. Thank you folks for reporting this. Just sent you PMs for zone details.

henshaw · October 12, 2021, 7:37pm

@firat, in the email that was sent out today, it said:

If you experience any problem, we kindly ask you to disable it and let us know.

Based on that, should I go ahead and turn it off until the caching issue is resolved? Which for me continues to be this:

cache-control: private

warning: 199 - “debug: content has ingestion error: Error fetching resource: Content is not cache-able or cache-able lifetime is too short”

eva2000 · October 12, 2021, 8:10pm

sounds like your HTML document’s content has <120 seconds cache time?

henshaw · October 12, 2021, 8:23pm

Sounds like it does, but that’s not the case. It’s set to 4 hours. I think the issue is that cache-control is being communicated as private, which is making it appear too short or uncachable. I have no idea how to fix this and am assuming it’s something that still has to be figured out by Cloudflare and Google during this beta.

eva2000 · October 12, 2021, 8:52pm

Which request are you looking at. Don’t mistake the Google cached/prefetched requests’ response headers for the cache-control you’re meant to be looking at. It’s the response header for your origin served request before CF generates the SXG cached request for Google to pick up and use.

This is close to my origin response headers which SXG criteria will look at cache-control/expires header for as Cloudflare would become the origin with respective to Google Search cache. AFAIK. In this case my HTML doc has been cached at CF CDN level for ~90000 seconds according to CF age header and for browser level cache max 86400 seconds.

This is Google served prefetch cached response headers

henshaw · October 12, 2021, 10:07pm

This is what I get. I don’t know anything beyond this. I represent the use case of toggling the option on and expecting it to work. I’m not doing anything special with my Cloudflare settings.

eva2000 · October 12, 2021, 10:22pm

@henshaw hey bumped into you on Twitter recently

Looks like you’re using CF Wordpress APO and it’s set to cf-cache-status = BYPASS with cf-apo-via: origin,no-cache so serving from origin rather than CF CDN cache. But you have x-cache: HIT: 3 which seems to come from WP Engine origin/web host side HTML caching so that is probably interfering with SXG?

Also, you have the presence of Vary header for vary: Accept-Encoding, Accept-Encoding, Accept-Encoding,Cookie which @firat said would prevent SXG generation by CF? Though my blog has SXG generated and I have vary: Accept-Encoding too.

Your non-SXG site response headers for index

alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
cache-control: max-age=600, must-revalidate
cf-apo-via: origin,no-cache
cf-cache-status: BYPASS
cf-edge-cache: cache,platform=wordpress
cf-ray: 69d3ad834e6df27f-BNE
content-encoding: br
content-type: text/html; charset=UTF-8
date: Tue, 12 Oct 2021 22:13:51 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
link: <https://www.coywolf.news/wp-json/>; rel="https://api.w.org/"
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=poJUqQwy3NCFN81RI9Tch99gBU5JW9cslqmG6cE1AjA3v%2BroUjxl9RsMC5gQO%2BP54eYya1H2efwkxKTzCdDudspnSeAXlWEuFA4Im7vz3jtBZxQ1PxrxXN7FKBXoZKGpeHg%3D"}],"group":"cf-nel","max_age":604800}
server: cloudflare
strict-transport-security: max-age=31536000; preload
vary: Accept-Encoding, Accept-Encoding, Accept-Encoding,Cookie
x-cache: HIT: 3
x-cache-group: normal
x-cacheable: SHORT
x-content-type-options: nosniff
x-powered-by: WP Engine

@firat @yevgen

eva2000 · October 12, 2021, 10:34pm

I think I see why, you have incorrect content encoding when you curl test the response headers for the Google served cache. Content type should be application/signed-exchange;v=b3 not text/html; charset=UTF-8

curl -s -i -H 'Accept: application/signed-exchange;v=b3' https://www-coywolf-news.webpkgcache.com/doc/-/s/www.coywolf.news/
HTTP/1.1 200 OK
Location: https://www.coywolf.news/
Cache-Control: private
X-Silent-Redirect: true
Warning: 199 - "debug: content has ingestion error: Error fetching resource: Content is not cache-able or cache-able lifetime is too short"
Content-Type: text/html; charset=UTF-8
X-Content-Type-Options: nosniff
Date: Tue, 12 Oct 2021 22:26:22 GMT
Server: sffe
Content-Length: 287
X-XSS-Protection: 0
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

<HTML><HEAD>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>Redirecting</TITLE>
<META HTTP-EQUIV="refresh" content="0; url=https://www.coywolf.news/">
</HEAD>
<BODY onLoad="location.replace('https://www.coywolf.news/'+document.location.hash)">
</BODY></HTML>

Also could be related to HTML meta refresh <META HTTP-EQUIV="refresh" content="0; url=https://www.coywolf.news/"> ? Or that is SXG failing and falling back to non-SXG URL?

whereas on my blog the correct content type is returned Content-Type: application/signed-exchange;v=b3

curl -s -i -H 'Accept: application/signed-exchange;v=b3' https://blog-centminmod-com.webpkgcache.com/doc/-/s/blog.centminmod.com/
HTTP/1.1 200 OK
NEL: {"report_to":"nel","max_age":604800,"success_fraction":0.05}
Report-To: {"group":"nel","max_age":604800,"endpoints":[{"url":"https://beacons.gcp.gvt2.com/nel/upload-nel"},{"url":"https://beacons.gvt2.com/nel/upload-nel"}]}
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Type: application/signed-exchange;v=b3
Content-Security-Policy: require-trusted-types-for 'script'; report-uri https://csp.withgoogle.com/csp/webpkgcache-team
Cross-Origin-Resource-Policy: cross-origin
Cross-Origin-Opener-Policy-Report-Only: same-origin; report-to="webpkgcache-team"
Report-To: {"group":"webpkgcache-team","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/webpkgcache-team"}]}
Content-Length: 223372
Date: Tue, 12 Oct 2021 22:30:04 GMT
Expires: Tue, 12 Oct 2021 22:30:04 GMT
Cache-Control: private, max-age=86399
Last-Modified: Tue, 12 Oct 2021 15:16:28 GMT
X-Content-Type-Options: nosniff
Server: sffe
X-XSS-Protection: 0
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

@firat is Report-To: {"group":"webpkgcache-team","max_age":2592000 in my response header what determines SXG cache time or Cache-Control: private, max-age=86399 as that matches what I set on CF CDN edge response for CF Worker full HTML page caching at 2592000 but browser cache control is 86400

henshaw · October 12, 2021, 10:47pm

Nice!

I only recently turned on APO to see if that would fix the issue. The cache private and debug message appears regardless of whether or not APO is being used.

It could certainly be something that WP Engine is doing. Although I wouldn’t know the first thing to do to fix that.

girirajnet · October 13, 2021, 3:40am

Should I do anything from my side?

I’m using Cache level as “Standard” and Browser Cache TTL “2 minutes”

eva2000 · October 13, 2021, 4:41am

hmm from webpackager/cache_requirements.md at main · google/webpackager · GitHub

Google SXG cache

The Google SXG cache sets these requirements in addition to the ones set by the SXG spec:

The SXG must have a freshness lifetime of at least 120 seconds, as computed for a shared cache from its outer headers.

The signed fallback URL must approximately equal the URL at which the SXG was served. Where possible, aim to make them byte-equal. The set of allowed differences is not precisely specified, but approximately:

Characters may be substituted by their percent encodings, and vice versa, with the exception of meaningful delimiters like / , ; , ? , & , and = .

Query parameters may be re-ordered.

Valueless query parameters may be encoded with or without a trailing = .

Extra & s in the query string are allowed.

The signed cert-url must be https .

The signature header must contain only:

One parameterised identifier.

Parameter values of type string, binary, or identifier.

The payload must be non-empty.

The signed cache-control header cannot have a no-cache or private directive, even with a value (e.g. no-cache=some-header is disallowed).

The content-type must satisfy the media-type grammar.

The link header, if present, must lead to successful substitution per the Loading spec. Specifically, it must meet these requirements, in addition to the ones set by the Link spec:

Each URI-Reference must be an absolute https URL.

Parameter names can only be as , header-integrity , media , rel , imagesrcset , imagesizes , or crossorigin .

All rel parameters must be either preload or allowed-alt-sxg .

All imagesrcset values must parse as a srcset attribute.

There may be no more than 20 rel=preload s.

All crossorigin values must either be the empty string, or anonymous .

Every rel=preload must have a corresponding rel=allowed-alt-sxg with the same URI, which in turn must contain a header-integrity parameter with a value that satisfies the CSP hash-source grammar using the sha256 variant.

The preloaded URLs, when requested with an SXG-preferring Accept header, must respond with valid SXGs that match their given header-integrity .

The link header must not be present on subresources, i.e. SXGs that are themselves preloaded from other SXGs.

There must not be a signed variant-key-04 or variants-04 header.

The signature’s lifetime ( expires minutes request time) must be >= 120 seconds.

The SXG must be no larger than 8 megabytes.

The page should be responsive, i.e. correct on all media. (In the future, a supported-media annotation should allow this constraint to be removed.)

Some of the above limitations are overly strict for an SXG cache’s needs, and were implemented as such for the sake of expediency. They may be loosened over time, especially in response to publisher feedback.

and RFC 7234 - Hypertext Transfer Protocol (HTTP/1.1): Caching

4.2.1. Calculating Freshness Lifetime A cache can calculate the freshness lifetime (denoted as freshness_lifetime) of a response by using the first match of the following:

If the cache is shared and the s-maxage response directive (Section 5.2.2.9) is present, use its value, or

If the max-age response directive (Section 5.2.2.8) is present, use its value, or

If the Expires response header field (Section 5.3) is present, use its value minus the value of the Date response header field, or

Otherwise, no explicit expiration time is present in the response. A heuristic freshness lifetime might be applicable; see Section 4.2.2.

Not sure if CF follows this, but that would mean in order of priority would be cache-control s-maxage, max-age, expires? Looks like cdn-cache-control header isn’t supported then.

Terms_G · October 13, 2021, 4:43am

So since we have HSTS enabled we will not be able to use signed exchanges?

Also, WordPress sites should have no issues with signed exchanges, correct?

Thanks.

firat · October 13, 2021, 7:12am

Hi @henshaw. This appears to be a Google bug. The problem is we are interpreting Cache-Control: must-revalidate as equivalent to max-age=0 . This seems wrong per the HTTP spec.

Short term, you can remove the must-revalidate.
Medium term, we’ll try to fix this bug or make some changes to remove must-revalidate.

cc @eva2000