Cached WebP image served to browser that doesn't support it

My Wordpress website (NGINX server) uses ShortPixel plugin to serve WebP images to browsers that support them and regular PNG or JPEG images to browsers that don’t.

The delivery method is the tag syntax. This means the replacement is only happening when the page is opened.

This also means that if the image is accessed directly, the JPG should be delivered.

I have found CF caching is 302 redirecting some JPG to WebP images - this is causing me problems on Safari 14 and 15 on MacOS Catalina, which doesn’t support WebP images, (although note - the same version of Safari can support WebP image when running on MacOS Big Sur) and when sharing images on sites like Pinterest.

When using CF development mode, (or initially after clearing CF cache), everything works as expected. After some time clearing cache, various images on my site start to have the problem.

It seems CF Caching is 302 redirecting JPG to WebP after a while for these images. For example:

is (currently) redirecting to WebP version, but other images such as this one:

are performing as expected:

These are images for posts promoted on my homepage https://poultrykeeper.com/

I have found a few other posts about this issue, but none of them answers this exact issue. They seem to propose alternative methods to deliver WebP using CF.

I would like to understand why CF is introducing this redirect and find a fix to continue to deliver WebP or JPG from my Website so I can use CF and Automatic Platform Optimisation.

I do NOT have Image Resizing, Polish, Rocketloader, APO, Mirage or Rocket Loader enabled, so these shouldn’t be interfering with the images in any way.

Thanks for your help,
Tim

Cloudflare is caching the HTTP 302 response from your server.

I believe this occurs when a WebP supported browser requests the image from your website, and your website detects the request is coming from a WebP supported browser, so your website sends a 302 redirect back to the browser.

Because Cloudflare caches JPG/PNG by default, the 302 response will also be cached.

Now, if you want to prevent caching 302 responses from your server, it is possible to use the Cache TTL by Status Code option to stop caching 302 responses, but this is only available for Enterprise plan.

Anyway, it’s rare to see a website redirects JPG/PNG to WebP URL. Most people just serve the WebP image in the same URL, just with a different content-type.

If you are caching images in Cloudflare, then you cannot vary the image type based on the device capabilities.

There are various solutions to this problem that you will find searching this community. Essentially, you need to ensure that the URL for the origin generated images are different for jpeg and WebP images.

Possibly the easiest way to fix this problem is to use a Transform Rule which alters the URL slightly so that a different cache is created for WebP images. Something like this should do the trick.

You may want to obfuscate the query parameter so that it is not accidentally requested by a user without an Accept: image/webp header. (The query parameter should never be seen by the public)

If the images on your Origin have either jpg or webp extensions, you could do the following:

That says if the URL contains .jpg and the Accept Header indicates webp support, then change the file extension from jpg to webp.

1 Like

Are you on the free plan? If you are on pro or higher I would use the webp image conversion using Cloudflare. Sadly I do not know much about short pixel but I can tell you of a much better option that is opensource with a good little community.

this beats most of the big players which are mostly paywall to get any good features and even then don’t really compare., You can install on your back end multiple options for image conversion and compare quality then select the option.

  • Replace tags with tags, adding the webp to srcset.
  • Dynamically load picturefill.js on older browsers

This works perfectly for old browser and webp on my site. Works for nearly all themes, unless you got something super customized and the coder is a black sheep doing things weird.

I will admit this plugin adds a tad more to the HTML, but the webp and image optimization and the options of what choice converter you can use will far outway the additional weight by a mile.

I use this personally even though I am on a pro play with access to Cloudflare webp it doesn’t stack up close to it.

Forget my post overall if not interested, just saying there are better options that can work if you cannot get short pixel to work properly :slight_smile:

Looks like a nice hack.

1 Like

Hello all,

Well, what a fantastic response, you have given me some brilliant ideas. Thank you.
So I have a fix. Here it is:

@erictung put me onto this path. Since it’s my website putting the 302 in place, I investigated my server first.

Shortpixel was doing what it was supposed to do, but since there is an NginX server, my host Kinsta came up with a fix for me :slight_smile:

They changed the server rules from:

rewrite ^(/wp-content/.+).(png|jpe?g)$ $scheme://$host$1.webp redirect;

to

rewrite ^(/wp-content/.+).(png|jpe?g)$ $1.webp break;

That uses a rewrite instead of a redirect. So while the URL will still show the .jpg, I will get a webp.

This will avoid the redirect, hence the cache will not save that 302.

But I like the little hack @michael came up with too.

Thank you, everyone, I’m going to go and test the fix now.

Tim.

1 Like

If you only make that change you will end up in the same situation.

A request to Cloudflare for a file will contain an Accept header. If the image is not already cached, then the request (including the Accept header) will be sent to the Origin. The origin will then serve an image file, which Cloudflare will cache. All subsequent requests for that URL will be responded to by the Cloudflare cache, regardless of the Accept header.

My first hack shards the Cache in two for each URL. One shard is only used for situations where the Accept header indicates webp support, and one shard for everything else. Origin requests for the two different shards will contain different Accept headers, which your Origin will serve as it is currently doing.

The second hack again creates two shards, but the file path is different in each shard. This one will essentially bypass the ReWrite rules, so should be a little bit more efficient. It has the advantage that it should be more reliable, in that the cache cannot be poisoned with the wrong image format.

Actually Cloudflare now supports Vary for images. So the WebP cache issue should be able to fix by sending an API request to Cloudflare.

https://developers.cloudflare.com/cache/how-to/enable-vary-for-images

3 Likes

Thank you again… I see what you mean, it’s not the right fix.

Vary for Images seems to be the solution for me to try, but (silly question) how do I go about getting to the Cloudflare API that’s required to enable Vary?

It seems to be quite advanced… and I was looking for something like a step by step guide to use it?

Thank you,
Tim

This documentation already shows you how to execute the curl command to use Vary for Images.

Just replace X-Auth-Email header value with your account email address and X-Auth-Key header with your Global API Key.

You can run the command in Windows or Linux terminal window, although I will prefer using Linux for such operations.

OK thank you. I appreciate it. I use Windows and have a Mac, I will try this.

1 Like

Hello again,

A quick update to provide additional details and one more question if you don’t mind :slight_smile:

From the Windows Command Prompt, I set Vary.
This is the code I used is below. There’s another support thread how I got to this here, huge thanks to Albert for his patience with me!

curl -X PATCH "https://api.cloudflare.com/client/v4/zones/**my zone ID here**/cache/variants" ^ 
-H "Authorization: Bearer **My API Token here**" ^ 
-H "Content-Type: application/json" ^ 
--data "{\"value\":{\"jpeg\":[\"image/webp\",\"image/avif\"],\"jpg\":[\"image/webp\",\"image/avif\"]}}"

Probably obvious to most people, but here are a couple of notes on why this is different to the recommended code in case you’re a beginner like me!

  • Windows Command Prompt requires you use ^ instead of \ to signal the command will continue on the next line.
  • Windows doesn’t like single quotes. You have to replace with \ instead like this:

--data "{\"value\":{\"jpeg\":[\"image/webp\",\"image/avif\"],\"jpg\":[\"image/webp\",\"image/avif\"]}}"

And on to my (hopefully final) question.

Do you think I should ask my host to set the server back to the original code now Vary is set?

Back to:

rewrite ^(/wp-content/.+).(png|jpe?g)$ $scheme://$host$1.webp redirect;

Excuse my ignorance, I’m not exactly sure what it’s doing…

Thank you,
Tim

2 Likes

Don’t. Your current config to rewrite is already fine.

1 Like

Thanks again @erictung - I really appreciate the help.

Unfortunately, I’m still seeing the same problem occurring.
I cleared CF Cache and slowly over time, various images stop showing up when testing with Safari 15 / MacOS Catilina.

For example, this image doesn’t show on my homepage:

https://poultrykeeper.com/wp-content/uploads/2021/09/Hen-Moulting_260x193.jpg

When I inspect it in the browser, it is called a jpg but the Content-Type is a WebP.
I guess that’s from the rewrite.

So, I’m still not 100% sure that the re-write is correct.

I guess Vary sends a jpg because it looks at the Accept header?
But in this case the jpg isn’t a jpg but a WebP.

The accept header I see from Safari is:

Accept: image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5

The Response from CF Cache:

Content-Type: image/webp

Thanks,
Tim

Looks like this topic has closed.
I still haven’t found a solution. I will come back and edit this if I do.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.