403s after creating application with rules per subdomain

I had cloudflared tunnels running fine using a wildcard subdomain application to add authentication, and then excluding the ones i did not want to have authentication by adding an application with a bypass.

Clearly no good, so i overhauled and fixed it yesterday by removing this wildcard subdomain application and adding an application with authentication per subdomain i wanted it on.
Then one application seemed to return a 403 which was odd.
Now today every single application that has authentication and that i created an application for within Zero Trust returns a simple browser 403.

So i go to check the application logs, with nothing.
Then the cloudflared logs on my machine and i see the following messages:

ERR  error="request filtered by middleware handler (AccessJWTValidator) due to: Invalid token in jwt: [<the AUD of the misbehaving Application>]" cfRay=xxxxxx-XX ingressRule=11 originService=http://xxxxx:80

Over and over again for each application misbehaving

Also just to add, the applications without Cloudflare Access authorisation going via the Tunnels are running perfectly well.

You likely enabled “Protect With Access” under your Tunnel’s Public Hostname settings, and now after deleting and recreating the app, you’ll have to add back all of the access applications you want allowed to access your tunnel.
You can find the setting within your tunnel’s settings, then the public hostname, then Additional Application Settings → Access → Protect with Access → Select all the applications you want.

Cloudflare Access sends a signed JWT (Json Web Token) in the headers/cookies of each request to prove a user passed through the Cloudflare Access Application you set. This can be used to ensure that the user really passed through your Cloudflare Access Application, preventing misconfigurations or other issues that could result in unauthorized requests to your origin/down your tunnel. Tunnels sit in a useful position – on your origin between Cloudflare and your actual web server, allowing them to validate these JWTs for you.

Right now, they are “helpfully” realizing that a request is coming down your tunnel that did not pass through an authorized application for that public hostname, and they are discarding it.

As for why Bypass does not work, Bypass “bypasses” the normal access flow, it even makes the request go through your zone’s normal security settings, and also, more critically, does not get a JWT added from the Access Application. To “fix” this, use the Action “Service Auth” rather than “Bypass”. Service Auth lets the request still flow through normal access, but without an identity, which means it gets a JWT with the request that your tunnel can verify.

TLDR: Add the applications you want to have access to a public hostname under that Public Hostname’s “Protect With Access” list selection, and use “Service Auth” rather than bypass.

More info about JWTs and CF Access: Validate JWTs · Cloudflare Zero Trust docs