Cloudflare Access Logout not quite working

I have a React web app that is secured with Zero Trust (using Warp) and using CloudFlare Access.

When first go to the website, I am correctly authenticated with my supplied identity provider and the JWT is correctly validated. I get a CF_Authorization cookie with a valid JWT that identifies me correctly.

The problem happens when I logout. What I do when I log out is clear my CF_Authorization cookie and redirect to the logout url: https://${domain}.cloudflareaccess.com/cdn-cgi/access/logout?returnTo=${rootLink}

The cookie is removed and the returnTo url redirects my back to my app. I expect to be redirected to my identity provider’s login, but that doesn’t happen. Instead my app attempts to run without a CF_Authorization cookie and nothing works.

To make a long story even longer, what might I be doing incorrectly?