Websockets not functional /w CF proxy and loopback

So this is totally backwards in my mind. I use CaddyV2 as a reverse proxy. All sites work EXCEPT when accessing from within my internal network and DNS cnames are proxied. To me this seems backwards as loopback problems would normally occur without proxy not with.

If I disable the proxy, the sites work without issue. If I use a VPN to route outside my network, the sites work. If I access externally they work. If I enable proxy and try to access the site, websockets are not available.

By all my logic, this is backwards from where loopback issues would normally happen. Any recommendations on where to start? My only idea to work around this is to override my DNS records to use a local IP, but that seems like more work than I should have to do.

Note: This happens across web server platforms… I’ve attempted both nginx and caddy.

Can you please clarify your architecture here?
Is the CaddyV2 reverse proxy configure between Cloudflare and your origin?
Or before Cloudflare?
When you say “websockets are not available”, can you provide more details?
Are you getting specific errors? Network error? Timeout?