Make cached URL accessible to workers only?

Say an URL is cached by cloudflare. Can I restrict access to to it from workers only?

I know I can restrict access to my origin URLs by allowing only Cloudflare IPs, but what about the CACHED urls?

P.S. I mean without buying the Enterprise plan of course

Why would you want to do that? Or better put, what would be the reason for doing such a thing? Because the answer depends on the use caseā€¦