It seems like the recommendation is to run cloudflared inside every application webservice pod?
This would mean as our application pods scale up and down as new tunnels are created and removed. Is it correct that I need to purchase load balance origins for the maximum amount of pods that will exist? For us we have many deployments of our application and each deployment can scale up dynamically. If I have 5 deployments in different regions each with capability to scale to 8 pods I should pay monthly for 40x load balance origins or is this billed dynamically?
Additionally it raises questions on scale down since our application might get the signal to scale down so it will finish processing the current requests and exit but then what about the balancer tunnel. It may still be sending new http requests to us? On the other hand when kubernetes signals cloudflared to shutdown we don’t want the tunnel to end before our service has had a chance to send back a response to a request. I can’t see anything in the docs about this or any timeout options or probes, etc.
I know your own ingress is not supported now but do you support other configurations? I was thinking maybe could set cloudflared to its own deployment but with its host as a kubernetes service like http://service-name.default.svc.cluster.local:8080 or maybe to sidecar cloudflared to our nginx ingresses?