Can not ssh to machine after warp is activated

I am on ubuntu 20.04 using the warp-cli to manage the service, unfortunately once i connect to Cloudflare network i can not ssh to the machine.
I can ssh to it if the source IP from where i connect to is added to the excluded-routes

Any ideas how i can ssh from any machine and still use warp ?