Cloudflare Zero Trust: WARP-to-WARP Connectivity

Hi there! Abe from the Zero Trust product team here :wave:

Today, we’re excited to share that we’ve been working on a new, easy way to start building a private network on Cloudflare! To get started, you just need the software Cloudflare WARP, that you’re likely already familiar with running on your mobile devices and laptops.

To start building your first network, you simply need to:

  1. Create an account
  2. Download WARP on the devices you want to connect
  3. Enroll each into your Zero Trust account
  4. Navigate to Settings > Network and enable Warp to Warp

Once enrolled, these two (or more) devices will become a part of the same virtual network and from there you can Ping, SSH, or access local web servers running on either machine as if they were in the same physical network. We’ll assign a private IP to each enrolled device that you can use. No additional configuration steps required.

If you want to add Zero Trust rules, you can! Otherwise you can make sure that everything enrolled in your Cloudflare One account can talk to everything else.

Needless to say, we’re excited to get this into your hands and get your early feedback. While this feature may simplify new, lightweight deployments, it also unlocks a number of use cases for more complex, existing deployments where you may want Warp devices to address one another. In either case, we’re excited to get your feedback and learn about what you’d like to see out of this feature-set next!

If you would like to be part of the beta, please fill out fill out this form and we will get this feature flag surfaced in Zero Trust account.

Happy testing!

5 Likes

Hi @abe — I filled out the form but wasn’t sure what “Cloudflare Account Tag” is… is that just my Teams domain? Thanks!

If you login to https://one.dash.cloudflare.com/ , you’ll find it in the URL as the alphanumeric number after you pick your account (if you have multiple ones)
Same if you go to https://dash.cloudflare.com/ — as well as after you pick a Zone (if you have any), then you’ll find it on the right bottom of the screen under “Account”

1 Like

Thanks @abe — just submitted the form again!

whoops! @nuno.diegues — didn’t notice until now that you were the one to reply to my inquiry, thank you.

also, @abe just wondering how long to expect before the feature might be enabled for my account? hoping to use this over Tailscale / ZeroTier since Zero Trust is already configured and would be preferable of course. thanks!

No problem at all. For context, we’ve been enabling accounts on weekly basis, but I’ll be sure to double check this weeks enablement to include your account. Looking forward to receiving your feedback as well!

@abe I am also keen to try this out to build a PoC, can you also add me to the week’s enablement?

Also, this feature seems to include all the features of cloudflared. Will Cloudflare-warp end up replacing cloudflared?

thanks @abe — still not seeing it in Settings » Network… but will check back later this week

Noticed the “WARP to WAPR (beta)” switch is now enabled for my zero trust account. Super excited to test it out :slight_smile:

1 Like

You mentioned that IPs would automatically be assigned that we can use. How do we see what IPs are assigned for our warp connected machines?

Under Cloudflare One dashboard’s Devices list, you will see an IPv4 and IPv6 assigned to each device.

Awesome, I see it now. I was able to get the warp-to-warp traffic going but wanted to point out that I am using include routes. Therefore I had to add the 100.96.x.x CGNAT address to the included routes in my warp config for this to work correctly. Might be worth adding to eventual documentation.

1 Like

I was able to get connectivity between a VM in the cloud and my local system working properly with warp to warp. It works great and I am really excited about using this in prod.

Since warp-cli can now expose services, what’s the future of cloudflared? Will it eventually be replaced by warp-cli?

Edit: further testing shows that warp to warp only works intermittently. I can ping both machines from each other, but running a simple http server on my desktop and trying to access it from my cloud vm only works some times. Currently curl is getting an empty response from the server when sending the request from the cloud vm. Then sometimes, ping will return destination unreachable.

Doing some further testing. Here’s my setup:
Windows machine running warp.
Ubuntu VM running in a cloud provider running warp without a public ip address and using a NAT gateway for egress.
Both machines are connected to warp and I can see them listed in the device list in the warp ui.

I can (usually) ping the machines for each other, although sometimes ping will fail randomly.
I can always ssh into the VM from the Windows machine using the VM’s Cloudflare assigned virtual ip. This is reliable and always works.

I am unable to access any other service running on the VM from the windows machine using the virtual ip.

This is the output of lsof-i -n -P | more on the VM:

COMMAND    PID            USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
systemd      1            root   50u  IPv4  16660      0t0  TCP *:111 (LISTEN)
systemd      1            root   53u  IPv4  16313      0t0  UDP *:111
systemd      1            root   54u  IPv6  16663      0t0  TCP *:111 (LISTEN)
systemd      1            root   55u  IPv6  16316      0t0  UDP *:111
rpcbind    529            _rpc    4u  IPv4  16660      0t0  TCP *:111 (LISTEN)
rpcbind    529            _rpc    5u  IPv4  16313      0t0  UDP *:111
rpcbind    529            _rpc    6u  IPv6  16663      0t0  TCP *:111 (LISTEN)
rpcbind    529            _rpc    7u  IPv6  16316      0t0  UDP *:111
systemd-n  598 systemd-network   18u  IPv4  20665      0t0  UDP 10.0.0.2:68
warp-svc  1656            root   22u  IPv4  23444      0t0  UDP 10.0.0.2:35050->162.159.193.2:2408
warp-svc  1656            root   25u  IPv4  26281      0t0  TCP 10.0.0.2:35284->162.159.137.105:443 (ESTABLISHED)
warp-svc  1656            root   27u  IPv4  26166      0t0  TCP 127.0.2.2:53 (LISTEN)
warp-svc  1656            root   28u  IPv4  26167      0t0  UDP 127.0.2.2:53
warp-svc  1656            root   29u  IPv4  26168      0t0  TCP 127.0.2.3:53 (LISTEN)
warp-svc  1656            root   30u  IPv4  26169      0t0  UDP 127.0.2.3:53
warp-svc  1656            root   31u  IPv6  26170      0t0  TCP [fd01:db8:1111::2]:53 (LISTEN)
warp-svc  1656            root   32u  IPv6  26171      0t0  UDP [fd01:db8:1111::2]:53
warp-svc  1656            root   33u  IPv6  26172      0t0  TCP [fd01:db8:1111::3]:53 (LISTEN)
warp-svc  1656            root   34u  IPv6  26173      0t0  UDP [fd01:db8:1111::3]:53
warp-svc  1656            root   43u  IPv4  24922      0t0  TCP 10.0.0.2:39170->162.159.36.1:443 (ESTABLISHED)
sshd      1665            root    3u  IPv4  21757      0t0  TCP *:22 (LISTEN)
sshd      1665            root    4u  IPv6  21759      0t0  TCP *:22 (LISTEN)
gomon     1846     snap_daemon   13u  IPv4  83277      0t0  TCP 10.0.0.2:35154->169.254.169.254:80 (ESTABLISHED)
systemd-r 1951 systemd-resolve   13u  IPv4  24874      0t0  UDP 127.0.0.53:53
systemd-r 1951 systemd-resolve   14u  IPv4  24875      0t0  TCP 127.0.0.53:53 (LISTEN)
vault     1997           vault    9u  IPv4  83372      0t0  UDP 127.0.0.1:51154->127.0.0.1:8125
vault     1997           vault   11u  IPv4  83425      0t0  UDP 127.0.0.1:49619->127.0.0.1:8125
vault     1997           vault   15u  IPv4  25047      0t0  TCP 10.0.0.2:8200 (LISTEN)
vault     1997           vault   16u  IPv4  25051      0t0  TCP 10.0.0.2:8201 (LISTEN)
sshd      2640            root    4u  IPv4  45393      0t0  TCP 172.16.0.2:22->100.96.0.14:12464 (ESTABLISHED)
sshd      2691          ubuntu    4u  IPv4  45393      0t0  TCP 172.16.0.2:22->100.96.0.14:12464 (ESTABLISHED)
sshd      3103            root    4u  IPv4  53080      0t0  TCP 172.16.0.2:22->100.96.0.14:12713 (ESTABLISHED)
sshd      3145          ubuntu    4u  IPv4  53080      0t0  TCP 172.16.0.2:22->100.96.0.14:12713 (ESTABLISHED)
nginx     5472            root    6u  IPv4  66022      0t0  TCP *:80 (LISTEN)
nginx     5472            root    7u  IPv6  66023      0t0  TCP *:80 (LISTEN)
nginx     5474        www-data    6u  IPv4  66022      0t0  TCP *:80 (LISTEN)
nginx     5474        www-data    7u  IPv6  66023      0t0  TCP *:80 (LISTEN)
nginx     5475        www-data    6u  IPv4  66022      0t0  TCP *:80 (LISTEN)
nginx     5475        www-data    7u  IPv6  66023      0t0  TCP *:80 (LISTEN)

Only the SSH service is accessible using the virtual IP and I am really confused as to why I can’t access anything else.

Thank you @abe !

I tested this feature here, and worked very well.

Would be very cool could set an IP, or create an alias (like a domain) for be more easy to access some relevant device, without have to check the Devices List.

One thing that I had to do for work is on the split tunnel settings remove the 10.XX.0.0/10 setting. (I’m using on exclude mode).

As a quick update, this feature is now available in all accounts. Check out our dev docs for more information on how to get started :rocket:

Thank you testing this out @robson. This is great feedback as we’re thinking through ways to improve the UX around this feature.

@f21 would you mind opening a support ticket at https://support.cloudflare.com/hc/en-us/requests/ (or you can just email support[at]cloudflare.com) and referencing this thread? We’ll likely want to collect some logs here to help us understand the issue a bit better. Sounds like it may be related to service bindings, but we’ll be able investigate further with a bit more information from the machines running warp :slight_smile:

Happy testing :hammer_and_wrench:

2 Likes

@abe
I am unable to open a support ticket as I am on the free plan. I emailed support@Cloudflare… and I got an automated response to open a support ticket instead.

Is there a direct email address I can email to debug this?

Sure thing. Please feel free to send me a note at abe[at]cloudflare.com in that case.

1 Like

@robson are you able to share what your setup looks like? I am trying to get warp-to-warp working, but other than ssh, I am not able to access any other services running on my warp clients.

Finally had time to really dive into this. The problem with the Oracle Cloud VM is the default iptables rules that come with their platform images.

There’s a rule in there that allows states RELATED and ESTABLISHED: -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

And another one after that that rejects everything: -A INPUT -j REJECT --reject-with icmp-host-prohibited

The solution is to modify the first rule to include the state NEW or to add another rule to allow NEW before the rule that rejects everything: -A INPUT -m state --state NEW -j ACCEPT

The reason why SSH was working while other services were not was because Oracle’s default iptable rules has an entry that allows NEW on port 22 (ssh).

1 Like