Hello! We’ve been using Cloudflare on Azure Virtual Desktops for about a year with no issues. But it appears since WinDivert has been replaced with WinTun, it conflicts with something (possible suspects at the moment are: Crowdstrike Falcon, Azure Guest Agent, Microsoft RDAgent, Microsoft Geneva Agent. Microsoft Defender). That “something” causes routes to bounce first disrupting user connectivity to VM and often ends with wintun crashing. I don’t expect a solution, just posting this in case someone else is googling this and may have any ideas on how to pinpoint the culprit. No other VPN software is installed on the VMs.
- This isn’t 100% reproducible. I’m still trying to find a pattern.
- This happens in both “exclude” (full tunnel with Azure service endpoints excluded, so only “user” traffic is tunnelled and not RDP) and “include” (three test class A private subnets) modes.
- The VMs have Cloudflare resolver (DoH via native Windows 11 support) configured as the default (that is – via ncpa.cpl, outside of WARP)
Log excerpt below:
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp: Connect finished
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp: warp_start_status=Ok(())
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp_service: self.warp future resolved
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="tunnel_taskset_errors_fut"
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp_service: Network change detected new_info=IPv4: [Ethernet 2; 10.1.0.4; Ethernet; 16]; DNS servers:; 127.0.2.2:53; 127.0.2.3:53; [fd01:db8:1111::2]:53; [fd01:db8:1111::3]:53; 127.0.2.2:53; 127.0.2.3:53; [fd01:db8:1111::2]:53; [fd01:db8:1111::3]:53; old_info=IPv4: [Ethernet 2; 10.1.0.4; Ethernet; 16]; DNS servers:; 127.0.2.2:53; 127.0.2.3:53; [fd01:db8:1111::2]:53; [fd01:db8:1111::3]:53; 127.0.2.2:53; 127.0.2.3:53; [fd01:db8:1111::2]:53; [fd01:db8:1111::3]:53;
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="status_change"
2023-05-09T19:22:57.174Z INFO main_loop: warp::warp_service: WARP status: Connected
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp_service::ipc_handlers: Sending IPC status update: Connected
2023-05-09T19:22:57.174Z DEBUG main_loop: warp::warp_service::ipc_handlers: Ipc Broadcast ResponseStatus: Connected
2023-05-09T19:22:59.127Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:00.143Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:00.143Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:01.142Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:01.142Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:01.373Z DEBUG warp::warp_service::network_change: Routes changed:
Added; Interface: 17; Destination: 162.159.192.0/31; Next hop: 0.0.0.0;
Added; Interface: 17; Destination: 2606:4700::2:0:0/95; Next hop: ::;
Deleted; Interface: 17; Destination: 162.159.192.1/32; Next hop: 0.0.0.0;
Added; Interface: 17; Destination: 2606:4700::1:0:0/96; Next hop: ::;
Added; Interface: 17; Destination: 2606:4700::8000:0/97; Next hop: ::;
Added; Interface: 17; Destination: 104.19.236.25/32; Next hop: 0.0.0.0;
{omitted for brevity}
Added; Interface: 17; Destination: fd01:db8:1111::3/128; Next hop: ::;
2023-05-09T19:23:02.158Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:02.158Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:03.346Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:03.346Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:04.160Z DEBUG warp_tun::win: Shutting down the wintun tunnel
2023-05-09T19:23:04.165Z DEBUG warp_tun::win: Stopping drive_read_wait_handle due to shutdown
2023-05-09T19:23:04.346Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:04.346Z DEBUG dns_proxy::errors: DnsProxy timeout target=254.169.254.169.in-addr.arpa.
2023-05-09T19:23:04.346Z WARN trust_dns_proto::udp::udp_stream: error sending message to [2606:4700:110:86cf:ac9d:3a8a:906e:f73a]:54106 on udp_socket, dropping response: The requested address is not valid in its context. (os error 10049)
2023-05-09T19:23:04.388Z DEBUG warp::warp_service::network_change: Routes changed:
Deleted; Interface: 17; Destination: 2606:4700:0:1000::/52; Next hop: ::;
Deleted; Interface: 17; Destination: 2606:4700:0:2000::/51; Next hop: ::;
Deleted; Interface: 17; Destination: 2606:4700:0:4000::/50; Next hop: ::;
Deleted; Interface: 17; Destination: 2606:4700:0:8000::/49; Next hop: ::;
Deleted; Interface: 17; Destination: 2606:4700:1::/48; Next hop: ::;
{omitted for brevity}
Deleted; Interface: 0; Destination: ::/8; Next hop: ::;
2023-05-09T19:23:04.391Z WARN main_loop: warp::warp: Tunnel task experienced error task_name="tun driver" err=TunDriverStopped
2023-05-09T19:23:04.391Z WARN main_loop: warp::warp_service: Tunnel connection experienced error error=Inflight(TunDriverStopped)
2023-05-09T19:23:04.391Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="tunnel_taskset_errors_fut"
2023-05-09T19:23:04.391Z DEBUG main_loop: warp::warp::dns_recovery::windows: Reverting DNS settings old_dns=RestoreDNS { name_servers: {16: InterfaceNameServers { idx: 16, name_servers: [] }} }
2023-05-09T19:23:04.391Z DEBUG main_loop:set_dns_settings{guid="{C27FCB6D-BFD2-4A49-8CB9-71270276FD46}" nameservers="" service="Tcpip6"}: network_info::win::iphelper: Updating registry entry service="Tcpip6" nameservers=""
2023-05-09T19:23:04.391Z DEBUG main_loop:set_dns_settings{guid="{C27FCB6D-BFD2-4A49-8CB9-71270276FD46}" nameservers="" service="Tcpip"}: network_info::win::iphelper: Updating registry entry service="Tcpip" nameservers=""
2023-05-09T19:23:04.391Z DEBUG main_loop: network_info::win::iphelper: Retrieved old name servers from registry old_nameservers=[127.0.2.2, 127.0.2.3, fd01:db8:1111::2, fd01:db8:1111::3]
2023-05-09T19:23:04.391Z DEBUG main_loop: warp::warp::dns_recovery::windows: DNS settings reverted (from "successfully applied RestoreDNS { name_servers: {16: InterfaceNameServers { idx: 16, name_servers: [] }} }")
2023-05-09T19:23:04.392Z DEBUG main_loop: firewall: Firewall reset to defaults
2023-05-09T19:23:04.393Z DEBUG main_loop: wfp::session: Initializing WFP Sublayer provider=ProviderKey(79D48B13-9D76-49C8-A6CA-FC68047EC58F) name="Cloudflare WARP Firewall"
2023-05-09T19:23:04.395Z WARN main_loop: warp::warp: Cancelled tunnel task experienced error task_name="Tunnel in/out" err=OsError(Os { code: 10022, kind: InvalidInput, message: "An invalid argument was supplied." })
2023-05-09T19:23:04.395Z DEBUG warp::warp::dns_forwarding::windows: Removed DNS IP alias
2023-05-09T19:23:04.395Z DEBUG warp::warp::dns_forwarding::windows: Removed DNS IP alias
2023-05-09T19:23:04.395Z WARN warp::warp::dns_forwarding::windows: Failed to revert remove IP DNS address error=Os { code: 1231, kind: NetworkUnreachable, message: "The network location cannot be reached. For information about network troubleshooting, see Windows Help." } ip=fd01:db8:1111::2
2023-05-09T19:23:04.395Z WARN warp::warp::dns_forwarding::windows: Failed to revert remove IP DNS address error=Os { code: 1231, kind: NetworkUnreachable, message: "The network location cannot be reached. For information about network troubleshooting, see Windows Help." } ip=fd01:db8:1111::3
2023-05-09T19:23:04.395Z DEBUG warp::warp::dns_forwarding::windows: Removing firewall rules for External DNS Servers ips=[fd01:db8:1111::2, fd01:db8:1111::3]
2023-05-09T19:23:04.415Z DEBUG warp::warp_service::network_change: Routes changed:
Deleted; Interface: 1; Destination: 127.0.2.2/32; Next hop: 0.0.0.0;
Deleted; Interface: 1; Destination: 127.0.2.3/32; Next hop: 0.0.0.0;
2023-05-09T19:23:04.429Z DEBUG main_loop: warp::warp_service: Reconnecting on connection error error=TunDriverStopped
2023-05-09T19:23:04.429Z DEBUG main_loop: firewall: Firewall allow private IPs
2023-05-09T19:23:04.430Z DEBUG main_loop: wfp::session: Initializing WFP Sublayer provider=ProviderKey(79D48B13-9D76-49C8-A6CA-FC68047EC58F) name="Cloudflare WARP Firewall"
2023-05-09T19:23:04.432Z WARN main_loop: warp::warp_service: Disconnected, but reason unknown net_info=IPv4: [Ethernet 2; 10.1.0.4; Ethernet; 16]; DNS servers:; 168.63.129.16:53;
2023-05-09T19:23:04.432Z DEBUG main_loop: firewall: Firewall allow private IPs
2023-05-09T19:23:04.433Z DEBUG main_loop: wfp::session: Initializing WFP Sublayer provider=ProviderKey(79D48B13-9D76-49C8-A6CA-FC68047EC58F) name="Cloudflare WARP Firewall"
2023-05-09T19:23:04.436Z INFO main_loop: warp::warp_service: captive_portal_fw_until: Indefinitely
2023-05-09T19:23:04.436Z DEBUG main_loop: warp::warp: Using auto fallback: true
2023-05-09T19:23:04.436Z DEBUG main_loop: warp::warp: Current Network: IPv4: [Ethernet 2; 10.1.0.4; Ethernet; 16]; DNS servers:; 168.63.129.16:53;
2023-05-09T19:23:04.436Z INFO main_loop: warp::warp: Initiate WARP connection
2023-05-09T19:23:04.436Z DEBUG main_loop: firewall: Firewall allow tunnel
2023-05-09T19:23:04.437Z DEBUG main_loop: wfp::session: Initializing WFP Sublayer provider=ProviderKey(79D48B13-9D76-49C8-A6CA-FC68047EC58F) name="Cloudflare WARP Firewall"
2023-05-09T19:23:04.439Z DEBUG main_loop: warp::warp::happy_eyeballs: Attempting Happy Eyeballs to 162.159.192.2:2408 / [2606:4700:d0::a29f:c002]:2408
2023-05-09T19:23:04.439Z DEBUG main_loop: warp::warp::happy_eyeballs: Start racer 10.1.0.4:60090 ---> 162.159.192.2:2408
2023-05-09T19:23:04.440Z DEBUG main_loop: warp::warp::happy_eyeballs: Sent handshake initiation to 162.159.192.2:2408
2023-05-09T19:23:04.440Z DEBUG main_loop: warp::warp::happy_eyeballs: Happy eyeballs to [2606:4700:d0::a29f:c002]:2408 failed: Os { code: 10051, kind: NetworkUnreachable, message: "A socket operation was attempted to an unreachable network." }
2023-05-09T19:23:04.440Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="status_change"
2023-05-09T19:23:04.440Z INFO main_loop: warp::warp_service: WARP status: Connecting
2023-05-09T19:23:04.440Z DEBUG main_loop: warp::warp_service::ipc_handlers: Sending IPC status update: Connecting
2023-05-09T19:23:04.440Z DEBUG main_loop: warp::warp_service::ipc_handlers: Ipc Broadcast ResponseStatus: Connecting
2023-05-09T19:23:04.448Z DEBUG main_loop: warp::warp::happy_eyeballs: Got response from 162.159.192.2:2408
2023-05-09T19:23:04.448Z DEBUG main_loop: warp::warp: Connected to 162.159.192.2:2408
2023-05-09T19:23:04.508Z INFO main_loop: wintun: Using existing driver 0.14
2023-05-09T19:23:04.515Z INFO main_loop: wintun: Creating adapter
2023-05-09T19:23:04.653Z DEBUG main_loop:set_dns_settings{guid="{DB484304-DB04-6AA0-A33D-7236836B364D}" nameservers="fd01:db8:1111::2,fd01:db8:1111::3" service="Tcpip6"}: network_info::win::iphelper: Updating registry entry service="Tcpip6" nameservers="fd01:db8:1111::2,fd01:db8:1111::3"
2023-05-09T19:23:04.653Z DEBUG main_loop:set_dns_settings{guid="{DB484304-DB04-6AA0-A33D-7236836B364D}" nameservers="127.0.2.2,127.0.2.3" service="Tcpip"}: network_info::win::iphelper: Updating registry entry service="Tcpip" nameservers="127.0.2.2,127.0.2.3"
2023-05-09T19:23:04.654Z DEBUG main_loop: network_info::win::iphelper: Retrieved old name servers from registry old_nameservers=[]
2023-05-09T19:23:04.657Z DEBUG main_loop:set_dns_settings{guid="{DB484304-DB04-6AA0-A33D-7236836B364D}" nameservers="fd01:db8:1111::2,fd01:db8:1111::3" service="Tcpip6"}: network_info::win::iphelper: Updating registry entry service="Tcpip6" nameservers="fd01:db8:1111::2,fd01:db8:1111::3"
2023-05-09T19:23:04.657Z DEBUG main_loop:set_dns_settings{guid="{DB484304-DB04-6AA0-A33D-7236836B364D}" nameservers="127.0.2.2,127.0.2.3" service="Tcpip"}: network_info::win::iphelper: Updating registry entry service="Tcpip" nameservers="127.0.2.2,127.0.2.3"
2023-05-09T19:23:04.658Z DEBUG main_loop: network_info::win::iphelper: Retrieved old name servers from registry old_nameservers=[127.0.2.2, 127.0.2.3, fd01:db8:1111::2, fd01:db8:1111::3]
2023-05-09T19:23:04.659Z DEBUG main_loop: firewall: Firewall allow interface iftype53_32768
2023-05-09T19:23:04.660Z DEBUG main_loop: wfp::session: Initializing WFP Sublayer provider=ProviderKey(79D48B13-9D76-49C8-A6CA-FC68047EC58F) name="Cloudflare WARP Firewall"
2023-05-09T19:23:04.664Z DEBUG main_loop: firewall: Firewall allow private IPs
2023-05-09T19:23:04.697Z DEBUG main_loop: wfp::session: Initializing WFP Sublayer provider=ProviderKey(79D48B13-9D76-49C8-A6CA-FC68047EC58F) name="Cloudflare WARP Firewall"
2023-05-09T19:23:04.767Z INFO main_loop: warp::warp::dns_forwarding::windows: Associated unicast address and added firewall rule ip=fd01:db8:1111::2
2023-05-09T19:23:04.785Z INFO main_loop: warp::warp::dns_forwarding::windows: Associated unicast address and added firewall rule ip=fd01:db8:1111::3
2023-05-09T19:23:04.796Z DEBUG main_loop: warp::warp::dns_recovery::windows: Applying DNS settings name_servers=[127.0.2.2, 127.0.2.3, fd01:db8:1111::2, fd01:db8:1111::3] v4_iface=Some(Ethernet 2; 10.1.0.4; Ethernet; 16) v6_iface=None
2023-05-09T19:23:04.796Z INFO warp::warp: DnsOverWarp. Client addr is 172.16.0.2:0
2023-05-09T19:23:04.796Z WARN trust_dns_resolver::hosts: could not parse an IP from hosts file
2023-05-09T19:23:04.796Z INFO dns_proxy::proxy: Default fallbacks configured default_fallback_ips=[168.63.129.16:53] config=ResolverConfig { domain: Some(Name(".")), search: [], name_servers: NameServerConfigGroup([NameServerConfig { socket_addr: 168.63.129.16:53, protocol: Udp, tls_dns_name: None, trust_nx_responses: true, tls_config: None, bind_addr: None }, NameServerConfig { socket_addr: 168.63.129.16:53, protocol: Tcp, tls_dns_name: None, trust_nx_responses: true, tls_config: None, bind_addr: None }], None) } sys_options=ResolverOpts { ndots: 1, timeout: 2s, attempts: 0, rotate: true, check_names: true, edns0: true, validate: false, ip_strategy: Ipv4thenIpv6, cache_size: 32, use_hosts_file: true, positive_min_ttl: None, negative_min_ttl: None, positive_max_ttl: None, negative_max_ttl: None, num_concurrent_reqs: 8, preserve_intermediates: true, try_tcp_on_error: false, server_ordering_strategy: QueryStatistics, recursion_desired: true, authentic_data: false }
2023-05-09T19:23:04.796Z DEBUG main_loop:set_dns_settings{guid="{C27FCB6D-BFD2-4A49-8CB9-71270276FD46}" nameservers="fd01:db8:1111::2,fd01:db8:1111::3" service="Tcpip6"}: network_info::win::iphelper: Updating registry entry service="Tcpip6" nameservers="fd01:db8:1111::2,fd01:db8:1111::3"
2023-05-09T19:23:04.797Z WARN trust_dns_resolver::hosts: could not parse an IP from hosts file
2023-05-09T19:23:04.797Z DEBUG main_loop:set_dns_settings{guid="{C27FCB6D-BFD2-4A49-8CB9-71270276FD46}" nameservers="127.0.2.2,127.0.2.3" service="Tcpip"}: network_info::win::iphelper: Updating registry entry service="Tcpip" nameservers="127.0.2.2,127.0.2.3"
2023-05-09T19:23:04.797Z DEBUG main_loop: network_info::win::iphelper: Retrieved old name servers from registry old_nameservers=[]
2023-05-09T19:23:04.823Z WARN main_loop: trust_dns_resolver::hosts: could not parse an IP from hosts file
2023-05-09T19:23:04.920Z DEBUG warp::warp_service::network_change: Routes changed:
Deleted; Interface: 17; Destination: 162.159.193.255/32; Next hop: 0.0.0.0;
Added; Interface: 17; Destination: 192.0.1.0/24; Next hop: 0.0.0.0;
Deleted; Interface: 17; Destination: 192.0.1.255/32; Next hop: 0.0.0.0;
Added; Interface: 17; Destination: 104.19.236.128/25; Next hop: 0.0.0.0;
Deleted; Interface: 17; Destination: 104.19.236.255/32; Next hop: 0.0.0.0;
{omitted for brevity}
Added; Interface: 1; Destination: 127.0.2.3/32; Next hop: 0.0.0.0;
2023-05-09T19:23:04.925Z DEBUG main_loop: warp::warp::connectivity_check: Resolved connectivity.cloudflareclient.com to [162.159.137.65, 162.159.138.65]
2023-05-09T19:23:04.925Z DEBUG main_loop: warp::warp::connectivity_check: Resolved warp-svc. to [127.0.2.2, 127.0.2.3]
2023-05-09T19:23:04.946Z DEBUG main_loop: warp::warp::connectivity_check: fl=572f8
h=engage.cloudflareclient.com
ip=20.10.13.192
ts=1683660184.941
visit_scheme=https
uag=
colo=IAD
sliver=none
http=http/2
loc=US
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519
2023-05-09T19:23:05.004Z DEBUG main_loop: warp::warp::connectivity_check: fl=573f3
h=connectivity.cloudflareclient.com
ip=104.28.210.133
ts=1683660184.994
visit_scheme=https
uag=
colo=IAD
sliver=none
http=http/2
loc=US
tls=TLSv1.3
sni=plaintext
warp=on
gateway=off
rbi=off
kex=X25519
2023-05-09T19:23:05.004Z DEBUG main_loop: warp::warp: Trace status: Ok(TraceResult { metal_id: "573f3", timestamp: 2023-05-09 19:23:04.993999872 +00:00:00, colo: "IAD", warp: On, gateway: Off })
2023-05-09T19:23:05.004Z DEBUG main_loop: warp::warp: Connect finished