ERR failed to connect to origin error="websocket: bad handshake"

I am setting up Zero Trust SSH connections according to this guide. It does not work. I would ask for more information on how to diagnose the issue. This is a fresh domain bought through Cloudflare today.

I am running and getting:

cloudflared access ssh --hostname tokyo1.xxx.pro
2022-02-03T14:02:33Z ERR failed to connect to origin error="websocket: bad handshake" originURL=https://tokyo1.xxx.pro
websocket: bad handshake

Here is the debug output I capture with cloudflared tunnel --loglevel debug run on the server side. The tunnel is definitely working because it attempts to connect.

2022-02-03T14:02:33Z DBG CF-RAY: 6d7c328f6c4269dc-MAD GET / HTTP/2.0
2022-02-03T14:02:33Z DBG Inbound request CF-RAY=6d7c328f6c4269dc-MAD Header="map[Accept-Encoding:[gzip] Cdn-Loop:[cloudflare] Cf-Access-Authenticated-User-Email:[[email protected]] Cf-Access-Jwt-Assertion:[eyJhbGciOiJSUzI1NiIsImtpZCI6IjIyMGY3MWI3YTUwNGM2OTI5YzdjOWY1M2JkMjc5ZDU2MDQxMWNmODBlY2ZlNmYwZDdjNDlmNzA3N2NjZmM0MDMifQ.eyJhdWQiOlsiNmFiYmI0YmMxNDM2MTRiOWY0N2JmOTc2ZDQ5ZmE2NTRjZDkzNmE1MTNmYTQwNjkyNzQzYTI2OWM1NDdjNjNjNSJdLCJlbWFpbCI6Im1pa2tvQHJlZGlubm92YXRpb24uY29tIiwiZXhwIjoxNjQzOTc5ODQ1LCJpYXQiOjE2NDM4OTM0NDUsIm5iZiI6MTY0Mzg5MzQ0NSwiaXNzIjoiaHR0cHM6Ly9kYW0tYm90cy5jbG91ZGZsYXJlYWNjZXNzLmNvbSIsInR5cGUiOiJhcHAiLCJpZGVudGl0eV9ub25jZSI6IkVQR1hYbklyZTVhcTRvazEiLCJzdWIiOiI4Mjg2NWYyNC03NTZjLTQ3NzktOWQwMC01M2YzYjNkNTNiNTkiLCJjb3VudHJ5IjoiR0kifQ.phTGYE6_CUUMrQs7ijUL8M9wm4lZrXXWQrOiDMeBHjkiuyQnZ9ENcKeJGh1-czFRaBm_cRiEOxuTgLNI0363ByqDoGgAYxerAzQFqhNwtdny9q4aQkmZnCxqkI81XhB02-2r2uwVUxKGxNqhJfZZXiwJMIUlUC926MPfbcxMnb9G7zeFRA9bpOh4EzEUo7Co65x_hNCO_1Wni7ZTKP35WOtZNmt2e1b24BBDeKMakiVjld7ATACu12gDgmlixKtRucp1_AvLB3lOGfCdSZheNw2G4FnFqqyGLlhH7kQRsk4IUUXg-s8lEEzQ9mAFznh3E0Yuehs9jdefQV2l45F-EQ] Cf-Access-Token:[eyJhbGciOiJSUzI1NiIsImtpZCI6IjIyMGY3MWI3YTUwNGM2OTI5YzdjOWY1M2JkMjc5ZDU2MDQxMWNmODBlY2ZlNmYwZDdjNDlmNzA3N2NjZmM0MDMifQ.eyJhdWQiOlsiNmFiYmI0YmMxNDM2MTRiOWY0N2JmOTc2ZDQ5ZmE2NTRjZDkzNmE1MTNmYTQwNjkyNzQzYTI2OWM1NDdjNjNjNSJdLCJlbWFpbCI6Im1pa2tvQHJlZGlubm92YXRpb24uY29tIiwiZXhwIjoxNjQzOTc5ODQ1LCJpYXQiOjE2NDM4OTM0NDUsIm5iZiI6MTY0Mzg5MzQ0NSwiaXNzIjoiaHR0cHM6Ly9kYW0tYm90cy5jbG91ZGZsYXJlYWNjZXNzLmNvbSIsInR5cGUiOiJhcHAiLCJpZGVudGl0eV9ub25jZSI6IkVQR1hYbklyZTVhcTRvazEiLCJzdWIiOiI4Mjg2NWYyNC03NTZjLTQ3NzktOWQwMC01M2YzYjNkNTNiNTkiLCJjb3VudHJ5IjoiR0kifQ.phTGYE6_CUUMrQs7ijUL8M9wm4lZrXXWQrOiDMeBHjkiuyQnZ9ENcKeJGh1-czFRaBm_cRiEOxuTgLNI0363ByqDoGgAYxerAzQFqhNwtdny9q4aQkmZnCxqkI81XhB02-2r2uwVUxKGxNqhJfZZXiwJMIUlUC926MPfbcxMnb9G7zeFRA9bpOh4EzEUo7Co65x_hNCO_1Wni7ZTKP35WOtZNmt2e1b24BBDeKMakiVjld7ATACu12gDgmlixKtRucp1_AvLB3lOGfCdSZheNw2G4FnFqqyGLlhH7kQRsk4IUUXg-s8lEEzQ9mAFznh3E0Yuehs9jdefQV2l45F-EQ] Cf-Connecting-Ip:[104.255.135.103] Cf-Ipcountry:[GI] Cf-Ray:[6d7c328f6c4269dc-MAD] Cf-Visitor:[{\"scheme\":\"https\"}] Cf-Warp-Tag-Id:[6817a80e-06af-439d-a81e-cad70aef3e58] Sec-Websocket-Key:[+OyGnE/RexUcfhKLC87Aqg==] Sec-Websocket-Version:[13] User-Agent:[Go-http-client/1.1] X-Forwarded-For:[104.255.135.103] X-Forwarded-Proto:[https]]" host=tokyo1.xxx.pro path=/ rule=1
2022-02-03T14:02:33Z DBG CF-RAY: 6d7c328f6c4269dc-MAD Request Content length unknown
2022-02-03T14:02:33Z DBG CF-RAY: 6d7c328f6c4269dc-MAD Status: 404 Not Found served by ingress 1
2022-02-03T14:02:33Z DBG CF-RAY: 6d7c328f6c4269dc-MAD Response Headers map[]
2022-02-03T14:02:33Z DBG CF-RAY: 6d7c328f6c4269dc-MAD Response content length 0

The error message is unhelpful. The guide provided by Cloudflare lacks any useful diagnostics help - how to troubleshoot issues. Is there any to force the tunnel daemon to tell why the connection fails?

I have extensively searched the past issues on this topic on

I have checked: Websockets, Flexible SSL, Cookie Settings in Zero Trust application config. I suspect it would be something in domain names, but I have triple checked those multiple times.

Here is my config.yml:

tunnel: 5ce3af1c-7efa-4228-aac3-9500ae0cabda
credentials-file: /root/.cloudflared/5ce3af1c-7efa-4228-aac3-9500ae0cabda.json

ingress:
  - hostname: tokio1.xxx.pro
    service: ssh://localhost:22
  - service: http_status:404

I did another attempt by forcing config.yml to go to SSH everytime and not have HTTP 404 handler. In this case, cloudflared recognises the incoming SSH connection, so the problem lies how cloudflared detects the host name.

ingress:
#  - hostname: tokio1.xxx.pro
   - service: ssh://localhost:22
#  - service: http_status:404

I am getting another error from the server cloudflared:

2022-02-09T11:39:01Z DBG Inbound request CF-RAY=6dacd08f7e6f666b-MAD Header="map[Accept-Encoding:[gzip] Cdn-Loop:[cloudflare] Cf-Access-Authenticated-User-Email:[[email protected]] Cf-Access-Jwt-Assertion:[eyJhbGciOiJSUzI1NiIsImtpZCI6IjIyMGY3MWI3YTUwNGM2OTI5YzdjOWY1M2JkMjc5ZDU2MDQxMWNmODBlY2ZlNmYwZDdjNDlmNzA3N2NjZmM0MDMifQ.eyJhdWQiOlsiNmFiYmI0YmMxNDM2MTRiOWY0N2JmOTc2ZDQ5ZmE2NTRjZDkzNmE1MTNmYTQwNjkyNzQzYTI2OWM1NDdjNjNjNSJdLCJlbWFpbCI6Im1pa2tvQHJlZGlubm92YXRpb24uY29tIiwiZXhwIjoxNjQ0NDgxNzIyLCJpYXQiOjE2NDQzOTUzMjIsIm5iZiI6MTY0NDM5NTMyMiwiaXNzIjoiaHR0cHM6Ly9kYW0tYm90cy5jbG91ZGZsYXJlYWNjZXNzLmNvbSIsInR5cGUiOiJhcHAiLCJpZGVudGl0eV9ub25jZSI6IlBwNVVhSXlxQ2pNMUFHQngiLCJzdWIiOiI4Mjg2NWYyNC03NTZjLTQ3NzktOWQwMC01M2YzYjNkNTNiNTkiLCJjb3VudHJ5IjoiR0kifQ.TVi9dyNAt_lIbVarCCAa1nBylp3Od2wq5Yxs6dl8D1ygZmUaBWE0i8vvNTRI2nxuTXafmPxab1DFIh8kxO6JXdRuXFUBcNA86wlzZR71LiEJZy0lN97QxMpuZxJmUVs5n3PDucRVVRvwPLEErR3O9AIi01uvUNrYe7twlqY4jicuD5rB6CjxFn2Q3_pMEvGF499VDFw4LZffNPMJIIRD9JTAnHJxJD9B9QhXoQ0PJslgV72nDcYTh59bsI4U7oL78-II-Iq-k0eGEBXxV4Tuy43zpJeYACgMTlZ-WYOrCkUpHZn4XQMTETyT0Nc-NrMeGruaIUoYR4cYv1bPavEOWA] Cf-Access-Token:[eyJhbGciOiJSUzI1NiIsImtpZCI6IjIyMGY3MWI3YTUwNGM2OTI5YzdjOWY1M2JkMjc5ZDU2MDQxMWNmODBlY2ZlNmYwZDdjNDlmNzA3N2NjZmM0MDMifQ.eyJhdWQiOlsiNmFiYmI0YmMxNDM2MTRiOWY0N2JmOTc2ZDQ5ZmE2NTRjZDkzNmE1MTNmYTQwNjkyNzQzYTI2OWM1NDdjNjNjNSJdLCJlbWFpbCI6Im1pa2tvQHJlZGlubm92YXRpb24uY29tIiwiZXhwIjoxNjQ0NDgxNzIyLCJpYXQiOjE2NDQzOTUzMjIsIm5iZiI6MTY0NDM5NTMyMiwiaXNzIjoiaHR0cHM6Ly9kYW0tYm90cy5jbG91ZGZsYXJlYWNjZXNzLmNvbSIsInR5cGUiOiJhcHAiLCJpZGVudGl0eV9ub25jZSI6IlBwNVVhSXlxQ2pNMUFHQngiLCJzdWIiOiI4Mjg2NWYyNC03NTZjLTQ3NzktOWQwMC01M2YzYjNkNTNiNTkiLCJjb3VudHJ5IjoiR0kifQ.TVi9dyNAt_lIbVarCCAa1nBylp3Od2wq5Yxs6dl8D1ygZmUaBWE0i8vvNTRI2nxuTXafmPxab1DFIh8kxO6JXdRuXFUBcNA86wlzZR71LiEJZy0lN97QxMpuZxJmUVs5n3PDucRVVRvwPLEErR3O9AIi01uvUNrYe7twlqY4jicuD5rB6CjxFn2Q3_pMEvGF499VDFw4LZffNPMJIIRD9JTAnHJxJD9B9QhXoQ0PJslgV72nDcYTh59bsI4U7oL78-II-Iq-k0eGEBXxV4Tuy43zpJeYACgMTlZ-WYOrCkUpHZn4XQMTETyT0Nc-NrMeGruaIUoYR4cYv1bPavEOWA] Cf-Connecting-Ip:[104.255.135.103] Cf-Ipcountry:[GI] Cf-Ray:[6dacd08f7e6f666b-MAD] Cf-Visitor:[{\"scheme\":\"https\"}] Cf-Warp-Tag-Id:[0990e029-c188-4019-a8d1-3b415670bb93] Sec-Websocket-Key:[TE8WPKA14idOVFe3EQW0Zw==] Sec-Websocket-Version:[13] User-Agent:[Go-http-client/1.1] X-Forwarded-For:[104.255.135.103] X-Forwarded-Proto:[https]]" host=tokyo1.xxx.pro path=/ rule=0
2022-02-09T11:39:01Z DBG CF-RAY: 6dacd08f7e6f666b-MAD Request Content length unknown
2022-02-09T11:39:07Z DBG tunnel to origin copy: readfrom tcp 127.0.0.1:44236->127.0.0.1:22: stream error: stream ID 3; NO_ERROR

There is a reply string from the SSH on the client though:

/usr/local/bin/cloudflared access ssh --hostname tokyo1.xxx.pro
SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.3

Invalid SSH identification string.

Now if I cannot with SSH:

Host tokyo1
ProxyCommand /usr/local/bin/cloudflared access ssh --hostname tokyo1.xx.pro
ForwardAgent yes
User ubuntu

The connection works.

I do not know how to debug cloudflared hostname matching issues.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.