SSH with short lived certificates: "invalid character '<' looking for beginning of value"

Website, Application, Performance Security
1

I’ve set up cloudflare SSH with short lived certificates today. Seems to work. But when I ssh from the command line, the first line of output on the command line is occasionally:

“invalid character ‘<’ looking for beginning of value”

It doesn’t actually cause a problem – the ssh session works afterward. But it seems suspicious.

It’s intermittent. Happens perhaps 1 in 10 attempts or so.

Anyone else experiencing this? Seems like a bug somewhere in cloudflare’s infrastructure.

2

It may be worth doing an SSH connection with the -v

It may help indicate where this flag is coming from.

EG:

ssh -v USER@IP_ADDRESS

I haven’t heard or seen something inside the Cloudflare network to indicate an ever-reaching issue.

3

Not much more informative, unfortunately.

% ssh -v [redacted1]

OpenSSH_8.1p1, LibreSSL 2.7.3

debug1: Reading configuration data /Users/[redacted2]/.ssh/config

debug1: /Users/[redacted2]/.ssh/config line 14: Applying options for [redacted1]

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: /etc/ssh/ssh_config line 47: Applying options for *

debug1: Executing proxy command: exec bash -c ‘/opt/homebrew/bin/cloudflared access ssh-gen --hostname [redacted3]; ssh -tt [redacted4]@cfpipe-[redacted3] >&2 <&1’

debug1: identity file /Users/[redacted2]/.ssh/id_rsa type 0

debug1: identity file /Users/[redacted2]/.ssh/id_rsa-cert type -1

debug1: identity file /Users/[redacted2]/.ssh/id_dsa type -1

debug1: identity file /Users/[redacted2]/.ssh/id_dsa-cert type -1

debug1: identity file /Users/[redacted2]/.ssh/id_ecdsa type -1

debug1: identity file /Users/[redacted2]/.ssh/id_ecdsa-cert type -1

debug1: identity file /Users/[redacted2]/.ssh/id_ed25519 type -1

debug1: identity file /Users/[redacted2]/.ssh/id_ed25519-cert type -1

debug1: identity file /Users/[redacted2]/.ssh/id_xmss type -1

debug1: identity file /Users/[redacted2]/.ssh/id_xmss-cert type -1

debug1: Local version string SSH-2.0-OpenSSH_8.1

invalid character ‘<’ looking for beginning of value

A browser window should have opened at the following URL:

4

It seems that it’s the “cloudflared access ssh-gen --hostname [redacted1]” command that’s outputting “invalid character ‘<’ looking for beginning of value”. When I invoke it directly, it occasionally is outputting “invalid character ‘<’ looking for beginning of value”.

Is there a way to increase the verbosity of the logging on the “cloudflared access…” command?

5

@MoreHelp could you take a look?

6

You may get some more useful output with ssh -vv. It seems like something isn’t escaping correctly in the configuration. If you could provide the redacted output for ssh -vv when you are able to replicate that may provide the information needed here.

You can also add --loglevel debug to your cloudflared access command to get the debug output.

7

ssh -vv seems like it’s barking up the wrong tree. I can reproduce the problem when I invoke cloudflared directly.

I’ve tried loglevel trace and loglevel debug. Neither gives additional output.

bash-3.2$ cloudflared --loglevel debug access ssh-gen --hostname [redacted1]
invalid character ‘<’ looking for beginning of value

bash-3.2$ cloudflared --loglevel trace access ssh-gen --hostname [redacted1]
invalid character ‘<’ looking for beginning of value

8

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.