Tail latency on load balancer traffic

We’re seeing occasional requests that are going via a CF load balancer to have much increased latency, anywhere from 900ms to 3.6 seconds additional latency. Examining logs & packet captures this seems to be a result of the request being delayed getting to the origin server. The client, CF POP and origin servers are all located in the SF bay area, no long network hops involved.

Is it expected that the tail latency is this bad? Are there any tools available in CF to help diagnose this? We don’t see this long tail latency when making requests direct to the origin server.