Post mortems - What happened with London today? ~9mins outage

Widely reported there was an outage effecting LDN from 9.37pm to 9.46pm (intermittent but heavy), status page as useless as ever, does CF plan to provide post-mortems on these now resolved incidients?

It doesn’t sit comfortably with me that these get swept under the carpet, transparency is underrated I feel.

Hi @matt53 sorry for the troubles, I don’t see any tickets from today related to LDN, but if the issues were not related to the cf network, they won’t show on that page. Did you open a ticket with Support, I can check further. And, what errors were you seeing?

The process of extracting blood from a stone is ongoing…

Anyway this from Matthew Prince a few minutes ago:
“Was UK ISP’s connection to us. So isolated to sites on us and eyeballs on them. They fixed so should be”

Hoping someone can share the info here - which ISPs connection went down? My hosting provider’s status page is as green as CFs… If I’m reading this right he suggesting an eyeball network problem yet this was observed by Google StackDriver, Pingdom, Monitor2 and myself several thousand miles away

1 Like

Thank you for sharing that, @matt53, if you’re not in the intersection of our sites on us and eyeballs on them, it seems like your issue is either unrelated (and strangely coincidental) or perhaps something related that we’ve not detected. In that case, it’s worth creating a ticket to see if Support can locate something in their logs that may pinpoint the issue. If you do, please share the ticket number here as I’d like to add myself to it to track.

Ive reached out to the other people effected on the bird network to see if we share any commonality, will aggregate and create a ticket.

Wait…
https://community.cloudflare.com/t/elevated-number-of-522-errors-in-lhr-london/268191

So it’s finally known and we’re not crazy lol. But what happened?

I’m more interested in how CF always seems to be the last to know, adspedia telling people on Twitter there is no London problem, no tickets internally related to LDN, status page nice and green throughout incident and hours later, and incorrect hypothesis from matthew price…

Pingdom (the fisher price of monitoring) first alerted me (confirmed via multiple test points) so it was hardly some hard to pinpoint issue.

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.