Severe Cloudflare APO page speed issues for Telekom Customers in Germany

For the past 2 weeks, there have been extreme loading times for Telekom customers in Germany when Cloudflare APO is used. Unfortunately, I am also a Telekom customer and regret it for several reasons.
Page loading times are extremely poor to almost non-existent. I have seen everything from 15+ seconds to 2:30 minutes. Normal loading time way below 1sec. This applies to both the backend and the Cloudflare dashboard. The forum here took over a minute to load.
I switch to a German VPN server, and the loading times are under 0.8 seconds as usual. Also, pausing Cloudflare resolves this serious issue.
The same problem occurs over Telekom’s LTE.

I have gone through these threads here, and all describe the same situation except for negotiation difficulties between Telekom and Cloudflare, where Telekom seeks payment for the traffic, and Cloudflare refuses to pay. This apparently comes at the expense of all customers. Disabling Http3 was suggested as a solution. This has, at best, only brought marginal improvement. This is a significant operational setback as our main customers are in Germany, and many are conservative and traditional, choosing to stay with Telekom. The average Joe does not tinker with their experimental Http3 settings of their browser, they just visit another site…

Are there any solutions other than kicking out Cloudflare and canceling the subscription?

Additional threads on the topic:

https://telekomhilft.telekom.de/t5/Festnetz-Internet/3-Tage-Cloudflare-Probleme/m-p/6342875/emcs_t/S2h8ZW1haWx8dG9waWNfc3Vic2NyaXB0aW9ufExNSFJZRFFVS1pKMkpQfDYzNDI4NzV8U1VCU0NSSVBUSU9OU3xoSw#M2131936

Hey, unfortunately I don’t have a solution, but are you sure that it’s the APO that causing this? Isn’t the case is that those websites are also on a Free plan while the rest of isn’t? Since atm the websites behind a Free plan are being routed to an US server if the visitor’s ISP is Deutsche Telekom or Magyar Telekom (Biggest ISP in Hungary, also the subsidiary of Deutsche Telekom).

Until Cloudflare and Deutsche Telekom agrees in something, you can only fix this by upgrading to a Pro plan, since as far as I know those are routed correctly at the moment.

Yes, I’m 100% sure its Cloudflare - Telekom issue. Posting this thread over here 30mins ago and the tab used to send it is still on loading. I couldn’t even log into the Cloudlfare dashboard a few hours ago. It is dead. As I said using paid VPN server in Düsseldorf or Frankfurt and all is back to normal with page loads below 1sec. Our Colleges with Vodafone and LTE do not see any issues.
Pausing Cloudlfare → Page loads back to normal. If we see this our customers do too. This is nuts.

Siteground Server is located in Frankfurt. Siteground was informed but they can not reproduce it and Pindom shows everything is fine.

The only issue I see that does not back up is that I read that service sites do not show this issues and using the CF network:

caseking.de

I guess they’re on a paid custom routing plans. To be honest this is over my experience and knowledge level.

? Yes, I know that this is happening because of the peering of Deutsche Telekom, in my previous reply I pointed out that this issue is not related to the Cloudflare APO in my opinion, since that’s what you said in the original post :slight_smile: I didn’t question that this is a CF - Telekom issue.

Other than upgrading your plans to any paid plan, you can only wait until Deutsche Telekom and Cloudflare makes an agreement with each other.

I guess they’re on a paid custom routing plans. To be honest this is over my experience and knowledge level.

Any paid plan fixes Deutsche Telekom’s weird routing since only websites on a Free plan are served from a US server (Newark, New Jersey)

1 Like

Ah sorry, didn’t get it then obviously. Well it is not an APO issue per se. But even with that small fee we are paid customers and do not get support. The support forms are blocked even for APO subs customers.Once a fee is paid, I see it as a paid plan regardless what companies try to play a terminus game.

Which paid plan do you mean and why should it solve those routing issues? To my understanding it is the transit routing which is the culprit. Is the premium package able to route it to pathways that are unaffected and this automatically or do you need to pay and tinker around even more?

You are not getting better routing since you are paying that 5 usd per month for the usage of the APO plugin. Customers on paid plans (Pro / Business / Enterprise) can get better routing since they are paying for a whole plan and a bunch of services and not just one extra service, but I understand your frustration.

Which paid plan do you mean and why should it solve those routing issues?

Any paid plan, even websites behind a Pro plan (The cheapest, it’s costs 20 USD per month if you pay annually or 25 usd per month normally) are served from the correct server for the users of Deutsche Telekom and Magyar Telekom. You don’t need to tinker with anything, after you upgrade your plan, Cloudflare’s system should move your website to another IP address (You don’t need to change anything).

Example:

You can also find confirmations about this on Deutsche Telekom’s forum where people are mentioning that the routing issue is only present for sites on a Free plan. This is because this whole situation is about money, Deutsche Telekom wants either money or more money (in case they already getting some) for (correctly) peering with Cloudflare. Better routes costs (more) money for Cloudflare.

1 Like

Thanks for clearing up. Yes, it is quite frustrating. In general, I am of the opinion that communication and public transportation should be made free of charge everywhere. I can understand that Telekom wants a piece of the pie but how the communicate that is outrages. The company is devilish and backward. Many say, somewhat cynically, that Germany is the third-world country of technologies, and Telekom has made a significant contribution to that perception. But then, I have a contract with them because the deal was good and I did regret that in the first week. Well, philosophizing about it doesn’t solve the problems.

Currently, I only see the option of canceling the Cloudflare service and choosing another CDN or reluctantly opting for this premium plan. APO has been perfectly sufficient until now. Seeing those old threads I’m not optimistic to see a resolution for that CF-Telek issue in a new future.

Just to catch upon that statement one get routed automatically to better pathway. I see the Argo feature is not included in the Pro plan which is advertising that it is doing just that. So even without Argo the Telekom issue is solved with “just” the Pro plan or do one need to add Argo? I see one can just add Argo to the APO / free plan. Wouldn’t this already solve the Telekom debacle?

We switched from the Free to the Pro plan because of exactly this problem and that was it. We didn’t book Argo.

I’m also not sure whether this might not be this “TCP Turbo” option that is NOT available in the Free plan but is automatically active in all other plans. We noticed around 24.01.2024 that all our pages were suddenly delivered via the US (Newark). Ping approx. 100-120ms. After a few days we tried to find out in more detail what was going on. This is described in detail here (unfortunately in German).

Hey, thanks for your response. Just exactly that shocking forum thread I discovered today. Even though I’m working on daily bases on the backend, customers using our page are more important. We did a lot of custom work to get our Google mobile page speed to a score of 90 and than it fails because of peering negotiations between companies on the shoulders of all internet users that pay a premium VDSL contract. Lets be true Telekom ain’t playing cheap with their contracts. The next time a Telekom representative knocks on the door for advertising a “future proof speed vdsl” I have some anecdote to tell. The one that was here got a white face after I told him what I’m paying for the Telekom contract compared what he was able to offer me …

//Edit: Ok, great I just have read your post! That sounds promising! thanks for posting. I was on page 1 and was still reading through that thread. I may subscribe today as I have no time to tinker around anymore… I will post my findings once subscribed.

//Edit: This is also quite remarkable. Cloudflare tests are awful. IPv4 is worse then IPv6 results…


No, only Argo wouldn’t solve your problem and you don’t need Argo AND the Pro plan to solve the routing issue, only the Pro plan. When a customer visits your site this is the path: Customer → CF node → Origin server. The argo service is for optimizing the route between the CF node and the Origin server, unfortunately it wouldn’t affect Deutsche Telekom’s awful routing at all.

1 Like

No, the issue gets resolved because customers on paid plans are getting better routes (I assume those costs more money to Cloudflare). After CF and Deutsche Telekom agrees regarding the money, the routing on the Free plan will be better. I assume Deutsche Telekom using this god awful routing (Since there is no reason to route EU traffic to another continent) so they can use it as a basis of negotiation or something.

OK, thanks for clearing up. This would have save some bucks as I see the need to upgrade only because of this present issue, not because I would like or need the Pro benefits. Interestingly that colleges that done some testing with DSL from 1&1, which is a reseller if my information are correct, are getting good metrics for our page.

@balogh.zsolti2002
Thanks for the info. That was basically my assumption as well. I only noticed this turbo option in passing and couldn’t really make heads or tails of the associated description of CF.

So I said I’m gonna update if we subscribe or have any positive news. After a day of watching the performance and logging it, it was like that

normal page load time ~850ms (eCommerce) APO enabled before the debacle
around 10am load time ~2.30sec APO enabled after Telekom debacle
around 2pm load time ~4.20sec APO enabled after Telekom debacle
around 5pm load time ~45sec - 3min - not loading at all APO enabled after Telekom debacle

Now within a minute after subscribing to the Pro Plan the load time is down to 1.3sec. The IP, as already stated here, did change. I did not change any Pro plan settings which come with additional performance benefits yet. Now, down to ~850ms again on prime time and like the performance we had before the debacle with Telekom started.

I can not say I’m happy, as we were forced to upgrade in an progressive way and without any notice about the issue with already damage taken by visitors that gave up on the site bc it wasn’t loading for them. From a business side perspective I can not see why Cloudflare would want to change this situation as they can grab the dollars keeping that situation like that. It is capitalism we all decided on, right?

We had no use to upgrade before the debacle, now they made sure we all have a reason. Pretty disgusting if you ask me. From both companies, to be clear.

At the moment the speed is good again, also on the free plan, perhaps they fixed something?
Can anybody else confirm this?

For me, the routing did not change and is still showing NYC hops.

It was yesterday with that awful performance I wrote down. I would be surprised if today would be any better. I can see an additional boost with the Pro plan and will be watching the performance closely this week. I see load time of 600ms with Pro compared to APO in free plan on the best days.
This issues is mostly present for Telekom DSL customers / user to remind everyone that reads that topic. The speed was alright with APO / Free for all other providers.

Doesn’t anyone in charge see this problem? People are victimized. I have stocks here.

Doesn’t anyone in charge see this problem? People are victimized. I have stocks here.

This is a business issue and not a technical one. The ones who are in charge for these things are aware of it since they are the one making the agreements with the ISPs. Until they find common grounds, your only choice is to pay for the 20-25 USD / month Pro plan if this causes you so much problems.

Thank you This solved my problem