Unable to change Crawler Hints

On one of my website I cannot change Crawler Hits.
It works perfectly in another of my website
Each time I click on I receive a error.

change_request_failed (Code: 10007)

Is it linked or blocked by another parameter ?

Why I want to change it?
It is a website deployed as a “Cloudflare Pages” with a “function”
I want to change it because I create a function for answering a sitemap.xml when crawler hits https://mywebsite/sitemap.xml
the sitemap.xml is correct, and mime type is text/xml… but Google says “unable to retrieve the sitemap” .
I use the same function in another Cloudflare Pages website and it works perfectly in Google.
The only difference I note is the “Crawler hits” parameter.
On the Cloudflare log, I got

AS396982 GOOGLE-CLOUD-PLATFORM : Block
each time I try to send my sitemap…

In the log, what is the reason that it was blocked? Expand the event to see

Thank you for your help:
The reason is “Rate Limiting”
Block
United States
2a06:98c0:3600::103
Rate limiting

This is the json event all are the same

{
  "action": "block",
  "clientASNDescription": "GOOGLE-CLOUD-PLATFORM",
  "clientAsn": "396982",
  "clientCountryName": "US",
  "clientIP": "2a06:98c0:3600::103",
  "clientRequestHTTPHost": "REMOVED",
  "clientRequestHTTPMethodName": "GET",
  "clientRequestHTTPProtocol": "HTTP/1.1",
  "clientRequestPath": "/",
  "clientRequestQuery": "",
  "datetime": "2022-02-12T21:14:02Z",
  "rayName": "6dc8d3029771106c",
  "ruleId": "worker",
  "rulesetId": "",
  "source": "ratelimit",
  "userAgent": "Mozilla/5.0 (Linux; Android 6.0.1; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.80 Mobile Safari/537.36 PTST/190509.230546 PTST/220129.163401",
  "matchIndex": 0,
  "metadata": [],
  "sampleInterval": 1
}

By the stated IP, it belongs to the Cloudflare network.

Why is your Worker getting rate limited … do you also se its getting challenged at the Firewall → Overview tab?

If so, you might want to “allow” this IP from Cloudflare Worker in Firewall → Tools → IP Access Rules?

For being complete this is the ./functions/sitemap.xml.js

const canonicalURL = "https://REMOVED";
const routes = [
    {
        path: "/index",
        name: "Index",
    },
    {
        path: "/tab_0",
        name: "Kite",
    },
    {
        path: "/tab_1",
        name: "Parapente",
    },
    {
        path: "/tab_2",
        name: "Cerf-Volant",
    },
    {
        path: "/tab_3",
        name: "Boomerang",
    },
    {
        path: "/tab_5",
        name: "Comité Directeur",
    },
];

let getRoutesXML = function (router,baseURL = canonicalURL) {
    //eslint-disable-line
    let list = "";
    for (let i = 0; i < router.length; i++) {
      if (router[i].path != undefined)
        list += `<url><loc>${baseURL}${router[i].path}</loc></url>\n`;
    }
    return `<?xml version="1.0" encoding="UTF-8"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
      ${list}
    </urlset>`;
  };

export async function onRequestGet(context) { //eslint-disable-line

    let stringXML = await getRoutesXML(routes,canonicalURL)
    return new Response(stringXML, {
        headers: {
          "content-type": "text/xml"
        }
      });


}

I’m having this same issue for my website. Can anyone help me with disabling Crawler Hints?

I am trying to enable crawler hints, every time I try to enable this setting I get the following error: change_request_failed (Code: 10007)

I am sorry to hear this, so all of you experience the same issue, @meesj, @user85512 and @eltorio ? :thinking:

May I ask if all of you are using Cloudflare Workers?
@azhao

From Cloudflare Status page → https://www.cloudflarestatus.com/, currently I do not se some incident related to this error. Might be a temporary issue? :thinking:

Can you confirm you can normally navigate through the Cloudflare dashboard? I wonder if this error you are seeing is somehow affecting, or not, the functionality of the Cloudflare dashboard.

Nevertheless, it’s still a beta feature.

May I ask has anyone written a ticket to Cloudflare Support about this so far? :thinking:

Kindly, I’d suggest you to write a ticket to Cloudflare support due to your account and/or domain issue and share the ticket number here with us so we could escalate this issue:

  • Login to Cloudflare and then contact Cloudflare Support by clicking on the Get More Help button. If you get automatic reply, reply and indicate to it you need more help and reference to this topic
  • Or send an an e-mail to support[at]cloudflare[dot]com from your e-mail associated with your Cloudflare account

@LucasCF

I’ve opened a ticket…

May I ask you to share a ticket number here so we could escalate it?

  • it should be located in the content or a title fo (automatic) reply from Cloudflare …

Thank you in advance.

I have not received yet the acknowledge but I’ll share it

1 Like

Hi Folks,

I would like to disable “Crawler Hints” for my website, but it fails with the error:
“change_request_failed (Code: 10007)”

I toggled a few other options to make sure there wasn’t an outage, but its only Crawler Hints that refuses to disable.

Could someone please help point me in the right direction?

Greetings,

Thank you for notice and writing. I am sorry to hear you are also experiencing this.

I have escalated this to Cloudflare support/engineering team as multiple users are experiencing this issue.

1 Like

I’m not using Cloudflare Workers.

I’ve been experiencing this issue for the last 1-2 weeks. I tried disabling Crawler Hints on different devices and on different internet connections, without success.

I can navigate normally though the Cloudflare dashboard. All other functionalities work fine.

I’m on the free plan so I can’t use email support.

1 Like

Hello everyone, Also I can’t activate this feature and it always gives me the error change_request_failed (Code: 10007) so I don’t know how to do it.

Can you tell me how and what to do? My website works perfectly.

Thank you

Greetings, thank you for writing. I am sorry to hear this.

Kindly, may I ask you to share a domain name and to send an e-mail to support[at]cloudflare[dot]com from your e-mail associated with your Cloudflare account.

When you got a reply (even automatic), please find a ticket number in the “title” or “content” of that e-mail and reply back here to share it with us so we could escalate it too. Thank you in advance!

As a free plan user, the answer of support[at]Cloudflare[dot]com is that there is no other support than community for free plans.

Just use and enable WARP (The VPN, not just 1.1.1.1), it will go through without a hitch.

Hi, Is it possible to enable “Crawler Hints” through the Cloudflare API? I am only able to find info about “Early Hints”: Cloudflare API v4 Documentation

Thanks.