The Woker can't fetch the IPFS content?

I built a simple Cloudflare woker service via the manual: developers. Cloudflare. com/workers , but I found it seems that the Cloudflare had prevented workers from fetching the ipfs gateway (Cloudflare-ipfs. com/ cf-ipfs. com).
It fetch the non-IPFS URLs works well, but get 429 when fetch the IPFS.
What step am I doing wrong?
Here is my Worker’s code:

export default {
    async fetch(
        request: Request,
        env: Env,
        ctx: ExecutionContext
    ): Promise<Response> {
        // return new Response("Hello World!");  // OK
        // return fetch(`https://google.com`);  // OK
        // return fetch(`https://k51qzi5uqu5dgqjy1i78mz3oumplzt0cye32w9m8ix8hg9chpz5trvj8luwv0c.ipns.cf-ipfs.com`); // failed
        // return fetch(`https://cf-ipfs.com/ipns/k51qzi5uqu5dgqjy1i78mz3oumplzt0cye32w9m8ix8hg9chpz5trvj8luwv0c`); // failed
        // return fetch(`https://cloudflare-ipfs.com/ipns/k51qzi5uqu5dgqjy1i78mz3oumplzt0cye32w9m8ix8hg9chpz5trvj8luwv0c`); // failed
        return fetch(`https://cloudflare-ipfs.com/ipfs/bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze`); // failed
    },
};

Thank you for making this post.

HTTP 429 is the status code for rate limiting. The problem here was that worker subrequests such as the fetch above share the same IP. This along with other characteristics of workers subrequests caused all worker subrequests to be grouped into the same rate limiting group.

I’ve added the cf-worker header as another factor considered in our rate limiting. Customers using workers should no longer be rate limited in the same group with workers from other zones

tl;dr it’s fixed now :slight_smile:

3 Likes

Has this fix been rolled out yet? I am seeing this on a worker with only my test requests (no other traffic). I have some content going through the IPFS DNSLink gateway working fine, but other content that is not possible to setup DNSLink for.

My worker is fetching from IPFS, extracting a CID then fetching again with it. I have ruled this out as the problem by calling an endpoint that does the second fetch only and passing the CID that would have come from the first request.

Do I need to pass the cf headers from the original request? I was under the impression that these headers were handled by Cloudflare.

I have tested my worker with ipfs.io as the gateway and it works fine. It is only when I use the Cloudflare gateway I have an issue.

@mathew5 ?

Hi,

Sorry I did not have notifications set up for this thread. It has been rolled out. Are you receiving a 429 response? Could you message me your zone name or ID so I can look into it?

I’ve just tested from workers sandbox and it appears to work both for cache hits and misses.

Thanks for the response @mathew5. I don’t seem to have the right trust to send a DM, but the zone is staxxinvaders.com.

I have added a workaround that does a client side redirect to cloudflare-ipfs.com/ipfs/<cid> that gets around the issue, but it does change the browser address. The last time I checked, I was getting 429 (from memory) and it would work intermittently.

I have setup a staging worker that has the flag set to fetch the image rather than redirecting. In my initial testing it looks like the images are being fetched correctly at the moment.

It looks like I just hadn’t hit the request limit yet:


  (log) Fetching image URL: QmRRFYfaWmZMpgyS6bvS18TeKqtv6WDjyt4aSzHJjwoHBi
  (log) Sending request to https://cloudflare-ipfs.com/ipfs/QmRRFYfaWmZMpgyS6bvS18TeKqtv6WDjyt4aSzHJjwoHBi
  (error) Image request failed: {}
  (error)   status: 429, reason: Too Many Requests
  (error)   headers: false, reason: undefined

Going directly to the link in the logs the image is found.