Scarping qouta


Can I get an API key for scraping small amount of data from a website?
Before you crucify me hear me out.

  1. I do not scrape all of the website, not even 0.0001 percent.
  2. I do not scrape it often aka one every 2 hours.
  3. I most often load cashed data.
    I have docker instance running on my home server. Every couple of hours runs a price search on local classified’s website for a good price for some used parts and materials for home improvement and if it finds it sends me a notification. My problem I have been running into cloudflare checkbox… I can spend some time and figure out a way around it but… this seams to be a service to protect against large scraping operations.
    Can I get an API key validated by credit card that will give me a limited ability to scrape website? Give me a ‘scraping qouta’ and I am sure I will fit under it or is there some API access that would let me check changes in cloudflares’ cashing?

I am not loading the website much more than a keen user and I am not interested in participating in scrapping arms race to feed AI models. You could have an option to optimize traffic knowing that it not human and can wait.

That is not something Cloudflare will ever offer. You can negotiate with the site owner(s) if you want their data. Their challenges are configured by them.

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.