Curl command getting 403 in my subdomain

I am using curl in my cron job. There are two cron job, the first one is for root domain and the second one is for sub-domain.

The First Cron job which is for root domain is working, but the second Cron job (for sub-domain).is not working. It’s returning 403 Forbidden. I have tried from Cpanel terminal.

[[email protected] ~]$ curl https://sub.domain.com/
<!DOCTYPE html>
<html style="height:100%">
<head>
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no" />
<title> 403 Forbidden
</title><script async src='/cdn-cgi/challenge-platform/h/b/scripts/invisible.js'></script></head>
<body style="color: #444; margin:0;font: normal 14px/20px Arial, Helvetica, sans-serif; height:100%; background-color: #fff;">
<div style="height:auto; min-height:100%; "> <div style="text-align: center; width:800px; margin-left: -400px; position:absolute; top: 30%; left:50%;">
<h1 style="margin:0; font-size:150px; line-height:150px; font-weight:bold;">403</h1>
<h2 style="margin-top:20px;font-size: 30px;">Forbidden
</h2>
<p>Access to this resource on the server is denied!</p>
</div></div><script type="text/javascript">(function(){window['__CF$cv$params']={r:'686597ff7dc14c73',m:'3aef224c08f4abfaec25ecbc0c11b8a81b9422cc-1630238161-1800-AS5DkXQi4MSfJpOKl77Z3aaxti1si3rR+dqYjc8bHmCpX7x58Txg5gNgGYGTdSokqMuWD3l2U6h6djJnJBU1NxL15ywVUHL8sw8Sy7kwJ4NhgGmw/Q0vtSgmH69u+hBEGQ==',s:[0xcdafa0022c,0xc197453dfe],u:'/cdn-cgi/challenge-platform/h/b'}})();</script><script defer src="https://static.cloudflareinsights.com/beacon.min.js" data-cf-beacon='{"rayId":"686597ff7dc14c73","version":"2021.8.1","r":1,"token":"ae55d96bddd34ef0b2933fd48fcd9d3e","si":10}'></script>
</body></html>
[[email protected] ~]$ curl -svo /dev/null https://sub.domain.com/
* out to connect() to sub.domain.com port 443 (#0)
  Trying ***.**.***.***...
Connected to sub.domain.com (***.**.***.***) port 443 (#0)
Initializing NSS with certpath: sql:/etc/pki/nssdb
  CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
SSL connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
Server certificate:
      subject: CN=sni.cloudflaressl.com,O="Cloudflare, Inc.",L=San Francisco,ST=California,C=US
      start date: Apr 02 00:00:00 2021 GMT
      expire date: Apr 01 23:59:59 2022 GMT
      common name: sni.cloudflaressl.com
      issuer: CN=Cloudflare Inc ECC CA-3,O="Cloudflare, Inc.",C=US
GET / HTTP/1.1
User-Agent: curl/7.29.0
Host: sub.domain.com
Accept: */*
HTTP/1.1 403 Forbidden
Date: Sun, 29 Aug 2021 12:02:00 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
cache-control: private, no-cache, no-store, must-revalidate, max-age=0
pragma: no-cache
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
CF-Cache-Status: DYNAMIC
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=7EgT741qrtuLPcQdyb%2FcoghPTNi4pl70BTnUZLyKs3zYYUwrHea1pO4P9dM2LYoAas1raino92AVZQj2peEFcy9Lmzn711rK9jFFTR60KWLx8cDAPUIyb26ABC01rXLjNWF6EGnFrw3UNcaV"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Strict-Transport-Security: max-age=2592000; includeSubDomains; preload
Server: cloudflare
CF-RAY: 6865a0bfe81700e8-AMS
alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400, h3=":443"; ma=86400
{ [data not shown]
* Connection #0 to host sub.domain.com left intact

What is the problem and how to solve this?

That 403 screen looks like it’s coming from your host. I’ve seen some hosts that block ‘curl’ from accessing. Try this just to be sure:

curl -skvo /dev/null https://sub.example.com --connect-to ::123.123.123.123

But change the IP address to the actual one for your server.

are you using super bot fight mode?

No, I am not using super bot fight mode.

https://sub.example.com was an example.

curl https://en.bismillahhomeocare.com/
<!DOCTYPE html>
<html style="height:100%">
<head>
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no" />
<title> 403 Forbidden
</title><script async src='/cdn-cgi/bm/cv/669835187/api.js'></script></head>
<body style="color: #444; margin:0;font: normal 14px/20px Arial, Helvetica, sans-serif; height:100%; background-color: #fff;">
<div style="height:auto; min-height:100%; "> <div style="text-align: center; width:800px; margin-left: -400px; position:absolute; top: 30%; left:50%;">
<h1 style="margin:0; font-size:150px; line-height:150px; font-weight:bold;">403</h1>
<h2 style="margin-top:20px;font-size: 30px;">Forbidden
</h2>
<p>Access to this resource on the server is denied!</p>
</div></div><script type="text/javascript">(function(){window['__CF$cv$params']={r:'687017c63cabee50',m:'8c5f24d726cc45510020a1b4f341ebff707004aa-1630348253-1800-Ab0eVM8CCnHQgC7Yf1gnMujykSD4FN8j6AcagaTJHxXDbj3CCTWrqKV6+eiUuZPRg+VA1mogYHgqNy6RXqk/tfqVFPKdvt1NTNAnuKsNK3J+GyqSXT3WiLeehTzqp6Sxpg==',s:[0x0424c1f580,0x4a51fce419],}})();</script><script defer src="https://static.cloudflareinsights.com/beacon.min.js" data-cf-beacon='{"rayId":"687017c63cabee50","version":"2021.8.1","r":1,"token":"ae55d96bddd34ef0b2933fd48fcd9d3e","si":10}'></script>
</body></html>
curl -svo /dev/null https://en.bismillahhomeocare.com/
* About to connect() to en.bismillahhomeocare.com port 443 (#0)
*   Trying 104.21.54.141...
* Connected to en.bismillahhomeocare.com (104.21.54.141) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=sni.cloudflaressl.com,O="Cloudflare, Inc.",L=San Francisco,ST=California,C=US
*       start date: Apr 02 00:00:00 2021 GMT
*       expire date: Apr 01 23:59:59 2022 GMT
*       common name: sni.cloudflaressl.com
*       issuer: CN=Cloudflare Inc ECC CA-3,O="Cloudflare, Inc.",C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: en.bismillahhomeocare.com
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< Date: Mon, 30 Aug 2021 18:30:37 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< cache-control: private, no-cache, no-store, must-revalidate, max-age=0
< pragma: no-cache
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< CF-Cache-Status: DYNAMIC
< Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=LvJvojcOTTqIa%2F1N0w43lzXML%2Fz3iSL1SpjitDuTJHKCiFJuz1XBicwohn4CoIJgmUludn8dPGb2oCVOeRr2AbMD%2B5Q%2BICERrF8BSy2Z7t%2B1nJmMZGS1tUqAV6cSwHsdlothHCbi2J6fq5wj"}],"group":"cf-nel","max_age":604800}
< NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< Strict-Transport-Security: max-age=2592000; includeSubDomains; preload
< Server: cloudflare
< CF-RAY: 68701764eb963316-CDG
< alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400, h3=":443"; ma=86400
<
{ [data not shown]
* Connection #0 to host en.bismillahhomeocare.com left intact

The First Cron job which is for root domain is working, but the second Cron job (for sub-domain).is not working. It’s returning 403 Forbidden. I have tried from Cpanel terminal, but same 403 error.

@sdayman any update?

en.bismillahhomeocare.com

Just a moment… JS Challenge (tested from Germany and Croatia).
cf-browser-verification cf-im-under-attack
DDoS protection by Cloudflare.
Ray ID: 687aa4965cccbf05

Maybe you blocked curl as an user-agent string in Firewall Rules too?

This: curl -skvo /dev/null https://en.bismillahhomeocare.com --connect-to ::62.210.142.141 gives me:

* Expire in 0 ms for 6 (transfer 0x562834c96fb0)
* Connecting to hostname: 62.210.142.141
*   Trying 62.210.142.141...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x562834c96fb0)
* Connected to 62.210.142.141 (62.210.142.141) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [15 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4049 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=en.bismillahhomeocare.com
*  start date: Aug 22 09:56:13 2021 GMT
*  expire date: Nov 20 09:56:12 2021 GMT
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x562834c96fb0)
} [5 bytes data]
> GET / HTTP/2
> Host: en.bismillahhomeocare.com
> User-Agent: curl/7.64.0
> Accept: */*
>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
} [5 bytes data]
< HTTP/2 200
< expires: Thu, 19 Nov 1981 08:52:00 GMT
< cache-control: max-age=2592000
< pragma: no-cache
< set-cookie: PHPSESSID=965c56af92309c25fc99ad2693031778; path=/; secure
< set-cookie: yasguid=0ce0755cfb43d128067f5710f692891966527796612ed507167ff7.53589714; expires=Sat, 27-Aug-2022 01:19:03 GMT; Max-Age=31104000; path=/; secure
< set-cookie: ya_store_theme_mode=normal; expires=Fri, 01-Oct-2021 01:19:03 GMT; Max-Age=2592000; path=/; secure
< content-type: text/html; charset=UTF-8
< date: Wed, 01 Sep 2021 01:19:03 GMT
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< vary: User-Agent,User-Agent
< alt-svc: quic=":443"; ma=2592000; v="43,46", h3-Q043=":443"; ma=2592000, h3-Q046=":443"; ma=2592000, h3-Q050=":443"; ma=2592000, h3-25=":443"; ma=2592000, h3-27=":443"; ma=2592000
<
{ [804 bytes data]
* Connection #0 to host 62.210.142.141 left intact

1 Like

Thank you for your time.

My problem is 403 Error.

I am using curl in cron job in my domain and sub-domain. In root domain curl is executing perfectly. but in sub-domain it’s returning 403.

My

But Same curl is working in my root domain!!.

After your last edit

curl -skvo /dev/null https://en.bismillahhomeocare.com
* About to connect() to en.bismillahhomeocare.com port 443 (#0)
*   Trying 104.21.54.141...
* Connected to en.bismillahhomeocare.com (104.21.54.141) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=sni.cloudflaressl.com,O="Cloudflare, Inc.",L=San Francisco,ST=California,C=US
*       start date: Apr 02 00:00:00 2021 GMT
*       expire date: Apr 01 23:59:59 2022 GMT
*       common name: sni.cloudflaressl.com
*       issuer: CN=Cloudflare Inc ECC CA-3,O="Cloudflare, Inc.",C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: en.bismillahhomeocare.com
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< Date: Wed, 01 Sep 2021 01:26:13 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< cache-control: private, no-cache, no-store, must-revalidate, max-age=0
< pragma: no-cache
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< CF-Cache-Status: DYNAMIC
< Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=PyFZO1qpPRO8vW4M%2BaGenIxfmqQyk%2BepV7OrsIHW4o7swRQ6KSd8BAscVOeE%2BQBfu6OSdGNqRLZYCPgO7ZTyf2SBILp%2BYicsLSZy5g6sJPw01QSnneBIi6XyksGU6CKADA2o4aLA4XxqDB42"}],"group":"cf-nel","max_age":604800}
< NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< Strict-Transport-Security: max-age=2592000; includeSubDomains; preload
< Server: cloudflare
< CF-RAY: 687ab58c6da33bda-CDG
< alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400, h3=":443"; ma=86400
<
{ [data not shown]
* Connection #0 to host en.bismillahhomeocare.com left intact

There’s definitely something interesting going on. We can ‘curl’ to the origin, but if it’s routed through Cloudflare, it gets a 403.

I don’t think we’ve asked, but have you checked the Firewall Events Activity Log at dash.cloudflare.com? If you Add Filter for that subdomain Host(name), it should make the list easier to scan.

1 Like

Yes

Latest Block (Js Challenge) -->. Maybe this one

But I didn’t find any log that proves that my curl has been blocked.

Other logs are normal. There is one Firewall Rules cf.client.bot → allow. I think this is not the problem, because I have toggled it and zero result

@MoreHelp

Ticket ID 2246184

Have you tried disabling Bot Fight Mode?

Yes! It’s working now, but my question is why it’s working in my root domain with Bot Fight Mode and why not in sub-domain? I want to use Bot Fight Mode.

Oops! Now it’s working after toggling Bot Fight Mode. Thank you everybody and specially @sdayman.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.