THen I see this article The Ideal WP Rocket Settings For 2021 (got me 1s load times), it said Cloudflare needs me to change the name servers to those in Cloudflare. It seems that Cloudflare will be webhosting for my website? Currently our website is hosted in a dedicated server. If using Cloudflare, does that means we need to replace the dedicated server with Cloudflare?
They are referring to the WP Rocket user interface, and telling you not to use the CDN option in WP Rocket if you use Cloudflare. Some CDNs can offload your static assets by serving them through their CDN with a different hostname. With those CDNs your website lives at www.example.com, but you configure WP Rocket to serve your static assets from your CDN with a hostname like images.example.com.akamaized.net. Cloudflare does not work like that, everything stays on www.example.com, and Cloudflare fetches everything from your dedicated server when it needs it.