THen I see this article The Ideal WP Rocket Settings to Pass Core Web Vitals in 2023, it said Cloudflare needs me to change the name servers to those in Cloudflare. It seems that Cloudflare will be webhosting for my website? Currently our website is hosted in a dedicated server. If using Cloudflare, does that means we need to replace the dedicated server with Cloudflare?
Cloudflare is a reverse proxy. It acts as a firewall in front of your site and also caches static resources like a CDN would. You still need your hosting.
They are referring to the WP Rocket user interface, and telling you not to use the CDN option in WP Rocket if you use Cloudflare. Some CDNs can offload your static assets by serving them through their CDN with a different hostname. With those CDNs your website lives at www.example.com, but you configure WP Rocket to serve your static assets from your CDN with a hostname like images.example.com.akamaized.net. Cloudflare does not work like that, everything stays on www.example.com, and Cloudflare fetches everything from your dedicated server when it needs it.