Cloudflare Access Policy problem

I have created an access policy for our internal app, but since we are not using it anymore, I deleted it from Cloudflare Access policy. The problem is the app is still showing on Access App Launcher portal when users logged in. Anybody encountered this? And how to fix?