As long as you haven’t loaded your page there before, otherwise it will certainly be cached as well. You can also try a different browser profile or a different machine.
As mentioned, there is no redirect at the moment, only the JavaScript challenge, but that’s not a redirect.
301s are generally not a good idea if you only want a temporary redirect.
No worries, but now your browser(s) have that locally stored and clearing the cache typically fixes that. Cloudflare itself does not send the redirect any more.
Even a permanent redirect will not redirect when disabled. The difference between 301 and 302 really is mostly the caching part. But yeah, if it is disabled it won’t redirect and it currently does not redirect either.