Bot Fight mode Vs Observatory?

Your conclusion is correct as long as you are talking about Bot Fight Mode (free) and not Super Bot Fight Mode (Pro or higher). Super Bot Fight Mode can be skipped with Custom Rules, a feature which was reported to eventually be making its way to free users: Super Bot Fight Mode is now configurable! (in the last paragraph).

It was stated a while ago that they were trying to make the new Speed Test a known bot excluded from Bot Fight mode as well: Bug: Cloudflare's speed test blocked by "Bot fight"-Mode - #2 by sdayman, but nothing yet as far as I know.

At the moment, your only option is to disable Bot Fight Mode. For what it’s worth, even Cloudflare’s docs don’t recommend enabling it when not under attack: FAQ · Cloudflare bot solutions docs

  • BFM and SBFM are high security features intended to quickly help customers under active attack stop as many bots as possible. Due to the high security threshold, false positives do sometimes happen.