Then it isn’t a Cloudflare problem.
It will only work like that if the actual application, e.g. the client, that you’re using to connect to Jellyfin with, is able to read and understand SRV records and how they work.
If that client doesn’t understand SRV records, what you’re trying to accomplish is never going to work the way you expect.
I Googl’ed a bit, and found that there are feature requests to add support for SRV records:
→ Support reading port from SRV DNS record · Issue #35 · jellyfin/jellyfin-meta · GitHub
Nothing that I see points in the direction that SRV records is, or have ever been supported with Jellyfin.
Since it doesn’t sound to me like Jellyfin supports SRV records, … at the time of writing this, another thing came up:
Could you eventually have forwarded port 80 (public IP) towards port 8096 (local machine), or something similar, so that the public port was actually different from the local/internal port because of your port forwarding set up?
Or maybe have run it at port 80 previously?
If SRV records aren’t supported, the problem needs to be found elsewhere, which doesn’t sound to be with your Cloudflare (and/or the DNS) configuration.
If you can share the domain, I can check from my end, if I see any kind of things that points towards reachability issues when using the domain, but I believe that’s all the Cloudflare Community can do now.
Other than that, I’m somehow starting to think that you should communicate with the Jellyfin Community about this.
They may be more specialized in what works with their software, and what does not, and can probably offer you much better troubleshooting than the Cloudflare Community.