The R2 API appear to only support HTTP/1.1, as tested with openssl s_client -alpn h2, curl and Firefox against the bucket URL.
Client-side applications accessing many small files directly using pre-signed URLs would benefit from HTTP/2 or HTTP/3. Is support for this planned?