I’m using Cloudflare Streams Live Input with the experimental WebRTC WHIP/WHEP support. I’ve ported the WHIP/WHEP JS example into C# inside the Unity game engine and using Unity’s WebRTC stack.
Considering just the WHIP side, at the point where the WHIP endpoint is contacted to get the SDP Answer, the API returns an error code: Expecting SDP to send two media streams with 1 video and 1 audio with direction sendonly
The peer connection has a video track and an audio track added, and both are configured for send-only.
I’ve added the local SDP description below. As can be seen, it includes m-lines for both video and audio and both have send-only attributes.
Is there something missing here?
Local SDP:
v=0
o=- 6031414581557964271 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 96 97 98 102 103 104 9 0 8 99 100 101 107 108 109 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:STzL
a=ice-pwd:0+VRdmV8s3uVWQ4LKfCWv3uZ
a=ice-options:trickle
a=fingerprint:sha-256 94:12:DA:FB:B1:9C:A4:E1:E0:CE:FE:8D:9B:83:0D:6E:C7:E4:73:14:BB:6C:16:5B:39:2E:5F:64:41:AE:BA:B9
a=setup:actpass
a=mid:0
s:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:- cbfb6a92-e854-4454-b0e0-def665684fdb
a=rtcp-mux
a=rtpmap:96 opus/48000/2
a=rtcp-fb:96 transport-cc
a=fmtp:96 minptime=10;sprop-stereo=1;stereo=1;useinbandfec=1
a=rtpmap:97 multiopus/48000/6
a=fmtp:97 channel_mapping=0,4,1,2,3,5;coupled_streams=2;minptime=10;num_streams=4;useinbandfec=1
a=rtpmap:98 multiopus/48000/8
a=fmtp:98 channel_mapping=0,6,1,2,3,4,5,7;coupled_streams=3;minptime=10;num_streams=5;useinbandfec=1
a=rtpmap:102 ILBC/8000
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:99 L16/8000
a=rtpmap:100 L16/16000
a=rtpmap:101 L16/32000
a=rtpmap:107 L16/8000/2
a=rtpmap:108 L16/16000/2
a=rtpmap:109 L16/32000/2
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:3453899210 cname:7wYHnlg/EBD1ixEU
a=ssrc:3453899210 msid:- cbfb6a92-e854-4454-b0e0-def665684fdb
a=ssrc:3453899210 mslabel:-
a=ssrc:3453899210 label:cbfb6a92-e854-4454-b0e0-def665684fdb
m=video 9 UDP/TLS/RTP/SAVPF 127 118 125 117 124 116 123 115 122 114 35 36 121 111 120
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:STzL
a=ice-pwd:0+VRdmV8s3uVWQ4LKfCWv3uZ
a=ice-options:trickle
a=fingerprint:sha-256 94:12:DA:FB:B1:9C:A4:E1:E0:CE:FE:8D:9B:83:0D:6E:C7:E4:73:14:BB:6C:16:5B:39:2E:5F:64:41:AE:BA:B9
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:- cf5a464f-d7a2-4437-b460-bcac49cc42f7
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:127 VP8/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 implementation_name=Internal
a=rtpmap:118 rtx/90000
a=fmtp:118 apt=127
a=rtpmap:125 VP9/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 implementation_name=Internal;profile-id=0
a=rtpmap:117 rtx/90000
a=fmtp:117 apt=125
a=rtpmap:124 VP9/90000
a=rtcp-fb:124 goog-remb
a=rtcp-fb:124 transport-cc
a=rtcp-fb:124 ccm fir
a=rtcp-fb:124 nack
a=rtcp-fb:124 nack pli
a=fmtp:124 implementation_name=Internal;profile-id=2
a=rtpmap:116 rtx/90000
a=fmtp:116 apt=124
a=rtpmap:123 H264/90000
a=rtcp-fb:123 goog-remb
a=rtcp-fb:123 transport-cc
a=rtcp-fb:123 ccm fir
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=fmtp:123 implementation_name=VideoToolbox;level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640c1f
a=rtpmap:115 rtx/90000
a=fmtp:115 apt=123
a=rtpmap:122 H264/90000
a=rtcp-fb:122 goog-remb
a=rtcp-fb:122 transport-cc
a=rtcp-fb:122 ccm fir
a=rtcp-fb:122 nack
a=rtcp-fb:122 nack pli
a=fmtp:122 implementation_name=VideoToolbox;level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:114 rtx/90000
a=fmtp:114 apt=122
a=rtpmap:35 AV1X/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=fmtp:35 implementation_name=Internal
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:121 red/90000
a=rtpmap:111 rtx/90000
a=fmtp:111 apt=121
a=rtpmap:120 ulpfec/90000
a=ssrc-group:FID 232129652 2050057267
a=ssrc:232129652 cname:7wYHnlg/EBD1ixEU
a=ssrc:232129652 msid:- cf5a464f-d7a2-4437-b460-bcac49cc42f7
a=ssrc:232129652 mslabel:-
a=ssrc:232129652 label:cf5a464f-d7a2-4437-b460-bcac49cc42f7
a=ssrc:2050057267 cname:7wYHnlg/EBD1ixEU
a=ssrc:2050057267 msid:- cf5a464f-d7a2-4437-b460-bcac49cc42f7
a=ssrc:2050057267 mslabel:-
a=ssrc:2050057267 label:cf5a464f-d7a2-4437-b460-bcac49cc42f7
A followup: Since the software encoder in Unity sends either VP8, VP9 or AV1 video, could this be the issue? Browsers appear to also offer H264. What video codecs are supported by Cloudflare Stream live inputs w/ WebRTC?
So forcing the encoder to use H264 didn’t change anything. SDP:
v=0
o=- 3560735882354195639 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 96 97 98 102 103 104 9 0 8 99 100 101 107 108 109 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:WOUL
a=ice-pwd:0Ir4631qpt6dYgDG/jVJEY4N
a=ice-options:trickle
a=fingerprint:sha-256 DB:88:3B:64:4D:F0:A2:CE:E2:2F:40:39:A0:75:46:E8:DC:BB:04:0A:AF:90:F5:03:CA:7C:F0:B7:04:2A:25:BF
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:- 47f2c124-acdf-44bb-accb-b2cf91ae6fe3
a=rtcp-mux
a=rtpmap:96 opus/48000/2
a=rtcp-fb:96 transport-cc
a=fmtp:96 minptime=10;sprop-stereo=1;stereo=1;useinbandfec=1
a=rtpmap:97 multiopus/48000/6
a=fmtp:97 channel_mapping=0,4,1,2,3,5;coupled_streams=2;minptime=10;num_streams=4;useinbandfec=1
a=rtpmap:98 multiopus/48000/8
a=fmtp:98 channel_mapping=0,6,1,2,3,4,5,7;coupled_streams=3;minptime=10;num_streams=5;useinbandfec=1
a=rtpmap:102 ILBC/8000
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:99 L16/8000
a=rtpmap:100 L16/16000
a=rtpmap:101 L16/32000
a=rtpmap:107 L16/8000/2
a=rtpmap:108 L16/16000/2
a=rtpmap:109 L16/32000/2
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:3795311088 cname:cGaBGbtnN0Lm8jUV
a=ssrc:3795311088 msid:- 47f2c124-acdf-44bb-accb-b2cf91ae6fe3
a=ssrc:3795311088 mslabel:-
a=ssrc:3795311088 label:47f2c124-acdf-44bb-accb-b2cf91ae6fe3
m=video 9 UDP/TLS/RTP/SAVPF 102 104
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:WOUL
a=ice-pwd:0Ir4631qpt6dYgDG/jVJEY4N
a=ice-options:trickle
a=fingerprint:sha-256 DB:88:3B:64:4D:F0:A2:CE:E2:2F:40:39:A0:75:46:E8:DC:BB:04:0A:AF:90:F5:03:CA:7C:F0:B7:04:2A:25:BF
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:- 33eafe89-a0f2-487d-908a-2b0a4b9baa89
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 implementation_name=VideoToolbox;level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640c1f
a=rtpmap:104 H264/90000
a=rtcp-fb:104 goog-remb
a=rtcp-fb:104 transport-cc
a=rtcp-fb:104 ccm fir
a=rtcp-fb:104 nack
a=rtcp-fb:104 nack pli
a=fmtp:104 implementation_name=VideoToolbox;level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=ssrc:1119883745 cname:cGaBGbtnN0Lm8jUV
a=ssrc:1119883745 msid:- 33eafe89-a0f2-487d-908a-2b0a4b9baa89
a=ssrc:1119883745 mslabel:-
a=ssrc:1119883745 label:33eafe89-a0f2-487d-908a-2b0a4b9baa89
OK, I found the issue. Unity’s UnityWebRequest client was URL-encoding the POST data. For those who stumble upon this later, to prevent the URL encoding:
using (UnityWebRequest www = new UnityWebRequest(endpoint))
{
www.uploadHandler = new UploadHandlerRaw(Encoding.UTF8.GetBytes(sdp));
www.downloadHandler = new DownloadHandlerBuffer();
www.method = UnityWebRequest.kHttpVerbPOST;
www.SetRequestHeader("content-type", "application/sdp");
yield return www.SendWebRequest();
// etc.
}
Hi philip,
I also try to use Cloudflare Stream with Unity WebRTC by WHIP. Everything looks fine (exchange sdp and connect), but I only get “Stream has not started yet.” in my Cloudflare dashboard. Do you encounter this problem? Thanks in advance.
@lookoutking — when experimenting with the WebRTC beta, use the example code here for playback, rather than the player in the Stream dashboard:
2 Likes
Thanks for your help! It works.
cg12
July 27, 2023, 3:33pm
8
Hi there!
I would also like to try cloudflare webRTC streaming to stream directly from Unity. I got the Unity Renderstreaming to work but the stream failes on some devices, even when using different TURN servers.
That’s why I would like to try Cloudflare Stream for streaming. Basically I want to control a Unity app via a browser from remote desktop and mobile.
I have some questions:
Can you also control Unity by using the Cloudflare WebRTC approach?
How can I start when I want to implement this stuff on my own? Do I have to edit Unity’s Renderstream package? Do you mind sharing some code?
Is the signaling process still the same or is this done by the Cloudflare network?
Do you have any issues connecting to the stream from behind firewall or NATs?
I appreciate any help!
@lookoutking @philip8
system
July 31, 2023, 8:22pm
9
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.