ClueCon 2019 WebRTC 2 VoIP Flowroute DIDs now support web originated calls
Julien Chavanton Lead software engineer - voice routing - flowroute.com presentation agenda : WebRTC 2 VoIP and share some insights about the design and features.
➔ How we are using Kamailio to do our web edge-proxies, providing SIP over WebSockets, Topology hiding and SDP normalization. (With sipwise RTP-engine)
➔ JS libraries, to help our customers integrate WebRTC for inbound audio calls on Flowroute DIDs using JSSIP, we will look at our provided Javascript libraries jssip_client and jssip_ui
➔ How we are monitoring QoS stats for audio sessions and audio streams with G.711 and Opus.
➔ The current state of our platform and how we were able to reuse our multiple point of presences and APIs to enable new possibilities for our customers. Addition to carrier infrastructure using Kamailio sip-proxy Kamailio SIP over websocket rfc7118 https://kamailio.org/docs/modules/stable/modules/websocket.html
5.1. ws_handle_handshake()
This function checks an HTTP GET request for the required headers and values, and (if successful) upgrades the connection from HTTP to WebSocket.
Below is an example of a WebSocket handshake in which the client requests the WebSocket SIP subprotocol support from the server:
GET / HTTP/1.1 Host: sip-ws.example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Origin: http://www.example.com Sec-WebSocket-Protocol: sip Sec-WebSocket-Version: 13
The handshake response from the server accepting the WebSocket SIP subprotocol would look as follows:
HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo= Sec-WebSocket-Protocol: sip
Once the negotiation has been completed, the WebSocket connection is established and can be used for the transport of SIP requests and responses. Topology hiding using Kamailio topos module
received
1 INVITE sip:[email protected]:5060 SIP/2.0 2 Record-Route:
proxied
16 INVITE sip:[email protected] SIP/2.0 17 Via: SIP/2.0/UDP 33.133.45.43:5060;branch=z9hG4bKd81f.ca02b8a41f4bcb1fcaddd44c35a0a17c.0 18 To:
Configuration : loadmodule "ndb_redis" loadmodule "topos" loadmodule "topos_redis"
modparam("topos", "storage", "redis") modparam("topos_redis", "serverid", "srv8")
➔ The topos state is stored redis, this is providing restart persistency. ➔ The module is registering core event callbacks to alter/restore the message after/before the routing script execution.
sr_event_register_cb(SREV_NET_DATA_IN, tps_msg_received); sr_event_register_cb(SREV_NET_DATA_OUT, tps_msg_sent); SDP Normalization example
v=0 o=mozilla...THIS_IS_SDPARTA-66.0.2 4010920269622071999 0 IN IP4 0.0.0.0 s=- t=0 0 a=msid-semantic:WMS * m=audio 14314 RTP/AVP 109 0 8 101 c=IN IP4 34.239.191.73 a=msid:{dad8d9f1-8be0-417e-a009-b56637915df2} {f4d0b092-c609-4546-a677-39b6d875136a} a=ssrc:807290859 cname:{420a5941-f46a-4695-a85d-b88a475bfe8d} Received : a=mid:0 after RTP-engine cleanup a=rtpmap:109 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:101 telephone-event/8000 a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1 a=fmtp:101 0-15 a=sendrecv a=rtcp:14315 a=end-of-candidates
v=0 o=- 2808372830592226719 0 IN IP4 34.239.191.73 s=- Proxied : c=IN IP4 43.139.191.73 After SDP rewriting t=0 0 m=audio 14188 RTP/AVP 109 0 8 101 a=rtpmap:109 opus/48000/2 a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1 a=rtpmap:101 telephone-event/8000 a=fmtp:101 0-15 SDP rewriting using Kamailio routing script
0 // get the request body 1 $avp(tempbody) = $rb; 2 3 // retrieve various values from SDP (example to retrieve Opus FTMP line) 4 ... 5 $var(match) = "a=fmtp:" + $avp(opus-payload-id); 6 $avp(opus-fmtp-line) = $(avp(fmtp-lines){line.sw,$var(match)}); 7 xinfo("opus-fmtp-line[$avp(opus-fmtp-line)]\n"); 8 9 // prepare a template 10 $var(v-line) = "v=0\r\n"; 11 $var(o-line) = "o=- "+$avp(sess-id)+" "+$avp(sess-version)+" IN IP4 12 +$avp(unicast-address)+"\r\n"; 13 $var(s-line) = "s=-\r\n"; 14 $var(c-line) = "c=IN IP4 "+$avp(unicast-address)+"\r\n"; 15 $var(t-line) = "t=0 0\r\n"; 16 $var(a-line1) = ""; 17 $var(a-linex) = ""; 18 20 // set the request body 21 set_body("$var(v-line)$var(o-line)$var(s-line) ...", "application/sdp"); Opus for speech
2. OVERVIEW OF OPUS
Opus operates in one of three modes:
• SILK mode (speech signals up to wideband), • CELT mode (music and high-bitrate speech), or • Hybrid mode (SILK and CELT simultaneously for super-wideband and fullband speech).
"Configuration changes that use CELT on both sides (or between wideband SILK and Hybrid mode) use the overlap of the transform window to avoid dis-continuities. However, switching between CELT and SILK or Hybrid mode is more complicated, because SILK operates in the time domain without a win-dow."
The Opus Codec, Xiph.Org Foundation Freeswitch Opus tuning using fmtp parameters configuration https://tools.ietf.org/html/rfc7587
/usr/local/freeswitch/conf/autoload_configs/opus.conf.xml
This is a good way tell Opus to use Silk and avoid using hybrid Silk/Celt mode with a normal voice call (not music), Silk only mode should be favored. And Opus will have more room to play with the target bitrate.
SDP Answer
11 v=0 12 o=FreeSWITCH 1565144490 1565144491 IN IP4 128.27.70.174 13 s=FreeSWITCH 14 c=IN IP4 128.27.70.174 15 t=0 0 16 m=audio 32724 RTP/AVP 109 101 17 a=rtpmap:109 opus/48000/2 18 a=fmtp:109 useinbandfec=1; maxplaybackrate=16000; stereo=1 19 a=rtpmap:101 telephone-event/8000 20 a=fmtp:101 0-16 21 a=ptime:20 22 a=sendrecv Opus tuning and ftmp parameters
// Tells the encoder about the highest sample rate the decoder is expected to // use when decoding the bitstream. The encoder would typically use this // information to adjust the quality of the encoding. The default // implementation does nothing. virtual void SetMaxPlaybackRate(int frequency_hz);
int16_t WebRtcOpus_SetMaxPlaybackRate(OpusEncInst* inst, int32_t frequency_hz) { ...
if (frequency_hz <= 8000) { set_bandwidth = OPUS_BANDWIDTH_NARROWBAND; } else if (frequency_hz <= 12000) { set_bandwidth = OPUS_BANDWIDTH_MEDIUMBAND; } else if (frequency_hz <= 16000) { set_bandwidth = OPUS_BANDWIDTH_WIDEBAND; } else if (frequency_hz <= 24000) { set_bandwidth = OPUS_BANDWIDTH_SUPERWIDEBAND; } else { set_bandwidth = OPUS_BANDWIDTH_FULLBAND; } return ENCODER_CTL(inst, OPUS_SET_MAX_BANDWIDTH(set_bandwidth)); } Our SIP over websocket client application
https://github.com/flowroute/jssip_client https://github.com/flowroute/jssip_ui
External dependencies : https://github.com/versatica/JsSIP https://github.com/webrtc/adapter
Thanks to Marcel, David and the Evolux team ! Demo application calling Freeswitch echo application.
https://demo.webrtc.flowroute.com/fr.html https://demo.webrtc.flowroute.com/ui.html Example 1: inlined Javascript integration of Example 2: Javascript integration of jssip_client + jssip_client jssip_ui QoS reports over SIP MESSAGE
MESSAGE sip:13125867146@sip.flowroute.com SIP/2.0 P-QoS-Call-ID:27vosgqamo35an134b31
"qos_data": { "rx_media": { “jitterBufferDelay": 156595.2, "jitterBufferEmittedCount": 1172160, "audioLevel": 0.40052491836298715, "totalAudioEnergy": 4.705616978591951, "totalSamplesReceived": 1200960, "totalSamplesDuration": 25.02111, }, “tx": { "bytesSent": 29795, "packetsSent": 497, ... QoS reports received stream example
rx_media description example
jitterBufferDelay It is the sum of the time, in seconds, each frame 156595 / takes from the time it is received and to the time it 1172160 exits the jitter buffer. This increases upon frames Average exiting, having completed their time in the buffer Jitter buffer (incrementing jitterBufferEmittedCount). The average jitter buffer delay can be calculated by size : dividing the jitterBufferDelay with the 130ms jitterBufferEmittedCount.
audioLevel The value is between 0..1 (linear), where 1.0 0.4 represents 0 dBov, 0 represents silence, and 0.5 represents approximately 6 dBSPL change in the sound pressure level from 0 dBov.
concealedSamples A concealed sample is a sample that is based on data 20159 / that was synthesized to conceal packet loss and 48000 does not represent incoming data. (sum) 410ms
totalSamplesReceived The total number of samples that have been 2892960 received by this receiver. This includes concealedSamples.
totalSamplesDuration Total duration in seconds of all samples received 60.26s
Statistics related to Audio Network Adaptation, not yet exposed ? https://www.w3.org/TR/webrtc-stats/ Our points of presences supporting WebRTC
us-east-nj us-east-va us-west-or us-west-sjc eu-west-ldn eu-west-ams JSSIP Websockets failover
Websocket do not support DNS NAPTR / SRV records, however JSSIP can manage failover over multiple sockets. export const FR_POINTS_OF_PRESENCE_DOMAINS = { 'us-east-va': [ 'wr-us-east-va-01.webrtc.flowroute.com', 'wr-us-east-nj-01.webrtc.flowroute.com', ], } const urls = FR_POINTS_OF_PRESENCE_DOMAINS[this.params.pointOfPresence]; const sockets = [ { socket: new WebSocketInterface(`wss://${urls[0]}:4443`), weight: 20, }, { socket: new WebSocketInterface(`wss://${urls[1]}:4443`), weight: 10, }, ]; Shortest media path using Flowroute preferred PoP / edge strategies
Depending where you VoIP endpoints are you can select to relay the media from any of our point of presence.
We will use an “edge strategy” to favor this point of presence and failover using the closest one if needed.
Our concept of edge strategy is used in inbound routes, our SRV records and is preset in our example JSSIP client.
Therefore, you can configure both the inbound route and the JSSIP client to be as close as possible to your VoIP endpoint. A few words about Freeswitch and Opus
Presenting this topic at Cluecon, was a great opportunity since Freeswitch is very well suited to handle Opus transcoding, in fact I did use it for a very similar use case handling millions of calls originating from Android devices few years ago.
As time passed, the situation as surely improved further the phones are being replaced with more powerful ones ARM v7 / v8 with support hardware optimizations NEON / assembly. Audio encoding and other audio processing requirements, should now be widely supported my the vast majority of mobile phones.
WebRTC and hardware manufacturers did solve many problems.
Looking forward helping our partners to achieve QoS best efforts and work with Freeswitch and Freeswitch users. BETA OPT-IN If you want to use our platform with you existing DIDs http://optin.webrtc.flowroute.com/
Contact me you have any feedbacks on this presentation ? [email protected]
Additional resources :
Flowroute https://www.flowroute.com/ RTP-Engine https://github.com/sipwise/rtpengine Kamailio https://github.com/kamailio/kamailio Building a WebRTC gateway Hacking with WebRTC native API https://www.youtube.com/watch?v=WpT7BWtNrcU WebRTC delay agnostic echo canceler ported to a mediastreamer2 filter https://github.com/jchavanton/ms-aec-webrtc