Project: Python Video Conferencing Client - pyVCC

Adrian Beutner1 and Christoph Kuhr2

Summer Semester 2013

[email protected] [email protected] Abstract

In this document, the concept and design of the Python Video Conferencing Client (pyVCC) and it’s development process is described. PyVCC utilizes the Session Initiation and Descrip- tion Protocols (SIP, SDP) to establish a session between two clients and a call server (CS). The two pyVCC clients share their audio and video (AV) data with the Realtime Transport Protocol (RTP). Python Video Conferencing Client Content Beutner - Kuhr

Contents

1 Introduction 2 1.1 Workpackages and Schedueling ...... 2 1.2 Project Description ...... 3

2 Concept and Design 3 2.1 Communication Concept ...... 3 2.1.1 SIP and SDP - Kuhr ...... 3 2.1.2 RTSP and SDP- Kuhr ...... 5 2.1.3 RTP - Beutner ...... 10 2.2 Callserver and MCU - Beutner ...... 10 2.3 Software Design ...... 11 2.3.1 libVLC ...... 11 2.3.2 SIPSimple Client SDK ...... 12 2.3.3 UML ...... 13 2.4 Proof of Concept ...... 15 2.4.1 Virtualbox - Firewall - Beutner ...... 15 2.4.2 Callserver and MCU - Beutner ...... 15 2.4.3 libVLC RTSP Streaming ...... 15 2.4.4 SIP Simple SDK - Blink ...... 19

3 Discussion 21 3.1 Callserver and MCU - Beutner ...... 21 3.1.1 OpenVCS and Asterisk ...... 21 3.1.2 Medooze ...... 21 3.2 pyVLCC ...... 22 3.2.1 libVLC ...... 22 3.2.2 SIP Simple SDK ...... 22 3.2.3 RTSP SDP vs SIP SDP ...... 22 3.2.4 Python ...... 22 3.2.5 Performance ...... 23

pyVCC i Python Video Conferencing Client Content Beutner - Kuhr

List of Figures

2.1 SIP Trapezoid [5, p. 4] ...... 3 2.2 SDP Body Scheme [4, p. 30] ...... 4 2.3 TEMP SIP Body Fields [4, p. 30] ...... 4 2.4 TEMP Field Values [4, p. 31] ...... 5 2.5 RTP Wireshark Capture ...... 11 2.6 Use Case Diagram ...... 13 2.7 Activity Diagram pyVCC ...... 14 2.8 Component Diagram pyVCC ...... 14 2.9 OpenVCS Communication ...... 15 2.10 pyVLCC GUI ...... 20 2.11 Blink GUI ...... 20 3.12 SIP Signaling Wireshark Capture ...... 23 3.13 RTSP Signaling Wireshark Capture ...... 23 3.14 SIP SDP Wireshark Capture ...... 23 3.15 RTSP SDP Wireshark Capture ...... 24

List of Tables

3.1 This table shows some data ...... 24

pyVCC 1 Python Video Conferencing Client Introduction Beutner - Kuhr

1 Introduction

1.1 Workpackages and Schedueling A list of single worksteps to perform:

MS2 04/18/2013 Concept and Design:

• Understand RTP (Beutner) • Understand SIP, RTSP, SDP, libVLC , SIPSimple Client SDK (Kuhr) • Develope communication concept (Beutner) • Design software (Kuhr)

MS3 06/27/2013 Implementation and Verification:

• Install call server(Beutner) • Implement software (Kuhr) • Install media control unit (Beutner) • Test software (Beutner, Kuhr)

MS4 07/11/2013 Conclusions and Presentation:

• Conclusions (Beutner, Kuhr) • Presentation (Beutner, Kuhr)

pyVCC 2 Python Video Conferencing Client Introduction Beutner - Kuhr

1.2 Project Description PyVCC is a video conferencing client written with the Python programming language. The application script pyVCC.py is executed within the Python interpreter CLI. Thus, the inter- action and debugging iniformation is printed to the Python CLI. The required functionality for SIP sessions management shall be implemented with the SIPSimple Client SDK [?]. The SIPSimple Client SDK provides an interface for external codecs, called IMediaStream. The RTP AV data transport is implemented with the libVLC library and is called via the IMedi- aStream API by the SIP components. libVLC capture the local AV data, streams it to the remote client and plays back the AV data stream received by the remote client. With the Session Initiation and Description Protocols (SIP, SDP), two pyVCC Clients es- tablish a session, managed by a Call Server (CS). The Call Server shall be realized with the OpenSource Application openVCS. OpenVCS provides SIP functionaity along with a MCU (Multipoint Control Unit). The MCU manages RTP AV streams encoded with the h264 codec for video and the G711 codec for audio.

2 Concept and Design

2.1 Communication Concept 2.1.1 SIP and SDP - Kuhr SIP describes two types of communication, the most common way with the so called SIP trapezoid and communication which is forwarded to a Multipoint Control Unit (MCU). In the most commonly used case, Voice over IP, the user clients send and receive their SIP messages to and from their next SIP Proxy. The SIP Proxy forwards the SIP messages to the appropriate remote SIP Proxy to reach the remote URI. After the session is initialized with the SDP parameters, the audio data ist transported, via RTP, to the remote user client based on a peer-to-peer connection on different ports.

Figure 2.1: SIP Trapezoid [5, p. 4]

The second case is used for video conferencing environments, where different RTP AVC Profile streams, the amount depends on the users registered to the processed session, are

pyVCC 3 Python Video Conferencing Client Introduction Beutner - Kuhr mixed to one RTP AVC Profile stream. The mixed stream is then return to each conference participant. This concept is described in detail in chapter 2.2 Callserver and MCU.

The SDP part of a SIP INVITE message containes different lines of parameters to nego- tiate a RTP AVC Stream. Figure 2.2 shows the rule, by which each line of the SDP body is build. In figure 2.3, an example for a SDP body is given.

Figure 2.2: SDP Body Scheme [4, p. 30]

Figure 2.3: TEMP SIP Body Fields [4, p. 30]

pyVCC 4 Python Video Conferencing Client Introduction Beutner - Kuhr

The fields shown in figure 2.3 have the following meaning:

Figure 2.4: TEMP Field Values [4, p. 31]

2.1.2 RTSP and SDP- Kuhr A RTSP stream uses HTTP like signaling to negotiate session parameters as well as SIP. But the signaling with RTSP is different. The mechanism to start a session uses a different data transport for signaling and media transmission, like with SIP signaling. For RTSP port 8554 is used over TCP and the ports 9000 and 9001 are used for RTP audio and video transport over UDP.

Mandatory to start a presentation stream, which means a combination of for example audio and video streams, is a presentation description file. This presentation description file is provided as a SDP file. It has to be communicated, over a seperate channel like email or a webpage, before the actual start of the session.

Once the servers presentation file is known, the requesting user client sends an OPTION request to the server. It contains information about the requested presentation stream by the preshared SDP file and the used user client.

The server responses with a 200 OK message. In this message the server gives information about the server application that is providing the presentation stream. Also a field named PUBLIC is contained in the response. Th field PUBLIC is a list of the operations, which are implemented by the server.

The most important operations, taken from the RTSP RFC2326 [7, p. 11f], are listed below:

pyVCC 5 Python Video Conferencing Client Introduction Beutner - Kuhr

SETUP: Causes the server to allocate resources for a stream and start an RTSP session.

PLAY and RECORD: Starts data transmission on a stream allocated via SETUP.

PAUSE: Temporarily halts a stream without freeing server resources.

TEARDOWN: Frees resources associated with the stream. The RTSP session ceases to exist on the server.

RTSP methods that contribute to state use the Session header field (Section 12.37) to identify the RTSP session whose state is being manipulated. The server generates session identifiers in response to SETUP requests (Section 10.4).

The most important RTSP messages and their responses are listed below:

DESCRIBE: The DESCRIBE method retrieves the description of a presentation or media object identified by the request URL from a server. It may use the Accept header to specify the description formats that the client understands. The server responds with a description of the requested resource. The DESCRIBE reply-response pair constitutes the media initialization phase of RTSP. Example: C->S: DESCRIBE rtsp://server.example.com/fizzle/foo RTSP/1.0 CSeq: 312 Accept: application/sdp, application/rtsl, application/mheg

RESPONSE with SDP S->C: RTSP/1.0 200 OK CSeq: 312 Date: 23 Jan 1997 15:35:06 GMT Content-Type: application/sdp Content-Length: 376 v=0 o=mhandley 2890844526 2890842807 IN IP4 126.16.64.4 s=SDP Seminar i=A Seminar on the session description protocol u=http://www.cs.ucl.ac.uk/staff/M.Handley/sdp.03.ps [email protected] (Mark Handley)

pyVCC 6 Python Video Conferencing Client Introduction Beutner - Kuhr

c=IN IP4 224.2.17.12/127 t=2873397496 2873404696 a=recvonly m=audio 3456 RTP/AVP 0 m=video 2232 RTP/AVP 31 m=whiteboard 32416 UDP WB a=orient:portrait

The DESCRIBE response MUST contain all media initialization information for the resource(s) that it describes. If a media client obtains a presentation description from a source other than DESCRIBE and that description contains a complete set of media initialization parameters, the client SHOULD use those parameters and not then request a description for the same media via RTSP. SETUP: The SETUP request for a URI specifies the transport mechanism to be used for the streamed media. A client can issue a SETUP request for a stream that is already playing to change transport parameters, which a server MAY allow. If it does not allow this, it MUST respond with error ”455 Method Not Valid In This State”. For the benefit of any intervening firewalls, a client must indicate the transport parameters even if it has no influence over these parameters, for example, where the server advertises a fixed multicast address. Since SETUP includes all transport initialization information, firewalls and other inter- mediate network devices (which need this information) are spared the more arduous task of parsing the DESCRIBE response, which has been reserved for media initialization. The Transport header specifies the transport parameters acceptable to the client for data transmission; the response will contain the transport parameters selected by the server. C->S: SETUP rtsp://example.com/foo/bar/baz.rm RTSP/1.0 CSeq: 302 Transport: RTP/AVP;unicast;clientport = 4588 − 4589

S->C: RTSP/1.0 200 OK CSeq: 302 Date: 23 Jan 1997 15:35:06 GMT Session: 47112344 Transport: RTP/AVP;unicast; clientport = 4588 − 4589; serverport = 6256 − 6257

The server generates session identifiers in response to SETUP requests. If a SETUP request to a server includes a session identifier, the server MUST bundle this setup request into the existing session or return error ”459 Aggregate Operation Not Allowed” (see Section 11.3.10). PLAY: The PLAY method tells the server to start sending data via the mechanism specified

pyVCC 7 Python Video Conferencing Client Introduction Beutner - Kuhr

in SETUP. A client MUST NOT issue a PLAY request until any outstanding SETUP requests have been acknowledged as successful. The PLAY request positions the normal play time to the beginning of the range specified and delivers stream data until the end of the range is reached. PLAY requests may be pipelined (queued); a server MUST queue PLAY requests to be executed in order. That is, a PLAY request arriving while a previous PLAY request is still active is delayed until the first has been completed. This allows precise editing. For example, regardless of how closely spaced the two PLAY requests in the example below arrive, the server will first play seconds 10 through 15, then, immediately following, seconds 20 to 25, and finally seconds 30 through the end. C->S: PLAY rtsp://audio.example.com/audio RTSP/1.0 CSeq: 835 Session: 12345678 Range: npt=10-15

C->S: PLAY rtsp://audio.example.com/audio RTSP/1.0 CSeq: 836 Session: 12345678 Range: npt=20-25

C->S: PLAY rtsp://audio.example.com/audio RTSP/1.0 CSeq: 837 Session: 12345678 Range: npt=30-

See the description of the PAUSE request for further examples. A PLAY request without a Range header is legal. It starts playing a stream from the beginning unless the stream has been paused. If a stream has been paused via PAUSE, stream delivery resumes at the pause point. If a stream is playing, such a PLAY request causes no further action and can be used by the client to test server liveness. The Range header may also contain a time parameter. This parameter specifies a time in UTC at which the playback should start. If the message is received after the specified time, playback is started immediately. The time parameter may be used to aid in synchronization of streams obtained from different sources. For a on-demand stream, the server replies with the actual range that will be played back. This may differ from the requested range if alignment of the requested range to valid frame boundaries is required for the media source. If no range is specified in the request, the current position is returned in the reply. The unit of the range in the reply is the same as that in the request. After playing the desired range, the presentation is automatically paused, as if a PAUSE request had been issued. The following example plays the whole presentation starting at SMPTE time code 0:10:20 until the end of the clip. The playback is to start at 15:36 on 23 Jan 1997. C->S: PLAY rtsp://audio.example.com/twister.en RTSP/1.0 CSeq: 833 Session: 12345678 Range: smpte=0:10:20-;time=19970123T153600Z

pyVCC 8 Python Video Conferencing Client Introduction Beutner - Kuhr

S->C: RTSP/1.0 200 OK CSeq: 833 Date: 23 Jan 1997 15:35:06 GMT Range: smpte=0:10:22-;time=19970123T153600Z For playing back a recording of a live presentation, it may be desirable to use clock units:

C->S: PLAY rtsp://audio.example.com/meeting.en RTSP/1.0 CSeq: 835 Session: 12345678 Range: clock=19961108T142300Z-19961108T143520Z

S->C: RTSP/1.0 200 OK CSeq: 835 Date: 23 Jan 1997 15:35:06 GMT

A media server only supporting playback MUST support the npt format and MAY sup- port the clock and smpte formats.

PAUSE: The PAUSE request causes the stream delivery to be interrupted (halted) temporarily. If the request URL names a stream, only playback and recording of that stream is halted. For example, for audio, this is equivalent to muting. If the request URL names a presentation or group of streams, delivery of all currently active streams within the presentation or group is halted. After resuming playback or recording, synchronization of the tracks MUST be maintained. Any server resources are kept, though servers MAY close the session and free resources after being paused for the duration specified with the timeout parameter of the Session header in the SETUP message. Example: C->S: PAUSE rtsp://example.com/fizzle/foo RTSP/1.0 CSeq: 834 Session: 12345678

S->C: RTSP/1.0 200 OK CSeq: 834 Date: 23 Jan 1997 15:35:06 GMT

The PAUSE request may contain a Range header specifying when the stream or pre- sentation is to be halted. We refer to this point as the ”pause point”. The header must contain exactly one value rather than a time range. The normal play time for the stream is set to the pause point. The pause request becomes effective the first time the server is encountering the time point specified in any of the currently pending PLAY requests. If the Range header specifies a time outside any currently pending PLAY requests, the error ”457 Invalid Range” is returned. If a media unit (such as an audio or video frame) starts presentation at exactly the pause point, it is not played or recorded. If the Range header is missing, stream delivery is interrupted immediately on receipt of the message

pyVCC 9 Python Video Conferencing Client Introduction Beutner - Kuhr

and the pause point is set to the current normal play time. A PAUSE request dis- cards all queued PLAY requests. However, the pause point in the media stream MUST be maintained. A subsequent PLAY request without Range header resumes from the pause point. For example, if the server has play requests for ranges 10 to 15 and 20 to 29 pending and then receives a pause request for NPT 21, it would start playing the second range and stop at NPT 21. If the pause request is for NPT 12 and the server is playing at NPT 13 serving the first play request, the server stops immediately. If the pause request is for NPT 16, the server stops after completing the first play request and discards the second play request. As another example, if a server has received requests to play ranges 10 to 15 and then 13 to 20 (that is, overlapping ranges), the PAUSE request for NPT=14 would take effect while the server plays the first range, with the second PLAY request effectively being ignored, assuming the PAUSE request arrives before the server has started playing the second, overlapping range. Regardless of when the PAUSE request arrives, it sets the NPT to 14. If the server has already sent data beyond the time specified in the Range header, a PLAY would still resume at that point in time, as it is assumed that the client has discarded data after that point. This ensures continuous pause/play cycling without gaps.

TEARDOWN: The TEARDOWN request stops the stream delivery for the given URI, freeing the resources associated with it. If the URI is the presentation URI for this presentation, any RTSP session identifier associated with the session is no longer valid. Unless all transport parameters are defined by the session description, a SETUP request has to be issued before the session can be played again. Example: C->S: TEARDOWN rtsp://example.com/fizzle/foo RTSP/1.0 CSeq: 892 Session: 12345678

S->C: RTSP/1.0 200 OK CSeq: 892

2.1.3 RTP - Beutner The Real-Time Transport Protocol is a widely used Protocol to carry media data. It works UDP-based. Poorly described, RTP completes UDP-datagrams with sequence-numbers, times- tamps and unique synchronization source identifiers to allow end to end transmission: By checking the sequence numbers, RTP notices if there was packet loss. It can synchronize Audio- and Video-data by using the timestamps as well. In addition, there is information about the Payload Type, for example which codecs are used. RTP is used in conjuncion with RTCP:

2.2 Callserver and MCU - Beutner To allow videoconferencing you need user agents, a call server and a multipoint control unit.

pyVCC 10 Python Video Conferencing Client Introduction Beutner - Kuhr

img/rtp_cap.png

Figure 2.5: RTP Wireshark Capture

2.3 Software Design 2.3.1 libVLC The libVLC library fully covers the functionality of the VLC player. It provides access to the complete internal VLC architecture. This way an application for capturing and displaying AV data, can rely on the processing of libVLC. libVLC is a C library with a complete Python API to it.

Installation: sudo apt-get libvlc libvlccore5

pyVCC 11 Python Video Conferencing Client Introduction Beutner - Kuhr

2.3.2 SIPSimple Client SDK The SIPSimple Client SDK is a SIP stack completely written in Python. It is an open source project which is funded by the European Union. SIPSimple completey handles a SIP sessions. Simple point to point session are possible, as well as conferences with multiple participants. Since AV data is handled by a seperate interface, it is possible to add custom AV data trans- port functionality to it.

Installation:

Add the following repository: deb http://ag-projects.com/ubuntu precise main deb-src http://ag-projects.com/ubuntu precise main sudo apt-get update sudo apt-get install python-sipsimple libgcrypt11

pyVCC 12 Python Video Conferencing Client Introduction Beutner - Kuhr

2.3.3 UML

Figure 2.6: Use Case Diagram

Use Cases:

UC10: Call User User 1 calls User 2 by entering the URI of the remote User. PyVCC uses the SIP signaling prtocol to establish a session between user agent 1 and user agent 2.

UC20: Incoming Call from User User 2 receives a call by User 1 and acknowledges it, so that a SIP session is established.

UC30: RTP AV Data Transport If a SIP session is established, the AV data shall be transported between the two user agents with the RTP using the codecs h264 for video and G711 for audio data.

UC40: Configuration, Maintance

pyVCC 13 Python Video Conferencing Client Introduction Beutner - Kuhr

Figure 2.7: Activity Diagram pyVCC

Figure 2.8: Component Diagram pyVCC

pyVCC 14 Python Video Conferencing Client Introduction Beutner - Kuhr

2.4 Proof of Concept 2.4.1 Virtualbox - Firewall - Beutner betriebssytem virtualbox (guest, switch XP/Linux, Asterisk Now) firewall (ports) router (portforwarding, problematik fh (8080,22))

2.4.2 Callserver and MCU - Beutner OpenVCS OpenVCS (Open Source Video Conferencing Server) is a SIP based Multipoint Control Unit that runs on Windows Systems. The App can manage various MCUs. Each MCU (it’s called Bridge here) can manage up to 64 participants. You can manage the Bridges via an xml config-file. For every Bridge, you have to set up the following:

id: The number that identifies the bridge realm: The domain name protocol: The Network Protocol that is used for the Bridge ipver: Version of the IP Protocol port: Portnumber that identifies the SIP-stack

After the App has started, these values are loaded from the config-file and the Bridge is added. Now, you can set up a connection from a SIP-Client to the Bridge by calling (SIP INVITE) sip:[email protected]. Then, the OpenVCS App establishes a connection between the SIP clients. Any SIP-client software that supports the Codecs G711 for Audio and H264 for Video can be used.

2 Medooze

Figure 2.9: OpenVCS Communication

2.4.3 libVLC RTSP Streaming This RTSP signaling is processed by libVLC itself, within the RTP streaming module. To make use of the RTSP signaling, a deep packet inspection has to be done to intercept those DESCRIBE messages, in order to take the appropriate measures.

For the purpose of deep packet inspection, the Python API to the C library libpcap is used.

pyVCC 15 Python Video Conferencing Client Introduction Beutner - Kuhr

import pcap

A simpler processing of the incoming Ethernet frames is done with the Python module Im- packet. from impacket.ImpactDecoder import EthDecoder from impacket.ImpactPacket importIP,TCP

PCAP deep packet inspection thread. class PCAP_Worker(QtCore.QThread): def __init__(self, net_dev, parent = None): QtCore.QThread.__init__(self, parent) self.instance = parent self.decoder = EthDecoder() self.p = pcap.pcapObject()

Open the Ethernet or WLAN interface. self.p.open_live(string.join(net_dev,’’), 1600, 0, 100)

Filter TCP messages. self.p.setfilter(’tcp’,0,0) self.p.setnonblock(1) self._running=False #______

Get the IP address of network interface in use. def get_interface_ip(self, ifname): s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) return socket.inet_ntoa(fcntl.ioctl(s.fileno(), 0x8915,struct.pack(’ 256s’, ifname[:15]))[20:24]) #______

def get_lan_ip(self): ip = socket.gethostbyname(socket.gethostname()) if ip.startswith("127."): interfaces = ["eth0","eth1","eth2","wlan0","wlan1"] for ifname in interfaces: try: ip = self.get_interface_ip(ifname) break; except IOError: pass return ip #______

def stop(self): self._running=False #______

def run(self): self._running=True

pyVCC 16 Python Video Conferencing Client Introduction Beutner - Kuhr

while(self._running==True): self.p.dispatch(1, self.callback) #______

Inspect packets for DESCRIBE message and save the remote IP address. def callback(self, jdr, data, something): l2 = self.decoder.decode(data) l3 = l2.child() if (l3.get_ip_src()!=self.get_lan_ip()): if isinstance(l3, IP): l4 = l3.child() if isinstance(l4, TCP): if l4.get_th_dport() == 8554: user_data = l4.get_data_as_string() data_lines = user_data.split(’\n’) data_fields = data_lines[0].split(’ ’) if(data_fields[0] == "DESCRIBE" and not self.instance. mediaplayerRemote.is_playing()): stream_address = data_fields[1].split(’/’) protocol = stream_address[0].split(’:’) socket = stream_address[2].split(’:’) session = stream_address[3] ipaddress = socket[0] port = socket[1] recon_stream_address = "%s://%s:%s/%s" %(protocol [0],l3.get_ip_src(),port,session) self.instance.stream_address = recon_stream_address self.instance.receive_remote() #______

And the libVLC implementation: import vlc.py class pyVLCC(QtGui.QMainWindow):

Local client configuration. def __init__(self): QtGui.QMainWindow.__init__(self) self.setWindowTitle("pyVLCC") self.instance1 = vlc.Instance() self.instance2 = vlc.Instance() self.instance3 = vlc.Instance() self.mediaplayerLocal = self.instance1.media_player_new() self.mediaplayerLocalStreamer = self.instance2.media_player_new() self.mediaplayerRemote = self.instance3.media_player_new() self.media_local = None self.media_loopback = None self.media_remote = None self.video_device = "v4l2:///dev/video0" self.audio_device = "alsa://hw:0,0" self.network_device = "lo"#"eth0"

pyVCC 17 Python Video Conferencing Client Introduction Beutner - Kuhr

self.remote_address = "localhost"#"139.6.19.157" self.stream_address = "rtsp://%s:8554/session.sdp" %(self. remote_address) self.ports = dict({"audio":9000, "video":9001, "rtsp":8554}) self.protocols={socket.IPPROTO_TCP:’tcp’, socket.IPPROTO_UDP:’udp’, socket.IPPROTO_ICMP:’icmp’} self.createLocalSectionUI() self.config() self.transmit_local() self.start_prev()

self.pcap_rtsp_thread = PCAP_Worker(self.network_device, self) self.pcap_rtsp_thread.start() #______

AVC Profile 3.0. def config(self): self.audio_codec="alaw" self.audio_sample_rate="48000" self.audio_channels="1" self.video_codec="h264" self.video_bit_rate="512" #______

Start or end a call.

def call(self): self.receive_remote() #______

def hang_up(self): self.stop() self.pcap_rtsp_thread.start() self.start_prev() #______

Start the webcam preview in the top left video frame. def start_prev(self): self.media_loopback = self.instance1.media_new(unicode("rtsp:// localhost:%s/session.sdp" %self.ports["rtsp"])) self.mediaplayerLocal.set_media(self.media_loopback) self.media_loopback.parse() if not self.mediaplayerLocal.is_playing(): if self.mediaplayerLocal.play()>=0: self.mediaplayerLocal.play() #______

Provide the local recorded media on the specified port. def transmit_local(self): self.media_local = self.instance2.media_new(unicode(self.video_device)) self.media_local.add_option(":input-slave=%s" % self.audio_device)

pyVCC 18 Python Video Conferencing Client Introduction Beutner - Kuhr

self.media_local.add_option(":live-caching=100") self.media_local.add_option(":sout=#transcode{vcodec=%s,{vcodec=%s, venc=x264{profile=baseline,level=1.3,nocabac,nobframes,ref=1},vb=% s,scale=0,acodec=%s,samplerate=%s,channels=%s}:rtp{sdp=rtsp://:%s/ session.sdp}" % % (self.video_codec,self.video_bit_rate,self.audio_codec,self. audio_sample_rate,self.audio_channels,self.ports["rtsp"])) self.media_local.add_option(":sout-keep") self.mediaplayerLocalStreamer.set_media(self.media_local) self.media_local.parse()

if not self.mediaplayerLocalStreamer.is_playing(): if self.mediaplayerLocalStreamer.play() >= 0: self.mediaplayerLocalStreamer.play() #______

Receive remote media, described by the incoming RTSP session and play it back in the main video frame. def receive_remote(self): self.media_remote = self.instance3.media_new(unicode(self. stream_address)) self.mediaplayerRemote.set_media(self.media_remote) self.media_remote.parse()

if not self.mediaplayerRemote.is_playing(): if self.mediaplayerRemote.play() >= 0: self.mediaplayerRemote.play() #______

def stop(self): self.mediaplayerLocalStreamer.stop() self.media_local.release() self.mediaplayerRemote.stop() self.media_remote.release() #______

Start the main loop. if __name__ == "__main__": app = QtGui.QApplication(sys.argv) vlcc = pyVLCC() vlcc.show() vlcc.resize(1024, 600) sys.exit(app.exec_())

Qt GUI:

2.4.4 SIP Simple SDK - Blink Blink is a SIP user agent that is written with the SIP Simple SDK. It has the functionalities for VoIP, messaging, chat and file transfer.

pyVCC 19 Python Video Conferencing Client Concept and Design Beutner - Kuhr

Figure 2.10: pyVLCC GUI

Figure 2.11: Blink GUI

A direct connection between blink and the well known Open Source Software Linphone, could be established.

pyVCC 20 Python Video Conferencing Client Concept and Design Beutner - Kuhr

3 Discussion

3.1 Callserver and MCU - Beutner 3.1.1 OpenVCS and Asterisk broken Video mixer Implementation

3.1.2 Medooze most recent open source version from 2003

broken dependencies due to library updates

pyVCC 21 Python Video Conferencing Client Concept and Design Beutner - Kuhr

3.2 pyVLCC 3.2.1 libVLC The Python API of libVLC is direct and automatically generated from the libVLC C API. Thus the Python API provices access to all functionality, that is available from the C API. An integration within the IMediaStream interface of the SIP Simple SDK would be possible.

3.2.2 SIP Simple SDK Due to complexity, the SIP Simple implementation failed, although SIP Simple SDK has a well documented and structured API. The comlexity of intercoorperation of the different components is not as simple, as the name suggests. An error, that could be identified, was the registration of user accounts on a SIP registrar. Since there has been no SIP Registrar to register at, SIP Simple returned a timeout. SIP Simple continues trying to register any user, that has been using the API. Even when the program and the Python interpreter get terminated and restarted. The error seems to be a fault usage, but could not be solved in the given time. Another, but not yet traceable error, was the usage of the IMediaStream interface. The mech- anisms that are used to control the RTP streams, that are implemented in libVLC, are not very well documented and lack of simple examples. Only the implementation of the native SIP Simple SDK RTP audio interface was available as an example. But very complex and without any comments. This made it very hard to work around the problems that occured. No SIP Invite message could be generated, due to these both errors.

3.2.3 RTSP SDP vs SIP SDP A SIP INVITE message contains a SDP body like it is explained in figure 2.2 and 2.4. In contrast the SDP body of a RTSP REPLY message is shown in figure 3.15.

The following figure shows a session initiation capture with SIP, between th SIP Simple SDK software blink and linphone-3.0. In the SDP body, show in figure 3.14, of the SIP INVITE message contains the following parameters:

3.2.4 Python In chapter 2.4.3 Proof of Concept - libVLC RTSP Streaming the deep packet inspection is described, which implements a simple way of signaling the RTSP streaming to both user clients within 80 lines of code. The code that is required to stream two different RTSP streams with libVLC, is written in about 100 lines of code. Since libVLC is a C library, the performance issues are not caused by the Python interpreter. The graphical user interface, written in PyQt, takes another 100 lines of code.

pyVCC 22 Python Video Conferencing Client Concept and Design Beutner - Kuhr

img/sip_signaling_capture.png

Figure 3.12: SIP Signaling Wireshark Capture

Figure 3.13: RTSP Signaling Wireshark Capture

img/sip_sdp_body.png

Figure 3.14: SIP SDP Wireshark Capture

Taken these aspect into account, makes the Python programming language a simple and fast way to write user client applications, even with a graphical interface.

3.2.5 Performance The user client without hardware acceleration is only stable for about 30 seconds, while transcoding the local video stream in AVC Baseline Profile 1.3. The following standard output of the x264 library, which is used for transcoding by libVLC,

pyVCC 23 Python Video Conferencing Client Discussion Beutner - Kuhr

Figure 3.15: RTSP SDP Wireshark Capture indicates in line three, that no hardware acceleration is used. x264 [warning]: frame MB size (40x30) > level limit (396) x264 [warning]: MB rate (30000) > level limit (11880) x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2 AVX x264 [info]: profile Constrained Baseline, level 1.3 x264 [info]: final ratefactor: 29,40 x264 [info]: using SAR=1/1

Because the CPU is used for transcoding, it has heavy load by the Python interpreter. The following excerpt from the standard output of the command top shows this:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 30968 root 20 0 1606m 131m 46m S 175 3.4 8:44.65 python

Table 3.1: This table shows some data

But even if the VLC player GUI the CPU usage rises to 70 percent.

pyVCC 24 Python Video Conferencing Client References Beutner - Kuhr

References

[1] Hubert Zimmermann, OSI Reference Model -The ISO Model of Architecture for Open Systems Interconnection, April 1980.

[2] Prof. Dr. Dipl. Ing. Andreas Grebe, NGN Introduction and Basics, July 2013.

[3] Prof. Dr. Dipl. Ing. Andreas Grebe, TCP/IP Protocols and Multimedia Services, July 2013.

[4] Prof. Dr. Dipl. Ing. Andreas Grebe, Service Signaling in NGN, July 2013.

[5] Prof. Dr. Dipl. Ing. Andreas Grebe, SIP Routing, Network Address Translation and Gateway Technologies in NGN, July 2013.

[6] Prof. Dr. Dipl. Ing. Andreas Grebe, Migration Scenarios in NGN and Corporate Net- works, July 2013.

[7] IETF, http://tools.ietf.org/html/rfc2326, Real Time Streaming Protocol, April 1998.

pyVCC 25