By CARLTON A. THOMPSON a DISSERTATION

A DESIGN AND PERFORMANCE STUDY OF A DISTRIBUTED IP-BASED TELECOMMUNICATION SYSTEM (D-IPTS)

By CARLTON A. THOMPSON

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2016 © 2016 Carlton A. Thompson

2 To my mother Hyacinth Thompson and to the memory of my father Carlton Thompson, for always supporting me during my studies and work.

3 ACKNOWLEDGMENTS The path to PhD has been very challenging and I have achieved a milestone in my career. I learned a lot about the ﬁeld of IP telecommunications, peformance analysis, and associated qualitative research methods. This dissertation would not have been written without the help of certain individuals. I would like to extend my gratitude towards my advisor Dr. Latchman and co-advisor Dr. McNair. They helped me with the selection of my topic and provided guidance during the writing of my dissertation. Their encouragement and insights have always been inspiring. In addition, none of this could have been possible without my family and loved ones providing their continuous support during my various course studies. Also, I would like to thank my friends and colleagues from the Electrical and Computer Engineering Department at the University of Florida. Finally, I would like to thank Texas Instruments ™ for providing ﬁnancial support for this work.

4 TABLE OF CONTENTS page ACKNOWLEDGMENTS...... 4 LIST OF TABLES...... 9 LIST OF FIGURES...... 10

LIST OF ABBREVIATIONS ...... 14 ABSTRACT...... 17

CHAPTER

1 INTRODUCTION ...... 19

Motivation...... 20 Voice Networks...... 21 Traditional Telecommunications Networks...... 21 VoIP Networks ...... 23 VoIP for telephone long distance cost reduction...... 23 VoIP for in-ofﬁce communication ...... 25 Aims and Objectives...... 26 Dissertation Structure ...... 27

2 BACKGROUND ...... 29

Overview of Existing and Emerging IP Telecommunications Systems ...... 29 IP Telecommunications Protocols...... 31 SCCP Signaling Protocol ...... 32 IAX Signaling Protocol ...... 32 SIP Signaling Protocol...... 32 SDP Signaling Protocol ...... 32 Data Transfer Protocols ...... 33 Real-time Transport Protocol ...... 33 Resource Reservation Protocol ...... 33 Real Time Streaming Protocol ...... 33 Voice Codecs...... 33 G.729 ...... 34 G.722 ...... 35 G.711 ...... 35 IP Telecommunications Availability...... 36 Availability...... 36 Reliability Block Diagram ...... 38 Performance Measures for IP Telecommunications...... 40

5 Quality of Service...... 40 Network QoS parameters...... 40 Measuring VoIP QoE...... 41 Queuing Theory ...... 41 Queuing techniques...... 43 M/G/1 model ...... 45 Summary...... 46 Literature Review...... 46 VoIP Overview ...... 46 System Design...... 47 Performance ...... 48 System Modeling and Reliability ...... 49 Proprietary Telephone Systems...... 55 Cisco ...... 55 ShoreTel ...... 56 Lync...... 57 8x8, Inc...... 58 Vonage ...... 60 Next Generation Systems...... 61 Skype for Business...... 61 OnSIP...... 64 2600hz ...... 66 Summary...... 68

3 DETAILED STUDY OF SESSION INITIATION PROTOCOL AND REAL TIME PROTOCOL ...... 70

Session Initiation Protocol...... 70 SDP Signaling Protocol ...... 73 Real-Time Protocol...... 74 SIP and RTP Security...... 75

4 DISTRIBUTED IP TELECOMMUNICATION SYSTEM (D-IPTS) ...... 76

Components of D-IPTS Testbed ...... 77 Alpine Linux...... 78 Kamailio...... 79 FreeSWITCH...... 80 Other Components...... 83 Web interface...... 83 DNS server...... 84 DHCP server...... 85 Provisioning server...... 86 D-IPTS Testbed...... 87 NATs Solution and Mobility ...... 90

6 RTPproxy and NAT traversal ...... 90 Mobility ...... 91 D-IPTS Testbed Implementation ...... 92 Call Flow ...... 92 Call Processing...... 93 Preliminary Tests...... 95 High Availability D-IPTS...... 96 D-IPTS database redundancy...... 97 D-IPTS DNS failover...... 98 System Conﬁgurations ...... 98 Multiple server ...... 98 LXC...... 98 Summary...... 100

5 TESTBED PERFORMANCE MODEL DEVELOPMENT AND CAPACITY STUDY 101

Testbed Design...... 102 SIPp Conﬁguration...... 104 Kamailio Software Customization ...... 105 Experiment 1: Call Scenarios Testing ...... 106 Experiment 2: Kamailio Router Test ...... 113 Concurrent calls ...... 113 Calls per Second Test ...... 116 Experiment 3: FreeSWITCH Stateful Call Processing ...... 116 Concurrent Calls Test ...... 117 Calls per Second Test ...... 117 D-IPTS Performance Comparison With Next Generation Systems ...... 118 Summary...... 119

6 QUEUING MODEL DEVELOPMENT AND VALIDATION ...... 121

Queuing Model...... 121 Queuing Experiments ...... 123 Utilization Comparison/Validation...... 126 Calls per Second Comparison/Validation...... 126 Waiting Time Comparison/Validation...... 127 D-IPTS Design Decisions...... 128 Call Rate and Utilization...... 129 Number of Servers...... 130 Summary...... 130

7 CONCLUSION AND FUTURE WORK ...... 132

7 APPENDIX

A SUMMARY OF RESEARCH CONTRIBUTIONS ...... 135

B KAMAILIO INSTALLATION ...... 137

C FREESWITCH INSTALLATION ...... 139

D SETTING UP MASTER-SLAVE REPLICATION ...... 140

E SETUP LXC HOST ...... 144

F CONNECTING TO A WIRELESS ACCESS POINT ...... 148

G BUILD OR COMPILE LINPHONE FROM SOURCE FOR IOS ...... 150 REFERENCES...... 152

BIOGRAPHICAL SKETCH ...... 167

8 LIST OF TABLES Table page

1-1 Cost comparison of IP telecommunications Systems (all prices in USD) [1] . 27

2-1 IP telecommunications codec comparison ...... 34

2-2 Mean Opinion Score ...... 35

2-3 Downtime for different availabilities ...... 36

2-4 Inter-arrival time distributions...... 43

2-5 Queuing service disciplines ...... 43

2-6 Client and server pricing...... 57

2-7 CALPricing ...... 58

2-8 Lync Online USL ...... 58

2-9 OnSIP calling rates...... 65

3-1 SIP response codes ...... 73

5-1 Test computer systems...... 102

5-2 D-IPTS call scenario tests ...... 106

6-1 Kamailio call processing Times ...... 123

9 LIST OF FIGURES Figure page

1-1 Distributed IP-based Telecommunication System architecture ...... 21

1-2 PSTN architecture ...... 22

1-3 VoIP calling using adapter...... 24

1-4 Redundancy in a traditional PBX [2]...... 26

2-1 Packet switching ...... 31

2-2 A-to-D conversion...... 34

2-3 System conﬁguration architectures...... 39

2-4 Combined system conﬁguration architectures...... 39

2-5 Network QoE factors...... 42

2-6 Queuing Model...... 42

2-7 Cisco’s Five-Nines System ...... 56

2-8 ShoreTel’s Distributed Call Control ...... 57

2-9 Vonage pricing ...... 61

2-10 Skype pricing...... 62

3-1 SIP triangular topology...... 71

3-2 SIP protocol architecture...... 72

3-3 SDP message for SIP INVITE...... 74

3-4 RTP protocol stack...... 75

4-1 Feature set of D-IPTS ...... 76

4-2 Components of D-IPTS Testbed...... 78

4-3 FreeSWITCH as SBC ...... 83

10 4-4 FreeSWITCH as B2BUA...... 83

4-5 D-IPTS web interface...... 84

4-6 D-IPTS DNS zone ﬁle ...... 84

4-7 D-IPTS DHCP conﬁguration...... 86

4-8 Phone provisioning using web interface ...... 87

4-9 Automatic provisioning of phones...... 87

4-10 D-IPTS System architecture...... 88

4-11 Topology of the D-IPTS...... 90

4-12 RTPproxy used to resolve NAT issues in D-IPTS ...... 91

4-13 User mobility in the D-IPTS ...... 92

4-14 D-IPTS call ﬂow...... 93

4-15 D-IPTS high availability design ...... 97

4-16 D-IPTS database redundancy...... 97

4-17 D-IPTS multiple server implementation...... 99

4-18 Virtual machine vs. container ...... 99

4-19 D-IPTS LXC implementation...... 100

5-1 D-IPTS system evaluation methodology: Relations of model types . . . 101

5-2 D-IPTS system evaluation methodology: Reliability evaluation ﬂow

chart...... 101

5-3 D-IPTS System architecture...... 102

5-4 Distributed IPTS test conﬁguration ...... 103

5-5 Register-Invite client ...... 103

5-6 top process monitoring sample screenshot ...... 104

11 5-7 htop process monitor sample screenshot...... 104

5-8 Call scenarios testbed ...... 106

5-9 OPTIONS message ﬂow...... 107

5-10 Average response times of messages ...... 107

5-11 Success rate of OPTIONS message ...... 107

5-12 MESSAGE call ﬂow...... 108

5-13 10000 CPS with 70 concurrent ...... 109

5-14 10000 CPS test result ...... 109

5-15 Registration call ﬂow...... 109

5-16 Registrations RTT...... 110

5-17 INVITE transmission results...... 111

5-18 Response times for REGISTER-INVITE ...... 112

5-19 Invite test results for r7000 l27...... 113

5-20 Kamailio test INVITE call ﬂow...... 114

5-21 Kamailio testbed conﬁguration...... 114

5-22 Number of concurrent calls vs CPU core performance ...... 115

5-23 Kamailio CPU utilization ...... 116

5-24 FreeSWITCH concurrent calls test-Intel at 15 CPS ...... 117

5-25 FreeSWITCH CPU utilization ...... 118

6-1 D-IPTS System queuing model ...... 122

6-2 Queuing model testbed ...... 123

6-3 D-IPTS queuing experiment call ﬂow...... 124

6-4 Mean processing time values ...... 125

12 6-5 Overall processing time vs call rate of calls ...... 125

6-6 Utilization of the D-IPTS ...... 126

6-7 Expected number of calls in the D-IPTS ...... 127

6-8 Waiting time in Queue ...... 128

6-9 Predicted values based on Queuing Model ...... 129

13 LIST OF ABBREVIATIONS

ACF Alpine Configuration Framework ATA Analog Telephone Adapter B2BUA Back to Back User Agent CAL Client Access Licenses CO Central Office CoTDMA Coarse-grained Time-division Multiple-access CS-ACELP Conjugate-structure Algebraic Code-excited Linear Prediction CSMA Carrier Sense Multiple Access D-IPTS Distributed IP-based Telecommunication System DHCP Dynamic Host Configuration Protocol DID Direct Inward Dialing DiffServ Differentiated Services FIFO First-In-First-Out HAD High Availability Daemon IAX Inter-Asterisk eXchange Protocol IP Internet Protocol ITSP Internet Telephone Service Provider LAN Local Area Network LBU Linux Backup Utility LXC Linux Containers MAC Media Access Control Address MOS Mean Opinion Score MTBF Mean Time Between Failures MTTR Mean Time To Restore NAPTR Name Authority Pointer

14 NAT Network Address Translation NGS Next Generation Systems NQoS Network Quality of Service PBX Private Branch Exchange PDF Probability Distribution Function PSTN Public Switched Telephone Network QoE Quality of Experience QoS Quality of Service QT Queuing Theory RAM Random Access Memory RBD Reliability Block Diagram RTCP RTP Control Protocol RTP Real-Time Transport Protocol RTP Real-time Transport Protocol SBC Session Border Controller SCCP/SKINNY Skinny Call Control Protocol SCTP Stream Control Transmission Protocol SDP Session Description Protocol SDP Session Description Protocol SIP Session Initiation Protocol SPS SIP Proxy Server SRV Service Record TCP Transmission Control Protocol TLS Transport Layer Security UDP User Datagram Protocol VM Virtual Machine VoIP Voice over Internet Protocol

15 VPN Virtual Private Network

16 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy A DESIGN AND PERFORMANCE STUDY OF A DISTRIBUTED IP-BASED TELECOMMUNICATION SYSTEM (D-IPTS) By Carlton A. Thompson May 2016 Chair: Haniph Latchman Cochair: Janise McNair Major: Electrical and Computer Engineering In this dissertation, it is shown that the commonly used centralized model for an IP-based telecommunication system fails to exploit the full capabilities of Internet- inspired communications, and that very inexpensive, elegant, and flexible solutions are possible by deliberately avoiding the centralized approach. The dissertation describes the philosophy, design, and implementation of a fully distributed IP-based Telecommunication System (D-IPTS) that provides the essential feature set for office and home telecommunications, including IP-based long-distance and local voice communications, as well as providing video, data, text, and mobility. A prototype D-IPTS was implemented using a customized SIP router and media server, in conjunction with a PostgreSQL database, to provide numerous features including support for dial plans, MOH, PSTN connectivity, DIDs, custom IVRs, voicemail, and conferencing. The D-IPTS leverages DHCP “option 66” and DNS SRV mechanisms, together with Python, XML, and shell scripts to provide automatic provisioning and failover. A NAT traversal solution was implemented using RTPproxy, as it is device independent. A statistical model of the D-IPTS was also developed using empirical measurements from the prototype, to facilitate simulation and call capacity studies. Validation tests showed close consistency between the model and the real system, with deviation in the range of 0-22% for call completions. The model was used to test the upper limits of the D-IPTS, and it was

17 determined that the D-IPTS built on an AMD A8-5600K CPU is capable of 3,300 calls per second, using Kamailio. At this high calling rate, calls in the D-IPTS will see a maximum signaling delay of 1.6 milliseconds. Results of experimentation show that the service time, µ, scales with CPU speed, and based on this the developed system model can be used to predict the performance of any CPU. The model was also used to predict that the D-IPTS would need to be scaled/load balanced if there is a performance requirement of greater than 2.8 million busy hour call attempts for Kamailio and 144 thousand for FreeSWITCH. From experiments, it was also determined that the call processing rate of Kamailio was largely dependent on CPU type, and Kamailio was able to process calls at double the rate when a quad core CPU was used versus a dual core (800 CPS vs. 400 CPS) before the CPU reached 100% utilization. FreeSWITCH in both CPU cases was only able to handle a maximum of 30 CPS without ever exceeding 50% CPU utilization. On an Intel Core(TM)2 Duo CPU, FreeSWITCH was determined to have a simultaneous call capacity of 100.

18 CHAPTER 1 INTRODUCTION With an ever expanding global community traditional phone systems, based on the Public Switched Telephone Networks (PSTN), lacks the ease of scalability, efficiency, and flexibility needed for the growing population demanding new and more sophisticated features. Voice over Internet Protocol (VoIP) systems provide a solution that addresses emerging and future communication needs. This work focuses more broadly on IP- based telecommunications featuring an integrated system of applications supporting voice, text, data, and video with mobility. Instead of a central IP Private Branch Exchange (PBX) model, a prototype for a fully distributed IP-based Telecommunication System (D-IPTS) will be thoroughly described and analyzed. This system uses a highly reliable and well designed local area network (LAN) infrastructure as the backbone (or essentially the back-plane – to use PBX/IP PBX analogies) and a fully distributed and Internet-inspired design to provide in-office IP-based local as well as long-distance, multimedia, and mobile communication. The use of an IP-based network also makes it extremely convenient to network multiple D-IPTS’s and relatively inexpensive to create a distributed regional or global network, without the requirement of special long distance trunks. Perhaps the biggest advantage of the D-IPTS design is the flexibility that it provides; this is leveraged to implement a fundamental paradigm shift in design philosophy. Instead of using centralized, expensive equipment and components at the core to drive the entire system, this design uses many inexpensive, replaceable components that can be upgraded, replaced, or removed without disrupting the operation of the active D-IPTS system. The end devices are intelligent IP phones, softphones, or applications on mobile devices that manage their presence and availability in the network. They also control parameters such as call forwarding rules and voicemail time-outs among other settings. Most of the updates and enhancements to the server are software based

19 and the system administrator can access the server remotely through the Internet or another IP-based network. Additionally, changes to the system such as modiﬁcation of particular parameters, the addition of new subscribers, or changing the privileges of existing subscribers can be done remotely and conveniently via an easy-to-use web-based interface. Also, upgrading the server is typically a software-only process and therefore proves to be an inexpensive system maintenance operation. IP-based systems are inherently highly scalable because of their modular design and extra modules can be added with additional software customization. Users beneﬁt from the portability that the IP-based network provides. By using a softphone, they can access the telephone network worldwide once connected to the Internet, and all that is required is the IP address or domain name of the server and the authentication information of the user.

Motivation

The objective of this research was comprised of two main goals, the first of which was to design and develop a viable prototype for a converged D-IPTS system that provides unified communications. The second goal of this research was to evaluate the performance, availability, and capacity scalability of the D-IPTS. To accomplish this latter goal I developed a theoretical model for the system and then validated the model against the prototype testbed. Theoretical and empirical performance results were obtained and shown to be consistent within reasonable tolerances. The model was then used to study the scalability of the D-IPTS, with respect to call capacity (incoming call rate and number of concurrent calls). The D-IPTS prototype system provides unified communications including voice, short message service (SMS), and video routed to a single device while on-the-go, with “follow-me” services, for home and office calls. The innovative system design is a feature-rich, open source, and fully functional D-IPTS developed in a manner that is well suited for a distributed design (see Figure 1-1). It features a Session Initiation Protocol (SIP) router, called Kamailio, running on Alpine Linux, with additional features like

20 music-on-hold, conference call, automated attendant, voicemail with email support, and other media services provided by FreeSWITCH, which also acts as a Session Border Controller (SBC). The innovative system also provides a web interface for common conﬁguration changes, an interface for automatic provisioning of IP telephones using DHCP option 66, as well as a DNS-based redundancy system. In the next section, a historical summary of the traditional telephone system is given with a view to highlight the desirability of the D-IPTS system.

Figure 1-1. Distributed IP-based Telecommunication System architecture

Voice Networks

Traditional Telecommunications Networks

The history of the Public Switched Telephone Network (PSTN), Figure 1-2, dates back to 1876 when Alexander Graham Bell patented the telephone [3]. The telephone system at that time began as a two phone peer-to-peer system with a single wire connection. As demand grew this model was no longer sufﬁcient so switching stations were incorporated. The switching station was a central location to which all phone lines were attached, and calls were connected by an operator. If a person wanted to make a call, they would pick up a phone, and connect to the operator. After telling the operator

21 to whom they would like to speak, the operator would manually connect the phone of the caller to the intended party.

Figure 1-2. PSTN architecture

This is no longer the case as a result of technological advancements. Circuit switching is now handled by computers that are capable of handling hundreds of calls simultaneously. The PSTN is made up of several components: • Edge Devices

– Analog/Digital Telephones • Local Loops • Central Ofﬁce (CO) Switches • Private Branch Exchange (PBX) • Toll Centers • Trunks • Regional Centers The edge devices are where the calls are terminated (connected), and are either an analog or digital telephone. Local loops are the two-wire connections that interface the subscriber to the PSTN network or a Private Branch Exchange (PBX). Central Ofﬁce (CO) switches and PBXs are the key components of the telephone network.

22 These are where all the end device signaling is generated. They are in charge of all call management, tone generation, powering end devices, and terminating local loops. In short they are the brain of the PSTN. If the called telephone is attached to a different CO then the call must go through a Toll Center. Regional Centers are the top level of the hierarchy and are used when a long distance call is being made. Toll centers, CO’s, and Regional Centers are connected via high bandwidth communications links called trunks. The PSTN was designed with voice in mind. Thus, it provides high quality and reliable voice calling. In today’s mobile age, people want to do more than just talk. People want to talk, text, email, watch videos, and transfer other data type content from a fixed or mobile platform. The PSTN was not designed for this type of high bandwidth traffic; therefore, it has left open the door for other types of voice networks. The D-IPTS is capable of efficiently leveraging the high bandwidth capabilities of the Internet to provide a more feature-rich communications environment.

VoIP Networks

Recently there has been a push by telecommunications service providers to move to IP-based telephony because of the cost savings, performance, ﬂexibility, and scalability. The two major application domains that have been addressed in the evolution of VoIP technologies are (i) VoIP for telephone call cost reduction and (ii) VoIP for in-ofﬁce communication.

VoIP for telephone long distance cost reduction

Providers, such as Vonage and MagicJack, have helped to popularize the idea of ‘Voice over IP’ (VoIP) [4, 5]. Such VoIP telephony has been made possible via an Internet connection by using a VoIP phone adapter, (see Figure 1-3) or a computer application. This allows the connection to an Internet Telephone Service Provider (ITSP ) to make free or low-cost VoIP calls to the PSTN worldwide. Such systems work by converting the analog signal to a digital format that is sent over the Internet. A VoIP user desiring to call regular PSTN numbers, must ﬁrst establish an account with an

23 ITSP which then enables VoIP calls via their analog telephone, computer, or mobile device software. VoIP users are able to make free calls to other users of the ITSP system and local, long distance, and international calls to PSTN numbers for a fraction of what it costs via traditional PSTN. For example, at the time of this writing, Vonage offers long distance calls to some countries for as little as $0.05 per minute (call to Belize) [4]. This is in contrast to the cost of $0.48 per minute using the PSTN or $0.55 using a cellphone [6]. This is nearly a ten times cost reduction by using VoIP versus the PSTN or cellphone. The ITSP typically has a contract with the destination country or intermediary PSTN provider. Pre-paid cellphones typical cost users upwards of $0.25/min compared to using VoIP which can be as low as $0.0198 / min. In some countries VoIP devices are considered illegal due to the issue of ‘revenue-bypass’, with outbound international calling being the most sensitive [7]. The tremendous cost savings in using VoIP for local, long distance, and international calls result from using the Internet and packet switching to replace the sequence of traditional telephone circuits made up of central ofﬁce, tandem switches, toll switches, and a variety of circuit based telecommunications links [8].

Figure 1-3. VoIP calling using adapter

24 VoIP for in-ofﬁce communication

The same IP-based technologies may also be used to reap tremendous cost, performance, and service benefits for in-office communications by replacing the central PBX with an IP-based system. The traditional PBX centralized design uses component redundancy to increase reliability (see Figure 1-4[ 2]). The major components (CPU, the Memory, the Switching Matrix and the Power supply), have failover backups in the case of an outage. It must be noted, however, that there is no redundancy at the line and trunk interface level. Thus, in the case of line or trunk interface failure, users will experience service disruptions. Though highly reliable, traditional PBXs for in-office communication are based on a centralized design that goes back to the early days of telephony, with all the intelligence of the telephone system located in the central PBX of the switching system. This design is inherently costly to purchase, maintain, operate, and expand. In addition, it also promotes vendor lock-in, where new features are only available only when the vendor supplies them, in a proprietary way, and usually with significant costs. These factors have motivated a great deal of interest in migrating to an open IP based system using the Session Initiation Protocol (SIP), which supports in-office telephony and at the same time allows access to the cost savings in VoIP calling. In contrast to proprietary PBX’s, as an IETF open standard, customer-driven SIP customizations are now possible and cost-effective. These benefits are highlighted in the implementation of the IP-based Telecommunication System (D-IPTS) described in this dissertation. The large majority of VoIP developers of in-office telephone solutions have adopted essentially the same legacy centralized philosophy of the traditional PBX and have built what may be termed IP PBX’s in which the RJ-11 from the PBX are replaced by RJ-45 Ethernet wires connected to IP telephones. In this dissertation, it is argued that this legacy centralized model grossly limits the exploitation of the full potential of Internet-inspired telecommunications.

25 Figure 1-4. Redundancy in a traditional PBX [2]

Aims and Objectives

For IP telecommunications to be widely adopted it must demonstrate the same or greater level of availability than the “Five Nines” (99.999%) performance of the current PSTN system [2]. However, IP telecommunication systems must also provide new levels of service/features to be widely adopted. Also, the telecommunication system must have a cost advantage over PSTNs. The D-IPTS system that is designed and described in this dissertation will offer converged communications for a fraction of what it costs compared to current IP PBX systems (see Table 1-1) with equal and potentially higher reliability and availability, along with support for a wide variety of new and advanced services and features. Table 1-1 shows cost to deploy 2,000 IP phones to 30 locations over ﬁve years by Avaya, Cisco, and Nortel [1]. The D-IPTS system uses the Session Initiation Protocol and features a fully distributed design and is run from random access memory (RAM). This D-IPTS system boasts a novel distributed architecture that is different from current state of art; it supports a convergence of voice, video, text, and data and also includes a mobility feature to provide hand-offs between different network

26 types. This novel D-IPTS system has wide applicability and will satisfy the demand among many users: • Small Businesses • Large Corporations • Universities • Hospitals • Travelers • Home users Table 1-1. Cost comparison of IP telecommunications Systems (all prices in USD) [1] Avaya Cisco Nortel D-IPTS Operational Start-up Costs (planning) 633,500 879,500 637,500 - Capital Costs (devices and equipment) 1,232,500 1,078,000 1,492,50 255,000 Ongoing Operational Costs 314,000 2,290,000 536,000 - Network Upgrades - - - - VoIP Management 50,000 50,000 50,000 0 Licenses 622,740 448,392 643,920 0 Training 33,600 33,600 33,600 33,600

In the literature there have only been reports on distribution and performance analysis of voice only networks. The contributions of this research are as follows: • Design of a fully functional, scalable and expandable, interactive multimedia IP telecommunication system • Design and implementation of a fully distributed IPTS architecture • Design of a robust and cost-effective USB-based run-from-RAM system • Development of a theoretical model of the D-IPTS • Comprehensive empirical performance analysis of the D-IPTS system used to validate the theoretical model and this is in turn used to study the capacity and projected performance limits of the D-IPTS

Dissertation Structure

The dissertation is organized as follows: Chapter 2 gives a background on IP telecommunications and a related literature review, Chapter 3 gives an overview of the

27 Session Initiation Protocol, which is at the core of the message exchange in the D-IPTS, and Chapter 4 gives a detailed description of the D-IPTS system. In Chapters 5 and 6 a theoretical model of the D-IPTS is developed and a performance analysis of the D-IPTS is performed based on call handling capability (call arrivals and concurrent calls), call delays, CPU and memory utilization, and scalability. Performance results are presented using empirical measurements using the prototype D-IPTS testbed as well as for the theoretical model. In addition, capacity and performance projections are presented based on the validated theoretical model. Finally, Chapter 7 gives concluding remarks and direction for future work.

28 CHAPTER 2 BACKGROUND

Overview of Existing and Emerging IP Telecommunications Systems

Voice over Internet Protocol (VoIP ) is often referred to as Internet telephony and encompasses real-time voice communication over the Internet. In a 2003 CNN interview, Federal Communications Commission Chairman Michael Powell stated, "What (VoIP) is going to do is start to weaken the foundation of the way we’ve done things for 100 years [9]." VoIP had its early start in 1994, when the co-founder (Alon Cohen) of an Israeli company VocalTec Inc., invented Internet based telecommunications when he dis- covered that is possible to send voice packets over the Internet [10]. In the years that followed the company produced the ﬁrst Internet phone software (softphone) and ex- panded technology to allow users to have PSTN connectivity by developing an Internet Telephone Gateway. From 1996 on, VoIP technology began to mature, enhanced with the development of the Asterisk [11] PBX software, and fueled by low-cost broadband service, which triggered the widespread adoption of VoIP telephony. The introduction of servers in the LAN environment established an infrastructure that allowed for the development of the IP PBX. More companies began adopting VoIP as an alternative to the PSTN, because of the cost savings for both long distance and LAN calling, as well as the addition of potential feature sets. VoIP was introduced to most home users in 2003 with the advent of Skype [12], which allows users to make free PC-to-PC calls over the Internet [13]. In 2006, the VoIP industry generated revenue of approximately $3 billion and in that same year, Tagg et.al developed the ﬁrst mobile VoIP application, Truphone [14]. By 2008, VoIP had taken over 80% of international PBX market [15], with some 210,000 VoIP subscribers in the USA. In 2010, VocalTec became a part of MagicJack. In 2011, the number of users hit a recorded 29 million, almost triple what it was the previous year [12]. The global market in

29 2012 generated $63 billion. In 2015 VoIP is predicted to generate $74 billion in revenue and the mobile VoIP expected to reach $1 billion in 2017. There are three main components of an IP telecommunications network: • Edge Devices

– IP Phones – Gateways • Communications Managers • IP data network Edge devices are used to connect users to the IP telecommunications network. These include analog phones connected via analog telephone adapter (ATA) and IP phones, which include PC softphones, hardware phones, and mobile devices. Gateways are used to interconnect VoIP users to the PSTN network and PSTN users to VoIP network. Communication managers are used for call control and signaling for edge devices as well as media services. IP telecommunication operates over the Internet Protocol (IP) data network, where data is routed via packet switching. In packet switching data is segmented into chunks of information called packets, which are routed through the network along varying routes till their ﬁnal destination (see Figure 2-1). Packet Switching uses a “Best-effort” model whereby the best effort is made for fast transmission, but packet delivery is not guaranteed [8]. This is a challenge for VoIP where loss of packets will cause choppy audio or silence. This will be further discussed in Chapter 5, where methods of overcoming this challenge will be explored. VoIP uses other protocols to achieve call signaling and media delivery. The Session Initiation Protocol (SIP) and Real-time Transport Protocol (RTP) are two such protocols that will be discussed later. Security is of great concern for IP telecommunications. It is not the objective of this dissertation to explore the security of IP telecommunications, however the D-IPTS implements security

30 measures for system security as well as call integrity. The study of these security measures is the subject of future work. It should be noted that this dissertation addresses not just VoIP, but the much broader field of interactive multimedia IP telecommunications and so references will be made throughout to IP telecommunications rather than to the much more limited field of VoIP. None the less the main signaling protocols used in IP telecommunications are the same as those used in VoIP (with appropriate variants to account for media types, data rates, etc.) and these protocols will be briefly discussed in the following section. Further details are provided in the dissertation as needed.

Figure 2-1. Packet switching

IP Telecommunications Protocols

In IP telecommunications, voice packets ﬂow end-to-end, while all signaling is done between the end device and its communications management application. Signaling protocols are needed to initiate or terminate calls and also are used in deﬁning all call parameters. Several different signaling protocols can be used in IP telecommunications. The main IP telecommunications protocols are: • SCCP • IAX • SIP • SDP

31 The following sections give a brief overview of the signaling protocols.

SCCP Signaling Protocol

The Skinny Call Control Protocol (SCCP/SKINNY) is a Cisco proprietary protocol used for signaling between IP devices and Cisco Uniﬁed Communications Manager. SCCP uses the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP ) to transport signaling to Call Managers and Real-time Transport Protocol (RTP) audio streams respectively. The main drawback to SCCP is that it is a proprietary protocol that is only available for use with Cisco hardware/software.

IAX Signaling Protocol

The Inter-Asterisk eXchange protocol (IAX) is an application-layer control and media protocol for creating, modifying, and terminating multimedia sessions [16]. IAX is a proprietary protocol that was developed by the creator of the Asterisk PBX software in order to minimize signaling bandwidth and provide support for network address translation (NAT) with no additional conﬁguration. The combination of control and media signaling in the same protocol is what makes IAX different.

SIP Signaling Protocol

Session Initiation Protocol (SIP) is a text-based application-layer protocol, and its syntax is very similar to the Hypertext Transfer Protocol (HTTP) [17]. It does not serve as a media gateway and is solely responsible for session setup/tear-down signaling. SIP does not deﬁne the media transfer protocol; it can be used via either TCP or UDP, and by default uses port number 5060. The similarity of SIP to HTTP allows compatibility with web browsers. The Distributed IP-based Telecommunication System (D-IPTS) that is developed for this research utilizes the SIP Protocol.

SDP Signaling Protocol

32 Data Transfer Protocols

Signaling protocols are used in the initiation and termination of calls, whereas media protocols are used for the digitization of voice trafﬁc before it can be transmitted over the Internet. The next sections describe commonly used protocols in IP telecommunications.

Real-time Transport Protocol

The Real-Time Transport Protocol (RTP), encapsulates media into UDP datagrams and then to IP packets. RTP is used to transmit data such as audio or video and works in conjunction with the RTP Control Protocol (RTCP) to monitor and identify packet delivery [19]. Secure Real-time Transport Protocol (SRTP) is an extension of RTP to provide encryption, message authentication and integrity, and replay protection [20].

Resource Reservation Protocol

Resource Reservation Protocol (RSVP) is an Internet control protocol used to reserve resources for a session by a router or a host [21]. RSVP allows a host to request a desired QoS and a router is able to require that level on all hops/nodes in the path.

Real Time Streaming Protocol

Real Time Streaming Protocol (RTSP) is used to stream multimedia content. RTSP establishes and controls streaming sessions [22]. RTSP allows applications to control media playback such as stop, rewind, and fast forward.

Voice Codecs

Voice information comes in the form of analog signals, which must be digitized before being transmitted via the Internet (see Figure2-2). Depending on the signal source the voice signal is converted using one of several different codecs. Cellphones use GSM-based codecs while traditional phones use G.711 and G.722, which are designed for IP telecommunications applications. Audio companding is a method

33 Figure 2-2. A-to-D conversion

for allowing signals with a large dynamic range to be transmitted through mediums that have a smaller dynamic range, and the standards for achieving this are set by the ITU-T [3]. The ITU-T is the telecommunications division of the International Telecommunication Unit, that sets the standards for telecommunications. G.711 and G.722 are both ITU-T standards while GSM is a standard deﬁned by the European Telecommunications Standards Institute (ETSI). The commonly used voice codecs used in IP telecommunications are, G.729, G.722, and G.711. Each of these offers a different level of bandwidth usage and voice quality (see Table 2-1[23]).

Table 2-1. IP telecommunications codec comparison Codec Bandwidth MOS Quality G.729 31.2 Kbps 3.92 Average G.722 87.2 Kbps 4.5 Very good G.711 87.2 Kbps 4.1 Good

Audio quality for voice codecs is rated using the Mean Opinion Score (MOS). MOS is a subjective test that rates voice quality based on human opinion of the perceived quality. It is on a scale of 1-5, with 1 being very bad and 5 being excellent[24].

G.729

G.729 is an audio encoding codec that uses an algorithm that uses a conjugate- structure algebraic code-excited linear prediction (CS-ACELP) to digitize voice signals at a rate of 8 KB/s [25]. G.729 offers “toll” quality voice and has low bandwidth requirements that make it ideal for IP telecommunications networks that have bandwidth

34 Table 2-2. Mean Opinion Score MOS Quality Impairment 5 Excellent Imperceptible 4 Good Perceptible but not annoying 3 Fair Slightly annoying 2 Poor Annoying 1 Bad Very annoying

limitations. G.729 has several alternate schemes called, “Annexes” that offer lower bit rate of 6.54 KB/s (Annex D,F,H,I, and C++) and 11.8 KB/s (Annex E,G,H,I, and C+). The variation in bit rates offers different levels of audio quality. The drawback to G.729 is that it requires exorbitant CPU resources, and may cause older hardware to fail. G.729 has an MOS value of 3.92.

G.722

G.722 provides a wider bandwidth for audio encoding allowing for an increase in voice quality while remaining relatively simple in comparison to G.711[26]. G.722 operates on an audio band of 50-7000 Hz. G.722 samples audio at a rate of 16 kHz with 14 bits per sample. This higher sampling rate results in HD quality audio of a much higher value than that of PSTN.

G.711

G.711 is a narrow-band audio encoding codec that allows the creation of 64bit/s bitstream from an 8 kHz signal with 8 bits per sample. G.711 uses two separate algorithms, A-law and µ-law. A-law is used primarily in the U.S. and Japan and provides more quantization at lower signal levels, while µ-law is capable of giving more resolution to higher ranged signals [27]. G.711 has a processing delay of 0.125ms. G.711 is supported by most IP telecommunications providers and is the same codec used by the PSTN so offers similar voice quality. With overhead included the codec can use up to 84 KB/s which can be strenuous on low bandwidth systems. G.711 has an MOS value of 4.1.

35 IP Telecommunications Availability

Availability

High Availability has been one of the deﬁning aspects of the PSTN for decades. Availability is the percentage of time that if you pick up the telephone you obtain a dial tone and are able to establish and successfully make a call. An important concept to consider when discussing availability is reliability, which is a measure of time before a service is no longer available. A system that is reliable is also available but availability does not imply reliability. Although the D-IPTS has reliability mechanisms in place, the objective of this dissertation is to evaluate availability. There are three metrics that can be used to classify Availability: Mean Time Between Failures (MTBF), Mean Time to Restore (MTTR), and performance. MTBF is the average time a system is expected to function without failure. MTTR is the time required to get the system back to a functional state after a failure. The performance is based on latency, dropped calls, or blocked calls. The system availability can be expressed by Equation 2–1[28]:

MTBF A = (2–1) MTBF + MTTR

Table 2-3. Downtime for different availabilities Availability Downtime per year Downtime per week 98% 7.3 days 3 hours, 36 minutes 99% 3.65 days 1 hour 68 minutes 99.9% 8 hours, 76 min 10 minute, 1 seconds 99.99% 52.6 minutes 1 minute 99.999% 5 minutes, 26 seconds 6.05 seconds 99.9999% 31.5 seconds 0.605 seconds

Traditional phone systems are classiﬁed as having “Five-Nines” availability, which refers to 99.999% system up-time. This means that in a given year the system is only down for 5 minutes and 26 seconds. When this value is advertised, it should be noted that this is only referring to the availability of the PBX or theCO. When all the elements that make up the entire network is factored in, ’Five-Nines’ is extremely hard to achieve.

36 To get a complete picture of availability, the end-to-end availability, which is a product of all the network availabilities, should be considered:

Ae2e = Ah1 × Alocal1 × Anetwork × Alocal2 × Ah2 (2–2)

Equation 2–2 represents the end-to-end availability. Ah1and Ah2represent the

availability of user handsets, Alocal1and Alocal2are the availabilities of the switching

networks ( PBX or theCO), and A network is the availability of the PSTN network. Phone

companies target Alocal1and Alocal2 when discussing availability as these are the factors most easily controlled. Loss of availability is not only a matter of inconvenience but also a ﬁnancial one. Service availability is critical in telecommunications because telephone use is an integral part of our daily personal and business lives, and any loss of service can create a loss of revenue, and also gives rise to life safety issues. A IP telecommunication system must be capable of meeting or exceeding the PSTN phone system’s “Fine-Nines” availability and performance, to pose as a viable alternative. The unplanned downtime can then be expressed as [29]:

1 − A Downtime = × (365 × 24 × 60)min/yr (2–3) 100 One way to measure the performance of PSTN systems is using Queuing Theory, which allows PSTN designers to determine the number of trunk lines and COs necessary to service an area. Poor and insufﬁcient planning will increase the probability of blocking, which is when a user is unable to make a call due to network congestion. An extreme example would be if a CO is only able to handle one hundred calls and there is an emergency situation where 200 people try to dial “911” at the same time. In this scenario, only the ﬁrst 100 people would get through to “911”, and all others would be unable to make a call. Queuing Theory allows designers to plan for these situations and modify their system design to meet day-to-day demand.

37 Another way to measure performance used by PSTN designers is the Erlang method. The Erlang method is used to describe the total trafﬁc volume in one hour for a voice path [30]. Using Erlang methods allows for the determination of how many lines are required between a telephone system and a CO or PBX [30]. In short the Erlang is a unit of measurement for call volume. Unlike the Erlang, Queuing theory is used to perform statistical analysis of the system. Queuing theory is able to take into account randomness of call arrivals and inter-arrivals. Both however can be used to make predictions and make design changes to meet call capacity requirements.

Reliability Block Diagram

Sydney Water in [31] uses a reliability block diagram (RBD) to aid in system design. The RBD should be used by all designers before any new system can be implemented. A system can have components in a series or parallel configuration. Series connections are easier to implement, but they suffer from the “Christmas tree light” effect where if one bulb is blown the rest of the lights along the link will not function (see Figure 2-3a). The same concept can be applied for any series configured system; all pieces must work for the system to operate. Parallel connections are more robust and best suited for redundant designs, since when other paths exist, a failed element will not interrupt traffic flow (see Figure 2-3b). Sydney Water [31] assumes the failure distribution is exponential and has a constant failure rate λ. With a constant failure rate they define: Cumulative Failure rate, F(t) = 1 − e−λt Reliability, R(t) = e−λt

1 Mean Time to Failure, MTTF = λ ; average time before element is non-repairable For a parallel system, reliability can then be determined from the following equation:

RS(t) = 1 − [(1 − R1)(1 − R2)(1 − R3) ... (1 − Rn)] (2–4)

The MTTF is then expressed as:

38 (a) Series Configuration (b) Parallel Configuration Figure 2-3. System configuration architectures

1 1 1 1 1 1 1 1 MTTF = + + + ... + − + + + ... + + λ1 λ2 λ3 λn (λ1 + λ2) (λ1 + λ3) (λ2 + λ3) (λi + λn)

1 + (2–5) (λ1 + λ2 + λ3 + ... + λn)

In a more complex system design the series and parallel conﬁgurations can be combined, as can be seen in Figure 2-4.

Figure 2-4. Combined system conﬁguration architectures

In this situation the resulting system reliability is then [31]:

39 RS = [1 − (1 − RB)(1 − RC)] (R6) (2–6)

Performance Measures for IP Telecommunications

Quality of Service

In a perfect world, we would be able to transfer an inﬁnite amount of data instantaneously. However, this is not possible due to network limitations, limited throughput, delays between end systems, and packet loss. These factor have an effect on the overall Quality of Service (QoS) that a system can provide. The ability of a network to handle the demands of applications or services is called Network QoS (NQoS) [32].

Network QoS parameters

Factors that affect NQoS are as follows: • Packet Loss • Delay • Jitter

Packet loss. Packet loss occurs when a packet arrives at a router with a full queue. Since a router’s queue is ﬁnite, the router will drop the packet. The probability of packet loss is an important factor when designing an IP telecommunication system, because lost packets result in a degradation of voice quality. Ideally this should be 0% loss, but this is impossible in a real world situation.

Delay. The delay is the time that it takes a packet to travel from one node to the next until it reaches its destination [33]. Processing delay, queuing delay, transmission delay, and propagation delay are the most signiﬁcant delays affecting networks. The processing delay is the time it takes a node to check a packet’s header and destination information and to inspect the packet for errors. After being processed the packets are put in a queue. The time the packets spend in the queue is known as the queuing delay. After

40 leaving the queue the packet is sent to the next node, and the time it takes all the bits of the packet to be transmitted onto the transmission link is known as the transmission delay. Once the bits are on the link, the propagation delay is the time it takes to reach the subsequent node. Propagation delay is a factor of the transmission media, e.g. ﬁber optics, cable, etc. Latency, is an important parameter, and is the amount of delay that an application can tolerate; which is an important factor for IP telecommunications applications.The total delay see by a packet is the sum of all the delays (processing + queuing + transmission + propagation), and is given by Equation 2–7 where N is the number of nodes:

d total = N(dproc + dqueue + dtrans + dprop) (2–7)

Jitter. Given the different network delays, there is a time variation from packet to packet as they travel from source to destination. This variation in delays is known as Jitter, and it will cause packets to be received out of order, which creates choppy or unrecognizable audio. A solution to this network congestion issue is the use of a Jitter buffer. The buffer collects all the packets and reorders them so they can be sent in the proper sequence.

Measuring VoIP QoE

These network parameters all combine to affect a users overall Quality of Expe- rience (QoE), which is a qualitative measure of the service based on the customer experience (see Figure 2-5)[34]. These factors must be thoroughly quantiﬁed and analyzed when designing a reliable IP telecommunication system. One popular method for analysis is Queuing Theory.

Queuing Theory

Queuing Theory (QT) is a scientiﬁc method to predict the waiting times of queues and the associated probabilities of packet loss and delays.QT is also used to modify

41 Figure 2-5. Network QoE factors system design to meet required performance [35]. The basic queuing model is shown in Figure 2-6. Performance measures that can be calculated are:

• The blocking probability, PB • The overall system delay, W¯

• The queuing delay, W¯Q • The overall throughput • Idle time of the server • Queue length

Figure 2-6. Queuing Model

Key characteristics of a queuing system are:

1. Arrival Process

2. Service Process

3. Number of Servers

42 4. Size of the Queuing System

5. Queuing Discipline

Queuing techniques

There are ﬁve parameters that are used to describe a queuing system [35]: • Distribution of the inter-arrival times (A) (see Table 2-4) • distribution of the service times (B) • number of servers (C) • total system size (D) • service discipline (E) (see Table 2-5) These parameters are typically expressed using a symbolic notation, called Kendall’s notation, represented as A/B/C/D/E.

Table 2-4. Inter-arrival time distributions Distribution Type Notation

Memory less: Exponential M Deterministic D General G

E k Erlang type K

Hk Hyper Exponential Type K

Ph Phase Type

Table 2-5. Queuing service disciplines Service Disciple (Symbol) Meaning

FCFS/FIFO First Come First Serve/ First In First Out LIFS/LIFO Last In First Served/ Last In First Out RSS/SIRO Random Selection For Service/ Service In Random Order PR Priority GD General Distribution

43 Before discussing the queuing model, some equations will be deﬁned.

La Traffic Intensity is defined as R packets • Where a is the average rate packets arrive (units of sec ) bits • R is the transmission rate (units of sec ) bits • L is the number of bits per packet (units of packet ) La If R > 1, then the average rate at which bits arrive at the queue exceeds the rate at which bits are transmitted from the queue. This means that the system is unstable and the delay will increase towards infinity. One of the main objectives in analyzing queuing systems is to determine the probability distribution function (PDF). Queuing systems are best defined using a Poisson Process as it best balances the trade-off between accuracy and tractability. The following axioms are used: Let τ → 0 and λ the mean arrival rate

1. For the Poisson process, there is at most 1 arrival in τ

2. The probability of an arrival τ is proportional to its length

(a) P(exactly 1 arrival in[t, t + τ] = λτ )

3. Arrivals come in two intervals[t1, t2]and [t3, t4],

(a) where [t1, t2] ∩ [t3, t4] = ∅ Then the probability that there are n arrivals in a time interval t for a Poisson process of rate λ is given by:

(λt)n P(n) = e−λt (2–8) n! Using the Poisson process it can be shown that the inter-arrival times are exponentially distributed, for the number of events occurring during a given time period.

44 M/G/1 model

There are several different queuing models that are available but the M/G/1 Model was the most suitable for the D-IPTS (detailed in Chapter 6). “M” signiﬁes that the arrivals follow a Poisson process. “G” signiﬁes the service rates are generally distributed. The “1” indicates that there is only one server. It is also useful to model this information from a customer’s perspective. For this Little’s Equations is used, which gives the average number of users in the system at any given time. Using this information a user can determine what call delays or call blocking they may experience. The variables used in Little’s Equations are:

• Nq : number of users in the queue

• W q : mean time a user spends in the queue

• NS : number of users in the system

• W S : mean time a user spends in the system Deﬁne:

Nq = λWq (2–9)

NS = λWS (2–10)

Substitute P0 = 1 − ρ,

(1 − ρ)ρ L = (2–11) S (1 − ρ)2 ρ = (2–12) (1 − ρ)

Given an equation for LS the other values are,

45 λ L = L + (2–13) S q µ

LS = λWS (2–14)

Lq = λWq (2–15)

Summary

There are several factors that affect the performance and QoS of a IP telecommunication system. QT is technique for evaluating performance and availability of IP telecommunication systems. The M/G/1 Model was well suited for the theoretical modeling of the D-IPTS system. This model is later used to determine the overall system

delay, W¯ , the queuing delay, W¯Q, and the queue length. This resulted in a D-IPTS that was designed to have a high QoE and availability.

Literature Review

This section will cover a literature review of related work in these areas. The areas that will be covered are VoIP overviews, system designs, performance studies, system modeling and reliability, proprietary systems, and next generation systems.

VoIP Overview

An overview of VoIP is given in [36]. This article gives background on traditional telephony, OSI layers involved in VoIP, QoE Factors, security, and basics of SIP. The difference in this paper and the D-IPTS system is that their VoIP analysis focuses on a centralized IP PBX design. Walker in [37] provides a reference guide on considerations to make when implementing VoIP or managing VoIP systems. The guide includes information on dealing with proper codec selection based on available bandwidth and performance characteristics such as delay, jitter, and packet loss. The E-model is discussed as a

46 better way of computing MOS values. This guide is only an informational study and a compilation of other research.

System Design

In [38] a simple VoIP system was designed and implemented. This system was used to provide an interconnecting gateway between the university’s existing PBX networks and allow users to make free VoIP calls to other users and mobile numbers. Similar to the D-IPTS this system was designed for high availability and utilizes Kamailio and FreeSWITCH. However, Kamailio and FreeSWITCH were only used to act as the entry point into the existing system. Kamailio speciﬁcally was used for SIP signaling and as a load balancer for multiple instances of FreeSWITCH. The user would direct the call to the Kamailio server which would then redirect the call to FreeSWITCH. FreeSWITCH was used as an SBC and also for user authentication. Once the call was redirected from Kamailio the user was required to enter an authorization PIN. If PIN was entered correctly the call was bridged to its destination. FreeSWITCH would stay in the call loop acting as an SBC to proxy media streams. If the PIN was not entered correctly, then the call was dropped. High availability was achieved by using clustering software and redundant elements. The clustering software would replicate all signaling to a main and backup Kamailio server. If a failure occurs the software would assign the public IP to the backup and start the Kamailio process. The drawback to this system design was that their system remains heavily centralized and does not harness the full potential of using an IP network design. Also, Kamailio and FreeSWITCH were only used as an add-on feature to their existing centralized PBX network. In the D-IPTS calls are fully routed via Kamailio and RTPproxy with FreeSWITCH only used for media services and B2BUA. Prasad et al. [39] proposed a scalable distributed VoIP conferencing using SIP. Their distributed architecture used separate SIP proxy servers and conference servers. They argued that separating call control from media signaling improves reliability and function. Unlike the D-IPTS, this system focused only on conferencing; and it does not

47 have a full featured telecommunication phone system, and uses a single proxy server and conference server therefore it lacks any means for high availability. In [40] ACULAB makes the case for the advantages of using a distributed architecture for VoIP. ACULAB is a cloud-based telecommunication system . They offer a comprehensive range of telephony platforms including cloud-based, software-based and hardware-based platforms, and a wide range of gateways. The drawback is that since it is a proprietary solution you must purchase their hardware to use.

Performance

In [41] Jiang assesses VoIP on the Internet using availability measurements. Their experimental design involved 14 nodes in various cities in the United States, Japan, and China. They used a call generator to produce thousands of calls over a period of two months on the different networks. They determined that packet loss occurs at a greater rate over international links, and that network outages were a signiﬁcant factor in determining availability. Unlike the D-IPTS, this was not a telecommunication system but only an availability study using a commercial VoIP system. In a related work [42], an end-to-end study of VoIP performance and reliability on a cable network was conducted and the results were compared to the PSTN. This study analyzed the IP backbone’s availability based on equipment failures such as hardware and software. They used three metrics to measure reliability: (1) end-to-end availability, (2) call cutoff likelihood, and (3) unsuccessful call setup. Availability was determined to be highly correlated to whether redundant equipment was used or not. A performance evaluation was done on VoIP in a campus environment in [43]. They conducted a QoS analysis of VoIP calls and measured delay, throughput, jitter, and packet loss. They used Wireshark [44] as their analysis and packet capture tool. This study, however, used a centralized PBX model driven by Asterisk [11]. Chan and Liew in [45] performed a performance study of VoIP in multiple co- located wireless LANs. Their study determined that in the presence of neighboring

48 cells the VoIP capacity of a given router was severely degraded and this degradation increases with the presence of additional neighbors. They determined the cause of this loss in performance was due to Carrier Sense Multiple Access (CSMA) operation of nearby cells. They proposed a coarse-grained time-division multiple-access (CoTDMA ) approach that resolves the interference. Using this approach each voice session was given a separate time slot, and this resulted in an increased VoIP capacity of an access point. Wang [46] studied the advantages of using a Differentiated Services (DiffServ) model over a First-In-First-Out service (FIFO) model. The waiting times of both service models with the presence of VoIP trafﬁc and background trafﬁc were observed. The authors determined that utilizing DiffServ can improve QoS for VoIP versus using FIFO, which introduced larger jitter delays.

System Modeling and Reliability

Kambourakis in [47] proposed a solution for failover of SIP and RTPproxy servers as well as for load balancing. To accomplish high availability they use a combination of a proprietary software, High Availability Daemon (HAD), and SIP/RTP server redundancy. All messages get routed to main and backup servers via the HAD, but only the main server gets a public IP address. If a failure of the main server occurs the HAD assigns all trafﬁc to ﬂow through to the backup server. A major drawback, however, is that the HAD requires unnecessary additional routing. Also if the HAD fails, there is no way to perform failover. Koutras in [48], studied the effects on availability and service reliability by software rejuvenation, which is a technique used to reclaim resources by ending/restarting system processes to prevent system failure. They studied the steady-state availability of a VoIP system and imposed rejuvenation policies, when necessary. The system was modeled as a semi-Markov process to characterize the system behavior and determine optimum rejuvenation time. Their work is only theoretical and focuses only on

49 a voice-only system, and furthermore, they do not discuss how their rejuvenation policy affects ongoing calls or in process calls. Pandit in [49] discusses the advantages of enhancing Queuing Theory by combining with Network Calculus, which is used to consider the worse case scenario for a network design. The drawback to using NC for design is the network is not fully utilized. Queuing Theory, on the other hand, ensures good network utilization. They were able to create a theoretical model that showed the advantages of combining Queuing Theory and Network Calculus. This study was purely theoretical, and only applies to voice-only systems. In [50] Gurbani, Jagadeesan, and Mendiratta created a queuing model to determine the response time and mean number of jobs in a VoIP system. Their queuing model treated each SIP message as its own “queuing station,” which handles the request/response of messages. Each station was modeled with a given service rate and arrival rate. Using a M/M/∞ model for the system, they measured the mean response time of the system under various distances of up to 1000 miles between SIP entities. The study determined that the response time increased in accordance with propagation delay and that the mean response time and mean number of jobs in the system increased as arrival rate increased. The same tests were done for multiple proxy servers on a single host, where they used a M/M/m model. A reliability analysis was also done in order to determine the steady-state system availability and probability of job loss. Garg et al. in [51] evaluated three different replication schemes: no replication, cold replication, and warm replication, using a Continuous Time Markov Chain with M/M/1/k Queuing model. It was determined that warm replication provided the highest availability with a value of 0.9999. For job loss they assumed an inﬁnite buffer. Again the probability of job loss was lowest for warm replication. The conclusions were that the formulas can aid in determining the right replication schemes, server downtime, and CPU throughput.

50 Krishnamurthy and Rouskas in [52] evaluated the performance of OpenSIPS proxy server. They characterized the performance of the SIP proxy server (SPS) under various call arrival rates and inter-arrival time distributions. Each call was broken down into individual messages of which the response time was measured based on the CPS (Calls Per Second). The SPS was modeled as an M/G/1 queue and the call rate was varied from 100 to 1200 CPS. The main objective was to determine the processing time a packet spends in the proxy server based on packet size. Ahmed and Mansor in [53] evaluated the CPU requirements for running an Asterisk PBX. They varied the number of running concurrent phone calls for each of the CPU’s they tested. The CPU utilization was recorded for each test and the results plotted for an AMD Sempron 1.5GHz CPU and a Pentium D 2.8GHz Dual Core CPU. They also tested the CPU performance with and without G711 transcoding. The results showed that the number of concurrent calls is mostly dependent on the CPU power. In [54], the objective was to analyze SIP server performance on multi-core systems. Three different systems were analyzed: an AMD Opteron 8218 2.6 GHz, Intel Xeon X5450 3.0 GHz, and a IBM POWER6. Forty-seven of these machines were used in combination to generate the system load. All tests were done on a private network in order to minimize spurious trafﬁc. OpenSER was used as the SIP server software, with a MySQL database on a Red Hat Linux OS. The study concluded that an OpenSER conﬁguration on one core supports 350,000 users. Adding a core improves performance by 28%, resulting in 450,000 users, and eight cores yield only an additional 150,000 users or an overall improvement of 71%. When they did authentication testing they found an increase performance by 76%, 90%, 34%, and 19% for one, two, four, and eight cores, respectively. Also by increasing the size of the database hash table they improved performance four fold. Wu and Wang in [55] evaluated the performance of a SIP media control gateway (MGC). They assumed an M/G/1 queue model and conducted an evaluation based

51 on queuing size, mean queuing delay, and variance of queuing delay. They derived formulas for the average number of SIP-T messages in the system, queuing size, and mean queuing delay. They then ran simulations on the MGC and compared these results against the predicted results to determine the ratio of cost to performance and the planning and design needed in order to meet the requirements of carrier class VoIP network. The need for VoIP to integrate the existing telephone network was the driving force in [56]. In this paper the authors developed test cases for the interoperation of VoIP systems. They used a graphical finite state machine (FSM) tool called ITIS, developed at Bell Laboratories, to run algorithms generated from an extended finite state machine (EFSM). The test cases that were modeled were for End users and rest of communication system; and End users using H.323 against the rest of the system. Modeling was done for cases that would generate a failure. Algorithms were developed for 22 test cases that fulfill the interoperability requirements that appear in Bellcore’s LATA Switching Systems Generic Requirements [57]. Markopoulou et al. in [58] performed a characterization of failures in an IP Back- bone. In particular they studied Sprint’s operational IP backbone. In the Sprint network, when an IP link fails, traffic is automatically routed to alternate routes around the failed link. Data was gathered for 6 months using passive listeners installed at geographically diverse locations each time a failure occurred. Data was separated based on the types of failures and the causes. The categories used were failures due to maintenance activities, router-related, and optical layer problems. Results of the study indicate that 20% of all failures are due to scheduled network maintenance. The remaining percentage are due to unplanned failures. A router-related problem or an optical layer fault was the cause of 30% of the remaining unplanned failures, while the remaining 70% of the unplanned failures were individual link failures caused by a variety of problems.

52 Authors in [50], created a queuing model to determine the response time and mean number of jobs in the system. Their queuing model treated each SIP message as a “queuing station,” which handles the request/response of messages. Each station was modeled with a given service rate and arrival rate, using an M/M/∞ model. They measured the mean response time of the system under various distance 0-1000 miles between SIP entities. It was determined that the response time increased in relative to the propagation delay. Also, the mean response time and mean number of jobs in the system increased as arrival rate increased. Similar tests were done for multiple proxy servers on a single host, using an M/M/m model. A reliability analysis then determined the steady-state system availability and the probability of job loss. The study evaluated three different replication schemes; no replication, cold replication, and warm replication using Continuous Time Markov Chain models. It was determined that warm replication provides the highest availability with a value of 0.9999. For job loss, they assumed an inﬁnite buffer, and observed the probability of job loss was lowest for warm replication. In [52] the performance of OpenSIPS proxy server was evaluated. They characterized the performance of the sip proxy server (SPS) under various call arrival rates and inter-arrival time distributions. Each call was broken down into individual messages of which the response time was measured based on the calls per second (CPS). The SPS was modeled as an M/G/1 queue. The call rate was varied from 100 to 1200 CPS. The objective was to determine how long a packet spends in the proxy server based on packet size. In [43] the performance of VoIP system was evaluated. This VoIP network developed for low-cost on-campus communications utilizes an IP PBX design using Asterisks software. The criteria used were the throughput of the audio received, the delay from the sender to the receiver, the jitter, and the MOS. Wireshark was used to gather the data and record the performance metrics.

53 In [53] the CPU requirements for running an Asterisk PBX on an AMD Sempron 1.5GHz CPU and a Pentium D 2.8GHz Dual Core CPU was evaluated. The number of running concurrent phone calls for each of the CPUs was varied and the CPU utilization was recorded for each test. CPU performance was verified with and without G711 transcoding. The results showed that the number of concurrent calls is mostly dependent on the CPU power, and therefore the expected call volume for any system is largely dependent on the CPU. In [59] the performance of an SPS based on packet drop probability was studied. Their M/G/c/K Queuing Model predicted the drop probability of the multi-threaded system. The authors measured the time a packet takes to move through physical layer and application layer. They determined that since only one server thread can run at a given instance due to only one core being available, an increasing number of server threads leads to increasing overhead in switching and results in increases in the waiting time. In [60] a set of performance metrics for evaluating SIP proxy servers was developed. The SIPstone software was used to measure the server registrations per second (RPS) and CPS. They determined that the following factors should be used to evaluate a system: a) The number of connections requested by the clients and accepted by the SUT per second b) CPU and memory utilization of server at various loads; c). A curve plotting the transaction response time (TRT) as a function of request arrival rates, with at least four plot points and one value at approximately 10% of the capacity d) CPS and registrations per second (RPS). In [54] the performance of SIP server on multi-core systems was analyzed. The different systems were: an AMD Opteron 8218 2.6 GHz, Intel Xeon X5450 3.0 GHz, and an IBM POWER6. Tests were conducted on a private network to minimize extra traffic and OpenSER was used as the SIP server software. The test results show that a OpenSER configuration on a single core can support 350,000 users. Adding an

54 additional core will improve performance by 28%, resulting in 450,000 users, however there is not a direct linear increase subsequent additional cores. Authentication testing similarly yielded non-linear performance increases from one core to, two, four, and eight cores. In [61] models for service availability and demand were developed using a Markov Reward Model and combining that with Continuous Reward Logic. They modeled a VoIP system consisting of one main proxy server and a backup server. The failure rates and processing rates were used to create a Markov chain. They came up with recommendations that if not met, then the system needed to be redesigned to meet speciﬁcations or a readjustment of the speciﬁcations. The fundamental difference between the D-IPTS and the prior works just discussed is that the D-IPTS is a telecommunications system that has a full featured, distributed and Internet-inspired design that provides IP-based local, long-distance, interactive multimedia communication, and mobility. Additionally the D-IPTS is highly reliable due to its redundant proxy server, media server, and database server, while using open-software and low-cost hardware.

Proprietary Telephone Systems

There has also been proprietary systems developed by different vendors such as Cisco, Shoretel, Lync, 8x8, and Vonage.

Cisco

A Cisco IP PBX system developed on traditional PBX architecture is shown in Figure 2-7[ 2]. Despite claims of high reliability, the drawback is its centralized design, which suffers from all the negative attributes of traditional PBX design and is only a small step in advancement for VoIP. The cost of Cisco’s system is high; a BTS Feature License for 6.0.3 Release SIP Update feature costs $4,200,000, not including the primary and backup hardware costs or IP telephone costs [62].

55 Figure 2-7. Cisco’s Five-Nines System

ShoreTel

ShoreTel’s distributed VoIP system 2-8, as described in [63], utilizes a distributed call control component and employs modules with a relatively smaller number of components, thus making it less vulnerable to failures. ShoreTel uses a N+1 redundancy model, which removes the possibility of a single point of failure. In this method of redundancy, instead of a central component to manage the entire system, the load is distributed amongst multiple units. ShoreTel claims their modular design makes the components easier and cheaper to repair, however it still uses an interconnected web of centralized systems. Software reliability was not available at the time of its publishing; however the average cost per call control unit is about $3000, not including the IP telephones [64].

56 Figure 2-8. ShoreTel’s Distributed Call Control

Lync

Lync, or Microsoft Lync, is a proprietary Microsoft communications platform. Originally known as Ofﬁce Communicator, Lync started out as an instant messaging client designed for enterprises [65]. Lync has support for presence, instant messaging, voice, video, and meetings. Lync can integrate with other Microsoft software, such as with PowerPoint, that allows for increased collaboration. Other features include screen sharing and collective whiteboard space. There are three ways to use Microsoft Lync, each with varying costs: • On-premises • Microsoft-hosted • Partner-hosted To host a Lync Server On-premises the costs are shown in Table 2-6. In addition to the Server/CAL (CAL) license, it is also necessary to acquire the appropriate client software licenses for their client devices (see Table 2-7).

Table 2-6. Client and server pricing Microsoft Lync Pricing Prices Lync 2013 (client) $31 Lync Server 2013 $3,646

For the Microsoft hosted type, this is known as, Lync Online. Two different plans can be purchased (see Table 2-8). Lync Online Plan 1 is a less-feature rich client and to access all the features, such as audio and video conferencing, it is necessary to

57 Table 2-7. CAL Pricing Lync Server CAL Pricing Prices Standard CAL (per device) $31 Standard CAL (per user) $36 Enterprise CAL (per device) $107 Enterprise CAL (per user) $123 Plus CAL (per device) $107 Plus CAL (per user) $123

purchase Lync Online Plan 2. As with the ’ On-Premise’ system, these plans are on a per user basis, and costs increase sharply for large companies.

Table 2-8. Lync Online USL Lync Online Plan 1 $2.00 user/month Lync Online Plan 2 $5.50 user/month

The ’Partner hosted’ option a Microsoft partner is responsible for building, deploying, and servicing the Microsoft Lync Server 2013 products and solutions. The costs for this will vary depending on the company chosen, and the costs will be in addition the number of servers and CALs required. For external users to access Lync Server 2013 an additional external license may be required. Microsoft has recently discontinued Lync for a newer product Skype for Business (SfB) [66]. SfB is further discussed in Section2.

8x8, Inc.

8x8, Inc is a Cloud-based business communications service provider. They provide hosted VoIP-communications and collaboration solutions [67]. Formerly known as Integrated Information Technology, Inc., or IIT, 8x8 began its endeavor in VoIP when it began producing semiconductor chips, software and other technologies for the videoconferencing, in early 1990s [68]. From there 8x8 continued to expand its reach into VoIP technology until in 2002 when it launched its won VoIP service Packet8 [68], and simultaneously began providing business services. Already known as "the third-largest independent Internet-phone service provider with 181,000 customers,” the

58 company gained an additional 200,000 customers when VoIP start-up SunRocket was liquidated in 2007 [69]. Today 8x8 is in use by over 40,000 companies [67] and is ranked #1 Cloud UC Provider by Infonetics [70]. Information technology research and advisory firm Gartner has named 8x8 a leader in the Gartner Magic Quadrant for Unified Communications, for the four past years [71]. 8x8 unlike other providers distinguishes their service offerings based on industry, business size, and business needs. 8x8 offers similar feature set to the other providers, but has released new features as of March 2105, which are the following [72]: • Cloud telephony/PBX capabilities • Multi-branch/multi-facility configurability and management

– used to associate a group of extensions • Browser-based switchboard • Integration with Salesforce, NetSuite and Zendesk. • Service-level Agreement (SLA)

– Industry first 99.99% end-to-end service up-time and call quality [73] • Virtual office analytics • MPLS network connectivity options 8x8 provides reliability via SSAE 16 audited geographically dispersed global data centers, that provide automatic and transparent failover [74]. The company offers two different business service plans Virtual Office and Virtual Office Pro, charged per extension [75]. The two service plans offer the same standard features except the ’Pro’ plan includes web conferencing with video, Internet faxing, and call recording. The cost for each of the plans are $19.99/mo/extension for Virtual Office and $49.99/mo/extension for Virtual Office Pro [76]. 8x8 Office offers numerous additional types of extensions each with varying costs.

59 The services are subscription based pricing model with 1, 2, 3 and ﬁve-year contracts, that automatically renews. If you wish to cancel the service early there is a there is a $59.99 termination fee [76].

Vonage

Vonage provides cloud based communication for business as well as residential customers. Vonage business services includes 40+ business-critical features as part of their plan. The plan is priced at $39.99 per month and per extension and includes unlimited calling in the U.S and Canada. Additional features are available at extra monthly cost (see Figure 2-9). For $14.99 you can get a virtual extension and or a metered extension. A virtual extension gives employees a dedicated direct dial number which is forwarded to their mobile device [77]. A metered extension (pay as you go) is charged $0.03 a minute and is ideal for extensions that are infrequently used. Vonage also supports teleworking, allowing employees to access all available features from anywhere via Vonage Business Mobile apps and web portal interface. Three different mobile apps are available for use: desktop app, mobile app, and softphone app. Vonage makes claims of being able to deliver 99.999% up-time reliability. Vonage offers customers live one hour training sessions every week day, four times per day. The desktop software includes integration with search engines, social media, a CRM software via an in-call pop-up menu.

60 Figure 2-9. Vonage Pricing [77]

As can be seen from the pricing, with more than one extension the cost of Vonage increases drastically. At the time of writing no information could be found on how Vonage provides reliability. Access to critical business features such as conferencing requires an extra charge of $14.99/mo + $0.03/min. It should also be noted that Vonage supports only audio conferencing for up to 30 participants, whereas other providers can support video and a higher number of callers. According to Sipera Systems, a security vulnerability allows hackers to tap into Vonage lines [78].

Next Generation Systems

Skype for Business

Beginning on April 14, 2015, Microsoft Lync was replaced for a newer product, Skype for Business (SfB) [66]. The new service features a client with a Skype-inspired design. All the current Lync features are still available but with the addition of connectivity to the entire Skype network . The service continues with similar pricing model as with legacy service Lync (see Figure 2-10). As can be seen the ﬁgure for advanced features will cost $5.50 per user, per month. For a small company of 1000 people this translates to $66,000 per year not including the cost of devices. In addition, users are bound by a yearly contract.

61 Figure 2-10. Skype pricing

According to [79] SfB is a re-branded version of Lync 2013, with additional features and bug fixes. Notable features of the new Skype for Business are call monitor,Skype Meeting Broadcast, PSTN calling, and cloud PBX. Call monitor is a multi-tasking addition where a thumbnail video window and call controls pops up if you switch away from the SfB client window. Skype Meeting Broadcast will enable companies to hold conference calls with over 10,000 people. With Office 365, PSTN calling and Cloud PBX is available which gives the ability to make and receive phone calls and voice conferencing worldwide, with the ability to hold, resume, forward and transfer their calls. One of the biggest advantages of SfB is its integration with Office products [80]. This feature allows added collaboration, so that users can work from anywhere with other users worldwide [81]. Partnering with Polycom, the entire line of VVX devices is able to provide additional functionality over competitors. Some of the newer features are [82]: • Better Together over Ethernet (BToE)

62 – adds the following features: provisioning, click to call/forward/hang up/transfer/hold/resume, and to join a Lync Online Meeting. • Boss/Admin (Shared Line Appearances)

– adds the ability to make and receive calls on behalf of another user, and call status monitoring. • Lync Address Book Web Query (ABWQ)

– devices can now perform directory searches against the entire Lync Address Book by sending requests to the Lync Server’s Address Book Web Query service. • Lync Device Update Service

– updates are now able to be provided by SfB instead requiring any type separate provisioning server • Call Park

– Full support for parking and call retrieval. • Peer Video Calling

– The VVX 500 and 600 models have the ability to handle video directly on these devices The drawback to using SfB (non online) is its extensive licensing requirements. There are three main types of licenses: (1) server licenses, (2) client access licenses, and (3) external connector licenses [65, 83]. A server license is required for each instance of the Skype for Business server software. A client access license (CAL) is required for each user or device that will need access. A CAL can be either per user or per device depending on organizational needs. There are three different pricing schemes for CALs: (1) Standard CAL, (2) Enterprise CAL, and (3) Plus CAL. If a business wants outside partners or clients to access SfB then an “external connector license” is required. The external connector license has the same three pricing schemes as the previous CAL, however the prices are higher ($2,021 for Lync 2013). It should be noted that

63 Microsoft has not yet disclosed what the prices for the new Skype for Business features will be [84]. A security concern for users is that SfB offers no End-To-End Encryption, therefore there exists ability of snooping trafﬁc. Of importance is that in order to get full functionality on premise version and special license is required [85]. In order to gain SIP functionality, it is necessary to implement a session border controller (SBC) and gateway [86]. Other issues arise with SfB when using IP phones [87]. Major ones include: (1) Transferring calls requires up to 7-10 button presses, (2) when dialing/calling, presence information is not available until the call is connected [88], (3) a result of the previous is that a 2nd incoming call will "interrupt" any actions currently performing on the device, (4) call Consult transfer requires the user to not hangup till the transfer is complete. In mixed phone environments (differing brands) the ’Boss Admin’ feature does not allow you to place a call on hold from a Polycom device and retrieve the call from a different branded device [88] . For Polycom VVX series there is an issue where Contact Groups show up empty, and according to a Polycom representative the feature will not be ﬁxed since it is no longer supported [89]. SfB is also based on a Microsoft proprietary protocol and not standardized with Industry [90].

OnSIP

OnSIP offers cloud based geographically distributed SIP proxies. OnSIP PBX style centralized services similar to Skype for Business with a similarly tiered pricing model [91]. There is a Pay-As-You -Go Plan and a Per Seat Metered Plan. Both tiers are non-contractual and you have a 30-day trial period upon sign-up. The Pay-As-You-Go, is promoted for small call volume business, so you pay a minimum $49.95/month fee and can add features as needed. The Metered Plan costs $8.95 per person with a 5 person minimum. The Metered Plan has the same features as the Pay-As-You-Go but increased cost savings as more users are added. Additional monthly charges are incurred if your business requires more advanced features such as Auto Attendant,

64 music on hold, call parking, and busy lamp ﬁeld. With OnSIP you are charged for both inbound and outbound calls. All the additional calling rates can be seen in Table 2-9. However, SIP calls within your network are free. With OnSIP a business is not limited to use any speciﬁc phone brands or models. The OnSIP admin portal allows admins to have complete control. Any changes or additions to the customer’s phone system can be easily and quickly accomplished [92].

Table 2-9. OnSIP calling rates Inbound Outbound Standard Inbound Calls 2.9 ¢/minute Standard Outbound Calls 2.9 ¢/ minute Inbound Toll-Free 3.9 ¢/ minute Extended Calling Area Variable On-Network Calling Free On-Network Calling Free Phone Numbers $7 setup, $2/month after E911 $1.80 / month per seat Number Porting $15 / number Number Porting $15 / number 25 Simultaneous Calls per Number Unlimited Outbound Call Limit

OnSIP’s distributed proxies ensures enhanced reliability, since it removes the requirement to register to a ﬁxed location. Instead end devices register to the closest SIP proxy [91]. OnSIP deploys redundant databases, servers, routers, and Internet connections in order to ensure reliability [93]. OnSIP’s pricing is attractive to many small businesses, however due to the per minute usage costs, with high voice trafﬁc other alternatives providers may be a better option. Current popular feature set of OnSIP includes [94]: • Seamless, Integrated Video Conferencing

– A free to use client called ’OnSIP InstaPhone,’ uses Web Real-time Communi- cations (WebRTC) to offer video collaboration from within a web web browser without the need to install extra applications • CRM and Telephony Integration

– Customer Relations Management (CRM) integration provides the callee additional information, provided by Salesforce, such as why the caller is calling, or any information that can be of aid in helping the caller

– RingCentral for Salesforce App simpliﬁes employees jobs by taking call notes

65 • Browser/Email and Telephony Integration

– OnSIP Call Assistant for Google Chrome provides click-to-call and Enhanced Caller ID

– Shortel Sky Plug-In for Microsoft Outlook offers click-to-call functionality • Messaging Presence and Telephony Integration

– My.OnSIP app allows an employee to make calls via their desk phone from within a web browser and obtain presence information • Call Center and Telephony Integration

– OnSIP Smart Queues provides call queue visibility, one click transfers, and remote agent support • Custom On Hold & Telephony Integration

– Easy on Hold provides free selection of hold music [95] On July 07, 2015 OnSIP announced a new feature called OnSIP Queues, which allows your callers to watch a video while on hold [96]. The new Smart Queues also now allows remote agent login/logout, and the ability to schedule email reports [97]. According to OnSIP over 40,000 businesses around the world use their services [98]. As of July 23 Jul, 2015 OnSIP has a customer satisfaction rate of 98.8% [99].

2600hz

Another company of interest is 2600hz. Although not a business VoIP provider, 2600hz offers a customizable cloud based VoIP products and services called Kazoo, that other companies/persons can use to develop a business VoIP service [100]. They offer redundant business PBX services. An added feature of 2600hz is that customers are able to re-brand the product because of the open development nature in order to provide a customizable experience. They can provide a cloud based system or software only solutions that can be used on private systems. 2600hz still however promoted a centralized telecommunications model. Three carrier options for call routing are available (a) 2600hz carriers, (b) bring your own, and (c) mix & match. At a higher price

66 the 2600hz carriers are Tier 1 carriers, but with this you get more reliable service and support for connectivity issues by 2600hz. With the “bring you own” option you select your own provider and manage all aspects of that relationship. Mix and match provides the benefits of the other two options in that you can select to use 2600hz for some services and your own carrier for others. 2600hz has developed a distributed telecommunications solution by using Kamailio as a session border controller (SBC) and FreeSWITCH as media server [101]. In this scheme registrations are processed by Kamailio and then to uses propriety AMQP protocol to send messages. AMQP is a secure messaging protocol that allows signaling messages to be transported between different entities over varying IP spaces [102]. With the switch to this new architecture the system saw a 25x improvement in registration handle versus when only FreeSWITCH was used [101]. Kazoo was designed to be “hardware agnostic”, meaning that their solution does not require any proprietary hardware [103]. They use a software-defined networking (SDN) solution that operators can manage on-demand.The Kazoo infrastructure is divided into several pieces: • Session border controller • Media server • Application server • Database • Operational and Billing Support Systems (OSS/BSS) The first two pieces have been discussed previously so the following will explain the other components. The application servers provide the core system functions such as voicemail,SIP trunking, and other call related features. The database is a meshed design that is replicated in different physical locations. OSS or operational support systems offers technicians a visible-clickable on-screen interface from which they can

67 a view and respond to arising issues. Billing Support System (BSS) provide integrated billing solutions that operators can access real time [103]. Key advantages of 2600hz are [100]: • Comprehensive Training • Competitive Pricing • True Support • Customizable Solutions • Scalability • Mobility Kazoo will soon be releasing an updated version of their application called SmartPBX. SmartPBX provides a graphical interface that allows admins to easily manage their telecommunication system. The main screen displays an overview of the system such as number of users, types of attached devices and conference bridges. Features include user creation, create extensions, and DID management [104]. It should be noted according to their webpage [105], 2600hz hasn’t released any major new features since 2014 and have only made improvements on their existing offerings.

Summary

IP telecommunications technology has undergone tremendous growth and development in the last decade. However, there is still a demand for a fully distributed and converged (voice, video, data, and text) system for IP-based telecommunications. System performance is a still a major hurdle for IP telecommunications, because all work thus far has been focused on theoretical systems or voice only systems, the prototype D-IPTS was modeled as an M/G/1 and this model was validated with actual system testing. The D-IPTS presented in this dissertation provides a IP telecommunication system design and implementation that addresses the performance, availability, scalability, and low-cost design of fully functional telecommunications system. The priors

68 works predominately evaluated proprietary systems, private networks, and provided data based on theoretical models and simulations. The D-IPTS system as designed operates on the public Internet, and also functions in both a local area network (LAN) and on mobile networks. Reliability is another important system criteria that needs to be further addressed. Most reliability solutions that have been proposed include the use of additional software and components to perform management functions. In the D-IPTS reliability is improved by using DNS routing as this requires no extra routing by the SPS or additional software and hardware.

69 CHAPTER 3 DETAILED STUDY OF SESSION INITIATION PROTOCOL AND REAL TIME PROTOCOL

Session Initiation Protocol

The Session Initiation Protocol (SIP) is one of the most popular protocols used for setting up IP telecommunications calls and is the core of the D-IPTS. It is responsible for initializing, modifying and tearing down communication sessions [17]. The addressing for SIP is based on the Uniform Resource Identifiers (URI) and the underlying SIP message allows a variety of information types to be transmitted. It may contain messages from other protocols such as Real Time Protocol (RTP), Session Description Protocol (SDP), Resource Reservation Protocol (RSVP), and Real Time Streaming Protocol (RTSP). SIP is responsible for: a) call initiations, transfers, holds, and session termination, b) determining the location of the endpoint to be used based on the URI, with the help of a DNS server and intermediary proxies, c) determine caller presence information prior to the flow of information/media. A SIP server accepts requests from a User Agent Client (UAC) and transmits call responses. The server may also act as a proxy server, in which case it can forward requests to another server on behalf of a client. The server also functions as a registrar, accepting REGISTER requests and checking if the UAC is authorized to register with the network. The SIP Proxy server forms a triangular topology with the user agent server and client as shown in Figure 3-1[ 17]. The proxy server receives requests from the UAC and decides where to forward that request. It may either forward it to a User Agent Server (UAS) or to another proxy. The response also follows the same path in reverse. If the server finds multiple destinations for the requests, it will fork the request and send it to all of them.

70 Figure 3-1. SIP triangular topology

The SIP protocol architecture is shown in Figure 3-2. The request contains four header sections, and the one of interest is the SIP header. The SIP header is comprised of three fields: the Request-Line, Message Header, and the Message Body. The Request-Line includes the request type and the destination address, and contained in the Message Header is the Via, From, To, Contact, and the Allow fields. The ’Via’ field contains the SIP version number and every node in the request path adds its IP address and port, and the ’From’ header field indicates the originator of the request. The ’To’ header field indicates the desired recipient of the request; the ’Contact’ header field where to send subsequent requests; and the ’Allow’ field lists the SIP Methods that the caller can support. The Message Body contains the SDP, and will be described in Section3. The basic SIP protocol [106, 107] uses six message types, which handle all functionality. These are:

71 Figure 3-2. SIP protocol architecture

• INVITE: This is a request sent from a UAC to establish a call with another UA. It includes the parameters and description of the media in the SDP format. • ACK: Used to acknowledge the receipt of the response to an INVITE message. It is the last message sent during the initial establishment of the call. • OPTIONS: Used to query the capabilities of another user agent, i.e codecs, content types, messages supported, etc. • BYE: Sent when a user agent wishes to terminate the call. • CANCEL: Used to cancel a pending request. • REGISTER: Used to register to a SIP server as explained earlier.

The six message types result in one of the response codes given in Table 3-1[17].

72 Table 3-1. SIP response codes Response Code Name Meaning

1xx Provisional Request received, processing 2xx Success Action successful (acts as ACK) 3xx Redirection Further action required 4xx Client Error Current server cannot process 5xx Server Error Server failed to process request 6xx Global Failure No server can process request

All the requests except the BYE request, have a message body that is described by the SDP protocol.

SDP Signaling Protocol

The Session Description Protocol (SDP) is an application layer protocol used for the negotiation of parameters for multimedia communications between endpoints such as media type, format, and other associated parameters [18]. The SDP message includes the session name and purpose, session time, and media info. The media info consists of the media type (audio/video), transport protocol (RTP, UDP, etc), media format (H.261 video, G.711, etc. ), and the media transport port. Figure 3-3 is a SIP INVITE request with an SDP description deﬁning the destination endpoint’s IP address “10.0.0.98” and the port “7076” on which it will receive an RTP audio stream. The media description section shows the codecs the source endpoint is offering, and the destination side will select one codec that will be used for voice transcoding.

73 Figure 3-3. SDP message for SIP INVITE

Real-Time Protocol

SIP is not designed for media transport, and therefore needs have other means of relaying media. However, once a session is established, the Real-Time Transport Protocol (RTP), is used to encapsulate media into UDP datagrams and then to IP packets [19]. The RTP packet contains: the Payload-type Id, which identiﬁes the payload as audio/video and the associated codec; the Sequence Numbering, which is used to correctly order the packets or detect packet loss; and Time stamping, which tells the time the packet was sampled. RTP works in conjunction with the RTP Control Protocol (RTCP) to perform delivery monitoring and stream synchronization. The information sent by RTCP is used for

74 diagnostics and modifying transmission rates [19]. Each RTP media session consists of IP addresses, the ports, and the codecs that will be used. Once packets are received they are not immediately decoded, instead they are ﬁrst put into the “Jitter buffer”, and transmitted with a short delay. On the receiver packets are reordered into their correct sequence. The RTP protocol stack can be seen in Figure 3-4.

Figure 3-4. RTP protocol stack

SIP and RTP Security

SIP is open to all Internet related attacks due to SIP entities residing on the Internet or access to SIP entities via communications ports. This section will discuss some of the security threats to SIP. Some of the threats to SIP are [90]: call hijacking, where calls to a destination URI gets redirected to a different user; registration hijacking, where the attacker fakes the registration information of a user, so all incoming traffic is routed to the attacker; Eavesdropping, which can be done on signaling or media; and denial of service, which occurs when the SIP entity becomes unavailable and is usually caused by flooding the network with packets. It was not the objective of this dissertation to explore security threats, but possible solutions to these threats are digest authentication, Transport Layer Security (TLS) encryption, Secure RTP, and IP flood banning.

75 CHAPTER 4 DISTRIBUTED IP TELECOMMUNICATION SYSTEM (D-IPTS) A fully functional distributed IP-based Telecommunication System was designed and implemented. The D-IPTS provides a converged distributed system that provides unified communications including voice, text, and video routed to a single device while on-the-go, with “follow-me” services, for home and office calls (see Figure 4-1). It also includes the common features of a traditional office PBX, namely music-on- hold, voicemail with email support, automated attendant, and conference calls. The combination of directory services and “presence,” allows users to publish their own availability status and willingness to connect. This system uses a highly reliable and well-designed LAN infrastructure as the backbone of the network. Instead of using centralized, expensive equipment and components at the core to drive the entire system, this design uses many inexpensive, replaceable components that can be upgraded, replaced, or removed without disrupting the operation of the active D-IPTS system. The D-IPTS system has been created in a manner that is flexible and highly suits the design of a distributed IP telecommunications system, and as a result, it is the first step in achieving a single device for all communication needs.

Figure 4-1. Feature set of D-IPTS

76 Components of D-IPTS Testbed

A fully functional D-IPTS consists of several components responsible for various aspects of the communication. A generic overview of each of these components and the role they play in the system is now given. This is followed by a detailed description of the actual components used and their roles in the system. • SIP Proxy Server: This is the central component of the system. It is responsible for registering users and maintaining a user database. It also takes care of setting up and tearing down of IP telecommunication connections. It does not handle the actual multimedia trafﬁc. • Media Server: A media server is required to deal with functions that require interactive media communication between the D-IPTS system and the user agents. • PSTN Gateway: The PSTN gateway enables VoIP subscribers to make calls to PSTN subscribers. • IPTS Gateway: The IPTS gateway enables D-IPTS subscribers to interconnect to legacy PBX systems. • ITSP Gateway: The ITSP gateway enables D-IPTS subscribers to connect to Internet telephone service providers. • 4G Gateway: The 4G gateway allows D-IPTS subscribers to connect to the system on mobile devices. • User Administration and Provision Portal: The administration portal allows users to manage their subscription, and the administrator to regulate the privileges of subscribers. • Media Proxy: Directing outgoing calls to the network of the VoIP provider requires the resolving of certain issues. One of the main concerns is NAT traversal. • Monitoring Tools: These tools are required to debug any problem in the SIP server. These may include a protocol analyzer, and packet snifﬁng tools like ngrep,

77 Wireshark, tcpdump and ethereal. Kamailio has a module called SIP Trace, which is also useful in detecting errors in the SIP server’s operation. The D-IPTS was designed using Kamailio, a SIP router and FreeSWITCH, which acts as the media server to provide extra features such as music-on-hold, voicemail, automated attendant, conference calling, etc (see Figure 4-2). Alpine Linux is the operating system used for implementing the Kamailio and FreeSWITCH since it is highly suitable for running IP-based telecommunication systems, because of its security, speed and run-from-RAM capabilities. The individual components of the system are described in more detail in the following sections. RTPproxy will be discussed later in Section4.

Figure 4-2. Components of D-IPTS Testbed

Alpine Linux

The D-IPTS was built on the Alpine Linux platform, which is specially designed for routers, ﬁrewalls, Virtual Private Networks (VPN), and VOIP [108]. The Alpine OS was installed and executed via a USB drive. Alpine Linux is intended for an experienced Linux user and allows full control over the system. Alpine Linux uses PaX security, which implements least privilege protection for memory access. This creates a highly secure kernel that is difﬁcult to penetrate, thereby protecting it from bugs and security attacks [109]. Alpine Linux requires a minimum of 4-5 MB of space, excluding the kernel, compared to gigabytes for most other operating systems. It was operated from RAM,

78 which reduced the risk of hard disk failure, and data was backed up using the Linux Backup Utility (LBU). All the configuration information can be saved in a single file, which can be used to restore the system to the previously saved configuration. These configuration files can also be copied to any compatible Alpine Linux system, and on any USB drive, resulting in an OS that is highly portable. A very useful feature is the Alpine Configuration Framework (ACF), which is a web configuration utility that provides a GUI for configuring nearly all the system settings.

Kamailio

Kamailio is the open source SIP router that was used in the D-IPTS. It handles thousands of calls per second even on low-budget hardware and acts as the registrar. Kamailio supports IP versions 4 and 6, as well as TCP, UDP, Transport Layer Security (TLS), and Stream Control Transmission Protocol (SCTP) [110]. Kamailio has been written entirely in the C language and also supports other languages such as Lua and Python. New modules for any specific purpose can be easily written using the C language, which makes it extremely portable and expandable [110]. Kamailio is capable of operating in stateless and stateful modes, and provides NAT traversal support for SIP and RTP traffic. It also has features for routing failover and replication for high availability, which was well suited for the D-IPTS. The accounting of the calls through Kamailio is event based and the parameters of the accounting are easily configurable. The user database, which stores all the necessary information about the subscribers, was maintained by PostgreSQL database management software. The fully functional D-IPTS was achieved by integrating Kamailio with Media Servers and PSTN gateways. These are some of the important characteristics that made that feasible: • Scalability: When operated on systems with 4GB memory, it serves approximately 300,000 online subscribers and in stateless mode it achieves rates of over 5000 call setups per second [110].

79 • Architecture: Kamailio consists of a core logic, which handles the functionality of a SIP server and Registrar. The majority of functions are handled by modules, which are software coded to meet new and existing functional needs. • Configuration: The Kamailio.cfg configuration file is the backbone of the routing for the entire D-IPTS system. All of the parameters, system settings, and routing criteria are in the configuration file. The file allows complete system administrative control to access and edit parameters along with defining SIP protocol actions in real time. Kamailio is the most significant component of the D-IPTS and it performs the role of the “core” of the network. Like backbone routers, the D-IPTS allows for multiple routers (pathways to UAC’s) providing high availability in case of component failures. Kamailio performs the duties of the SIP registrar and verifies the authenticated users based on the PostgreSQL database. Call origination and setup was established by Kamailio server, which then determined the type of call requested and performs authentication checks. Based on the type of call, the routing decision was then made and calls were immediately routed to the destination. In normal operation, Kamailio ran in the stateless mode meaning that once calls are established, Kamailio is removed from the communication loop until the call is being terminated. Removing Kamailio improves D-IPTS system performance by reducing the load on the SIP router, and allows it to support more users simultaneously.

FreeSWITCH

FreeSWITCH is an open source IP telecommunications platform that is highly scalable and works with a comprehensive range of communication protocols. It can operate as a simple switching engine, a PBX, a media gateway, or a media server to host interactive voice response (IVR) applications [111]. It makes complicated functions like voicemail and music-on-hold fairly straight forward to incorporate into the

80 telecommunication system. In the D-IPTS, FreeSWITCH functions as a media server and an SBC. The design of FreeSWITCH is similarly based on a stable central core with modules for specific functionality. The core provides interfaces to these modules, allowing the developers to manage the system efficiently, and if needed, add their own modules conveniently. The various modules that operate on the core are completely independent of each other. The most important modules are the Endpoint modules, which provide an interface to one of the supported communication protocols. The connection between FreeSWITCH and the SIP protocol is known as a session, and the parameters and settings are defined in the XML configuration files. The basic operation of FreeSWITCH is the following. First, a SIP phone sends call setup message to the SIP module. The call is then passed to the core state machine. This brings FreeSWITCH in the routing state, which then looks for a module called the Dial-plan, which is an XML file describing call specific actions for each scenario, known as an extension. The Dial-plan now builds a task list for the current call and inserts instructions into the session object. FreeSWITCH then goes into the execute state, sequentially implementing the tasks described in the Dial-plan. A task list is created by the Dial-plan in the form of the application name and the arguments it requires. The applications listen for these tasks and executes the ones intended for them. The configuration files are written in a modular fashion, where separate XML files are used for various purposes. The files can then be included in the central configuration file, where they are processed. This is an important feature when creating user accounts, because each user account can be created as a separate file, therefore making it extremely convenient to manage the privileges of the individual users. The users may also be grouped together and have a common account file. A session border controller (SBC) is located at the logical boundary of two networks, and controls the communication between them. It resolves compatibility

81 issues arising as a result of differences in the administration and protocols of the bordering networks, providing interoperability despite the mismatch. It can also provide security features, such as regulating traffic entering the network and shielding its topology from surrounding networks. FreeSWITCH is the SBC for the D-IPTS phone system. FreeSWITCH in this design serves as the network shield for the entire D-IPTS architecture and also can resolve NAT traversal issues. FreeSWITCH was used to access the PSTN network and allows authorized external networks to communicate with the D-IPTS, as illustrated in Figure 4-3. In this topology, FreeSWITCH is seen as a single interface by external networks, and shields the SIP proxy and the rest of the topology, resulting in a more secure D-IPTS system that is less vulnerable to attacks. FreeSWITCH also acts as a Back-to-back User Agent (B2BUA), where it proxies SIP requests. The difference between FreeSWITCH and Kamailio is that all the data, including the media streams, passes through FreeSWITCH and therefore it stays in the loop of the communication for the entire duration of the call (see Figure 4-4). Some of the other functions an SBC can perform are as follows [112]: • Network topology hiding • Protection from intentional and unintentional flooding of a network • Protection from unauthorized access to a network • Resolving Network Address Translation (NAT) issues • Protocol conversion • Transcoding of media traffic • Translation of phone number formats • Providing additional QoS support

82 Figure 4-3. FreeSWITCH as SBC

Figure 4-4. FreeSWITCH as B2BUA

Other Components

The D-IPTS is comprised of several other components which include a web- interface, a DNS server, a DHCP server, and a provisioning server.

Web interface

The D-IPTS system also provides a web interface for common conﬁguration changes as well as an interface for provisioning of IP telephones (see Figure 4-5). The web interface was designed to give remote secure access for administrative functions. The D-IPTS system health can be monitored to ensure it has the necessary resources

83 available. The web interface also allows for remote system software upgrades and management, and user account creation.

Figure 4-5. D-IPTS web interface

DNS server

In order to perform DNS failover, a DNS server was created in order to resolve IP addresses. TinyDNS was used in the D-IPTS in order to create a domain. TinyDNS is a DNS server program. The following is the zone ﬁle, which identiﬁes all the IP addresses and domain names associated with the D-IPTS system (Figure 4-6):

Figure 4-6. D-IPTS DNS zone ﬁle

84 Line 1 and 2 starting with “.” are the “Address records” (A) for the domain name of the system. Lines 3 and 4 are used to resolve the canonical name to an IP address of the service. Lines 5-6 are pointers from the domain name to the IP address. The following lines, 7-9, identify the mail, text, and FTP servers. Lines 11 and 12 are the “Name Authority Pointer” (NAPTR) records with preferences (lower is more preferred). NAPTR record defines that the preferred method for SIP service is UDP. Lines 14-16 consists of “Service records” (SRV), which specify where to route SIP traffic. Each line has a priority and weight field. The priority field is used for failover and the weight field is for load balancing. The server with a lower priority number is used as the main route for traffic and in the event the main cannot be reached, traffic is automatically routed to the next server. Servers with the same priority can be given different weights which identifies the percentage of traffic that is sent to the server. These SRV record lines are critical to the D-IPTS design because load balancing and server failover affect the overall performance and reliability.

DHCP server

The D-IPTS uses the Dynamic Host Configuration Protocol (DHCP) to automatically assign network information to connected devices [113]. Devices make requests to the DHCP server and the server assigns an IP address. The IP address is “leased” to the device for a specified time period. The D-IPTS DHCP server was designed to also support “option 66”. This feature specifies the Trivial File Transfer Protocol (TFTP) server where IP phones can download/upload configuration information. This important feature will be discussed further in Section4. The D-IPTS DHCP file is shown in Figure 4-7 where each lines purpose is described below:

1. Declares the domain name of the server

2. Sets the DNS update style to none, which means clients cannot modify the DNS

3. List of DNS servers

5. TFTP server address

85 8. Speciﬁes IP address lease time

9. Max time a client can lease an IP address

10. Declares that this is only DHCP server on the network

11. Subnet declaration

12. Range within which clients are assigned an IP address

13. Netmask declaration

14. Default broadcast address to be used by clients

15. IP address of default router

18. Time server IP address for clients to use

19. Time offset (in secconds) is used to specify time zone (Eastern)

Figure 4-7. D-IPTS DHCP conﬁguration

Provisioning server

The provisioning system was used to simplify the conﬁguration of IP phones (Poly- com 335s were used). The web interface of the D-IPTS system allows administrative access for rapid deployment of phones. To provision a user, the admin enters the phones media access control address (MAC), phone extension, and user password (see Figure 4-8). The database is then populated with this information and the accounts

86 created on the SIP registrar. A conﬁguration ﬁle with all the account information is stored on the provisioning server, which is retrievable by the IP phone.

Figure 4-8. Phone provisioning using web interface

When the IP phone is connected to the network it receives its information from the DHCP server. Included in this information is the location of the provisioning server. The IP phone downloads its conﬁguration ﬁle by utilizing DHCP option 66 (see Figure 4-9).

Figure 4-9. Automatic provisioning of phones

D-IPTS Testbed

The D-IPTS offers a novel approach to IP telecommunications; it provides all the features present in existing IP PBXs, but at a much lower cost. Moreover running it in

87 RAM makes it highly reliable. The USB disks running the system can be plugged into any computer with the appropriate amount of RAM and an internet connection, resulting in the same functionality. The USB is made read-only during normal operation, which leads to a highly protected tamper-proof system. Duplicating the USB disks is possible using the Linux Backup Utility (LBU), a feature that is beneficial for backing up the system as well as expanding the system. The prototype system is Internet-inspired, i.e., the intelligence of the network lies at the terminal equipment. The server does not impose any form of features on the phones, which allows the user flexibility to tweak system parameters by means of their SIP phones, irrespective of the type of end system. It is a distributed type of system, in the sense that the SIP server is not in the communication loop at all times and it is only involved in the setting up of the call. The media does not traverse through the SIP server, it goes directly from one end device to the other, which reduces the traffic flow through the server making it much more efficient. As a result, the number of dropped calls is reduced and also allows the server to support more calls simultaneously.

Figure 4-10. D-IPTS System architecture

The distributed and end system oriented design of the D-IPTS phone system also gives it structure, which centralized servers lack. Different functions are divided in an organized manner across the system, instead of one server doing it all. For example,

88 Kamailio is responsible for the call set up and FreeSWITCH handles the media functions and also acts as the Back to Back User Agent (B2BUA). IP phones can handle call transfers and call forwarding. They can also control parameters of other functions such as the ringer timeout. The structure of the system is depicted in Figure 4-10, in contrast to traditional phone systems. As can be seen, the system will connect to the PSTN network as well and can support Direct Inward Dialing (DID). External calls to any PSTN number is accomplished by redirecting the call to FreeSWITCH and using it as a B2BUA. SIP phones used in the system can automatically self-provision by downloading configuration files from the provisioning server. Additionally, the phones can handle call transfers and allow the user to modify any of their settings. The phone will then inform the provisioning server of these changes by uploading the new configuration. The D-IPTS system was designed to have interconnection with all current telecommunication networks by using gateways. The D-IPTS is able to connect to legacy PBX system by utilizing IPTS gateways, PSTN gateways allow D-IPTS users to call POTS telephones, 4G gateway allows mobile users to connect, and an ITSP gateway allows the system to connect to ITSPs. This web of interconnected networks also gives D-IPTS users mobility, which allows calls to be handed off when users traverse the different networks. The system provides several extra functions apart from basic voice calls. The functions include:

89 • Internal Calls • External Calls • Music on Hold • Voicemail (with email support) • Conference Calls • Automated Attendant • Video • SMS • Presence • Automatic provisioning The distributed nature of the system means that each of the functional elements will be run on separate machines (see Figure 4-11). This helps in that if a hardware failure occurs on one machine, then the system will not lose all functionality. The ﬂexibility of the D-IPTS distributed design also allows the possibility of the servers being geographically distributed. Also it can have warm replacements, gracefully expanding the system by adding servers running services as needed.

Figure 4-11. Topology of the D-IPTS

NATs Solution and Mobility

RTPproxy and NAT traversal

The D-IPTS system created currently resides in a private network. This creates a NAT traversal issue because devices can only convert IP addresses at Layer 3 and not on the Application layer, where IP telecommunications resides [112]. In this scenario

90 when calls are connected, the audio data was unable to traverse the network (no sound). RTPproxy resolved this well-known NAT issue by changing the private IP address of the SIP devices to the D-IPTS public IP address. RTPproxy must be used to enable IP telecommunications to user agents (UA) behind NATs [114]. RTPproxy had to be conﬁgured within Kamailio with the Unix domain socket and the UDP port range at start-up. Once these were enabled, Kamailio replaces the “media ip:port” in the SDP for an INVITE request and forwards the request to the destination. Once the session is established, RTPproxy listens on the designated port for a UDP packet from each user. RTPproxy replaces the ip:port structures associated with each call with the source ip:port of that packet [115]. At this point, media packets can be communicated freely (see Figure 4-12).

Figure 4-12. RTPproxy used to resolve NAT issues in D-IPTS

Mobility

The D-IPTS was implemented with advanced features and routing algorithms that allow for user mobility (see Figure 4-13). When a user registers, their location information is stored in the location server. When the user changes locations, a new REGISTER message is generated which updates the location information in the server. Once this happens, all future calls are routed based on this location. Intelligent routing of calls is possible due to the D-IPTS use of IP-based call routing. In the event that a user is on a call while changing locations, there is a seamless hand-off to the new network

91 location and the call remains connected. Only the voice packets that are in transit at the moment of hand-off are lost.

Figure 4-13. User mobility in the D-IPTS

D-IPTS Testbed Implementation

Call Flow

The D-IPTS call flow chart is illustrated in Figure 4-14. Before a user can make a call, he/she must first register to the D-IPTS domain. The D-IPTS system has a DNS server that directs SIP traffic to the primary SIP Proxy. If the primary proxy is not busy or down, then the call is processed by the system. The SIP proxy checks the call URI to determine whether the call is bound for an internal or external extension. If the call is an internal extension, the proxy routes the call to the proper destination, then there are two possible scenarios . First, if the call is connected (user answers), then the call is completed. The second is if the user rejects or the call is unanswered, then the call is routed to FreeSWITCH’s voicemail application. Other functionality is available via the SIP proxy while calls are in progress, such as routing a call placed on hold to the media server for “music on hold”. If the primary proxy server is down, the DNS server routes the traffic to the secondary SIP proxy that has redundant functionality of the primary so calls are handled

92 in the same way. If both the primary and the secondary servers are down, which is highly unlikely, calls are routed to FreeSWITCH voicemail.

Figure 4-14. D-IPTS call ﬂow

Call Processing

Users must be fully authenticated before using the D-IPTS system. When a user dials an internal or external number, the SIP-enabled phone generates an INVITE message with the appropriate destination URI, which is then sent to Kamailio. The message is checked for corrupt or illegal ﬁelds, message size, and the max number of hops. If the message fails a check it gets rejected. Based on the destination URI, Kamailio takes the following actions if the message passes the checks:

93 • Internal Calls: An internal call is defined as a call whose source and destination are both registered to the same SIP server. These calls are handled only by Kamailio. Once the initial processing of the INVITE message is completed, the INVITE message is then relayed to the destination extension. The SIP-enabled telephone at the destination receives the INVITE message, and replies with a 200 OK message, signaling that the call has been answered. Once the calling SIP phone receives the 200 OK response, it replies with an ACK, and the call is then established. A BYE message is generated and relayed to the other participant through the Kamailio server, when either participant hangs up. The call ends when the BYE message is acknowledged. • External Calls: An external call is when the user dials a number outside the D-IPTS network. These calls are handled by Kamailio and FreeSWITCH, and require a VoIP account with a service provider. When a user dials a “9” before the telephone number, Kamailio identifies that it is an external call and redirects the call to FreeSWITCH for proper routing. Located within the FreeSWITCH configuration files are the credentials of the VoIP account that will allow a user to make the external call. The call is bridged to the VoIP service provider’s server using the FreeSWITCH mod_sofia module. FreeSWITCH remains in the communication loop, acting as a B2BUA. • Music on Hold: The D-IPTS system provides music on hold (MOH) capability when a caller places another participant on hold. Streamed MOH requires a media proxy, which is FreeSWITCH. The MOH extension is defined in the FreeSWITCH dial-plan. When a call is placed on hold, Kamailio redirects the call to the MOH extension. • Voicemail: In the D-IPTS system if a user does not answer a call, it is redirected to voicemail, where a message can be recorded. Every registered user requires a FreeSWITCH voicemail account to be able to receive messages. This can be

94 done using Alpine Linux’s web configuration utility GUI or by creating individual configuration files for each of the users in the FreeSWITCH directory. There is an entry for the voicemail extension in the FreeSWITCH dial-plan that uses the voicemail application. The D-IPTS also has additional capability to forward voicemail messages to an e-mail address if needed. • Conference Calls: The D-IPTS system conference call feature allows more than two users to talk to each other simultaneously. In the D-IPTS system the 30XX extension numbers are reserved for conference calls. So users initiate a conference call by dialing any 30XX extension. The FreeSWITCH application called conference handles these 30XX extensions. Conference calls follow the same processing as internal calls as discussed above. • Automated Attendant: The D-IPTS system has an Interactive Voice Response (IVR) system capability. This allows callers to interact with the D-IPTS system via a telephone keypad. This is used to redirect incoming calls without an operator. The call is redirected through FreeSWITCH in a manner similar to the conference calls.

Preliminary Tests

Preliminary Tests of the D-IPTS were done using students in an undergraduate communications course. There were 63 students that participated in the testing. The students were asked to test the systems voice, video chat, SMS, and conferencing capabilities. During a two week period, the students were able to use the system and were required to keep a log of their experience. Any connection issues were to be noted along with voice quality using MOS. Results of the test are as follows: • Voice Quality

– MOS Average=3.7 • 95% Availability (End-to-End )

95 The results from this test were very promising. The voice quality is determined by codec selection and for these tests the codec choices were random. Voice quality of the D-IPTS can be further improved by using a higher quality codec such as G722. The 95% availability of the system was an end-to-end availability. The students were able to test the system on various networks (WiFi, Cellular, and PSTN) and end-to-end availability is not guaranteed. The actual availability of the D-IPTS excluding external networks was 100%.

High Availability D-IPTS

The reliability of the D-IPTS system was increased by adding features for high availability (HA). The HA features of the D-IPTS were implemented with redundant proxy servers (as shown in Figure 4-15a). The master and backup server are active at all times to reduce failover times. The servers are managed by DNS records to route SIP traffic to one of the proxy servers. There is a single public IP shared between the servers, and the DNS records were set to prioritize the master server so that all traffic will be directed there. Traffic will be automatically routed to the backup in the event of a master server failure. This redundant failover design requires no action by users, administrators, or any additional hardware. This alternative software-based solution has significantly lower cost compared to traditional PSTN/PBX hardware based solutions (see Figure 4-15b).

96 (a) D-IPTS HA (b) PSTN/PBX HA Figure 4-15. High availability design. A) D-IPTS HA. B) PSTN/PBX-HA.

D-IPTS database redundancy

The D-IPTS system has increased availability by having a secondary PostgreSQL user database. PostgreSQL has a very useful function called Streaming Replication, which continuously backs up the database files and duplicates it to the secondary PostgreSQL database (see Figure 4-16). Streaming Replication was configured and enabled on the D-IPTS by modification of the PostgreSQL configuration. Each of the servers were enabled to have the ability to communicate via IP. This enhances availability by allowing the servers to be location independent. SSH-key only access was also configured on the servers to increase security.

Figure 4-16. D-IPTS database redundancy

97 D-IPTS DNS failover

DNS server records perform the fail-over and load balancing in the D-IPTS. Using DNS records allows multiple SIP proxies to be active simultaneously without conflicting. The failover feature is handled by setting different priorities for each proxy. The main SIP Proxy (1) is given a lower value as this directs all the SIP traffic to its IP. The other SIP Proxy (2) is given a higher value. If a failure of (1) occurs, traffic is automatically sent to the other proxy (2). This is an elegant solution as it requires no additional processing to be done by the individual proxies. Load balancing is done in a similar way, except that the “weight” field of the SRV record is used. The weight field allows the ability to define what percent of traffic to be directed to individual IP addresses. This method of failover is preferred as it does not require the changing of IP addresses. This method also does not require additional hardware or software to be implemented into the system design.

System Conﬁgurations

The D-IPTS was designed and implemented in two different platforms, Linux Containers (LXC) and multiple servers.

Multiple server

The distributed nature of the D-IPTS system means that it is possible for each of the functional elements (SIP server, media server, DHCP/DNS server, and provisioning server) to be run on separate machines. Four servers were used and Alpine Linux and the associated software for each element was installed on a USB drive (see Figure 4- 17). For each system it was necessary to modify the software code with IP addresses to communicate to the other elements. It was also necessary to modify the Linux operating system to allow for incoming and outgoing connections.

LXC

LXC is a virtualization software for Linux-based computers and it is a user- space interface for the Linux kernel containment [116]. Each virtualized environment

98 Figure 4-17. D-IPTS multiple server implementation

“container” has a separate process and network space. Containers appear as system processes on the host system and offer the same features as virtual machines (VM ), with lower (minimal) overhead [117]. Unlike VMs, containers do not require the installation of separate operating systems (see Figure 4-18) or the inherently long boot times of these installations.

Figure 4-18. Virtual machine vs. container

The D-IPTS was built using LXC virtualization software, see Appendix D. Figure 4-19 shows the basic concept of the design. In the ﬁgure each container served as the host for D-IPTS functionality. Each block represents one of the major components of the D-IPTS system. There are containers for the SIP proxy (Kamailio), media server (FreeSWITCH), DHCP, DNS, and provisioning. This created a completely isolated environment as if they were run on separate machines. All functionality was retained but was able to be delivered in a smaller footprint.

99 Figure 4-19. D-IPTS LXC implementation

Summary

The D-IPTS system was designed using Kamailio as the proxy server and FreeSWITCH as the media server. In addition in the D-IPTS system design, Kamailio and FreeSWITCH are used in conjunction to provide a full featured system where in the others they are used as standalone products (each as a PBX). The D-IPTS provides interactive multimedia communication of voice, text, email, video, as well as mobility. A web-interface was designed to provide a graphical interface for system conﬁgurations and user provisioning. A NAT traversal solution was implemented by using RTPproxy. A DNS server and DHCP server were also designed and implemented on the system. The DHCP server provides option 66 to IP phones and automatic provisioning is performed by a Python script. Two implementations of the D-IPTS were accomplished by using a multiple server conﬁguration and a smaller footprint LXC based design. The high availability of the D-IPTS was accomplished by redundant servers and DNS routing and fail-over.

100 CHAPTER 5 TESTBED PERFORMANCE MODEL DEVELOPMENT AND CAPACITY STUDY The design and implementation of the prototype D-IPTS successfully achieved the primary objective of a converged distributed system that provides unified communications including voice calls, text, email, and video routed to a single device while on-the-go, with “follow-me” services, for home and office calls. The secondary objective was to evaluate the performance and availability using the D-IPTS as a testbed and also diagnose failure modes of the D-IPTS. The results could be used to re-optimize the D-IPTS system. A flow chart for developing reliable communication systems that was utilized for the D-IPTS is shown in Figure 5-1[ 49] and a process flow chart for evaluating system’s reliability is shown in Figure 5-2[29].

Figure 5-1. D-IPTS system evaluation methodology: Relations of model types

Figure 5-2. D-IPTS system evaluation methodology: Reliability evaluation ﬂow chart

101 Testbed Design

The performance and availability analysis of the D-IPTS system was done using the multiple server D-IPTS system (see Figure 5-3). It must be noted that is the ﬁrst and only comprehensive empirical and theoretical analysis of a USB-drive IP telecommunications system implementation. To perform the experiments and analysis of the fully functional D-IPTS system, low-cost open source software was utilized. Kamailio was used as the router, registrar, and location server. FreeSWITCH was used as the media server a session border controller (SBC). Alpine Linux is the operating system (OS) on which the D-IPTS was installed. Two separate x86 systems were used, an AMD and a Intel Core(TM)2 Duo CPU T6600 (see Table 5-1). As shown in Figure 5-4, the SIP router and media servers are physically located on different machines. To minimize experimental variability, all of the measurements were conducted on a private network, which eliminated the effects of external network trafﬁc. SIPp was installed on a machine running Alpine Linux 2.64 with 4 gigabytes of RAM and a 2.0 GHz Intel Xeon CPU. The D-IPTS was built on the Alpine Linux 2.64 operating system.

Figure 5-3. D-IPTS System architecture

Table 5-1. Test computer systems Processor Speed (MHz) Memory (GB) Cores AMD AMD A8-5600K CPU with Radeon(tm) HD Graphics 3710.857 3.0 4 Intel Core(TM)2 Duo CPU T6600 2001.000 4.0 2

102 Figure 5-4. Distributed IPTS test conﬁguration

The SIPp traffic generator was used to generate voice and data traffic. Customized SIPp test case XML files were created to implement the D-IPTS authenticated user access. A sample call flow for the XML file is shown in Figure 5-5.

Figure 5-5. Register-Invite client

In order to measure the CPU loading information of the different conﬁgurations two programs were used to monitor the system, top and htop. The ’top’ program shows the CPU load information for each process that is currently running (see Figure 5-6). The

103 ’htop’ program provides CPU load information and additionally provides process memory usage and uses a graphical interface ( Figure 5-7).

Figure 5-6. top process monitoring sample screenshot

Figure 5-7. htop process monitor sample screenshot

SIPp Conﬁguration

As stated previously, the D-IPTS system requires user authentication. Whenever the client sends out an INVITE to the server, the server sends back a code 407 response to indicate Proxy Authentication Required. To address this server authentication requirement, a client XML ﬁle was created. This XML ﬁle acknowledges the response by the server and then re-sends an INVITE containing the authentication parameters. The server then responds with a 200 OK, meaning the request was successful. The steps and code to install SIPp on Alpine Linux are as follows:

104 1. Install required packages:

(a) apk add make gcc g++ automake autoconf libncurses5-dev python build- essential openssl libpcap-dev libssl-dev libnet1-dev libgsl0-dev gsl-bin libgsl0ldbl

2. Download and extract the source ﬁles:

(a) wget http://sourceforge.net/projects/sipp/ﬁles/sipp/3.2/sipp.svn.tar.gz (b) tar -xzf sipp.svn.tar.gz

3. Compile with pcpap play and TLS features:

(a) make pcapplay_ossl

4. At this point, SIPp is installed and can be run with the command “./sipp”, a “./sipp -h” will bring up the manual for the program.

5. To run the program with custom XML execute the following:

(a) sipp –s [call number] –sf [UAC Test File.xml] –inf [CSV contain Username and Password.csv] –d [length of call] –l [number of concurrent calls] [D-IPTS ip address]

Kamailio Software Customization

It was necessary to customize the Kamailio core source code to make it possible to collect more detailed performance metrics on the various packet latencies. The first of these changes was to configure Kamailio to use high resolution timers when collecting performance metrics. This was accomplished by modifying the build scripts and makefile. Kamailio’s C source code was then modified to capture and log the SIP request and response timestamps. Both these changes required rebuilding the Kamailio binaries from source code. Timestamps were generated when a SIP message was initially received and when its server side processing was completed. This data was logged to persistent storage using the standard, Kamailio configured logging daemon. To minimize the latency overhead of the profiling code, the data collected was kept to a minimum and only the message descriptors of interest and their duration was recorded and stored.

105 Additional processing was done ofﬂine using Python scripts to perform further analysis of the data. The D-IPTS system performance and availability experiments were divided into three experiments: the ﬁrst experiment was evaluating various call scenarios, the second experiment was evaluating the Kamailio SPS, and the third experiment was evaluating the performance of FreeSWITCH.

Experiment 1: Call Scenarios Testing

Several different test scenario ﬁles were used for the call scenario test experiments. These tests were important for observing and evaluating the variation in call delays due to call volume and call rate. The round trip time (response time) was calculated as the call rates were varied. The response times observed are from the UAC (SIPp) perspective, and they include the network delays and the response time of the UAS. The series of tests that were performed are given in Table 5-2 and the testbed conﬁguration can be seen in Figure 5-8.

Table 5-2. D-IPTS call scenario tests Test Component Tested

1 SIP OPTIONS Kamailio 2 SIP MESSAGE Kamailio 3 REGISTER Kamailio 4 INVITE Kamailio and FreeSWITCH

Figure 5-8. Call scenarios testbed

106 SIP OPTIONS (Kamailio): The SIP OPTIONS request allows a UA to perform a query in order to discover the capabilities of another UA or a proxy server [90]. SIP OPTION query was sent to registered users. The request was sent 20000 times at call rates of 440 to 3000 CPS. Figure 5-9 shows the message call ﬂow. Figure 5-10, shows the average response time (RTT) of the SPS for the OPTIONS message.

Figure 5-9. OPTIONS message ﬂow

Figure 5-10. Average response times of messages

Figure 5-11. Success rate of OPTIONS message

107 In the 500 CPS test, 96.4 % (19284 out of the 20000) of the requests were successful with an average RTT of 2.99 ms. When the call rate was increased to 1000 CPS, there was almost a 10x increase in the number of failed calls, with the total successful being only 67 % (13402), and the average RTT was also increased to 3.34 ms. For veriﬁcation purposes, the test was also done at a lower rate and increased to see when failures began to occur. The call rate could be increased up to 440 CPS after which there would be failures. It should be noted that this was an extreme test as there was no limit to the number of concurrent calls; it was done to ﬁnd the upper limits of error-free performance of the system. In a normal situation, it is unlikely that 400 or more calls will be received simultaneously by a server. The results are shown in Figure 5-11.

SIP MESSAGE (Kamailio): A SIP MESSAGE was sent in order to assess the time it takes to receive the 200 OK response (see Figure 5-12). A series of tests were carried out: 10000 CPS and 10000 CPS with a limit of 70 concurrent calls. Figure 5-13 shows the response times of the 10000 CPS with 70 concurrent call limitation. All messages were successfully received and an average RTT of 23.72 ms was observed. However, when the concurrent call limitation was removed, only 169631 (84.8 %) of the calls were successful and the average RTT time increased to over 100 ms.

Figure 5-12. MESSAGE call ﬂow

108 Figure 5-13. 10000 CPS with 70 concurrent

Figure 5-14. 10000 CPS test result

REGISTER (Kamailio): A SIP REGISTER request was sent order to establish the time it takes to receive the 200 OK response (see Figure 5-15). The request was sent 20000 times at call rates of 100 to 1000 CPS.

Figure 5-15. Registration call ﬂow

109 Figure 5-16. Registrations RTT

Results in Figure 5-16 show the max RTT at 1000 CPS was only 5.9 ms. The results show a D-IPTS system performance that is able to handle all the most common registration scenarios.

INVITE (Kamailio and FreeSWITCH): A SIP INVITE request was sent to the media server, FreeSWITCH, to assess its load handling capabilities. A series of tests were conducted using 20 concurrent calls and 120 calls with ﬁxed call rate, and 15 concurrent calls with a varying call rate. The results show that all messages were transmitted successfully with no need for any re-transmissions (see Figure 5-17a). However, when the number of simultaneous calls was increased to 120, Figure 5-17b, it was observed that in some instances the BYE message was not properly received and the message was re-transmitted 32 times during the test period. This is attributed to FreeSWITCH’s inability to process the larger number of concurrent calls. It was observed that when the number of concurrent calls approached or exceeded 100, there is a high incidence of packet loss. This is attributed to FreeSWITCH’s inability to process a large volume of concurrent calls instantaneously.

110 (a) 20 Call test scenario

(b) 120 Call test scenario

Figure 5-17. INVITE Transmission results for (a) 20 and (b) 120 concurrent calls

The next test was limited to only 15 concurrent calls, with varying call rates: at 440, 500 CPS, and 1000 CPS. Figure 5-18 shows the average response times of the three

111 tests. In both the 440 and 500 CPS tests, all calls were completed successfully with an average RTT time of 421 ms for 440 CPS and 491 ms for 500 CPS. When the call rate was increased to 1000 CPS, 64.6% of the calls were successful and the RTT increased to 501 ms. The increase in RTT that is observed when the call rates increased is due to the number of messages and the required processing time.

Figure 5-18. Response times for REGISTER-INVITE

In the previous test, FreeSWITCH was used as the UAS. The test was repeated with Kamailio to observe how it performed when it is configured as a UAS. To configure Kamailio as a UAS, the routing logic was modified to accept requests and respond 200 OK. A series of tests were conducted using call rates of 440, 500, 1000, and 7000 CPS with 27 concurrent calls. The results show that for call rates of 440, 500, and 7000, all calls were transmitted successfully with RTT less than 20 ms. For a call rate of 7000 CPS, all calls were successful and the average RTT was 12 ms (see Figure 5-19). What these results show is that when Kamailio is configured as a UAS, it performs better than FreeSWITCH. Based on this observation, Kamailio was used in the D-IPTS design

112 as the SIP router and FreeSWITCH only as the media server. In the event that more media handling capacity is needed, an additional FreeSWITCH can be integrated into the D-IPTS system.

Figure 5-19. Invite test results for r7000 l27

Experiment 2: Kamailio Router Test

A set of tests were designed to assess how the D-IPTS system handles concurrent call volume and calls per second over a given time period. FreeSWITCH and Kamailio were tested independently and compared to see how they performed. For the Kamailio tests, the testbed conﬁguration had to be slightly modiﬁed. A secondary server running another instance of Kamailio was used, which acted as an UAS, for the purpose of transmitting the "200 OK" responses. The CPU load information was collected over the duration of the test for each call rate. The results for each case were averaged and then plotted.

Concurrent calls

Concurrent or simultaneous calls refers to the number calls allowed in the system during a time period. A series of concurrent call experiments were conducted to stress the D-IPTS SPS (Kamailio) while monitoring the CPU performance (see Figure 5-20). A ﬁxed call rate of 200 CPS was used and the total number of concurrent calls varied from 100-2000. These tests were speciﬁcally designed to determine the load handling capability of Kamailio. The D-IPTS system was installed and run on a virtual machine server to allow for system hardware customization. The virtual machine was created on

113 Figure 5-20. Kamailio test INVITE call ﬂow

Figure 5-21. Kamailio testbed conﬁguration an Intel Core i7-2600K CPU, with a 3.4 GHz clock speed and 2 GB of RAM. The number of CPU cores available to the virtual machines were varied from single (1), double (2), and quad (4), while the amount of RAM remained constant.

114 (a) Kamailio CPU Utilization-Intel

(b) SIPp Execution Times

Figure 5-22. Number of concurrent calls vs CPU core performance

The results from Figure 5-22a show that as the number of concurrent calls increased, the CPU utilization remained almost constant. It was also observed that there was no signiﬁcant advantage when the CPU core was increased from single to double, however the utilization increased by 57% for the quad core. This increase is most likely due to extra processing overhead needed to split the process across the extra cores.

115 The biggest advantage of increasing the number of CPU cores was that the execution time decreased when the number of core increased; single to double 55% and single to quad 97% in Figure 5-22b. This implies that there is a performance trade-off between CPU utilization and execution (processing) times to consider in the D-IPTS system design.

Calls per Second Test

Figure 5-23. Kamailio CPU utilization

This test was designed to evaluate the call handling capability of Kamailio based on calls per second (call rate). The call rate was varied from 100-1000 and the CPU load information collected. Two separate x86 systems were used, an AMD Quad core and a Intel Core(TM)2 Duo CPU T6600 (see Table 5-1). The CPU utilization for the Intel dual core spiked to 100% once the call rate was 400 CPS. At 400 CPS, the utilization for the AMD quad core was only 47%. The AMD CPU was able to achieve 800 CPS at 100% CPU utilization. The addition of the two extra cores doubles the achievable call rate. In all cases, even at 100% CPU load, Kamailio did not experience any software errors (Figure 5-23).

Experiment 3: FreeSWITCH Stateful Call Processing

The tests from the previous section were repeated for the media server, FreeSWITCH. For FreeSWITCH the call rate ’r’ was varied from 5 to 50 CPS, for a total of 20000 calls. The CPU load information was collected over the duration of the test for each call rate.

116 The calls were directed to a conference extension for FreeSWITCH, which would answer the call and each call lasted for 30 seconds. The results for each case was averaged and then plotted.

Concurrent Calls Test

Figure 5-24. FreeSWITCH concurrent calls test-Intel at 15 CPS

The concurrent calls were generated at a rate of 15 CPS, and the number of concurrent calls were varied from 5 to 100. Calls were initiated to a music on extension, where they stay connected for 30 seconds and then terminated. When the number of concurrent calls was increased, there was a steady increase in the CPU utilization (see Figure 5-24). This observation is in contrast to Kamailio, where concurrent calls were seen to have minimal impact on CPU performance. An explanation for these results is attributed to differences between Kamailio and FreeSWITCH operations. Kamailio acts only as a SIP router and processes and forwards messages; once a call is connected, the UACs talk directly to each other. FreeSWITCH, on the other-hand, stays in the communication loop for the duration of call (for media trans-coding, logging, etc.) and therefore requires more resources.

Calls per Second Test

At 30 CPS the CPU utilization was 32% for the Intel dual core and 18% for the AMD quad core machine (see Figure 5-25). At 40 CPS FreeSWITCH experienced “throttle

117 Figure 5-25. FreeSWITCH CPU utilization errors” because it exceeded its processing capabilities. At 50 CPS, although the CPU utilization was less than 60% for both INTEL and AMD CPU’s, FreeSWITCH crashed and needed to be restarted. The program can safely operate at 30 CPS with no errors.

D-IPTS Performance Comparison With Next Generation Systems

The performance of the D-IPTS system was compared to commercially available Next Generation Systems (NGS). The D-IPTS can support well over 100 participants in a conference call. 8x8’s conference bridge can only support calls with up to 15 participants [118]. The D-IPTS, using Kamailio, is capable of handling over 800 inbound calls per second, while OnSIP can handle only 25 simultaneous inbound calls (see Figure 6-9)[119]. Skype for Business Online limits the number of open chat conversations to 50 [120]. The D-IPTS system imposes no such limitations so that users can have as many conversations open as necessary. The Standard Edition Skype for Business server has a maximum rate of calls retrieved by users of 25 per minute [121]. As previously stated, the D-IPTS can support well over this number. 2600hz uses FreeSWITCH as their call router, so the inbound max CPS before scaling is 30. The call router, Kamailio, used in the D-IPTS has been shown to have a superior performance advantage over FreeSWITCH and thereby the D-IPTS has a signiﬁcant call handling capability advantage over 2600hz.

118 Real world testing was done of FreeSWITCH by the FreeSWITCH community [122]. The most notable test was done on an Intel Xeon E5-2650L 1.80Ghz Processor with 4GB of RAM and running Ubuntu 12.04. Results from this test showed that FreeSWITCH was able to handle 2-180 concurrent calls, but having a CPU utilization of 80-100%. The D-IPTS system with FreeSWITCH as the media server was tested with up to 100 concurrent calls with CPU utilization being under 50%. The D-IPTS, however, achieved CPU utilization of 30% at 2000 concurrent calls, when Kamailio was used as the call router. This D-IPTS ﬂexible design feature exceeds the current industry performance, when the D-IPTS is designed with FreeSWITCH, and far exceeds it with Kamailio as the core router. This results in a D-IPTS system with a longer CPU lifetime and reduced failure rates. CommuniGate Systems performed VoIP scalability tests on their CommuniGate Pro Dynamic Cluster, and published results in [123]. The hardware used in their system is an HP Integrity Superdome. The HP Integrity Superdome has 64 x 1.5 GHz Itanium 2 processors and 256 GB memory. Their system demonstrated 2,160,000 successfully completed registrations over a one-hour period, for a call rate of 600 CPS. A similar test performed on the D-IPTS completed 3,600,000 registrations in a one-hour period, at a rate of 1000 CPS. An alternate perspective to emphasize the difference between these systems is the cost; the HP Integrity Superdome costs USD $11,978,134 [124] while the D-IPTS was built for under $500.

Summary

The performance and availability of the D-IPTS was evaluated using a series of call scenario tests. FreeSWITCH and Kamailio were evaluated independently and it was necessary to customize Kamailio to collect more detailed performance metrics. The analysis of the full featured D-IPTS system included CPU and signal delay experiments. The results of the call scenario testing showed that Kamailio as the SIP router had a signiﬁcant advantage over FreeSWITCH as the SIP router. The concurrent calls

119 test results show that the number of cores used in the D-IPTS system has a direct correlation to CPU utilization. When the number of concurrent calls was increased, there was a steady increase in the CPU utilization, when FreeSWITCH was used. This is in contrast to Kamailio, where concurrent calls were seen to have minimal impact on CPU performance. The results of the CPS test showed that there was a performance limitation for FreeSWITCH once the call rate reached 30 CPS. Kamailio was able to handle over 800 CPS at 100% CPU utilization on the system platform that was tested; however the D-IPTS system was able to exceed 800 CPS while at 100% utilization with no errors or failures. The D-IPTS leverages the individual advantages of FreeSWITCH and Kamailio in a manner that boosts overall system performance and availability. The results of this study show that in almost all scenarios the D-IPTS system delay performance is less than the ITU-T Recommendations for One-way transmission delay value of 150 ms [125]

120 CHAPTER 6 QUEUING MODEL DEVELOPMENT AND VALIDATION

Queuing Model

Queuing Theory (QT) is the mathematical study of queues, their lengths, associated wait times, throughput, and blocking probability. QT principles are used to develop a queuing model which can then be used to optimize and scale the D-IPTS system’s design to meet required performance. Using a queuing model for the D-IPTS allows for the prediction of the number of users in the system at steady state, the number of users in the queue (queue size), the length of time in the system (system delay), and the length of time in the queue (queue delay). The D-IPTS queuing model is shown in Figure 6-1. Several studies on IP telephony systems using M/M/1 models have appeared in the literature, however, empirical measurements of the D-IPTS testbed conﬁrmed that the M/M/1 model was not suitable for the D-IPTS. In the model development for the D-IPTS system I therefore only consider the M/G/1 queuing model. Characteristics of this model are:

1. A Poisson arrival process, with intensity λ

1 2. A service time X, with mean E(X) = X = µ

2 2 1 3. A second moment of service time X = σ + µ2

λ 4. A single server, with load ρ = λX = µ

5. A First Come First Serve (FCFS) Queuing Discipline

121 Figure 6-1. D-IPTS System queuing model

To determine empirically the average service rate (µ) of the Kamailio SPS, SIPp was used to generate 10000 calls at 1 CPS for the D-IPTS testbed. Each call was assumed to consist of five different packet types: INVITE, 200 OK (INVITE), ACK, BYE, and 200 OK (BYE). Most other studies only evaluate call setups which only include the INVITE, 200 OK, and ACK packets, but I wanted to evaluate the processing of the entire call sequence. The calls were sent to a modified version of Kamailio, as described in Section5, and the processing time for each packet was collected. A Python script was used to parse the log file and sort it based on packet type and processing time. From this data the mean processing time for each packet type was determined (see Table 6-1). From this data, the overall mean call processing (service) time was calculated to be 61 micro-seconds, which is approximately 16,000 PPS (packets per second). The testbed configuration that was used can be seen in Figure 6-2.

122 Table 6-1. Kamailio call processing Times Average Processing Time (µsec)

INVITE 87 200 OK 43 ACK 65 BYE 81 200 OK 31

Figure 6-2. Queuing model testbed

Queuing Experiments

For the queuing experiments, calls were initiated between the UAC (User Agent Client) and UAS (User Agent Server) via the SPS. For each call arrival rate, 10,000 calls were generated and each call, the messages exchanged between the UAC and UAS are as shown in Figure 6-3. For each call, the SPS receives and processes ﬁve different messages: three for call setup and two for call tear-down. Therefore, a total of 50 thousand messages (packets) may be processed by the SPS for each experiment (the number is smaller when messages are dropped due to overload). For each packet processed by the SPS, the waiting and service times are calculated.

123 Figure 6-3. D-IPTS queuing experiment call ﬂow

Each experiment was characterized by two parameters: • Call arrival rate. The call arrival rate ranges from 100 CPS to 1500 CPS. A maximum call rate was determined from the CPU call rate experiments; at this point the CPU utilization was at 100%. • Call inter-arrival time distribution. A modiﬁed version of SIPp tool was as a UAC to generate call arrivals. Calls were generated and the inter-arrival times were exponentially distributed around the reciprocal of a mean call rate. The performance measures for the M/G/1 queuing model are given by the mean waiting times and mean numbers in queue and in service, which are given below [126]:

λX 2 1. Mean waiting time in the queue, WQ = 2(1−ρ)

λ2X 2 2. Mean number in the queue, NQ = λWQ = 2(1−ρ)

1 λX 2 3. Mean waiting time, in queue and in service, W = WQ + µ = X + 2(1−ρ)

1 λ2X 2 4. Mean number in the system, N = λW = ρ + 2 (1−ρ)

5. Throughput, C = µ(1 − ρ) Wireshark, a packet analysis tool, was used to analyze and monitor for packet loss. For all the tests Wireshark reported that no packets were dropped or blocked. This means that the M/G/1 was an accurate model since no calls were blocked due to a full queue, for the D-IPTS system platform that was studied.

124 Figure 6-4 shows the mean processing times from the data collected for each packet time. It was observed that as the call rate is increased, there is a noticeable increase in the processing times, especially for the larger packets for INVITE and BYE. There does not seem to be a direct correlation between an exponentially distributed call inter-arrival times and the packet processing time. Figure 6-5 shows the overall processing time of the combination of packets for varying call rates.

Figure 6-4. Mean processing time values

Figure 6-5. Overall processing time vs call rate of calls

125 Utilization Comparison/Validation

Utilization plots are useful for assessing the performance of the D-IPTS system and can indicate system capacity thresholds. The utilization is a function of the call rate and

λ the service rate and is determined by µ . Figure 6-6, shows the expected number in the queue and the expected number in the system as a function of the utilization. In both cases it can be observed that the experimental and theoretical results from the model are analogous with deviation range of 0-5%. These results indicate that the Kamailio call service rate was faster than the incoming call rate, keeping the system under 50% utilization.

(a) Utilization in Queue (b) Utilization in System

Figure 6-6. Utilization of the D-IPTS. A) Expected number vs. utilization in Queue. B) Expected number vs. utilization in System.

Calls per Second Comparison/Validation

This section will show the expected number of calls in the queue and in the system as functions of the incoming call rate. Figures 6-7a and 6-7b show the expected number of calls in the queue and in the system as the call rate increases. From the ﬁgures, it can be observed that the experimental and theoretical model are similar, and there was a maximum deviation in the results of 15.3% for the queue and 15.8% for the system.

126 (a) Expected number in the (b) Expected number in the Queue System

Figure 6-7. Expected number of calls in the D-IPTS. A) Expected number in the Queue. B) Expected number in the System.

Waiting Time Comparison/Validation

Waiting time plots show the average delay a call will see before it is processed. These results allow for “busy hour” call planning, whereby a system designer can scale the D-IPTS based on a desired service rate for a given hour. Figure 6-8a shows the expected call waiting time in the queue as a function of the call rate, with a maximum deviation of 22.6% from experimental results. Figure 6-8b, shows the waiting time in the queue as a function of the utilization.

127 (a) vs. calls per second (b) vs. utilization

Figure 6-8. Waiting time in Queue. A) Waiting time in Queue vs. calls per second. B) Waiting time in Queue vs. utilization.

D-IPTS Design Decisions

The D-IPTS system model was validated as described in the previous section. This model can be used to make predictions of system capacity and performance limitations.

128 Call Rate and Utilization

(a) Predicted call rate

(b) Predicted utilization

Figure 6-9. Predicted values based on Queuing Model. A) Predicted call rate. B) Predicted utilization

The Queuing model was used to predict the point at which the D-IPTS system will become fully utilized ( see Figure 6-9a& 6-9b). Results show that the system becomes fully utilized when the call rate is 3,300 CPS. This is in contrast to the call rate CPU experimental results which shows 100% CPU utilization at 800 CPS (AMD CPU). The system is capable of handling a higher call volume, however the strain of running the CPU above 800 CPS will create system bottlenecks, increase call latency, shorten CPU life, and increases the chances of a D-IPTS system failure. The D-IPTS system performance will be supported and guaranteed up to the 800 CPS threshold, at greater than 80% utilization. According to[42], the telecommunications industry recommends

129 a 70% system operational threshold. The D-IPTS system at 100% CPU utilization is capable of handling 800 CPS. If the industry’s recommendation is used, the D-IPTS system is only required to handle 560 CPS. Additional servers would need to be added to support increased call capacity. The D-IPTS in its current conﬁguration can handle more call capacity than recommended without the need for additional hardware at the expense of performance.

Number of Servers

The call processing capability for the D-IPTS system with Kamailio as the SIP router is 800 CPS. In a given hour, Kamailio’s call processing capability is 2,880,000 SIP calls per hour. If there is a requirement for more call processing capability, additional servers are needed. For example, to handle 3,000,000 BHCA (Busy Hour Call Attempt), then two servers are required [127]. For FreeSWITCH, the media server, call processing capability is 40 CPS. The call processing capability for a single server is 144,000 calls per hour. If the requirement of call processing capability for each server is designed to handle 3,000,000 BHCA (Busy Hour Call Attempt), then 21 FreeSWITCH servers will be required.

Summary

The results from the experimental and the M/G/1 queuing model are quite similar. This implies that the M/G/1 queuing model with the empirical parameters obtained from the testbed measurements is an accurate and ideal predictor for assessing the mean call delay, number of calls in the queue, and system utilization for the D-IPTS system. The M/G/1 queuing model assumes an inﬁnite buffer size (no limit on number of packets), therefore the probability of packet loss in this model zero, and hence in the D-IPTS the probability of packet loss is also zero for the range of parameters that were studied. Additionally, the probability of capacity overload that will lead system latency is minimized. Therefore, the D-IPTS system performance is well optimized. The veriﬁed

130 D-IPTS system model is a now a useful tool for design and engineering decisions for network planning.

131 CHAPTER 7 CONCLUSION AND FUTURE WORK A fully functional distributed IP-based Telecommunication System was designed and implemented with an Internet-inspired distributed design using open source software, with customizations and configurations. It provides the common features of a traditional office PBX, namely music-on-hold, voicemail, automated attendant and conference calls. External calls to any PSTN extension can be made by redirecting the call to FreeSWITCH and using it as a B2BUA. Intelligent SIP phones used in the system can directly take the configuration settings from the provisioning server, and get automatically registered. The phones can handle call transfers on their own. The user can also modify any of the settings as per his/her requirement. The phone will then inform the provisioning server of these changes. The system has been designed in a manner that is highly flexible and well suited for a distributed IP telecommunication system. This novel D-IPTS system is the first step in achieving a single device for all communication needs. The performance and availability of the D-IPTS system was evaluated using the system as a testbed. The performance and availability of the D-IPTS was evaluated using a series of call scenario tests. FreeSWITCH and Kamailio were evaluated independently and it was necessary to customize Kamailio to collect more detailed performance metrics. The results of the call scenario testing showed that Kamailio as the SIP router had a significant advantage over FreeSWITCH. A system model was developed by using empirical data from the D-IPTS testbed. The results from the Queuing analysis show that the experimental and the M/G/1 queuing model are identical. The M/G/1 queuing model was determined to be an accurate and ideal predictor for assessing and validating the performance of the D-IPTS system design. The Queuing model was used to predict the point at which the D-IPTS system will become fully utilized. Results further show the point at which the D-IPTS system achieves its

132 operational thresholds and how to properly scale the system to meet requirements. The D-IPTS leverages the individual advantages of FreeSWITCH and Kamailio in a manner that boosts overall system performance and availability, therefore the D-IPTS system performance is well optimized. The prototype D-IPTS system has become a testbed or evaluation tool for a distributed IP-based telecommunications system. Opportunities for future work with the D-IPTS system are described in this section. The D-IPTS has many built-in security features. An exhaustive test of these features was not performed, however an evaluation on the impact on the performance should be conducted. All of the security features require user authentication. Combination of security tests for example packet spoofing and call flooding can be performed. The D-IPTS system can support the addition of additional calling features. Some examples group messaging, multi-party group conferencing, and the ability to send files within the phone app. To add new features requires the Kamailio routing logic to be modified and the performance impact assessed. The D-IPTS system open-source design allows the use of any codec. Tests can be performed to evaluate the implications of the different codecs. One such test is a detailed study to see how varying voice codecs affects CPU usage of concurrent calls on FreeSWITCH. The D-IPTS system performance in its current configuration has been shown to be well optimized. Additional features or increased capacity demands will require additional servers. For Kamailio and FreeSWITCH there is already a guideline for call capacity scaling that is detailed in Section6 (1 Kamailio: 2,880,000 BHCA, 1 FreeSWITCH : 144,000 BHCA ). A performance evaluation using similar methodology (CPS, concurrent, and queuing) must be conducted to establish impact of load balancing and fail-over schemes.

133 The Linphone open-source SIP client was used for testing the D-IPTS system. The integration of the D-IPTS system and Linphone is the ‘Gatorfone’. The Gatorfone app has been designed and launched on the iOS platform. New features need to be added such as picture messaging, geo-location, and file sharing. The source code (Objective-C) will need to be modified to incorporate these or any other feature. The D-IPTS system is currently unable to support group video chats. To support group video chat, modification of the source code is necessary. A group mesh topology will need to be implemented where users transmit data to all participants. RTPproxy is the NAT solution that is used for the D-IPTS system. An evaluation of RTPproxy’s load handling capabilities needs to be performed. To accomplish this, the UAC (SIPp) will need to be NATted and load testing performed. A suitable NAT solution that works for all scenarios was utilized. Other potential candidates for evaluation are Session Traversal Utilities for NAT (STUN), Traversal Using Relay NAT (TURN), and Application Layer Gateway (ALG) intelligence. These options need to be evaluated to assess their impact on the D-IPTS system. Additionally there is interest in the evaluation of different database replication schemes, and the implications each will have on system availability. The D-IPTS was implemented using redundant database servers. The reliability and performance of the failover should be evaluated. One experiment that could be done is to disable the main database and measure the time it takes for the backup database to take over.

134 APPENDIX A SUMMARY OF RESEARCH CONTRIBUTIONS Major contributions: • Designed a distributed IP Telecommunication System (D-IPTS)

– A fully functional, scalable and expandable, interactive multimedia IP telecommunications system

– Fully distributed architecture – System used as a testbed – Highly availability – Reliability mechanisms • Performed a comprehensive performance analysis of the D-IPTS

– Determined the failure modes and requirements for scalability • Developed and validated a performance model of the D-IPTS using Queuing Theory and empirical parameters from the D-IPTS testbed

– Model used to study the capacity and projected limits of performance of the D-IPTS • Designed a robust and cost-effective USB-based run from RAM system

– Single server D-IPTS System – Multiple server D-IPTS system – LXC hosted D-IPTS System • Kamailio 2013 Award Winner • Gatorfone iOS App development Other contributions of this dissertation : • D-IPTS Security implementation

– User Authentication – SSH-key only logins – IP bans and ﬂood protection

135 • Linux CLI proficiency • Created LIST Lab IVR menu • Designed a DHCP Server • Designed a Provisioning Server • Designed a DNS server • Implemented NAT Traversal Solution • Shell Scripting proficiency • Kamailio routing logic and modules proficiency • FreeSWITCH XML code proficiency • Developed IP address changing script • Developed a Provisioning via web interface

– Assisted by Python code • PostgreSQL Streaming Replication

136 APPENDIX B KAMAILIO INSTALLATION • #Install PostgreSQL

– apk add postgresql postgresql-client • Setup postgresql

– /etc/init.d/postgresql setup • #Direct logging information to syslog

– sed "/^[# ]*log_destination/clog_destination = ’syslog’" -i /var /lib /postgresql /9.3/data/postgresql.conf • #Start postgresql and enable auto start

– /etc/init.d/postgresql start – rc-update add postgresql • #Create ﬁle for Kamailio to start after pg-restore when booting:

– echo ’rc_after=pg-restore’ > /etc/conf.d/kamailio • #Conﬁgure postgresql and setup LBU backup for the database:

– lbu include /var/lib/postgresql/ – mkdir -p /var/lib/postgresql/backup – sed ’/^[# ]*PGDUMP/cPGDUMP="/var/lib/postgresql/backup/databases.pgdump"’ -i /etc/conf.d/pg-restore

– rc-update add pg-restore – mkdir /etc/lbu/pre-package.d – echo "#!/bin/sh" > /etc/lbu/pre-package.d/postgresdump – echo "/etc/init.d/pg-restore dump" >> /etc/lbu/pre-package.d/postgresdump – chmod +x /etc/lbu/pre-package.d/postgresdump • Install the Kamailio packages

– apk add kamailio kamailio-presence kamailio-postgres • Conﬁgure Kamailio

137 – sed "/^[# ]*SIP_DOMAIN/cSIP_DOMAIN=10.0.0.160” -i /etc/kamailio/kamctlrc – sed ’/^[# ]*DBENGINE/cDBENGINE=PGSQL’ -i /etc/kamailio/kamctlrc – sed ’/^[# ]*DBHOST/cDBHOST=localhost’ -i /etc/kamailio/kamctlrc – sed ’/^[# ]*DBNAME/cDBNAME=openser’ -i /etc/kamailio/kamctlrc – sed ’/^[# ]*DBRWUSER/cDBRWUSER=openser’ -i /etc/kamailio/kamctlrc – sed ’/^[# ]*DBRWPW/cDBRWPW="openser"’ -i /etc/kamailio/kamctlrc – sed ’/^[# ]*DBROUSER/cDBROUSER=openserro’ -i /etc/kamailio/kamctlrc – sed ’/^[# ]*DBROPW/cDBROPW=openserro’ -i /etc/kamailio/kamctlrc – sed ’/^[# ]*DBROOTUSER/cDBROOTUSER="postgres" ’ -i /etc/kamailio/kamctlrc – sed ’/^[# ]*OSER_FIFO/cOSER_FIFO="/tmp/kamailio/kamailio_ﬁfo" ’ -i /etc/kamailio/kamctlrc

– echo postgres > /root/.pgpass chmod 600 /root/.pgpass • Create initial database

– yes|kamdbctl create openser • Set Kamailio to start on boot and after Postgres

– rc-update add kamailio – echo ’rc_after=postgresql’ >> /etc/conf.d/kamailio • Create the directory for pid ﬁle:

– mkdir -p /var/run/kamailio – # set ownership to /var/run/kamailio

* chown kamailio:kamailio /var/run/kamailio • Start Kamailio

– /etc/init.d/kamailio start

138 APPENDIX C FREESWITCH INSTALLATION

1. apk add freeswitch freeswitch-sounds-en-us-callie-8000 freeswitch-ﬂite acf- freeswitch freeswitch-sample-conﬁg freeswitch-sounds-music-8000 ssmtp

2. Start FreesWITCH and it to rc-update:

(a) /etc/init.d/freeswitch start && rc-update add freeswitch

3. #include the freeSWITCH ﬁles to lbu:

(a) lbu include var/lib/freeswitch (b) lbu include usr/sounds

4. #to get voicemail to work:

(a) mkdir /usr/storage chown freeswitch:freeswitch /usr/storage cp -avr /usr/share/freeswitch/sounds /usr/ (b) mkdir /usr/conf cp /etc/freeswitch/tetris.ttml /usr/conf

5. apk update

6. apk cache sync

7. lbu ci

139 APPENDIX D SETTING UP MASTER-SLAVE REPLICATION Install the appropriate packages with these commands. apk update apk add postgresql postgresql-client acf-postgresql PostgreSQL creates a user called "postgres" in order to handle its initial databases. I will configure ssh access between my servers to make transferring files easier. I will need to set a password for the postgres user so that i can transfer the key files initially. If you desire, you can remove the password at a later time: passwd postgres Make SSH directory for postgres mkdir /var/lib/postgresql/.ssh/ Change permissions of SSH directory chown -R postgres /var/lib/postgresql/.ssh/ chmod -R u+rX /var/lib/postgresql/.ssh/ The first command above sets the user ’postgres’ as the owner of the directory and the second command gives users of the directory ’read’ and ’write’ permissions. Next is to switch over to the postgres user and generate an ssh key. su - postgres ssh-keygen Transfer the keys to the other server: ssh-copy-id IP_address_of_opposite_server Configure the Master Server Begin by configuring my master server. All of these commands should be executed with the postgres user. First, i will create a user called "replicator" that can be used solely for replication: psql -c "CREATE USER replicator REPLICATION LOGIN CONNECTION LIMIT 1 ENCRYPTED PASSWORD ’yourpassword’;" Next, move to the postgres configuration directory:

140 cd /var/lib/postgresql/9.2/data/ Modify the access file with the user i just created: nano pg_hba.conf At any place not at the bottom of the file, add a line to let the new user get access to this server: host replication replicator IP_address_of_slave/32 md5 Save local replication postgres trust and close the file. Next, open the main postgres configuration file: nano -w postgresql.conf Find these parameters. Uncomment them if they are commented, and modify the values according to what is listed below: listen_addresses = ’localhost,IP_address_of_THIS_host’ Save wal_level = ’hot_standby’ archive_mode = on archive_command = ’cd .’ max_wal_senders = 3 max_wal_senders = 5 wal_keep_segments = 32 and close the file. Change back to root account su root Restart the master server to implement your changes: /etc/init.d/postgresql restart Configure the Slave Server Begin on the slave server by shutting down the postgres database software: Run as root /etc/init.d/postgresql stop Make some similar configuration changes to postgres files, so change to the configuration directory: cd /var/lib/postgresql/9.2/data/ Adjust the access file to allow the other server to connect to this. This is in case of a failure then the slave turns into the master. nano pg_hba.conf

141 Add the host access information in the file: host replication replicator IP_address_of_master/32 md5 local replication postgres trust Save and close the file. Next, open the postgres configuration file: nano -w postgresql.conf You can use the same configuration options you set for the master server, modifying only the IP address to reflect the slave server’s address: listen_addresses = ’localhost,IP_address_of_THIS_host’ wal_level = ’hot_standby’ archive_mode = on archive_command = ’cd .’ max_wal_senders = 5 wal_keep_segments = 32 hot_standby = on Save and close the file. Before the slave can replicate the master, i need to give it the initial database to build off of. This is because it reads the logs off of the master server and applies the changes to its own database. I need that database to match the master database. On the master server, i can use an internal postgres backup start command to create a backup label command. I then will transfer the database data to my slave and then issue an internal backup stop command to clean up: su - postgres psql -c "SELECT pg_start_backup(’replbackup’);" tar cf /tmp/db_file_backup.tar /var/lib/postgresql/9.2/data psql -c "SELECT pg_stop_backup();" On slave computer stop postgresql and move the existing data directory to a new folder and erase the old data directory. cd / mv /var/lib/postgresql/9.2/data/ /var/lib/postgresql/9.2/data.old cd /var/lib/postgresql/9.2/main/ rm -rf * Unzip master server data snapshot file that is copied into this server.

142 tar -xjvf /tmp/db_file_backup.tar Make sure that the contents of this directory are owned by the postgres user and group. If not, then: $ chown -R postgres:postgres /var/lib/postgresql/9.2/data I now have to configure a recovery file on my slave. On the slave navigate to the postgresql data directory. Here, i need to create a recovery file called recovery.conf: nano recovery.conf Fill in the following information. Make sure to change the IP address of your master server and the password for the replicator user you created: standby_mode = ’on’ primary_conninfo = ’host=master_IP_address port=5432 user=rep password=yourpassword’ trigger_file = ’/tmp/postgresql.trigger.5432’ The last line in the file, trigger_file, is one of the most interesting parts of the entire configuration. If you create a file at that location on your slave machine, your slave will reconfigure itself to act as a master. This will break your current replication, especially if the master server is still running, but is what you would need to do if your master server goes down. This will allow the slave to begin accepting writes. You can then fix the master server and turn that into the slave. Update permissions on recovery.conf file and start PostgreSQL. chown postgres.postgres /var/lib/postgresql/9.2/data/recovery.conf /etc/init.d/postgresql start

143 APPENDIX E SETUP LXC HOST

• Boot Alpine USB – Follow the instructions on http://wiki.alpinelinux.org/wiki/Create_a_Bootable_USB about how to create a bootable USB.

• Alpine Setup (version 2.7+) – setup-alpine • Upgrade packages

– apk update && apk upgrade • Save Changes

– lbu commit • Finish Setup with a reboot

– reboot • Install LXC Install the LXC and Bridge packages

– apk add lxc bridge • Set up a bridge on the host.

– Example /etc/network/interfaces:

* auto br0 * iface br0 inet dhcp * bridge-ports eth0 • Create a network conﬁguration template for the guests, /etc/lxc/lxc.conf:

– lxc.network.type = veth – lxc.network.link = br0 – lxc.network.ﬂags = up • Restart Networking

– /etc/init.d/networking restart • Install the SIP Container

144 – Create and Conﬁgure the container

* lxc-create -n sip -f /etc/lxc/lxc.conf -t alpine • Create the startup Script

– ln -s /etc/init.d/lxc /etc/init.d/lxc.sip • Set lxc folder to Backup

– lbu include /var/lib/lxc • Start the container

– /etc/init.d/lxc.sip start • Conﬁgure the container to automatically start

– rc-update add lxc.sip • Enter the sip container

– lxc-console -n sip – Login as root Note: If the need arises to exit the container press Ctrl+ a + a+a+q • Remove obsolete /etc/network/interfaces

– rm /etc/network/interfaces • Create and conﬁgure the new /etc/network/interfaces as shown below:

– Contents of /etc/network/interfaces: – auto lo – iface lo inet loopback – auto eth0 – iface eth0 inet static – address 10.2.0.4 – netmask 255.255.255.0 – gateway 10.2.0.1 • Startup networking

– /etc/init.d/networking start

145 • Conﬁgure remote administration

– apk update – setup-sshd -c openssh • Conﬁgure a passwd for the container

– passwd • Setup acf for web administration

– setup-acf • see Appendix ?? for Kamailio installation instructions • Install the SIP Media container – Create and Conﬁgure the container

* lxc-create -n sipmedia -f /etc/lxc/default.conf -t alpine • Create the startup Script

– ln -s /etc/init.d/lxc /etc/init.d/lxc.sipmedia • Start the container

– /etc/init.d/lxc.sipmedia start • Conﬁgure the container to automatically start

– rc-update add lxc.sipmedia • Enter the SIP Media container

– lxc-console -n sipmedia • Create and conﬁgure the new /etc/network/interfaces as shown below:

– Contents of /etc/network/interfaces

* auto lo * iface lo inet loopback * auto eth0 * iface eth0 inet static * address 10.0.0.4 * netmask 255.255.255.0

146 * gateway 10.0.0.1 • Startup networking

– /etc/init.d/networking start • Conﬁgure remote administration

– apk update – setup-sshd -c openssh • Conﬁgure a passwd for the container

– passwd • Setup acf for web administration

– setup-acf • Install FreeSWITCH see Appendix ??

147 APPENDIX F CONNECTING TO A WIRELESS ACCESS POINT • Install wireless-tools and wpa_supplicant dbus (system message service needed for alpine <2.7 , if using alpline 2.7 dbus gets added automatically).

– apk add wireless-tools wpa_supplicant dbus • Bring the link up so i can look for wireless networks. (An error here means you probably need extra drivers/ﬁrmware.)

– ip link set wlan0 up • Find a network to connect to. Look for the ESSID. In this example i will use the ESSID "MyNet" (this step not necessary if you know ssid but does help because it shows if wiﬁ is working).

– iwlist wlan0 scanning • Lets set the ESSID:

– iwconﬁg wlan0 essid MyNet • I need to create a shared key for wpa_supplicant.

– wpa_passphrase MyNet > wpa.conf • It will wait for the password from stdin. Enter the password and enter. Now you will have a wpa.conf ﬁle with the preshared key. Start wpa_supplicant with the generated conﬁg:

– wpa_supplicant -Dwext -iwlan0 -c ./wpa.conf • From another console, start dhcpcd:

– udhcpc -i wlan0 • You should get an IP address. You then want to make the connection process automatic on boot-up. Open /etc/network/interfaces and add the following stanza:

– auto wlan0 iface wlan0 inet dhcp • You will also need to set wpa_supplicant to start automatically on boot:

– rc-update add wpa_supplicant default

148 • Next, create /etc/wpa_supplicant/ (permissions of 755 with root:root are ﬁne), and move wpa.conf into that folder, renaming it to wpa_supplicant.conf.

– cp wpa.conf /etc/wpa_supplicant/wap_supplicant.conf • Reboot and check that you are associated with the access point:

– iwconﬁg wlan0 • and check that you got a DHCP lease:

– ifconﬁg wlan0 | grep addr • To conﬁgure static IP address for wlan: auto wlan0 iface wlan0 inet static address 10.0.0.165 netmask 255.255.255.0 gateway 10.0.0.1 up ip addr add 10.0.0.162/24 dev wlan0 hostname gfone3

149 APPENDIX G BUILD OR COMPILE LINPHONE FROM SOURCE FOR IOS

1. Install XCode Install XCode

2. Install required linux package through MacPort Install MacPort, then install these ports: Note, the following commands are for bash, might need to modify slightly for other shell: #sudo port install coreutils automake autoconf libtool intltool wget pkg- conﬁg cmake gmake yasm grep doxygen ImageMagick optipng antlr3 nasm #wget –no-check-certiﬁcate https://raw.github.com/yuvi/gas-preprocessor/master/gas- preprocessor.pl #sudo mv gas-preprocessor.pl /opt/local/bin/. #sudo chmod +x /opt/local/bin/gas-preprocessor.pl Then link macport libtoolize to glibtoolize #sudo ln -s /opt/local/bin/glibtoolize /opt/local/bin/libtoolize Then link host strings to simulator SDK For Xcode prior to 4.3: #sudo ln -s /usr/bin/strings /Devel- oper/Platforms/iPhoneSimulator.platform/Developer/usr/bin/strings For newer XCode:

(a) #sudo ln-s/usr/bin/strings/Applications/Xcode.app/Contents/Developer/Platforms (b) Note, on my Mountain Lion OS, I can’t get “sudo” to work, so I enable the root via this link: http://support.apple.com/kb/PH11331 and use the following commands #su - #port install coreutils automake autoconf libtool intltool wget pkgconﬁg cmake gmake yasm grep doxygen ImageMagick optipng gtk2 nasm #wget –no-check-certiﬁcate https://raw.github.com/yuvi/gas- preprocessor/master/gas-preprocessor.pl #mv gas-preprocessor.pl /opt/local/bin/. #chmod +x /opt/local/bin/gas-preprocessor.pl

3. Install or Update Git Install git. I followed the instruction on this great github article. To update an existing git install (if you got an git version error in the later steps) #git clone git://github.com/gitster/git.git

4. Obtain linphone source code Download linphone source from this git tree page #git clone git://git.linphone.org/linphone-iphone.git –recursive

5. Install JDK Some submodules require JDK. You can download JDK from Oracle site. I used JDK7 (http://www.oracle.com/technetwork/java/javase/downloads/jdk7- downloads-1880260.html) After installation, verify JDK by opening a terminal and type #java -version

6. Update iOS SDK Version IMPORTANT!!! Update SDK_VERSION in submodules/build/iphone-conﬁg.site if appropriate To check for iOS SDK, at the commandline #ls /Applica- tions/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs My result is: iPhoneOS7.0.sdk Therefore I changed my iphone-conﬁg.site to: ... SDK_VERSION_MAJOR=7 SDK_VERSION=7.0 ...

150 7. Build LinPhone SDK Build SDK #cd submodules/build #make veryclean #make all The build can take a long while, so grab yourself a good book or go make yourself a good cup of instant noodle.

8. Build LinPhone in XCode Open linphone.xcodeproj in linphone-iphone directory. Just build as normal.

151 REFERENCES [1] R. Gareiss, “The Business Case for Voice Over IP,” white paper, Nemertes Research, 2007. pages 9, 26, 27 [2] C. Systems, “IP telephony: The ﬁve nines story,” tech. rep., Cisco Systems, 2002. pages 10, 25, 26, 55 [3] Tanenbaum, Computer Networks. Pearson Education(singapore) Pte. Limited, 3rd ed., 1993. pages 21 [4] Vonage, “Vonage Home Page.” http://www.vonage.com/, October 2013. pages 23, 24 [5] M. Jack, “Magic Jack Home Page.” Website, October 2013. pages 23 [6] AT&T, “AT&T International Rates,” October 2013. pages 24 [7] GAO, “Telephone Communications: Bypass of the Local Telephone Companies,” August 1986. pages 24 [8] Alberto, Leon-Garcia and W. Indra, Communication Networks. McGraw-Hill, 2 ed., 2004. pages 24, 30 [9] E. H. Ben Charny, “Court’s call: Hands off VoIP.” online, 2003. pages 29 [10] VoIPUSA.com, “History of Voice over IP Phones and VoIP Technology.” online, 2008. pages 29 [11] Asterisk, “Asterisk Home Page,” October 2013. pages 29, 48 [12] vopium, “global voip revolution,” 2011. pages 29 [13] Microsoft, “Skype Home Page.” http://www.skype.com/en/, 2014. pages 29 [14] R. Pepper, “The History of VoIP and Internet Telephones.” http://getvoip.com/blog/2014/01/27/history-of-voip-and-internet-telephones, January 2014. pages 29 [15] Wikipedia, “Voice over Internet Protocol.” http://en.wikipedia.org/wiki/VoiceoverInternetProtocol, 2014. pages 29 [16] M. Spencer, B. Capouch, E. Guy, F. Miller, and K. Shumard, “RFC5456IAX: Inter-Asterisk eXchange Version 2,” February 2010. pages 32 [17] J. Rosenberg, H. Schulzrinne, G. Camarillo, A. Johnston, J. Peterson, R. Sparks, M. Handley, and E. Schooler, “RFC 3261-SIP: Session initiation protocol,” June 2002. RFC 3261. pages 32, 70, 72

152 [18] M. Handley and V. Jacobson, “RFC2327: SDP: Session Description Protocol,” April 1998. pages 32, 73 [19] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, “RFC3550:RTP: A Transport Protocol for Real-Time Applications,” July 2003. pages 33, 74, 75 [20] M. Baugher, D. McGrew, C. S. Inc., M. Naslund, E. Carrara, K. Norrman, and E. Research, “RFC3711: The Secure Real-time Transport Protocol (SRTP),” March 2004. pages 33 [21] R. Braden, Ed.ISI, L. Zhang, S. Berson, S. Herzog, and S. Jamin, “rfc2205: Resource ReSerVation Protocol (RSVP),” September 1997. pages 33 [22] R. L. H. Schulzrinne, A. Rao, “Real Time Streaming Protocol (RTSP) (RFC 2326),” April 1998. pages 33 [23] C. Systems, “Voice Over IP - Per Call Bandwidth Consumption,” tech. rep., Cisco Systems, http://www.cisco.com/c/en/us/support/docs/voice/voice-quality/7934- bwidth-consume.html, 2006. pages 34 [24] “Mean opinion score,” 2006. pages 34 [25] I.-T. G.729, “G.729,” June 2012. pages 34 [26] I.-T. G.722, “G.722,” September 2012. pages 35 [27] G.711ITU, “G.711 : Pulse code modulation (PCM) of voice frequencies,” Novem- ber 1988. pages 35 [28] Networkers, “Availability Measurements,” tech. rep., Cisco, 2004. pages 36 [29] C.-H. Chu, H. Pant, S. Richman, and P. Wu, “Enterprise VoIP Reliability,” in Telecommunications Network Strategy and Planning Symposium, 2006. NET- WORKS 2006. 12th International, pp. 1–6, Nov 2006. pages 37, 101 [30] W. Engineers, “What is an Erlang.” http://www.erlang.com/whatis.html, January 2014. pages 38 [31] S. A. Management, “Reliability Block Diagram (RBD),” tech. rep., SYDNEY WATER, 2010. pages 38, 39 [32] Microsoft, “Chapter 2 - What is Network QoS?,” tech. rep., Microsoft, http://technet.microsoft.com/en-us/library/bb742481.aspx, September 1999. pages 40 [33] J. Kurose and K. Ross, Computer Networking: A Top-Down Approach. Always learning, Pearson Education, Limited, 2012. pages 40 [34] “Quality of experience,” 2014. pages 41

153 [35] H. A. Latchman, Computer Communication Networks and the Internet. McGraw-Hill, 1997. pages 42, 43 [36] A. Papakotoulas, “Voice over Internet Protocol,” Journal of Computations & Modelling, vol. 4, no. 1, pp. 299–310, 2014. pages 46 [37] J. Q. Walker and J. T. Hicks, “The Essential Guide to VoIP Implementation and Management,” tech. rep., NetIQ Corporation, 2002. pages 46 [38] M. Pohancenik, Design and implementation of a system CERN¡Šs telephony networks. PhD thesis, University of Zilina, 2013. pages 47 [39] H. S. J. R. Venkatesha Prasad, Richard Hurni, “A scalable distributed VoIP conferencing using SIP,” in Computers and Communication, 2003. (ISCC 2003). Proceedings. Eighth IEEE International Symposium on, pp. 608–613, 2003. pages 47 [40] Aculab, “Distributed architecture for VoIP telephony Solutions,” tech. rep., 2007. pages 48 [41] W. Jiang and H. Schulzrinne, “Assessment of VoIP Service Availability in the Current Internet,” 2003. pages 48 [42] D. S. Chi-Hung Kelvin Chu, Ramesh Nagarajan and W. Yang, “End-to-End Performance and Reliability Estimation of PacketCable VoIP Services,” tech. rep., Bell Laboratories, Lucent Technologies, 2013. pages 48, 129 [43] R. Munadi, I. H. Santoso, and A. Mulyana, “Performance Evaluation for VoIP on Campus,” INTERNATIONAL JOURNAL OF COMPUTERS & TECHNOLOGY, vol. 10, no. 9, pp. 2027–2035, 2013. pages 48, 53 [44] G. Combs, “Wireshark,” 2013. pages 48 [45] A. Chan and S.-C. Liew, “Performance of VoIP over Multiple Co-Located IEEE 802.11 Wireless LANs,” Mobile Computing, IEEE Transactions on, vol. 8, pp. 1063–1076, Aug 2009. pages 48 [46] D. Wang, X. Li, and J. McNair, “An evaluation of the performance advantage of DiffServ with respect to voice over IP ﬂows,” in Military Communications Conference, 2009. MILCOM 2009. IEEE, pp. 1–7, Oct 2009. pages 49 [47] G. Kambourakis, D. Geneiatakis, S. Gritzalis, C. Lambrinoudakis, T. Dagiuklas, S. Ehlert, and J. Fiedler, “High Availability for SIP: Solutions and Real-Time Measurement Performance Evaluation.,” International Journal of Disaster Recovery & Business Continuity, vol. 1, no. 1, 2010. pages 49 [48] V. P. Koutras and A. Platis, “VoIP Availability and Service Reliability through Software Rejuvenation Policies,” in Dependability of Computer Systems, 2007.

154 DepCoS-RELCOMEX ’07. 2nd International Conference on, pp. 262–269, June 2007. pages 49 [49] K. Pandit, Quality of Service Performance Analysis based on Network Calculus. PhD thesis, Darmstadt, 2006. pages 50, 101 [50] V. K. Gurbani, L. J. Jagadeesan, and V. B. Mendiratta, “Characterizing session initiation protocol (SIP) network performance and reliability,” in Service Availability, pp. 196–211, Springer, 2005. pages 50, 53 [51] S. Garg, Y. Huang, C. Kintala, K. Trivedi, and S. Yajnik, “Performance and reliability evaluation of passive replication schemes in application level fault tolerance,” in Fault-Tolerant Computing, 1999. Digest of Papers. Twenty-Ninth Annual International Symposium on, pp. 322–329, June 1999. pages 50 [52] R. Krishnamurthy and G. Rouskas, “Evaluation of SIP proxy server performance: Packet-level measurements and queuing model,” in Communications (ICC), 2013 IEEE International Conference on, pp. 2326–2330, June 2013. pages 51, 53 [53] M. Ahmed and A. M. Mansor, “CPU Dimensioning on Performance of Asterisk VoIP PBX,” in Proceedings of the 11th Communications and Networking Simula- tion Symposium, CNS ’08, (New York, NY, USA), pp. 139–146, ACM, 2008. pages 51, 54 [54] C. P. Wright, E. M. Nahum, D. Wood, J. M. Tracey, and E. C. Hu, “SIP Server Performance on Multicore Systems,” IBM J. Res. Dev., vol. 54, pp. 79–90, Jan. 2010. pages 51, 54 [55] J.-S. Wu and P.-Y. Wang, “The performance analysis of SIP-T signaling system in carrier class VoIP network,” in Advanced Information Networking and Applications, 2003. AINA 2003. 17th International Conference on, pp. 39–44, March 2003. pages 51 [56] N. Griffeth, R. Hao, D. Lee, and R. Sinha, “Interoperability testing of VoIP systems,” in Global Telecommunications Conference, 2000. GLOBECOM ’00. IEEE, vol. 3, pp. 1565–1570 vol.3, 2000. pages 52 [57] Telcordia, “Call Processing Generic Requirements,” tech. rep., TEL Telcordia, http://telecom-info.telcordia.com/site- cgi/ido/docs.cgi?ID=110226225SEARCH&DOCUMENT=GR-505;, December 2006. pages 52 [58] A. Markopoulou, G. Iannaccone, S. Bhattacharyya, C.-N. Chuah, and C. Diot, “Characterization of failures in an IP backbone,” in INFOCOM 2004. Twenty-third AnnualJoint Conference of the IEEE Computer and Communications Societies, vol. 4, pp. 2307–2317 vol.4, March 2004. pages 52

155 [59] R. Krishnamurthy and G. N. Rouskas, “Performance Evaluation of Multi-Threaded SIP Servers,” pages 54 [60] H. Schulzrinne, S. Narayanan, J. Lennox, and M. Doyle, “SIPstone - benchmark- ing SIP server performance,” tech. rep., 2002. pages 54 [61] F. Lipson, “Veriﬁcation of Service Level Agreements with Markov Reward Models,” in South African Telecommunications Networks and Applications Conference, pp. 1–7, 2003. pages 55 [62] C. Systems, “Cisco FL VOIP Price List - Industry Solutions.” Online, October 2013. pages 55 [63] Shoretel, “IP telephony: Reliability you can count on,” white paper, 2009. pages 56 [64] Shoretel, “ShoreTel Pricelist.” Online, October 2013. pages 56 [65] Microsoft, Lync Licensing Guide November 2013 Customer and Partner. Microsoft, November 2013. pages 57, 63 [66] Microsoft, “Skype for Business,” 2015. pages 58, 61 [67] 8x8, “8x8, Inc.,” 2015. pages 58, 59 [68] Wikipedia, “8x8,” 2015. pages 58 [69] K. Hart, “For SunRocket Customers, Sounds of Silence,” Washington Post, 2007. pages 59 [70] D. Myers, “Cloud UC North America Service Provider Scorecard,” tech. rep., Infonetics Research, Inc., 2015. pages 59 [71] B. E. Daniel O’Connell, “Magic Quadrant for Uniﬁed Communications as a Service, Worldwide,” tech. rep., Gartner, 2015. pages 59 [72] E. Gately, “8x8 Speeds Up Cloud Migration With New Services,” 2015. pages 59 [73] L. Stapleton, “Disaster Reliability¡xWhat Old PBXs and Phone Systems Can¡Št Give You,” 2015. pages 59 [74] 8x8, “8x8 reliability and uptime,” 2015. pages 59 [75] 8x8, “8x8 Business Service Plans,” 2015. pages 59 [76] R. Pepper, “Vonage Business vs 8¡Ñ8 Comparison,” 2014. pages 59, 60 [77] Vonage Marketing LLC, “We Help You Build Your Own,” tech. rep., Vonage, 2015. pages 60, 61 [78] E. Oswald, “Hackers Can Tap Into Vonage Lines, Says Security Firm,” betanews, 2007. pages 61

156 [79] M. Vale, “Skype for Business NEW Differences between Lync 2013 and Skype for Business,” 2015. pages 62 [80] M. Branscombe, “A look at the new Microsoft Skype for Business Server 2015,” 2015. pages 62 [81] A. Burlacu, “Microsoft Shows Off New Enterprise-Class Features For Skype,” Tech Times, 2015. pages 62 [82] J. Schertz, “Polycom UCS 5.0 for VVX Phones,” 2013. pages 62 [83] “Microsoft Lync Server 2010 LICENSING GUIDE,” tech. rep., Microsoft, 2010. pages 63 [84] V. Perera, “Microsoft Skype For Business; Latest Features Detailed,” tech news today, 2015. pages 64 [85] J. P. Bruzzese, “What you need to know about Microsoft Skype for Business,” 2015. pages 64 [86] C. Williams, “The SBC and Its Role in Skype for Business,” 2015. pages 64 [87] M. Landis, “Feedback From the Field: Challenges & Solutions Using Lync Phone Edition in Very Heavy Enterprise Voice Scenarios,” 2012. pages 64 [88] M. Landis, “Q&A About Microsoft Lync Boss Admin and Shared Line Appearance,” 2013. pages 64 [89] P. Community, “VVX series - Lync Contact Group issue,” 2015. pages 64 [90] H. Sinnreich and A. B. Johnston, Internet Communications Using SIP: Delivering VoIP and Multimedia Services with Session Initiation Protocol. New York, NY, USA: John Wiley & Sons, Inc., 2001. pages 64, 75, 107 [91] J. Networks, “OnSIP.” online, 2015. pages 64, 65 [92] J. Networks, “Business VoIP: What to Consider,” tech. rep., OnSIP, 2015. pages 65 [93] J. Networks, “Hosted PBX: Why go hosted?,” tech. rep., OnSIP, 2015. pages 65 [94] K. Bartley, “Integrated Telephone System - The Latest Features of 2015,” 2015. pages 65 [95] K. Bartley, “New Feature: Stream External Music Files With Music On Hold While Your Customers Wait For Service,” 2015. pages 66 [96] K. Bartley, “Announcing Queue Video, Plus 3 More Queue Features!,” 2015. pages 66 [97] K. Bartley, “New Feature: Email Reports Now Available for Smart Queues,” 2015. pages 66

157 [98] voip info.org, “OnSIP Reviews,” June 2015. pages 66 [99] K. Bartley, “Top Notch Business VoIP Support, By The Numbers,” 2015. pages 66 [100] “2600hz Cloud Telecom,” 2014. pages 66, 68 [101] 2600hz, “High Performance Telecom, Section 1: Distributed BLF + Kamailio Module,” 2015. pages 67 [102] K. Anderson, “Kamailio and Kazoo,” 2015. pages 67 [103] 2600hz, “Carrier Dominance,” tech. rep., 2600hz, https://www.2600hz.com/docs/2600hz_CarrierDominance.pdf, 2015. pages 67, 68 [104] 2600hz, “SmartPBX Overview,” 2015. pages 68 [105] 2600hz, “2600hz News,” 2015. pages 68 [106] J. Rosenberg and H. Schulzrinne, “RFC 3262-Reliability of Provisional Responses in the Session Initiation Protocol (SIP),” June 2002. pages 71 [107] C. Holmberg, E. Burger, and H. Kaplan, “RFC 6086-Session Initiation Protocol (SIP) INFO Method and Package Framework,” June 2011. pages 71 [108] Alpine Linux Development Team, “About Alpine Linux,” October 2013. pages 78 [109] H. A. Latchman, N. Angelacos, and N. Copa, “Enterprise VOIP solutions with alpine Linux,” 2011. pages 78 [110] Kamailio SIP Server Project, “Kamailio,” October 2013. pages 79 [111] FreeSWITCH, “FreeSWITCH Home Page,” October 2013. pages 80 [112] P. Park, Voice over IP Security: Networking Technology: IP Communications. Cisco Press, 1 ed., 2008. pages 82, 90 [113] A. Kostur, “DHCP Options in Plain English,” 2013. pages 85 [114] “RTPProxy,” 2003. pages 91 [115] sobomax, “About RTPproxy,” Feb 2013. pages 91 [116] IBM, Google, E. Biederman, D. Lezcano, and S. Hallyn, “LXC Introduction,” 2008. pages 98 [117] D. Strauss, “Containers¡xNot Virtual Machines¡xAre the Future Cloud,” Linux Journal, vol. http://www.linuxjournal.com/content/containers2013. pages 99 [118] I. 8x8, “Conference Bridge,” 2015. pages 118 [119] J. Networks, “Simple Pricing,” 2015. pages 118

158 [120] Microsoft, “Skype for Business Online Limits,” 2015. pages 118 [121] Microsoft, “Plan for Group Call Pickup in Skype for Business 2015,” 2015. pages 118 [122] F. community, “Real-world results,” 2013. pages 119 [123] CommuniGate, “Telco VoIP Scalability Test Results,” tech. rep., Communi- Gate, https://www.communigate.com/Papers/CommuniGate-Superdome-VoIP- Benchmark.pdf, 2006. pages 119 [124] H. Packard, “HP Integrity Superdome,” Tech. Rep. Revision 5.8, Hewlett Packard, http://c970058.r58.cf2.rackcdn.com/individual_results/HP/tpcc.hp.SD.es.082107.pdf, February 2007. pages 119 [125] “One-way transmission time,” May 2003. pages 120 [126] D. Bertsekas and R. Gallager, Data Networks. Prentice-Hall international editions, Prentice Hall, 1992. pages 124 [127] T. Viswanathan and M. Bhatnagar, Telecommunication Switching Systems and Networks. PHI Learning, 2015. pages 130 [128] E. Altman, C. Barakat, and V. M. Ramos Ramos, “Queuing analysis of simple FEC schemes for voice over IP,” Computer Networks, vol. 39, pp. 185–206, June 2002. pages [129] F. Li, J. Yang, H. Zhang, S. Li, X. Wang, and J. Wu, “Evolution of network configurations: High-level analysis of an operational IP backbone network,” in Network Operations and Management Symposium (APNOMS), 2014 16th Asia-Pacific, pp. 1–4, Sept 2014. pages [130] W. Van Heddeghem, B. Lannoo, D. Colle, M. Pickavet, and P. Demeester, “A Quantitative Survey of the Power Saving Potential in IP-over-WDM Backbone Networks,” Communications Surveys Tutorials, IEEE, vol. PP, no. 99, pp. 1–1, 2014. pages [131] B. Amarasekara, A. Nirmalathas, and R. Evans, “Analysis of ip-based communication backbone over shared wide area-network for Smart Grid applications,” in Wireless Personal Multimedia Communications (WPMC), 2014 International Symposium on, pp. 601–606, Sept 2014. pages [132] D.-H. Shin, D. Moses, M. Venkatachalam, and S. Bagchi, “Distributed mobility management for efficient video delivery over all-IP mobile networks: Competing approaches,” Network, IEEE, vol. 27, pp. 28–33, March 2013. pages [133] F. Kocsis, A. Kurokawa, and J. Reilly, “Next-generation telco IT architectures and transformation to support service production and operation in all-IP (NGN)

159 networks,” Communications Magazine, IEEE, vol. 48, pp. 94–95, August 2010. pages [134] T. Szymanski and D. Gilbert, “Internet Multicasting of IPTV With Essentially-Zero Delay Jitter,” Broadcasting, IEEE Transactions on, vol. 55, pp. 20–30, March 2009. pages [135] K. Papagiannaki, S. Moon, C. Fraleigh, P. Thiran, and C. Diot, “Measurement and analysis of single-hop delay on an IP backbone network,” Selected Areas in Communications, IEEE Journal on, vol. 21, pp. 908–921, Aug 2003. pages [136] C. Fraleigh, S. Moon, B. Lyles, C. Cotton, M. Khan, D. Moll, R. Rockell, T. Seely, and S. Diot, “Packet-level trafﬁc measurements from the Sprint IP backbone,” Network, IEEE, vol. 17, pp. 6–16, Nov 2003. pages [137] J. Stanek and L. Kencl, “SIPp-DD: SIP DDoS Flood-Attack Simulation Tool,” in Computer Communications and Networks (ICCCN), 2011 Proceedings of 20th International Conference on, pp. 1–7, July 2011. pages [138] I. 8x8, 8x8 Call Queuing. 8x8, Inc., call queuing administrator guide v 2.0 ed., February 2011. pages [139] 2600hz, “2600hz-Powerful Products,” 2015. pages [140] 2600hz, “Our Other Border Brother: Kamailio,” October 2012. pages [141] 8x8, “8x8 Virtual Ofﬁce Extension Pricing,” 2015. pages [142] Alpine Linux Development Team, “About Alpine Linux,” October 2013. pages [143] Y. Amir, C. Danilov, S. Goose, D. Hedqvist, and A. Terzis, “An Overlay Architecture for High-Quality VoIP Streams,” Multimedia, IEEE Transactions on, vol. 8, pp. 1250–1262, Dec 2006. pages [144] F. Andreasen and B. Foster, “RFC3435: Media Gateway Control Protocol (MGCP),” January 2003. pages [145] Asterisk, “Asterisk Home Page,” October 2013. pages [146] E. Basart, “Building reliable IP telephony systems,” tech. rep, ShoreTel, Oct 2006. pages [147] E. H. Ben Charny, “Court’s call: Hands off VoIP.” online, 2003. pages [148] S. Bera, S. Misra, and J. Rodrigues, “Cloud Computing Applications for Smart Grid: A Survey,” Parallel and Distributed Systems, IEEE Transactions on, vol. 26, pp. 1477–1494, May 2015. pages [149] 2600hz Blog, “High Performance Telecom, Section 1: Distributed BLF + Kamailio Module,” 2015. pages

160 [150] D. Bushmitch, W. Lin, A. Bieszczad, A. Kaplan, V. Papageorgiou, and A. Pakstas, “A SIP-based device communication service for OSGi framework,” in Consumer Communications and Networking Conference, 2004. CCNC 2004. First IEEE, pp. 453–458, Jan 2004. pages [151] I. L. Cincunegui, Quality of Service for VoIP in Wireless Communications. PhD thesis, Newcastle University, 2011. pages [152] G. Combs, “Wireshark,” 2013. pages [153] L. N. Corporation, “How Does Hosted VoIP Compare to Vonage Voice over IP?,” 2008. pages [154] D. S. Dash, A. Durresi, and R. Jain, “Routing of VoIP traffic in multilayered satellite networks,” 2003. pages [155] D. S. Dash, A. Durresi, and R. Jain, “Routing of VoIP traffic in multilayered satellite networks,” 2003. pages [156] J. DiAdamo, “SIP: The Clear Choice for Smart Grid Communications,” June 2009. pages [157] W. Engineers, “What is an Erlang.” http://www.erlang.com/whatis.html, January 2014. pages [158] P. Faltstrom, “The international public telecommunication numbering plan,” September 2000. pages [159] Z. Fan, P. Kulkarni, S. Gormus, C. Efthymiou, G. Kalogridis, M. Sooriyaban- dara, Z. Zhu, S. Lambotharan, and W. H. Chin, “Smart Grid Communications: Overview of Research Challenges, Solutions, and Standardization Activities,” Communications Surveys Tutorials, IEEE, vol. 15, pp. 21–38, First 2013. pages [160] B. R. Flynn, “Key Smart Grid Applications,” tech. rep., GE Energy, 2009. pages [161] FreeSWITCH, “FreeSWITCH Home Page,” October 2013. pages [162] FreeSWITCH Core Team, “How does FreeSWITCH compare to Asterisk?,” 2008. pages [163] F. E. Goncalves, Building Telephony Systems with OpenSER: A Step-by-step Guide to Building a High Performance Telephony System. Packt Publishing, 2008. pages [164] D. Greenfield, “Open Source VoIP: Asterisk or FreeSwitch?,” 2008. pages [165] J. Guilbault, “8x8 Wins Communications Solutions Product of the Year Award for Virtual Office Analytics,” New York Times, 2015. pages

161 [166] S. Haryadi and F. Niramaya, “Study of unfair competition between regulated and unregulated VoIP providers in the mixed of non and all-IP network era,” in Telecommunication Systems Services and Applications (TSSA), 2014 8th International Conference on, pp. 1–5, Oct 2014. pages [167] J. Hassell, “A look at the new Microsoft Skype for Business Server 2015,” 2015. pages [168] R. Horak, Telecommunications and Data Communications Handbook. Wiley- Interscience, 2007. pages [169] JabRef Development Team, JabRef, 2015. pages [170] M. Jack, “Magic Jack Home Page.” Website, October 2013. pages [171] J. Janitor, “Efﬁcient VoIP Solution for the Environment of the Technical University of Kosice,” Master’s thesis, Technical University of Kos£Ÿice, 2008. pages [172] B. Jansen, C. Binding, O. Sundstrom, and D. Gantenbein, “Architecture and Communication of an Electric Vehicle Virtual Power Plant,” in Smart Grid Communications (SmartGridComm), 2010 First IEEE International Conference on, pp. 149–154, Oct 2010. pages [173] G. Jayadevan, V. K. Gurbani, and R. M. Arlein, “Adapting SIP for application server requirements in 3G networks,” Bell Labs Technical Journal, vol. 9, no. 3, pp. 57–71, 2004. pages [174] S. Jelassi, G. Rubino, H. Melvin, H. Youssef, and G. Pujolle, “Quality of Experi- ence of VoIP Service: A Survey of Assessment Approaches and Open Issues,” Communications Surveys Tutorials, IEEE, vol. 14, pp. 491–513, Second 2012. pages [175] W. Jiang and H. Schulzrinne, “Assessment of VoIP Service Availability in the Current Internet,” 2003. pages [176] W. Jiang and H. Schulzrinne, “Assessment of voip service availability in the current internet,” in Proceedings of the 4th International Workshop on Passive and Active Network Measurement (PAM 2003), 2003. pages [177] C. Johnson, Y. Kogan, Y. Levy, F. Saheban, and P. Tarapore, “VoIP reliability: a service provider’s perspective,” Communications Magazine, IEEE, vol. 42, pp. 48–54, July 2004. pages [178] Kam, “Kamailio Modules - v4.0.x (stable),” 2013. pages [179] Kam, “Kamailio Modules - v4.0.x (stable),” 2013. pages [180] Kamailio SIP Server Project, “Kamailio,” October 2013. pages

162 [181] J. Y. Kim, G. W. Bond, E. Cheung, T. M. Smith, and H. Schulzrinne, “An Evaluation Framework for Highly Available and Scalable SIP Server Clusters,” in Proceedings of the 5th International Conference on Principles, Systems and Applications of IP Telecommunications, IPTcomm ’11, (New York, NY, USA), pp. 1:1–1:10, ACM, 2011. pages [182] Y.-J. Kim, M. Thottan, V. Kolesnikov, and W. Lee, “A secure decentralized data- centric information infrastructure for smart grid,” Communications Magazine, IEEE, vol. 48, pp. 58–65, November 2010. pages [183] V. P. Koutras, C.-P. Salagaras, and A. Platis, “Software Rejuvenation for Higher Levels of VoIP Availability and Mean Time to Failure,” in Dependability of Computer Systems, 2009. DepCos-RELCOMEX ’09. Fourth International Conference on, pp. 99–106, June 2009. pages [184] M. Landis, “Lync Qualiﬁed/Comptabile and Optimized IP Phone Features Compared: Which Devices are More Advanced?,” 2013. pages [185] H. A. Latchman, N. Angelacos, and N. Copa, “Enterprise VOIP solutions with alpine Linux,” 2011. pages [186] D. Li and S. Jayaweera, “Distributed Smart-Home Decision-Making in a Hierarchi- cal Interactive Smart Grid Architecture,” Parallel and Distributed Systems, IEEE Transactions on, vol. 26, pp. 75–84, Jan 2015. pages [187] X. Liu and H. Cao, “A Study of Smart Grid Communication Architecture,” in Proceedings of International Conference on Computer Science and Information Technology (S. Patnaik and X. Li, eds.), vol. 255 of Advances in Intelligent Systems and Computing, pp. 189–196, Springer India, 2014. pages [188] C.-W. Lu, S.-C. Li, and Q. Wu, “Interconnecting ZigBee and 6LoWPAN wireless sensor networks for smart grid applications,” in Sensing Technology (ICST), 2011 Fifth International Conference on, pp. 267–272, Nov 2011. pages [189] C.-W. Lu, S.-C. Li, and Q. Wu, “Interconnecting ZigBee and 6LoWPAN wireless sensor networks for smart grid applications,” in Sensing Technology (ICST), 2011 Fifth International Conference on, pp. 267–272, Nov 2011. pages [190] E. Marcus and H. Stern, Blueprints for High Availability: Designing Resilient Distributed Systems. New York, NY, USA: John Wiley & Sons, Inc., 2000. pages [191] V. Marketing, “Business Phone Solutions Buyers Guide,” tech. rep., Vonage, 2015. pages [192] M. Mealling, “RFC 3403-Dynamic delegation discovery system (DDDS) part three: The domain name system (DNS) database,” October 2002. pages [193] Microsoft, “IP Phones Lync Tested Features Comparison,” 2015. pages

163 [194] A. Minessale, D. Schreiber, and M. S. Collins, FreeSWITCH 1.0.6. Packt Publishing, 2010. pages [195] D. Minoli and E. Minoli, Delivering Voice over IP Networks. New York, NY, USA: John Wiley & Sons, Inc., 1998. pages [196] D. Niyato, L. Xiao, and P. Wang, “Machine-to-machine communications for home energy management system in smart grid,” Communications Magazine, IEEE, vol. 49, pp. 53–59, April 2011. pages [197] U. G. A. Ofﬁce, “Telephone Communications: Bypass of the Local Telephone Companies,” August 1986. pages [198] U. G. A. Ofﬁce, “Telephone Communications: Bypass of the Local Telephone Companies,” August 1986. pages [199] S. Pal, R. Gadde, and H. A. Latchman, “On the reliability of Voice over IP (VoIP) telephony.” The SPRING 9th International Conference on Computing, Communications and Control Technologies, March 2011. pages [200] S. Pal, R. Gadde, and H. A. Latchman, “On the reliability of Voice over IP (VoIP) telephony.” The SPRING 9th International Conference on Computing, Communications and Control Technologies, March 2011. pages [201] A. Patel, J. Aparicio, N. Tas, M. Loiacono, and J. Rosca, “Assessing communications technology options for smart grid applications,” in Smart Grid Communica- tions (SmartGridComm), 2011 IEEE International Conference on, pp. 126–131, Oct 2011. pages [202] D. Patnaik, A. Bijlani, and V. Singh, “Towards high-availability for IP telephony using virtual machines,” in Internet Multimedia Services Architecture and Application(IMSAA), 2010 IEEE 4th International Conference on, pp. 1–6, Dec 2010. pages [203] I. Performance Enhancements, “Microsoft Lync 2013 ¡v How to Park Calls and Retrieve Them,” 2013. pages [204] J. Peterson, “RFC 3953-Telephone number mapping (ENUM) service registration for presence services,” January 2005. pages [205] V. Prasad, R. Hurni, and H. Jamadagni, “A scalable distributed VoIP conferencing using SIP,” in Computers and Communication, 2003. (ISCC 2003). Proceedings. Eighth IEEE International Symposium on, pp. 608–613 vol.1, June 2003. pages [206] K. S. S. Project, “Kamailio History,” 2015. pages [207] R. Qiu, Z. Chen, N. Guo, Y. Song, P. Zhang, H. Li, and L. Lai, “Towards a Real- Time Cognitive Radio Network Testbed: Architecture, Hardware Platform, and

164 Application to Smart Grid,” in Networking Technologies for Software Deﬁned Radio (SDR) Networks, 2010 Fifth IEEE Workshop on, pp. 1–6, June 2010. pages [208] F. Y. Rashid, “Skype for Business,” 2015. pages [209] O. J. Richard GAYRAUD, “SIPp.” online, October 2013. pages [210] O. J. Richard GAYRAUD, “SIPp.” online, October 2013. pages [211] J. Rosenberg, “RFC 5411 -A Hitchhiker’s Guide to the Session Initiation Protocol (SIP),” January 2009. pages [212] A. Sabbah, A. El-Mougy, and M. Ibnkahla, “A Survey of Networking Challenges and Routing Protocols in Smart Grids,” Industrial Informatics, IEEE Transactions on, vol. 10, pp. 210–221, Feb 2014. pages [213] H. Schulzrinne, “RTP Proﬁle for Audio and Video Conferences with Minimal Control,” July 2003. pages [214] K. Sheridan, “Skype For Business: New Features Ready To Preview,” 2015. pages [215] Shoretel, “ShoreTel Pricelist ¡v Peppm.” Online, October 2013. pages [216] S. S. Sreenadh, R. Depuru, L. Wang, and V. Devabhaktuni, “Smart meters for power grid: Challenges, issues, advantages and status,” Renewable and Sustainable Energy Reviews, vol. 15, no. 6, pp. 2736 – 2742, 2011. pages [217] S. Subramanian and R. Dutta, “Comparative Study of M/M/1 and M/D/1 Models of a SIP Proxy Server,” in Telecommunication Networks and Applications Conference, 2008. ATNAC 2008. Australasian, pp. 397–402, Dec 2008. pages [218] D. Sun, J.-P. Joseph, F. R. Magee, Jr., A. Mukhopadhyay, and B. Tang, “A SIP- enabled all-IP architecture for converged next-generation networks,” Bell Labs Technical Journal, vol. 9, no. 3, pp. 15–37, 2004. pages [219] C. Systems, “Cisco FL VOIP Price List - Industry Solutions.” Online, October 2013. pages [220] C. A. Thompson, H. A. Latchman, N. Angelacos, and B. K. Pareek, “A Distributed IP-Based Telecommunication System using SIP,” CoRR, vol. abs/1312.2625, 2013. pages [221] S. Tomic and P. Todorova, “SIP meets ZigBee,” in Mobile and Wireless Communi- cations Summit, 2007. 16th IST, pp. 1–5, July 2007. pages [222] VoIPUSA.com, “History of Voice over IP Phones and VoIP Technology.” online, 2008. pages [223] Vonage, “Vonage Home Page.” http://www.vonage.com/, October 2013. pages

165 [224] vopium, 2011. pages [225] J. Wang and V. Leung, “A survey of technical requirements and consumer application standards for IP-based smart grid AMI network,” in Information Networking (ICOIN), 2011 International Conference on, pp. 114–119, Jan 2011. pages [226] X. G. Wang, G. Min, and J. Mellor, “Improving VOIP application’s performance over WLAN using a new distributed fair MAC scheme,” in Advanced Information Networking and Applications, 2004. AINA 2004. 18th International Conference on, vol. 1, pp. 126–131 Vol.1, 2004. pages [227] Wikipedia, “IP Multimedia Subsystem,” October 2013. pages [228] Wikipedia, “IP Multimedia Subsystem,” October 2013. pages [229] H. Wook, S. Kang, and D. Kim, “Performance Enhancement of SIP proxy server by using lhash for matching transaction,” in Advanced Communication Technology, The 9th International Conference on, vol. 2, pp. 1290–1293, Feb 2007. pages [230] M. Xiang, S. Tauch, and W. Liu, “Dependability and Resource Optimation Analysis for Smart Grid Communication Networks,” in Big Data and Cloud Computing (BdCloud), 2014 IEEE Fourth International Conference on, pp. 676–681, Dec 2014. pages [231] A. Zaballos, A. Vallejo, and J. Selga, “Heterogeneous communication architecture for the smart grid,” Network, IEEE, vol. 25, pp. 30–37, September 2011. pages [232] “SIP - Open Communications for Smart Grid Devices,” tech. rep., Siemens Enterprise Communications, June 2009. pages [233] “High Availability for SIP: Solutions and Real-Time Measurement Performance Evaluation,” International Journal of Disaster Recovery and Business Continuity, vol. 1, p. 20, 2010. pages [234] “NIST Framework and Roadmap for Smart Grid Interoperability Standards Release 1.0,” September 2009. pages [235] X. Liu and H. Cao, “A study of smart grid communication architecture,” in Proceedings of International Conference on Computer Science and Information Technology (S. Patnaik and X. Li, eds.), vol. 255 of Advances in Intelligent Systems and Computing, pp. 189–196, Springer India, 2014. pages

166 BIOGRAPHICAL SKETCH Carlton Thompson completed BS and MS degrees at the University of Florida in the Department of Electrical and Computer Engineering. He completed his PhD in May 2016 at the University of Florida in the Department of Electrical and Computer Engineering. Previous research was done using Electrical Impedance Tomography (EIT) for the development of a system for capturing and reconstructing impedance images of the head, and extracting information about bleeding-related impedance changes from background noise. Results gathered from two and three-dimensional ﬁnite element models of the head will be used to test reconstruction methods on a skull model. He is currently a Research Assistant in the Laboratory for Information Systems & Telecommunications (LIST). Current work is on development of a completely distributed IP-based Telecommunication System (D-IPTS) and the performance of such a system. His research interests lies in IP telecommunications, open-source software, cloud systems, and networking.

167