Real-World IPTV Network Measurements

Georgios Baltoglou Eirini Karapistoli Periklis Chatzimisios School of Technology and Health Department of Electronics Department of Informatics KTH, Stockholm, Sweden Alexander TEI of Thessaloniki, Greece Alexander TEI of Thessaloniki, Greece [email protected] [email protected] [email protected]

Abstract—Internet Protocol Television or IPTV is defined as while an over-provisioned network might be expensive to an alternate method of distributing television content over IP by deploy or have extravagant leasing costs. Finally, throughout a telecom carrier or an Internet service provider (ISP) and it measurement data the optimal time for upgrade and the extent is increasingly gaining market share in modern communication networks. IPTV poses specific requirements on the network in- of the upgrade can be decided. frastructure and at the same time it requires that several network The relation between the service providers or telcos, and the performance characteristics are maintained under defined levels end users can be summarized by what is called Service-Layer in order for the end user to have an assured viewing experience. Agreement or SLA. The SLA is a form of contract containing In this paper, our main motivation is to investigate how a real the responsibilities of both sides and the definition of the level life network performs in terms of distributing this inelastic and high-bandwidth type of service utilizing traffic measurements. of service that has to be provided by the telcos and experienced Furthermore, our goal is to verify and analyze whether the by the subscribed users. Again, measurements are of use.In studied network is suited for multicast IPTV traffic. order for the users to be able to observe the compliance of the operator to the agreed level of provided service. In contrast I. INTRODUCTION to telcos, users are rarely interested in quantitative network The latest entry into the television programming industry are performance metrics (i.e. packet loss rate, packet delay, delay telecommunication companies or telcos - which have started variation, etc.). What is of interest to them is the application providing television content over the internet commonly re- performance, a set of qualitative measurements that perplex the ferred to as IPTV. IPTV is a service with great expectations. QoS measured by the operators with the overall experience of It thrives to replace the former methods of TV broadcasting the end user. Modern traffic measurement techniques directly such as over the air, cable or satellite TV distribution, making provide QoE results, allowing the users to evaluate if their use of the packet-switched Internet infrastructure. In order current level of service is in par with their expected level of to succeed, it also provides a variety of new services such experience. as Video-on-Demand (VoD) and VCR-like functionality for This paper is organized as follows. In Section II, we the distributed TV media, together with the ability to co-exist discuss the work that has already been accomplished and with high rate data surfing and VoIP traffic in what is called the contribution of our paper. In Section III we describe a triple-play service. Yet, the highly maladaptive to network the architecture of the studied real-world, commercial IPTV deficiencies IPTV streams pose great demands in all separate network. In Sections IV and V we provide information on layers of the network architecture. why active measurements are ideally suited for IPTV traffic In modern network engineering problems dealing with along with a description of the equipment used for holding inelastic traffic, such as IPTV, the efficient use of network the measurement sessions, as well as the metrics that we resources that satisfy the expected levels of Quality of Service specifically focus on to evaluate the network’s performance in (QoS) is a major issue. At the same time, Quality of Experi- IPTV deployment. Finally, Section VI illustrates the obtained ence (QoE) or how the QoS metrics are perceived by the end real-time measurement results, accompanied by evaluation users is also of vital importance. In order to respond to that reports. The conclusion of the paper is found in Section VII. and to characterize the performance of an IP network, traffic measurements are utilized, which play very important roles for II. IPTV MEASUREMENTS AND CONTRIBUTION both the ISPs, network and telecommunication operators, as well as for the end users and even the measurement equipment Obtaining traffic measurements, especially for IPTV, inflicts vendors. certain problems that need to be researched: From the telcos’ perspective, network measurements pro- • Different types of network measurements exist, namely vide the means for network monitoring. As different important active and passive, each having its own advantages and metrics are gathered, such as traffic load and packet delay the disadvantages. As IPTV multicast traffic is peculiar and operators can identify the current state of the network and even demanding, not all traffic measurements are suited for predict eminent failures prior to happening. At the same time assessing the IPTV’s network performance. measurements are useful for network dimensioning. An under- • Real life networks resemble great heterogeneity. In terms provisioned network would result in dissatisfied customers of QoS implementation, multiple architectures exist such

& &

(4 56-)7

,)7 ,-- ,)7 *7 7 O2P ,-- 7 O$P ,7

Fig. 1. Studied Network Map

as ATM, MPLS, IPv4, IPv6, and others. As such, ob- III. STUDIED NETWORK taining measurements of only a specific network layer In this section the exact network architecture of the real-life provides little with regard to the end-user QoS/QoE. At commercial IPTV network, on which our measurements were the same time, end-to-end measurements including totally conducted. A full network map was designed for this reason, unknown portions of a network provides poor information depicted in Fig. 1. The IPTV streams in the studied network for the subsequent analysis of the obtained data. are originating from Stockholm, Oslo and Copenhagen. The • While substantial work on traffic measurements and net- transportation and interconnection of these cities is provided work performance concerning IPTV has been made ([1] by the network provider Norrsken. Norrsken uses optical fibers and [2]), they focus on P2P traffic and passive mea- to connect several cities within Sweden and in Scandinavia surements, or they are usually produced via simulation in general, in an SDH ring topology. They use STM-16 software ([3] and [4]), mathematical modeling [5], or transmission frames, yielding a bitrate of 2.4 Gbps. The IPTV studying proof of concept lab networks, which have a streams transported to the city Borlänge are by the service more predictable behavior [6]. providers Canal-Digital and FastTV. Canal-Digital is transmit- • Studying only the performance of a high-level QoS ting multicast IPTV streams with 700 Mbps bandwidth, while priority class, as given in IPTV multicast traffic, is usually FastTV uses 500 Mbps. not enough for the end-user expectations. As IPTV is marketed in triple play bundles, including VoIP and data A. Metro / Core traffic, successive studies on these lower priority classes The Metro/Core layer is a Gigabit Ethernet network over are important, since they might be affected by the high optical fibers interconnecting cities Borlänge (where the IPTV priority class. stream is firstly received) and Falun. The link has a bandwidth of 10 Gbps and on its edges lie Cisco 7600 Series routers. From the above analysis, it is evident that no compre- These routers were specifically chosen when the network hensive study that we know of, has so far presented active was designed because they deploy high-performance Carrier measurements of a real-world commercial IPTV network. As Ethernet, IP/MPLS features and Enterprise WAN aggregation such, this is our work’s main contribution. Furthermore, with and core routing over 10 Gbps interfaces. Additionally, they the obtained results of our measurement scenarios and their support interface processors capable of controling voice, video, subsequent analysis we intend to: and data, offering a rich set of QoS features as specifically needed. Moreover, the Metro/Core routers have been configured with MPLS creating a QoS-guaranteed network. a provide tangible information and present the reasons why a network architecture as the one studied, verifies as being B. Distribution an exemplary IPTV distribution infrastructure. The Distribution network is composed of 1 Gbps Ethernet b be able to pinpoint the parts of the network that are links carried over optical fiber. The Distribution network sources of problems for IPTV delivery. situated in Falun consists of five redundantly interconnected c use them as a reference point for evaluating other net- Cisco 3750 ME switches featuring hierarchical QoS, traffic works and assessing the distributed IPTV service quality. shaping, intelligent 802.1Q tunneling, VLAN mapping and

978-1-4577-0681-3/11/$26.00 ©2011 IEEE 831 MPLS. Spanning Tree Protocol is further implemented to V. M ETRICS OF INTEREST prevent switching loops. All provided services are divided Due to the fundamental design of IP networks as a best- into different VLANs and each VLAN supports different QoS effort delivery platform independent of application require- classes. The highest priority class is given to the IPTV VLAN ments, packets and data are susceptible to loss. For the inelas- followed by VoIP, Management and finally the Internet VLAN. tic nature of the IPTV service, packet losses are a major issue. As mentioned, IPTV deployments commonly transport packets C. Access via UDP, providing no protection to packet loss. Packet loss of the audio stream can be exhibited as dropouts, squeaking On the Falun Access network the nodes are connected noises, variations in sound volume, and skipping. For the video using 100 Mbps Ethernet links, originating from cabinets that stream, the impact is varying, depending on the video frame support 350 customers each. As the Distribution network links that was affected. The effects result from a mild pixelization are optical fibers, the Access network can be considered as on the viewed content to blocked pixels for a duration of a a Fiber-To-The-Node Access network. Having Ethernet for few frames, unnecessary elongation or repetition of frames last mile connection poses great advantages, in comparison (stuttering), frame freezes, or in worst case, freezing of the to other technologies such as DSL (i.e. higher bandwidth, set-top-boxes (STBs). simplified network architecture, utilization VLANs as the In contrast to the other metrics that follow, measuring loss service delivery mechanism over the entire access-network, over long term periods and analyzing the average is inappro- etc.). The Access network supports currently 2000 clients, out priate. To exemplify, an impulse noise that usually occurs in of which approximately 400 have also VoIP services enabled. bursts of 10 to 50 milliseconds, would be hidden in a sample The providers of CPE devices are Allied Telesis and Packet- of one hour as a 0.001% packet loss. While the operator Front. Each Ethernet port on the CPE connects the client to would pay no attention to such a value identifying it as a good the respective VLANs of IPTV, VoIP and Data traffic. The measurement, the end-user might in fact have an undesirable Ethernet link from the CPE is directly connected to a specific viewing experience. As such, packet loss measurement should port on the (Allied Telesis) Distribution Switch. Finally, all be conducted with regard to both the frequency of occurrences IPTV subscribers use Motorola STBs for the reception of the over a time period and of the occurring duration. IPTV services as it is used by both Canal-Digital and FastTV. A. Latency and Jitter IV. ACTIVE MEASUREMENTS AND EQUIPMENT Latency, or the time it takes for packets to traverse from the source to the destination is another parameter that affects the The approach of active measurements requires inserting test IPTV content. High latency values result in corrupted images, packets into the network for measurements purpose. As the picture blocking and frozen frames on the TV sets of the end generated traffic will traverse the same network infrastructure, users. Due to the fact that the packet propagation delay is not it will eventually be affected by it, with the same manner that constant over time, latency across a network is varying. The the real traffic is. Introducing artificial (emulated) packets to metric that measures this variability is named packet delay the network provides the advantage of having absolute control variation or jitter. As the IPTV decoders and STBs require of the nature of the traffic parameters. As such, parameters can a steady IP stream, jitter can cause both buffer overrun and be adjusted to have traffic of an exact amount of volume, gen- underrun. In the first case the end users might see pixelization eration intervals, sampling frequency, scheduling, packet sizes of the viewed content or horizontal/vertical lines, while in the and types. Yet, active measurements do create extra traffic as latter case, frame freezing is the most usual experienced issue. the packets are inserted into the network. If the extra generated volume is too high, problems arise, appointing the obtained B. Join and Leave Delay measurements inaccurate. Despite that, active measurements In order to assess the performance of zapping, channel join remain the ideal type of measurements for the inelastic IPTV and leave delay metrics are used. Join delay is defined to be the multicast traffic. As such, unlike the passive measurements that elapsed time from issuing the IGMP join request until arrival need to capture all types of packets, including packets that do of the first packet in the multicast packet stream. The join not need to be monitored, active measurements can generate operation will be said to fail if no packet has arrived when the traffic to reflect only the data of the studied application service. time has come to issue the IGMP Leave request. Accordingly, To conduct our active measurement study we used the leave delay is defined to be the elapsed time from issuing the Prosilient Technologies PTAnalyzer; a distributed probe-based IGMP leave request until reception of the last packet in the end-to-end Service Performance Management, QoS measure- multicast packet stream. These performance indicators dealing ment and SLA supervision system. It consists of one or more with channel changing are the ones that greatly affect user probes, one or more reflectors and the PTAnalyzer client, perceived QoS. Inherently, IPTV can not achieve changing which is a graphical user interface (GUI) application provid- of channels instantaneously as supported by OTA and cable ing the functionality to configure and control a supervision TV broadcasting, since the whole process of IPTV operation domain, retrieve and present status and result information in starting at encoding and multicasting and ending at STB layer graphical as well as textual formats. decoding involves a certain number of delays.

978-1-4577-0681-3/11/$26.00 ©2011 IEEE 832 C. MOS as simply annoying by end users. Even so, currently there is A subjective measurement indication is Mean Opinion Score no knowledge about the explicit relation between the channel (MOS), which ranks the perceived video and audio quality zapping time and the user experience (QoE) as expressed with based on user feedback. In MOS users determine the video Mean Opinion Score (MOS), but only approximate criteria quality by rating displayed video sequence on a scale of 1 are given for its evaluation, [9]. The PTAnalyzer client that (very bad) to 5 (very good). The mean averages of multiple controls the PT1404 probes, evaluates zap Join and Leave users are taken and so the MOS value is computed. As such, delay using the time margins shown in Table I. the MOS provides a numerical value for the QoE of the end VI. MEASUREMENTS METHODOLOGY AND RESULTS user. When comparing MOS scores it is of vital importance A. IPTV, VoIP and Data traffic to consider that certain video types inherently produce a higher level of quality than others, as viewers tend to form In order to conduct the measurement scenarios, our first expectations of quality based on the perceived capabilities of concern was to identify the exact parameters of the IPTV the medium. For example, HDTV delivers a higher resolution traffic that traverses the network. For this reason, Wireshark and picture size than regular SDTV, so maintaining all other packet sniffing software was used. From the captured packets factors identical, the MOS for a HD video stream will be the transported video streams were analyzed, which pro- higher than the MOS for the same sequence delivered in SD. vided us with the necessary information to construct identical traffic on the active probes. Specifically, the highest D. IPTV Quality margins bitrate channel transported was 641 packets per second with From the above descriptions of the QoS/QoE metrics, 1328 bytes per packet payload, yielding 6.854 Mbps including it is evident that IPTV quality is not equally affected by IP overhead.The MPEG TS originating from the source was them. As a result, defining its tolerance margins and exact encapsulated in UDP packets for transportation. The routers requirements is more than a difficult task. While QoS classes situated in both sites were configured with PIM-sparse mode have been standardized [7], IPTV is comprised by different of Multicasting. The additional channel traffic that traversed services, varying from simple streaming video to interactive the network was accordingly constructed on the probes using applications. As such, no standardization has been made so far the extracted bitrate, TTL and ToS values from the real traffic. providing the exact limits beyond which IPTV service quality As it is common, IPTV is usually provided in a triple- is not acceptable. Considering that PTV can be described as play bundle, and as such it was interesting to evaluate the very bandwidth demanding, very sensitive to packet loss, and performance of the VoIP and Internet traffic services on the simply sensitive to latency and jitter, and based on [8], some same network, identifying wether these services were affected margins for the QoS/QoE metrics can be derived. As such, by the IPTV traffic. In order to obtain VoIP and data traffic latency should be kept less than 200 ms, whereas PDV or measurements, in addition to that of IPTV, the probes were jitter that can deteriorate the video quality at greater extend connected to the respective VLANs and required additional should be kept under 50 ms. Packet loss for IPTV, that has the configuration was made. The measurement scenarios held, greatest impact in video quality should be kept below 10−4. were created with bi-directional data traffic transmissions, with To further specify, as losses are bound to happen in bursts, varying data-rates, and TCP protocol encapsulation, while the the bandwidth stream of the IPTV video should be related VoIP traffic was generated from probe A towards probe B, to the loss rate tolerance. Data from [8] provide packet loss using G711 codec with 64 Kbps bitrate, in accordance with rate values for streams of bitrates varying from 3 Mbps up to real VoIP traffic traversing the network. Measurements for the 5 Mbps. As a result, for the studied IPTV streams that average previous scenarios were obtained for fifteen consecutive days. above 6 Mbps (as later described), a loss rate tolerance of Having accurately synchronized probes is mandatory in 6.98x10−5 is accepted, with the maximum duration of every order for the obtained measurements to be valid. The graph single error not exceeding 16 ms. shown in Fig. 2a shows the synchronization reliability that the probes achieved, which was always maintained at very high Experience Join Delay Leave Delay levels, between 87 and 91%, translating to an accuracy in the Excellent 50 250 order of 50μsec. Good 150 750 Fair 300 1500 In the over-provisioned network of our study, packet losses Poor 500 2500 should be expected to be at very low levels. In our measure- Bad 1000 5000 ment scenarios, 830.598.268 packets were sent and all were received, yielding a 0% packet loss. The obtained results for TABLE I ZAP JOIN AND LEAVE EXPERIENCE MARGINS packet delay and jitter are presented in Table II. As shown and expected from the over-engineered network in study, packet delays are kept at very low levels. Even the maximum value Establishing margins of satisfactory performance for the of 8.125 ms is more than 95% lower than the margin of accept- “zapping” function, is more complicated. Generally, a time able quality, which is 200 ms. The very low observed values is less than 1 second can be safely called satisfactory, while a a direct outcome of the network architecture. By having a high value more than 2 seconds seems to exceed being characterized data-rate Ethernet last mile connection, mode conversions from

978-1-4577-0681-3/11/$26.00 ©2011 IEEE 833 Ethernet to ATM, common in DSL environments, are avoided circumvent possible QoE degrading issues, fast-leave has been and so is the potential additional delay that they may cause. implemented on the Access network switches. This important Jitter measurements show a great resemblance to packet delay, addition to the original IGMP protocol, that was included with the highest value obtained being 3.125 ms, or 93% lower in the second version of RFC2236, allows the switch to than the proposed upper margin of 50 ms. remove an interface from the forwarding-table entry without the need of previously sending out group specific queries to the IPTV VoIP interface. The VLAN interface is pruned from the multicast Metric Delay (μs) Jitter (μs) Delay (μs) Jitter (μs) tree for the multicast group specified in the original leave Min: 471 - 586 0 - 58 188 - 312 0 - 0 Mean: 554 - 667 111 - 125 214 - 333 1 - 17 message. Fast-leave processing ensures optimal bandwidth Median: 548 - 662 73 - 126 208 - 330 1 - 2 management for all hosts on a switched network, even when Max: 719 - 8185 232 - 3125 285 - 470 17 - 169 multiple multicast groups are in use simultaneously. This IGMP feature can only be used in VLANs where exactly one TABLE II IPTV AND VOIP DELAY AND JITTER RESULTS host is connected to each interface, as it is in our case. If In addition, neither medium sensitivity VoIP traffic, nor our network design had VLANs in which more than one host best effort data delivery delays were affected by the high was connected to a switch interface, fast-leave would cause the priority IPTV streams. Zero packet losses were also found hosts to be dropped inadvertently. With the fast-leave enabled, in all measurement sessions of the triple-play services, while our measurements in channel leave time are expected to be packet delay and jitter were also restricted within confined short. margins (Table II). While a lot measurement scenarios were held, they all had a Overall, and as a direct outcome of the previous metrics, common behavior concerning zap Leave delays. An indicative IPTV MOS values were constantly kept at 4.4, as presented graph is depicted in Fig. 2b. As observed zap Leave times were in Fig. 2a. This shows that statistically all the users receiving kept at very low levels. In most cases zap Leave delay was in the IPTV streams would be very satisfied with its quality and the vicinity of 100 ms, with very low values in the proximity of that the network can sufficiently handle the inelastic services 10 ms not being absent. These results illustrate that indeed fast- comprising the triple-play bundle. This was of course the leave IGMP snooping implementation played an important role initial purpose of building such an over-engineered network, in decreasing zapping Leave delay. These times could have with IPTV distribution being a primary concern. been even lower, if the probe was connected to a CPE that supported IGMP “fast-leave”. Additionally, the zapping Join B. HD Channel measurements delay showed great resemblance in the distribution uniformity. While at the time that the measurements were held, only However, while zapping leave times were sufficiently low for SD channels were being multicasted to IPTV subscribers, the channel leave experience to be characterized as “excellent”, we decided to generate some multicast traffic the size and the same cannot be said about the channel Join delays. properties of which would resemble a HD channel, verifying Evidently from Fig. 2c, Join times frequently surpassed 150 ms the fact that the network would have no problem supporting its yielding a “good” channel experience. Furthermore, there were distribution. For that reason, one HD channel was created on measurement sessions with a more peculiar behavior, such as probe A, with a bitrate of 14 Mbps. The measurement session the one depicted in Fig. 3a. Taking into account that 430 ms that we conducted had the same outcome: zero packet losses, of zapping time (zap Join plus Leave delays) translates to a mean delay of 555-675μs and a mean jitter of 112-131μs, a MOS value of 3.5, it is evident that delays of 1005 ms yielding an overall “excellent” QoS chart (a chart that shows and 492 ms, depicted in the aforementioned figure are simply the amount of time spent in the different QoS levels, in percent unsatisfactory. of the complete test period). It should also be noted that in few isolated measurement scenarios there were even Join failures, as depicted in Fig. 3b, in C. “Zapping” Measurements Scenarios which At 14:04:38.124 (UTC time) of our zap measurements The same configuration as in the previous scenarios was session, it is reported that no join packet is received. While used for the “zapping” measurements session. For this scenario this result cannot be considered poor due to the low number more channels had to be created using PTAnalyzer GUI, of cases that it happened, it mandates that certain things need originating from probe A. Again, the Wireshark output that to be examined to further understand and optimize zap Join was captured and analyzed in the previous case was used to delay. For instance, in our traffic measurement session, there replicate channels with parameters identical to the real IPTV is only one subscriber to the multicast channel, as only one channel characteristics. As a result five more channels were probe was to our availability. As we are zapping through our emulated with the same attributes and datarates ranging from generated channel list, Leave messages stop the stream at the 6.211 up to 6.737 Mbps. switches that are connected to the CPE, but the IGMP message In contrast to the first scenarios that were held for con- still travels upwards, reaching the metro router situated in the secutive days, we created several zapping sessions, and ob- city of Falun. At that point however, that router notifies the tained multiple different measurements ranging from 2 up to metro router in Borlänge that there are no subscribers for the 24 hours. As the network has been engineered in order to measurements multicast group, causing the multicast stream to

Fig. 3. a) Excessive Join delays Chart, b) Zapping Join failure and c) Overall QoS Join delay experience.

be stopped right at Borlänge. Because of that, when a new Join VII. CONCLUSIONS IGMP message is issued, it has to travel up to the latter router, In this paper, we investigated how a real life network in order to start receiving the multicast traffic. In reality, there performs in terms of distributing inelastic and high-bandwidth is a quite high probability that there more than one subscribers IPTV data utilizing active traffic measurements. The obtained per channel exist, meaning that the multicast traffic never stops results of this ideally suited IPTV distributing network can act at router located in Borlänge. This could actually be one reason as a benchmark for evaluating different network architectures for the good but not excellent zap Join values. in the same operations and identify problematic sources, To further locate points that could add delay to IPTV leading to problem-free network infrastructures and changing zapping, we can refer to all the interconnected devices of the way IPTV content is consumed. Fig. 1 in the studied network map. In case the Join delay is caused at the router, then the problem might be a PIM- REFERENCES SM multicast routing misconfiguration. Accordingly, if delay [1] T. Hossfeld and K. Leibnitz, “A qualitative measurement survey of is caused at the Distribution switches, then an IGMP snooping popular internet-based iptv systems,” June 2008, pp. 156–161. [2] Y. Won, M.-J. Choi, B.-C. Park, J. Hong, H.-W. Lee, C.-K. Hwang, and J.- misconfiguration might be causing it. Yet, these types of errors H. Yoo, “End-User IPTV Traffic Measurement of Residential Broadband would probably create more serious problems and far more Access Networks,” April 2008, pp. 95–100. frequent Join failures. As a result, the most probable point that [3] E. Shihab, F. Wan, L. Cai, A. Gulliver, and N. Tin, “Performance Analysis of IPTV Traffic in Home Networks,” 2007, pp. 5341 – 5345. further degrades channel zap times, are the CPE and STBs [4] D. Agrawal, M. Beigi, C. Bisdikian, and K.-W. Lee, “Planning and connected to it. Since a large number of prioritized packets Managing the IPTV Service Deployment,” in Integrated Network Man- traverse through the CPE, it is logical in case of very high agement, 2007, pp. 353–362. [5] D. Emir, E. Mohamed, and B. Ammar, “Network performance evaluation CPU utilization, for packets to be delayed, or even dropped. using traffic measurements,” 2004, pp. 523–526. At the same time we acknowledged the fact that continuous [6] L. Ciavattone, A. Morton, and G. Ramachandran, “Standardized Active zapping for a long time, stresses the STB to its limits, causing Measurements on a Tier 1 IP Backbone,” IEEE Communications, pp. 90–97, 2003. additional delays and freezes. [7] ITU-T Recommendation, “Network Performance Objectives for IP-Based The cumulative channels result of zapping Join experience Services,” ITU-T Y.1541 Recommendation, Feb 2003. [8] D. Forum, “Triple-play Services Quality of Experience (QoE) Require- is represented in Fig. 3c, illustrating that even in this over- ments,” Technical Report TR-126, Dec 2006. engineered network IPTV zap frailties still exist, limiting the [9] X. L. et al., “Channel Zapping in IP over Optical Two-layer Multicasting excellent zap channel Join experience under 5% and yielding for Large Scale Video Delivery,” Dec. 2007, pp. 1–4. an overall 91% Good zapping Join QoE.