DESIGN like alphabet soup, take comfort in the fact that you are not alone. Throughout FORUM this discussion, we’ll identify the various technical performance and compatibility issues an AV professional should consider when developing a streaming system.

What Is a Streaming Protocol? A streaming protocol is a transmission protocol used to deliver encoded video and audio data across networks from hardware encoders or media servers to hardware decoders or client software players operating on PCs, tablets, or . A variety of streaming protocols have been developed to provide the speed, accuracy, and compatibility that ensures moving pictures and audio will fulfill various platform or application requirements.

Streaming Protocols Work Together with Network Transport Protocols Streaming protocols rely on one of two network transport protocols to Understanding Streaming deliver audio and video data. Either the Transport Protocols Transmission Control Protocol – TCP or There’s more to ensuring streaming system compatibility the User Datagram Protocol – UDP is than using the H.264 compression standard used. Each offers different strengths to a streaming application: By Karl Johnson, Director of Product Marketing • TCP applies an active connection ne of the most overlooked and Previously, in AVSD Issue 3, we between the client and server, O misunderstood areas of AV examined a variety of compression delivering accurate, ordered data streaming is the topic of streaming codecs, each of which had been through use of an error-checking transport protocols. We’ll refer to them developed by different organizations to mechanism that recovers and resends as streaming protocols in this discussion. fulfill specific technical or commercial lost packets. TCP only supports unicast Streaming applications are transforming objectives. The H.264 standard clearly connections and the retransmission of the way organizations and individuals sets itself apart as the mainstream codec data can produce significant latency. communicate and entertain. The growth applied to many consumer and industrial • UDP uses a “best effort” in the variety of mobile devices, set top applications. However, the H.264 standard connectionless transmission service boxes, and media players has produced only defines how the compressed video that offers speed and minimal latency. a wide variety of streaming protocols data is decoded. Different streaming It does not guarantee accurate, that are used to deliver H.264 video protocols are used to deliver the audio ordered delivery of data, but supports between devices. This variety creates and video across different network multicast one-to-many applications. unique challenges for the AV professional environments for use on different devices. designing systems that can provide access A sample of these protocols include: Applied alone, neither TCP nor UDP to video anywhere while delivering unique RTP, RTMP, RTSP, MMS, HLS, and transport protocols can fulfill the following performance for specific applications. MPEG-2 TS. If this list of acronyms looks requirements that exist for reproducing

70 Design Forum AV SYSTEMS DESIGN Fall 2014 AVSD.com 71

avsd_fall2014_cc.indd 70 10/7/14 4:42 PM DESIGN FORUM

Figure 1 locations within an enterprise, but the bandwidth required to support streaming applications may be cost Hardware Encoder Hardware Decoder prohibitive and WANs rarely support Or Streaming Protocol Or Software Player multicast traffic. Network • The Internet – offers broad access to

Server Client virtually any location in the world, but like WLANs, end-to-end bandwidth Streaming Protocols deliver encoded audio and video between servers and clients. and QoS may not be satisfactory. Firewalls must be traversed and low latency transport will be a challenge. The use of a public network also motion video, which include: support high scalability on many LANs. introduces requirements for securing • Wireless LANs – WLANs provide broad data transmissions. • Sustaining consistent data rates that access to video from mobile devices, will support the decoding of live, but bandwidth and QoS can vary and While exceptions exist, the following moving pictures drop below levels required for streaming generalizations can be made concerning • Delivery of accurate, sequential data applications. The performance of a the best network transport protocol to use for image decoding typical WLAN will degrade as the in each of these network environments: • Supply of synchronizing data to number of wireless clients saturates support accurate AV decoding and when interference is experienced. • UDP will be ideal for use in applications • Varying latency requirements which • Wide Area Networks – WANs provide requiring low delay or multicast delivery, differ by application the opportunity to stream to distant particularly on LANs or WANs. • System scalability which can range from one to thousands of endpoints

Streaming protocols apply either TCP or UDP to fulfill these and many other application requirements on different types of devices and network environments.

Variety of Network Environments Private, public, wired, and wireless networks all offer varying bandwidth, latency, scalability, and Quality of Service – QoS. The performance offered by different streaming protocols will make them applicable for use on certain types of networks. A summary of the Extron and AV Streaming different network conditions that will be Extron offers two different types of AV streaming products to meet applications in corporate, encountered are identified here: education, government, and other commercial AV market segments: the SME 100 Series, based upon standard H.264 compression technologies; and the VN-Matrix Series, which utilizes • Local Area Networks – LANs can offer Extron’s high performance, low latency PURE3 codec. To complement these products, we offer abundant bandwidth, manageable entry-level network training courses online, at www.extron.com. For more advanced system designers, our instructor-led Emerging Technologies course offers a comprehensive review of Quality of Service – QoS, and an streaming technology, including compression, networking, streaming protocols, and deployment opportunity to support low latency considerations. Extron S3 product support specialists are also available throughout the AV streaming applications, but the user’s application system development process to provide pre-sale design assistance, product commissioning, must exist within the physical reach and other services to ensure your success and complete satisfaction with Extron products. of the LAN. Multicast services can

70 AV SYSTEMS DESIGN Fall 2014 AVSD.com Design Forum 71

avsd_fall2014_cc.indd 71 10/7/14 4:42 PM DESIGN FORUM

• TCP offers reliable transport for Figure 2 data across the Internet or wireless Video Streaming Application Specific connections, providing mechanisms Protocols TS, RTMP, HLS, or Smooth Streaming to overcome poor QoS. Proprietary Protocols

While the user experience may appear Application Layer RTP, RTSP similar, we have identified why protocols Data Transport applied to streaming on wired LANs Protocols Transport Layer UDP, TCP and WANs are frequently different from those used on the Internet and WLANs. Internet Layer Internet Protocol Stack Protocol differences increase as various decoding devices are considered. Link Layer

Decoding Devices and The relationship between Network Transport and Streaming Protocols Operating Platforms Customers expect to use many different devices to decode AV streams and each class of product can use a Many Streaming Protocols transmission standards such as different electronic processing platform, Applied in Different Ways the Advanced Television Systems operating system, or player software Both standards-based and proprietary Committee – ATSC and Digital to decode AV streams. Manufacturers protocols are used in streaming Video Broadcasting – DVB. It is and developers of these devices have applications. Examples of commonly used extensively in broadcast, IPTV, applied both standards-based and used standards-based protocols include: cable television and digital signage proprietary streaming protocols to meet applications. Set-top box decoders their technical or commercial objectives. • RTP – Real Time Transport Protocol: typically use TS and many H.264 Each type of device will offer different RTP is used broadly by hardware hardware encoders and decoders compatibility and performance: encoding and decoding products, but use it as well. it may not be applied in the same • Hardware encoders, decoders, and manner on every streaming product. Proprietary streaming protocols have set top boxes are designed to support H.264 video may be delivered directly been developed for use on software specific streaming performance using RTP, or RTP may be used to operating systems and media player operating on private LANs and WANs carry MPEG-2 Transport Streams. applications. Commonly encountered that can deliver predictable bandwidth, • RTSP – Real Time Streaming Protocol: examples include: scalability, or low latency. RTSP is a session management • Desktop and laptops offer protocol for streaming but it has also • Microsoft Media System – MMS: flexible operating systems that support been applied in a non-standard method MMS was developed for use by installation of different media players to deliver AV streaming payloads. Media Services. and software decoding applications. MMS has been frequently deployed in Sustainable real-time processing Both RTP and RTSP are identified as enterprise-wide streaming applications capacity may have limits. part of the Internet Protocol Suite. that use to • Tablet and mobile devices have decode and present AV streams. operating systems and processing • MPEG-2 Transport Stream – TS: The • Real Time Messaging Protocol – capacities that can be expected to be TS was established by the Moving RTMP: Adobe has developed RTMP, lower than those offered by desktop Pictures Expert Group – MPEG and RTMPS, RTMPE, RTMFP, and HTTP computers and laptops. Since they it is frequently used in streaming Dynamic Streaming protocols for use are wireless devices, connection applications. It is a container format with Player. Each protocol bandwidth and QoS can vary. that uses RTP and UDP protocols for offers unique security, encryption, Customer expectations to stream to transport in streaming applications. latency, or firewall traversal capabilities. these devices continues to grow. It has been applied in broadcast RTMP and Adobe Flash Player are

72 Design Forum AV SYSTEMS DESIGN Fall 2014 AVSD.com 73

avsd_fall2014_cc.indd 72 10/7/14 4:42 PM DESIGN FORUM

used extensively for streaming on players. Some compatibility is available protocols are ideal for streaming content the Internet to PCs. Flash Player was from standalone streaming player across the Internet or WLANs where it once supported on and applications and plugins developed using may be necessary to adapt to changing tablet PCs using the Android operating specifications for either proprietary or bandwidth and QoS conditions. systems and RTMP protocol was used. standards-based protocols. However, This is no longer the case. the device platforms and applications Examining Streaming • HTTP Live Streaming – HLS: The may not support the reliability required Protocol Specifications Apple HLS streaming protocol is used for industrial applications. For instance, When examining specifications for extensively streaming to iPad and an application that is compatible with various encoding and decoding devices, iPhone devices. These Apple devices RTP on a tablet PC may not adequately the information concerning streaming use iOS operating systems and manage processing for high bit rate protocols is often difficult to locate HTML5 serves as the media player. streams or provide recovery mechanisms and interpret. For instance, a device New smartphone and tablet PCs with to address network errors. supporting RTP may use UDP and RTP the Android operating system support Hypertext Transfer Protocol – HTTP to carry a TS payload, or UDP may be the HLS protocol. and TCP network transport protocols are used alone to deliver TS. RTSP or HTTP • Smooth Streaming, an extension used by HTTP Dynamic Streaming, HLS, protocols may be used purely to control of Microsoft IIS Media Services, is and Smooth Streaming protocols. TCP a device or manage a streaming session, another Microsoft streaming protocol, provides reliable transport and HTTP or they may also be used to transport AV which has been developed for use allows video streams to tunnel through streaming payloads. When considering with Microsoft Silverlight and other firewalls using port 80, which will be open encoding and decoding devices from media players. for HTTP Web traffic. These protocols different manufacturers, there is no also support Adaptive Bit Rate – ABR substitute for reviewing specifications The proprietary protocols identified capabilities that allow decoding devices to directly with an applications engineer or above are the preferred streaming request higher or lower streaming bit rates conducting a compatibility test. transport methods for the identified as network throughput or processing smartphones, tablet PCs, and media capacity increases or decreases. These Supporting AV Streaming to Many Different Devices The wide variety of devices, operating systems, media players, and protocols Extron SME 100 and Streaming Protocols identified here creates a challenge for the AV designer that must offer live streaming to any device, anywhere. Media servers offer a solution. Media servers accept streamed inputs, converting bit rates, resolution, and streaming protocols to support compatibility with different types Since it was first introduced, Extron has expanded the variety of protocols that are available at decoding devices. This conversion from the SME 100 for streaming audio and video across IP networks. The SME 100 now capability is referred to as “multi-screen supports MPEG-2 Transport Streams – TS, the streaming protocol used by many third party delivery.” Media servers also support H.264 devices including set-top box decoders. TS can be applied in unicast and multicast push system scalability by serving multiple streaming applications. Interleaved RTSP transport is available to provide reliable delivery when unicast AV streams to decoding devices low network QoS conditions exist. In addition, HTTP tunneling is supported, allowing RTP and RTSP to pass through firewalls when ports used by streaming protocols have been closed. when a network cannot support IGMP multicast traffic. Currently shipping units include these new streaming protocols. A firmware upgrade is available Developers of media servers, such for units shipped prior to February 2013, free of charge. as Wowza Media Systems, recommend Model Version Part No. the following when encoding for live, SME 100 HD H.264 HD Encoder 60-1061-01 multi-screen streaming applications: SME 100 SD H.264 SD Encoder 60-1061-02 For the greatest compatibility with decoding devices, apply standards-based

72 AV SYSTEMS DESIGN Fall 2014 AVSD.com Design Forum 73

avsd_fall2014_cc.indd 73 10/7/14 4:42 PM DESIGN FORUM

Figure 3 Where streaming protocols are considered as a part of a complete system TS Set Top Box and design, choices will often need to be made Hardware Decoders between performance and compatibility. Small, point-to-point applications using RTMP, RTMPS specific devices will be afforded the luxury Adobe Flash Player on PCs or legacy Android smartphones and tablet PCs of applying the best performing protocol High Quality H.264 without making compromises.

H.264 Encoder Projects that must serve a large RTP, RTSP or Media Smooth Streaming Server number and variety of decoding devices, PCs and Software Players including personal computers, tablets, or smartphones, should consider that the streaming protocols used by the media HLS players on these devices may change Quicktime Player on PCs or HTML5 on iPhone and iPad or new Android smartphones and tablet PCs over time. Wherever possible, a system design should apply a standards-based Media servers allow a single H.264 stream to be compatible with many different decoding devices. protocol for AV encoding and distribution on a wired LAN. Media servers can be used to convert protocols for use on wireless consumer devices or the compression such as H.264, and use Dynamic Adaptive Streaming over HTTP Internet. Platform design choices that are commonly used streaming protocols – DASH also known as MPEG-DASH was based on use of proprietary protocols such as RTP or RTMP. AV streams published as an international standard for specific media player software or should be encoded at the highest quality in April 2012. Broad implementation wireless consumer devices may be costly possible to preserve image quality of the DASH standard may reduce to maintain and support over time as for higher resolution streams, leaving the variables to consider streaming to technology and devices change. down-conversion to other devices. wireless consumer devices and Internet destinations. However, until that standard Karl Johnson is Director of Product Marketing at Extron Streaming Application Tip: Media is applied broadly, the AV professional will Electronics for streaming technologies and videowall servers are intended for use in streaming still need to include streaming protocols processing systems. He worked at Electrosonic for over 20 years, most recently as General Manager of the video in one-way applications. When as an important factor in system designs. Electrosonic product division. designing any streaming system, be sure to consider if low latency is a requirement. Low latency is required for bidirectional Extron and Multi-Screen Streaming Applications applications and should also be considered if live streaming content will be used as a visual reinforcement on personal displays in a presentation area. If the user can see or hear the original source material, noticable delay introduced by a media server will make it ineffective.

How Should the AV Designer Manage Streaming Protocols? Extron’s SME 100 has been installed into multi-screen streaming applications in educational and corporate environments, delivering H.264 streaming video to Apple iPad and iPhones, Android Streaming protocols are an important tablet PCs, and smartphones, as well as Windows desktop computers. These multi-screen variable in H.264 applications. Failure applications include both live streaming and playback of content recorded from SME 100 using to consider their role can result in a big Wowza Media Server. Complete instructions for integrating SME 100 with Wowza Media Server compatibility gap for a system. can be found in the SME 100 User Guide available at www.extron.com/sme100guide. A new adaptive bit rate technology,

74 Design Forum AV SYSTEMS DESIGN Fall 2014 AVSD.com 75

avsd_fall2014_cc.indd 74 10/7/14 6:22 PM Architectural Considerations Critical to Streaming Protocols

Streaming Destinations Inside the Enterprise Outside the Enterprise

Decoding Devices

Electronic Decoding Computing Device Appliance Software O/S

Multi-Screen Applications Delivery Media Servers or CDNs Paradigm deliver standards based and proprietary streaming protocols One to One Few to Few One to Many

Firewalls

Inside the Firewall Outside the Firewall

Appropriate Proprietary based protocols: HLS, RTMP, Smooth Streaming Streaming Protocols Standards based Protocols: RTP, RTSP, TS

Design Questions Critical to Streaming Protocols

The following questions can help the AV professional identify important architectural and performance requirements to factor into streaming system designs

Will the video streaming content What type of network path must the Identify the system scalability be used inside or outside streaming video pass over? LAN, that will be required. How many the enterprise? WAN, WLAN, or the Internet? consumers of video streaming • Inside: Opportunities exist to • Managed LANs or WANs: content must be served? identify the decoding devices and A standards-based protocols • One-to-many applications will the best protocols to use. such as RTP will be better typically require support for many • Outside: A variety of protocols positioned to serve low delay and different types of decoding devices must be supported. A media multicast applications. and streaming protocols. A media server or Content Delivery • WLANs or the Internet: Will be server or CDN may be required. Network – CDN may be able to better served using HTTP or HTTP fulfill that requirement. ABR protocols. Will the application require low latency streaming or is delay not Determine the types of decoding Will the AV streaming need to a concern? devices that will be used by cross firewalls? • Low latency applications will be the customer. • HTTP and HTTP ABR protocols will better served using RTP protocols • If a wide variety of devices may be offer the convenience of tunneling streaming directly from encoding used, a media server can provide through firewalls that have closed to decoding devices on wired LAN the means to convert streaming ports used by commonly used and WAN connections. protocols for those devices. streaming protocols. • Wireless consumer devices will probably require support for proprietary streaming protocols.

74 AV SYSTEMS DESIGN Fall 2014 AVSD.com Design Forum 75

avsd_fall2014_cc.indd 75 10/7/14 4:42 PM