Transfer Protocols for Digital Media Workflows Contents 2 Introduction
Introduction to Protocols Introduction to Protocols 2
1 User Datagram Protocol (UDP) 3 With the many advances that have been made in digital media file formats, compression, and distribution – the RUDP 6 technology for the transfer of rich media has remained relatively unchanged for over 40 years. UDPS 6
New lightweight codecs and file choices are regularly introduced to provide content viewing and a downloadable proxy for production. However, with a growing influx of content makers looking to maintain the value of a File Transfer Protocol (FTP) 6 2 large, complex master file for productions, the need to transfer, share and move these large files has various FTPS 8 SFTP 8 challenges in speed, bandwidth and security.
Here we take a look at some of the transfer building blocks commonly used throughout the video industry. For the purpose of this document we are going to focus on the three main protocols in practice today – UDP, 3 Hypertext Transfer Protocol (HTTP) 9 HTTP 1.1 11 FTP and HTTP.
4 Key Considerations 12 11 Transfer Speed Times 13 The Future 14
5 About Aframe 15
Share this... Transfer Protocols for Digital Media Workflows Transfer Protocols for Digital Media Workflows
1 UDP User Datagram Protocol 4 UDP
User Datagram Protocol (UDP)
Essentials
• Typically used for streaming media assets over IP. • UDP will optimize your transfer rates for large data sets when there is high latency or packet loss over the network.
Background
Developed and released by David P Reed in 1980, thanks to its fast transfer rates and capability of handling large files, UDP is quite often the method to be leveraged for video conferencing and video streaming needs. Compared to some standard Transmission Control Protocol (TCP) communications, it offers a simple lightweight messaging format, where the stream of data is lacking formality. UDP is commonly leveraged along with compression algorithms for improved speed with video file transfers and in local storage systems for speed of access and playback from a Storage Area Network (SAN).
The principle advantage of UDP is speed – particularly when transferring large files across wide geographic expanses. The protocol allows for the sending of multiple data streams in parallel, and does not require acknowledgement from the recipient in order to progress. Compare this with TCP-based protocols such as FTP, where each packet requires acknowledgement before the next is sent – and factor in the hundreds, if not thousands of miles of network infrastructure these requests are sent back and forth across - and you can begin to see the advantages of UDP for file transfer.
Share this... Transfer Protocols for Digital Media Workflows Part 1 5 UDP
These advantages are further emphasized in the world of media, where files are typically many times larger than text-based documents, or even image-based documents. The fact that video production is increasingly becoming a geographically dispersed environment, and the implications this has on collaboration and access for project stakeholders, add further weight to the argument in favor of making UDP the de facto file transfer protocol of the media and entertainment industry.
Did you know ? Can you depend on it? Because of its unprecedented speed, The transmissions consist of a header, the datagram for the message and a checksum (optional) for protection against tampering or corruption. The datagram can handle payloads up to 65,535 bytes, (8 of which are for UDP is the preferred the header information). transfer protocol of the London Stock Exchange In the past, UDP has been called out as unreliable due to its non-transactional method for packet checking and the optional use of check sums on the messages in Internet Protocol version 4 networking. The datagram is sent without checking to see if the server side host is ready to receive. It is a stateless protocol, where a socket is created from the client to the server and messages are sent as quickly as the bandwidth of the network connection will allow. There is no overriding control as to which packets in the transfer are sent first. This process can drop packets, which can be lost without application controls.
Share this... Transfer Protocols for Digital Media Workflows 6 UDP
RUDP - Reliability User Datagram Protocol
To answer concerns about UDP’s reliability, RUDP (conceived by Bell Labs), has been adopted by some networking and application companies to specifically address the following concerns:
• Acknowledgment of received packets. • Flow control. • Retransmission of lost packets. • Over buffering – or compressing for faster than real time transmissions.
UDPS - Extension
UDPS is an extension to the UDP standard, where client side applications request that the UDP session be encrypted. UDPS relies on Secure Socket Layer (SSL) and a Transmission Layer Security (TLS) for long public and unrelated session keys, which encrypts the transmission or message for the length of the transmission session.
Share this... Transfer Protocols for Digital Media Workflows Transfer Protocols for Digital Media Workflows
2 FTP File Transfer Protocol 8 FTP
File Transfer Protocol (FTP)
Essentials
• FTP may run in active or passive mode, which determines how the data connection is established. • Typically used for point-to-point media movement. • Fast and compatible with most networks.
Background
File Transfer Protocol, or more commonly FTP, was developed in 1971 by Abhay Bhushan. As a pre–Internet protocol, FTP continues to be the primary way that users talk about digital file transfers. It was originally used for moving large text files from point to point with a simple username and password or anonymous logins for access to publically shared files - FTP was not designed with file sizes in the high gigabytes or even terabytes in mind. This makes something of a square peg for the video industry, where file sizes continue to rise exponentially.
As is the case with each protocol discussed here, there will always a be a handoff to TCP at the end of the transfer process. Based on how that is configured on the network it can have an impact on the transfers regarding what’s allowed to pass and not pass through customer network. An example being a TCP policy that doesn’t allow encrypted transmissions on the network.
Share this... Transfer Protocols for Digital Media Workflows Part 2 9 FTP
Can you depend on it?
FTP inherits TCP’s robustness and is generally very reliable for transferring files. Security and the ability to resist brute force attacks (like DOS) on FTP servers, has been a key point for both vendors and customers to examine because of the impact on performance within a datacenter or FTP server. Adding security procedures within the client application and server responses can mitigate these threats. Did you know ? FTPS - Extension
TCP guarantees the This increasingly popular protocol version handles client side application requests that the FTP session be delivery of data between encrypted. The server side has the option of allowing or denying connections that do not request TLS. This two hosts and also boasts method uses other ports as part of the encryption for the data transfer. that packets are delivered in the same order in which they were sent SFTP - Secure File Transfer Protocol
This uses the secure shell to transfer files, offering encryption for both commands and data, as well as preventing passwords and sensitive information from being transmitted openly over the network.
However, SFTP does not interoperate well with common FTP software. This makes collaboration and setup of sharing pools more difficult to administer and run with a wide set of partners.
Share this... Transfer Protocols for Digital Media Workflows Transfer Protocols for Digital Media Workflows
3 HTTP Hypertext Transfer Protocol 11 HTTP
Hypertext Transfer Protocol (HTTP)
Essentials
• HTTP is the foundation of data communication for the World Wide Web. • The protocol defines how messages are formatted and transmitted. • HTTP is slower in comparison to FTP & UDP but more likely to work on all networks.
Background
This protocol is the basis for communications on the World Wide Web and was developed in 1989 by Sir Tim Berners-Lee et al. The function is based on client request and server response for text or other media depending on the application. Unsurprisingly, as the language of the Internet, it’s also the most commonly leveraged transfer communication. HTTP provides logging, caching of repeated content and has been leveraged to provide persistent connections for multiple transfers in the recent improvements to the standard.
As with FTP and UDP, it can also utilize acceleration “over buffering” with applications designed for that purpose, but can be seen as “best effort” based upon the request-response and stateless architecture. Its chatty in that regard and there is a larger overhead for instructions compared with FTP and UDP.
Share this... Transfer Protocols for Digital Media Workflows Part 3 12 HTTP
Can you depend on it?
As the use of the Internet has grown exponentially for the consumer, business and enterprise, so have the number of security attacks against web servers. Media applications that require logins, encrypt the transfer Did you know sessions and have unique identifiers (user and application cookies) are more successful in keeping legitimate ? transfers and transactions to take place between the client and server. HTTP is called a stateless protocol because each HTTP 1.1 command is executed independently, without any The latest version of Hypertext Transfer Protocol (HTTP 1.1) has streamlined some of the archaic processes knowledge of the commands of previous models. It now provides a persistent connection that allows multiple requests to be batched or that came before it pipelined to an output buffer. The Transmission Control Protocol layer can put multiple requests into one TCP segment that gets forwarded to the Internet Protocol layer for packet transmission.
Because the sum of connection and disconnection requests for a sequence of “get a file” requests is condensed, fewer packets need to flow across the Internet. Since requests are pipelined, TCP segments become far more efficient - resulting in less Internet traffic and faster performance for the user.
Share this... Transfer Protocols for Digital Media Workflows Transfer Protocols for Digital Media Workflows
4 Key Considerations Part 4 14 Speed Test
Transfer Speed Times
Time (minutes) Test Rundown: 0 5 10 15 20 25 30 35 40 45 50
HTTP Upload Bandwidth Speed:
80 Mbps FTP
File Size:
UDP 10GB file
The chart above shows transfer times of each of the protocols discussed from London to Los Angeles.
HTTP and FTP times in this example transfer were distinctly slower than UDP, which was expected with a large file size over long distances. UDP transfers with larger file sizes become more efficient than FTP performance over the course of the transfer time.
Share this... Transfer Protocols for Digital Media Workflows Part 4 15 Key Considerations
The Future for Digital Media Transfer
While not extinct, the use of videotape dubs, baseband satellite transfers and baseband video over IP transfers are drastically reduced in today’s production environment. There is an ongoing need to move key increasing amounts of file-based media at higher speeds to serve the media production life cycle. The size considerations of of files from media acquisition is not getting smaller. High-speed bandwidth for upload and download of 5 large files is becoming more of an available reality. Additionally, there is a move to collaborate with remote file transfer partners allowing content to be centralized, managed and available for use at any minute by any partner.
All of these existing transfer protocols work to move digital media files. Since video based media files can Time be very large in size, it pays to understand which protocol is best for your transfer needs. There are several applications that accelerate media transfers and other applications that add a user interface. Cost To that end here are some questions you should consider when selecting solutions to support a digital Security media workflow:
Reliability 1) What are the bandwidth requirements for our business and our acquisition partners?
Accessability 2) What networking policies are in place to protect and facilitate high volume transfers of our media?
3) Can we afford to maintain a separate network for media workflows to accommodate high-speed transfer and transmission of large media files?
4) What technology and products can leverage our available bandwidth to improve our collaboration and retain our value of our media?
Share this... Transfer Protocols for Digital Media Workflows Transfer Protocols for Digital Media Workflows
5 About Aframe Part 5 17 About Aframe
Introducing Aframe
Aframe is the video collaboration platform in the cloud that gives you one central location where everyone can instantly view, access and work with high resolution media. Aframe drives business efficiencies and dramatically increases the usage and value of media assets. Aframe’s private cloud solution provides an easy Fastest upload speeds, guaranteed mechanism for making Aframe pain-free upload technology solutions help transfer files quickly from camera to the cloud. Our new content easily accessible desktop app makes the process even smoother by automatically syncing content between your computer “for people who need it and your Aframe project. immediately, in a self- The Aframe uploader utilizes UDP based file transfer technology to effectively max out your Internet service model. connection, resulting in faster uploads and less time waiting around. If your organization hasanover- sensitive firewall and blocks this uploader, you have the option to switch to FTP (which is still fast and compatible with many networks) or HTTP (which is slower but more likely to work with all networks). In Wendy Allen, short, if you can browse the Internet, you can upload media into Aframe. VP of Media Engineering
Review and approval Logging and tagging Share just the clips you choose, with the people Add logs to the timeline to make everything ” that need it and decide whether they’ll need a searchable with simple keyboard shortcuts. download or a secure web stream – whether Alternatively, have our team of taggers log it’s rushes, rough cuts or finished content. your footage for you.
Timecode based commenting Streamline the edit Aframe’s HTML5 player enables granular time Hassle-free pre-edit organization and reduced measurement, frame accurate metadata input time in the edit suite. Organize your media, log and commenting. Simply add your comments clips and make timecode-based notes. Export to the video clip’s timeline, collection or the metadata into the edit system of choice. document and manage your entire production from a single work-space.
Share this... Transfer Protocols for Digital Media Workflows Part 5 18 About this eBook
About the Author
Bruce Jones Senior Solutions Architect Senior Solutions Architect [email protected] [email protected] 516-426-2186 978-799-9400
Bruce joined Aframe from Avid where for the past 7 years he has helped design and implement complex production and broadcast solutions. Bruce has also worked for several US television stations including KPHO-TV, KCNC-TV and KNXV-TV.
See Aframe’s unique file transfer technology in action: aframe.com/demo
Share this... Transfer Protocols for Digital Media Workflows