Diplomarbeit A Java-Based Streaming Media Server ausgeführt am Institut für Interaktive Medien Systeme der Technischen Universität Wien unter der Anleitung von ao.Univ.Prof. Dr. Horst Eidenberger durch Harald Psaier Matr.Nr. 9826727 Bergsteiggasse 7/1/20 Wien, 19th October 2005 2 Abstract The distribution of multimedia content has become a prospective sector of the Internet business. Multimedia content is transmitted by media streaming servers to stream-controlling clients. These multimedia architectures are available for many different operating systems. This diploma thesis describes the implementation of such a multimedia client/server architecture. The main task of a media streaming server is to provide transparent network services for a collection of media objects. The server acts as a proxy server and allows different access methods to the provided multimedia content. Accessing the multimedia content includes streaming media objects, accounting media ob- jects and their access rights and, furthermore, media stream-control during media content transmission. At the early stages of the work existing solutions and methods of multime- dia implementations were examined . The resulting software project consists of a client component and a server component based on the Java programming lan- guage and its media extension, the Java Media Framework. Java and the frame- work provide all the means to implement a multi-threaded listening server and a controlling and accounting client. The communication between the counterparts is established by the Real Time Streaming Protocol. It has been extended to support media information and user profile accounting. A relational database manages the media information and user profiles. Media streaming is implemented by seg- menting the media objects into network packets using the approach described by the Real-Time Transport Protocol. The resulting software projects comprises a server and a client package. The server package contains facilities for streaming, device capturing and caching of media streams. The client provides an user interface for both control of media transmissions and administration of media information. The final implementation is a multimedia client/server architecture written in Java code, thus making it independent from the operating system. Equipped with real-time media stream control and real-time media object transport capabilities, it is compatible with modern multimedia streaming solutions such as the RealNet- works and the QuickTime client. CONTENTS 3 Contents 1 Introduction 5 2 Related solutions and background 8 2.1 Streaming protocols . 8 2.1.1 RFC 2326 - The Real Time Streaming Protocol (RTSP) . 9 2.1.2 RFC 1889 - The Real Time Transport Protocol (RTP) . 9 2.2 Multimedia frameworks . 10 2.2.1 OpenML . 12 2.2.2 Apple QuickTime . 12 2.2.3 GStreamer . 13 2.2.4 Java Media Framework . 13 2.3 Streaming media overview . 14 2.3.1 Apple streaming solutions . 15 2.3.2 RealNetworks streaming solutions . 16 2.3.3 VideoLAN streaming solutions . 16 2.4 Caching . 17 2.4.1 Caching objectives . 18 2.4.2 Memory caching policies . 18 3 Design and architecture 22 3.1 Requirement analysis . 22 3.1.1 Media management and control . 23 3.1.2 RTSP/RTP server . 24 3.1.3 Interaction with RealNetworks and QuickTime clients . 25 3.1.4 JMF MPEG-4 RTP integration . 25 3.1.5 Caching . 26 3.1.6 UI enhancement . 26 3.1.7 Media capturing timer . 27 3.2 Deployment strategy . 27 3.3 Static structure . 28 3.3.1 The server package . 28 3.3.2 The client package . 32 3.3.3 Database tables . 35 3.4 Dynamic behavior . 36 3.4.1 RTSP/RTP stream setup . 37 3.4.2 Extended RTSP . 39 3.4.3 Authentication process . 40 3.4.4 VCR timer sequence . 43 CONTENTS 4 3.4.5 Multi-unicast session . 44 3.4.6 Patterns - singletons and factories . 44 3.5 Implementation challenges . 46 3.5.1 JMF creation . 46 3.5.2 JMF RTP engine . 47 3.5.3 JMF MPEG-4 RTP stream setup . 47 3.5.4 JMF AVI Container with audio/video streams . 48 3.5.5 Interval Caching . 48 4 Implementation 49 4.1 Implementing with the Java Media Framework . 50 4.2 Project description . 50 4.2.1 System overview . 51 4.2.2 Server package . 51 4.2.3 Sub-packages in the server package . 56 4.2.4 Client package . 59 4.2.5 Mp4 package . 60 4.2.6 Util package . 61 4.2.7 MediaDB database tables . 61 4.3 User Interfaces . 63 5 Usage description 64 5.1 System requirements . 65 5.1.1 Hardware requirements . 65 5.1.2 Software requirements . 65 5.1.3 Building requirements (ANT) . 66 5.2 System usage description . 66 5.2.1 Running the projects server and client . 66 5.2.2 Transmitting and controlling an RTP stream . 66 5.2.3 Adding and deleting media . 67 5.2.4 Editing media object properties . 67 5.2.5 Starting a multi-unicast transmission . 67 5.2.6 Adding and deleting a recorder timer . 67 5.2.7 Running other clients . 68 5.3 Performance . 68 5.4 Limitations & known bugs . 70 6 Conclusions 71 6.1 About the project . 71 6.2 Outlook . 72 1 Introduction 5 1 Introduction This diploma thesis describes my work in the field of multimedia streaming archi- tectures. A multimedia streaming architecture is characterized by server components, client components and a component-connecting network. With the growth of the Internet and the local networks with their manifold possibilities, the streaming of multimedia content has become a new challenge and a profitable busi- ness. Current computer networks were initially built to handle text-based information only. In order to integrate a multimedia server into these networks, new procedures and problem-specific solutions have be defined. The initial documents, implementations and supplier of media streams appeared in the middle nineties when the Real-Time Transport Protocol (RTP) and the Real Time Streaming Protocol (RTSP) were composed and RealNetworks was founded. Never- theless, transmitting reliable and continuous multimedia streams is still a complex task. Many obstacles have to be passed before an undisturbed delivery can be achieved. Huge amounts of data need to be delivered. Hence, large network bandwidths are required. Moreover, the layers below the network transport layer do not guarantee reliable deliv- ery. Appropriate methods have to be implemented by the upper layers. Finally, from the marketing point of view, consumers still need to be attracted to such services. One solution to aforementioned bandwidth problems would by to "throw bandwidth on con- gestion problems", augmenting the networks reliability. This strategy would solve the problem on local networks. However, streaming through public networks worldwide, such as the Internet, is still only of interest if connected to a cheap and steady web, using flat rates such as DSL technology. Public streaming services started with radio streams at different compression and quality rates, followed by download and stream- ing portals provided amongst others by such well-known companies as RealNetworks (Listen.com), Microsoft (MSN Music) or Apple (iTunes Music Store). The broadcast of audio-visual content is just starting to establish itself, with only a few and hesitating providers. The newest addition to streaming implementations for the Internet is Internet telephony also known as Voice over IP. The purpose of the project is to develop a streaming and accounting media server which can handle multiple client requests simultaneously. The media server’s main function is to hosts a streaming and accounting service for a collection of media objects. The ser- vices are supported by a relational database that stores records of media content infor- mation. Furthermore, the server tries to balance the streaming request load by caching intervals of media streams. The client offers access to the server’s services through an user interface. Along with media stream capturing, the client’s main function is to co- 1 Introduction 6 ordinate media content presentation with media stream flow-control. Both server and client are written in Java programming language and include the Java Media Framework (JMF). The framework adds streaming and presentation functions to the components. Figure 1 shows the main components of the media server implementation of this thesis work. Figure 1: Project overview. CT1, CT2 represent the different communication types. The projects main components the database, the multimedia server and the clients in- teract using two communication types (CT1 and CT2). The database holds media infor- mation and user authentication data. The connection between server and database CT1 requires a database dependent protocol for communication. It is implemented using a library interface. The link to the interface allows the server to submit standard queries to the database tables and to fetch the returned query results. In order to satisfy the user the client requires real-time communication with the server. Hence, CT2 actually consists of two communication protocols supporting real-time demands. These are the Real-Time Transport Protocol (RTP) and the the Real Time Streaming Protocol (RTSP). RTP is the state-of-the-art method to transmit multimedia data in real-time over networks. Its functions are embedded into the projects software with the help of the Java Media Framework. RTSP provides control over multimedia streams. This protocol was implemented to put remote-control methods over media content streams at the client’s disposal. RTSP was extended to allow authenticated media object accounting. Additionally, to the just stated main objectives my work comprises the following as- pects. On the server’s side, not only the processing of multimedia streams and protocol communication are of interest but also some proposals about clever resource handling have been made. Several caching policies for multimedia streaming were considered to 1 Introduction 7 support the server in load balancing and better utilization of free primary and fast mem- ory, hence increasing server capacity and resource usage.