Using the Java Media Framework to build Adaptive Multimedia Groupware Applications Martin Bateman, Alan Ruddle, Colin Allison School of Computer Science University of St Andrews St Andrews, KY16 9SS {mb,alanr,ca}@dcs.st-andrews.ac.uk Abstract principles of distributed groupware remain, there is now a clearly preferred means of implementation Realtime audio and video conferencing has not yet and deployment. been satisfactorily integrated into web-based groupware environments. Conferencing tools are at best only loosely linked to other parts of a shared working environment, and this is in part due to their implications for resource allocation and management. The Java Media Framework offers a promising means of redressing this situation. This paper describes an architecture for integrating the management of video and audio conferences into the resource allocation mechanism of an existing web-based groupware framework. The issue of adaptation is discussed and a means of initialising multimedia session parameters based on predicted QoS is described. 1 Introduction Figure 1: A Warp-based Multimedia Groupware Multimedia conferencing tools are specific Application applications within the general scope of real-time The Warp-based RDG application shown in Fig.1. distributed groupware (RDG). Ellis and Gibbs [1] used X-windows, IP Multicast, and RPC2, and is itemised the following requirements of RDG therefore an example of pre-web groupware. It is systems: also a system that required a significant degree of • response times must be short technical expertise to install and use, in strong • participants are not required to be connected to contrast to the ease of use and deployment the same machine associated with web-based approaches. • participants are free to come and go at any time The TAGS framework for groupware [4] differs • participants are not required to follow a pre- from Warp in that it is completely web-oriented. planned script TAGS is a project to research the development and • there is the possibility of a high degree of access deployment of distributed learning environments conflict and is currently used to support the development of • participants can communicate with each other via groupware in the form of distributed, multi-user an audio-visual channel resources. In TAGS groups form the basis of (i) Warp [2] produced an exemplar RDG application privileges and access control, (ii) information in the form of a multi-user, distributed, shared dissemination and event awareness and (iii) spreadsheet [3] that meets all of the above allocation of shared resources. A home page, or requirements. (See Fig. 1). However, a new portal, is generated dynamically for each TAGS important requirement must be added to the RDG user, and presents access via hyperlinks to all the requirements list, namely that RDG, in the interests resources allocated to all the groups that that of interoperability, should be usable on the web. individual belongs to. It has however proved Web-oriented component technologies adhere to the difficult to incorporate realtime audio and video 'anytime anywhere' requirements of distributed conferences into this model, due in part to the groupware and are largely platform neutral. The general lack of mechanisms for integrating realtime growth of the web as the preferred medium for conferences into the Web. Efforts to date have group work has meant that, while the underlying resorted to starting applications such as vic[5] and rat[6] either totally independently or from a browser sending the data to its final destination - this could script. This is not satisfactory as it means that (i) the be to the screen, to a file, or to the network. session is not integrated in any useful way with the resource allocation mechanisms, and (ii) a degree of technical expertise beyond that found in typical web users is required. This paper describes an approach to tackling these problems in the form of an architecture for synchronous communication within a group-based distributed learning environment. Given the special needs of such resources the architecture provides for adaptation to predicted and current network conditions. In the broader sense it is an application of the principle that distributed learning environments should be QoS aware in order to provide a reliable service. The remainder of the paper is structured into three sections. Firstly the Java Media Framework (JMF) Figure 2: A simple JMF application and its features that facilitate web based JMF supports various network protocols for the synchronous communication are described and the transmission of multimedia, the most interesting of limitations imposed by such an approach are these from this projects point of view is multicast identified. Secondly, the issues that arise when the RTP[8] which can be used to disseminate realtime JMF is used within the context of a system that multimedia to a group of users. supports the allocation of resources to users through A/V Media Cross Native Bit a group mechanism are discussed. Finally, the Encoding Platform Library Rate automatic configuration of multimedia resource (sample) instances based on past traffic measurements to A G.711 (U-law) Rcv/Tx Rcv/Tx 64 kb/s, ensure fairness with competing traffic and 8Khz, 8 bits utilisation of available bandwidth is discussed. mono A G.723 mono Rcv Rcv/Tx 6.5 kb/s A 4-bit mono Rcv/Tx Rcv/Tx 32 kb/s 2 The Java Media Framework DVI, 8 Khz A 4-bit mono DVI Rcv/Tx Rcv/Tx 44 kb/s The Java Media Framework (JMF) [7] is a 11.025 Khz A 4-bit mono DVI Rcv/Tx Rcv/Tx Varies 0.0 - collection of APIs which aim to provide a method 22.05 Khz 100 kb/s for handling time based multimedia within Java. It A MPEG Layer I 64 kb/s does this by allowing the capture, transmission, 48 Khz @ 16 bits per sample, storage and displaying of various formats of audio mono, and video. There are two main distributions for the A MPEG Layer II JMF, a pure Java implementation and a native 22 Khz @ 16 32 kb/s library version. The pure Java implementation is bits per sample, mono, quite limited in ability since it is unable to capture A MPEG Layer Rcv/Tx Rcv/Tx or transmit video/audio. It can however, render III 64 kb/s 44 Khz @ 16 audio/video streams that it receives and is intended bits per sample for use in situations where the native version of mono JMF has not been installed but the user still wishes V JPEG (411, Rcv Rcv/Tx 1.5Mb/s @ 422, 111) * 320x200 @ to receive the audio/video streams. In the native 19fps library version of the JMF the majority of the V H.261 - Rcv - processing is performed in platform specific code. V H.263 ** Mode A Rcv/Tx 120kb/s @ Only 176x144 @ This has significant performance advantages over a 25fps pure Java implementation and offers greater V MPEG-I *** Tx Rcv/Tx Depends on encoded capabilities for the programmer and user. media The simple JMF application shown in Fig.2 has * Video dimensions must be multiples of 8 pixels three main parts. There is (i) a DataSource, which ** Can only be transmitted as one of 128x96, 176x144, 352x288 receives multimedia data from devices such as *** Only from pre-encoded media e.g. mpeg encoded file. video capture cards or sounds cards; (ii) a Processor Table 1: JMF Supported Multicast RTP payloads that can be used to change the format, frame rate, Table 1 shows the RTP payload types that are image depth or bitrate of the multimedia data - for supported by the default installation of the JMF. example, video could be encoded to H.263 at 23 fps Due to the lack of H.261 encoding support, JMF is 120Kbps; (iii) a DataSink which is used when only able to interoperate with vic using Motion using groups as the basis for the mapping. Users JPEG, which is bandwidth hungry in comparison to and groups are unique by name; resources are H.261 or H.263. unique by name and type. Access rights can be JMF's major advantage when compared to other specified when a resource is allocated to a group. A video conferencing technologies is that since it is resource may simply be distinguished as Read-only Java based, it can be incorporated in applets that or Read-Write, or it may export a more subtle set of can then be integrated into a web environment. access methods. Figure 4 illustrates an example set Additional multiplexers, demultiplexers, filters and of relationships between users, groups and codecs can be added to the JMF via means of a resources. plug-in architecture. These plug-ins must be registered on the host machine before they can be used. Users Groups Resources There are several ways in which the bandwidth RW A used within a conferencing session can be varied. John Students These include, video frame rate, video size, image Alice Group A RW B resolution, image depth, audio resolution, audio R Mark Group B S sample rate, changing codec, changing media. R The primary channel of communication in a video Jane Tutors RW T conference is the audio channel, with the video link providing additional information, such as the Figure 4: Group Resource Allocation expression of the other person. This generally aids John, Alice and Mark are members of the Students in communication and leads to fewer group. There are no resources shown allocated to misunderstandings, even when the bit rate used for the Students group. Jane is a Tutor and has created audio is low. Groups A and B. John, Alice and Jane are in Group A and have Resources A and S allocated.