The following paper was originally published in the Proceedings of the Sixth Annual Tcl/Tk Workshop San Diego, California, September 14–18, 1998 Creating a Multimedia Extension for Tcl Using the Java Media Framework Moses DeJong, Brian Bailey, and Joseph A. Konstan University of Minnesota For more information about USENIX Association contact: 1. Phone: 510 528-8649 2. FAX: 510 548-5738 3. Email: [email protected] 4. WWW URL:http://www.usenix.org/ Creating A Multimedia Extension for Tcl Using the Java Media Framework Moses DeJong, Brian Bailey, and Joseph A. Konstan University of Minnesota Computer Science and Engineering Department {dejong, bailey, konstan}@cs.umn.edu Abstract models in use today, it is no wonder the core Tcl group has shied away from attempting to directly As multimedia capable computers become cheaper provide multimedia support. One of the more popular and more pervasive in the consumer and corporate multimedia extensions for Tcl is the Continuous markets, and as the availability of digital information Media Toolkit (CMT) [10] developed at California- increases, the need for low-cost, cross-platform Berkeley by the Plateau project group. CMT is a multimedia applications will steadily rise. However, flexible, low-level multimedia toolkit supporting a because Tcl lacks native support for continuous variety of media formats, transport protocols, and media streams, such as audio, video, and animation, playout devices. Although CMT has proven to be it is not well suited for this emerging application very useful in our research, it suffers from the same domain. At the same time, Java now provides a set of problems as other Tcl extensions: class libraries, called the Java Media Framework (JMF), which provides the multimedia support that • Portability. Designing extensions that are Tcl lacks. With the recently introduced integration of portable across multiple flavors of Unix, Tcl and Java, Java can now be used to provide the Windows, and the Macintosh is a non-trivial cross-platform multimedia support required by Tcl; task. In fact, most extensions available today, whereas Tcl can be used to provide the easy-to-use including CMT, do not provide this level of programming environment required for building cross-platform portability. multimedia applications. In this paper, we introduce a • Tcl extension that provides a high-level scripting Compatibility. With each new release of the interface to the Java Media Framework. In addition, Tcl/Tk core, an extension developer must update we will highlight some interesting problems in the and test their extension to ensure that it still current Tcl/Java package as well as suggest some functions properly. Unfortunately, this has not potential solutions. This paper will benefit Tcl been easy as Tcl/Tk has changed substantially in programmers who would like to learn more about at least the following core areas; file I/O, image using Tcl to build multimedia applications, support, sockets, event handling, and object integrating Tcl and Java, or the multimedia support APIs, to name just a few. For some extensions, provided by the JMF. such as CMT, this has caused a lag of at least 6 months behind new releases of Tcl/Tk. Keywords • Configurability. Extensions cannot easily access Multimedia, Tcl extension, Synchronization, Jacl, the configuration information Tcl uses for its TclBlend, Java, Java Media Framework own compilation. Extension writers must effectively recreate this same information, which 1 Introduction is why almost every Tcl extension comes with its own set of configuration scripts. Tcl [9] has never directly supported continuous • media streams, such as audio, video, or animations. Dependencies. Many Tcl extensions are not Tcl programmers wanting to include these types of standalone; i.e., they require other extensions in media objects in order to create desktop conferencing order to function properly. For example, CMT applications, electronic manuals, voice activated requires the Tcl-Dp [12] extension in order to interfaces, or even simple multimedia presentations, leverage distributed services. As the number of have had to rely on fellow Tcl developers to build, extension dependencies increases, so does the distribute, and support multimedia extensions. Due to difficulty of upgrading an extension to support the huge variety of media formats, transport the latest release of Tcl/Tk. protocols, playout devices, and synchronization Within the last year, JavaSoft has introduced the Java sections of the paper, we present our conclusions and Media Framework (JMF) [4]. The JMF is a Java [1] the results of recent work to improve the Tcl/Java package that provides support for the local playback package. of a wide variety of audio (AIFF, AU, DVI, MIDI, and MPEG-1), video (H.261, H.263, MJPEG, 2 The Tcl/Java Package MPEG-1, AVI, QuickTime), and animation formats (Apple Animation). As the JMF evolves, it promises To better integrate Tcl and Java, several key to also support audio recording, audio mixing, video technologies have been developed. Jacl [6] is an capture, streamed playback, and emerging implementation of a Tcl interpreter written entirely in multimedia standards such as MPEG-4. The JMF the Java language. Jacl currently provides most of the already provides roughly the same functionality functionality of the Tcl 8.0 interpreter. TclBlend [5] offered by CMT, but without many of the problems provides a native code library that accesses the Java inherent in Tcl extensions as described above. Thus, Virtual Machine (JVM) using the Java Native the JMF deserves serious consideration as the Interface (JNI). The Java Package [5] is a set of Tcl underlying toolkit used in commercial multimedia commands providing access to the Java Reflection applications or even research projects. However, system. The Java Package provides commands that building multimedia applications in pure Java code allow a Tcl interpreter to both allocate and invoke using the JMF has several drawbacks: methods on Java objects. Tcl code using these commands will run seamlessly in either the TclBlend • Complexity. The JMF currently supports only or Jacl implementations. In addition, extensions primitive media events such as notification of a written in Java using the Java Package can be playback rate change. Exactly how these events accessed in either Jacl or TclBlend, and are instantly affect the rest of the application must be defined portable to a wide variety of systems without by the developer in low level Java code. configuration or recompilation. For the purposes of this paper, we use the term Tcl/Java package to • Lack of higher-level synchronization models. collectively refer to Jacl, TclBlend, and the Java Without higher-level synchronization support, Package. building complex multimedia applications becomes difficult. This is roughly equivalent to building user interfaces by invoking low level X 3 Integration Utilities routines instead of using a higher-level toolkit Before describing our multimedia extension for the like Tk. Java Media Framework in detail, several • supplemental packages need to be introduced. These Edit/Compile/Debug cycle. Strongly typed, utility packages provide the underlying support compiled languages like Java are great for required by our multimedia extension and consist of developing low level systems, but they make the following: poor tools for rapid prototyping and development of high-level multimedia applications. • Object-oriented support. The lack of object- oriented support in Tcl makes modular Because of these drawbacks, we argue that the JMF development difficult. For several reasons that would benefit from a scripting interface that would will be explained shortly, none of the current increase the usability and decrease the development object-oriented extensions were usable in this effort required to build complex multimedia project, and thus a Tcl-only object-oriented applications. Thus, the goal of this project was to use package was created. the Tcl/Java package [5, 6, 13] to provide a scripting • interface to the JMF called TJMF (Tcl/Java Media Event mapping system. A package to map Java Framework). This paper describes our experiences events to Tcl callbacks was created so that events and lessons learned from building this Tcl interface from the JMF components could be managed in to the JMF. In section 2, a general overview of the user-defined Tcl code. features already provided by the Tcl/Java package is • Pack geometry manager for the Java AWT. The presented. Section 3 outlines the utility packages that existing Java layout managers are inflexible and provide the foundation for the TJMF extension. In difficult to use. In response, a new layout Section 4, an example of using the TJMF extension manager, based upon the pack geometry manager to playback a MPEG movie as well as a detailed of Tk, was created for Java AWT components. discussion of the extension is provided. In Section 5, we reflect on our experiences and lessons learned • Package command for Jacl. Initial versions of from using the Tcl/Java package. In the remaining Jacl did not provide an implementation of the Tcl package command, but since our project allows for rapid prototyping of user interfaces. In required this functionality, the Jacl interpreter order to gain these same advantages when developing was extended with an equivalent Java interfaces with the AWT, a pack geometry manager implementation. was created by porting the packer layout algorithm to the AWT 1.1 LayoutManager interface. Java 3.1 Object-Oriented Support programmers can use this layout manager to gain pack functionality for AWT components. Although From our experience, the most severe impediment to this functionality could be accessed from Tcl using modular development in Tcl is the lack of object- the Tcl/Java package, the layout manager would be oriented support. Tcl has no built-in support for easier to use if wrapped in a Tcl command. For this aggregate data types, other than lists and arrays, and reason, a jtk::pack command was created that also lacks support for encapsulation of data.