Interactive Shared Educational Environment (ISEE): Design, Architecture, and User Interface

Xiangming Mu and Gary Marchionini School of Information and Library Science Interaction Design Laboratory, UNC-CH [mux, march]@ils.unc.edu

Abstract The Interactive Shared Educational Environment (ISEE) is an advanced real-time multimedia application that supports highly interactive collaboration and distance learning activities within a heterogeneous network context. The ISEE not only takes full advantage of fast LAN campus networks or Internet2 wide area networks by providing peer-to-peer multicast support, but it also can be used in the less advanced settings of home users. Media (e.g., digital video) are integrated into a desktop style interface in the ISEE. The ISEE allows users to interact with live multicasts , a shared web browser, shared video/audio with thumbnails for quick navigation, and text chat. Collaborative work is supported via a shared time line across the multiple ISEE tools. For example, each comment in the chat text panel is associated with a timestamp, which indicates at which point during the video viewing (i.e., in what context) the comment was made. Clicking that timestamp by another user will jump that user’s video player to the same timestamp. One challenge for a collaborative distance learning (CDL) system is to support a high degree of interaction between users and the video player due to the delays associated with re-buffering the video. With pre-buffering and a novel collaboration protocol, ISEE not only supports dynamic user-media interactions in real time, but also guarantees synchronization across participants. The system interface and architecture are discussed. 1. Introduction

Collaborative distance learning (CDL) applications are the result of many years of advances in computer network and computer technology. The number of Internet hosts has grown from 4 in 1969 to more than hundreds of millions today. The connection speed for the US backbone has grown from 50 kbps to several GBps [1]. Personal computers are powerful enough to easily process high quality multimedia information like real-time videos. Multimedia information (e.g. images, graphics, and videos) is widely produced and can be easily accessed through the fast growing Internet. Using multimedia, especially video, to help distance learning is the main characteristic of current CDL systems. Collaborative distance learning is a process that allows participants (e.g. students) to join in and form a learning group to exchange comments or multimedia information (e.g. images). During the learning process, a mediator (e.g. instructor) can join and guide the learning. Or it can be a process for several students to get together to overview the previous classes (videos) and to prepare for an examination. If guided under a mediator to watch a video, this type of learning is often referred as Distributed Tutored Video Instruction (DTVI) or Tutored Video Instruction (TVI). Studies [2-5] have indicated that the DTVI or TVI students outperform students who attended the live lectures in the traditional classroom and students who watched the live streaming lecture individually, in both the situations with or without tutors (mediators). The success of DTVI or TVI is attributed to the collaboration when the students watch the video. Collaboratively watching a video means a group of students share the same video at the same time. Physically the students can stay together or in distributed locations. The advantage of distributed collaborative learning is obvious: it increases the flexibility and scalability. Students located hundreds or thousands miles away do not have to travel a long distance to “form a group” and the number of attending students can be much more than a traditional classroom’s occupation. The system we will discuss in this paper belongs to this category. The video being watched in CDL system can be synchronized or non- synchronized. One characteristic for synchronized video watching is the so-called “what I see is what you see”. In most synchronized CDL systems, every student in a study group shares one copy of the video clip. Student can not control the video playback (pause, restart, etc.) unless he/she gets the control token based on various rules of floor control [13]. With the token, the student can pause the video at a time and incur a discussion. All other group members’ video players are also paused at the same time due to the synchronizing token mechanism. On the contrary, Non-synchronized CDL systems allow students watching and controlling their video independently without interrupting other group members’ watching. In addition to the video, other media or communication “channels” can also be added to build a rich, multiple dimensions collaborative learning environment. Image the following scenario:

Two UNC students, Mike in Mexico City and John in Asheville, are taking a class Multimedia Communication via distance education. Dr. Hughes, the instructor, lectures online and incorporates a variety of audiovisual materials supporting her lectures. John and Mike are required to watch a video clip: Senses and Sensitivity during the class. From their respective homes they start their CDL system and connect to the Internet. Immediately they see a live video from Dr. Hughes and they can also hear the live lecture. Dr. Hughes completes the lecture part of class and opens the discussion phase. Mike asks a question in the chat room of the CDL system:“ I am very interested in the structure of the human ear, where can I find additional information?”. Dr. Hughes sees the message from Mike in her chat room window. “I will give you a website”. After Dr. Hughes types this message in the chat room, she opens the shared browser and input the URL. Immediately, Mike’s and John’s browser are also redirected to the same URL as Dr. Hughes’. John also joins the discussion and sends a message to the chat room: “ Here is a picture about the ear structure, http://www.bioanim.com/CellTissueHumanBody3/uhoMorf1lgws.html ”. Mike sees the URL in the chat room and double clicks on it to load the picture into his browser. After a while, Mike recalls an example about antennae in the previous part of the video that is related to the current content . He drags his video slider control back to that point and plays the clip. Then Mike sends a message to the chat panel: “look at this example, how can the antenna (ear) of the mouse send the messages to their brain?” By looking at the time-stamp following by the sender’s name, Dr. Hughes realizes that Mike is talking about something early in the video. Clicking on the timestamp on the chat room text, the video player of Dr. Hughes immediately jumps back to the exact same timestamp as Mike’s. After browsing the example, Dr. Hughes says : “it works this way..” . Finally, when the class is finished, Mike, John and Dr. Hughes each save the annotations in the chat panel for later reference along with their personal notes.

In this scenario, communication components like the video player, chat panel, and shared browser are used combined to achieve an effective collaborative learning environment. Each component contains one or more communication channels: the video player provides visual and audio channels for both pre-recorded video and real-time live streaming; the chat panel enables text communication; and the browser builds a hypertext communication channel across participants. One of the challenges for CDLs is to provide a service to enable students to interactively browse video content while maintaining a state of synchronization in order to collaborate with others. For example, a student can “pause” the video to take notes, to ask a question or just go get a cup of coffee. When the student “resumes” the video after a while, he/she can “synchronize” his/her video with other group members at will. The synchronization part of the service is achieved in most collaborative CDL and Computer Supported Cooperative Work (CSCW) systems by sharing one copy of the video playback among all the participants (students). And the collaboration is achieved via the floor control mechanism. On one hand, students will be reluctant to control (i.e. pause) the video playback if they know all the other team members’ video player will also be affected. On the other hand, some students may dominate floor control. The Interactive Shared Educational Environment (ISEE) is an advanced real-time multimedia distance learning system that allows students not only control the playback of the video without affecting others, but also can synchronize with the group at any point. Another challenge for collaborative distance learning is how to effectively integrate multiple information channels into an efficient system. In the ISEE, a real-time video streaming channel, a shared browser channel, a pre-cached video channel, and a shared chat room channel are provided and integrated in a desktop style user interface (Fig.1). Highly interactive collaboration and distance learning activities within a heterogeneous network context are supported. Information is shared across these various information channels in the format of message objects. With the novel concept of “smart alignment”, these shared message objects allow students do “alignment” across channels. For example, double clicks on the URL presented in the shared chat room can align the shared browser and update it to the same URL. This paper is organized as follows: first, related research for CDL systems is reviewed. Next, the user interface as well as the various visual components in ISEE are introduced. Then a list of design requirements and the system architecture are described. Finally, innovative features of ISEE are presented and results from informal testing are discussed.

2. Related Work In terms of application domains, the current state of distance learning (CDL) applications can be grossly classified into four categories: general tools, educational tools, commercial tools, and research tools.

2.1 General Tools: A generic video player is implemented in such tools to support direct video playback. Video clips can be downloaded via the network or distributed through CD, or DVD. A number of such tools are available. Representative products include Microsoft Windows Media Player (www.microsoft.com), RealOne Player from RealNetworks(www.real.com), and Quicktime from Apple Computer Inc. (www.apple.com). The main advantage of these tools is that they are easy to use and provide control over basic functions such as pause, play, stop, and jumping backwards or forwards. The disadvantage for such applications under the CDL context is there is no support for communication and collaboration, for example, no video conferencing, chat, shared browsing or shared whiteboard are implemented in these generic tools.

2.2 Educational Tools: There are many production-level systems for using video in distance education. Notably, Stanford-online (http://stanford-online.edu) [6] provides online access to pre- recorded video of live courses. This technology is often referred to as on-demand streaming. Using this tool, the content of the video can be played by distance students via Internet connection. A synchronized slide displayer is also provided. Slides or screenshots of the scripts on the blackboard are synchronously displayed with the class video. Choice of video qualities is available for different network connection speeds. Other services such as digiscript (http://library.digiscript.com/) provide similar services for conferences and professional meetings. The advantages for such tools are ease of use ( no extra installation except a network connection), choice of connection speeds, and multiple information channels (visual, audio, and images). The disadvantage is problems for scalability (the total bandwidth on the server is limited), long delays for each reposition of the video playback (it takes time to re-buffer), and no collaboration services (real-time chat-room, shared browser, or shared whiteboard) are supported.

2.3 Commercial Tools: There are many commercial tools in the market that can be used for the purpose of CDL [7-8]. In terms of the media transmission technology, these tools can be roughly classified into two sub-categories: one that only supports one-way steaming, the other that supports two-way or multiple way communication. Representative products for one-way streaming includes RealONE Player (www.real.com), QuickTime (www.apple.com), Microsoft Media Player, Netshow and Vxtreme (www.microsoft.com). In the context of CDL, the one-way steaming can be utilized to broadcast either real-time lectures, or on-demand video streaming. The advantages for this kind of tools are their scalability and flexibility. A great number of users can receive the streaming lecture simultaneously with no restriction on the connection number. The disadvantages are no supports for two-way communication, and no collaboration among group members. On the market, there are various video conferencing and Internet meeting software, including: Intel video Phone(www.intel.com), White Pine’s CU- SEEME(www.cuseeme.com),Meeting Point, and Microsoft’s NetMeeting (www.microsoft.com) which supports real-time two-way or multiple way communication. As they are not specifically designed for CDL, the majority of the tools in this sub-category do not provide specific supports for collaboration and information sharing across students. For example, functions like integration of multiple information channels are not supported in most systems. Furthermore, scalability and adaptability are two main challenges. Limitations include: how to choose a videoconference system to adapt to various network connection speeds to support a large number of students to join in from distributed locations; how to allow students take full control of their video player, as well as synchronizing with other students and allowing students to save the logs or annotations for later use.

2.4 Research Tools: A number of tools or prototypes are available as part of larger research and development efforts. Some of the tools are MBONE tools such as vi (www- nrg.ee.lb1.gov/vic) and wb (www-nrg.ee.lb1.gov/wb), IRI [9] ,IRI-h[10], and virtual classroom systems [11]. Multicast is widely used in the design of the research tools. Multicast dramatically reduces the bandwidth consumption in the Internet, especially for high quality real time applications like the multimedia distance learning systems. Compared with multiple Unicast transmissions, the network latencies can also be highly reduced by adopting Multicast. The disadvantage for research tools is that more works are needed to make the tools and prototype more robust , more stable, and more usable.

2.5 Interactive Shared Educational Environment (ISEE) Although we can find various advantages in the applications discussed above and each of them offer a solution toward distance learning with video, the following functions are rarely covered, let alone integrated: 1. Provide students a system that supports multiple views of the video in the CDL context; allow students to preview the video or related annotation before a CDL class; allow students to view the video and make annotations in real-time; and allow students to overview videos and make annotations on videos from previous CDL classes. 2. Coordinate and integrate independent communication channels (video, audio, text chat, web browser) into a uniform user interface. Information can be shared information across these channels to achieve optimal system performance. For example, a timestamp appear in the chat room is linked to the video player’s timestamp and a single click on the timestamp in the chat room can update the video player to the same point. 3. Provide students the services for both interactively controlling the video player and synchronizing with other team members. 4. Accept and support metadata (like MPEG-7 compatible data in XML format) for the video and effectively present the metadata in the user interface. The Interactive Shared Educational Environment (ISEE) is a prototype application that implements all the above functions.

3. User Interface ISEE provides a desktop style Graphic User Interface (GUI) that integrates multiple communication components (Fig.1). In the ISEE, students (or instructors) can rearrange the layout of the GUI at will according to different learning contexts and personal preferences. The positions of various components can be rearranged by dragging and dropping and their size can also be changed (Fig.3). This is particularly useful when multiple communication components are open simultaneously in limited screen real estate. Such technology also provides a way to take full advantage of the high resolution of large screens. For example, students can enlarge the relative size of a video player in a high-resolution (1600*1248) monitor to get a better video display rather than use the default size, which is designed for a screen resolution of 1024*768. Another reason we choose a desktop style user interface for ISEE is the consideration of information flow management. With various information channels (text, images, multiple video and audios and hypertext), the issue of information overloading arises. A desktop with internal frames to contain alternative information channels will provides users (students) the choice to filter these multiple information flows by opening or closing the internal frames.

Pre-cached Interactive User name and icons Video Player (IVP): for Current Users Multicast Streaming Login check Video Player (SVP)

Shared Web Tooltip: indicate the current timestap and Shared Chat Room (SCR) : Browser (SWB) Storyboardthe duration for video with annotation and clip in the IVP timestamp followed by each message

Figure 1: Overall interface for ISEE

3.1 Streaming Video Player (SVP) For any real-time multimedia distance learning system, the key component is the video player that can receive streaming video sent from the instructor or a live stream sent directly from the classroom. In ISEE, after students successfully log in with the correct user name and password (Fig. 2), a button entitled “Live Streaming” is enabled and students can thus press the button to begin receiving streaming video multicasted from the remote server. An internal frame with a video player will pop up at the top-left of the interface with the same size of the sending streaming video (Fig.1). Students can determine the size by rearranging the frame’s size. A control panel is also presented beneath the Streaming Video Player (SVP) that allows students to pause/resume the stream, to adjust/mute the voice volume, and browse the media properties (media source, video/audio format, video encoding type, frame rate, and current time-stamp).

Figure 2: Log in to the ISEE

3.2 Interactive Video Player (IVP) One of our primary goals is to support usage of interactive video by students as they study. A second window/channel is provided in ISEE for local control of a pre- cached video on the client machine. Students not only can pause/resume the video playback, but also can determine what happens next. For example, students can jump forward/backward to an arbitrary point of the video and begin playing from that point, or they can fast-forward the video to quickly browse the content of the video. In some cases students need to pause the video and discuss with other students or ask questions to an online instructor when they come across difficulties to understand the video (an abstruse formula or a perplexing concept). For a synchronized steaming video player, pausing the player will lose the remaining stream up to the point it is resumed. This is not what students want. The Interactive Video Player (IVP) is provided in ISEE to allow students full control of the video player without losing information. Pressing on the resume button will resume the video at the same point the video was paused. The IVP can also support the synchronization video playback among members in a group. Figure 3. Example for re-arranging the size of IVP in the ISEE interface

After selecting a video clip from the drop-down list of available pre-cached video clips in the top panel of the ISEE interface (Fig. 4), the IVP will be presented in the center of the ISEE and begin to play back. Below the IVP is a storyboard that contains a series of frames, which refer to the start points of segments in the video clips (Fig.1). Single clicking on one of the frame in the storyboard will update the IVP to the same timestamp of that selected frame and begin the playback from there (like the functions provided by DVDs to let users select a segment to begin to play). At the same time the Shared Web Browser (SWB) will be updated to display a web page related to the video clips, such as an outline of the video (Fig.4). The more detailed introduction about SWB is provided next. A similar video control panel is also provided for IVP that can not only allow students to pause/resume or stop the video, but also allow student to jump forward/backward to an arbitrary position of the video by dragging the scroll bar beneath the video player (Fig.1). Students can even drag the scroll bar to quickly navigate the content of the video. We are investigating other surrogates in addition to the storyboard in the IVP. A tooltip along with the scroll bar indicates the corresponding time-stamp. An important innovation in the ISEE is the provision of the “synchronization” function for the IVP. When a student logs in to the ISEE, he/she will join a virtual learning group. Usually a mediator will also join the group to guide the study, but this is optional. The person who established the session serves as the mediator. The mediator can be a class instructor or a teaching assistant depending on the learning context. By a single click on the “synchronize” button, which is located on the top of the ISEE interface with the icon of an envelop and an arrow on the envelop (Fig. 1), users are able to synchronize their IVP with the IVP of the mediator. For example, in a virtual class scenario, the instructor acts as the mediator. Students join the class virtually from distributed locations and open their own IVP to watch the video playback. For students who join late, pressing the “synchronize button”, will immediately synchronize their IVP with the instructor’s to share the same video timestamp. That is the so-called “what I see is what you see”. During the class, students can navigate the video content by dragging the scroll bar back and forth, or jumping around by clicking on the frames listed on the storyboard, or pausing the IVP to take notes. Doing any of these operations results in the non-synchronization of their IVP with others. Clicking on the synchronize button “brings them back” immediately and synchronizes with the IVP of mediator.

Outline for video clip Select a video clips (HTML format) available

Figure 4: Example for displaying a class outline in the SWB after the video clip for that class is selected (Senses and Sensitivity lecture 1)

3.3 Interactive Chat Room (ICR) Chat rooms are now widely used online as a media for virtual communication. In ISEE, an Interactive Chat Room (ICR) is implemented (right bottom in Fig. 1). The student can post messages in the ICR to join a discussion on a specific topic, or can passively receive messages posted by other class members. One important characteristic for ICR is linking the chat discussion to other information resources. As with hyperlinks in an HTML page, a link mechanism is provided in the ICR that coordinates display in the IVP and shared web browser. Clicking one of the links in the ICR will update either the HTML page of the shared browser, or the current timestamp of the IVP. For example, consider a student, John, taking a class on web design. While watching the video “tutorial about web design” at 12min 33secs into the video, John posts a question “ how do I change the color of the font in this example?”. The message will appear on the ICR window of all the class members as follows:

JOHN: (12:33) how do I change the color of the font in this example? Suppose at this point, the instructor’s IVP timestamp is 23min, 22secs and the topic in the instructor’s IVP is “how to insert an image into your web page”. Most students are synchronized with the instructors’ IVP and have no idea to which “example” John refers. However, by double clicking the hyperlink timestamp (12:33) associated with John’s posting, another student, Sue, can immediately jump his own IVP to the timestamp of 12min,33secs and begin to play from that point . Sue now knows what John is talking about and she can post her response in the ICR “ you can…..”. After she is finished, Sue can come back to “re-join” the class by clicking on the “synchronize” button. ICR also supports hyperlinks. Any messages posted in the ICR that contains a substring started with “http://; HTTP://; WWW. ; or www.” is added asa hyperlink automatically to that substring . For example, if Sue sends a message that contains a URL, it will be presented like this:

SUE: (22:33) You can get more information about ear structure at http://www.sbu.ac.uk/~dirt/museum/ear_ct.html

By double clicking the underscored URL displayed in the ICR, the referred web page will be presented in another component of the ISEE—the Shared Web Browser.

3.4 Shared Web Browser (SWB) It is important that students can not only share text messages in the chat room, but also share other information media like images, pictures, or even web pages. A component called the Shared Web Browser (SWB) is supported in the ISEE (lower left in Fig.1). SWB is a web browser style component that can present hypertext web pages written in HTML. Hyperlinks are also enabled in SWB. Beneath the SWB in the same internal frame, a text field is provided that allows students or instructors to input URLs which will be propagated to all other students who joined the same study group or virtual class. That means all the group members are able to share hypertext web pages via SWB. For instance, the instructor can present class slides via SWB and students can exchange music, images, etc.

3.5 Other widgets 3.5.1 Login widget: This is a widget that provides user authorization to ISEE sessions. Before the user is authorized, buttons for chat and live streaming are disabled and no information about other users is presented.

3.5.2 User profile widget: Awareness of other group members is important for a virtual class. So a user profile frame is provided which lists all the users’ login names and their preferred image icons (upper right in Fig. 1). The view for this widget is dynamically updated and always displays the current members in the study group. We are investigating adding more metadata information about the user in the next iteration of the ISEE. For example, the number of messages posted to the ICR by a particular user. 3.5.3 User control panel At the top of the ISEE interface, a set of buttons are provided which allow users to open or to re-open the various components mentioned above. The “synchronize” button also resides in this panel. Additionally, each internal window can be minimized, maximized, or closed just as in other GUI applications.

4. Design requirements and system architecture

With a number of CDL tools available, the initial motivation for us to design the ISEE was the absence of systems that coordinated information resources into a seamless environment.

4.1 Design requirements for the ISEE 4.1.1 Flexibility: The ISEE system can be used in various tasks in CDL. It supports real-time live class Multicast, ad hoc study sessions (with multiple class videos available) controlled by students preparing for an examination or collaborating on a project, and virtual classes using pre-recoded lectures (videos) under the mediation of a Graduate Assistant. Another kind of flexibility provided by ISEE is the support of heterogeneous network settings. The ISEE communication protocol we defined (discussed later), uses only text messages to transmit among group members if the video is pre-cached and the Multicast channel is disabled. For slow network connections (i.e. phone line), the video can be obtained through other approaches, such as the distribution of CD instead of directly downloading from Internet or downloading in the background while other tasks are foregrounded. A peer-to-peer Multicast is also supported in ISEE to take full advantage of the advanced Internet connections like Ethernet or Internet2. In these cases, video clips can be dynamically updated by directly downloading from the server (Class Center). 4.1.2. Interactivity ISEE is not another video player passively receiving streaming video and playing it the beginning to end with few controls. ISEE supports two distinct classes of interaction: a person can use the information resources in private mode, thus interacting with video or text individually, and a person can interact with other people mediated by shared resources. The advantage of the ISEE is that users can move easily across these types of interaction, using some information resources locally and sharing others. Figure 5 summarizes these interaction features. We believe that the integration of these private and shared uses will lead to emergent new capabilities for students and teachers. Resource/Function Private Shared SVP Start/stop X X Audio/mute X X IVP Start/stop X X Audio/mute X X Jump X X Search X Annotate X X Overview/preview X SCR View notes X* X Take notes X* X Link X SWB Search X X** Browse X X**

Figure 5. ISEE Private and Shared Interaction Functions Note: * private notes can be made in a client-side word processor. ** shared URLs are passed through the chat resource.

4.1.3 Scalability ISEE can be used by a large number of distributed users. Using Multicast technology for the real-time streaming, ISEE greatly enhances the scalability by just sending one copy of the streamed content. The architecture (discussed later) is a peer-to- peer structure in general instead of client/server structure. So there are fewer demands on the server’s processing and bandwidth resources.

4.1.4 Portability Java and Java compatible APIs were used to develop the system. ISEE is a pure Java application and can be directly deployed on various operating system like Windows, Unix, Linux, or Solaris, and on various platforms like IBM compatible PCs, Macintoshes, or Solaris workstations with only minimal modifications. This will save time and effort for extending, re-developing, and adapting the source code.

4.1.5 Extensibility Based on the Real-time Transport Protocol (RTP), Multicast streaming is used for real-time video. Currently only one pair of Multicast video/audio channels from the mediator to students are supported in ISEE. It will be straightforward to extend this design strategy to a peer-to-peer architecture in advanced networks such as Internet2. 4.1.6 Interoperability Metadata, also called ‘data about data,’ provide information and descriptions on the content of the video. In ISEE, we include metadata on video properties, video summary ( keyframes displayed in the storyboard that presenting segments of the video), tooltips for the key frame, and video outline in hypertext HTML format. Even though there are multiple choices like Dublin Core(http://purl.org/DC), The Society of Motion Picture and Television Engineers (SMPTE), Metadata Dictionary (www.smpte.org), OpenSMEF from Standard Media Exchange Framework(SMEF), INDECS (http://indecs.org), we decide to use the MPEG-7 as our metadata standard. XML is adopted to define and to describe the definitions and definition schemas based on MPEG- 7. Some of the reasons we choose MPEG-7 are: it is media independent, it is supported from both the academic and commercial sectors, and it has a rich set of descriptions. And more importantly, such a metadata schema provides the data interoperability for our ISEE system.

4.2 System architecture: 4.2.1 Java Shared Data Toolkits (JSDT) In ISEE, a Session Sever (SS) is implemented on top of the Java Shared Data Toolkits (JSDT) (www.javasoft.com) Application Programming Interfaces (APIs). A user ( a student or an instructor) can initialize and start a new session by sending a request to the session manager which is running on the session server. Whenever a session is started, other users can log in or leave that session freely. Each user is treated equally in a session as one Session Client (SC). Through the SS, each SC can send messages to all other SCs.

Session Session Internet Session Server Internet Client Client(SC) (SC)

Session Session Session ClientSession ClientSession ClientSession Client(SC) Client(SC) Client(SC) (SC) (SC) (SC) Figure 6. Relations between Session Server and Session clients and the peer-to-peer structure among Session clients.

4.2.2 Java Media Frame (JMF) JMF is the platform that we used to develop video players for both ICP and SVP. A rich collection of APIs are provided by JMF for video playback, video capturing, and video conference (www.javasoft.com/jmf). JMF supports most standard video formats, such as Cinepak, MPEG, H.261, H.263 and JPEG. For streaming video application, RTP is supported in JMF, which facilitates our implementation of transmitting, receiving, and processing Multicast video media. The version of APIs we used for ISEE development is JMF 2.1.1a.

4.2.3 Peer-to-peer Multicast Multicast is used for the transmission of video streams. In the current version of ISEE, only one-way multicast is implemented. Students can log into a session (group), and after authorization begin receiving multicast streams from the live class or from the instructor. As opposed to traditional client/server architecture, the instructor is also a common member for a given session. Thus it is straightforward to support multiple channels for multicast from other students. This is the so-called peer-to-peer multicast architecture.

Live Class ICEE Live Class Multicas t network

ICEE ICEE ICEE

Figure 7. Multicast of Video Streams Supported in ISEE. Solid lines show the current streaming directions implemented in ISEE. Dotted lines show peer-to-peer multicast architecture

4.2.2 ISEE communication protocol Each message sent to and received from the SS is encoded and decoded based on a communication protocol we defined for the ISEE. This protocol covers five types of information that are transmitted across SS and SCs. They include: comment messages issued from a user (session client), the status information ( an user joins/leaves the session), the hyperlink/URL (to be shared across session members), the font color for messages sent from a SC ( to display messages on the chat room), and the video timestamp from the mediator SC (so the SC can synchronize with each other). After being decoded, these messages can be utilized by the SC that receives it to trigger corresponding actions (table 1). For example, if the messages received by a SC contain a chat message, it will be presented on the ICR of ISEE; if a timestamp is identified, it will be presented on the ICR with a link that can update the IVP. The font information received will decide the color of the message presented on the ICR (to help distinguish different participants). The URL information will provide a hyperlink on the ICR. The following pair of tags are used in the ISEE communication protocol:

Tags Functions … Text message from users … Color of the text from users … The status information for users … URL to be shared … The timestamp for synchronization

Table 1: Tags used in the ISEE Protocol for Information Sharing

5. Innovations in the ISEE The ISEE provides several innovations. 5.1 Private/Shared binary learning space Studying in a conventional live classroom, students share common learning processes: they see the same examples, listen to the same lecture from the instructor, join the same discussion, and look at the same slides provided by the instructor. However, students usually cannot control the pace of the class: they cannot slow it down when coming across problems, or speed it up for the familiar content. The same problem exists in most collaborative CDL systems. For most collaborative CDL systems, a shared video stream (real-time or pre- recorded ) is available for students to share the learning process. But either the students are not allowed to control the playback of the video stream, or they are reluctant exercise control (e.g. stop or rewind) because they usually do not want to interrupt other members’ watching, especially when the students are not familiar with each other [28]. And in such systems, any operations on the streaming video will lead the video player to re-buffer, which is time consuming, especially for slow network connections. But students may still prefer to have video control functions (e.g. stop/resume, jump backward/forward). In order to test our simple hypothesis, an informal survey was conducted. In our survey, we walked around the campus of the University of North Carolina at Chapel Hill and interviewed 20 students with the following question : Imaging you are a member of a virtual class with 200 students and you are now watching a video synchronously with others. You can discuss the content of the video with others through a chat room. Do you think the control functions (like pause/resume, jump backward/forward) are necessary and helpful to your study if you can always synchronize with others at will? How about if you can not synchronize with others after you exercise control?

18 of the 20 students said the control functions are necessary and helpful. One said she did not know and one said he did not care. If no synchronizing function is available, 13 of the 20 students answered they needed the control functions. One student said he should be able to play back the video segments if he missed something important. Clearly, this was not a controlled user study, but this informal assessment suggests that students do want to have control over video they use in learning. However, video control and synchronization are conflicting operations and most of the collaborative distance learning systems which support synchronizing video do not provide functions to let users fully control the playback of the “shared” video. ISEE provides a shared/private binary mode mechanism that not only provides a private version of the video player, which allows the students to fully control the playback of the video, but also provides a synchronizing function (synchronize button) that allows the students to synchronize their video player with other students with just a single click. Students can easily shift between the two modes. Any operations to control the video player will switch from shared mode to the private mode. A single press on the synchronizing button will automatically bring it back to the shared mode.

5.2 2D video navigation For the distance learning activity, there are two basic “dimensions” involved: a temporal dimension and a spatial dimension. In the video playback, the temporal dimension is defined as the process from the point when the video starts to the point when the video ends. For example, real-time live video streaming is a one-way linear process in the temporal dimension. The student is unable to stop, rewind, or jump around in this dimension. The temporal dimension is definded by the duration of the video. For distributive collaborative learning, students are arrayed in various locations. We define this as the spatial dimension of distance learning. The spatial dimension is determined by the current active number of students in a specific study session. Thus the spatial dimension will change when students join or leave the session. Learning individually has no variation on the spatial dimension. Both the temporal and spatial dimensions exist in most CDL systems. However, in most cases students can only work in one dimension. For example, students are able to download a video clip and jump around in the temporal dimension individually, but unable to synchronize with other students. Or students can join a collaborative system and share information across group members who are located in the spatial dimension but can not control the shared video player in the temporal dimension. Students cannot jump backward or forward in the streaming video. ISEE is designed to allow students be able to navigate in both the temporal and spatial dimensions. We call it a 2D distance learning system. For a pre-cached video in the IVP, a scroll bar is provided (center of Fig.1) to allow students determine what to see next in the temporal dimension. A single click on the “Synchronize” button will immediately re-synchronize the student’s IVP with the mediator’s, which is also synchronized by most group members. With the Interactive Chat Room (ICR), students can exchange messages with other group members located remotely in the spatial dimension. Timestamps following the username of each message make it quite straightforward to track the timestamp of the message sender’s IVP.

5.3 Smart Alignment In ISEE, services are provided for students to receive information from multiple channels: real-time video streaming channel, shared browser channel, pre-cached video channel, and shared chat room channel. A mechanism called “smart alignment” is provided in ISEE that enables students to automatically align one channel with another. For example, a single click on the timestamp followed by the sender’s name in the chat panel will align your IVP with the sender’s IVP. A double click on the hyperlink in the chat panel will update the URL of the shared browser. Pressing the “synchronize” button will align your IVP with the mediator’s. Inputting an URL in the text field beneath the SWB will “broadcast” that URL to all the group members and immediately update all the SWB.

6. Pilot user study and future work 6.1 pilot user study A pilot test of the ISEE was conducted on April 2, 2002 in a class in the School of Information and Library Science, University of North Carolina at Chapel Hill. There were 37 students enrolled in this class and 8 students volunteered to test the ISEE. Except for a web page to illustrate how to download and install the ISEE on their laptops, we did not provide any training on how to use ISEE. The subjects (students) had ISEE installed in their own laptops before the class and connected with the class through the wireless Ethernet. The wireless network is Multicast enabled. The class lasted for about 1 hour and 15 minutes. After the class started, subjects were asked to go to places outside the classroom with their laptop and to login as a virtual student. The live class was captured by a IBM ThinkPad UltraPort Camera which was mounted on a laptop and the live streaming video was sent out using Multicast. The Instructor also logged in to mediate and answer questions. A video clip, Senses and Sensitivity 1, was downloaded to the subjects’ laptops before class. During the class, the instructor opened the ISEE user interface and displayed it through the project. All the students in the classroom could see the operations of the instructor as he used ISEE. Fifteen minutes before the class was over, subjects were invited to return to the classroom for a brief discussion about the ISEE. Finally, after the class, several students and the instructor were interviewed. The results for the class discussion and the interviews are summarized as follows: From the students’ point of view: 1. Students were excited about the tools. With no extra equipment, a very simple installation and no special training, they could see the live video of the class from other places. 2. The audio channel is very important. They could accept low quality video, but the audio needed to be high quality so they could feel they are in the class. They complained that sometimes they could not hear clearly from the live classroom stream even though the video was clear enough. 3. They suggested that any actions issued from the instructor should be “broadcast” to students. For example, when the instructor paused the video playback and began to discuss it, the remote students do not realize it and their IVP still played the video. 4. There were still some bugs in the system. For example, the chat room sometimes failed to refresh the screen freezes for a while until a new message is received or sent. From the instructor’s point of view: 1. Even with a carefully planned presentation, there was substantial overload. The instructor had difficulty reading the remote chat while lecturing and leading discussion. 2. There is need for an assistant. Having someone control the live classroom camera is important so that the camera can capture whomever is talking rather than being fixed on one scene. Likewise, having an assistant read and respond to the chat is necessary during lecture/presentations and helpful during class discussions.

From the saved logs of the chat room we found that most of the conversation was about how to use the system instead of focusing on the content of the video and class. For example, except the hellos from each user, discussions about how to control the two sound channels ( from video clips and from live class) and how to mute one of them, about how to synchronize the video and complaints about the sound and excitement upon finding new functions were the dominant comments in the chat room. 6.2 Demos An earlier version of ISEE has been demonstrated in the NCNI student fellowship presentation conference (RTP,2002). The demo was manipulated by the first author who developed the system. Feedback was very positive with great interest in Smart Alignment, 2D navigation and the private/shared view functions demonstrated by ISEE. Suggestions included further development of peer-to-peer multicast and interface design. Another demo will be held at that Internet2 Spring 2002 meeting in Alexandria.

6.3 Future work: Before we say that ISEE provides some helpful functions for Collaborative Distance Learning, formal user studies are needed to evaluate the user (instructor and students) acceptance of the technology and the effects of the technology on learning. We are planning various user studies next: lab tests, interviews, surveys, classroom testing for a whole semester, and the applications of ISEE in other settings like discussions and seminars. Other work includes fixing the existing bugs, equipping some high quality capturing device for audio, adding shared whiteboard, providing real-time image (frame) grabbing, presenting a thumbnail of frames in the chat room and extending the peer-to- peer architecture.

Acknowledgements: This work was supported by grants from the North Carolina Networking Initiative (www.ncni.net) and NSF grant number IIS 0099638. Thanks are also due to instructors and students who have observed and used ISEE.

References: 1. R.H Zakon, “Hobbes’s Internet Imeline” http://info.isoc.org/guest/zakon/Internet/History/hit.html 2. Gibbons,J.F., Kincheloe, W.R., and Down, K.S.(1977). Tutored videotape instruction: a new use of electronics media in education. Science. 195:1139-1146. 3. Stone,H.R.(1990). Economic development and technology transfer: Implications for video-based distance education. In M.G.Moore(Ed.), Contemporary issues in American distance education(pp231-242). Oxford, England: Pergamon. 4. Sipusic, M., Pannoni, R., Smith, R., Dutra,J., Gibbons., J.and Sutherland, W. (1999). Virtual Collaborative learning: A Comparision between Face-to-Face tutored Video Instruction and Distributed Tutored Video Instruction(DTVI). Sun Microsystems Laboratories, Inc. TR-99-72. 5. Smith, R., Sipusic, M., and Pannoni, R. (1999). Experiments Comparing Face-to- Face with Virtual Collaborative Learning. Sun Microsystems Laboratories, Inc. TR-99-0285. 6. D.Harris and A.DiPaolo, “Advancing asynchronous distance education using high –speed networks” IEEE Trans.Education. vol 39, pp.444-449, Aug 1996. 7. B.Peyser, which streaming media tool do I need? http://www.streamingmediaworld.com/gen/tools/ipw/ (2000) 8. community tolls evaluation: http://cscw.telin.nl/communitytools//products/chatspace.html 9. Maly,K., Abdel-Wahab,H., Overstreet,C.M., Wild,C., Gupta,A., Youssef,A., Stoica,E., and Al-share,E. 1997. Distance learning and training over intranets. IEEE Internet Comput. 1,1(May/June), 60-71. 10.Maly,K., Abdel-Wahab,H., Wild,C., Overstreet,C.M., Gupta,A., Abdel-hamid,A., Ghanem,S., Gonzalez,A. , and Zhu,X. 2001. ACM Journal of Educational Resources in Computing, Vol. 1,No.1. 11. J.-N. Hwang, S.G.Deshpande, and M.-T. Sun, “A virtual classroom for real-time interactive distance learning” , Proc. ISCAS,vol.3, June 1998, pp.611-614. 12. JJ Cadiz, Anand Balachandran,Elizabeth Sanocki, Annop Gupta, Jonathan Grudin,Gavin Jancke. Distance Learning Through Distributed Collaborative Video Viewing. 2000. MSR-TR-2000-42. http://www.research.microsoft.com/research/coet/DTVI/TRs/paper.pdf 13.H.P. Dommel, J.J. Garcia-Luna-Aceves, Floor control for multimedia conferencing and collaboration. Mulitmedia Systems, vol 5, 23-38, 1997