<<

Special Issue Advanced Technologies and Solutions toward Ubiquitous Network Society

Communication Door: Real-Time Middleware

By Masahiro TABUCHI,* Kazuaki NAKAJIMA,* Akiko KAWAMOTO,* Takashi OSHIBA* and Shujun YOSHIZAKA*

We describe a real-time communication middleware system that enables solution providers to ABSTRACT construct various applications, such as web conferencing systems and IP-based call centers. The middleware platform, called Communication Door, is a web-based application platform for real-time groupware. It includes a real-time web sharing function implemented with an HTTP-based push protocol and an audio-video communication function. By using this middleware, many real-time groupware applications can be customized by rewriting their designs and functionalities through a server-side web application while utilizing thin clients. Solution providers can customize the user interface design and the functions of real-time groupware by revising their server-side web applications. In this paper, we describe the architecture of the middleware and its applications.

KEYWORDS Real-time communication, Web conferencing, Groupware, Web application, Middleware

1. INTRODUCTION lutions in the same manner as the development of web applications. It supports three major features: This paper describes a real-time communication 1) real-time synchronization of contents among web middleware platform called Communication Door. browsers connected to the same web application; This system enables solution providers to develop 2) real-time voice and video communication; and tailored solutions, such as VoIP (Voice over IP), web 3) an HTTP-based push server that can autono- conferencing, and call center applications, for various mously send differences in data maintained by the customers. server to web browsers connected to the application Recent progress in network access technologies (as compared to a general HTTP server, which can such as ADSL (Asymmetric Digital Subscriber Line) send data only when a web browser requests it). By and FTTH (Fiber To The Home) has increased the use utilizing these capabilities of Communication Door, of broadband connections. Broadband various communication services can be developed. Internet is becoming a new infrastructure for commu- nication services, and it is reducing the costs of voice 2. OUTLINE OF COMMUNICATION DOOR communication. Customers also now require commu- nication services that provide higher value than con- Communication Door is a middleware application ventional voice services. To satisfy various require- supporting various interactive communication forms ments for communication services, solution providers via broadband Internet. It offers solution providers should rapidly deploy tailored solutions to their cus- various functions for real-time communication. tomers. Since conventional communication services, such as service, require both dedicated ter- 2.1 Purpose minals and a dedicated network in general, it is too The purpose of Communication Door is to provide expensive to rapidly satisfy varied customer require- leverage to enlarge communication solutions in busi- ments. ness by integrating technology with net- Communication Door is a middleware platform working technology. that enables solution providers to develop custom so- As a new infrastructure for communication ser- vices, broadband Internet reduces the cost of voice communication, as in the case of telephone service. As a result, customers require higher value communica- *Internet Systems Laboratories tion services, such as web conferencing and

176 NEC J. of Adv. Tech., Summer 2004 Communication Door: Real-Time Communication Middleware web-based call centers, as compared with voice com- Many desktop conferencing systems based on ITU munication services. standards such as H.323 and T.120[9] have been de- To satisfy these requirements, solution providers veloped. NetMeeting[10], developed by Microsoft Cor- must rapidly develop various solutions suited to their poration, is an example of a desktop conferencing customers’ businesses. Therefore, solution providers application based on the H.323 standard. Recently, require middleware enabling this sort of rapid devel- other desktop conferencing systems have been based opment based on their customers’ requirements. on proprietary protocols. Examples include Macromedia’s Flash Communication Server[11] and 2.2 Related Work and Issues Microsoft’s Office Live Server. Several researchers[1-5] have proposed various These systems have rich functionality but require a communication functions that will be required in a higher level of expertise for solution providers to uti- real-time groupware system. These previous ap- lize them in customizing applications. proaches have been implemented as dedicated sys- tems. Therefore, custom applications based on these 2.3 Concept of Communication Door systems cannot work in a general-purpose system. As described above, the purpose of Communication Other researchers[6,7] have proposed web-based Door is to enable solution providers to develop com- groupware systems. The benefit of this approach is munication systems in the manner of web develop- the ease of customizing applications by using develop- ment rather than the development of dedicated sys- ment tools for web applications. Web-based applica- tems. Users of web applications can select services tions, however, cannot support real-time functions according to their businesses by specifying URLs because of the standard behavior of HTTP. Specifi- (Uniform Resource Locators) in their web browsers. cally, HTTP is a transactional protocol, that Users of Communication Door can thus select real- a web browser can update contents on-screen only by time communication services in this way, by specify- accessing an HTTP server. ing URLs. Another previously proposed approach, called “e- Figure 1 shows the relationship between real- office”[8], supports real-time web conferencing but time communication services and Communication requires a dedicated server for real-time communica- Door. Such real-time communication services as TV tion. In this system, a user’s view is produced by a telephony, web conferencing, and call centers can be web application, but the application program has to implemented as applications based on Communica- be implemented as a client program of the dedicated tion Door. Communication Door supports the common server. functions of real-time communication, including voice

Fig. 1 Outline of Communication Door.

NEC Journal of Advanced Technology, Vol. 1, No. 3 177 Special Issue and video communication, real-time sharing, and con- browsers. Communication Door supports functions tent synchronization. for propagating the differences in web contents, i.e., The advantages of Communication Door are that it HTML documents, to reduce the time for sending supports flexible customization and ensures real-time data. When Push Server sends both an identifier and performance. the differences in an HTML document to a web browser, LiveComm, running on the web browser, can (1) Flexible Customization identify parts of the HTML document from the identi- Communication Door is designed to support flex- fier. Then, LiveComm replace the specified parts in ible customization as a middleware platform for web the document with the differences sent by Push applications. The methodology of developing web ap- Server. plications is widely accepted as providing such flex- ibility. Conventional web applications, however, do 3. ARCHITECTURE not support real-time updating of the contents of web browsers. Our middleware extends to both server Communication Door is designed as a system plat- technology and client technology. The server technol- form for web-based real-time communication applica- ogy, called Push Server, pushes data from a web tions. application to web browsers. The client technology, called LiveComm, receives the data sent by Push 3.1 Overview Server and updates the contents of a web browser. Communication Door consists of a web server, A web application based on Communication Door Push Server, and web browsers. It is characterized by can update the contents of all connected web browsers the establishment of a persistent, real-time connec- in real time. Thus, by utilizing the APIs (Application tion over HTTP with low latency time. Applications Programming Interfaces) of Push Server and based on Communication Door can support real-time LiveComm, solution providers can develop real-time communication with voice, video, image, and text communication services as web applications synchro- data over HTTP. nizing among web browsers. Communication Door provides solution venders with server-side modules and ActiveX controls run- (2) Ensuring Real-Time Performance ning in web browsers, as shown in Fig. 2. By using Communication Door is designed to ensure the the APIs of these modules, solution venders can de- real-time performance of web applications. As men- velop web applications based on ASP (Active Server tioned above, Push Server can send any data to web Pages) to control the modules.

Fig. 2 System architecture.

178 NEC J. of Adv. Tech., Summer 2004 Communication Door: Real-Time Communication Middleware

3.2 ASP Applications and COM Interfaces cation, it downloads an HTML document with a script The ASP applications are implemented on a web program calling the control. After downloading the server, by applying IIS (Internet Information Server), document, the browser interprets it and activates the developed by Microsoft Corporation. An ASP applica- control. The activated control then connects to Push tion consists of HTML documents that include script Server and establishes a persistent bi-directional con- programs calling COM (Component Object Model) in- nection. After being connected, Push Server can send terfaces. Since Communication Door supports COM and receive data to and from the control in real time. interfaces for shared resources and Push Server, pro- Also, web applications can request Push Server to grammers can develop ASP applications to control send data to the ActiveX controls on web browsers exclusive access to shared resources and Push Server. through Push Server’s API. In general, ASP applications create HTML docu- ments for each web browser accessing the applica- 3.5 Audio/Video Relay Server tion, and they send these documents back to the The audio/video relay server relays the audio and browsers. The browsers then display the documents video streams sent by the audio/video sender-receiver through HTML rendering. As a result, ASP applica- (ActiveX control) on one browser to the audio/video tions can customize individual users’ views by creat- sender-receiver controls on others. Since the server is ing individual HTML documents. implemented on the HTTP tunneling module, it can relay audio/video streams through firewalls or NAT 3.3 ActiveX Controls routers in an Intranet or the Internet. ActiveX controls are modules running on Microsoft’s Internet Explorer web browser to extend 4. DEVELOPING AN APPLICATION WITH COM- its functionality. Communication Door supports two MUNICATION DOOR major ActiveX controls: an audio/video sender and receiver, and a data sender and receiver. The audio/ We now explain how to build and customize a web video sender encodes audio/video streams and sends conferencing application, as a typical example of a them to an audio/video relay server. The receiver web application, by using Communication Door. receives the encoded audio/video streams from the relay server, decodes them, and plays them back. The 4.1 Web Conferencing System data sender sends data created by script programs A web conferencing system is one example of a running on web browsers to Push Server. The re- desktop video conferencing system. In a conventional ceiver receives the data from Push Server and trans- desktop conferencing system, users have to install a fers it to the script programs on the web browsers, client application and configure its settings in order which render the data. to connect to the conferencing server in their network. An HTTP tunneling module provides these Users of web conferencing systems, however, need ActiveX controls with persistent bi-directional con- only a web browser. First, a user accesses a web nections, i.e., an upload connection and a download server for conferencing, then he or she downloads the connection, between the controls and Push Server client application. The downloaded application is acti- through firewalls, without utilizing dedicated lines vated automatically by the web browser and joins a [14,15]. virtual conferencing room on the server. In many web In the conventional way, a client application based conferencing systems, the client application is imple- on HTTP standards connects to a web server to utilize mented as an independent application, i.e., as a user resources on the server, and it disconnects afterward. interface or a conferencing function built into the Since stream data, such as audio and video, continue application. Also, many web conferencing systems are to flow, persistent bi-directional connections are re- designed as package applications. As a result, the quired. Also, through such bi-directional connections costs of customizing web conferencing systems tend to for text and image data, client applications can send be higher. and receive data to and from server applications in In contrast, a web conferencing system based on real time. Communication Door is implemented as a web appli- cation. Since it describes the user interface through 3.4 Push Server an HTML document, the user interface can easily be Push Server sends data to the data sender-receiver customized by upgrading the web application to pro- ActiveX controls on the web browsers via HTTP or duce an appropriate HTML document. HTTPS. When a web browser accesses an ASP appli-

NEC Journal of Advanced Technology, Vol. 1, No. 3 179 Special Issue

4.2 Design System Behavior ment. Since the application can maintain differences Developers can design the system behavior of web between the original document and the new docu- applications based on Communication Door, as we ment, it can immediately send the differences to the describe here for the example of a web conferencing web browser by calling Push Server. The differences system, as illustrated in Fig. 3. are received by the data sender-receiver control on First, a user accesses the initial page on the web the web browser. After receiving the differences, this server and downloads the ActiveX controls (the audio/ control transfers them to the script program in the video and data sender-receivers) to his computer. HTML document, which then rewrites the HTML Then, he navigates to the login page on the web document displayed in web browser accordingly. As a server and logs in with a username and password to a result, the HTML documents displayed in each web virtual conferencing room. After the login procedure, browser are synchronized in real time. he downloads the initial HTML document including In addition, the audio/video sender-receiver on the the definition of the user interface. For example, the user’s web browser automatically sends his voice and main view shown in Fig. 4 is described by an HTML video data to the audio/video relay server. document consisting of many frames, including a main frame for displaying shared documents, a video 5. OTHER APPLICATIONS frame, and frames for tools. After loading the initial HTML document, the web We now describe two other applications based on browser activates the ActiveX controls. The activated Communication Door: an IP-based call center system, controls establish persistent connections to both Push and a TV-VoIP system. Server and the audio/video relay server. When a user manipulates objects in the frames for (1) IP-Based Call Center System tools, a script program in the initial HTML document An IP-based call center system enables users to creates data and sends it to the web application contact a company’s help desk. Figure 5 shows ex- through the HTTP connection. The web application amples of screens for an IP-based call center system: processes the data and creates a new HTML docu- the left side shows the customer’s view, while the

Fig. 3 Primary system behaviors.

180 NEC J. of Adv. Tech., Summer 2004 Communication Door: Real-Time Communication Middleware

Fig. 4 Web conferencing view on-screen.

(a) Customer view. (b) Operator view.

Fig. 5 IP-based call center screens. right side shows the operator’s view. Both the cus- can add image communication and a function for tomer and the operator can communicate with each sharing information to an ordinary VoIP system. A other while watching the same web page. Since the user can search for and confirm other users’ statuses shared page is synchronized, the images, input text, and call them. Users can also communicate with each and pointers on the page appear on both screens in other by sharing contents, such as written materials real time. As a result, the customer can consult with or papers, as shown in Fig. 6. Thus, they can achieve the operator through voice and video communication close meeting and send various notifications to re- while watching a synchronized web page. mote locations as if they were actually maintaining face-to-face contact by viewing each other’s facial ex- (2) TV-VoIP System pressions and sharing contents. A TV-VoIP system based on Communication Door

NEC Journal of Advanced Technology, Vol. 1, No. 3 181 Special Issue

Fig. 6 TV-VoIP view on-screen.

Since these applications, including the web Study in the Design of Computer-Supported Meeting En- conferencing system, are implemented as web appli- vironments,” Proceedings of CSCW ’88. Portland, Oregon, 1988, ACM SIGCHI and SIGOIS, pp.257-270. cations running on a web server, users can select [3] H. Ishii, and M. Kobayashi, “Clear Board: A Seamless communication services by simply entering URLs in Medium for Shared Drawing and with Eye their browsers. Contact,” Proceedings of CHI ’92, ACM, pp.525-532, May 1992. [4] H. Watanabe, S. Sakata, et al, “Distributed Multimedia 6. CONCLUSION Desktop Conferencing system MERMAID,” IPSJ Jour- nals, 32, 8, pp.1200-1208, 1991, in Japanese. We have described Communication Door, a [5] T. Suganuma, S. Fujita, et al., “Flexible Videoconference middleware platform that enables solution providers System Based on Multiagent-based Architecture,” IPSJ Journal, 38, 6, pp.1214-1224, Jun. 1997, in Japanese. to build various communication services as web appli- [6] S. Ichimura, R. Nakamura, et al., “QuickBoard: A Web cations. By using Communication Door, solution pro- System for Realtime Delivery of Computer Screen,” IPSJ viders can implement communication systems tai- Journal, 45, 1, pp.244-254, Jan. 2004, in Japanese. lored to their customers’ requirements in the same [7] Y. Shinazawa, T. Uetake and S. Takao, “The Electronic Bulletin Board System ‘S-Board,’” IPSJ Journal, 45, 2, manner as the development of web applications. It is pp.623-634, Feb. 2004, in Japanese. easier to customize web applications than conven- [8] K. Sakakibara, M. Kato, et al., “‘e-office’: A Communica- tional client-server applications. Hence, Communica- tion Support System for Distributed Workers Using Me- tion Door provides a system platform for real-time dia Space,” IPSJ Journal, 43, 8, pp.2821-2831, Aug. 2002, in Japanese. communication services. [9] T.120: ITU website (http://www.itu.int/). Our future work is to ensure scalability for a large [10] Microsoft Corporation: NetMeeting, (http:// number of users. For example, in a web conferencing www.microsoft.com/windows/netmeeting/). system, increasing the number of participants re- [11] Macromedia Corporation: Flash Communication Server, (http://www.macromedia.com/software/flashcom/). duces the performance of real-time communication. [12] Microsoft Corporation: Office Live Communication We plan to extend Communication Door to ensure a Server, (http://www.microsoft.com/office/livecomm/ high level of scalability. prodinfo/default.mspx). [13] Yano Economic Research Institute: The actual condition and a strategy view of a call center / contact center REFERENCES market, Dec. 2003, in Japanese. [14] K. Nakajima, T. Oshiba, et al., “A Real-time E- [1] M. Stefik, G. Foster, et al., “Beyond the Chalkboard: Marketplace System for Advertising Opportunities on Computer Support for Collaboration and Problem Solving Datacasting,” Tenth International World Wide Web Con- in Meetings,” Communications of the ACM, 30, 1, pp.32- ference, May 2001, (http://www10.org/cdrom/posters/ 47, Jan. 1987. p1133/index.htm). [2] M. Mantei, “Capturing the Capture Concepts: A Case [15] K. Nakajima, T. Abe and M. Tabuchi, “A Real Time

182 NEC J. of Adv. Tech., Summer 2004 Communication Door: Real-Time Communication Middleware

Sharing Control System for Multi-Media Data on the [18] A. Kawamoto, S. Yoshizaka, et al., “A Proposal of Con- Internet,” IPSJ DiCoMo2000 Symposium, pp.265-270, trolling Functions of Terminal in Realtime Communica- Jun. 2000, in Japanese. tion,” IPSJ 66th Annual Report 5B-02, Mar. 2004, in [16] K. Nakajima, S. Matsumoto, et al., “Servers Linkage Japanese. Middle-ware for Real-time Synchronized Web Page Asso- [19] Communication Door website, (http:// ciated with Viewer Participation Programs,” IPSJ www.communicationdoor.com/). DiCoMo2000 Symposium, pp.507-510, Jul. 2002, in Japa- nese. [17] K. Satoda, J. Takada, and Y. Hiraike, “Selection Control- ling Scheme of MPEG-4 Frame for Live ,” IPSJ 61st. Annual Report 3N-04, Sep. 2001, in Japanese. Received July 14, 2004

* * * * * * * * * * * * * * *

Masahiro TABUCHI received his B.E., M.E. Takashi OSHIBA received his B.S. and M.S. and Ph.D. degrees in computer science from degrees in computer science from Tsukuba Waseda University, Japan, in 1987, 1989, and University, Japan, in 1998 and 2000, respec- 1993, respectively. He joined NEC Corporation tively. He joined NEC Corporation in 2000 and in 1993 and is now a principal researcher in is now a researcher in the Software Architec- the Software Architecture Technology Group, ture Technology Group, Internet Systems Re- Internet Systems Research Laboratories. He leads a research search Laboratories. His research interests include ubiquitous team in the fields of ubiquitous communication and collabora- communication and collaboration technology, Internet infor- tion technology and Internet information service middleware. mation service middleware, personalization methodology, and Dr. Tabuchi is a member of the Information Processing electronic commerce. Society of Japan. Mr. Oshiba is a member of the Information Processing Society of Japan.

Kazuaki NAKAJIMA received his B.E. and M.E. degrees in computer science from Tokyo Shujun YOSHIZAKA received his B.A. degrees University of Agriculture and Technology, Ja- in design from Tama Art University, Japan in pan in 1993 and 1995. He received a Dr. Eng in 1993. He joined NEC Corporation in 1993, and 1998. He is currently an assistant manager at is now a researcher of the Software Architec- Internet Systems Research Laboratories, NEC ture Technology Group, Internet Systems Re- Corporation. His research interests include ubiquitous commu- search Laboratories. His research interests nication, groupware and collaboration technology. User Interface Design for ubiquitous communication and col- Dr. Nakajima is a member of the IPSJ. laboration.

Akiko KAWAMOTO received her B.E. degrees in computer science from Tokyo University of Agriculture and Technology, Japan in 1998. She joined NEC Corporation in 1998, and is now a researcher of the Software Architecture Technology Group, Internet Systems Research Laboratories. Her research interests include ubiquitous com- munication and collaboration technology, Internet information service middleware, and Terminal adaptation methodology.

* * * * * * * * * * * * * * *

NEC Journal of Advanced Technology, Vol. 1, No. 3 183