ABSTRACT With the distributed and rapidly increasing volume of data and expeditious de- velopment of modern web browsers, web browsers have become a possible legitimate vehicle for remote interactive multimedia presentation and collaboration, especially for geographically dispersed teams. To our knowledge, although there are a large num- ber of applications developed for these purposes, there are some drawbacks in prior work including the lack of interactive controls of presentation flows, general-purpose collaboration support on multimedia, and efficient and precise replay of presentations. To fill the research gaps in prior work, in this dissertation, we propose a web-based multimedia collaborative presentation document system, which models a presentation as media resources together with a stream of media events, attached to associated media objects. It represents presentation flows and collaboration actions in events, implements temporal and spatial scheduling on multimedia objects, and supports real-time interactive control of the predefined schedules. As all events are repre- sented by simple messages with an object-prioritized approach, our platform can also support fine-grained precise replay of presentations. Hundreds of kilobytes could be enough to store the events in a collaborative presentation session for accurate replays, compared with hundreds of megabytes in screen recording tools with a pixel-based replay mechanism. A WEB-BASED COLLABORATIVE MULTIMEDIA PRESENTATION DOCUMENT SYSTEM by Chunxu Tang B.S., Xiamen University, 2013 M.S., Syracuse University, 2015 Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical and Computer Engineering. Syracuse University June 2019 Copyright c Chunxu Tang 2019 All Rights Reserved To my parents. iv ACKNOWLEDGMENTS Owning our story and loving ourselves through that process is the bravest thing that we will ever do. | Bren´eBrown, The Gifts of Imperfection The work presented in this thesis could not have been created without the encour- agement and guidance provided by many others. I would like to acknowledge those who have helped me throughout this effort. First and foremost, I would like to thank my advisor, Dr. C.Y. Roger Chen, who has been my mentor in my Ph.D. life. I am really grateful that he decided to accept a random master student to his group, who just walked into his office to ask to work with him on a sunny day. He has not only taught me how to accomplish excellent research work, but also the perseverance and courage in overcoming difficulties. Without his help, I could not even start my research, nonetheless struggling through the doctoral program. There are many other faculty members helped me a lot in the past few years. Dr. Fawcett introduced me to the world of programming through his hard coursework and showed me his passion for cultivating young minds. Dr. Yu, who I have worked with in several courses, provided lots of advice on teaching and research. I would like v to thank the rest of my dissertation committee: Dr. Sponsler, Dr. Qiu, and Dr. Du for their insightful advice on my research work. My labmates and many other friends also helped me a lot to make the long Ph.D. life more enjoyable. In particular, Beinan, Maria, Elie, and Eyoel made the cold and snowy Syracuse winters easier to bear. Ao, Yiou, Zhiruo, Qinyun, Zhi, Zilong, and Zhiyuan encouraged me in the most difficult and depressing part of the journey. Last but not least, I would like to acknowledge my mother Fengyun Liu and my father Jun Tang for all of their support and encouragement in the past four years through my doctoral life. They told me the meanings of human lives and the essential courage to purse for the meanings. Without their patience and help, I could not make it through. This is dedicated to them. vi TABLE OF CONTENTS Page ABSTRACT ::::::::::::::::::::::::::::::::::: i LIST OF TABLES :::::::::::::::::::::::::::::::: xi LIST OF FIGURES ::::::::::::::::::::::::::::::: xii 1 Introduction :::::::::::::::::::::::::::::::::: 1 1.1 Multimedia Systems :::::::::::::::::::::::::: 1 1.2 Real-Time Collaboration :::::::::::::::::::::::: 3 1.3 Design Features and Contributions :::::::::::::::::: 5 1.4 Organization of the Dissertation :::::::::::::::::::: 8 2 Survey of Technologies :::::::::::::::::::::::::::: 11 2.1 Rendering of Web Pages :::::::::::::::::::::::: 11 2.2 Event Loop in JavaScript :::::::::::::::::::::::: 13 2.3 Chrome Extensions ::::::::::::::::::::::::::: 15 3 Interactive Multimedia Presentation Document System :::::::::: 17 3.1 Model Structure ::::::::::::::::::::::::::::: 17 3.2 Object-Oriented Design ::::::::::::::::::::::::: 18 3.2.1 Introduction ::::::::::::::::::::::::::: 18 3.2.2 Document Object :::::::::::::::::::::::: 18 3.2.3 Document Event :::::::::::::::::::::::: 21 3.2.4 Document Panel :::::::::::::::::::::::: 21 3.2.5 Document Annotation ::::::::::::::::::::: 22 3.2.6 Document Presentation ::::::::::::::::::::: 25 3.3 Temporal Modeling ::::::::::::::::::::::::::: 27 3.3.1 Introduction ::::::::::::::::::::::::::: 27 3.3.2 Timeline ::::::::::::::::::::::::::::: 29 vii Page 3.3.3 Sequence Diagram ::::::::::::::::::::::: 30 3.3.4 Activity Diagram :::::::::::::::::::::::: 31 3.3.5 Petri Net :::::::::::::::::::::::::::: 33 3.3.6 Timed Petri Net :::::::::::::::::::::::: 37 3.3.7 Document Net :::::::::::::::::::::::::: 39 3.3.8 DocEvent Structure ::::::::::::::::::::::: 47 3.3.9 Temporal Constraints :::::::::::::::::::::: 50 3.4 Spatial Modeling :::::::::::::::::::::::::::: 51 3.4.1 Introduction ::::::::::::::::::::::::::: 51 3.4.2 Slicing Tree ::::::::::::::::::::::::::: 52 3.4.3 Modeling Updates ::::::::::::::::::::::: 54 3.5 Implementation ::::::::::::::::::::::::::::: 55 3.5.1 Overview :::::::::::::::::::::::::::: 55 3.5.2 Presentation Preparation :::::::::::::::::::: 55 3.5.3 Presentation Control :::::::::::::::::::::: 56 3.5.4 Presentation Replay :::::::::::::::::::::: 58 3.6 Related Work :::::::::::::::::::::::::::::: 59 3.7 Conclusion :::::::::::::::::::::::::::::::: 61 4 Continuous Updates of External Web Resources :::::::::::::: 63 4.1 Introduction ::::::::::::::::::::::::::::::: 63 4.2 Challenges :::::::::::::::::::::::::::::::: 65 4.3 Continuous Updates with a Server ::::::::::::::::::: 65 4.3.1 Implementation ::::::::::::::::::::::::: 65 4.3.2 Change Monitoring ::::::::::::::::::::::: 67 4.3.3 Communication ::::::::::::::::::::::::: 68 4.3.4 Summary :::::::::::::::::::::::::::: 69 4.4 Continuous Updates without a Server ::::::::::::::::: 70 4.4.1 Implementation ::::::::::::::::::::::::: 70 4.4.2 Change Monitoring ::::::::::::::::::::::: 71 viii Page 4.4.3 Communication ::::::::::::::::::::::::: 73 4.4.4 Summary :::::::::::::::::::::::::::: 73 4.5 External Web Resources Loading ::::::::::::::::::: 73 4.5.1 Introduction ::::::::::::::::::::::::::: 73 4.5.2 Web Elements with Styles ::::::::::::::::::: 74 4.5.3 Screenshot :::::::::::::::::::::::::::: 74 4.5.4 Summary :::::::::::::::::::::::::::: 76 4.6 Related Work :::::::::::::::::::::::::::::: 76 4.7 Conclusion :::::::::::::::::::::::::::::::: 78 5 Distributed Context-Aware Collaboration Framework ::::::::::: 79 5.1 Challenges :::::::::::::::::::::::::::::::: 79 5.2 Framework Methodology :::::::::::::::::::::::: 82 5.2.1 Communication Models :::::::::::::::::::: 82 5.2.2 Stateless Events ::::::::::::::::::::::::: 87 5.2.3 Scalable Cloud Service ::::::::::::::::::::: 88 5.2.4 Uniform Interfaces ::::::::::::::::::::::: 90 5.2.5 Object-Prioritized Collaboration :::::::::::::::: 91 5.2.6 Non-Intrusive Collaboration :::::::::::::::::: 93 5.2.7 Access Control ::::::::::::::::::::::::: 94 5.3 Architecture Components ::::::::::::::::::::::: 99 5.3.1 Collaboration Subject ::::::::::::::::::::: 99 5.3.2 Media Event Capturer ::::::::::::::::::::: 100 5.3.3 Media State Recorder :::::::::::::::::::::: 104 5.3.4 Media Event Replayer ::::::::::::::::::::: 104 5.3.5 Event Messages ::::::::::::::::::::::::: 107 5.3.6 Message Serializer/Deserializer ::::::::::::::::: 112 5.4 Evaluations ::::::::::::::::::::::::::::::: 114 5.4.1 Web Application :::::::::::::::::::::::: 114 5.4.2 Comparison with Screen Sharing Tools :::::::::::: 117 ix Page 5.5 Related Work :::::::::::::::::::::::::::::: 122 5.6 Conclusion :::::::::::::::::::::::::::::::: 125 6 Collaborative Web Browsing ::::::::::::::::::::::::: 127 6.1 Introduction ::::::::::::::::::::::::::::::: 127 6.2 Challenges :::::::::::::::::::::::::::::::: 128 6.3 Synchronization of Web Browsing Actions :::::::::::::: 131 6.4 Related Work :::::::::::::::::::::::::::::: 135 6.5 Conclusion :::::::::::::::::::::::::::::::: 137 7 Conclusion and Future Work ::::::::::::::::::::::::: 138 LIST OF REFERENCES :::::::::::::::::::::::::::: 140 VITA ::::::::::::::::::::::::::::::::::::::: 152 x LIST OF TABLES Table Page 5.1 Comparison of supports for multimedia collaboration of four studies with our work. :::::::::::::::::::::::::::::::::: 82 5.2 Compatibilities of XMLHttpRequest, Server-sent events, and WebSocket on major web browsers. 1 ::::::::::::::::::::::::: 86 5.3 Access control matrix. ::::::::::::::::::::::::::: 95 5.4 Access control matrix representation of our web-based collaborative document system. :::::::::::::::::::::::::::::: 97 5.5 Summary of media events and related represented DOM events. :::: 103 5.6 Summary of elements in a media event message. ::::::::::::: 109 5.7 Summary of elements in a control event message. :::::::::::: 112 5.8 Total bytes transmitted in one minute of four types of media, under the four situations. :::::::::::::::::::::::::::::::