REAL TIME MULTIPLE CODECS SWITCHING ARCHITECTURE FOR VIDEO CONFERENCING By USMAN SARWAR Thesis submitted in fulfilment of the requirements for the degree of Master of Science MAY 2008 REAL TIME MULTIPLE CODECS SWITCHING ARCHITECTURE FOR VIDEO CONFERENCING By USMAN SARWAR Thesis submitted in fulfilment of the requirements for the degree of Master of Science MAY 2008 ACKNOWLEDGEMENTS بســـــــــــــــم ﷲ الرحمن الرحيـــــــــــــــم "اقرأ باسم ربك الذي خلق" I would like to take this opportunity to convey my sincere thanks and deepest gratitude to my supervisor: Assoc. Prof. Dr. Sureswaran Ramadass, and to my co- supervisor Dr.Rahmat Budiarto, for all the help and valuable guidance provided to me during the preparation of this thesis. I consider myself privileged to have had the opportunity to work under their guidance. Moreover, I would like to convey my appreciation to the MCS ver. 6 core team, all Network Research Group members, School of Computer Sciences, Institute of Postgraduate Studies, the university library and the Mlabs staff for their help and support. Finally and most important of all, my sincere gratitude also goes to my parents; Dr.M. Sarwar, Riffat Bano, Song Jing Sheng and Dong Feng, my wife Shen, brother (Imran), sisters (Ainee and Ayesha), uncle (Dr.Tahir ), aunt (Fatima) and friends (Ali, Arshad and many more). I thank them for their support, understanding, and encouragement during every step of my study and my writing of this thesis. ii TABLE OF CONTENTS Page ACKNOWLEDGEMENTS ii TABLE OF CONTENTS iii LIST OF TABLES vi LIST OF FIGURES vii LIST OF ABBREVIATION ix LIST OF PUBLICATIONS & SEMINARS ix ABSTRAK x ABSTRACT xi CHAPTER ONE : INTRODUCTION 1.0 Introduction 1 1.2 Problem statement 1 1.3 Objectives of this thesis 2 1.4 Background 3 1.4.1 Multimedia real time interactive services 3 1.4.2 Digitizing video 4 1.5 Codec 5 1.5.1 Hardware codec 6 1.5.2 Software codec 6 1.6 Video Compression 6 1.6.1 Lossless compression 7 1.6.2 Lossy compression 7 1.7 Digital video paradigm 8 1.8 Contribution 8 1.9 Organization of the thesis 9 iii CHAPTER TWO : LITERATURE REVIEW 2.1 Introduction 11 2.2 Multimedia Communications 11 2.3 Video Conferencing 12 2.4 Existing video framework for video conferencing 13 2.4.1 ITU-T H.323 Video Conferencing Suite 13 2.4.2 Session Initiation Protocol (SIP) 17 2.4.3 vic - Video Conferencing Tool 19 2.4.4 INRIA Videoconferencing System 21 2.4.5 Vaudeville 22 2.4.6 ALX video conference 23 2.4.7 Seodang 24 2.4.8 GCSVA 26 2.4.9 The HKUST video conferencing system 27 2.4.10 OCTOPUS 28 2.4.11 DIGIPARTY - a decentralized multi-party video 30 conferencing system 2.4 Summary 30 CHAPTER THREE : RMCS ARCHITECTURE 3.1 Introduction 34 3.2 Video coding standardization 34 3.2.1 Evolution of codecs 35 3.3 The proposed real time multiple codecs switching architecture 36 3.3.1 RMCS Architecture 37 3.3.2 Integration of RMCS codec extensibility in video 42 components 3.3.3 RMCS Protocol 46 3.3.4 Simultaneous multiple codec support 48 3.3.5 Conference initiator video codec preference 54 3.3.6 Real time multiple codec change without handshaking 55 3.3.6 Codec capabilities 59 3.4 Evaluation method 59 3.5 Summery 59 iv CHAPTER FOUR : IMPLEMENTATION 4.1 Introduction 60 4.2 Implementation details 60 4.2.1 Hardware platform 60 4.2.2 Operating system 60 4.2.3 Programming languages 61 4.3 Mlabs Multimedia Conferencing System 64 4.4 RMCS architecture program organization 65 4.4.1 Video Capture component organization 66 4.4.2 Video network source filter component organization 68 4.4.3 Video Playback component organization 70 4.5 RMCS architecture integration 72 4.6 Summary 73 CHAPTER FIVE : TEST RESULTS AND DISCUSSION 5.1 Introduction 74 5.2 Test setup 74 5.3 Installation of MPEG-4 codec 75 5.4 RMCS architecture testing 75 5.5 Existing systems 82 5.5.1 H.323 82 5.5.1 SIP 87 5.6 Summery 89 CHAPTER SIX : CONCLUSION 6.1 Introduction 91 6.2 RMCS Architecture 91 6.3 Future Work 92 6.4 Summary 93 REFERENCES 94 v LIST OF TABLES Page Table 2.1 Summary of different video conferencing systems 33 Table 3.1 RMCS Media Tag description 43 Table 4.1 RMCS architecture minimum hardware requirements 61 Table 4.2 List of codecs 67 Table 5.1 Test plan 74 Table 5.2 System specifications for testing 75 Table 5.3 Bandwidth and system resource summary 78 Table 5.4 Test description for real time multiple codec change 80 Table 5.5 Participant 2 bandwidth consumption with different codecs 81 configurations Table 5.6 Summary results of MCS client 82 Table 5.7 Summary results of H.323 86 Table 5.8 Summary results of SIP client 88 Table 5.9 Summary of test results 90 vi LIST OF FIGURES Page Figure 2.1 H.323 Stack 13 Figure 2.2 Capability table 15 Figure 2.3 Capability Descriptor 15 Figure 2.4 SIP Architecture 18 Figure 2.5 VIC 20 Figure 2.6 GCSVA Architecture 26 Figure 2.7 Structure of OCTOPUS 29 Figure 2.8 Contribution of RMCS architecture 32 Figure 3.1 Evolution of the standard codecs over time 36 Figure 3.2 RMCS Architecture 38 Figure 3.3 Video sender module 40 Figure 3.4 Video receiver module 41 Figure 3.5 Communication layer 41 Figure 3.6 Structural template of RMCS repository 43 Figure 3.7 RMCS architecture acquiring video codec 45 Figure 3.8 Customized RTP header 45 Figure 3.9 Customized RTP header structure 46 Figure 3.10 Inter-process communication 47 Figure 3.11 Remote-process communication 47 Figure 3.12 Messaging for version verification 48 Figure 3.13 Layer procedure 50 Figure 3.14 Each participant video sender and receiver modules 50 Figure 3.15 Video sender packetization procedure 51 Figure 3.16 Video receiver de-packetization procedure 52 Figure 3.17 Codec change and transmission procedure 53 Figure 3.18 Different systems with multiple video codecs 54 Figure 3.19 Messaging between chairman and the participants 55 Figure 3.20 Runtime codec change procedure 57 Figure 3.21 Codec independent session 58 Figure 4.1 Microsoft Directshow architecture 63 Figure 4.2 Different processes of video capture and playback 65 vii Figure 4.3 Interface for accessibility of video capture functionality 66 Figure 4.4 Interface for accessing video network source filter 69 functionality Figure 4.5 Interface for video playback 70 Figure 5.1 Multimedia Conferencing System version 6 client 75 Figure 5.2 Media repository configuration 76 Figure 5.3 MCS client options 77 Figure 5.4 Test setup for MCS using MCS server 77 Figure 5.5 Simultaneous multiple codec support 78 Figure 5.6 Simultaneous multiple codec support bandwidth utilization 79 Figure 5.7 Video resolution dialog box in MCS 80 Figure 5.8 Participant 2 bandwidth utilization with different codecs 81 Figure 5.9 Real time codec change by participant 2 to VDO Wavelet 82 Figure 5.10 Test setup for H.323 client using MCU 83 Figure 5.11 Polycom PVX 83 Figure 5.12 Microsoft Netmeeting 84 Figure 5.13 Participants in H.323 conference 85 Figure 5.14 Participants in H.323 test-2 86 Figure 5.15 Test setup for SIP client using SIP server 87 Figure 5.16 X-Lite SIP client 87 Figure 5.17 Participants in SIP 89 viii LIST OF ABBREVIATION UDP User Datagram Protocol MCS Multimedia Conferencing System SIP Session Initiation Protocol MTU Maximum Transmission Unit TCP/IP Transmission Control Protocol/Internet Protocol LAN Local Area Network RMCS Real Time Multiple Codec Switching LIST OF PUBLICATIONS & SEMINARS 1.1 Usman Sarwar, “Presentation on RMCS architecture” in Prince of Songkla University, Thailand. 3 May 2007. ix SENIBINA PENUKARAN PELBAGAI CODECS PADA MASA SEBENAR UNTUK PERSIDANGAN VIDEO ABSTRAK Trend terkini yang berhubung kait dengan khidmat video internet telah merubah dengan mendadak cara kita berfikir dan berkerja. Perkhidmatan-perkhidmatan ini telah memperluaskan ruang lingkup teknologi serta memberi peluang kepada perkembangan idea-idea penyiaran tanpa sempadan dan memendekkan jarak komunikasi secara geografi melalui telesidang video. Perkhidmatan-perkhidmatan ini harus dicapai dengan kaedah yang betul dan efisyen bagi mengelakkan ia menyekat penggunaan sumber terhad yang di kongsi bersama terutama yang berhubung kait dengan lebar jalur. Penyelidikan yang dilaporkan dalam tesis ini mencadangkan satu arkitektur bagi menyelesaikan isu-isu telesidang video yang berkaitan dangan penggunaan lebar jalur dan sumber sistem secara optima. Codec video yang berlainan mempunyai kesan yang berbeza terhadap lebar jalur serta kualiti gambar. Arkitektur ini membenarkan penggunaan pelbagai codec untuk sistem video telesidang secara serentak. Pendekatan ini membolehkan pengguna telesidang video menggunakan codec video yang berbeza bagi melaraskan lebar jalur tujuan pelarasan berdasarkan sumber sistem dan kepelbagaian jenis rangkaian. Ia juga membolehkan penyelarasan secara dinamik dan pertukaran codec video bagi membolehkan penggunaan optimum lebar jalur rangkaian semasa sesi telesidang berlangsung. Arkitektur cadangan juga menambah codec video yang piawai dan bukan piawai kepada sistem telesidang video asas melalui pendekatan mudah pasang. Arkitektur cadangan ini telah diintegrasi ke dalam sistem persidangan multimedia (MCS). Ia telah diuji dan di bandingkan dengan H.323 dan sistem berasaskan SIP. x REAL TIME MULTIPLE CODECS SWITCHING ARCHITECTURE FOR VIDEO CONFERENCING ABSTRACT Current trends related to internet video services have drastically transformed the way we think and work. These services have expanded the new scope of technology and have given opportunity for broadcasting ideas without any boundaries as well as shortened the vast geographical distances for communications through video conferencing.