HABILITATION A` DIRIGER DES RECHERCHES DE L'UNIVERSITE´ PIERRE ET MARIE CURIE HABILITATION A` DIRIGER DES RECHERCHES Sp´ecialit´e: Sciences pour l'ing´enieur pr´esent´eepar Cyril CONCOLATO Interactive Media Streaming Soutenue le 25/10/2016 devant le jury compos´ede M. Vincent CHARVILLAT Professeur, Universit´ede Toulouse Rapporteur M. Dick BULTERMAN Professeur, Vrije Universiteit Amsterdam Rapporteur M. Carsten GRIWODZ Professeur, Universitetet i Oslo Rapporteur M. Pierre SENS Professeur, Universit´eParis 6 Examinateur M. Gwendal SIMON Ma^ıtrede Conf´erences,HDR, Telecom Bretagne Examinateur M. Christian TIMMERER Associate Professor, Universit¨atKlagenfurt Examinateur M. Jean-Claude DUFOURD Directeur d'´etudes,Telecom ParisTech Examinateur ii Contents 1 Foreword 1 2 Introduction 5 3 From Scene Descriptions to Web Applications 7 3.1 Scene Description Activities . 8 3.1.1 MPEG-4 BIFS . 8 3.1.2 Mixing Scene Description formats . 8 3.1.3 Extending the SVG format . 9 3.2 Interactive Media Applications . 11 3.2.1 Synchronized and Interactive Streams . 11 3.2.2 Personalized Interactive Application . 13 3.3 Communicating Web Applications . 15 3.3.1 Widgets . 15 3.3.2 Multi-screen Web Applications . 17 3.4 Conclusion . 17 4 Multimedia Application Adaptation 19 4.1 Adaptive Images . 20 4.2 Multimedia Documents Adaptation . 21 4.3 Broadcast-Friendly Adaptation using Scalable Multimedia Doc- uments . 23 4.4 Conclusion . 26 5 HTTP Adaptive Streaming 27 5.1 Standardization activities . 28 5.2 Adaptive Streaming of Text Content . 28 5.3 HTTP Streaming Latency . 30 5.4 Adaptive Streaming of Tiled Videos . 33 5.5 Conclusion . 34 iii 6 Multimedia Application Delivery 35 6.1 Document Streaming . 36 6.2 Electronic Program Guide Streaming . 36 6.3 Application broadcasting . 38 6.4 Application delivery over MPEG-DASH . 38 6.5 Application delivery over ISOBMFF . 39 6.6 Hybrid Multimedia Delivery . 39 6.7 Application delivery over Hybrid networks . 40 6.8 Conclusion . 41 7 Conclusion and perspectives 43 Bibliography 47 iv Chapter 1 Foreword At the time of writing of this report, I have been carrying research activities for the past 15 years, mostly at Telecom ParisTech, formerly Ecole Nationale Sup´erieurede T´el´ecommunications, in Paris, France. Telecom ParisTech is a public institution and as such dissemination is a very important part of the research. Dissemination often takes the form of published papers. I have published so far 60 papers in journals or con- ference proceedings. I have also applied for more than 18 patents. A list of my published papers available online in my publications page1 and many are listed in the bibliography chapter of this report. Figure 1.1 presents a graphical overview of some of my publications, since my PhD, between 2008 and 2015, along a timeline, highlighting the topics I have covered and the links between publications. These topics will be covered in this report. The impact of the dissemination of a researcher is often measured with paper citations. According to my Google Scholar's profile2, I've been cited 423 times. This does not unfortunately measure the impact of my contribu- tions to open-source software, which is a particularity of my research activity and that I consider a complementary and important dissemination activity. Most, if not all, of my research leads to publication of open-source software, in particular within GPAC3, a multimedia platform developed at Telecom ParisTech. My GitHub profile page4 indicates a total of 568 contributions to open-source software in the last year. My OpenHub profile page5 indicates 1Full list of publications https://concolato.wp.mines-telecom.fr/publications/ 2Google Profile page https://scholar.google.fr/citations?user= ySXws0oAAAAJ&hl=fr&oi=ao 3Website of the GPAC project http://gpac.io 4Personal GitHub page https://github.com/cconcolato 5Personal OpenHub page see https://www.openhub.net/p/gpac/contributors/2598440182341747 1 Web & & Web Widget & & Widget Streaming MultiScreen Video Streaming MMSP MMSP 2015 Web 3D 2015 internet delivery internet Multimedia 2015 Multimedia XML Prague XML 2015 Latency and Hybrid Hybrid and Latency Streaming Services Graphical Web 2015 mediaand easier for MPEG-DASH for Low MPEG-DASH Fast DASH Bootstrap Fast Packaging web graphics Packaging Synchronized Delivery of Synchronized Scene 3D Scenes with Audio and 3D Scenes & Hybrid & In-Browser XML Document In-Browser XML Broadcast Broadcast Adaptation Adaptation Adaptation & Streaming & DocEng 2015 Oriented Web Page Oriented Web Page MSoS: A Multi-Screen- MSoS: A Segmentation Approach Segmentation Scene Scene Open Open Source Description DASH IISA 2014 IISA of Low LiveLatency of Browser Streaming using MPEG-Streaming using DocEng 2014 DocEng Overhead and Performance Overhead Performance and MMSys 2013 MMSys Refactoring Web Refactoring Web The Virtual Splitter: Virtual The Applications for the for Applications Multiscreen Environment Live HTTP Streaming of HTTP Streaming of Live Legend Video and Subtitles Video withinand a Subtitles on the onWeb the MMSP MMSP 2013 SIGGRAPH SIGGRAPH 2013 curve imagescurve from boundary elements Biharmonic diffusion diffusion Biharmonic Graphical Graphical Web 2013 Streaming Server Streaming SVG animations Streaming SVG animations DashCast, A DASH DashCast, Live MMSys 2013 MMSys Distributed DASH Dataset Distributed The MPEG The MPEG Digital Media Springer 2012 Springer Representation ofRepresentation and playback MediaSync 2013 MediaSync using MPEG DASH MPEG using MMSys 2012 Hybrid Broadcast Services Broadcast Hybrid broadband networks broadband GPAC, toolbox for GPAC,interactive for toolbox SIG Multimedia SIG 2012 Records multimedia packaging, delivery packaging, delivery multimedia multimedia content over content over multimedia Synchronized delivery of of Synchronized delivery uncoordinated broadcast uncoordinated broadcast Figure 1.1: Timeline of my academic publications MMSys 2011 Applications EuroiTV 2011 Recording and Recording and Usages of DASH for DASH Usagesfor of Rich Services Rich Media Delivery of HbbTVDelivery of multimedia documents MTAP 2011 Communicating and and Communicating migratable interactive migratable Multimedia 2011 Multimodal Recordings Dance from Performance Enhanced Visualisation of VisualisationEnhanced of Automatically Synchronised Automatically SAPMIA 2010 Adaptive Video and Video and Adaptive Multimedia Documents Metadata Display using Display using Metadata the Distance the using GPAC using Movid 2010 MELECON 2010 MELECON Multimedia 2011 Multimedia MTAP 2010 Mobile Devices and Devices Mobile and Experimenting with Experimenting SVG Electronic SVG Gestures to aControl Program Guides SVG Open 2010 Open SVG Multimedia Advances Multimedia A of Analysis Formative Annotation based Annotation based mobile applications mobile A Model for the for Model Delivery A Multimedia Application from Multimedia Application over Broadcast Channels Broadcast over of Interactive Applications Interactive Applications of presentation of videos for of presentation personalized adaptation personalizedand MMEDIA MMEDIA 2010 graphics graphics formats primitive for vector vector for primitive SuperPath: a SuperPath: necessary MTAP 2009 MTAS 2009 MTAS Widgets Widgets Mobility Widgets Authoring Authoring of Scalable Multimedia Documents Multimedia 2 SVG 2009 Open DocEng 2009 Experiences SVG Communicating Communications Documents UCMEDIA 2009 UCMEDIA WIAMIS 2009 Mobile Multimedia Mobile 2009 MM for Dynamic Widgets Dynamicfor Declarative Interfaces Declarative Interfaces mobile applications mobile and presentation of and of presentation Summarization of Summarization annotated videos for videos annotated EuroiTV 2009 EuroiTV Scalable Multimedia Scalable Multimedia Conversion of Images to Conversion of Natural-Language-based Natural-Language-based Declarative EPG Declarative Personalized adaptation Personalized adaptation and Presentation of Presentation and Multimedia Multimedia Documents Generation, Streaming Generation, DocEng 2008 Playback of Mixed Playback Radio ICCE 2009 ICCE DocEng 2008 An MPEG-based An MPEG-based Scalable Multimedia Scalable Multimedia Documents for Digital Documents and devices mobile Widget System forWidget System CE DocEng 2008 Trans. CE 2008 Trans. Vector Scalable Graphics Player for Player Graphics Constrained Devices Design of an Efficient of an Design Adaptation of Scalable of Adaptation Multimedia Documents ICME 2008 TVC/CGI 2008 TVC/CGI in Broadcast Home Networks Home Environments Multimedia Access in AccessMultimedia User-Centric Universal Spatial Scene Adaptation 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 TPT 2007 TPT optimisations Descriptions de de Descriptions représentations et représentations scènes multimédia : scènes a total of about 700 contributions to GPAC since the beginning of these statistics. A confirmation that my open source activity is important for my research is that my most cited paper is the first paper presenting GPAC at the ACM Multimedia Conference [44]. When analyzing someone's research record, it is also interesting to see who he/she publishes with. I've published my academic papers with 81 different persons from 20 different institutions, but mostly research centers and few universities. This highlights an important aspect of my research activities, that most of them have been carried through collaborations with companies, rather than with other universities. This is the consequence I believe of two characteristics. First, most of my research has been funded either by the French research