Vmoox cross-platform video publishing solution for mobile devices A Thesis Presented to the Faculty of the Department of Computer Science Pace University In Partial Fulfillment of the Requirements for the Degree of Master of Science in Software Development and Engineering by Lior D. Shefer October, 2009 Under the Direction of Professor Francis Marchese - 2 - Table Of Content Table Of Content 3 List Of Figures 6 I. Introduction 7 1. No Longer Science Fiction – Ubiquitous Computing And The Smart 8 Phone Explosion. 2. All Hail The King - The Rise Of The iPhone 9 3. Watch It Online – Video On The Internet 9 4. It's Not Me, It's You – The Flash Problem 10 II. H.264/AVC Smartphone Support 12 III. FFmpeg/libavcodec 14 IV. Software as a Service (SaaS) 15 V. XStream 16 VI. Vmoox Requirements 17 1. Publisher Service Interface 17 a. Account Setup 17 b. Content Input 17 c. Pre-Roll Content Input 17 d. Content Update 18 e. Thumbnail Creation 18 f. Customize Smart Phone Web or Native Application 18 2. Transcoder 19 a. Initial Transcode 19 b. Requested Transcode 19 3. Video Encoding Decision Algorithm (VEDA) 19 a. Fully Transcoded Videos 20 b. Partially Transcoded Videos 20 4. Database 20 a. Original Content Files 20 b. Meta-Data (including thumbnails) 20 c. Fully Transcoded Video Files 20 d. Partially Transcoded Video Files 20 e. Pre-Roll Content 20 - 3 - 5. Web Service/ API 21 a. Featured Video List 21 b. Video Playlist 21 c. Search List 21 6. Ad Engine 21 7. Front End Application (native and/or web based) 21 a. Featured View 22 b. Video View 22 c. Play View 22 8. Loosely Coupled Modular Architecture 22 VII. Vmoox Design 23 1. Vmoox Design Overview 23 2. VPSI - Design 26 3. Transcoder - Design 28 a. Initial Transcode 28 b. Requested Transcode 28 4. Video Encoding Decision Algorithm (VEDA) 29 5. Vmoox Database - Design 31 6. Vmoox Web Service / API - Design 33 7. Ad Engine - Design 35 8. Front End Application - Design 36 VIII. Vmoox Implementation 39 1. Vmoox Implementation Overview 39 2. VPSI - Implementaion 39 a. The Model Objects 39 b. Publisher Service 42 c. Downloader 43 3. Transcoder - Implementation 45 a. Transcoding Using FFmpeg 46 4. VEDA - Implementation 49 5. Database - Implementation 51 a. Video 51 - 4 - b. H264 51 c. Ad 52 6. Vmoox Web Service 55 a. Implementing Get Network Videos Service 56 b. Implementing Get List of Videos Media URLs for a Given Video 59 Id c. Ad Serving 59 7. iPhone Native and Web App 60 a. Web Based App 60 i. The Model 60 ii. The Controller 61 iii. The View 62 IX. Summary and Future Work 69 X. References 70 - 5 - List of Figures Figure 1: Pre-iPhone: BellSouth/IBM Simon Personal Communicator 7 (1993) Figure 2: H.264 Wide Range of Implementations. 11 Figure 3: Top US Smartphones Web Traffic. 12 Figure 4: Supported Formats of Popular Smart Phones in the US 12 Figure 5: Typical Publisher Implementation. 24 Figure 6: A Sample XML Submitted by Client Including Content Meta- 22 Data Figure 7: !!!!!!Transcoder - Initial Transcoding Overview 27 Figure 8: Transcoder - Real Time Transcoding Overview 28 Figure 9: Services VEDA Provides to VPSI 29 Figure 10: VEDA Chapter Name Creation 29 Figure 11: Services VEDA Provides to Vmoox Web Service 30 Figure 12: Services Vmoox Web Service Provides to the iPhone App 33 Figure 13: Vmoox Web Service JSON Formatted Response to a Get 34 Videos’ Media URLs Request, With an “ads” Array Figure 14: MVC Design Pattern 35 Figure 15: Vmoox Controller 36 Figure 16: Video Object UML 39 Figure 17: Video Chapter Object UML 40 Figure 18: Ad Object UML 40 Figure 19: VPSI UML 41 Figure 20: Transformation Class Services 44 Figure 21: Vmoox DB(s) Overview 52 Figure 22: Featured Videos View 62 Figure 23: Video View 65 Figure 24: A Video is Being Played by the iPhone Native Media Player 67 - 6 - I. Introduction This thesis introduces Vmoox [1] - a cross-platform video publishing solution for mobile devices. It will detail the need, timeliness and profit potential of a real-time video transcoding solution for mobile devices as well as the suggested implementation. While this model is intended for use in all smart phones, the current prototype implementation is for the iPhone. This implementation can be modified to fit other and future smart phones. 1. No Longer Science Fiction – Ubiquitous Computing And The Smart Phone Explosion The term Ubiquitous Computing has been growing in popularity in recent years, as has the phenomena it attempts to describe. Simply put, this means that computers are everywhere. More than ever before, computers have ceased to stand alone as “computing devices” but rather, have been integrated into everyday items to help them become “smarter”. We now use computers in our offices, homes, stores, cars, banks, hospitals, kitchens and gyms. From washing machines to coffee makers, to car alarms to defibrillators; like it or not, it is hard to deny that computers have made their way into nearly every aspect and activity of our modern lives. Similarly, the Internet, which 30 [2] years ago seemed to most of us the stuff of science fiction, has become an inseparable part of our lives and culture. As a society, especially in the United States, we have come to expect and rely a great deal on both computers and the Internet. Perhaps the greatest manifestation of this phenomenon is the smart phone explosion. IBM introduced the first smart phone in 1993 [3]. Its then revolutionary features included games, a pager, send and receive fax capabilities, a calculator and a notepad. Costing around $900 [4], this was a gadget that was far from ubiquitous. But it was the beginning of a trend that would change the way we access and use information. - 7 - Figure 1: Pre-iPhone: BellSouth/IBM Simon Personal Communicator (1993) In 2001, RIM introduced the first Blackberry - a device optimized for wireless email use. By June 2007 over 8 million people were hooked. In the 8 years since Blackberries came on to the scene, we have witnessed incredible advances in both the number and functions of smart phones. Today, the vast majority of cellular phones are not just phones. A recent survey showed that around the world, one out of ten phones is a smart phone [5], a number that is expected to double by 2013 [6]. Smart phones have changed the way we access and use information in a drastic way. They have become all-in-one devices that we rely on for a great deal of information, accessible at all times. Email servers, clocks and calendars, maps and navigation software, photo and movie cameras, video games, music players and calculators are just a few of the features that now come standard in every smart phone. Many of these features rely on the Internet to function. It’s the Internet that tells the calendar feature what day and time it is if, lets say, your phone’s battery died last night at 3 am. When your Blackberry buzzes with a new email, it’s because your phone connected to the Internet and retrieved that email from your email server. Until recently, smart phones relied on the Internet for purely ‘behind the scenes’ assistance. Until the iPhone changed everything. 2. All Hail The King – The Rise Of The iPhone Within the smart phone revolution, one major player is widely acknowledged as a game changer – the iPhone. While the Blackberry allowed users to access email on their cell phone, the iPhone brought the Internet in all its glory into the palm of our hand. In no small part thanks to Apple’s signature user-friendly design and aggressive marketing, the iPhone turned the smart phone from a tech-y solution for workaholics, into the must-have gadget for everyone. - 8 - But behind what is, depending on your point of view, either brilliant design or overrated hype was the true technological revolution of the iPhone – the Internet was no longer a background player. It was front and center, just as we were used to seeing it on our desktop computers. On the iPhone, our maps look like the maps we are used to using when looking up directions to our favorite restaurant; we can watch our favorite you Tube videos, check the weather and Google just like we’re used to. The iPhone was no longer a phone with extra capabilities - it was the Internet, in the palm of your hand. Besides being a miniature version of our desktop computers (with a phone and music player built in) the iPhone changed our technological landscape by copying the Internet not only in form but also in concept, it was a device specifically designed for growth and the sharing of information. The iPhone’s true genius lies in its 'apps'. With over 25,000 different downloadable applications and more than 800 million downloads [7], the AppStore allows the community of users to continuously devise and implement new functions for the iPhone. But for all its brilliance, the iPhone (along with its smart-phone counterparts) has hit a major stumbling block as it intersects with the other major Internet revolution of recent years – video content. 3. Watch It Online – Video On The Internet In recent years, online video content has skyrocketed. As of November of 2008, Americans spent 12.5 percent of their time online viewing videos [8], watching over 14 billion clips a month [9]. You Tube’s ‘most watched’ clip has more than 123 million views [10] compared to 2008’s record setting Super Bowl which managed a ‘modest’ audience of 97.5 million [11].