Vmoox cross-platform video publishing solution for mobile devices
A Thesis Presented to the Faculty of
the Department of Computer Science
Pace University
In Partial Fulfillment
of the Requirements for the Degree of
Master of Science
in
Software Development and Engineering
by
Lior D. Shefer
October, 2009
Under the Direction of
Professor Francis Marchese
- 2 - Table Of Content Table Of Content 3 List Of Figures 6 I. Introduction 7 1. No Longer Science Fiction – Ubiquitous Computing And The Smart 8 Phone Explosion. 2. All Hail The King - The Rise Of The iPhone 9 3. Watch It Online – Video On The Internet 9 4. It's Not Me, It's You – The Flash Problem 10 II. H.264/AVC Smartphone Support 12 III. FFmpeg/libavcodec 14 IV. Software as a Service (SaaS) 15 V. XStream 16 VI. Vmoox Requirements 17 1. Publisher Service Interface 17 a. Account Setup 17 b. Content Input 17 c. Pre-Roll Content Input 17 d. Content Update 18 e. Thumbnail Creation 18 f. Customize Smart Phone Web or Native Application 18 2. Transcoder 19 a. Initial Transcode 19 b. Requested Transcode 19 3. Video Encoding Decision Algorithm (VEDA) 19 a. Fully Transcoded Videos 20 b. Partially Transcoded Videos 20 4. Database 20 a. Original Content Files 20 b. Meta-Data (including thumbnails) 20 c. Fully Transcoded Video Files 20 d. Partially Transcoded Video Files 20 e. Pre-Roll Content 20
- 3 - 5. Web Service/ API 21 a. Featured Video List 21 b. Video Playlist 21 c. Search List 21 6. Ad Engine 21 7. Front End Application (native and/or web based) 21 a. Featured View 22 b. Video View 22 c. Play View 22 8. Loosely Coupled Modular Architecture 22 VII. Vmoox Design 23 1. Vmoox Design Overview 23 2. VPSI - Design 26 3. Transcoder - Design 28 a. Initial Transcode 28 b. Requested Transcode 28 4. Video Encoding Decision Algorithm (VEDA) 29 5. Vmoox Database - Design 31 6. Vmoox Web Service / API - Design 33 7. Ad Engine - Design 35 8. Front End Application - Design 36 VIII. Vmoox Implementation 39 1. Vmoox Implementation Overview 39 2. VPSI - Implementaion 39 a. The Model Objects 39 b. Publisher Service 42 c. Downloader 43 3. Transcoder - Implementation 45 a. Transcoding Using FFmpeg 46 4. VEDA - Implementation 49 5. Database - Implementation 51 a. Video 51
- 4 - b. H264 51 c. Ad 52 6. Vmoox Web Service 55 a. Implementing Get Network Videos Service 56 b. Implementing Get List of Videos Media URLs for a Given Video 59 Id c. Ad Serving 59 7. iPhone Native and Web App 60 a. Web Based App 60 i. The Model 60 ii. The Controller 61 iii. The View 62 IX. Summary and Future Work 69 X. References 70
- 5 - List of Figures Figure 1: Pre-iPhone: BellSouth/IBM Simon Personal Communicator 7 (1993) Figure 2: H.264 Wide Range of Implementations. 11 Figure 3: Top US Smartphones Web Traffic. 12 Figure 4: Supported Formats of Popular Smart Phones in the US 12 Figure 5: Typical Publisher Implementation. 24 Figure 6: A Sample XML Submitted by Client Including Content Meta- 22 Data Figure 7: !!!!!!Transcoder - Initial Transcoding Overview 27 Figure 8: Transcoder - Real Time Transcoding Overview 28 Figure 9: Services VEDA Provides to VPSI 29 Figure 10: VEDA Chapter Name Creation 29 Figure 11: Services VEDA Provides to Vmoox Web Service 30 Figure 12: Services Vmoox Web Service Provides to the iPhone App 33 Figure 13: Vmoox Web Service JSON Formatted Response to a Get 34 Videos’ Media URLs Request, With an “ads” Array Figure 14: MVC Design Pattern 35 Figure 15: Vmoox Controller 36 Figure 16: Video Object UML 39 Figure 17: Video Chapter Object UML 40 Figure 18: Ad Object UML 40 Figure 19: VPSI UML 41 Figure 20: Transformation Class Services 44 Figure 21: Vmoox DB(s) Overview 52 Figure 22: Featured Videos View 62 Figure 23: Video View 65 Figure 24: A Video is Being Played by the iPhone Native Media Player 67
- 6 - I. Introduction
This thesis introduces Vmoox [1] - a cross-platform video publishing solution for mobile devices. It will detail the need, timeliness and profit potential of a real-time video transcoding solution for mobile devices as well as the suggested implementation. While this model is intended for use in all smart phones, the current prototype implementation is for the iPhone. This implementation can be modified to fit other and future smart phones.
1. No Longer Science Fiction – Ubiquitous Computing And The Smart Phone Explosion
The term Ubiquitous Computing has been growing in popularity in recent years, as has the phenomena it attempts to describe. Simply put, this means that computers are everywhere. More than ever before, computers have ceased to stand alone as “computing devices” but rather, have been integrated into everyday items to help them become “smarter”. We now use computers in our offices, homes, stores, cars, banks, hospitals, kitchens and gyms. From washing machines to coffee makers, to car alarms to defibrillators; like it or not, it is hard to deny that computers have made their way into nearly every aspect and activity of our modern lives.
Similarly, the Internet, which 30 [2] years ago seemed to most of us the stuff of science fiction, has become an inseparable part of our lives and culture. As a society, especially in the United States, we have come to expect and rely a great deal on both computers and the Internet. Perhaps the greatest manifestation of this phenomenon is the smart phone explosion.
IBM introduced the first smart phone in 1993 [3]. Its then revolutionary features included games, a pager, send and receive fax capabilities, a calculator and a notepad. Costing around $900 [4], this was a gadget that was far from ubiquitous. But it was the beginning of a trend that would change the way we access and use information.
- 7 - Figure 1: Pre-iPhone: BellSouth/IBM Simon Personal Communicator (1993)
In 2001, RIM introduced the first Blackberry - a device optimized for wireless email use. By June 2007 over 8 million people were hooked. In the 8 years since Blackberries came on to the scene, we have witnessed incredible advances in both the number and functions of smart phones. Today, the vast majority of cellular phones are not just phones. A recent survey showed that around the world, one out of ten phones is a smart phone [5], a number that is expected to double by 2013 [6]. Smart phones have changed the way we access and use information in a drastic way. They have become all-in-one devices that we rely on for a great deal of information, accessible at all times. Email servers, clocks and calendars, maps and navigation software, photo and movie cameras, video games, music players and calculators are just a few of the features that now come standard in every smart phone. Many of these features rely on the Internet to function. It’s the Internet that tells the calendar feature what day and time it is if, lets say, your phone’s battery died last night at 3 am. When your Blackberry buzzes with a new email, it’s because your phone connected to the Internet and retrieved that email from your email server. Until recently, smart phones relied on the Internet for purely ‘behind the scenes’ assistance. Until the iPhone changed everything.
2. All Hail The King – The Rise Of The iPhone
Within the smart phone revolution, one major player is widely acknowledged as a game changer – the iPhone. While the Blackberry allowed users to access email on their cell phone, the iPhone brought the Internet in all its glory into the palm of our hand. In no small part thanks to Apple’s signature user-friendly design and aggressive marketing, the iPhone turned the smart phone from a tech-y solution for workaholics, into the must-have gadget for everyone.
- 8 - But behind what is, depending on your point of view, either brilliant design or overrated hype was the true technological revolution of the iPhone – the Internet was no longer a background player. It was front and center, just as we were used to seeing it on our desktop computers. On the iPhone, our maps look like the maps we are used to using when looking up directions to our favorite restaurant; we can watch our favorite you Tube videos, check the weather and Google just like we’re used to. The iPhone was no longer a phone with extra capabilities - it was the Internet, in the palm of your hand.
Besides being a miniature version of our desktop computers (with a phone and music player built in) the iPhone changed our technological landscape by copying the Internet not only in form but also in concept, it was a device specifically designed for growth and the sharing of information. The iPhone’s true genius lies in its 'apps'. With over 25,000 different downloadable applications and more than 800 million downloads [7], the AppStore allows the community of users to continuously devise and implement new functions for the iPhone.
But for all its brilliance, the iPhone (along with its smart-phone counterparts) has hit a major stumbling block as it intersects with the other major Internet revolution of recent years – video content.
3. Watch It Online – Video On The Internet
In recent years, online video content has skyrocketed. As of November of 2008, Americans spent 12.5 percent of their time online viewing videos [8], watching over 14 billion clips a month [9]. You Tube’s ‘most watched’ clip has more than 123 million views [10] compared to 2008’s record setting Super Bowl which managed a ‘modest’ audience of 97.5 million [11]. Fifty-seven percent of all Americans now have high-speed Internet access at home, a number that is expected to rise to ninety percent by 2012 [12]. With Americans increasingly connected to an ever more powerful network, it seems that the appetite for online video will continue to grow in the coming years. And there is certainly no shortage of content producers and owners willing and able to satisfy that appetite. From major television networks to bloggers with a webcam, video is being uploaded almost as fast as it can be consumed.
It is only natural, therefore, that the mobile computing and online video revolutions intersect. Just as people have come to expect their Internet in the palm of their hand, they want to be able to use it in the same way they use their desktop browser. But, in the video market, mobile devices still have a long way to go. And the major hurdle they face at the moment is Flash.
- 9 - 4. It's Not Me, It's You – The Flash Problem
Flash has become an online video favorite mainly due to its high-quality/low bitrate (small file size) capabilities. As a result, nearly ninety-nine percent of users are now able to view Flash files, which in turn leads most of the newly created online video content to be predominantly flash-based [13][14].
Due to the nature of mobile devices, where power consumption is a major factor [15], Flash enabled video files, are not supported. The iPhone, for example, supports video playing in H.264/AVC [16] format, an alternative to Flash which is not nearly as popular (yet!).
Several solutions are currently available which attempt to address this incompatibility problem. Sorenson Media’s Sorenson 360 Video Delivery Network provides users the ability to convert any media into an H.264/AVC video format [17] and publish on their sites.Sorenson 360 also provides additional features like management and reports that help users track and analyze their users' video consumption. However, Sorenson 360 is an offline solution i.e., users first need to upload their videos and perform the transcoding themselves. When dealing with a large amount of media that is constantly being changed and updated, this solution is overly cumbersome and without the ability to automate becomes unfeasible to most large-scale web publishers.
Encoding.com also provides a video encoding platform. Customers send in their source content in any popular format, and receive encoded media at a location of their choice, in the format of their choice [18]. However, Encoding.com is also an offline solution, users need to request a transcoding action and manage the new iPhone enabled video themselves. One advantage Encoding.com has over Sorenson 360 is that Encoding.com provides the option to automate portions of the process by offering an API to their services.
Despite their usefulness for certain purposes, these solutions and others like them currently being offered, are not optimal for large-scale video publishers, like television networks or online video sites. They require manual management of files which is both time consuming and costly. In order to fully address the Flash compatibility problem, these publishers need a cost-effective, fully automated solution, which can operate in real time and convert large amounts of video in a relatively short amount of time. Vmoox aims to offer this sort of solution - a real- time, cost effective conversion tool to bridge the format gap between flash video content on the web and mobile devices.
Vmoox uses a SaaS [19] (Software as a Service) model in which publishers pay
- 10 - per conversion. This allows both small and large publishers to use the transcoding services in a cost-effective manner. Alternately, a publisher who anticipates very large volumes of transcoding could purchase the software at a flat rate. Another way in which the Vmoox solution helps publishers keep costs low is by allowing them to keep their existing Flash-based infrastructure in place. Publishers can keep their web-services the same (Flash-based) while reaching new audiences in the smart phone market.
The Vmoox system offers a full service solution - from real time transcoding of video content through thumbnail generation all the way to a customized smart phone app.
Publishers will submit their video meta-data content to the Vmoox system. This content will include information about the video such as the title, description, duration and a link to the original video file. Vmoox will then run an offline process adding those videos to its database and transcoding the first 3 minutes of each clip. Publishers will receive a customized smart phone application, which they can distribute to their users. For the publisher, this is a simple, one-step process which makes it both time and cost effective.
Another one of the main advantages of Vmoox is the ability to decide in real time if transcoding is needed. Many publishers offer a very large amount of content, not all of which is accessed equally frequently. The Vmoox system offers a 'smart' solution, which avoids wasted server space and computation resources by only performing a full transcode on files which are actually viewed. As mentioned above, the system initially transcodes only the first 3 minutes of each clip. The remainder of the clip will begin its transcode as soon as the file is accessed by a user, the initially transcoded 3 minutes providing the buffer with which to avoid any wait time on the user-end. Simply put, while the user watches the first 3 minutes of the clip, the rest of the clip is transcoding in the background.
Although this duration is configurable, 3 minutes is the recommended setting as this allows enough time for the transcoding process to stay ahead of the user. Vmoox video encoding decision algorithm (VEDA) will decide if and when a video needs to be encoded. More on information on VEDA, developed specifically for Vmoox, is provided in section VII2.
Research has shown that the average duration of video consumption on mobile device is 3.2 minutes [13]. Future versions of Vmoox and VEDA will include a feature which will stop the transcoding (even if it has not finished) if the user has stopped watching the clip, thus saving even more resources.
- 11 - II. H.264/AVC Smartphone Support
Though Vmoox is designed as a multi-codec transcoding tool, the prototype application demonstrates conversion from Flash Video Files - the most popular internet video format - to H.264/AVC, an emerging codec supported by the majority of smart phones currently on the US market.
H.264 is an open standard developed by ITU-T Video Coding Experts Group [20] [21]. In recent years, H.264 has become an industry standard in wide range of applications [22], from home DVR and TV services to mobile devices. Of note is also the fact that Apple has adopted the H.264 codec as a preferred codec and it's popular Quicktime application and all its plugins use it heavily[23].
The main benefits of H.264 are:
1. High quality, with relatively small file sizes (lower bit rates) – H.264 uses an advanced video compression technology that enables the creation of files with very high quality (comparable to MPEG-2 files), but up to half the data rate. 2. Wide range of implementation opportunities – H.264 reaches great compressions effectiveness for a large range of application types.
Figure 2: illustrates H.264 Wide Range of Implementations [23]
Smartphones are gaining increasing shares in the handset market. A recent survey showed that around the world, one out of ten phones is a smart phone [5], and this dominace is growing each month. The major players in this market today include Apple’s iPhone, RIM’s Blackberry line, Palm’s Pre and HTC’s G1 with Google. All of these devices support playback of the H.264/AVC codec. Figure 3 shows top US Smartphones web traffic according to a research by AdMob Mobile Metric - February 2009[23] .
- 12 - Figure 3: Top US Smartphones web traffic according to a research by AdMob Mobile Metric.
Figure 4: Supported Formats of Popular Smart Phones in the US
Device Supported Formats RIM BlackBerry Storm MPEG4 H.263, MPEG4 Part 2 Simple Profile, H.264, WMV [25]. Apple iPhone H.264 video, up to 1.5 Mbps, 640 by 480 pixels, 30 frames per second, Low-Complexity version of the H.264 Baseline Profile with AAC-LC audio up to 160 Kbps, 48kHz, stereo audio in .m4v, .mp4, and .mov file formats; H.264 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Baseline Profile up to Level 3.0 with AAC-LC audio up to 160 Kbps, 48kHz, stereo audio in .m4v, .mp4, and .mov file formats; MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps, 48kHz, stereo audio in .m4v, .mp4, and .mov file formats [16] Google Android OS based on PacketVideo's OpenCORE; the libraries support playback and recording of many popular audio and video formats, as well as static image files, including MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG [26].
- 13 - Palm Pre MPEG-4, H.263, H.264 [27].
As demonstrated in the above data, all four major players in the smart phone market support the H.264 codec. In accordance with this market trend, Vmoox will provide a solution based on the H.264/AVC format. As it's core technology, Vmoox will perform real time H.264/AVC video transcoding.
III. FFmpeg/libavcodec
Several of Vmoox's key features will be implemented using the FFmpeg and Libavcodec tools.
FFmpeg defines itself as “ a complete, cross-platform solution to record, convert and stream audio and video. It includes libavcodec - the leading audio/video codec library. [28]"
Libavcodec is an open source free software licensed under LGPL that includes video decoders and encoders for a variety of video formats including flash video and H.264/AVC formats [29].
FFmpeg based projects include among others, the popular VLC media player from VideoLan - which provides a cross-platform media player as well as a streaming solution [30] [31], and Perian, an open source plug-in to Apple’s QuickTime. Perian enables users to play a range of video formats that are not natively supported by QuickTime for example: FLV (i.e, flash video), Divx, 3ivx and much more [32].
Vmoox will utilize FFmpeg and libavcodec to transcode flash video files to H.264/ AVC, create thumbnails of video files, create chapters within the original video files and obtain information about the files themselves.
IV. Software as a Service (SaaS)
Vmoox will be designed and implemented using the Software as a Service (SaaS) model.
SaaS is model in which users pay by "action", as opposed to a larger one-time
- 14 - licensing fee. This model is ideal for smaller or gradual users as it avoids the need to pay large sums in advance of future usage of the [33]. The main advantages of adopting SaaS model are [34] [35]:
1. Centralized feature updating– SaaS providers can push new features and functionality easily. Since the software 'lives' with the creator and is accessed by the user, there are no updates to install and bugs can be addressed easily on the provider's side. 2. Low-Risk – Customers can avoid the heavy costs and risks of implementing or paying high licensing fee for services that don't work for them. Because the business model of SaaS is an on-demand model i.e., pay per user or pay per request customers can keep cost effective features and terminate other services. 3. Fast Implementation – In most SaaS applications customers can easily sign up and start using the service. There is no need to install, deploy and configure any software.
Vmoox's SaaS model will function as follows:
Publishers will sign an agreement with Vmoox that includes a one-time setup fee and receive a customized smart phone app (prototype version is an iPhone app), and then be charged per video - i.e. each new video added will incur a charge as will each access request from a user. Publishers anticipating a large amount of data will be able to pay a flat licensing rate to be determined by their expected usage.
V. XStream
Vmoox customers (publishers) will receive a smart phone app with which their users will be able to view their content. Communication between the smart phone application and the Vmoox server will be via the JSON format. The XStream will be used to serialize the Java objects coming from the Vmoox server (video and ad content) into the JSON format.
Xstream is an open source library that serializes Java objects to XML and JSON and vise versa [36]. When designing a web service it is often required to
- 15 - implement several end-points to your services. For example: some clients will prefer to interact with the service’s data thru XML while other types of applications will prefer JSON or pure Java objects thru RPC or SOAP. XStream provides an abstraction layer on top of your business logic rules layer by separating business rules and presentation. This helps your web service design to be loosely coupled i.e., business rules are separated with the output presentation format such as XML, JSON or Java objects.
The main advantages of Xstream are: 1. Ease of use – Xstream provides an abstraction layer on top of object serialization. Common use-cases are easy to implement and use. 2. No complicated object mapping - most objects can be serialized without need for specifying mappings. 3. Performance – Xstream serelization is fast and low memory consumption is a cure requirement in the design and implementation of Xstream.
Vmoox's web service will utilize XStream in order to provide clients a JSON/ JSONP RESTful [37] service. Vmoox web service will respond to a HTTP GET request and will generate java object using Hibernate [38]. XStream will then serialize these Java objects into JSONP and post it back to the Request.
- 16 - VI. Vmoox Requirements
The goal of the Vmoox system is to enable web publishers to distribute their video content on mobile devices that don't support Flash. In order to do this, Vmoox will follow the "Software as a Service" (SaaS) model and implement the following requirements:
1. Publisher Service Interface
As part of the SaaS model, Vmoox will provide an interface where publishers can input original content. This will be the "entry point" of the content into the Vmoox system and include the following features: (a) Account Setup
This one-time process will generate an internal Vmoox Customer ID for the publisher. This Customer ID will identify the content provider (publisher) as well as the video’s meta-data in the Vmoox database(s). Once Vmoox provides the client with a Customer ID, they can start initializing the account. (b) Content Input
As soon as a client has set up an account, they can begin inputing content into the Vmoox system. Publishers can choose to store content internally or externally. In an internal integration, the Vmoox service will "live" in the publisher's data center but information about the content will be stored in the Vmoox database. In an external integration content and information will "live" on the Vmoox server. Publishers will provide the location of the original content and the VPSI will import either the data or files and data into the Vmoox database and then pass the video on to the Transcoder for an initial transcode.
(c) Pre-Roll Content Input
Publishers will have the option to inject an advertisement (or any other content of thier choosing) into the pre-roll section of their video. In order to do this a publisher must input the pre-roll content into the Vmoox system and define it as such. Publishers must also define criteria for pairing Pre-Roll and primary content.
- 17 - (d) Content Update An existing client will be able to add or update content in the Vmoox system.
(e) Thumbnail Creation When new content is input into the Vmoox system, the VPSI will create a thumbnail for each clip that will serve as the icon the user sees for the clip as part of the Front End Application. These thumbnails will also be stored in the Vmoox (or client) database and linked to the appropriate video file.
(f) Customize Smart Phone Web or Native Application As part of the initializing process, the publisher will define criteria for the customized smart phone application for the device(s) of their choosing. The publisher will be able to choose between a native app (specific to a particular device) or a web app (runs on the browser of any device but requires web browsing capabilities) or both. This information will be used to create the Front End Application which will be delivered to the publisher and can be distributed directly by the publisher or by a third party. The prototype implemented as part of this thesis is an iPhone app for CNN. More requirements related to the iPhone application will be described in section 5.
- 18 - 2. Transcoder Vmoox will perform offline as well as real-time transcoding. The main functionality requirement of the Vmoox system is the ability to transform Flash- based video content to the H.264/AVC codec, which users can access directly on their smart-phones. The Transcoder will peform the following functions in the following instances:
(a) Initial Transcode Any time a publisher inputs content into the Vmoox system, whether a new client performing an initial input or an existing client updating/ adding content, the Transcoder will transcode the first section of each clip. The length of the initial section should be a configurable variable (i.e., each publisher should be able to detrmine how long of a section they would like transcoded) but the default is 3 minutes. This allows a user to begin watching a video with no wait/ load time while still optimizing storage/ server space by only transcoding fully those videos that have been requested. (b) Requested Transcode When a user requests a video for the first time via the Front-End App, the Transcoder will receive a request from VEDA (see below) to begin transcoding the remainder of the clip in 3 minute (or other duration determined by publisher) increments- called Chapters. Each Chapter will be named according to a list generated by VEDA.
3. Video Encoding Decision Algorithm (VEDA)
The Video Encoding Decision Algorithm (VEDA) plays a crucial role in the conservation of resources - a major advantage of the Vmoox system. VEDA recieves a request from a user via the Front End Application and Web Service/ API and immediately determines whether a video is fully transcoded or needs to be transcoded. Based on that determination it routes the videos to the appropriate actions:
- 19 - (a) Fully Transcoded Videos Videos that have either been viewed once already or are up to 3 minutes in length are retrieved from the Database and delivered directly to the user via the Front End App. (b) Partially Transcoded Videos Videos that have never been viewed and are over 3 minutes in length are sent to the Transcoder. Asynchronously, VEDA assigns chapter names to each 3 minute (or other length as determined by the client) increment of the clip and sends a Playlist of chapters to the Front End Application and to the Transcoder. The result is that at the moment a user selects a video to view, the Front End Application begins playing a series of video chapters, only the first of which is an existing file (the rest are virtual) with no wait time on the user end. By the time the first chapter (3 minute section) has finished playing, and the Front End Application goes to retrieve the second section, the Transcoder has already finished creating it and it is no longer virtual, but actually exists and can be retrieved, and so on with all subsequent chapters.
4. Database
All content will be referenced in the Vmoox Database. There are essentially 5 types of content:
(a) Original Content Files Flash or other non-smart-phone compatible files coming directly from the publisher. (b) Meta-Data (including thumbnails) Information about a clip including: title, description, duration, keywords, publisher, thumbnail image. (c) Fully Transcoded Video Files A collection of 3-minute chapters making up the whole. (d) Partially Transcoded Video Files (e) Pre-Roll Content
- 20 - 5. Web Service/ API
The Web Service facilitates communication between the Front End Application and the Back End Components (VEDA, Transcoder, Database, Ad Engine)
The Web Service will output JSON/JSONP data format, or XML and provide the following services, which correspond with the functions of the Front End App:
(a) Featured Video List A list of the latest (or preferred) video content from the publisher. Criteria for inclusion in the Featured List can be determined by the publisher (i.e., newest or most popular videos). In the Featured List, the following will be provided for each video: thumbnail URL, title, duration, video ID and description. (b) Video Playlist Information about an individual clip including: thumbnail URL, title, duration, video ID and description; as well as a list of chapters (3 minute segments) and Pre-Roll content. (c) Search List A list of videos including the search term in thier meta-data. For each video the following will be provided: thumbnail URL, title, duration, video ID and description.
6. Ad Engine
This component pairs a publisher's Pre-Roll content with its Primary content based on criteria defined by the publisher. The Pre-Roll content will play before the primary content and the user will not be able to skip or fast-forward it. This component will also ensure that the same pre-roll ad does not repeat on the same user session, unless defined otherwise by the publisher.
7. Front End Application (native and/or web based)
A major component of Vmoox is the Front End Application. This is the means of delivery of the transcoded content to the smart-phone user. Publishers can customize the Front End Application in both appearance (to fit thier brand identity and traget audience) as well as functionality. Publishers can choose to create either a native smart-phone application (specific to a particular device) or a web application (runs on the browser of any device but requires browsing
- 21 - capabilities), or both.
The basic template, which can be expanded or revised by the publisher, will include the following 3 modes, or Views: (a) Featured View The Featured View is the first view a user sees when launching the application. The Featured View includes the latest (or preferred) video content from the publisher. Criteria for inclusion in the Featured view can be determined by the publisher (i.e., newest or most popular videos). In the Featured View, the following should be displayed for each video: thumbnail, title, duration, and a link to the video view. The user selects a video and is taken to the Video View.
(b) Video View After selecting a video from the list, users will be taken to the Video View - where the thumbnail and information for their selected video is displayed. Clicking on the video thumbnail (or adjacent button) initiates the video content - playing first any pre-roll content and then the actual video. (c) Play View In this view users watch first the pre-roll content as determined by the publisher, and then the actual video. There is no load or wait time on the user end. (See explanation in VEDA section)
The prototype Front End Application will be a native iPhone application for CNN (sample publisher).
8. Loosely Coupled Modular Architecture
Vmoox will be designed as a loosely coupled collection of modules. Each of the above components will interact with each other through an API regardless of how each component is implemented.[39] This will improve maintainability and readability of the software [40] as well as open Vmoox to third-parties who may use Vmoox in combination with their own software applications.
- 22 - VII. Vmoox Design
1. Vmoox Design Overview
The goal of the Vmoox system is to provide a customizable, scalable video transcoding and publishing solution for web publishers. In order to accomplish this goal, Vmoox provides publishers with a generic yet powerful integration interface, a loosely coupled Trancoder and Encoding Decision Algorithm and a customizable Front End Application which can be integrated with the publisher's brand identity and functionality requirements.
The Vmoox solution includes the following components:
a. Vmoox Publisher Service Interface (VPSI) - provides the entry point for data from the publisher (video content and information). This component receives incoming content and data, classifies and routes it appropriately. b. Transcoder - recieves video content, transcodes it into the desired format and creates thumbnails for each clip which will be used to identify it in the Front End Application. c. VEDA (Video Encoding Decision Algorithm) - identifies whether videos have or have not been transcoded. d. Database - stores and classifies all data. e. Web Service/API - serves as a bridge between the front-end applications and the back-end components. f. Ad Engine - provides a pre-roll ad before video clip. g. Front End Application - enables users to consume publisher video content on a mobile device (native or web-based).
The components interact with each other using a loosely coupled architecture model, where each component operates independently and communicates with the other components via an API.
The typical flow of the Vmoox system (illustrated in Figure 5) is: 1. A publisher sets up an account with Vmoox via the Vmoox Publisher Service Interface (VPSI); creates a user ID and password, defines criteria for Front End Application and Pre Roll content pairing and provides the location of content. Publisher recieves a customized Front End Application per specifications and distributes to users directly or via third-party (offline).
- 23 - 2. VPSI retrieves content from the specified location, updates the Vmoox Data- base and sends videos to the Transcoder for initial transcoding. 3. Transcoder recieves video content, performs an initial transcode (ie. transcodes the first three minutes of each clip), creates a thumbnail image for each clip. 4. VPSI delivers initially transcoded materials and meta-data to Database. 5. User activates Front End Application and selects a video for viewing. 6. Front End Application sends request for video via the JSON/API Web Service 7. VEDA recieves the request via the JSON/API Web Service and determines whether the requested clip has been transcoded or not. 8a. If a clip has already been transcoded, VEDA sends the Front End Application a list of chapters (3-minute segments) that make up the clip to be retrieved from the Database and delivered to the user via the JSON/API Web Service. 8b. If a clip has not yet been transcoded (first time viewed), VEDA will generate a (virtual) list of chapters (3-minute segments) based on the clip's length. VEDA will send this list to both the Front End Application and the Transcoder. Asynchro- nously, the Front End Application will begin playing the chapters in sequence and the Transcoder will create the chapters and deposit them in the Database. The result is that by the time the Front End Application attempts to retrieve a chapter file, it will have already been created. The initially transcoded first chapter pro- vides the buffer which results in zero wait/load time on the user end. Chapter files will be deilvered to the user through the Front End Application via the JSON/API Web Service. 9. As the first chapter of each clip is being retrieved, the Ad Engine will select an appropriate Pre-Roll content and deliver it to the Front End Application via the JSON/API Web Service to be played before the first chapter of the user selected primary content. 10. User enjoys his selected video content on his mobile device preceeded by the Pre-Roll content selected by the publisher.
- 24 - Figure 5: Typical Publisher Implementation
- 25 - 2. VPSI - Design
The Vmoox Publisher Service Interface (VPSI) is the "entry point" of the content into the Vmoox system. When fully implemented, the VPSI will allow clients to perform the following actions via a web-based application: Account Setup; Con- tent Input; Pre-Roll Content Input, Content Update and Smart Phone App Cus- tomization Requirements. In the implementation for this thesis, the client was a virtual one (CNN was used as a model but without it's official participation). As a result, only the input and update components were implemented.
Figure 6 presents a sample case (simulated) wherein a client "provided" Vmoox with an XML of meta-data for the content it "wanted" included in it's smart-phone app.
Figure 6: A Sample XML Submitted by Client Including Content Meta-Data
- 26 -
- 27 - 3. Transcoder - Design
Vmoox performs offline as well as real-time transcoding. In this implementation, videos are transcoded from Flash to the H.264/AVC codec. The transcoder provides transcoding services in the following 2 instances:
(a) Initial Transcode A request for this service comes from the VPSI. When new content is input, the VPSI sends all files to the Transcoder and the Transcoder transcodes the first section of each clip. The length of the initial section is a configurable variable (i.e., each publisher is able to detrmine how long of a section they would like transcoded) in this implementation the duration is 3 minutes. (b) Requested Transcode A request for this service comes from VEDA. When VEDA receives a requese for a video that has not yet been fully transcoded, it routes this file to the Transcoder for a full transcode. As part of the request, VEDA sends the transcoder a list of chapters to generate.
Figure 7: Transcoder - Initial Transcoding Overview
- 28 - Figure 8: Transcoder - Real Time Transcoding Overview
4. Video Encoding Decision Algorithm (VEDA)
VEDA identifies whether or not a video is fully transcoded. During the initial transcode, files are flagged as partially transcoded. After a clip has been watched once and fully encoded, VEDA marks it as such.
VEDA receives requests from the Web Service and differentiates between fully and partially transcoded files. It routes fully transcoded files (previously viewed or shorter than 3 minutes) directly to the Front End App via the Web Service, and sends partially transcoded files to the Transcoder for a full transcode.
When a partially transcoded file is sent to the Transcoder, VEDA also generates a list of chapter names for the Transcoder to generate. This list is simultaneously sent to the Front End App via the Web Service and serves as a Playlist. The Playlist instructs the Front End App of the order in which the chapters should be played. This design eliminates wait/load time on the user end, by using the initially transcoded first chapter as a buffer while the rest of the video is being transcoded.
Figure 9: Services VEDA Provides to VPSI
- 29 - Figure 10: VEDA Chapter Name Creation
Figure 11: Services VEDA Provides to Vmoox Web Service
- 30 - 5. Vmoox Database - Design
The Vmoox Database supports and contains information about all the Vmoox components. 3 types of objects are described in the database: Video, Vide Chap- ter (H264) and Ad. Each object is represented by a table that mirrors the object's properties and contains the information which helps each component perform as specified. (a) Video The table for the Video object reflects the following properties: (i) Network Id - Each publisher is assigned a unique number which identifies its content. VPSI appends this information the object as video files are
- 31 - imported into the system. This helps assure that the correct videos are channelled to the publisher's Front-End App. (ii) Title - Provided by the Publisher. Delivered by the Web Service to the Front End App which displays the title to the User along with the video. (iii) Description - Provided by the Publisher. Delivered by the Web Service to the Front End App which displays the description to the User in the Video View. (iv) Image - Reference to a file created by the Transcoder during the Initial Transcode and stored on the Vmoox Server. Delivered by the Web Service to the Front End App and used as the representation of the video to the user in the Featured and Video Views. (v) Original Media URL - Used by VPSI to deliver the files to the transcoder module. (vi) Original Media - Reference to a file created by the Transcoder during the Initial Transcode and stored on the Vmoox Server. (vii) Insert Date - Generated by the DB. Represent a video insert date to Vmoox DB. (viii) H264 Ready (Y/N) - A flag that indicates if a video is fully transcoded i.e., H264 ready or not. (ix) Duration (in sec) - The duration in seconds of a video. This value is being calculated by the Transcoder module. (x) H264 Media URL - Reference to a file created by the Transcoder during the Initial Transcode and stored on the Vmoox Server. Delivered by the Web Service to the Front End App and used as the the first video chapter. (xi) Has Chapters (Y/N) - A flag that indicates if a video has chapters or not.
(b) H264 The table for the Video Chapter object reflects the following properties: (i) Video Id - Reference to a video entry in the Video table. Each video entry can have multiple instances in the H264 table indicates the video has multi- ple chapters. (ii) H264 Media - Reference to a file created by the Transcoder in real time and stored on the Vmoox Server. Delivered by the Web Service to the
- 32 - Front End App and used as the the video chapters. (iii) Image - Reference to a file created by the Transcoder during the the real time Transcode and stored on the Vmoox Server. Delivered by the Web Service to the Front End App and used as the representation of the video chapter to the user in the Video View. (iv) Title - Provided by the Publisher. Delivered by the Web Service to the Front End App which displays the title of a chapter to the User. (v) Duration (in sec) - The duration in seconds of a video. This value is being calculated by the Transcoder module.
(c) Ad The table for the Ad object reflects the following properties: (i) H264 Media - Provided by the Publisher. Delivered by the Web Service to the Front End App and used as the the pre- roll video ad. (ii) Duration (in sec) - Provided by the Publisher. The duration in seconds of a video ad.
6. Vmoox Web Service / API - Design
The Vmoox Web Service provides services to both the web and native iPhone applications. The web service provides information for the following criterias: a. List of randomly selected video’s meta-data such as: title, description, duration and image. b. Relevant meta-data for a given video id. c. List of video’s meta-data that contain a given keyword in the video title and/or description – this will support the search functionality. d. List of videos’ media URLs for a given video id. e. Media ad URL for a given video id.
- 33 - Figure 12: Services Vmoox Web Service Provides to the iPhone App
As illustrated in Figure 12 Vmoox web service uses an Ad serving algorithm module to append a pre-roll video ad before serving the publisher content. When a request of type “Get videos’ media URLs” enters the Vmoox web service, the web service uses VEDA in order to decide if an asynchronies real time transcoding is needed and to generate the list of videos’ chapters (including the pre-roll ad).
The mentioned requests are HTTP GET requests, when the response for these requests is also an HTTP JSON/JSONP formatted response. This will enable the clients to easily read and manipulate data on the client side.
There are several benefits of using JSON/JSONP web service as Vmoox web service protocol:
1. The use of web service as the data delivery mechanism reinforces a good software engineering design practice when components are loosely coupled [41].
2. JSON/JSONP is a lightweight protocol that enables developers to easily
- 34 - connect to Vmoox’s web service and to use Vmoox’s data in their own applications. JSON is a data format that is more naturally fit for browser data consumption [42], because it is a subset to JavaScript and can be easily parsed by the browser. Also Objective-C the iPhone programming language provides easy integration with JSON enabled web service [43].
7. Ad Engine - Design
Vmoox provides publishers with the ability to generate revenue from their content. According to eMarketer by 2010 one in ten dollars devoted to Internet advertising will go towards video placements [44]. This means that publishers will have the inventory and the advertisement budget to sponsor further video distribution of content. As illustrated in figure 12 when a request to get videos’ media URLs enters vmoox web service, the web service in addition to get the relevant videos chapters generate a request to the ad serving engine to receive the proper ad(s).
The following JSON formatted text i.e., Figure 13 illustrates how vmoox web service response to a request to get videos’ media URLs for the video Id = 11246. The “ads” property in this JSON response demonstrates how the client application can append this video to the publisher video content.
Figure 13: Vmoox Web Service JSON Formatted Response to a Get Videos’ Me- dia URLs Request, with an “ads” Aray.
[{"video":{"id":11246,"title":"Quest's Profitable Moment-6\/ 11","image":"Quest_s_Profitable_Moment_6_11_0_72.jpg","h264Media":"Quest_ s_Profitable_Moment_6_11_0_72.mp4","h264Ready":false,"hasChapters":false," duration":72,"networkId":1,"insertDate":"2009-06-11 17:46:14.0","ads":{"com.vmoox.model.Ad":[{"id":3,"h264Media":"BMW_z4_30_3D tag.mp4","duration":30}]}}}]
In order to maximize ads efficiency and provide better user experience the ads serving engine randomly selects ads. This will enable the application to offer different pre-roll video for different vides, and to allow a better diversity. This of course is not only a technical challenge, but also a marketing/sales challenge to insure the publisher has a sufficient number of ads inventory.
- 35 - 8. Front End Application - Design
The Vmoox Front End application delivers the transcoded content from Vmoox servers to the end users. While designing the Front End application several key attributes were considered:
1. Customization - Each publisher will be able to customize the Front End application in both appearance to fit their brand identity and target audience as well as functionality. Some publisher would prefer controlling the videos presented in the Featured view, while other could prefer a random selection of videos. Another example is the ability to control the amount and content of pre- roll ad before each video.
2. Platform - Publishers will have the ability to receive either a native smart- phone application (specific to a particular device) or a web application (runs on the browser of any device but requires browsing capabilities), or both. All these platform share most of the code base i.e., Vmoox Web Service provides services to all these platforms and Vmoox other modules VEDA and the Transcoder are platform independent.
3. Following MVC design pattern -
The Front End application’s architecture uses the MVC design / architecture pattern.
Figure 14: MVC Design Pattern
The benefit of using MVC design pattern is the ability to create an independency
- 36 - between component. Data access code, business logic code and presentation code are all separated. This allow us to create different views for different publishers according to their specific specifications. Keeping low coupling between different types of classes or decoupling data access code, business logic code and presentation code will make classes easier to maintain and reuse [49].
(a) The Model Objects:
The same model objects are used by VPSI, VEDA, Vmoox Web Service and Vmoox Front End Application (web based)
These objects are Ad.java, Video.java and VideoChapter.java
(b) The Controller Objects: (Web Based app)
The controller needs to do the heavy lifting of getting the data and building the model that will be displayed. The sequence of events for the controller starts when a user starts the application. The controller then gets information related to the specific publisher from Vmoox Web Service. This logic is implemented in the main controller class Vmoox.java. This class implements two interfaces the EntryPoint interface which indicates this is the entry point of the application, and the VmooxListener.
The following UML describes this relationship:
Figure 15: Vmoox Controller
- 37 - (c) The View Objects: (Web Based app)
The view objects purpose is to create and display to the user three views:
(i) Featured View
The Featured View is the first view a user sees when launching the application. The Featured View includes the latest (or preferred) video content from the publisher. Criteria for inclusion in the Featured view can be determined by the publisher (i.e., newest or most popular videos). In the Featured View, the following should be displayed for each video: thumbnail, title, duration, and a link to the video view. The user selects a video and is taken to the Video View.
(ii) Video View
After selecting a video from the list, users will be taken to the Video View - where the thumbnail and information for their selected video is displayed. Clicking on the video thumbnail (or adjacent button) initiates the video content - playing first any pre-roll content and then the actual video.
(iii) Play View
In this view users watch first the pre-roll content as determined by the publisher, and then the actual video. There is no load or wait time on the user end. (See explanation in VEDA section)
- 38 - VIII. Vmoox Implementation
1. Vmoox Implementation Overview
The main development language that was used to implement Vmoox was Java. VPSI, VEDA, Vmoox Web Service, and the iPhone web application were all im- plemented utilizing Java and open source projects. The Vmoox native iPhone app was implemented using apple's iPhone SDK OS 3.0 [47]. The following open source frameworks were used to implement Vmoox: 1. Hibernate: "Relational Persistence for Java and .NET" [38]. 2. MySql: "The world's most popular open source database" [45]. 3.FFmpeg: "A complete, cross-platform solution to record, convert and stream audio and video" [30]. For more details see section III of this paper. 4. libavcodec: "A library of codecs for encoding and decoding video and audio data" [29]. 5. Xstream: "A simple library to serialize objects to XML and back again" [36]. For more details see section V of this paper. 6. GWT: "An open source framework that allows web developers to build rich internet application using the Java language. GWT cross-compiles Java source code into optimized JavaScript that automatically works on all major browsers" [46].
2. VPSI - Implementaion
VPSI provides web publishers an interface to Vmoox system. Publishers can easily provide VPSI an XML feed with the data to be published. Web Publishers should not be concern how Vmoox implements this integration process. Also changes in this specific implementation are not visible to publishers.
(a) The Model Objects:
The same model objects are used by VPSI, VEDA and Vmoox Web Service. These objects are Ad.java, Video.java and VideoChapter.java
- 39 - Figure 16: Video Object UML
Video private int id; private String title; private String description; private String image; private String originalMedia; private String originalMediaLocalName; private String h264Media; private Boolean h264Ready; private Boolean hasChapters; private Integer duration; private int networkId; private Timestamp insertDate; private List
The Video object holds information about a video. This information includes an internal video Id, the video title, description, link to the original flash video file, link to a thumbnail, a link to a H.264/AVC video, the duration of the video and if the video was fully transcoded to H.264/AVC or not.
Figure 17: Video Chapter Object UML
- 40 - VideoChapter private int id; private int videoId; private String title; private String image; private String h264Media; private Integer duration; public int getId() public void setId(int id) public int getVideoId() public void setVideoId(int videoId) public String getTitle() public void setTitle(String title) public String getH264Media() public void setH264Media(String media) public Integer getDuration() public void setDuration(Integer duration)
The Video Chapter object is not been used by VPSI, but it is an important data structure. A Video object holds a reference to its video chapters. A video has video chapters if its duration is longer then three minutes.
Figure 18: Ad Object UML
Ad private int id; private String h264Media; private Integer duration; public int getId() public void setId(int id) public String getH264Media() public void setH264Media(String media) public Integer getDuration() public void setDuration(Integer duration)
The Ad object is also currently not been used by VPSI. If required a publisher can integrate his ads. In this cases the Ad data structure will be used by VPSI.
- 41 - Vmoox Web Service uses the Ad object in order to provide the client app a list of pre-roll video ads. Figure 19: VPSI UML
(b) Publisher Service
The PublisherService.java is an abstract class that holds a reference to the Downloader class and the Transformation class. Classes that extends this class must implement the abstract run method. The PublisherService in its constructor assign its local verbal transformation us- ing the static getInstance() method: public abstract class PublisherService { protected Downloader downloader; protected Transformation transformation;
public PublisherService() { super(); downloader = Downloader.getInstance();
- 42 - transformation = Transformation.getInstance(); }
public abstract void run(); }
Implementing the run() method:
The class CNNService extends PublisherService therefore it must implement the abstract run() method. The purpose of this method is to connect to the publisher XML service and create a list of Video objects. Once all videos objects are avail- able this method checks which videos are new and which are already available in the Vmoox system. Then new videos are download and the transcoding process begins.
(c) Downloader
The only purpose of the Downloader class is to download a remote file and save it locally. This class has one important method i.e., download(String local- FileName) that receives a string that specify what is the local file name to be created. The method uses Java's URL and URLConnection objects in order to connect to the remote site, and save the byte array into a local buffer. When the entire file is available locally the method writes the byte array into a file using Java's PrintWriter class. Below you can see Downloader's download method public String download(String localFileName) { BufferedInputStream in = null; ByteArrayOutputStream baOut = null; try { URL url = new URL(urlStr); URLConnection urlConn = (URLConnection) url.openConnection(); in = new BufferedInputStream(urlConn.getInputStream()); baOut = new ByteArrayOutputStream(); byte[] buffer = new byte[50 * 1024]; // 50K buffer while (true) { int n = in.read(buffer);
- 43 - if (n == -1) break; baOut.write(buffer, 0, n); } localFileName = Utils.formatText(localFileName); localFileName += ".flv"; String outPutFile = localMediaPath+localFileName; PrintWriter out = new PrintWriter(new BufferedWriter( new FileWriter(outPutFile))); out.print(baOut); out.flush(); out.close(); baOut.flush(); baOut.close(); return localFileName; } catch (Exception e) { //handle exception } finally { if (in != null) try { in.close(); } catch (Exception e) { } if (baOut != null) try { baOut.close(); } catch (Exception e) { } } return ""; }
Once a file was downloaded using the Downloader class VPSI uses the Transfor- mation class in order to initialize a first 3 minutes off-line transcoding action. The last step of VPSI is updating the Vmoox database. VPSI uses the DBProxy class that provides generic reading and writing operation on the database. When VPSI needs to decide if a video that comes from the publisher interface is new, VPSI uses the DPProxy class to retrieve all publishers videos, and checks if new videos are needed to be added to Vmoox's DB. The following is this logic implementation:
- 44 - DBProxy dbProxy = DBProxy.getInstance(); boolean newVideo = false; List
3. Transcoder - Implementation
The transcoder module responsibility is to perform partially or full video transcod- ing. VPSI and Vmoox Web Service uses this module by interfacing with the Transformation class.
Figure 20: Transformation Class Services
Transformation Video transform(Video video, boolean fullTransformation,Integer chapterDuration) List
(a) Transcoding Using FFmpeg
- 45 - VEDA uses ffmpeg in order to transcode flash videos to H.264/AVC videos. Below is the implementation of generateChapter(final String inputFile- Name,final Integer from, final Integer duration) method. This method as input receives the path to the original flash video file, the from and duration of the transcoding. As output this method returns the path to the new H.264/AVC video file. private String generateChapter(final String inputFileName, final Integer from, final Integer duration) { ArrayList
// any output? StreamGobbler outputGobbler = new StreamGobbler(proc .getInputStream(), "OUTPUT"); errorGobbler.start(); outputGobbler.start(); return inputFileName.replace(".flv", "") + "_" + from + "_" + duration; }
This method uses Java's RunTime and Process classes in order to execute a shell script that class the ffmpeg program. Below is the implementation of the h264_h shell script:
/opt/local/bin/ffmpeg -y -threads 0 -i $1 -vcodec libx264 -s 480x272 -flags +loop -cmp +chroma -deblockalpha 0 -deblockbeta 0 -
- 46 - crf 24 -bt 256 -refs 1 -coder 0 -me_range 16 -subq 5 -partitions +parti4x4+parti8x8+partp8x8 -g 250 -keyint_min 25 -level 30 -qmin 10 -qmax 51 -trellis 2 -sc_threshold 40 -i_qfactor 0.71 -acodec libfaac -ab 96k -ar 48000 -ac 2 -t $2 -ss $3 $4
This script as an input receives the original flash video file name (i.e.,-i $1), the duration of the transcoding to perform (i.e., -t $2), the time stamp to start the transcoding from (i.e., -ss $3) and the output file name (i.e., $4). The video codec that is being uses is specified in the -vcodec flag (i.e., -vcodec libx264).
The Transformation class use the singleton design pattern. The single design pattern insures that only one instance of a class is created, provides a global point of access to the object and also allow multiple instances in the future with- out affecting a singleton class's clients [48]. Below is the first few lines of Trans- formation.java that illustrates the singleton design patten in action. public class Transformation { private Properties properties; private String localMediaPath; private String h264MediaPath; private String h264Transformation; private String fileOutPutExtension; private Integer chaptherDuration;
private static Transformation instance = null;
public static Transformation getInstance() { if (instance == null) { instance = new Transformation(); } return instance; }
protected Transformation() { //implementation... } } The constructor visibility of this class is protected, this insure that clients of this class are able to access it only through its static getInstance() method. As described above the VPSI module uses the Transcoding class to perform an initial transcoding. If a video duration is less then three minutes then the
- 47 - transcoding interface transcode the video and signal that the file is H.264 ready. If the video duration is more then three minutes then the transcoding interface generates a list of chapters for the video and transcode the first one. Below is this logic implementation: public Video transform(Video video, boolean fullTransformation, Integer chapterDuration) { final String sourceFileName = video.getOriginalMediaLocalName(); Integer videoDurationInSeconds = getVideoDuration(sourceFileName); video.setDuration(videoDurationInSeconds); final List
This method is also used when the VEDA in real-time decides if a full transformation is needed. In this case the parameter fullTransformation is passed with a 'true' value.
- 48 - 4. VEDA - Implementation
VEDA identifies whether or not a video is fully transcoded. During the initial transcode, files are flagged as partially transcoded. After a clip has been watched once and fully encoded, VEDA marks it as such.
VEDA receives requests from the Web Service and differentiates between fully and partially transcoded files. It routes fully transcoded files (previously viewed or shorter than 3 minutes) directly to the Front End App via the Web Service, and sends partially transcoded files to the Transcoder for a full transcode.
When a user requests a video, VEDA in real time decides if real time encoding is needed. If this is the case the service generates the chapters name, start an asynchronous job that transcode the video and immediately returns the output the the client. This insures that users will be able to immediately start to watch the video when in the background a transcoding task is being executed.
Below is part of this logic implementation: if (video.isH264Ready()) { if (video.isHasChapters()) { video.setVideoChapters(getVideoChapters(video)); } video.setAds(getAd(video)); jsonResult.append(xstream.toXML(video.getToJSONObj())); } else { video .setVideoChapters(getVideoChaptersAndStartTransformation(video)); video.setAds(getAd(video)); jsonResult.append(xstream.toXML(video.getToJSONObj())); }
At first the code checks if the file is already H264 ready. This can be the case if a user already watched the video, thus the video was already transcoded, or if the original video duration was less them 3 minutes. If the video needs further transcoding them the code calls getVideoChaptersAndStartTransforma- tion(video). This method as you can see below start a new thread that transcode the file and immediately return the video chapters names:
- 49 - private List
After the video was fully transcoded the method change the state of the video ob- ject by indicating that the video is H.264 read and it has chapters and updates the DB with the new video setting and the new video chapters that was just created.
- 50 - 5. Database - Implementation
Vmoox's schema contains three tables.
(a) Video
(b) H264
- 51 - (c) Ad
DBProxy class uses Hibernate in order to map the model objects (i.e., Video, VideoChapter, Ad) to the above database tables. Below are the relevant mapping specified in the Hibernate configuration file:
This mapping declarations indicate that the above resources files are used to map objects to tables.
For example the following is the XML that describes the mapping between the Video.java class and the DB table video:
- 52 -
From this XML file one can see how the Video class properties are mapped to the Video table columns.
Two components uses Vmoox DB(s). VPSI reads and updates the DB as already described above, and Vmoox Web Service reads from the DB and provide web services to client applications. (i.e., the iPhone apps).
Figure 21: Vmoox DB(s) Overview
VPSI Vmoox Web Service
Vmoox DB(s)
DBPoxy.java provides updates and reads services for VPSI and Vmoox Web Service. Below you can find the implementation of the method insert- NewVideos(List
- 53 - public void insertNewVideos(List
This method provided services to VPSI and insures that only new videos will be added to the DB. (i.e., no duplicates).
- 54 - 6. Vmoox Web Service
As describes in the design section of this document Vmoox Web Service pro- vides services to both the web and native iPhone applications. Vmoox Web Ser- vice provides two entry points to it services. Those two end-point are defined in the web application web.xml file:
The first service Vmoox Web Service provides is retrieving a publisher videos data. Client can submit a request by generating the following HTTP GET request: http://www.vmoox.com/web/videos?networkId=PUBLISHER_NETWORK_ID The response to this request will be a JSON format data as follow:
[{"video":{"id":11259,"title":"SOTU_Help in the Hardest Times","image":"SOTU_Help_in_the_Hardest_Times_0_180.jpg","h264Media":"S OTU_Help_in_the_Hardest_Times_0_180.mp4","h264Ready":false,"hasChapter
- 55 - s":false,"duration":245,"networkId":1,"insertDate":"2009-08-09 11:45:01.0"}},{"video":{"id":11260,"title":"SOTU Oregon Diner","image":"SOTU_Oregon_Diner_0_180.jpg","h264Media":"SOTU_Oregon_ Diner_0_180.mp4","h264Ready":false,"hasChapters":false,"duration":225,"netwo rkId":1,"insertDate":"2009-08-09 11:25:17.0"}},{"video":{"id":11257,"title":"The 'public option'","image":"The__public_option__0_180.jpg","h264Media":"The__public_op tion__0_180.mp4","h264Ready":false,"hasChapters":false,"duration":522,"networ kId":1,"insertDate":"2009-07-28 17:28:22.0"}}...]
This request returns a randomly selected video’s meta-data such as: title, de- scription, duration and image per network Id.
The second service Vmoox Web Service provides is retrieving a videos' play list data. Client can submit a request by generating the following HTTP GET request: http://www.vmoox.com/web/video?videoId=VIDEO_ID
The response to this request will be a JSON format data as follow:
[{"video":{"id":11246,"title":"Quest's Profitable Moment-6\/ 11","image":"Quest_s_Profitable_Moment_6_11_0_72.jpg","h264Media":"Quest_ s_Profitable_Moment_6_11_0_72.mp4","h264Ready":false,"hasChapters":false," duration":72,"networkId":1,"insertDate":"2009-06-11 17:46:14.0","ads":{"com.vmoox.model.Ad":[{"id":5,"h264Media":"Sprint_Wireless_ Wedding.mp4","duration":30}]}}}]
This response also contains a pre-roll Ad that can be played before the publisher video plays.
(a) Implementing Get Network Videos Service
As defined in the web.xml configuration file specified above the class that imple- ments this service is JSONServiceImpl. This class extends Java's HttpServlet class and overrides the following method: void doGet(HttpServletRequest req, HttpServletResponse res) The class in the doGet method retrieves the request parameters map and de- cides which request is being made. If the the request parameters map contains
- 56 - the parameter networkId then this method calls the private List
- 57 - } videos = criteria.list(); if (key.equals("featured")) { // shuffle the result Collections.shuffle(videos); } } catch (HibernateException e) { } finally { session.close(); } return videos;
}
This is a generic method that all Vmoox's supported requests uses in order to ac- cess the DB and retrieve Videos objects. These requests include for example a get video by Id or get videos by a keyword in the video title or description.
As mentioned before the response that Vmoox Web Service outputs is in a JSON format and that was implemented using Xstream. In order to serialize the Video object to JSON one needs to create an instanse of the XStram object passing a JettisonMappedXmlDriver driver:
XStream xstream = new XStream(new JettisonMappedXmlDriver());
Then we need to set the right class alias: xstream.alias("video", Video.class); and in order to output the JSON format data structure we need to call the XS- tram's toXML method and write it to the request object:
PrintWriter out = res.getWriter(); StringBuilder jsonResult = new StringBuilder(); jsonResult.append(xstream.toXML(video.getToJSONObj())); out.print(jsonResult.toString());
This will generate a JSON format data structure like the example below.
- 58 - (b) Implementing Get List of Videos Media URLs for a Given Video Id
As defined in the web.xml configuration file specified above the class that imple- ments this service is JSONPlayListServiceImpl. This class extends Java's HttpServlet class and overrides the following method: void doGet(HttpServletRequest req, HttpServletResponse res) The class in the doGet method retrieves the request parameters map and de- cides which video Id is being request. Then the service needs to decide if all the video chapters are available or if a real time transcoding is necessary. The code that implement this logic was already discussed under the VEDA implementation in this chapter.
(c) Ad Serving
When Vmoox Web Service generates a list of videos' chapters for a given video, the service append to the video play list a video Ad. This logic is part of the get videos play list service request. The following method retrieve an Ad per three vides' chapters: private List
- 59 - 7. iPhone Native and Web App
The last part of the Vmoox system is a Front End app that delivers the transcod- ed content from Vmoox servers to the end users. As explained in the requirement and design sections of this paper users are not aware that not all videos are "ready" to be played, and an additonal transcoding in real time is needed. Users are able to browse the publisher video catalog and click on a video and instantly start watching it.
(a) Web Based App
The app was implementing using GWT and follows the MVC design pattern [49].
(i) The Model
The model objects of this app are the same objects as described in VPSI imple- mentation i.e., Video, VideoChapter and Ad objects in addition to a VideoTag ob- ject that help with generating a QuickTime HTML tag. Below you can see how the class generates this tag using the class properties: String posterUrl, String width, String height, String h264Uri, List
(ii) The Controller
The controller of this app provides the necessary data to the application views which then displays the featured videos and generates the QuickTime tag that lets users play the video public interface VmooxListener { public void getFeaturedVideos(); public void getVideoTag(int videoId); }
The controller users Vmoox Web Service in order to retrieve the data in JSON format, parse the data structure and generate a list of Videos that will be passed to the view in order to generate the user interface. Below is part of the controller implementation: public void asyncGetFeaturedVideos() { StringBuilder jsonUri = new StringBuilder(); jsonUri.append(GWT.getHostPageBaseURL() + "web/ videos?networkId=1"); jsonUri.append("&callback="); JSONRequest.get(jsonUri.toString(), new JSONRequestHandler() { public void onRequestComplete(JavaScriptObject json) { JSONObject jsonObj = new JSONObject(json); handleVideosData(jsonObj); } });
- 61 - }
This method builds the request parameters and generate an HTTP GET request to Vmoox Web Service. When a request is completed a new JSONObject instance is crated and a call to handleVideosData is being executed. This method then parse the JSONObject data structure and send the result to the View: for (int i = 0; i < 20; i++) { JSONValue v = jsonObject.get(new Integer(i).toString()); JSONObject collectionJSONObject = v.isObject(); JSONValue videoBeanJSONValue = null; videoBeanJSONValue = collectionJSONObject.get("video"); videoObj = videoBeanJSONValue.isObject(); String posterUrl = "h264/"+ (String) JSONValueMappingUtil.getJsonValue( videoObj, "image",JsonValueType.stringType); String title = (String) JSONValueMappingUtil .getJsonValue(videoObj, "title",JsonValueType.stringType); int duration = (Integer) JSONValueMappingUtil .getJsonValue(videoObj, "duration", JsonValueType.intType); int videoId = (Integer) JSONValueMappingUtil .getJsonValue(videoObj, "id",JsonValueType.intType); video = new Video(); video.setTitle(title); video.setImage(posterUrl); video.setDuration(duration); video.setId(videoId); videos.add(video); } (iii) The View
The app has three different views. This first view displays a randomly selected list of twenty videos for a given publisher. Each video section contains the video thumbnail, title and duration of the video.
- 62 - Figure 22: Featured Videos View
This view is being generated by FeaturedVideosView.java class. Below is the implementation of void loadFeaturedView(List
- 63 - public void loadFeaturedView(List
videoPanel.add(image); videoPanel.setCellWidth(image, "100px"); videoPanel.add(metaDataPanel); Image arrow = new Image("img/arrow_b.png"); arrow.addClickHandler(new ClickHandler(){ public void onClick(ClickEvent event) { loadVideoView(video); } }); arrow.setStyleName("arrow"); HorizontalPanel arrowPanel = new HorizontalPanel();
- 64 - arrowPanel.add(arrow); videoPanel.add(arrowPanel); hubRow1Panel.add(videoPanel); mainHubPanel.add(hubRow1Panel); } contents.add(mainHubPanel);
This method generates twenty horizontal panels each panel containe an image (the video thumbnail), two HTML objects (a GWT wrapper class of the HTML DIV element) and a blue arrow (another image) that when a user clicks on it the app loads the VideoView : arrow.addClickHandler(new ClickHandler(){ public void onClick(ClickEvent event) { loadVideoView(video); } });
The VideoView displays a QuickTime tag that contains a video poster (i.e., the same thumbnail image) and a playlist of videos. The view also includes two HTML elements that displays the video title and duration.
- 65 - Figure 23: Video View
This view is being generated by VideosView.java class. Below is the implementation of public void loadVideoView(VideoTag videoTag) This method as in input received a VideoTag object and build the view: public void loadVideoView(VideoTag videoTag){
- 66 - VerticalPanel vp = new VerticalPanel(); HorizontalPanel hubRow1Panel = new HorizontalPanel(); hubRow1Panel.setStyleName("videoPanel"); HorizontalPanel videoPanel = new HorizontalPanel(); VerticalPanel metaDataPanel = new VerticalPanel(); metaDataPanel.setStyleName("metaDataPanel"); HTML videoTitle = new HTML(video.getTitle()); videoTitle.setStyleName("videoTitle"); int durationInSeconds = video.getDuration(); int minutes = durationInSeconds/60; int seconds = durationInSeconds - (60*minutes); String duration = ""; if(seconds==0){ duration = new Integer(minutes).toString()+":00"; }else if(seconds<10){ duration = new Integer(minutes).toString()+":"+new Integer(seconds).toString()+"0"; }else{ duration = new Integer(minutes).toString()+":"+new Integer(seconds).toString(); } HTML durationDisplay = new HTML(duration); metaDataPanel.add(videoTitle); metaDataPanel.add(durationDisplay); videoPanel.add(videoTag); videoPanel.add(metaDataPanel); hubRow1Panel.add(videoPanel); mainHubPanel.add(hubRow1Panel); vp.add(mainHubPanel); contents.add(vp); }
The last view is the video play-list with pre-roll ad view. This view loads when the user clicks on the video thumbnail on the video view. The application uses the native media player build in the iPhone device, by generating a QuickTime tag. This tag allow the application to create a playlist of videos which in our cases in- cludes a pre-roll ad and also to display a video poster in our case the video thumbnail which VPSI originally generated.
- 67 - Figure 24: A Video is Being Played by the iPhone Native Media Player
- 68 - IX. Summary and Future Work
While the CNN native iPhone application works well as a prototype, some changes to the design of the Vmoox system will be made as individual publishers purchase the service. Each client has a unique set of requirements and both the application and service will be modified slightly in each case to fit those. One of the major components of this prototype is the division into 3 - minute chapters, a feature that may change per client needs and specifications. In this prototype the decision to break video files up into 3 - minute chapters comes from a desire avoid wasting server space and computation resources by only performing a full transcode on files which are actually viewed. The initially transcoded section provides the buffer with which to avoid any wait time on the user- end. Simply put, while the user watches the first 3 minutes of the clip, the rest of the clip is transcoding in the background. In order to improve efficiency even more, a future modification of the Vmoox algorithm will include a “stop transcode” feature which will identify when a user has stopped viewing a clip and instruct the system to cease transcoding any remaining chapters. This is significant because recent research indicates that the average duration of video consumption on mobile devices is 3.2 minutes [13]. Data collected by publishers about their individual audiences may impact the way the Vmoox service interacts with their content (i.e., CNN viewers might have a lower or higher average view time, calling for an adjustment to chapter lengths). We believe that customization is a key part of Vmoox’s potential for success. Though several transcoding solutions exist in the market, we have not yet found one that offers a customizable ‘one-stop-shop’ approach to content management, transcoding and delivery.
- 69 - X. References
[1] http://www.vmoox.com
[2] Scott Ruthfield. "The Internet's History and Development From Wartime Tool to the Fish-Cam". http://www.acm.org/crossroads/xrds2-1/inet-history.html
[3] Schneidawind, J: "Big Blue unveiling", USA Today, November 23, 1992, page 2B
[4] PC World Staff (PC World) 19/09/2007. "In Pictures: A History of Cell Phones". http://www.pcworld.idg.com.au/article/194761/ pictures_history_cell_phones?img=486&ssid=1
[5] Alana Semuels: "Smartphones now have one-third of market share globally, report says", March 24, 2009. http://latimesblogs.latimes.com/technology/ 2009/03/android-smartph.html
[6] SCOTTSDALE, Ariz: "Smartphones Will Double Their Share of the Handset Market by 2013", March 2, 2009. http://www.instat.com/ press.asp?ID=2465&sku=IN0804050WHFigure
[7] "The Ongoing Smartphone Revolution", March 23, 2009. http:/ /seekingalpha.com/article/127294-the-ongoing-smartphone-revolution
[8] Sarah Radwanick: "The 2008 Digital Year in Review", comScore Whitepaper, January 30, 2009. http://www.comscore.com/Press_Events/ Presentations_Whitepapers/2009/2008_Digital_Year_in_Review
[9] A2/M2 Three Screen Report: "Television Internet and Mobile Usage in the U.S.", 1st Quarter 2009. http://blog.nielsen.com/nielsenwire/wp-content/uploads/ 2009/05/nielsen_threescreenreport_q109.pdf
[10] "Evolution of Dance". http://www.youtube.com/browse?s=mp&t=a&c=0&l=
[11] Paul Thomasch: "Giants and Patriots draw record Super Bowl audience", February 4, 2008. http://www.reuters.com/article/topNews/ idUSN0420266320080204
[12] Nielsen "An Overview of Home Internet Access in the U.S.", December, 2008. http://blog.nielsen.com/nielsenwire/wp-content/uploads/
- 70 - 2009/03/overview-of-home-internet-access-in-the-us-jan-6.pdf
[13] "U.S. Online Video Viewing Surges 13 Percent in Record-Setting December ", comScore Press Release, February 4, 2009. http://www.comscore.com/ Press_Events/Press_Releases/2009/2/US_Online_Video_Viewing_Sets_Record
[14] Serge Jespers: "Flash Video market share continues to grow", February 5, 2009. http://www.webkitchen.be/2009/02/05/flash-video-marketshare-continues- to-grow/
[15] University Of Gottingen: "Approaches to Reduce Energy Consumption of WLAN Devices", September, 2004. http://whitepapers.techrepublic.com.com/ abstract.aspx?docid=175825
[16] iPhone Technical Specifications, August, 2009. http://www.apple.com/ iphone/specs.html
[17] Seronia media, August, 2009. http://www.sorensonmedia.com/video- delivery-network/
[18] Cameron Christoffers: "Encoding.com Relaunches as SaaS Video Encoding Platform", June 26, 2008. http://www.techcrunchit.com/2008/06/26/encodingcom- relaunches-as-saas-video-encoding-platform/
[19] Wikipedia: "Software as a service", August 2009. http://en.wikipedia.org/wiki/ Software_as_a_service
[20] ITU Corporate Strategy Newslog: "ITU Corporate Strategy Newslog", April 29, 2005. http://www.itu.int/osg/spu/newslog/ ITU+H264+The+Advanced+Video+Coding+Standard.aspx
[21] ITU-T H.264 The Advanced Video Coding Standard: "New opportunities for video communication". http://www.itu.int/itudoc/gs/promo/tsb/87066.pdf
[22] Wikipedia: "List of video services using H.26", August 2009. http:/ /en.wikipedia.org/wiki/Software_as_a_service
[23] QuickTime Technologies - H264: " H.264: Stunning clarity from 3G to HD.", August, 2009. http://www.apple.com/quicktime/technologies/h264/
[24] Erick Schonfeld: "iPhone Makes Up 50 Percent of Smartphone Web Traffic In U.S., Android Already 5 Percent", March 24, 2009. http://www.techcrunch.com/ 2009/03/24/iphone-now-50-percent-of-smartphone-web-traffic-in-the-us/
- 71 - [25] BlackBerry Storm Specification, August, 2009. http://na.blackberry.com/eng/ devices/blackberrystorm/storm_specifications.jsp
[26] What is Android, August, 2009. http://developer.android.com/guide/basics/ what-is-android.html
[27] Palm Pre: "Details", August, 2009. http://www.palm.com/us/products/phones/ pre/
[28] FFmpeg: "Project Description", August, 2009. http://ffmpeg.org/
[29] Wikipedia: "libavcodec", August, 2009. http://en.wikipedia.org/wiki/ Libavcodec
[30] FFmpeg, August, 2009. http://wiki.videolan.org/Ffmpeg
[31] VideoLan: "Streaming", August, 2009. http://www.videolan.org/vlc/ streaming.html
[32] Perian: "The swiss-army knife for QuickTime", August, 2009. http:/ /perian.org/
[33] Bennett, Keith; et al: "Service-based software: the future for flexible software" (PDF), December, 2000. Seventh Asia-Pacific Software Engineering Conference, 2000. APSEC 2000.. Proceedings: Seventh Asia-Pacific Software Engineering.
[34] Sahil Parikh: "Going SaaS? The pros & cons", September 17, 2008. http:/ /www.saasburst.com/tag/saas-advantages/#http://www.saasburst.com/tag/saas- advantages/
[35] Traudt, Erin; Amy Konary: "2005 Software as a Service Taxonomy and Research Guide 7", June, 2005 IDC.
[36] Xstream: "About Xstream", August, 2009. http://xstream.codehaus.org/
[37] Wikipedia: "Representational State Transfer (REST) ", August, 2009. http:/ /en.wikipedia.org/wiki/Representational_State_Transfer
[38] Hibernate: "Relational Persistence for Java and .NET", August, 2009. https:/ /www.hibernate.org/
[39] Wikipedia: "Coupling (computer science)", August 2009. http:/
- 72 - /en.wikipedia.org/wiki/Coupling_(computer_science)
[40] Joanne M. Atlee: "Software Engineering: Theory and Practice, Third Edition"
[41] Lawrence Wilkes: "ROI - The Costs and Benefits of Web Services and Service Oriented Architecture". http://roadmap.cbdiforum.com/reports/roi/
[42] Regina Lynch: "Article: JSON for Ajax Web services", October 19, 2006. http://www.theserverside.com/news/thread.tss?thread_id=42722
[43] json-framework: "A strict JSON parser/generator for Objective-C", August, 2009. http://code.google.com/p/json-framework/
[44] CornerWorld: "Online Video Advertising Will Gain Considerable Market Share", April 26, 2009. http://cornerworld.com/online-video-advertising-will-gain- considerable-market-share/
[45] MySql: "The world's most popular open source database", August, 2009. http://www.mysql.com/
[46] GWT: "Write AJAX apps in the Java language, then compile to optimized JavaScript", August, 2009. http://code.google.com/webtoolkit/overview.html
[47] iPhone SDK 3.0: "Developing for iPhone OS 3.0", August, 2009. http:/ /developer.apple.com/iphone/
[48] David Geary: "Simply Singleton, Navigate the deceptively simple Singleton pattern", April 25, 2003. http://www.javaworld.com/javaworld/jw-04-2003/jw-0425- designpatterns.html
[49] MVC: "Model-View-Controller", August, 2009. http://java.sun.com/blueprints/ patterns/MVC.html
- 73 -