sign in sign up
<<
Home , Speex, WavPack

Solving the puzzle of multimedia on or how to handle multimedia on linux

Muriel Moscardini

Fluendo SA Agenda

1. Introduction 2. Multimedia 3. Linux as a solution 4. Stakes of multimedia on linux 5. What solutions do we have 6. In practice… 7. Market solutions 8. Conclusion 1. Introduction

 A talk for all, Distributors and Users, companies, IT directors, device makers, software vendors, developers, product managers,…to understand how to cope with patented multimedia formats

 Not a talk about legitimacy of patent on software but solutions to live with it

 Here to inform, each being responsible for taking his own decisions

 Share experience, give tips, explain, make it clearer

 The talk is open to all the feedback you may have to share.

2 - Multimedia 2.1. Multimedia everywhere 2.2. Market Figures Multimedia Consumption : A Vibrant Market

Among all VOD and Digital Downloads online consumers, Streaming 2010/2011 + On-line subscription

+31%

-11%

DVD and BluRay rentals 70% watch online video* 2010/2011

$8 billion by 2014*

*source Nielsen *source Digital Media Post *source PricewaterhouseCooper's 2.3 -Multimedia in the enterprise

EXTERNAL CONTENT

On line or not Streamed On demand or Live

Content Multimedia content Delivery Network Internal or CONSUMPTION external IN HOUSE PRODUCTION E learning

Recording Training Business content Recording events All company divisions Marketing productions Social share Corporate webcasts Multimedia is everywhere we need to care. It is important for your “customers” whether you sell them devices or software, or whether they are your employees

3 – Linux as a solution 3.1 Why Linux is a good answer? For device makers and enterprises  Low cost  Ease of customization  Very large choice of projects available, high potential of differentiation  Stable kernel  Ported to numerous platforms  Innovation stimulated by the open community  Very large ecosystem of software development company providing support and project services  Allow a mix between free and proprietary solutions to take advantage of both

3.2 Solution recognized for Embedded and for final users

 Linux has been chosen for Embedded solutions a long time ago and drove the innovation, we even didn’t know.

 Final users acceptance has taken longer time, but thanks to new types of man-machine interfaces (smartphone, tablets) Linux based OS have become a solution for millions of people. 3.3 Challenges of Linux in the enterprise

Proprietary OS Linux distribution

- Pay by unit - Free, and “a la carte” set up of - All included paying apps - No or few custom - Large functional scope available - Used for years - Free and Proprietary applications - Usage habits available - Pay for upgrades - Diverse - Ecosystem of proprietary solutions - New - Paying support services available

 Ensure users acceptation  Maintain efficiency  Maintain scope coverage

 Multimedia may be a major gap

4 – Stakes of multimedia on linux 4.1 Generic model

Multimedia Application Free or proprietary license Multimedia Framework open: Gstreamer, VLC, Phonon,

Free license

A/V Codecs Free Formats Patented Linux OS (Fedora, Suse, Mandriva, Ubuntu, customer OS,…) Formats 4.2 Why it can become a constraint?

• Some countries recognize these patents • Ie. US, Germany recognize patent on software

• Use/Distribute patented format would require to be patent licensee

• Many of the patent licenses are incompatible with most licenses so it may be incompatible to distribute both together. • Ie. Software codec under a GPL license together with a patent license

4.3 Licenses incompatibility • Some terms in patent licenses are incompatible with some terms in open licenses*

• Mainly the fact that open source (free software) licenses require to give the right to use, copy, modify, distribute

• Patent licenses often forbid copy, reverse engineering, release source code and often bear per-copy running royalties and therefore request tracking

• The main impact may be to be obliged to get some software with compatible license (restricted rights – not GPL)

*ref. Section 11 of the GNU LGPL (version 2) 4.4 Why we didn’t care before?

BEFORE

• Use of proprietary OS • Pay for license • All in one • We didn’t know or look for what was included in the price • Patent where included in the license price when the format was supported • If not we were even not used to wonder why • Deal as a black box

4.5 Now what has changed?

NOW, what is changing with Linux

• We build and choose the modules of our solution, no more a black box • we don’t pay upfront, nothing patented is included a priori • If we decide to include an open and free project, there is no business model (specific for companies or resellers) allowing you to distribute these pieces of software (patented codecs) in an obvious way

4.6 Why should we care?

• Linux concerns millions of people

• Linux is growing and interest is also growing

• In the patent world, Patent holder claim infringment if not licensed

• Now patent holders care about linux

4.7 Why it impacts companies?  In countries where patents on software are recognized, infringing the patent can lead to negative legal consequences

• Companies distributing thousands of units may be concerned about being sued

• Companies who have patents themselves may be concerned about respecting patent licenses

• Enterprises deploying thousand of workstations /desktops may not want to endorse responsibility

4.8 First level of impact When choosing a solution, implementing a sofware, distributing it

 Pay more attention and analyse what we use, distribute, bundle on a licensing point of view

 Find and accept new ways to build the solutions to comply licenses, accept to make compromises . Ie. Proprietary mix with OS

 Be Geography sensible 4.9 Second level of impact

 The solution may not be completely free anymore

• Be patent license compliant often means - be licensee, sign agreements - may imply some royatly to be paid to the patent holder, - tracking units, tracking distribution/use - may imply choices of specific software to comply with the patent license terms

 Require new skills and help from experts in licensing

5 – What solutions do we have? 5.1 Use free formats

 Safer way is to use the free formats available (open and free of patent and copyright)

 When you can control the source of content, it is the best option, with no constraints Examples of free formats

Audio codec ALAC— lossless , previously a proprietary - video format of Apple Inc. PNG — a raster image format standardized by ISO/IEC CMML — timed metadata and subtitles SMIL — a media playlisting format and multimedia DAISY Digital Talking Book— a talking book format integration language FLAC — lossless audio codec SVG — a vector image format standardized by W3C — an audio codec VRML/X3D — realtime 3D data formats standardized ADPCM audio by ISO/IEC LPCM audio WebM — a video/audio format ILBC audio JPEG 2000— an image format standardized by ISO/IEC XSPF — a playlist format for multimedia DASH demuxer, for adaptive streaming — speech codec HLS demuxer for adaptive streaming WavPack — "Hybrid" (lossless/lossy) audio codec MNG — moving pictures, based on PNG

Containers — container for , FLAC, Speex and Opux (mkv) — container for all type of multimedia (audio formats) & Theora formats (audio, video, images, subtitles)

5.2 Don’t provide directly the support for patented formats  Bundle and resell: choose not to provide the support directly but rely upon the fact the final user can get it: integrate some “plug-in” management program, where the user will be directed to a source providing solutions to get them, or rely upon source of content providing their own (direct user dowload)  Company: block the content of any source that you don’t cover, and rely upon content that don’t need patent license to be read 5.3 Use implementation with patent license

• As it is not always possible to choose the formats to be supported depending on the source of content you want to support.

• In this case, the approach is to use proprietary implementations and and get the license for the patent.

 Bundle with software under compatible licenses 5.3 Use implementation with patent license

•User satisfaction (internal/external) •Cope with laws in country of Distribution or use •Reduce patent infrigment risk and law pursuit

Codec and other multimedia application from the market + = answering licensing requirements READY Patent licenses... TO SHIP/USE PRODUCT Some of the patented formats

Format Known patent licensing entities/ patent holders

Windows Media Audio Media Video Microsoft Windows Media network protocol Microsoft Smooth Streaming demuxer Microsoft H264 /MPEG4part10/AVC) video MPEGLA MPEG2 video MPEGLA MPEG4part2 video MPEGLA AAC/HEAAC audio Via Licensing (Dolby) DD/AC3 audio Dolby DD+/EAC3 audio Dolby TrueHD audio Dolby Dolby Headphones audio Dolby Pulse audio Dolby AMR NB WB audio Voice Age MP3/MPEG1LayerIII audio Thomson H265 video Not yet defined SILK audio Skype DTS audio DTS RMV video Real networks RMA audio Real networks Containers ASF Microsoft 3GPP 3GPP 5.4 What if we deal with hardware codecs

•To cope with multimedia requirements, harware manufacturers more and more include the encoding(decoding directly in the platform

•They may be implementation licensee but will never be final products

•Ie. If you resell an APP considering you will use the hardware decoder of the platform, you are responsible

7 – In practice… 7.1 Generic matters when you start Define the need and scope: - for device makers define the target and identify the multimedia consumption behavior of your target - for companies identify the need for each category of worker and ensure where multimedia must be supported - choose the OS/distro the most adapted to your business Define the technical configuration - define the multimedia framework to be used and list players available - list the codecs needed to be supported 7.2 Define if you are eligible

 Main patent licensors will require the final OEM or final user to be the licensee.

 Some patent licensors will provide 2 levels of license - One for implementation: when you provide the software (codec) - One for distribution

 You need to identify where you are in the chain and what you provide 7.3 How does that work? Step 1 1. You identify your needs and possible providers

- You define the codec list and identify possible vendors of implementation

- For each format you identify known licensors. Nobody will never commit to an exhaustive list of patent holders, as some may come up and nobody knows it. Be sure to identify the ones that count for you. 7.3 How does that work? Step 2 2. You get the codec under the appropriate license  You buy the implementation from a codec vendor  You develop or make develop for you - Some codecs may require to acquire a specific license for implementation different from the one for use or distribute - Some may be developped following public specs even if it needs then a license to be distributed  You buy from codec licensors - Some licensors provide their own implementation

7.3 How does that work? Step3

3. You get the appropriate patent license

 Main patent licensors will require you first to define - The device: hand held, Desktop like, embedded - The target : internal use, bundle and resell, ODM, OEM, Software vendor,… - For the entire system or for only one application?

 Depending on this you may have different licenses. Be careful you may have to sign different licenses if you want to cover different use 7.4 Codec implementation Business model

 It completely depends on the software vendors  It can be a fix fee or a per unit fee, or even a buy out  It will often come as a normal , with an offer on support services  It will mainly be binary

7.5 Patent license Business model

 It is usually based upon a per quarter / semester / year royalty reporting per unit (reporting and payment after sales)  Some propose % of revenue generated on a service (for profesionnal application / streaming servers,…)  Some propose annual caps or volume discount (ie. MPEGLA) that can be paid upfront

7.4 Patent license costs

 It depends on Formats  Sometimes it depends on the use Systeme wide or per application (ie: Via licensing)  It globally goes from $0,10 to $2 per unit  Price per unit is usually a price per machine or per application  Price may differ depending on the type of device (ie: Desktop like, hand held, ...) 7.4 Patent license costs

 Some require up front fee, or minimum fee per year whatever the units sold  Some propose a one year fee for one application for PC software (ie: Via licensing)  Some propose free units for a certain volume 7.6 What process to follow?

STEP 1 DEFINE THE SCOPE Device makers Enterprise

Target users Market target Employees

Users behaviour Mobile, home, work, Work, which division, perso,…. perso too?

HDW/OS/Framework/Player Cross platform? Linux distro by default? used Hardare codecs?

Source of content Depend on market May depend on division

Features supported Decode? Encode? Decode? Encode?

Formats needed User consumption User consumption fixed depend on known by the company sources Quantity Distribution volume Number of users forecast Country Distribution Sites (if multinational) 7.6 What process to follow?

STEP 2 TESTING Device makers Enterprise Evaluation Testing in target Test or buy trial environment versions

STEP3 FORMALIZE

Product license Reseller agreement Site licensing to bundle and agreement for use distribute

Or pay for development Business model options Per unit royalty fee One time fee license + Annual fee support agreement Buy out

8 – Market solutions Solution providers

 License a full proprietary solution (application): Corel, Cyberlink, Realnetworks,

 Get proprietary codecs implementations: Fluendo, Corecodec, Rovi (main concept), , Packet video, Dolby, DTS, H264 licensing, Enthropy wave,… + Get patent licenses: Microsoft, MPEGLA, Via licensing, Dolby, DTS, VoicAge,Technicolor,…

 Get a packaged product one-stop-shop on Gstreamer: Fluendo

Example: Gstreamer

Stable, supported

Video editors Streaming server Media Players Modular (A/V, DVD,…) Gstreamer Video Streaming, Adaptive Codecs, conferencing streaming, containers, Cross platform muxers, demuxers Post processing, Rendering Specific elements HDW acceleration audio/video (Gstreamer SDK) edition

Operating system Wide coverage Hardware platform Widely used

LGPL V2 Gstreamer.net and gstreamer.com Conclusion  Linux growth introduces specific consideration of multimedia  When talking about legal issues on linux, we often open Pandora’s box, but it should not be a big deal, and solutions can be found.  Now solutions are available meanwhile things on pure legal side may take time to evolve.  The most important may be as usual to know what we do, to rely upon experts who exist, and to be able to measure the risks we take.

 To feel comfortable, work with lawyers and developers and believe the ecosystem we are part of, brings value, and helps also make things evolve enabling all types of needed configurations. So get in touch with the appropriate partner and you will find solutions.

Contact

Muriel Paumier-Moscardini [email protected]

Skype: muriel.moscardini

Download slides on http://events.linuxfoundation.org/events/collaboration-summit/slides