Perceptually-Driven Video Coding with the Daala Video Codec
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Network Working Group A
Network Working Group A. Filippov Internet Draft Huawei Technologies Intended status: Informational A. Norkin Netflix J.R. Alvarez Huawei Technologies Expires: May 17, 2017 November 17, 2016 <Video Codec Requirements and Evaluation Methodology> draft-ietf-netvc-requirements-04.txt Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. The list of current Internet-Drafts can be accessed at http://datatracker.ietf.org/drafts/current/ Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on May 17, 2017. Copyright Notice Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust’s Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with <Filippov> Expires May 17, 2017 [Page 1] Internet-Draft Video Codec Requirements and Evaluation November 2016 respect to this document. -
C-Based Hardware Design of Imdct Accelerator for Ogg Vorbis Decoder
C-BASED HARDWARE DESIGN OF IMDCT ACCELERATOR FOR OGG VORBIS DECODER Shinichi Maeta1, Atsushi Kosaka1, Akihisa Yamada1, 2, Takao Onoye1, Tohru Chiba1, 2, and Isao Shirakawa1 1Department of Information Systems Engineering, 2Sharp Corporation Graduate School of Information Science and Technology, 2613-1 Ichinomoto, Tenri, Nara, 632-8567 Japan Osaka University phone: +81 743 65 2531, fax: +81 743 65 3963, 2-1 Yamada-oka, Suita, Osaka, 565-0871 Japan email: [email protected], phone: +81 6 6879 7808, fax: +81 6 6875 5902, [email protected] email: {maeta, kosaka, onoye, sirakawa}@ist.osaka-u.ac.jp ABSTRACT ARM7TDMI is used as the embedded processor since it has This paper presents hardware design of an IMDCT accelera- come into wide use recently. tor for an Ogg Vorbis decoder using a C-based design sys- tem. Low power implementation of audio codec is important 2. OGG VORBIS CODEC in order to achieve long battery life of portable audio de- 2.1 Ogg Vorbis Overview vices. Through the computational cost analysis of the whole decoding process, it is found that Ogg Vorbis requires higher Figure 1 shows a block diagram of the Ogg Vorbis codec operation frequency of an embedded processor than MPEG processes outlined below. Audio. In order to reduce the CPU load, an accelerator is designed as specific hardware for IMDCT, which is detected MDCT Psycho Audio Remove Channel Acoustic VQ as the most computation-intensive functional block. Real- Signal Floor Coupling time decoding of Ogg Vorbis is achieved with the accelera- FFT Model Ogg Vorbis tor and an embedded processor both run at 36MHz. -
Blackberry QNX Multimedia Suite
PRODUCT BRIEF QNX Multimedia Suite The QNX Multimedia Suite is a comprehensive collection of media technology that has evolved over the years to keep pace with the latest media requirements of current-day embedded systems. Proven in tens of millions of automotive infotainment head units, the suite enables media-rich, high-quality playback, encoding and streaming of audio and video content. The multimedia suite comprises a modular, highly-scalable architecture that enables building high value, customized solutions that range from simple media players to networked systems in the car. The suite is optimized to leverage system-on-chip (SoC) video acceleration, in addition to supporting OpenMAX AL, an industry open standard API for application-level access to a device’s audio, video and imaging capabilities. Overview Consumer’s demand for multimedia has fueled an anywhere- o QNX SDK for Smartphone Connectivity (with support for Apple anytime paradigm, making multimedia ubiquitous in embedded CarPlay and Android Auto) systems. More and more embedded applications have require- o Qt distributions for QNX SDP 7 ments for audio, video and communication processing capabilities. For example, an infotainment system’s media player enables o QNX CAR Platform for Infotainment playback of content, stored either on-board or accessed from an • Support for a variety of external media stores external drive, mobile device or streamed over IP via a browser. Increasingly, these systems also have streaming requirements for Features at a Glance distributing content across a network, for instance from a head Multimedia Playback unit to the digital instrument cluster or rear seat entertainment units. Multimedia is also becoming pervasive in other markets, • Software-based audio CODECs such as medical, industrial, and whitegoods where user interfaces • Hardware accelerated video CODECs are increasingly providing users with a rich media experience. -
Detail Streaming Support Protocols
Encore+ User Guide Detail Streaming Support Protocols Supported Audio Codecs Supported Container Formats • MP3 • WAV • AAC • M4A • FLAC • OGG • LPCM/WAV/AIFF • AIFF • ALAC Supported Protocols • WMA, WMA9 • SHOUTcast • Ogg Vorbis • HTTPS Supported Playlist • WMA streaming • ASX • RTSP/SDP • M3U • PLS • WPL 43 Detail Audio Codec Support Encore+ User Guide Supported MP3 encoding parameters • Sampling rates [kHz]: 32, 44.1, 48 • Resolution [bits]: 16 • Bit rate [kbps]: 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, VBR • Channels: stereo, joined stereo, mono • MP3PRO playback • MP3 File extensions: *.mp3 • Decoding of ID3v1, ID3v2, MP3 ID tags including optional album art in .jpeg format up to 2 megapixels • Gapless MP3: Playback is gapless if the container provides LAME encoder delay and padding tags. Supported Vorbis encoding parameters • Sampling rates [kHz]: 32, 44.1, 48 • Resolution [bits]: 16 • Nominal bit rate [kbps] (quality level): 80 (Q1), 96 (Q2), 112 (Q3), 128 (Q4), 160 (Q5), 192 (Q6), • Channels: stereo • The audio player supports reading of Vorbis content stored in Ogg containers. Supported file name extensions: *.ogg and *.oga. • The audio player supports decoding of Vorbis comments. NOTE: There is no specification for tag names. The system relies on the OSS implementation. • Tag names decoded: TITLE, ALBUM, ARTIST, GENRE. • Binary data (e.g. for album art) is not supported. • The audio player supports gapless Vorbis playback. Supported FLAC encoding parameters • Sampling rates [kHz]: 44.1, 48, 88.2, 96, 176.4, 192 • Resolution [bits]: 16, 24 • Channels: stereo, mono • The audio player supports reading of FLAC content stored in native FLAC containers. -
DVP Tutorial
IP Video Conferencing: A Tutorial Roman Sorokin, Jean-Louis Rougier Abstract Video conferencing is a well-established area of communications, which have been studied for decades. Recently this area has received a new impulse due to significantly increased bandwidth of Local and Wide area networks, appearance of low-priced video equipment and development of web based media technologies. This paper presents the main techniques behind the modern IP-based videoconferencing services, with a particular focus on codecs, network protocols, architectures and standardization efforts. Questions of security and topologies are also tackled. A description of a typical video conference scenario is provided, demonstrating how the technologies, responsible for different conference aspects, are working together. Traditional industrial disposition as well as modern innovative approaches are both addressed. Current industry trends are highlighted in respect to the topics, described in the tutorial. Legacy analog/digital technologies, together with the gateways between the traditional and the IP videoconferencing systems, are not considered. Keywords Video Conferencing, codec, SVC, MCU, SIP, RTP Roman Sorokin ALE International, Colombes, France e-mail: [email protected] Jean-Louis Rougier Télécom ParisTech, Paris, France e-mail: [email protected] 1 1 Introduction Video conferencing is a two-way interactive communication, delivered over networks of different nature, which allows people from several locations to participate in a meeting. Conference participants use video conferencing endpoints of different types. Generally a video conference endpoint has a camera and a microphone. The video stream, generated by the camera, and the audio stream, coming from the microphone, are both compressed and sent to the network interface. -
Ogg Audio Codec Download
Ogg audio codec download click here to download To obtain the source code, please see the xiph download page. To get set up to listen to Ogg Vorbis music, begin by selecting your operating system above. Check out the latest royalty-free audio codec from Xiph. To obtain the source code, please see the xiph download page. Ogg Vorbis is Vorbis is everywhere! Download music Music sites Donate today. Get Set Up To Listen: Windows. Playback: These DirectShow filters will let you play your Ogg Vorbis files in Windows Media Player, and other OggDropXPd: A graphical encoder for Vorbis. Download Ogg Vorbis Ogg Vorbis is a lossy audio codec which allows you to create and play Ogg Vorbis files using the command-line. The following end-user download links are provided for convenience: The www.doorway.ru DirectShow filters support playing of files encoded with Vorbis, Speex, Ogg Codecs for Windows, version , ; project page - for other. Vorbis Banner Xiph Banner. In our effort to bring Ogg: Media container. This is our native format and the recommended container for all Xiph codecs. Easy, fast, no torrents, no waiting, no surveys, % free, working www.doorway.ru Free Download Ogg Vorbis ACM Codec - A new audio compression codec. Ogg Codecs is a set of encoders and deocoders for Ogg Vorbis, Speex, Theora and FLAC. Once installed you will be able to play Vorbis. Ogg Vorbis MSACM Codec was added to www.doorway.ru by Bjarne (). Type: Freeware. Updated: Audiotags: , 0x Used to play digital music, such as MP3, VQF, AAC, and other digital audio formats. -
Opus, a Free, High-Quality Speech and Audio Codec
Opus, a free, high-quality speech and audio codec Jean-Marc Valin, Koen Vos, Timothy B. Terriberry, Gregory Maxwell 29 January 2014 Xiph.Org & Mozilla What is Opus? ● New highly-flexible speech and audio codec – Works for most audio applications ● Completely free – Royalty-free licensing – Open-source implementation ● IETF RFC 6716 (Sep. 2012) Xiph.Org & Mozilla Why a New Audio Codec? http://xkcd.com/927/ http://imgs.xkcd.com/comics/standards.png Xiph.Org & Mozilla Why Should You Care? ● Best-in-class performance within a wide range of bitrates and applications ● Adaptability to varying network conditions ● Will be deployed as part of WebRTC ● No licensing costs ● No incompatible flavours Xiph.Org & Mozilla History ● Jan. 2007: SILK project started at Skype ● Nov. 2007: CELT project started ● Mar. 2009: Skype asks IETF to create a WG ● Feb. 2010: WG created ● Jul. 2010: First prototype of SILK+CELT codec ● Dec 2011: Opus surpasses Vorbis and AAC ● Sep. 2012: Opus becomes RFC 6716 ● Dec. 2013: Version 1.1 of libopus released Xiph.Org & Mozilla Applications and Standards (2010) Application Codec VoIP with PSTN AMR-NB Wideband VoIP/videoconference AMR-WB High-quality videoconference G.719 Low-bitrate music streaming HE-AAC High-quality music streaming AAC-LC Low-delay broadcast AAC-ELD Network music performance Xiph.Org & Mozilla Applications and Standards (2013) Application Codec VoIP with PSTN Opus Wideband VoIP/videoconference Opus High-quality videoconference Opus Low-bitrate music streaming Opus High-quality music streaming Opus Low-delay -
High Efficiency, Moderate Complexity Video Codec Using Only RF IPR
Thor High Efficiency, Moderate Complexity Video Codec using only RF IPR draft-fuldseth-netvc-thor-00 Arild Fuldseth, Gisle Bjontegaard (Cisco) IETF 93 – Prague, CZ – July 2015 1 Design principles • Moderate complexity to allow real-time implementation in SW on common HW, as well as new HW designs • Basic building blocks from well-known hybrid approach (motion compensated prediction and transform coding) • Common design elements in modern codecs – Larger block sizes and transforms, up to 64x64 – Quarter pixel interpolation, motion vector prediction, etc. • Cisco RF IPR (note well: declaration filed on draft) – Deblocking, transforms, etc. (some also essential in H.265/4) • Avoid non-RF IPR – If/when others offer RF IPR, design/performance will improve 2 Encoder Architecture Input Transform Quantizer Entropy Output video Coding bitstream - Inverse Transform Intra Frame Prediction Loop filters Inter Frame Prediction Reconstructed Motion Frame Estimation Memory 3 Decoder Architecture Input Entropy Inverse Bitstream Decoding Transform Intra Frame Prediction Loop filters Inter Frame Prediction Output video Reconstructed Frame Memory 4 Block Structure • Super block (SB) 64x64 • Quad-tree split into coding blocks (CB) >= 8x8 • Multiple prediction blocks (PB) per CB • Intra: 1 PB per CB • Inter: 1, 2 (rectangular) or 4 (square) PBs per CB • 1 or 4 transform blocks (TB) per CB 5 Coding-block modes • Intra • Inter0 MV index, no residual information • Inter1 MV index, residual information • Inter2 Explicit motion vector information, residual information -
Encoding H.264 Video for Streaming and Progressive Download
W4: KEY ENCODING SKILLS, TECHNOLOGIES TECHNIQUES STREAMING MEDIA EAST - 2019 Jan Ozer www.streaminglearningcenter.com [email protected]/ 276-235-8542 @janozer Agenda • Introduction • Lesson 5: How to build encoding • Lesson 1: Delivering to Computers, ladder with objective quality metrics Mobile, OTT, and Smart TVs • Lesson 6: Current status of CMAF • Lesson 2: Codec review • Lesson 7: Delivering with dynamic • Lesson 3: Delivering HEVC over and static packaging HLS • Lesson 4: Per-title encoding Lesson 1: Delivering to Computers, Mobile, OTT, and Smart TVs • Computers • Mobile • OTT • Smart TVs Choosing an ABR Format for Computers • Can be DASH or HLS • Factors • Off-the-shelf player vendor (JW Player, Bitmovin, THEOPlayer, etc.) • Encoding/transcoding vendor Choosing an ABR Format for iOS • Native support (playback in the browser) • HTTP Live Streaming • Playback via an app • Any, including DASH, Smooth, HDS or RTMP Dynamic Streaming iOS Media Support Native App Codecs H.264 (High, Level 4.2), HEVC Any (Main10, Level 5 high) ABR formats HLS Any DRM FairPlay Any Captions CEA-608/708, WebVTT, IMSC1 Any HDR HDR10, DolbyVision ? http://bit.ly/hls_spec_2017 iOS Encoding Ladders H.264 HEVC http://bit.ly/hls_spec_2017 HEVC Hardware Support - iOS 3 % bit.ly/mobile_HEVC http://bit.ly/glob_med_2019 Android: Codec and ABR Format Support Codecs ABR VP8 (2.3+) • Multiple codecs and ABR H.264 (3+) HLS (3+) technologies • Serious cautions about HLS • DASH now close to 97% • HEVC VP9 (4.4+) DASH 4.4+ Via MSE • Main Profile Level 3 – mobile HEVC (5+) -
Game Audio the Role of Audio in Games
the gamedesigninitiative at cornell university Lecture 18 Game Audio The Role of Audio in Games Engagement Entertains the player Music/Soundtrack Enhances the realism Sound effects Establishes atmosphere Ambient sounds Other reasons? the gamedesigninitiative 2 Game Audio at cornell university The Role of Audio in Games Feedback Indicate off-screen action Indicate player should move Highlight on-screen action Call attention to an NPC Increase reaction time Players react to sound faster Other reasons? the gamedesigninitiative 3 Game Audio at cornell university History of Sound in Games Basic Sounds • Arcade games • Early handhelds • Early consoles the gamedesigninitiative 4 Game Audio at cornell university Early Sounds: Wizard of Wor the gamedesigninitiative 5 Game Audio at cornell university History of Sound in Games Recorded Basic Sound Sounds Samples Sample = pre-recorded audio • Arcade games • Starts w/ MIDI • Early handhelds • 5th generation • Early consoles (Playstation) • Early PCs the gamedesigninitiative 6 Game Audio at cornell university History of Sound in Games Recorded Some Basic Sound Variability Sounds Samples of Samples • Arcade games • Starts w/ MIDI • Sample selection • Early handhelds • 5th generation • Volume • Early consoles (Playstation) • Pitch • Early PCs • Stereo pan the gamedesigninitiative 7 Game Audio at cornell university History of Sound in Games Recorded Some More Basic Sound Variability Variability Sounds Samples of Samples of Samples • Arcade games • Starts w/ MIDI • Sample selection • Multiple -
The Daala Video Codec Project Next-Next Generation Video
The Daala Video Codec Project Next-next Generation Video Timothy B. Terriberry Mozilla & The Xiph.Org Foundation ● Patents are no longer a problem for free software – We can all go home 2 Mozilla & The Xiph.Org Foundation ● Except... not quite 3 Mozilla & The Xiph.Org Foundation Carving out Exceptions in OIN (Table 0 contains one Xiph codec: FLAC) 4 Mozilla & The Xiph.Org Foundation Why This Matters ● Encumbered codecs are a billion dollar toll-tax on communications – Every cost from codecs is repeated a million fold in all multimedia software ● Codec licensing is anti-competitive – Licensing regimes are universally discriminatory – An excuse for proprietary software (Flash) ● Ignoring licensing creates risks that can show up at any time – A tax on success 5 Mozilla & The Xiph.Org Foundation The Royalty-Free Video Challenge ● Creating good codecs is hard – But we don’t need many – The best implementations of patented codecs are already free software ● Network effects decide – Where RF is established, non-free codecs see no adoption (JPEG, PNG, FLAC, …) ● RF is not enough – People care about different things – Must be better on all fronts 6 Mozilla & The Xiph.Org Foundation We Did This for Audio 7 Mozilla & The Xiph.Org Foundation The Daala Project ● Goal: Better than HEVC without infringing IPR ● Need a better strategy than “read a lot of patents” – People don’t believe you – Analysis is error-prone ● Try to stay far away from the line, but... ● One mistake can ruin years of development effort ● See: H.264 Baseline 8 Mozilla & The Xiph.Org -
IP-Soc Shanghai 2017 ALLEGRO Presentation FINAL
Building an Area-optimized Multi-format Video Encoder IP Tomi Jalonen VP Sales www.allegrodvt.com Allegro DVT Founded in 2003 Privately owned, based in Grenoble (France) Two product lines: 1) Industry de-facto standard video compliance streams Decoder syntax, performance and error resilience streams for H.264|MVC, H.265/SHVC, VP9, AVS2 and AV1 System compliance streams 2) Leading semiconductor video IP Multi-format encoder IP for H.264, H.265, VP9, JPEG Multi-format decoder IP for H.264, H.265, VP9, JPEG WiGig IEEE 802.11ad WDE CODEC IP 2 Evolution of Video Coding Standards International standards defined by standardization bodies such as ITU-T and ISO/IEC H.261 (1990) MPEG-1 (1993) H.262 / MPEG-2 (1995) H.263 (1996) MPEG-4 Part 2 (1999) H.264 / AVC / MPEG-4 Part 10 (2003) H.265 / HEVC (2013) Future Video Coding (“FVC”) MPEG and ISO "Preliminary Joint Call for Evidence on Video Compression with Capability beyond HEVC.” (202?) Incremental improvements of transform-based & motion- compensated hybrid video coding schemes to meet the ever increasing resolution and frame rate requirements 3 Regional Video Standards SMPTE standards in the US VC-1 (2006) VC-2 (2008) China Information Industry Department standards AVS (2005) AVS+ (2012) AVS2.0 (2016) 4 Proprietary Video Formats Sorenson Spark On2 VP6, VP7 RealVideo DivX Popular in the past partly due to technical merits but mainly due to more suitable licensing schemes to a given application than standard video video formats with their patent royalties. 5 Royalty-free Video Formats Xiph.org Foundation