Learning AV Foundation Addison-Wesley Learning Series
Total Page：16
File Type：pdf, Size：1020Kb
Learning AV Foundation Addison-Wesley Learning Series Visit informit.com/learningseries for a complete list of available publications. The Addison-Wesley Learning Series is a collection of hands-on programming guides that help you quickly learn a new technology or language so you can apply what you’ve learned right away. Each title comes with sample code for the application or applications built in the text. This code is fully annotated and can be reused in your own projects with no strings attached. Many chapters end with a series of exercises to encourage you to reexamine what you have just learned, and to tweak or adjust the code as a way of learning. Titles in this series take a simple approach: they get you going right away and leave you with the ability to walk off and build your own application and apply the language or technology to whatever you are working on. Learning AV Foundation A Hands-on Guide to Mastering the AV Foundation Framework Bob McCune Upper Saddle River, NJ • Boston • Indianapolis • San Francisco New York • Toronto • Montreal • London • Munich • Paris • Madrid Cape Town • Sydney • Tokyo • Singapore • Mexico City Many of the designations used by manufacturers and sellers to distinguish their products Editor-in-Chief are claimed as trademarks. Where those designations appear in this book, and the Mark Taub publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. Senior Acquisitions Editor The author and publisher have taken care in the preparation of this book, but make no Trina MacDonald expressed or implied warranty of any kind and assume no responsibility for errors or Development omissions. No liability is assumed for incidental or consequential damages in connection Editor with or arising out of the use of the information or programs contained herein. Chris Zahn For information about buying this title in bulk quantities, or for special sales opportunities Managing Editor (which may include electronic versions; custom cover designs; and content particular to Kristy Hart your business, training goals, marketing focus, or branding interests), please contact our Project Editor corporate sales department at [email protected] or (800) 382-3419. Elaine Wiley For government sales inquiries, please contact [email protected] . Copy Editor For questions about sales outside the U.S., please contact [email protected] . Barbara Hacha Visit us on the Web: informit.com/aw Senior Indexer Cheryl Lenser Library of Congress Control Number: 2014944245 Proofreader Copyright © 2015 Pearson Education, Inc. Katherine Matejka All rights reserved. Printed in the United States of America. This publication is protected Technical Reviewers by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, Chris Adamson Ryder Mackay electronic, mechanical, photocopying, recording, or likewise. To obtain permission to Jon Steinmetz use material from this work, please submit a written request to Pearson Education, Inc., Permissions Department, One Lake Street, Upper Saddle River, New Jersey 07458, or you Editorial Assistant may fax your request to (201) 236-3290. Olivia Basegio ISBN-13: 978-0-321-96180-8 Cover Designer ISBN-10: 0-321-96180-3 Chuti Prasertsith Text printed in the United States on recycled paper at Courier in Westford, Massachusetts. Senior Compositor Gloria Schurick First printing: October 2014 ❖ I dedicate this book to my loving wife, Linda, who supports me in all my crazy endeavors. ❖ Contents Preface xiii Part I AV Foundation Essentials 1 1 Getting Started with AV Foundation 3 What Is AV Foundation? 3 Where Does AV Foundation Fit? 4 Decomposing AV Foundation 6 Understanding Digital Media 7 Digital Media Compression 13 Container Formats 18 Hello AV Foundation 19 Summary 23 Challenge 24 2 Playing and Recording Audio 25 Mac and iOS Audio Environments 25 Understanding Audio Sessions 26 Audio Playback with AVAudioPlayer 28 Building an Audio Looper 30 Configuring the Audio Session 34 Handling Interruptions 36 Responding to Route Changes 40 Audio Recording with AVAudioRecorder 42 Building a Voice Memo App 45 Enabling Audio Metering 52 Summary 57 3 Working with Assets and Metadata 59 Understanding Assets 59 Creating an Asset 60 Asynchronous Loading 63 Media Metadata 65 Working with Metadata 70 Building the MetaManager App 76 Saving Metadata 98 Summary 101 Challenge 101 Contents vii 4 Playing Video 103 Playback Overview 103 Playback Recipe 107 Working with Time 109 Building a Video Player 110 Time Observation 118 Creating a Visual Scrubber 124 Showing Subtitles 129 Airplay 133 Summary 136 Challenge 136 5 Using AV Kit 137 AV Kit for iOS 137 AV Kit for Mac OS X 140 First Steps 140 Control Styles 144 Going Further 147 Working with Chapters 151 Enabling Trimming 157 Exporting 159 Movie Modernization 161 Summary 165 Challenge 166 Part II Media Capture 167 6 Capturing Media 169 Capture Overview 169 Simple Recipe 174 Building a Camera App 175 Summary 208 Challenge 208 viii Contents 7 Using Advanced Capture Features 209 Video Zooming 209 Face Detection 216 Machine-Readable Code Detection 228 Using High Frame Rate Capture 241 Processing Video 247 Understanding CMSampleBuffer 249 Summary 257 Challenge 258 8 Reading and Writing Media 259 Overview 259 Building an Audio Waveform View 265 Advanced Capture Recording 276 Summary 293 Challenge 293 Part III Media Creation and Editing 295 9 Composing and Editing Media 297 Composing Media 297 Working with Time 300 Basic Recipe 303 Introducing 15 Seconds 307 Building a Composition 311 Exporting the Composition 316 Summary 321 Challenge 322 Contents ix 10 Mixing Audio 323 Mixing Audio 323 Mixing Audio in the 15 Seconds App 327 Summary 333 Challenge 333 11 Building Video Transitions 335 Overview 335 Conceptual Steps 337 15 Seconds: Adding Video Transitions 348 Summary 360 Challenge 360 12 Layering Animated Content 361 Using Core Animation 361 Using Core Animation with AV Foundation 363 15 Seconds: Adding Animated Titles 367 Preparing the Composition 378 Summary 383 Challenge 384 Foreword Yeah, we knew QuickTime’s goose was cooked. It had served us well for two decades, but that’s the problem. Apple’s essential media framework was a product of the late 1980s. By the mid 2000s, it had accumulated plenty of cruft: old programming practices, dependencies on system APIs that had fallen out of favor, and features that didn’t stand the test of time (wired sprites, anyone?). Heck, it preferred big-endian numeric values, because that’s what the Motorola 68000 series CPUs used. That’s right, QuickTime had already made two CPU transitions: from 680x0 to PowerPC, and then again to Intel x86. And QuickTime’s evolution in the first decade of the new century was hard to make sense of. Apple built a Java wrapper around QuickTime, then updated it again and left out half the features. They did the same incomplete job in Objective-C and called it QTKit. And then there was a Windows version of QuickTime that stopped getting meaningful updates, and nobody at Apple would tell us why. Usually with Apple, that means they’re up to something. What they were up to was the iPhone, of course. But the first SDK we got for iPhone shipped with a minimum of media support: a full-screen video player that took over your application and the low-level Core Audio library. There was an obvious, enormous hole in the media software stack for what Apple insisted was “the best iPod we’ve ever made.” Yet we knew they couldn’t port QuickTime over to the iPhone, considering they were already walking away from it on the desktop. Oh yeah, they were up to something. Bits and pieces of new media functionality on iPhone popped up here and there over the next few years: a “Media Player” framework to let us query and play the music library songs, and some Objective-C wrappers around Core Audio’s Audio Queue, so that playing from or recording to a file was no longer a 200-line exercise in drudgery. These latter classes were curiously assigned to a new framework—“AV Foundation”—which seemed a misnomer in the iPhone OS 3 era, when it was all “A” and no “V.” In retrospect, we really should have known they were up to something. Then, in 2010, Apple was finally ready to show us what they’d been up to all this time. Apple’s Meriko Borogove stood up at the WWDC “Graphics State of the Union,” showed off iMovie for iPhone, and said that everything Apple used to make this video editor was now available to iOS developers. AV Foundation, formerly consisting of those odd little Core Audio wrappers, was now 40 classes of audio-video processing power. Capture, editing, playback, and export—pretty much everything we ever actually did with QuickTime (sorry, wired sprites)—were all present and accounted for. And now they fit in your pocket or purse. Foreword xi Relieved of 1990s legacies, the new classes were products of genuinely modern thinking. On iOS, they were among the first to make use of Objective-C blocks to handle asynchronous concerns like lengthy media export, practices that now seem like second nature to iOS developers. Back on the Mac, a few of us looked enviously to iOS, given that our choices now consisted of 32-bit-only QuickTime with all its archaic bits or the bowdlerized QTKit. It was hardly surprising in OS X 10.7 (“Lion”) when AV Foundation made its debut on the Mac, or in OS X 10.9 (“Mavericks”) when QuickTime was formally deprecated in favor of AV Foundation. Don’t assume from this story that it’s all rainbows and puppies for us, though.