Audio for Virtual and Augmented Reality
Total Page:16
File Type:pdf, Size:1020Kb
2016 AES CONFERENCE AUDIO FOR VIRTUAL AND AUGMENTED REALITY FRIDAY, SEPT 30 THRU SATURDAY, OCT 1 LOS ANGELES CONVENTION CENTER CONFERENCE PROGRAM PLATINUM SPONSORS GOLD SPONSORS SPONSORS MESSAGE FROM THE CONFERENCE CO-CHAIRS Welcome to the first AES International Conference on Audio for Virtual and Augmented Reality! We are really proud to present this amazing technical program, which is the result of many months of extremely hard teamwork. We aimed for the best content we could possibly provide, and here you have it. We are extremely thankful to our great presenters, authors, keynote speakers and sponsors. Together, we made this possible and we sincerely hope you will take away a lot of useful information. Also, I´d like to extend our special thanks to our delegates, coming from all over the world to ANDRES MAYO attend this truly unique event, and to our really hard working team of volunteers, which ultimately made it possible to Conference Co-chair pack this awesome quality and quantity of knowledge in 2 full days crammed with papers, workshops, tutorials and even a technical showcase. Welcome to the show! I would like to extend a warm welcome to all of our delegates, authors, presenters and sponsors. This conference has been a dream of Andres’ and mine since May of 2015. The world of VR / AR has grown so quickly, so fast that we knew we had to bring a conference dedicated specifically to this topic to the audio community. We could not have done this without the hard work and dedication of an incredible conference committee. VR / AR provides entirely new opportunities for audio as it is now part of the experience, not just an aid in conveying story. It has been speculated by Wall Street that VR / AR will LINDA GEDEMER be “as game changing as the advent of the PC”; so we’re in for an incredible journey. I believe the authors, presenters and Conference Co-chair sponsors here are some of the best visionaries to lead us on that journey. Please, enjoy the conference. 1201 SOUTH FIGUEROA STREET LOS ANGELES LOS ANGELES, CALIFORNIA 90015 CONVENTION (213) 741-1151 CENTER LACC Attendee Directions LA Live Entertainment District Visitor Information Centers Metro Guide First floor of Los Angeles Convention Center, showing main entrance to West Hall. Second floor of Los Angeles Convention Center, showing stairs to Lecture Theater and Workshop/Tutorials areas. CONFERENCE COMMITTEE ANDRES LINDA RICHARD BRIAN GREG MAYO GEDEMER WOLLRICH SEAGRAVE KADORIAN Program Content Event Logistics Conference Conference Business & Finance Assistant/Secretary Chair Co-Chair Co-Chair Chair/Treasurer GARY PABLO "TANGO" STEPHEN GRAHAM SARAH EPSTEIN FORMICA O’HARA KIRK STEVENSON Website Social Media Convention Volunteer Sales Session Coordinator Coordinator Coordinator Director Chair PROGRAM AND SIGNAGE DESIGNER: EZEQUIEL COLAVITA The committee would also like to recognize the hard work of these contributors (in alphabetical order): VIVIANA AKEL Plugged Minds CHRIS CAIN University of California, Santa Cruz. MICHELLE GOSSMAN Event Manager, Los Angeles Convention Center MEL LAMBERT Content-Creators.com DAVID SCHEIRMAN President-Elect, AES PROGRAM THEATER OVERVIEW SET UP | 8:00AM TUTORIAL 3 | 2:00PM OPENING KEYNOTE Spatial Audio and Sound | 8:30AM Propagation Philip Lelyveld for VR: New Developments, Implementations, and BREAK | 9:30AM Integration TUTORIAL 1 | 9:45AM BREAK | 3:30PM Audio Recording and Production for TUTORIAL 4 ° | 3:45PM Virtual Reality/360 3D Audio Post-Production Applications Workflows for VR BREAK | 11:15AM BREAK | 4:30PM TUTORIAL 2 | 11:30AM TUTORIAL 5 | 4:45PM Creating Immersive & How Can Audiology and Aesthetic Hearing Auditory Spaces for Science Inform AVAR Virtual and Augmented and Vice Versa? Reality BREAK LUNCH | 6:15PM FRI | 12:30PM TUTORIAL 6 | 6:30PM LUNCH TIME SESSION 1 | 12:45PM VR Audio - The Creating Scientifically Valid Convergence DAY Spatial Audio for VR and AR: of Sound Professions SEPTEMBER 30 Theory, Tools and Workflows ROOM 409A SET UP | 8:00AM PAPER SESSION 1 | 9:45AM BREAK | 3:00PM Sound Localization in 3D Space PAPER SESSION 4 | 3:15PM BREAK Capture, Rendering and | 11:15AM Mixing for VR Part 1 PAPER SESSION 2 | 11:30AM BREAK | 5:30PM Real -World Case Studies Part 1 PAPER SESSION 5 |5:45PM Streaming Immersive LUNCH | 12:45PM Audio Content PAPER SESSION 3 | 2:00PM Real -World Case Studies Part 2 THEATER SET UP | 8:00AM WORKSHOP 5 | CANCELLED Real-Time Production Chain WORKSHOP 1 | 8:30AM for Immersive 3D Audio End-to-End VR Audio Solution for Fully Immersive/ WORKSHOP 6 | 2:45PM Interactive Experience Positioning Sounds in VR Post Production BREAK | 9:15AM BREAK | 3:30PM WORKSHOP 2 | 9:30AM Object Based Audio Mixing WORKSHOP 7 | 3:45PM for AR/VR Applications OZO Audio Workflow BREAK | 10:15AM WORKSHOP 8 | 4:30PM Using MEMS Microphones WORKSHOP 3 | 10:30AM for Ambisonic Audio in a Facebook 360 Spatial Live Streaming Spherical Workstation: Video Camera Tools, Workflows and Best Practices BREAK SA | 5:15PM BREAK | 11:15AM WORKSHOP 9 | 5:30PM A Lightweight & Versatile TUR WORKSHOP 4 | 11:30AM 3D-Audio Codec for Storing Immersive Sound Capture & Transmitting Immersive for Cinematic Virtual Reality Audio and its Application to Live VR 360 Events DAY LUNCH | 12:30PM BREAK | 6:15PM OCTOBER 01 LUNCH TIME SESSION 2 | 12:45PM Challenges in Live Virtual CLOSING KEYNOTE | 6:30PM Reality Audio George Sanger ROOM 409A SET UP | 8:00AM WORKSHOP 10 | 8:30AM PAPER SESSION 7 | 2:00PM Audio Content Creation for Music for VR VR/AR Projects with Standard DAWs BREAK | 3:45PM BREAK | 9:15AM PAPER SESSION 6 | 9:30AM PAPER SESSION 8 | 4:00PM Perceptual Consideration Capture, Rendering and for VR/AR Mixing for VR Part 2 LUNCH | 12:30PM OPENING KEYNOTE CLOSING KEYNOTE THE JOURNEY FUTURE NOSTALGIA, INTO VIRTUAL AND HERE AND NOW: AUGMENTED REALITY LET’S LOOK BACK ON TODAY by Philip Lelyveld - VR/AR Initiative FROM 20 YEARS HENCE Program Manager, USC Entertainment Technology Center by George Sanger - Magic Leap Virtual, Augmented, and Mixed Reality have the potential of delivering Two decades of progress can change interactive experiences that take us to how we live and think in ways that places of emotional resonance, give boggle the mind. Audio is a small us agency to form our own experiential piece of that, but it’s our piece. 20 memories and become part of the years ago, the PC got rudimentary everyday lives we will live in the future. sound cards; now the entire Philip Lelyveld will define what Virtual, “multitrack recording studio” lives Augmented, and Mixed Reality are, on our computers. Some of us saw present recent developments that will that development as inevitable, but shape how they will potentially impact in 1996 those smart people sounded entertainment, work, learning, social fairly edgy, to say the least. And of interaction, and life in general, and those smart people, who among them raise rarely-mentioned but important saw that texting would be the “killer issues that will impact how VR/AR/MR app” for smart phones, in many ways is adopted. Just as TV programming trumping audio communication? progressed from live broadcasts of staged performances to today’s very Let’s take our accumulated wisdom complex language of multithread long- from the past 20 years of growth and form content, so VR/AR/MR will progress non-growth of audio and computing, from the current ‘early days’ of projecting and see if we can’t get some feel existing media language with a few for what it will be like in this room 20 tweaks into a headset experience to a years from now, looking back. new VR/AR/MR-specific language that both the creatives and the audience With luck, we will be nodding our understand. Philip’s goal is to bring you heads sagely, saying, “Yep, we saw up to speed on the current state, the that one coming way back in 2016!” potential, and the known barriers to adoption of Virtual, Augmented, and Mixed Reality. Saturday, October 01 | 6:30 PM AVAR Theater Friday, September 30 | 8:30 AM AVAR Theater TUTORIALS THEATER | 9:45AM THEATER | 12:45AM AUDIO RECORDING AND PRODUCTION LUNCHTIME SESSION 1: FOR VIRTUAL CREATING REALITY/360° THEATER | 11:30AM SCIENTIFICALLY APPLICATIONS CREATING VALID SPATIAL AUDIO IMMERSIVE FOR VR AND AR: & AESTHETIC THEORY, TOOLS AND Chair: AUDITORY SPACES WORKFLOWS JAN PLOGSTIES FOR VIRTUAL AND Fraunhofer Instiute AUGMENTED REALITY RAMANI DURAISWAMI Participants: ADAM O’DONOVAN CHRIS PIKE VisiSonics Corp. BBC Research & Development CHANEL SUMMERS University of Southern ABSTRACT DR. NILS PETERS California Qualcomm Technologies Inc. The goal of VR and AR is to immerse Syndicate 17 LLC DILLON COWER the user in a created world by fooling Google VR the human perceptual system into ABSTRACT perceiving rendered objects as real. ABSTRACT This presentation will This must be done without the brain discuss the challenges experiencing fatigue: accurate audio At IBC, AES 139th, CES and other representation plays a crucial role events Virtual Reality has been a and provide specific solutions for creating in achieving this. Unlike vision with huge topic. VR producers more and a narrow foveated field of view, more realize the potential and need audio within interactive virtual and augmented human hearing covers all directions for spatial audio processing for VR in full 3D. Spatial audio systems applications. This workshop will reality experiences. Audio techniques will must provide realistic rendering discuss the following topics: of sound objects in full 3D to 1. How to record audio for 360° video? be revealed that can be used today to advance complement stereo visual rendering. - Can we use the same techniques We will describe several areas of as for Movie/TV productions? Does a storytelling and gameplay in virtual environments our research, initially conducted at B-format mic do the trick? the University of Maryland over a 2.