AKITO VAN TROYER Assistant Professor | Berklee College of Music Research Affiliate | MIT Media Lab 1140 Boylston Street, Boston, MA 02215-3693, USA 617.821.1306 | [email protected] | https://vantroyer.com/ EDUCATION Doctor of Philosophy in Media Arts and Sciences 2012.9 – 2018.2 Massachusetts Institute of Technology Media Arts and Sciences Master of Science in Media Arts and Sciences 2010.9 – 2012.8 Massachusetts Institute of Technology Media Arts and Sciences Master of Science in Music Technology 2008.9 – 2010.6 Georgia Institute of Technology Music Technology Bachelor of Arts in Interdisciplinary Studies for Multimedia 2001 – 2008.6 University of Hawaii at Manoa Interdisciplinary Studies for Multimedia RESEARCH INTERESTS Computational Acoustic, Music, and Visual Design. Performance Systems. Audience Participation. Human-Computer Interaction. Ubiquitous Computing. Music Information Retrieval. Signal Processing. Embedded System. Digital Fabrication. RESEARCH EXPERIENCE MM-RT: Electromagnetic Tabletop Musical Instrument 2015 – present MIT Media Lab, Research Assistant under Tod Machover • Investigates new paradigm of musical instruments to trigger sonic curiosity towards music-making. • Initiated concept, produced hardware, and programmed software. • Paper published and presented at the 2017 New Interfaces for Musical Expression conference. • People’s Choice Award for Most Unusual Musical Instrument (2017 Guthman Musical Instrument Competition). Constellation: An Audio Web App for Music Score Making 2013 – present MIT Media Lab, Research Assistant under Tod Machover • Enables website visitors to intuitively explore and co-create a music piece through graphical input. • Designed and produced both front and back-end software. • Paper published in 2013 Computer Music Modeling and Retrieval proceedings. Sound Dérive 2013 – present MIT Media Lab, Research Assistant under Tod Machover • A community-based sound/music application where users can record, upload, view, and share sounds collected in a particular city. • Designed and produced both front-end web and mobile apps as well as back-end server architecture. • Used by thousands of people in Detroit, Miami, Akron, Lucerne in collaboration with organizations such as Detroit Symphony Orchestra, New World Symphony, and Lucerne Festival. Curriculum Vitae : Akito van Troyer 1 Repertoire Remix: Audience Participation-based Networked Music Performance 2013 – 2015 MIT Media Lab, Research Assistant under Tod Machover • Performance system for audience members to dynamically suggest musical preferences for live web-streaming musical improvisation sessions. • Designed software performance system using web interface and server infrastructure that runs in real-time. • Book chapter published for the Computational Music Science book series on Ubiquitous Music (Springer). Faceloid: Controlling Vocaloid using Facial Expressions 2011 – 2012 MIT Media Lab and Yamaha Corporation, Research Assistant under Tod Machover • A computer vision-based software that captures facial gestures to control the hardware Vocaloid synthesizer board in real time. • Developed Java-based Vocaloid controller interface to allow rapid experimentation and mapping of facial gestures to synthetic vocal expressions such as breathiness, brightness and mouth opening. Remote Theatrical Immersion: Extending Sleep No More 2010 – 2011 MIT Media Lab and Punchdrunk, Research Assistant under Tod Machover • Collaborative project to enhance the immersive experience of live participants in the non-verbal theatrical show called Sleep No More using wearable computing devices, sensors, augmented portal objects, operators, and actors. • Developed wearable computing system with bone conduction headsets and sensors for live participants to communicate non-verbally with remote participants. • Paper published from the 2013 ACM international workshop on Immersive Media Experiences. Social Influences on Individual’s Live Performance Experience 2011 MIT Media Lab under Rosalind Picard • Investigates live social inputs from audience members on a visual display to help in increasing the engagement level in a live performance experience. • Designed an experimental study in a lab setting with a custom performance system for audience members with electrodermal activity sensors to report their emotional state in real-time. DrumTop: Everyday Objects as Percussive Instruments 2011 – 2012 MIT Media Lab under Joseph Paradiso • Investigates transforming everyday objects into percussive musical instruments to engage people in exploring the musical potential of their surroundings. • Initiated concept, produced hardware, and programmed software. • The work was exhibited in various venues including MIT Museum and the Curious Sound Objects. • Paper published in the 2012 New Interfaces for Musical Expression conference. Chroma District: Site-specific Interactive Audiovisual Installation 2011 MIT Media Lab, Research Assistant under Tod Machover and Meejin Yoon Collaborators: Eyal Shahar and Seung Jin Ham • A fully automated outdoor interactive artwork that responds with lights and sounds to pedestrians as they walk through the installation. • Designed and fabricated lanterns, electronics, and installation site. • Exhibited at the MIT Festival of Art + Science + Technology in 2011. Acoustically Reconfigurable Musical Instrument 2010 MIT Center for Bits and Atoms under Neil Gershenfeld • Developed a mechanically adjustable sound post structure for the violin to acoustically extend the instrument’s timbre quality in real-time. • Developed the electronic hardware system and fabricated the mechanical system to install inside the body of the violin. Curriculum Vitae : Akito van Troyer 2 Laptop Orchestra Live Coding (LOLC) 2009 – 2010 Georgia Tech Center for Music Technology, Research Assistant under Jason Freeman • Developed a collaborative textual performance system for the laptop orchestra to create and share rhythmic motives based on a collection of recorded sounds. • Designed networked performance system including lexical analysis and parsing. • Several papers published. Received the Best Artwork Award at the 2011 ACM Creativity and Cognition conference. Real-time Beat Tracking System for a Voice and Guitar 2009 – 2010 Georgia Tech Center for Music Technology under Parag Chordia • Researched and developed a system to beat track voice and guitar in real-time for a smartphone application. • Implemented a context-dependent beat tracking algorithm using C/C++ under iOS. Measuring Musical Creativity in Musical Performances through Neural Activities 2009 – 2010 Georgia Institute of Technology, Research Assistant under Parag Chordia, Jason Freeman, and Melody Moore Jackson • Investigating the neural correlates of musical creativity by comparing the EEG signal between improvising and sight-reading performers. • Developed a custom software to run the experiment in a lab setting and recorded EEG signals from the participants while they improvise, reproduce, and rest, based on the instructions from the software. Also developed software to topologically analyze the recorded EEG signals. Beatscape: A Mixed Virtual-Physical Environment for Musical Ensembles 2009 Georgia Institute of Technology under Gil Weinberg Collaborators: Aaron Albin, Brian Blosser, Oliver Jan, and Sertan ¸Sentürk • Mixed virtual-physical environment for musical ensembles where sound objects interact with temporal waves to create rhythmic grooves. • Developed basic concept and built core software components for live performance. Aspects of Sahnsi and Polos in Balinese Gamelan Instruments 2007 – 2008 University of Hawaii, Undergraduate Thesis with Jim Hearon • Project to sample, analyze, and create a composition of computer music simulating the Balinese gamelan instruments. • Analyzed pitch frequencies of Gamelan Instruments, and simulated harmonic beating, taking place between the Sahnsi and Polos of several instruments. TEACHING EXPERIENCE EP-390: Interactive Multimedia Immersive Performance (3 Units) 2020.1 – present Assistant Professor at Berklee College of Music, Boston, Massachusetts Leading a team of Berklee students in working with Nona Hendryx and artist/singer/songwriter Chagall on an interactive and immersive performance premiering Imogen Heap’s Mi.Mu gloves. This course guides students into designing, prototyping, programming, fabricating and evaluating systems and interfaces that enable new technologically-mediated interactions in a live performance setting. Course topics include motion sensing, artificial intelligence, 3D animation, projection mapping, surround sound system, and virtual and augmented reality. EP-435: Audio Programming for iOS (2 Units) 2018.9 – present Assistant Professor at Berklee College of Music, Boston, Massachusetts This senior-level course teaches students to use the Apple XCode IDE and the iOS SDK to develop synthesis, signal processing, and MIDI apps for the iOS devices. Swift is used as the primary programming language, and C programming language is reviewed throughout the class. Curriculum Vitae : Akito van Troyer 3 EP-353: Audio Programming in C (2 Units) 2018.9 – present Assistant Professor at Berklee College of Music, Boston, Massachusetts A junior-level course for teaching students program musical examples and projects in C programming language. Students learn the terminal application, IDE, and GNU/LLVM Compiler on macOS to write musical programs that do algorithmic composition, software synthesis, and signal processing and in the process more deeply understand the