Mastering openFrameworks: Creative Coding Demystified A practical guide to creating audiovisual interactive projects with low-level data processing using openFrameworks Denis Perevalov BIRMINGHAM - MUMBAI Mastering openFrameworks: Creative Coding Demystified Copyright © 2013 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: September 2013 Production Reference: 1160913 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-84951-804-8 www.packtpub.com Cover Image by Asher Wishkerman ([email protected]) Credits Author Project Coordinators Denis Perevalov Shiksha Chaturvedi Hardik Patel Reviewers Mathias Paumgarten Proofreader Tim Pulver Mario Cecere Acquisition Editor Indexer Nikhil Karkal Priya Subramani Lead Technical Editor Production Coordinator Sweny M. Sukumaran Arvindkumar Gupta Technical Editors Cover Work Sharvari Baet Arvindkumar Gupta Aparna Kumari Hardik B. Soni Copy Editors Aditya Nair Brandt D'Mello Gladson Monteiro Adithi Shetty Foreword Confuсius said, "Choose a job you love, and you will never have to work a day in your life". Probably this piece of wisdom is meant right for you. Maybe you have already found your vocation. Maybe you are just looking for it. Anyway there's something that made you take this book. If so, let me tell you a little story. My name is Igor Tatarnikov, also known as Sodazot. I'm an artist, although not by my education, but rather by my way of life. I keep on looking for a new and interesting occupation to take up. I've had different hobbies. I've tried a lot of jobs. Three years ago I didn't even think about the interactive video, although my major occupation at that time was making video clips. I used to employ the stop-motion technique combined with computer animation. I also did some filming. By the by I grew more and more interested in the live VJ performances and generative video. I began experimenting with different techniques and posted my clips on the Internet. At the same time there appeared the first available depth camera - Microsoft Kinect on offer. After it, things went crazy. Hundreds of video clips with reviews and tests of this camera's possibilities appeared every day on the Net. I took an interest in it all and kept track of the news. Watching the possibilities displayed I got a lot of new creative ideas. And finally my friends presented me the gadget. Of course I was happy, but at first I lost my head and didn't know what to do with it. Also, it was not long since one person had written me an e-mail asking for permission to use my experimental video clip in his lecture for the students. I gladly agreed and since then we've been communicating by e-mail. It happened that this man lived in Ekaterinburg and we were thousands of kilometers apart. As we communicated, we found a lot of common interests and decided to do something together. My head was boiling with ideas. I drew lots of sketches, shared them all with my friend, and he knew how to put them to life using the openFrameworks toolkit. We took counsel and the discussions brought us even more new ideas. So we created our first commercial project for a special event at one of the Moscow cinemas. It was a funny video installation, where the picture responded to the visitor's movements. Coming up to the screen, a visitor saw himself as a funny character. The visitors of the event liked it very much and our big customers were happy—they played before the screen of the installation like children. We enjoyed the result of the work and it inspired us to create something new. We've been working together for two years since and we've made several interesting and successful projects, using openFrameworks as our main tool. This is how our visual laboratory Kuflex was created. Our projects now grow more complicated and interesting, and our team became more and more numerous. We also work with musicians, artists, dancers, architects, and we have recently created a project with a real symphony orchestra. If you still haven't guessed, I'll tell you that the friend with whom everything started is the author of this book Denis Perevalov. The content of the book is based on our experience of creating interactive installations and performances. It teaches you the openFrameworks' multimedia capabilities and the principles of their usage for building interactive projects, which work with video, 3D graphics, sound, and cameras. The core of the book consists of the real working examples of projects for openFrameworks. Some of them are based on our works, the others were designed exclusively for this book. Besides, across the text you will find hints that will help you avoid many pitfalls in the practical use of openFrameworks. Study openFrameworks, invent, and create your own projects and soon you'll realize that you can implement practically everything you can imagine using it. Igor (Sodazot) Tatarnikov, Artist About the Author Denis Perevalov is a computer vision research scientist. He works at the Institute of Mathematics and Mechanics of the Ural Branch of the Russian Academy of Sciences (Ekaterinburg, Russia). He is the co-author of two Russian patents on robotics computer vision systems and an US patent on voxel graphics. Since 2010 he has taught openFrameworks in the Ural Federal University. From 2011 he has been developing software for art and commercial interactive installations at kuflex.com using openFrameworks. He is the co-founder of interactive technologies laboratory expo32.ru (opened in 2012). Acknowledgement I would like to thank my family—wife Svetlana and son Timofey for their patience and suggestions. And many thanks to my parents and grandparents for great pirozhki, which was a necessary part of the book-writing process. My creative coding experience and the desire to write the book appeared, thankfully, to artist Igor Sodazot, who invented and designed most of the interactive installations, which I program for him using openFrameworks. He is the coauthor of most of the book's examples and its video/audio contents. Thanks to my scientific supervisor Victor Borisovich Kostousov for expending so many efforts to shape my scientific style of thinking and writing. I would like to thank my colleagues working at interactive media art, experimental music, and dance fields—Prof. Yoichi Nagashima, Tatyana Komarova, Ekaterina Zharinova, and my first curator Ksenia Fedorova, for their teaching and influence. And big thanks to my friends and scientific colleagues for supporting me and helping me with ideas: Nikolay Mikhalev, Sergey Samuraev, Kirill Kostousov, Fedor Kornilov, Elizaveta Sayfutdinova, and Prof. Pavel Konstantinovich Kuznetzov. This book would be impossible without hard work on proof-reading by Angelina Poptzova, and technical reviewing by Mathias Paumgarten and Tim Pulver. Thanks to Packt Publishing, who made this book possible. Thanks to the openFrameworks' creators and openFrameworks community for developing this amazing toolkit. All the book's examples were developed together with Igor Sodazot, except the Dancing cloud example, which is based on the idea of nCode installation by Andrey Krel, Igor Sodazot, Denis Perevalov, and Pavel Tikhonenko (2011, Moscow). All the video and audio content for the examples were made by/with Igor Sodazot, except: Image sunflower.png, which was contributed by ©iStockphoto.com/Andrew Johnson Music track surface.wav, which was contributed by Ilya Orange (soundcloud.com/ilyaorange) About the Reviewers Mathias Paumgarten is a creative developer from Austria. He is currently living and working in Santa Monica, California. Starting with a background in Flash development, Mathias found his passion for code-driven animation at a very young age. Over the years while working for and at several agencies he has broadened his skillset by leaving the web platform and working on installations using low-level languages such as C/C++. After graduating with a Bachelor's degree at the University of Applied Sciences, Salzburg, Austria, he decided to leave Austria while focusing on modern web technologies such as HTML5 and JavaScript, currently working as a frontend JavaScript developer. Mathias has worked for several renowned agencies such as B-Reel, Soap Creative, and Firstborn working on projects for Sony, Fox Entertainment, Pepsi Co., and many more. After receiving recognitions such as FWA and other awards, Mathias has also contributed to publications such as HTML5 Games Most Wanted. Tim Pulver is an interaction design student from Potsdam, Germany. As a teenager he was fascinated by the demo scene and how people were able to transform code into something beautiful. He now uses Processing, openFrameworks, and Arduino to create interactive installations, data-visualizations, and user-interface prototypes. One of his recent projects is interactive fulldome data visualization, where users can playfully explore global crop production. In another project, Tim wrote a program that translated an image of an eye based on its structure into unique jewelry, which was printed out using a 3D printer.