Download at Boykma.Com Programming Interactivity A Designer’s Guide to Processing, Arduino, and openFrameworks Joshua Noble Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo Download at Boykma.Com Programming Interactivity by Joshua Noble Copyright © 2009 Joshua Noble. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/institutional sales department: (800) 998-9938 or [email protected]. Editor: Steve Weiss Indexer: Ellen Troutman Zaig Production Editor: Sumita Mukherji Cover Designer: Karen Montgomery Copyeditor: Kim Wimpsett Interior Designer: David Futato Proofreader: Sumita Mukherji Illustrator: Robert Romano Production Services: Newgen Printing History: July 2009: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. Programming Interactivity, the image of guinea fowl, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information con- tained herein. TM This book uses RepKover™, a durable and flexible lay-flat binding. ISBN: 978-0-596-15414-1 [M] 1247251884 Download at Boykma.Com Table of Contents Preface . .................................................................... xv Part I. Introductions 1. Introducing Interaction Design . .......................................... 3 What This Book Is for 3 Programming for Interactivity 4 The Nature of Interaction 5 Messages and Interaction 7 Interfaces and Interaction 8 Languages of Interaction 10 Design and Interaction 12 Art and Interaction 13 Data Exchange and Exploration 15 Working Process 19 2. Programming Basics . .................................................. 21 Why You’ll Read This Chapter More Than Once 22 The Nature of Code 22 Variables 23 Simple Types 24 Arrays 29 Casting 33 Operators 33 Control Statements 37 if/then 37 for Loop 38 while Loop 39 continue 40 break 40 Functions 41 iii Download at Boykma.Com Defining a Function 41 Passing Parameters to a Method 42 Some Suggestions on Writing Functions 44 Overloading Functions 44 Objects and Properties 46 Scope 49 Review 50 3. Processing . 53 Downloading and Installing Processing 54 Exploring the Processing IDE 54 The Basics of a Processing Application 56 The setup() Method 56 The draw() Method 57 The Basics of Drawing with Processing 60 The rect(), ellipse(), and line() Methods 60 RGB Versus Hexadecimal 62 The fill() Method 63 The background() Method 65 The line() Method 65 The stroke() and strokeWeight() Methods 65 The curve() Method 66 The vertex() and curveVertex() Methods 66 Capturing Simple User Interaction 67 The mouseX and mouseY Variables 68 The mousePressed() Method 69 The mouseReleased() and mouseDragged() Methods 70 The keyPressed and key Variables 73 Importing Libraries 77 Downloading Libraries 77 Loading Things into Processing 79 Loading and Displaying Images 79 Displaying Videos in the Processing Environment 81 Using the Movie Class 81 Reading and Writing Files 83 Running and Debugging Applications 85 Exporting Processing Applications 86 Conclusion 88 Review 89 4. Arduino . ............................................................. 91 Starting with Arduino 92 Installing the IDE 93 iv | Table of Contents Download at Boykma.Com Configuring the IDE 96 Touring Two Arduino Boards 97 The Controller 97 Duemilanove Versus Mini 97 Touring the Arduino IDE 102 The Basics of an Arduino Application 105 The setup Statement 106 The loop Method 106 Features of the Arduino Language 108 Constants 109 Methods 110 Arrays 111 Strings 112 How to Connect Things to Your Board 115 Hello World 117 Debugging Your Application 122 Importing Libraries 124 Running Your Code 126 Running Your Board Without a USB Connection 126 Review 127 5. Programming Revisited . .............................................. 129 Object-Oriented Programming 129 Classes 130 The Basics of a Class 131 Class Rules 132 Public and Private Properties 133 Inheritance 135 Processing: Classes and Files 137 C++: Classes and Files 139 .cpp and .h 140 A Simple C++ Application 142 Pointers and References 144 Reference 146 Pointer 146 When to Use Pointers 147 Large Data Objects 148 Pointers and Arrays 149 When Are You Going to Use This? 150 Review 151 6. openFrameworks . ................................................... 153 Your IDE and Computer 154 Table of Contents | v Download at Boykma.Com Windows 154 Mac OS X 155 Linux 155 Taking Another Quick Tour of C++ 156 Basic Variable Types 157 Arrays 157 Methods 158 Classes and Objects in C++ 159 Getting Started with oF 160 Touring an oF Application 166 Methods 166 Variables 168 Creating “Hello, World” 168 Drawing in 2D 171 Setting Drawing Modes 172 Drawing Polygons 174 Displaying Video Files and Images 176 Images 176 Video 178 Importing Libraries 180 ofxOpenCv 181 ofxVectorGraphics 181 ofxVectorMath 182 ofxNetwork 182 ofxOsc 182 Compiling an oF Program 183 Compiling in Xcode 183 Compiling in Code::Blocks 184 Debugging an oF Application 184 Using the printf Statement 184 Using the GNU Debugger 185 Using the Debugger in Xcode 186 Using the Debugger in Code::Blocks 188 Review 188 Part II. Themes 7. Sound and Audio . ................................................... 193 Sound As Feedback 194 Sound and Interaction 197 How Sound Works on a Computer 199 Audio in Processing 202 vi | Table of Contents Download at Boykma.Com Instantiating the Minim Library 202 Generating Sounds with Minim 204 Filtering Sounds with Minim 208 Sound in openFrameworks 214 openFrameworks and the FMOD Ex Library 221 The Sound Object Library 228 The Magic of the Fast Fourier Transform 233 Physical Manipulation of Sound with Arduino 238 A Quick Note on PWM 239 Creating Interactions with Sound 242 Further Resources 242 Review 243 8. Physical Input . ...................................................... 245 Interacting with Physical Controls 245 Thinking About Kinetics 246 Getting Gear for This Chapter 247 Controlling Controls 248 The Button As an Electrical Object 248 The Button As an Interactive Object 248 The Button As a Value in Code 248 Turning Knobs 249 The Dial As an Interactive Object 249 Potentiometers 249 Using Lights 251 Wiring an LED 252 Detecting Touch and Vibration 253 Reading a Piezo Sensor 254 Getting Piezo Sensors 255 Communicating with Other Applications 259 Sending Messages from the Arduino 262 openFrameworks 263 Detecting Motion 265 PIR Motion Sensor 265 Reading Distance 267 Reading Input from an Infrared Sensor 269 Understanding Binary Numbers 270 Binary Numbers 270 Bits and Bit Operations 271 Why Do You Need to Know Any of This? 273 Detecting Forces and Tilt 273 Introducing I2C 278 What Is a Physical Interface? 283 Table of Contents | vii Download at Boykma.Com What’s Next 284 Review 286 9. Programming Graphics . .............................................. 289 The Screen and Graphics 289 Seeing Is Thinking, Looking Is Reading 292 Math, Graphics, and Coordinate Systems 293 Drawing Strategies 296 Use Loops to Draw 296 Use Arrays to Draw 298 Draw Only What You Need 303 Use Sprites 303 Processing and Transformation Matrices 303 Creating Motion 307 Shaping the Gaze 308 Setting the Mood 308 Creating Tweens 310 Using Vectors 315 Using Graphical Controls 325 ControlP5 Library 326 Event Handling 326 Importing and Exporting Graphics 328 Using PostScript in Processing 329 Using PostScript Files in oF 330 What’s Next 333 Review 334 10. Bitmaps and Pixels . .................................................. 337 Using Pixels As Data 338 Using Pixels and Bitmaps As Input 340 Providing Feedback with Bitmaps 341 Looping Through Pixels 342 Manipulating Bitmaps 345 Manipulating Color Bytes 347 Using Convolution in Full Color 348 Analyzing Bitmaps in oF 349 Analyzing Color 350 Analyzing Brightness 351 Detecting Motion 353 Using Edge Detection 355 Using Pixel Data 361 Using Textures 368 Textures in oF 369 viii | Table of Contents Download at Boykma.Com Textures in Processing 373 Saving a Bitmap 375 What’s Next 376 Review 377 11. Physical Feedback . .................................................. 379 Using Motors 380 DC Motors 381 Stepper Motors 384 Other Options 386 Using Servos 386 Connecting a Servo 387 Communicating with the Servo 387 Wiring a Servo 388 Using Household Currents 392 Working with Appliances 393 Introducing the LilyPad Board 395 Using Vibration 397 Using an LED Matrix 404 Using the Matrix Library 404 Using the LedControl Library 407 Using the SPI Protocol 410 Using LCDs 412 Serial LCD 416 Using Solenoids for Movement 417 What’s Next 420 Review 421 12. Protocols and Communication . ........................................ 423 Communicating Over Networks 425 Using XML 426 Understanding Networks and the Internet 429 Network Organization 429 Network Identification 430 Network Data Flow 431 Handling Network Communication in Processing 432