Art/Art History Music Context Cultural Studies/Communications Computer Science Course Design Strategies Historical Critical Performative Preparation 3 Software 4 Subversive and Syncretic Notions Solo OVERVIEW 14 Integrating Course With Performance 15 Ambient AUDIO Personnel linear Rhythmic narrative design non-linear Performance Design Stratgegies 2 Main Audio Mix Video Mixer Live Camera VIDEO Personnel Live Camera Assist/Effects DataMosh Vocabularies 6 Pitch Correction Documentation Camera YouTube Video Wall 8 Experimental Digital Media 1 New node Mobile Smart Mobs Distributed Systems 7 Multiple Screens Approachs Database Film NEW & EMERGING: Search Engine Art 9 Flarf Poetry new idea . Populist: App Culture 11 Controllers: Multitouch 10 Boutique (ReacTable) 12 iPod Opera 13 Convergence: Performance Horizons Avant Dinner Theatre Conclusions 5 Notes 1) Experimental Digital Media Performative media--including video jamming, live and networked media colaboration, and mobile-device based performance--is reaching a critical mass in digital culture. in Experimental Media Horizons, we will examine a number of outlying practices where the materials of digital culture are beginning to coalesce, and place current activity into a broader historical and aesthetic context. Included is an overview of tech requirements and performance suggestions for developing your own course. 2) Performance Design Stratgegies Perhaps the most exciting development (among countess, rich and diverse developments) is the emergence of an open platform for visual and sonic improvisation--video jam, VJ, audio/visual culture. This movement has begun to appear in venues other than the dance club, specifically in gallery spaces, installations, and as visual accompaniment to live music events outside pop genres. 3) Preparation PREPARATION 1) build library of video clips, both sound and silent, 2) build a library of interactive clips (entry level: Flash. More advanced: Max/Jitter, or Processing) 3) Build a three part sonic library: - Sustained, evolving ambience - Rhythmic backgrounds - Narrative through-lines (voice overs) VIDEO Video clips should be short (under two or three minutes), and they should either posess an ambient soundtrack, or one which makes clear connections between visual and sonic events (i.e., a visual event accompanied by a synch sound), or they should have no soundtrack. For now, the most useful format is uncompressed photo-Jpeg Quicktime movies, at 24 frames per second, at about DV frame size or a bit smaller (640 X 480 pixels or so - - DV is 720 X 480). Any file size with a larger dimension is a waste of precious processing energy. FLASH/PROCESSING: Try out your generative programs (Flash, Processing, etc.) in your VJ software to make sure it actually works, and to discover the further limits of both your authoring and your mixing software. We build Flash animations in Flash CS3, with Actionscript 2.0, since these work fine in Arkaos NuVJ. These clips cannot have preloaders, and all the actionscript has to execute on Flash's main stage/ main timeline. Also, Flash clips cannot be scratched, time-shifted, or reversed, nor can they contain dynamically loaded media. Clips with sound, and videos compressed in .flv format do not work. SOUND Uncompressed .wav or .aif files are preferred, but compressed .mp3 or .aac files are usually good enough. PRESENTATION STRUCTURE If time and resources permit, build an ensemble around six or seven devoted and talented individuals. If this is not possible, try to produce a complete audio soundtrack to play back while the ensemble is creating a live visual performance. In subsequent performances, leave more and more audio elements out of the soundtrack (such as a narrative voice-over component, or a solo interludes) so they can be supplied live during performance and mixed in. Ultimately, create an ensemble where all sonic and visual elements are performed and mixed live. Video jams can also be inserted as impromptu seques between short student films. Members of the ensemble should follow a division of labor that clearly distinguishes each member's role (see chart below). While it's not optimal, pre-produced material can replace any of the members of the ensemble. Also, each member should lear how to automate his particular function, in the even he would need to contribute in another role. A set of about 20 minutes can include short, individual jams created by four to six individuals. Remember, the author of a video jam does not necessarily need to perform his own work. Video jams can be notated ( see blog entry) or improvised with whatever degree of structure is necessary. An experienced ensemble can easily give three or four performances in a semester; if this is a new (WHAT'S IT CALLED? ADVENTURE? EXPERIMENT? ), it's probably best to prepare one show per semester. Because of their open narrative structure and inclusive approach (APPROACH? OR SOMETHING ELSE?), experimental media performances are great venues for collaboration. Ask a dancer to improvise on stage, invite readings by poets or storytellers, invite (solicit?) scores or electronic compositions from local composers as a background for video experimentation. 4) Software Final Cut Pro, Motion (video); Flash CS3 or CS4, Quartz Composer, Processing, Isadora, MaxMSP/Jitter (additional generative visuals); SoundHack, Audacity, and Energy XT2, Circle, MetaSynth, and Live (demo or full versions), sound and music design; iTunes Visualizer, Arkaos Grand VJ, Livid Union, 3rLL (Thrill), Resolume Avenue (demo or full versions), Arkaos NuVJ (full version), video jam and visualizing software. Most essential in the production stage of a program, is to encourage the student to not worry if there seems to be something missing from the work he produces. If he's doing it right, there should be at least TWO things missing (these things are added during the performance/mixing). Each of the library elements are designed to be modular and remixable. The only way to achieve this is to under-design these modules so they don't have two of the three main elements of a live performance: visuals (usually in motion), multi track (multi-function) sound, and some sort of narrative strand that holds our attention and engages us in the performance. This doesn't mean a visual progression of images should not suggest one or many narratives, nor does it mean an ambient soundtrack should not imply or evoke a rhythmic or solo musical utterance. It only means there should be multiple evocations, suggestions, and implications, and none of them should come to fruition. The more suggestive and implicit a visual or sonic element, the richer the total collage of multiple suggestive and implicit elements. DEMO SOFTWARE: Try out a vast number of software tools as demos for two reasons: First, you'll eventually find the software you can actually work with, which seems intuitive and useful to you, and second, use demos to generate material which might be captured in part by just recording material on your monitor with an HD camera. The quality is not as good as an export to file option from a full version, but it might be just good enough to generate placeholder material you'll rework in one of the visual or audio editing programs, plus this process may also push your work in a different direction. 5) Conclusions Certainly, the path for experimental media, if it is to be viable, must be weird, unintelligible, bordering on irrelevance, unpredictable. And yet, the most immediate and fecund source of new material is surely the subversion of code and process itself: transforming the technology by subverting it, by asking it to do that which it was never designed to do, to render that piece of software useless in its original intention, but useful to the expressive digital artist. 6) Vocabularies The next vocabulary may well derive from the tools, process, and shortcomings themselves. One only needs to examine techniques such as pitch-correction (T.Pain) and data moshing (Kanye West, THAT OTHER BAND) to realize that digital artifacts and byproducts of attempts to achieve perfection (pitch) are prima materia for the next generation of digital media artists. 7) Distributed Systems Mass "spontaneous" performances (such as smart mobs) and technology-based performance ensembles (including laptop ensembles and collaborative online groups, i.e., YouTube bands) tend to distribute structural elements among its members, creating an effective contemporary approach to the ensemble convention of division of labor. 8) YouTube Video Wall A relatively new development in experimental digital media is the utilization of the digital object as part of a distributed performance system. Applying simple principles of division of labor to a system like YouTube, it is easy to create a random chaotic video wall from even a modest collection of videos - - as little as four videos playing side by side can easily saturate the sonic and visual landscape. Recent artists like (DOOD IN Bb) have restrained visual and musical vocabulary into a neo-minimalist ambience of 16 (CHECK THIS). Tan Dun's highly underwritten and over exposed YouTube Symphony, in contrast, fails to use digital means to break through the conceptual framework of the 19th century western orchestra: the work is still a 9) Search Engine Art Marshaling the means of outstanding success (the Google search engine) can also result in memorable mashups, as the present author's image search-engine database film Projek Iaght confirms. 10) Controllers: Multitouch Multitouch technology is a critical shift in the digital landscape because it essentially eliminates the interface. We no longer approach data through the interface, we merely interface with data. The absolute genius behind this development will forever be Jeff Han (with perhaps an assist from Mr. Spielberg's Minority Report). Multitouch technology has simply and effectively subverted the principles of screen interface design as espoused by Jacob Nielsen and others. The vocabulary of multitouch is the language of the iPhone/iPod Touch, and also the Palm Pre: drag, scroll, tap, pinch.