VISAP’20, Pictorials and annotated portfolios. video art,moderncomposition lapse, SuperCollider,electronicmusic, information art,parameterspace,time stochastic music,datasonification, Authors Keywords at playin data guidesthevariousmusicalparameters process. Thispictorialdescribeshowcolor space, ratherthantheendresultofacausal as movementthroughavirtualpossibility sonification imaginesthemusicalexperience stochastic of musicalsound-eventsovertime.This probability vectorsthatgovernthebehavior time lapsevideo,constituteever-changing The colorsofLakeMichigan,capturedin color datatogenerateitsmusicalsoundtrack. Leander Abstract through SPACEMUSICAL POSSIBILITY LEANDER isanexperimentalfilmthatsonifies , orprobabilisticapproachtodata Leander throughweightedchance. COLOR DATACOLOR SONIFICATION : NAVIGATING Rather thanmappingthedata directlyto film, inasense,writesitsown soundtrack. the colordatainsyncwith the video.The SuperCollider, sonifying (orperhaps into theaudioprogrammingenvironment values whichcanbereadlikeamusicalscore Processing intoatableoftimestampedcolor quantifiable colordataofthevideousing and assemblingthefootage,Iextracted surrounding environment.Aftergathering sunset. Ialsomadeaudiorecordingsofthe before sunsetandendedthirtyminutesafter March 2020.Eachvideobeganthirtyminutes County, Wisconsinoveraseriesofdaysin from thesamepointonacliffinMilwaukee behavior, Itooktime-lapsevideosofthelake of itswaves.Todocumentthischameleonic atmospheric conditionsintheappearance in thesky,tides,temperature,andother daily, reflectingthemostminutechanges Michigan seemstochangeitscolorsalmost Even inthedreariestearlyspringmonths,Lake Introduction musifying ) www.lawtonhall.com/leander. Leander discovered onlybyexperiencingthepiece. taken isaproductofchanceandalgorithms, compass andbearing,butthespecificpath solitude. Thelakeanditscolorsprovidea landscape onajourneyofpatienceand Leander toward hisloverHero’s lighthouse, who swamnightly acrosstheDardanelles Named after the youngmanofGreekmyth structure andaudio-visualsynchronicity. alike, whilemaintaininganoverallmusical no tworealizationof to weightedrandomchancemeansthat Leaving themoment-by-momentspecifics will occurateachmomentinthefilm. sound-events withparticularcharacteristics representing thelikelihoodthatparticular color valuesareusedasprobabilityvectors fixed soundsormusicalparameters,the [email protected] United States Milwaukee, Wisconsin53211 2400 EKenwoodBlvd. University ofWisconsin–Milwaukee Music CompositionandTechnology Lawton Hall canbeviewedat isaguidefornavigatingmusical Leander areexactly VISAP’20, Pictorials and annotated portfolios. Bells Hue IGHT D Flowchart oftheprocessgenerating music fromvideothroughcolor Saturation data sonificationin Pitch Pads Blue PARAM T RS from videopixels M SICAL COLOR d d DATA et et e e r r PROBABILIT mi mi n n e e Soundscape s s Brightness Leander Rhythm Green . Timbre Red predates thedigitalage.Musicalgames assist withmusicalchanceprocedureslong and theuseofmechanizedautomationto or Delegating compositionaldecisionstorandom, Chance Music relying on coin flips, graphical scores, and (most relying on coin flips,graphical scores,and (most Music ofChanges Many ofhisworksfromthis era,suchas chance proceduresbeginning inthe1950s. American composerJohn Cage championed reaction tothesemodernist tendencies, nineteenth andtwentiethcenturies. complex anddeterministicinthelate as musiccompositionbecameincreasingly This proto-chancemusicfailedtocatchon without theuseofdice intervention. or human can produceover53trilliondifferentpieces device tochoosefragmentsrandomlyand The Componiumusesaroulettewheel-like pinned woodencylinderslikeamusicbox. from materialpre-programmedonto plays combinatorially-generatedmusic Componium dice gameswithaninstrumentcalledthe Nikolaus Winkelmechanizedtheearlier In 1821,Dutch-GermaninventorDiederich from itspossiblecombinationsofmeasures. producing over45trilliondifferentwaltzes würfelspiel in eighteenth-centuryEurope.Anotable1792 from precomposedfragmentswerepopular build piecesofmusicbyrollingdicetochoose ( musikalisches würfelspiele aleatoric attributedtoMozartiscapableof , chanceisnotanewtechnique , aclockworkpipeorganthat , are almost entirely aleatoric, , arealmostentirelyaleatoric, ) inwhichplayers 4 In In
2
1
3 at eachmoment asapieceisrealized. determined bythevalueof eachparameter and thelocationofoneparticular outcomeis represents adimensioninpossibility space values maychangebetween realizations) musical parameter(i.e.parameters whose point withinsuchaspace. possible outcomesthateachoccupyadiscrete space but rathera works, thepieceofmusicisnotfixedentity, In the resonant frequenciesoftheroomitself. each iterationuntilallthatremainsarethe room becomingincreasinglyapparentwith times,with theacousticqualitiesof recorded. Thisprocessisrepeatednumerous being playedintoanemptyroomandre- 1969 featuresarecordingofhisspokenvoice Alvin Lucier’s ambient soundsintheconcerthall. allowing thelistenertofocusexclusivelyon play) fortheentiredurationofpiece, the soloperformerisinstructedto visual landscape.InCage’s1952piece like how experience insomealeatoricworks,much The sonicenvironmentgeneratesthemusical mapping in data visualizationand sonification. (pitch, rhythm,etc.)muchliketheparametric chance techniquestomusicalparameters musical parameters.Cage mappedthese famously) IChingdivination todetermine ) thatencompassesalargenumberof würfelspiele Leander possibility space I AmSittinginaRoom generatesmusicfromits andmorerecentaleatoric 8 Everydynamic (or from parameter tacet 6 Composer 4’33” (not 7 , 5 VISAP’20, Pictorials and annotated portfolios. parameters directly.Theelement ofsurprise while avoidinghavingtomanipulate musical results, itcansimplybeadjusted andrepeated tweakability; ifonefailsto producepleasing criteria. possibility spacethatmeetcertainaesthetic are ameansoflocatingthoseregionsin Such causal,deterministicalgorithms of rollingdiceorbyusinganintegerseries. choosing thefifthoptioneverytimeinstead in theory,beexploredsystematically,e.g.by baggage. that carriedhistoricalorextramusical to avoidareasofmusicalpossibilityspace composition methodsusedrigidalgorithms century, serialismandothermodernist of polyphonicmusic. rhythms toexploretheunchartedfrontier used fixed,repeatingcyclesofpitchesand isorhythmic motetsinthefourteenthcentury to exploreapossibilityspace.Composersof Of course,randomchanceisnottheonlyway especially whentheprocessisautomated. are greaterthanthesumoftheirparts, and allowforthecreationofworksthat The resultsareoftenexcitingandsurprising possibility spaceisincomprehensiblyvast. landscape evenwhenthefullbreadthofa work providesadifferentviewofmusical thinking. Eachrealizationofanaleatoric often blindustoout-of-the-boxcreative space withoutthepersonalbiasesthat unexpected locationwithinapossibility performer, orevencomposerintoan Random chance,then,dropsthelistener, 11 10 Algorithmshavetheadvantage of EvenMozart’s 9 Inthetwentieth würfelspiel could, of aleatoricandcausaltechniquesbyusing composer IannisXenakisdetailedasynthesis In his1963book “good” trajectoriesthroughparameterspace. values, totakeahands-onroleinfinding allow thecomposer,guidedbyhertastesand strictly causal,butinteractivealgorithms may belostwhenexploratorymethodsare deterministic clarityofdata sonificationand indeterminacy ofthe Leander to specificsubspacesasthe musicrequires. work, butsound-eventscanbeconstrained probability spaceexistsforthedurationofa pieces ofmusicovertime.Thesamevast variables thatacomposerusestoshape not fixed,butrathertheprimarydynamic the probabilitiesusedinstochasticmusicare of causaldeterminism.Mostimportantly, to assertcreativeagencywithouttherigidity probability distributions,allowingacomposer the othersbutallaregovernedbysame Each sound-eventbehavesindependentlyof of largenumbersdiscrete determine themorphologyandcharacteristics musically byusingstatisticalprobabilityto summer songsofcicadas,canbeimitated Complex naturalphenomena,suchasthe in stochastic music Xenakis calledthiscompositionmethod others throughstatisticaldistribution. make someoutcomesmorelikelythan outcomes inanaleatoriccontextandto probability tonarrowtherangeofpossible Pithoprakta hybridizesthecombinatorial andmanylaterworks. Formalized Music anddemonstrateditsuse würfelspiele sound-events , the , Greek 13 . 12 find themselvesateverymomentinthepiece. in thismusicallandscapewherelistenersmay of thetimelapsevideodeterminethoseregions defined bythefixed-mediavisuals.Thecolors and awide,windingpaththroughthisspace by asmallnumberofdynamicparameters, generative musicalpossibilityspacedefined methods ofstochasticmusic.Itisbotha medieval isorhythmsandtheprobabilistic Camera placement for Leander VISAP’20, Pictorials and annotated portfolios. occur asaresult ofthechangingvisualcolor dataofthevideo. emerges fromtheamalgamation ofsound-eventsovertime,which The large-scaleformof states thatrepresentspecific localeswithinthepossibilityspace. (discussed individuallybelow) arelimitedtoafinitenumberofdiscrete stereo fieldtocreateanimmersivesoundscape.Allother parameters by sonification;sound-eventsdriftslowlyandrandomly aroundthe sound-events) istheonlycontinuousparameterand notdetermined values ineachdimension.Panning(theleft-rightspatial positionof boundaries ofwhicharedelineatedbythegamut of parametric Each parameterrepresentsadimensioninpossibility space,the They canbedividedintothreecategories: At rightisamapofthedynamicmusicalparametersatplayin continuous) toimposeboundariesonitspossibilityspace. varying textures.In changing therelativevolumelevelofdifferentrecordingstocreate of parameters.Thefieldrecordingvoiceisparameterizedby characteristic qualitiesaregovernedbyasmallandfinitenumber digitally synthesizedusingfairlysimpletechniques,sotheir of eachsound-eventplayedbyvoice.Thebellsandpadsare A varietyofparametersdeterminethecharacteristicqualities performing individualsound-events. types ofsoundmaybeconsideredavoiceinvirtualtrio,responsiblefor and acollageof with ashortattackandimmediatedecay;sustainedvocal-like Only threedifferenttypesofsoundsareusedin The Primary parameters that aremapped from colordata. vectors derived to probability Leander PossibilitySpace field recordings Leander Leander Secondary parameters Secondary parameters primary parameters primary parameters linked tocolordata. but are not directly but arenotdirectly , theseparametersarediscrete(non- that result from that resultfrom isnotpredefined,butrather . Inmusicalterms,eachofthesethree Leander Tertiary parameters that but governedby than colordata. processes other : bell tones
are dynamic, pads Leander ; ,
.
I S PADS BELLS FIELD pitch pitch SI A PA A S pitch pool scales moment- rhythm moment by- fundamental harmonics harmonics wavetable timbre timbre above spacial position (all voices) panning density event volume ofeach sound collage event occurs event occurs when/if an when/if an temporal temporal position position layer in mix rhythm macro- overall form VISAP’20, Pictorials and annotated portfolios.
vectors persampledframe. to createsixprobability brightness colorproperties blue, hue,saturation,and this wayforthered,green, Every pixelinaframeissorted their upperandlowerbounds. brightness valuesbetween sixteen binsholdpixelswith of brightnessvalues.The frame sortedintobins Ten pixelsfromonevideo
B r i _ B 1 ri_16
B
B r
r i _ i _ 1 2 5
B B
r r
i i
_ _
3 1
4
B
r B
i
_ r
4 i
_
1 3
B
B
r
i r
_
i
5 _ 12
B
Br
r
i i_
_
6
B B 1
1
r r
B B
i i
_ _
i r
r
7 1
i
_ _ 0
8 9
its corresponding parametricvaluewill occur. particular bincorrelatesto agreaterlikelihoodthat music in mapped totheparameters that generatethe constitute six properties. Theseconstantly-changing percentages overall pixelsineachbinforofthesixcolor it ispossibletodeterminethepercentageof Once allpixelsinaframehavebeensorted, contains thosebetween0.0625and0.125,etc. brightness valuebetween0.0and0.0625;Bri_2 example, thebinBri_1containsallpixelswitha than RGB,hencethegreaterresolution).For to haveagreaterimpactoncolorperception sixteen bins(smallchangesinHSBvaluestend hue, saturation,andbrightnessaredividedinto gamuts areeachdividedintoeightbins,while range ofpossiblevalues.Thered,green,andblue for eachpixelissortedinto“bins”containinga frame usingProcessing.Eachofthesixvalues HSB valuesofeverypixelthecurrentvideo second), IextractedandtabulatedtheRGB At regularintervalsinthevideo(fourtimesper the probabilitytablesthatdrivemusicin scale). to humanvisualperception (often ona0.0–1.0 three-dimensional modelthatcorresponds roughly brightness uses acoordinate system of 0–255 at8bitsperchannel). TheHSBcolorspace are indicatedexplicitly(oftenonascalefrom the color models.InRGBspace,theintensity of RGB Probability VectorsfromColorValues red and 14 , These six color properties are used to create Thesesixcolorpropertiesareusedtocreate green Leander HSB values to place colors within a virtual valuestoplacecolorswithin avirtual probability vectors , and are two of the most common digital aretwoofthemostcommondigital . Agreaternumberofpixels ina blue components of a color components ofacolor hue , saturation , whichare , and , and Leander . VISAP’20, Pictorials and annotated portfolios. Color analysis(probabilityvectors)—04:55:750 Hue Saturation Brightness four timespersecond.Theprobabilityvectorsfromthecoloranalysis are storedascolumnsinaCSVtable.Eachrowoftheisread back Color sortingProcessingsketch.Thecurrentvideoframeisanalyzed at thesamerateinSuperCollider patchthatgeneratesthemusic. Red Green Blue Right: VISAP’20, Pictorials and annotated portfolios. 0:00 1:00 4:00 3:00 2:00 Below: Left: (pitch pool) scales Hue Changes inthe Color analysis(probabilityvectors)—00:41:500
hue probabilityvectoroverthefirstfourminutesof Saturation timbre bells: Brightness temporal position bells: recordings: field mix Red Leander . temporal position Green pads: timbre pads: Blue VISAP’20, Pictorials and annotated portfolios. Color analysis(probabilityvectors)andparametermappings—07:36:500 (pitch pool) scales Hue Hue Saturation Saturation timbre bells: Brightness temporal Brightness position bells: recordings: field mix Red Red temporal position Green pads: Green timbre pads: Blue Blue a subtleambientbackground collage. which mixesdifferent audio loopstocreate are mappedtothe foreground voice.The bell tonesandfillinsonicspacearoundthe notes movemuchmoreslowlythanthe and the padvoicearedeterminedby The impression ofapolyphonicmusicaltexture. timbres The bins, creatingrhythmicvarietyandinterest. values tendtobespreadacrossnumerous apparent dynamictrajectory,and brightness (due tothesetting sun) is The ever-decreasingbrightnessinthevideo event densityandoverallformofthepiece. of thebells,whichinturn influences the mapped tothe more parametricvariety. the largersizeofHSBvectorstocreate dynamic ofthethreevoicesandbenefit from The belltonesarethemostprominentand of notes inthebelland padvoices. piece, whichhelpdeterminethepitches (pitch collections) Hue mapped toonlyonemusicalparameter. Each ofthesixprobabilityvectorsis Parameter MappingOverview blue temporal position saturation determinesthemusical ofthebelltones,creating the vectors,respectively.Thepad rhythm (temporalposition) vectordeterminesthe red usedthroughoutthe probabilityvector, field recordings and Leander’s Brightness scales timbre most green of of is is
VISAP’20, Pictorials and annotated portfolios. d b a c Leander of thefourpitchstreams.Thebellvoicein the likelihoodthatnoteswilloccurinone sample probabilityvector(above)determines an aggregatetexture(bottomtwostaves).The rhythmic durations(a,b,c,d)combinetoform Four examplepitchstreamswithdifferent c usessixteensimultaneousstreams. b d a Leander isdeterminedbyselecting notes position when intimetheyoccur.The sound-events, itisnecessary todetermine Before otherparameterscanbeappliedto same underlyingrhythmicgridandtempo. and soon.Allstreamsaresynchronizedto is one-thirdthespeed(dottedeighth notes), is halfthespeedof#16(eighthnotes),#14 fastest (constantsixteenthnotes),stream#15 is determinedbyitsindex.Stream#16the of eachnoteinastream (therateofchange) octaves inregulardurations.Theduration and descendsfive-notescalesoverseveral Each pitchstreamconstantlyascends from sixteensimultaneous Brightness >BellRhythm (or rhythm ) ofbell-eventsin pitch streams temporal . dynamic characteristicofLeander. a givenwindowoftime)isanimportant (the numberofsound-eventsoccurringin goes on.Thischanging are, inturn,lessnotesoverallasthepiece and lessinthehigher(faster)ones.There occurring inthelower(slower)pitchstreams more prevalent,resultinginnotes darker (low-brightness)pixelsbecome As thesunsetsovercourseof background whetherornotrestsoccur. pitch stream.Allstreamscontinueinthe vector aremappedtoacorresponding Each ofthesixteenbinsinbrightness stream willoccurorifitbearest. the likelihoodthatnextnoteinapitch The brightness probabilityvectorgoverns event density Leander
,
VISAP’20, Pictorials and annotated portfolios. scales usedin Up anddownarrowsindicate tuning (equaltemperament). their correspondinghuebin. as whole-numberfrequency The sixteenjust-intonation The samescalesinmusical notation color-codedwith deviations fromstandard ratios inSuperCollider. Leander expressed Above: Right: down overthespanofseveraloctavesforduration (chosen randomlyinarandomoctave)andclimbsup Each pitchstreambeginsononeofthefivescaledegrees The pitchofeachnoteisnotparameterizeddirectly. strange andcolorfuldepending onthefrequencyratios. tuning andmaybeverysweet-sounding andconsonantor intervals oftenfalloutside ofstandardequal-tempered by integerratiosbetweenfrequencies. Just-intonation intonation pitch streams,creatingclashes.Thescalesarein from differentscalestooccursimultaneouslyin harmonic varietyandtension.Itispossibleforpitches and dissonantpullawayfromtheCtonic,creating consonant andwanttoreturnC,othersareunstable the otherfourscaledegreesvary;somearestableand Each scalebeginsonthenoteCbutintervalsof pool hue-weighted chance.Thescalesconstitutea scale degreestoplayinthatmomentisdeterminedby pitch stream,thechoiceofwhichsixteensecond bins. When,forexample,scaledegree#2occursina different five-notescalescorrespondtothesixteenhue scale changebasedonthe fixed, thespecificpitchesthatcompriseunderlying While thestreams’movementthroughscale several notesoccurinsuccessionfromasinglestream. or harmonic.Tunefulmelodicfragmentsemergewhen stepwise motionisobscuredandthenotessoundtextural particular streamissparse(lowbrightnessprobability),the of thepiece.Whenoccurrencenotesfromone fromwhichthepitchstreamsretrievenotes. Hue >Scale(PitchPool) , atuningsystemthatdeterminespitches hue probabilityvector.Sixteen degrees pitch just is 15
VISAP’20, Pictorials and annotated portfolios. sounds theircharacteristictonecolors. of are oneofthefactorsthataffectourperception same pitch.Thesevaryingharmonicspectra different overtonesevenwhentheyplaythe Different instrumentstypicallyproduce (multiples) ofthefundamentalfrequency. frequencies oftheseovertonesare to themain numerous When anoteisplayedonaninstrument,
FRE UENCY timbre im 0 0 20 0 click p 0 0 ulse Saturation >BellTimbre , theacousticqualitiesthatgive overtones fundamental f i lter resonance arecreatedinaddition pitch.Often,the harmonics 16
a uniquecombinationofharmonics,are sixteen differentarrays,eachrepresenting To createavarietyofbell-liketimbres, resonator ringsatonlyaspecificfrequency. through abankoftunedresonators.Each by filteringthesoundofadrumstickclick The belltonesin characterized bytheirharmonicspectra. resonators fuseintounifiedtimbres bank whenanoteoccurs.Theringing the tuningsoffiltersinresonator produced bythepitchstreamstodetermine multiplied bythefundamentalfrequencies example (440Hz).Differenttimbresarecreatedfromvariouscombinationsofharmonicsabovethefundamental. Spectrogram ofthesixteenbelltimbresin Leander BELL TIMBRE INDE aresynthesized Leander 17
. Thefundamentalfrequencyisthesameforeachnoteinthis character ofadialoguebetweenvoices. two alternatingtimbresmaytakeonthe duller timbresfadeintothebackgroundor may emergeasaforegroundvoicewhile from thepitchstreams;brightertimbres that givepassagesaunifyinglogicseparate Distinct timbresresultinperceptualgroupings different partstoinstruments. orchestrating apianopiecebyassigning timbres inthiswayisakintoacomposer vector. Dividingpitchstreamsamong determined bythe The choiceofthetimbreeachnoteis saturation probability 18
VISAP’20, Pictorials and annotated portfolios. FRE UENCY 20 0 0 0 The eightpadtimbresinLeanderarestoredaswavetablesandeachemphasizeadifferent band ofharmonicsaboveafundamentalpitch(440Hzinthisexample). PAD TIMBREINDE corresponding greenbin Pitch stream index and Pitch streamindexand 8 7 6 5 4 3 2 1 Durations ofnotesinpadpitchstreams. note duration 192 beats 168 beats 144 beats 120 beats 96 beats 72 beats 48 beats 24 beats pads takeona moreprominentrole. harmonics becomemorecommon andthe the film,timbreswithlower, morestable low bluevaluesbecomemore prevalentin above andbelowtheprimary timbre.When shift betweenthetwoneighboring wavetables timbres, andthepadssubtlyerratically The synthcaninterpolatesmoothlybetween is governedbytheblueprobabilityvector. to generatenotes.Thechoiceofwavetable stored aswavetables progressively higherbandofharmonics,are Eight differenttimbres,eachemphasizinga of notesoccurringintheeightpitchstreams. probability vectordeterminesthelikelihood a harmonicbackdroptothebells.Thegreen beats each.Theselong,sustainedtonesprovide the padstreamsrangebetween24and192 much moreslowly;thedurationsofnotesin use onlyeightdifferentstreamsandmove streams todeterminerhythm.However,they and useasimilarsystemofprobabilisticpitch The padssharethesamepitchpoolasbells occupy alowerregisterthanthebells. wash insectionsofLeander.Thepads hazy andunstable,creatingabackground away slowlyandtheirtuningtimbreare bell tones.Eachpadnoteemergesandfades and fillinsonicspacebetweenthepercussive Sustained padsprovideacontrastingtexture Blue >PadTimbre Green >PadRhythm 19 usedbythepadsynth VISAP’20, Pictorials and annotated portfolios. prominent, distincttextureastheothervoicesfadeaway. around loops#1and#2thefieldrecordingsemergeasamore ambient collage.Inthefinalminutesof are dividedamongseveraldifferentloops,creatinganindistinct were fartheraway.Throughoutmostofthefilm,redvalues were recordedclosertothelakewhilehigher-numberedloops directly tothevolumelevelsofloops.Lower-numberedloops that playthroughoutthefilm.The The recordingswereeditedintoeightlongloopsofdifferentlength of thefilm.Theyarealsoleastaleatoricelement. and actasabridgebetweentheaudiovisualcomponents a counterpointtothemoreabstractsonicelementsin location wherethevideofootagewasshot.Theserecordingsare Audio fieldrecordings Red >FieldRecordings werecapturedaroundthesame red probabilityvectormaps Leander , redvaluescoalesce Leander
Sample vectorsshown atone-minuteintervals. eight fieldrecordingloopsthroughout The redvectordeterminesthevolume levelsofthe Right: Field recordingnearLakeMichigan. Above andleft: Leander . 2
2 00 00 0 00 00 00 00 00 00 00 00 2 00 00 VISAP’20, Pictorials and annotated portfolios. vectors overthecourseof a piece. a differentmorphology ofprobability be vastlydifferent,since thiswillgenerate sonification system,thesonicresultswill times ofdayarefedinto thissamedata- videos ofdifferentlandscapesat minute changesin thevisuals.Iftime-lapse musically satisfyingresultsandrespondto find pathsthroughthespacethatproduce entire parameterspace.Thedataisusedto to provideacomprehensivelookatits The datamappingin maximize thefractionofgoodresults.” parameter set,butwithcleverdesignshecan what isgoingtohappenforeverysingle composer toknowallofitandpredict can behugeandthereisnowayforthe “The parameterspaceofanalgorithm Composer PalleDahlstedthaswrittenthat exist thatareabsentinthetimelapsevideos. unexplored, simplybecausemanyothercolors the this opportunity.Therearevastareasof and. Theminimaliststyleof low-saturation bandofthecolorspectrum careful focusonchangeswithinanarrow, Lake Michigan’sdiversityofcolorsrequires is afairlygray,dullplace.Toappreciate Even ontheclearestdays,MarchinWisconsin Future Paths Leander possibilityspacethatremain Leander Leander isnotdesigned allows 20 data. Withthedatafixedandparameter sonification reflectschangesintheinput if thatcourseisagoodone,since“good”data taken andalsoprovidesameansofjudging heart of The generative-stochasticsystematthe composition insyncwithtime-based data. used togenerateastochasticmusical palettes thatbecomethesonicmaterial prescribe harmonic,rhythmic,andtimbral The composerishascreativefreedomto arrangement oftheparameterstochance. the visualswhileleavingspecific by choosingmusicalmaterialthatsuits over theaestheticexperienceindirectly This hybridapproachgivestheartistcontrol the musicisnotdependentonvisuals. sonic works,oraudiovisualworksinwhich souces otherthanvideotoproducestrictly generative music.Datacanalsobedrawnfrom hybrid stocastic-deterministicsystemtocreate processing methodsmayfindnovelusesfora edge detection,pixelsorting,andothervideo types ofvideoinputdata.Artistsworkingwith The systemcouldbeadaptedtoacceptother with thecolorprobabilityvectors. specifics ofhoweachparameterinteracts success ofthesonificationdependson space limitedtoonlysixdimensions,the Leander directsthemusicalcourse The moonoverLake Michigan.March2020. VISAP’20, Pictorials and annotated portfolios. Leander of fascinationand thedriving forcebehind captured intimelapse,remained asource mark itspassage. Thecolorsofthelake, absence ofthemanyquotidianeventsthat Time itselfseemedtoebbandflowinthe classmates suddenlyfeltanoceanaway. cancelled. Friends,family,colleagues,and chat. Weddings and familyreunions were meeting, andappointment was nowavideo and abundantly clear.Everyclass,lesson, term “socialdistancing” becameswiftly state. Thefullmeaning ofthenow-familiar Home” order,effectivelylocking downthe On March 24,Wisconsinissueda“Saferat quickly settingin acrosstheUnited States. gravity oftheCOVID-19 pandemicwas between March 16and30,2020asthe I recordedthe Discussion , buttheacceleratedsunsetsand Leander timelapsevideos with hisloverHero,wholitalantern to swam acrosstheDardanellesstraittobe in verseby Christopher Marlowe), Leander In Greekmythology(andlaterimmortalized drag onandflybysimultaneously. apropos asquarantinehoursseemedto hypnotic blurring ofdaysfeltespecially come intofocusin futuremonthsandyears. as-yet-undetermined newnormalthatwill suddenly feltlikeadistant memoryoran perhaps) —eithertheoldnormal that side” (theviewfromHero’s lighthouse, center ofeveryframetorepresentan“other decision toinvertthethreerectangles inthe assemble itintoacohesivewhole.Imadethe nostalgic characterasIattemptedto footage tookon anincreasingly distant, of waiting andisolation. Thegrainylake determination requiredduring thisperiod This seemedafittingsymbolforthepatient guide hisvoyageacrossthedarkwaters. 21
to accompanyeachcomingtwilight. then lettingitsingmegenerativelullabies and suggestingchangestoitsbehavior, generate something beautiful,listening providing itwith thematerialsneededto kind ofdialoguewithmyownartwork by context. Italsoallowedmetoengageina be possibleinastrictlydeterministic time Ilisten,afreedomthatwouldnot possibility thatitmaybedifferentnext at thetimewhile remainingopen tothe areas initspossibilityspacethatfeltright to gently guidethemusictowardthose approach todatasonification allowedme a housein possibility spaceasmyescapeinstead of happened tocreateaconceptual musical fantasies intimesofcrisis way. Itmaybenaturaltoentertain escapist to expresstheminacreativeandmeaningful living throughthemandevenmoredifficult It isdifficulttoprocesseventsasyouare Looking eastover LakeMichiganatsunset,March 2020. Animal Crossing 22 . Astochastic ; Ijustso VISAP’20, Pictorials and annotated portfolios. 10. 9. 8. 7. 6. 5. 4. 3. 2. 1. References for theirfeedbackandencouragement. thanks toPhillipSinkandMarkHirsch assisting withthefieldrecordings.Special generative musicandtoMaggieHallfor his researchandguidanceintheworldof I amindebtedtoWarrenEnströmfor Acknowledgements Allan F.Moore, “Serialism andItsContradictions,” 2001), accessedJune3,2020. Music Online Margaret Bent,“Isorhythm,” 6, no.10(2001),121-124. Through Evolution,” Parameter Space:InteractiveComposition Palle Dahlstedt,“AMutaSynth in a Room Nicolas Collins,AlbumNotesfor Ibid. Wesleyan UniversityPress,1961),57-61. Lectures andWritings John Cage,“Composition,”in 2001), accessedJune5,2020. Online Paul Griffiths,“Aleatory,” componium-dietrich-nikolaus-winkel/ accessed June2,2020.https://interlude.hk/ Nikolaus Winkel,” Georg Predota,“TheComponiumofDietrich 2001), accessedJune5,2020. Music Online Arthur W.J.G.Ord-Hume,“Componium,” (New York:Springer,2010). Paradigms ofAutomatedMusicGeneration Gerhard Nierhaus, (OxfordUniversityPress, byAlvinLucier(LovelyMusic,1990). (OxfordUniversityPress, (Oxford UniversityPress, Interlude Algorithmic Composition: (Middletown,CT: Organised Sound Grove Music , August19,2018, Grove Silence: I AmSittingin
Grove
17. 16. 15. 14. 13. 12. 11. of the5thInternational Conference Synthesis ofBell-LikeSounds,” (Proceedings Paulo A.Esquef,“Efficient Modeling and Matti Karjalainen, VesaVälimäki,and oxfordhb/9780190637224.001.0001 accessed July26,2019,https://doi.org/10.1093/ Press, 2018), and AlexanderRehding(OxfordUniversity Oxford HandbookofTimbre Alexander Rehding,“Timbre/Techne,”in University ofIllinoisPress,2006),32-40. and Other Writings on Music Musical PitchRelations,”in Ben Johnston, “Proportionality and Expanded com/blog/rgb-vs-hsb-vs-hsl-demystified/. June 1,2020.https://www.anaghsharma. Demystified,” September30,2019,accessed Anagh Sharma,“RGBvsHSBHSL– NY: PendragonPress,1992),1-42. in Composition Formalized Music:ThoughtandMathematics Iannis Xenakis,“FreeStochasticMusic,”in Nutshell,” Margaret Boden,“Creativityina Dahlstedt, “MutaSynth.” Sociology ofMusic International ReviewoftheAestheticsand Think Oxford HandbooksOnline , revised.ed.(Stuyvesant, 5(2009),83-96. 26,no.1(1995),77-95. , ed.EmilyI.Dolan Maximum Clarity (Chicago: , The 22. 21. 20. 19. 18. Psychology Today Go aLongWayinCOVID-19World,” Joshua Wilder,“ALittleEscapismMay org/poems/44674/hero-and-leande.r 2020, https://www.poetryfoundation. Poetry Foundation,accessedApril4, “Hero andLeanderbyChristopherMarlowe,” Dahlstedt, “Mutasynth” 2017, https://theproaudiofiles.com/ Synthesis,” Philip Mantione,“TheBasicsofWavetable (Cambridge, MA.:MITPress,1994),92-103. The PerceptualOrganizationofSound Albert S.Bregman, Germany, September26-28,2002). on DigitalAudioEffects,Hamburg, Pro AudioFiles , March29,2020. Auditory SceneAnalysis: , October13,