SMC2011 Template
Total Page:16
File Type:pdf, Size:1020Kb
From Live Coding to Virtual Being Nikolai Suslov Tatiana Soshenina Fund for Supporting Moscow Institute of Architecture Development of Russian Technology State Academy! Vologda, Russia Moscow, Russia [email protected] [email protected] ABSTRACT "e self#explorative, colla%orative environments and virtual worlds are se&ing up the new standards in so'ware engineering for today( In this, live coding is also re)uired in reviewing as for programmers and as for artists too( "e most popular live coding frameworks, even %eing %uilt %y using highly dynamic, re+ective languages, still suffer on tight %indings to single#node or client#server architecture, language or platform dependence and third#party tools. "at leads to ina%ility nor to develop nor scale to the Internet of things the new created works using live coding. In the paper we introduce the prototype of integration of o%ject#oriented language for pa&ern matching .Meta onto Virtual /orld Framework on 0avaScript( "at integration will allow using the live coding in virtual worlds with user-de1ned languages. Also we explore the possi%ilities of a conformal scaling of live coding in the case of augmented reality systems and Internet of things. In summary, the paper descri%es the efforts %eing done for involving virtual worlds architecture in live coding process. All prototypes that are descri%ed in the paper are availa%le for experimenting with on 2restianstvo SD2 open source project3 h&p344www(*restianstvo(org 1. FROM CO !R TOOLS TO S!LF E"#LORAT$VE% COLLABORAT$VE ENVIRONMENTS Fundamentally, live coding is a%out highly dynamic programming languages, re+ectivity, homoiconicity and Meta properties. For example, families of functional languages li*e 5isp, Scheme, 6lojure or pure o%ject# oriented Smalltalk, etc( could %e considered as "ready8 for live coding. And adding to that languages some rich li%rary of functions for working with network, multimedia audio, video! and integrated program development environment ID9! %ased on the de%ugger, ma*es them a complete environment for live coding. "e most popular live coding frameworks still share this tools-centric approach, %ut the current developments in virtual : augmented reality, Internet of "ings, ro%otics etc( shows, that this approach re)uires some review( "e revival and emergence of the self explorative environments and virtual worlds, li*e Self language environment, 9toys, Scratch, .pen6ro)uet, 5ively2ernel, Virtual /orld Framework is o%served now( "is will change the role of the programmer and artist in the process of live coding and also will change the process of live coding itself( See Figure ; for the prototype of controlling several instances of Super6ollider using self explorative, colla%orative .pen 6ro)uet %ased virtual world: 2restianstvo SD2( Figure 1. Installation Man<0 (Soshenina 2>;>! &. L$VE CO ING IN VIRT(AL WORL S W$T* US!R D!FIN! LAN'(A'!S /e want to introduce the prototype of integration of o%ject#oriented language for pa&ern matching .Meta on to Virtual /orld Framework( "e integration will allow to de1ne on any V/F component it's own language grammar and replicate it through the application instances, then have a running scripts %ased on that shared grammar for that component( For example, one could have all the languages down from 5ogo Turtle graphics) to 5isp, Smalltalk, even Super6ollider, availa%le for scripting the virtual world just in the /e% %rowser in real-time, see Figure =( "e Virtual /orld Framewor* V/F! provides a synchroni@ed colla%orative AD environment for the we% %rowser( 6ontinuing the .pen 6ro)uet research effort, V/F allows easy application creation, and provides a simple interface to allow multiple users to interact with the state of the application that is synchroni@ed across clients, using the notion of virtual time( A V/F application is made up of prototype components, which are programmed in 0avaScript, which allows a shared code and %ehaviors used in distri%uted computation, to %e modi1ed at runtime Virtual /orld Framework =>;B!( .Meta is a new o%ject#oriented language for pa&ern matching. It is %ased on a variant of Carsing 9xpression Drammars C9Ds), which have %een extended to handle arbitrary data types. .Meta's general-purpose pa&ern matching facilities provide a natural and convenient way for programmers to implement to*eni@ers, parsers, visitors, and tree transformers /arth =>>E!( /e use AD5 Sand%o$ project as a real-world application, which is %uilt on top of V/F( Figure 2. Integration of OMeta onto Virtual World Framewor* To reali@e that integration we implemented the .Meta driver for V/F( "e drivers in V/F de1ne the autonomic actions that happen within a system, dividing responsi%ility and delegating work for each action of the system. "ese actions include things such as creating or deleting a node, ge&ing or se&ing a property, calling methods, and 1ring events. To load the driver we were needed to include it in the 1les, that respond for V/F %ooting process. As an example, see Figure A for a simple 5#system parser-generator and Turtle#Sphere for drawing it Suslov =>;F!( "e virtual world holds two o%jects, one for drawing and another for parsing and generating 5# system strings. For that purposes, the last one has two grammars, one for parsing user-de1ned input string and another for parsing the generated structure( "e rule of the generator is de1ned recursively in .Meta /arth =>>G!( /e used an enhanced Script editor inside the virtual world framework for editing the source of the grammar( Figure 3. Editing L#system grammar in virtual world "at Script editor is working as Smalltalk %rowser in self#exploratory environment S)uea* Ingalls ;GGE!( It allows to inspect the properties, methods, and events of the selected V/F component visually and to modify it( Here is the source code of .Meta parsers of 5#System in V/F and sample expression, which is de1ned %y one of the live coding session participant in real time3 ometa ParseString { digit = ^digit:d -> d.digitValue(), symbols = ("-" | "+"):n -> n, readRule = letter:n -> n | symbols, expr = <digit*>:d %-% spaces letter:f "=" spaces readRule*:a %-% spaces letter:g "=" spaces readRule*:b ";" spaces readRule*:n spaces -> [d, [[f, a], [g, b]], n] 1 ometa ParseStructure { symbols = ("-" | "+"):n -> n, line = (letter | symbols)*, rules = [letter [line]]*, takeRule:g = [takeRuleFor(g)*:m] -> (vwf.callMethod(nodeID,"flatCollection",[m])), takeRuleFor:g = letter:k -> (vwf.callMethod(nodeID,"selectRule",[g, k])) | symbols:n -> [[n]], gen 0 [rules] [line]:t -> t, gen :n :g takeRule(g):k = gen(n-1, g, k):m -> m 1 tree = LSystem.matchAll('=- 2=2-2''2-2- >=- 2''2''2', 'expr') LSystemGen.matchAll(tree,'gen') V/F could contain a lot of simulations running in it( "ese simulations could %e %ased on data, generated %y so'ware, %ut also the data, that is go&en from any external hardware, li*e sensors. .Meta language will %e a natural solution for parsing that incoming data in user-de1ned way( +. CONFORMAL SCALING OF L$VE CO ING IN T*! CAS! OF INT!RN!T OF THINGS AN AUGMENT! REAL$T, SYST!MS In general, a lot of existed live coding programming languages and their tools are %ased on single#node or client#server architectures. And these nodes are extended with a huge amount of communication protocol li%raries, li*e .pen Sound 6ontrol, MIDI, raw pac*ets, user-de1ned ACI etc( for working with controllers and external nodes. So, the artist works in form of a Ipuppeteer”, who actually has a )uite different approaches in live coding of itself and connected to him nodes. In other words, very o'en the scaling mechanism from one to many nodes lies at the ACI and li%raries level, instead of the programming language itself( "e virtual worlds, which are %ased on the model of distri%uted computation, li*e .pen 6ro)uet and Virtual /orld Framewor*, solve that pro%lem %y generali@ing the model of communication, omi&ing it up to the virtual machine level of the programming language( So, that we could o%serve a conformal scaling from one node to many, while never using live coding techni)ues in terms of communication protocols. Figure 4. Photo of CAV9 environment, running by OpenJwa)42restianstvo virtual world /e %uilt the prototype of 6AV9 cave automatic virtual environment!, which is made of si$ nodes personal computers), see Figure F( 6AV9 application is %ased on our modi1ed version of the .penJwa)4.pen 6ro)uet virtual world development *it3 2restianstvo SD2( Four nodes are used for projecting virtual world onto the cu%e walls, one node is used for controlling movements in virtual world and the last one is used for se&ing the parameters of the 6AV9( .pen 6ro)uet architecture was %uilt for creating replicated virtual worlds, which are distri%uted over the network( Several running virtual machines and their corresponded images with source code, de1ne the replicated state of the whole computation K Island Smith =>>A!( 5ive coding could %e done from any node, as all nodes are e)ual to each other( So, the coder is working in Smalltalk %rowser, which could %e opened in virtual world also( Any modi1cation in source code is replicated immediately to all instances of one shared Island( /hile coding, an artist uses well-*nown Smalltalk language constructions. Scaling the 6AV9 with the new nodes is done just %y starting the new ones and connecting them to the running virtual world( Lsing an avatar an artist adjusts the properties of the 6AV9 system, see Figure B( "e early prototype was used for installation in "e State Tretya*ov Dallery in Moscow and "e National Cush*in Museum in Saint Cetersburg for turning one of the exhi%ition halls into 6AV93 7A( Ivanov Ni%lical sketches" installation( So the artist gets the full featured connected virtual/real space to manipulate with( For example3 it is easy to model augmented lightning for the physical room.