artists’ article Artistic Exploration of the Worlds of Digital Developmental Swarms
a b s t r a c t
This paper presents artwork Sebastian von Mammen, that was inspired by a compu- tational model called Swarm Thomas Wißmeier, Joyce Grammars. In this work, the “liveliness” of swarms is combined with the generative Wong and Christian Jacob capabilities of more established developmental representa- tions. Three of the authors followed their individual artistic approaches to explore the cre- ativity and dynamics of Swarm Grammar structures. One chose to breed structures interactively to compose virtual spaces. The second explores the movement and construction dynamics of interactive swarms. The third ith an open mind and open eyes, we dis- ships develop because Swarm Gram- artist translated developmental W mars combine the developmental processes of Swarm Gram- cover breathtaking forms and colors, textures and shapes in mars into interactions of paint nature. The ridged desert sands, the reflection of the sun on aspect of L-Systems with an agent- particles driven by friction and the sea, or serene alpine landscapes have served as motifs based modeling approach [6]. Each gravity. for countless paintings, pictures and movies. Nature’s glam- symbol is considered an individual orous constituents of organic origin, such as butterflies and that perceives and reacts to stimuli shells, have been collected for their beauty over centuries. in its environment while grammati- Through algorithms that retrace the processes of growth in cal rules drive its reproduction and construction processes. plants and other forms of life we can also produce beauty. Thus, highly dynamic, complex networks of interactions Indeed, digital art has grown far beyond a binary heritage that emerge [7]. is only suited to re-shaping the reflections of real objects or The Swarm Grammar examples in this article combine the creatures. coordination of movement as seen in birds, reproduction as Algorithms can simply implement nature-inspired processes modeled in the growth of cell colonies or plants and the in- of growth and development by repeated substitution of sym- direct communication through the environment used by so- bols in a string according to a set of rules. L-Systems, which do cial insects. These aspects are found in many natural systems, exactly this, can simulate the growth of cells [1], plants [2], e.g. chemotaxis, quorum sensing and bio-film aggregation at organs [3] and architectural designs [4]. From a larger per- the cellular level. Algorithmically, those aspects have mainly spective, generative processes or developmental models have been studied independently. We utilize Swarm Grammars that extended the computational realms of creation and creativity combine these aspects to create life-like artifacts that both re- [5]. semble organic forms [8] and capture the dynamics of the Besides the distinction of individual symbols, e.g. A or B, any construction processes [9]. information about the relations among the replicating units has to be inferred from the L-Systems’ strings by an external interpreter. Co-authors Sebastian von Mammen and Christian Fig. 1. All mates within the conic field of perception of the dark Jacob have designed Swarm Grammars as a novel computa- agent are considered its neighbors. The perception range is deter- tional representation and algorithmic approach to simulated mined by a distance d and an angle α. (© Sebastian von Mammen) growth. In Swarm Grammars, complex networks of relation-
Sebastian von Mammen (scientist, artist, teacher), Department of Computer Science, University of Calgary, 2500 University Dr. NW, T2N 1N4 Calgary, Alberta, Canada. E-mail:
©2011 ISAST LEONARDO, Vol. 44, No. 1, pp. 5–13, 2011 5
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021 mathematical methods—including fractals—intriguing sculptures of artifi- Fig. 2. Swarm Grammars in Action: (a) Three cial aesthetics can be evolved [11]. Whole agents, A, B, and C worlds of seemingly living organisms (represented as small can be made to grow in virtual spaces pyramids), are head- [12]. ing upwards after their initialization. Agent A An important aspect of developmen- creates the vertically tal systems is their ability to evolve. Real striped centered stem. creativity is conjured when computa- (b) Agents B and C drift tional evolution drives the development apart (indicated by the of structural and graphical representa- slim arrows). Before they run out of energy, agents tions. A human “breeder” can manu- B and C create the spiky ally select “fit” individuals [13,14]. construction elements Alternatively, automatic evolutionary seen in (c). The spikes runs can foster structural attributes that occur because the agents’ energy levels are linked optimize mathematical fitness measures, to their construction ele- e.g. for increasing design complexity ments’ diameters. In the [15] or to introduce architectural func- meantime, agent A, often tionality. In an attempt to automate the hidden from the camera evaluation of generated computer art, by construction elements, has produced new off- human-made works can provide refer- spring and the construc- ence points for assessments. The combi- tion module created so nation of computational evolution and far is repeatedly added developmental models has produced to the growing structure (d–g). (© Sebastian von many artistic works; see for example Part Mammen) III in The Art of Artificial Evolution: A Handbook on Evolutionary Art and Music by Romero and Machado [16]. However, artificial life methodologies have instilled other important aspects in computer arts. Artificial swarms, for instance, regularly enthuse public audi- ences in interactive art installations [17]. Through the interactions of large num- bers of flocking individuals, or boids (see Reynolds [18]), a certain “liveliness” is communicated. As an underlying prin- ciple, each swarm individual (referring to a single unit of the swarm) has a lim- ited perceptional field that determines its neighborhood. In Fig. 1 the field of While artists and computer scientists as Basis for Experiments,” Wißmeier perception is defined by the viewing have explored computational develop- looks at how the artworks serve as vision- distance d and the angle α. Further- mental models and artificial swarms to ary playgrounds for artistic expression, more, at each time-step of the simula- a large extent, the exploration of Swarm media and forms. In the conclusion we tion, the individual computes its current Grammars has begun only very recently. provide an outlook on possible future acceleration in accordance with its neigh- Nest constructions by social insects such work. bors. As a consequence, the individual as wasps, ants and termites hint at the changes its position and its neighbor- power of swarm-based developmental hood configuration, as do the neigh- models. In this paper we present the Related Work boring flock mates. Hence, a feedback works of three artists—the paper’s co- Machines help us in performing re- loop of interactions is triggered that can authors Sebastian von Mammen, petitive tasks. The more powerful the lead to a variety of flocking behaviors Thomas Wißmeier and Joyce Wong— machine, the more complex the con- [19]. who work from Swarm Grammar struc- ductible action can be. Analogously, a Primarily, the flocking dynamics of ar- tures to create real-world artworks. Their more powerful, expressive and abstract tificial swarms feed into animations. But works relied on simulations and simula- computational representation often the spatio-temporal interaction network tion tools conceptualized by co-authors yields more complex computable results. of swarms also supports computational von Mammen and Jacob. The section Multi-cellular organisms are outstand- music generation [20]. Furthermore, “Creating Space” discusses von Mam- ingly complex systems. With L-Systems, simply by leaving traces in space, the men’s utilization of Swarm Grammars Lindenmayer successfully found a gram- flocking patterns can be “solidified” and to create artificial spaces. In the sec- matical representation for the growth their dynamics captured in virtual 3D tion “Abstraction of Dynamics,” Wong of clusters of cells [10]. In retracing sculptures. Some of these virtual sculp- offers a window into the inspirational the development in natural systems, L- tures have served as inspirations for tra- manifestations of interaction dynamics Systems can create natural aesthet- ditionally crafted collages and paintings in complex systems. Finally, in “Models ics. Based on L-Systems and other [21].
6 von Mammen et al., Digital Developmental Swarms
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021 been placed by an agent of type A. Al- ready, the first production rule has been executed, leaving three agents A, B, C, illustrated as the small pyramids floating at the top of the construction. Subse- quently, in Figs 2b to 2g the reproduc- tion and construction process continues: Agents B and C quickly drift apart (em- phasized by the two arrows in Fig. 2b), losing most of their internal energy. The loss of energy is reflected in the shrinking diameter of their constructions. Hence, after another reproduction on each side, the construction processes stop. Only the initial agent A keeps enough energy for persisting cycles of reproduction and construction. In the Swarm Gram- mar configuration used, the loss of en- ergy is type-dependent and linked to the agents’ movements and replication processes. The rules for reproduction and differentiation are triggered after a type-dependent interval of simulated steps. Fig. 3. Evolved Swarm Grammar Samples. (© Sebastian von Mammen)
Fig. 4. Diptych of the two pieces (b) caméléon and (c) bighorn sheep by Sebas- tian von Mammen, acrylic medium on canvas, 23 × 38 inches, 2008. Selections of Swarm Grammar structures bred for the diptych are displayed in (a) and Developmental Creativity (d), respectively. (© Sebastian von Mammen) of Swarms When termites build their nests, they do not work from a blueprint of the new construction. Instead, they follow their instinct and build wherever they see fit. In fact, the construction activi- ties of social insects are determined by pheromone trails left by other individu- als, construction cues in their immediate environment, physical gradients such as heat and humidity, or by plain chance. Based on this idea, simple sets of proba- bilistic behavioral rules have led to con- structions similar to those observed with ants and wasps [22]. Swarm Grammars are a computational concept that integrates (1) the power of constructive swarms with (2) the ability to instantaneously reproduce and (3) the boid flocking paradigm outlined earlier. Figure 2 shows an example of Swarm Grammar development. In Figs 2a through 2g Swarm Grammar agents split and leave construction elements along their flocking paths. The depicted Swarm Grammar hosts three agent types, A, B and C, each exhibiting a different flocking behavior and leaving different construction elements along its path. With an initial, “axiomatic” agent, A, reproduction is performed according to the rule-set {A → ABC, B → A, C → AA}. In Fig. 2a, a construction element has
von Mammen et al., Digital Developmental Swarms 7
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021 Different sculptures emerge when we change the reproduction rules and the agent properties (Fig. 3). We can change this Swarm Grammar con- figuration manually or by means of computational evolution. Many ap- proaches have been tested for the latter case:
1. Through interactive evolution an external breeder rates an array of Swarm Grammars [23]. 2. In an immersive (co-)evolution ap- proach a “gardener” tinkers with Swarm Grammars in a virtual space by replenishing their energy to fur- ther their growth, by inducing mu- tations or by crossbreeding selected specimens [24]. 3. Using automatic evolution a fitness measure is formulated mathemati- cally, which serves to evaluate the construction processes and the emerging structure [25]. Fig. 5. During an interactive evolutionary run (a), the depicted rose-like structure emerged (b). (© Sebastian von Mammen) This structure inspired the piece Pirouette in Red by Joyce Wong, 12 × 12 inches, acrylic and oil on glass, 2008: (c) the sculpture in its initial position; (d) bearings allow the plates to rotate into different configurations. (© Joyce Wong) Creating Spaces Stimulated by the architectural capabili- ties of Swarm Grammars, artist-author mar growth processes are maintained. Sebastian von Mammen combined a Abstraction of Dynamics The interdependent flocking of the collection of swarm structures to cre- The smooth, interwoven curves with Swarm Grammar agents can be replayed ate surreal, artificial worlds. In about sporadically grown thorns seen in Figs by rotating the glass plates, causing a 40 interactive evolutionary experi- 5a and 5b emphasize the emergent great number of alternative structures ments, von Mammen bred the utilized flocking dynamics of the constructing of the same constituents to emerge. The Swarm Grammar structures relying on Swarm Grammar. Figure 5a shows a layer-wise mapping of a two-dimensional Christian Jacob’s Mathematica library screenshot from the interactive breed- image results in a fully 3D sculpture. Both Evolvica for the evolutionary algorithm ing procedure: Several specimens are the glass plates and the bearings have a [26] and the user interface Inspirica growing according to their configura- significant height (about 0.3 inches). [27]. The breeding experiments yielded tion, independently in isolated spaces. Hence, although the images are flat as the sets of Swarm Grammar structures After close inspection, the external on a computer screen, they materialize displayed in Figs 4a and 4d. In the cor- breeder, in this case the team of Wong as if they are three-dimensional due to responding paintings, Figs 4b and c, and von Mammen, rates present Swarm the interplay of transparent plates and respectively (see also Color Plate A), a Grammars. Based on their received fit- bearings. chameleon and a bighorn sheep are nesses, individuals are selected for the immersed in complementary artificial next generation of Swarm Grammar environments. simulations. During the transition from During the evolutionary runs, von one generation to the next, a certain Models as Basis Mammen followed two main objec- percentage of selected Swarm Grammars for Experiments tives. First, robust-looking beams should are interbred to combine some of their Publishing about art inherently requires emerge to form a structural mesh, thus characteristics. The so-called mutation photographs to show the discussed opening vast spaces by their mere exis- operator introduces small configuration pieces. Artist Thomas Wißmeier, how- tence. Second, fuzziness, continuity and changes to some of the other selected ever, intentionally chose the medium of resemblance are sought to warrant the specimens. photography for another reason. Pho- authenticity of the generated virtual Artist Joyce Wong chose the Swarm tography supports Wißmeier’s experi- worlds. The color gradients in the back- Grammar structure depicted in Fig. 5b mental work with an immense degree of grounds reflect the extreme climates of to inspire her real-world sculpture shown freedom, particularly the works based on the habitats of the portrayed animals. in Figs 5c and 5d (see also Color Plate sculptures presented at the end of this They also highlight the wholesome, flu- A). Four glass plates, painted with match- section. As pointed out by the Austrian ent structural architecture in Fig. 4b (see ing Swarm Grammar graphics, are con- arts professor Christian Reder [28]: “As also Color Plate A), and the liveliness and nected through bearings that allow them a multitude of considerations, pictures, dynamics caught in the erratic structures to turn (Fig. 5d; see also Color Plate A). layers, associations, the model is always of Fig. 4c, with “warm” and “cold” pal- Through this innovative sculptural de- a working model at the same time.” This ettes, respectively. sign, the dynamics of the Swarm Gram- statement is a creative imperative—to
8 von Mammen et al., Digital Developmental Swarms
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021 Fig. 6. (a) Swarm Grammar sculpture (© Sebastian von Mammen) that provided inspiration for (b) Thomas Wißmeier, Gate, acrylic media on wood, 22.4 × 39.4 in, 2008. (© Thomas Wißmeier)
use the model is to work with it and to graph shown in Fig. 6b: With monochro- Fig. 7b. The photograph is the result improve on it; it is not an effort of chas- matic coloring and extrapolation the of an intricate experimental process: A ing the initial idea to its immediate final- artwork goes beyond the original struc- glass plate was primed on one side; on ization. In the context of the presented ture by adding a third column. Like the the other side varnish was applied with a work, we follow several iterations of a art pieces presented in the “Creating sponge in a subtle manner and reworked self-similar methodological approach: Spaces” section, the photograph accen- with shades of white and blue; in addi- Scientific biological models were uni- tuates the architectural characteristics tion, undesired light reflections, which fied by Swarm Grammars, a computa- of some Swarm Grammar structures. But occur in photographs when working tional (meta-)model of developmental even though the stylized elements were with varnish, were removed. The piece processes. In a second step, unfold- reduced in number, the architectural reflects the similarity between Swarm ing an instance in the computational and sculptural features of the original Grammar structures and clouds. Clouds, model space led to 3D structures in structure are strengthened. Wißmeier too, are spatial structures that continu- virtual space that served as models for painted the underlying material piece ously change. artistic work. with varnish on a large wooden panel The same process led Wißmeier from and smoothened the strong contrasts the Swarm Grammar structure in Fig. 8a Translating between and hard edges in order to facilitate in to the unnamed photograph displayed Virtual Spaces viewers an engagement with the photo- in Fig. 8b. Rhythmic collisions in the The Swarm Grammar sculpture of Fig. graph’s spatial aspects. Swarm Grammar sculpture emerged 6a has been captured by the artist in a The Swarm Grammar structure in Fig. through synchronized differentiation rather abstract fashion in the photo- 7a inspired Wißmeier’s photograph in into swarm agents that attract and re-
Fig. 7. Inspired by the semi-transparent branching Swarm Grammar structure in (a) (© Sebastian von Mammen), Thomas Wißmeier used experimental processes involving painting with mixed media and reworked photographs to create (b) Thomas Wißmeier, Cloud, acrylic media on wood, 2008. (© Thomas Wißmeier)
von Mammen et al., Digital Developmental Swarms 9
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021 virtual sculpture, Wißmeier chose wires and tin foil as construction elements Fig. 8. The Swarm Gram- for a corresponding real-world model. mar structure shown in (a) emerged from the inter- These materials were chosen to mimic actions of swarm agents the compact wrappings and outgrowths attracting and repelling of the Swarm Grammar sculpture. Al- one another. (© Sebastian though the original swarm structure is von Mammen) Inspired by not directly identifiable in the photo- the process at play in (a), Thomas Wißmeier worked graph (Fig. 10b), several analogies are with black and white color evident: compounds having enough surface tension to repel each 1. The partition of the aperture: few other, thus competing for surface space on this 17.3 light elements on the left-hand × 25.2 inch glass plate (b), side and impenetrable surfaces to acrylic media on wood, 2008. the right. (© Thomas Wißmeier) 2. Interwoven offshoots to the left and a protruding one in the top-right pane. 3. A rippling spiral similar to the seg- mented outgrowths in the Swarm Grammar structure.
pel each other. The resulting structure run in different directions. Thereby, the Most importantly, however, very gen- arose from a competition between repel- artist only indirectly affected the out- eral, common features stand out. First, ling color compounds. Black and white come. the unity and the role of the interwo- colors competed for surface space on a ven elements are apparent. Second, the glass plate. This competition came about Finding the In-between World smoothness and the dynamics of the through a thinning of the varnish that In the works we present in this section, sculpture are also noteworthy. allowed it to flow quickly with gravity. At the Swarm Grammar sculptures were Searching for other suitable materials the same time, the compounds retained used as inspirational starting points for to simulate Swarm Grammar processes, enough surface tension to repel each real sculptures. A newly gained spatiality Wißmeier relied on paper for the piece other. fully springs to life in the pieces in Figs presented in Fig. 11b (see also Color Allowing color particles to find their 10b and 11b (see also Color Plate A). The Plate A). A soft light reflection is intro- own paths is very similar to the idea of interplay of light and shade creates deep, duced through the material. The swirly autonomous Swarm Grammar agents smooth textures, provoking tactile urges. twist of the Swarm Grammar agent illus- leaving traces in space. To further ex- The photographs convey an organic look trated in Fig. 11a is directly reproduced plore this idea, Wißmeier emulated and the fuzziness that we know from nat- in the sculpture in a manifold way. In the branching processes that led to the ural phenomena. The pieces look soft this case a craggy, concave surface was Swarm Grammar sculpture depicted in and gnarly, like skin, but have their very devised. The radial gradients in combi- Fig. 9a to create the work shown in Fig. own unique structures. nation with the fuzziness of the photo- 9b, in which he used a finish with a thick Figure 10a shows a Swarm Grammar graph create the impression of circular texture on a sloped wood panel with a structure with a compact yet spiky core to motion. The white background of the coarse surface; the effects of gravity pro- the right-hand side and seven thick, en- simulation screenshot was turned into pelled the color particles to disperse tangling outgrowths. Their regular seg- black for the photograph, spotlighting across the surface. Moreover, Wißmeier mentation and their pointy ends capture the sculpture and allowing for an adroit rotated the piece, causing the color to the viewer’s attention. Inspired by this illumination.
Fig. 9. Inspired by the Swarm Grammar structure shown in (a) (© Sebastian von Mammen), Thomas Wißmeier caused color particles to disperse by means of gravity across a coarse 13.8-×-27.6-in wood panel to create (b) the work Blue, acrylic media on glass, 2008. (© Thomas Wißmeier)
10 von Mammen et al., Digital Developmental Swarms
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021 Fig. 10. The Swarm Grammar structure shown in (a) (© Sebastian von Mammen) inspired Thomas Wißmeier’s sculpture (b), clear varnish, acrylic media, tin foil, mixed media (acrylic, varnish, wood, glass, tin foil, wires, paper), 7.9 × 9.4 × 2.8 in, 2008. (© Thomas Wißmeier)
Summary and Future Work point, we rely on layering propelled Acknowledgments color particles—inspired by some of the Scientists and artists have collaborated We would like to thank Gerald Hushlak of the De- early works by Canadian artist Gerald partment of Art of the University of Calgary for his in producing the interdisciplinary work Hushlak [29] as well as Wong’s sculpture continuing encouragement of our interdisciplin- discussed in this paper. We believe that ary work, his invaluable feedback and his logistic Pirouette in Red (Fig. 5) and Wißmeier’s these pieces both enrich the art world support. photographs seen in Figs 8b and 9b. and can inspire those involved with mak- A series of photographs of experimen- ing models on any level of abstraction— tal trials by von Mammen is displayed in in silico, in vivo or in-between. Fig. 12. Although this approach has sim- In the future, we want to refine sev- ilarities to modern 3D plotting devices eral concepts and approaches. The ap- [30] and feeds on real-world experi- plied Swarm Grammar model is still ments on self-assembly [31], we believe restrictive in many ways: The assort- that the swarm perspective provides a ment of construction elements is very playground for novel ideas. small, limiting imaginative possibilities. Broader selections and more intricate shapes could enhance the artistic value of the swarm-built structures. Due to Fig. 11. (a) Swarm Grammar graphic (© Sebastian von Mammen), which served as inspira- the overwhelming number of param- tion for (b) Wißmeier’s swarms formed from acrylic media on paper, 2008. (© Thomas Wißmeier) eters that determine a Swarm Grammar construction, the only way currently to implement and realize a conception is through computational evolution. Al- though we believe in the power of this approach, a working artist expects greater freedom for individual arrange- ments in many cases. Therefore, embed- ding an interactive design tool into the Swarm Grammar simulation framework would be beneficial. Regarding new concepts for Swarm Grammar art, we are currently develop- ing mechanisms for the self-organized assembly of 3D structures, following the idea of autonomous color particle paintings. In order to do this, the step- wise extension of the particles’ attri- butes in tandem with means to affect the particles’ situations and interactions needs to be investigated. As a starting
von Mammen et al., Digital Developmental Swarms 11
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021 Fig. 12. Series of photographs showing developmental swarm structures: (a) While color runs down a sloped canvas, a dryer is used to coun- ter the effects of gravity by blowing upwards; (b) The interplay of forces creates branching structures; (c) New layers of color and plastic foil are introduced into the artificially created physical world; (d) Complex structural relationships emerge through the interacting layers of mixed media; (e) The emergent texture has grown into three dimensions. (© Sebastian von Mammen)
12 von Mammen et al., Digital Developmental Swarms
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021 References and Notes 14. K. Sims. Artificial evolution for computer graph- 29. G. Hushlak. Website of Gerald Hushlak,
von Mammen et al., Digital Developmental Swarms 13
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00087 by guest on 02 October 2021