T e c h n i c a l N o t e A Variational Art Algorithm for Image Generation
Y o o n Y o u n g K i m , J a e C h u n R y u , E u n I l K i m ,
H y o u n g k e e K i m a n d B y u n g s e o n g A h n
The authors propose a variational art algorithm: a virtual system-based approaches have been developed, such as the parametric optimization algorithm developed for generating images. Observing method [9] and algorithms to simulate fungal hyphae growth
T that the topology optimization method used for multiphysics system
S [10]. In the literature described above, images were mostly design can produce two- or three-dimensional layouts without baselines, ABRACT the authors propose to expand it beyond engineering applications for generated by simulating biological or physical phenomena. generating images. They have devised a virtual physical system---a heat- Unlike the aforementioned approaches to simulate physi- path system---that “interprets” the optimization-based process of image cal phenomena, we are proposing an alternative method of generation as the simultaneous drawing of multiple strokes in a painting. image generation—a topology design optimization-based method—as illustrated in Fig. 1. To create the images in Fig. 1, we used randomly generated multiple starting and ending points, regarded as “heat sources and sinks” in the topology Since computational algorithms for art or aesthetic design optimization algorithm [11] for each of the different colors. were suggested in the early 1960s, there has been a growing This algorithm has been used for structural design optimiza- interest in utilizing them in nonengineering and nonscien- tion, but we suggest using the algorithm for image generation tific fields. Among pioneers in computer and algorithmic art, [12]. To realize this idea, we need to answer a fundamental we note the following as relevant to this discussion: Verostko question: What kind of optimization problems should be [1] presented computer-assisted algorithmic drawing in solved to generate images? In theory, any physical system which pen strokes controlled by his own algorithm created may be considered, but to facilitate the generation of diverse artistic images; Cohen [2] developed the painting program images, we need to come up with a virtual design problem AARON for abstract and representational drawings; Nake [3] for which the process of the topology optimization can be and his colleagues presented pioneering works in the field; intepreted in the context of painting. For a virtual problem, and Mandelbrot [4] reported fractal images that were gen- we are proposing a topology optimization problem to find an erated by computer graphics and mathematical equations. efficient two-dimensional, heat-dissipating structural layout. Recently, McCormack [5], Sims [6], Heijer and Eiben [7], and Interestingly, the following interpretation may be possible in Todd and Latham [8] used evolutionary algorithms for image this case: A brush stroke drawn on a canvas by an artist may generation. Besides evolutionary algorithms, several other be viewed as an optimal heat path connecting a heat source and a heat sink in a two-dimensional plate under a constraint Yoon Young Kim (educator), School of Mechanical and Aerospace Engineering, Institute of Advanced Machines and Design, Seoul National University, 1 Gwanak-ro, imposed on the mass used to form the path. The interpreta- Gwanak-gu, Seoul 08826, Korea. Email:
Eunil Kim (student), School of Mechanical and Aerospace Engineering, virtual problem, one can use multiple heat sources (starting Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 151-744, Korea. points) and sinks (end points) simultaneously. This inter- Email:
226 LEONARDO, Vol. 49, No. 3, pp. 226–231, 2016 doi:10.1162/LEON_a_00914 ©2016 ISAST
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021 Fig. 1. Two cases of image generation by the proposed algorithm: (a) without and (b) with a base image (j: iteration number). (© Yoon Young Kim)
Fig. 2. A conceptual similarity between (a) brushstrokes in real painting and (b) the topology optimization problem to fi nd an effi cient heat path on a two-dimensional plate. (© Yoon Young Kim)
the amount of paint or ink) quite arbitrarily and even juxta- VARIATIonAl ART AlGoRIThm pose multiple solutions (images) if they are properly colored. As explained in the previous section, the variational art algo- While there were interesting studies related to brush- rithm is based on the topology optimization method, which strokes (e.g. [13–15]), our method does not aim to develop it employs to fi nd effi cient heat-dissipating paths on a two- brushing techniques or to investigate brush stroke eff ects. dimensional structure for a given set of heat sinks, sources Brushstrokes are only used to construct a conceptual similar- and allowable mass usage. Because the detailed procedure ity between the topology optimization and painting. In sub- to perform the heat-path topology optimization has been sequent discussions, the proposed algorithm will be called developed by Kim et al. [17], our main intent here is the ex- the variational art algorithm, because the topology optimiza- planation of the underlying procedure and focus on unique tion method solves a minimization problem formulated by aspects that need to be addressed for image generation. Th e the variational principle [16]. virtual physical system used for image creation is schemati-
Kim et al., A Variational Art Algorithm for Image Generation 227
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021 Fig. 3. (a) A virtual system of heat transfer that is used in the variational art algorithm. (b) Discretized model of the virtual physical system for the fi nite element analysis. (© Yoon Young Kim)
cally shown in Fig. 3. Note that only steady-state conduction as a function of ρe. In this study, a polynomial interpolation e = p and normal convection will be considered in this system. of k ke ρe is employed to express the element conductiv-
Following Ryu [18] and referring to Fig. 3, we begin with ity depending on material presence, where ke is the nominal the governing diff erential equation conductivity and ρ the penalty exponent. To solve the topol- ogy optimization using a gradient-based optimizer such as k(x, y)2θ(x, y) − h(x,y)θ(x, y) + Q(x, y) = 0 the optimality criterion (OC) algorithm (see e.g. Bendsoe for a temperature fi eld θ(x, y) distributed on a two-dimen- and O. Sigmund [19]), the sensitivity of the objective func- sional plate . Th e symbols k(x, y) and h(x, y) are the coef- tion with respect to ρe is calculated. Referring to Fig. 1,
fi cients of thermal conductivity and convective heat transfer. ρe = M is used everywhere at j = 1 and updated at every itera- Heat sources are denoted by Q(x, y) and prescribed as Q = tion using the sensitivity. Q* in S while the coeffi cient of convective heat transfer is used to model heat sinks such that h = h* in E. ColoRInG TeChnIQue To solve the virtual problem by using the fi nite element To use several colors in generating images, we propose the method, the region where heat transfer takes place for following technique. Let us defi ne the number of colors as given sources S and sinks E (shown in Fig. 3a) is dis- the number nL of layers in the variational algorithm. For each cretized by two-dimensional fi nite elements (i.e. pixels or layer, the topology optimization is solved separately by the voxels) as illustrated in Fig. 3b. By following the standard formulation given above. Aft er topology optimization prob- fi nite element procedure, one can construct an element-level lems are solved n times, the converged images obtained for e e L stiff ness matrix ( ) and nodal vectors (temperature θ and n layers are then superimposed to make a fi nal image. Th e e L load F ) and then form a system matrix equation θ = F color of each layer is represented by the triplet RGB color where , θ and F denote the system-level counterparts. l l l l l l model C such that C = [r , g , b ] (l = 1, 2, ∙∙∙, nL), where r , With these variables, the virtual design optimization prob- g l, and bl take on integer values between 0 and 255. Th e color lem is set up to minimize thermal compliance as follows: l th distribution C e of the e fi nite element is conceptually pro- minimize = θ Tθ portional to the value of the element density design variable (ρ1, ρ2, ρ3, ∙∙∙, ρe) (ρe ) as explained in Fig. 4a. Th is means that the brightness of the color Cl depends on the density value. Once Cl is de- where ρe denotes the continuous density design variable al- e located to every element and represents the state of material termined for all elements for all color layers (l = 1, ∙∙∙, nL), the presence in an element (0 for void and 1 for material pres- results can be superimposed based on the subtractive color mixture for each layer. Figure 4b illustrates the superimposed ence) such that 0 ρe 1 (e = 1, 2, ∙∙∙, Ne [the number of elements]). A mass constraint images for nL = 3. Th e images in the bottom row represent the superimposed images for each optimization iteration. Ne ρeVe − M 0 (0 M 1) eΣ=1 PARAmeTeRs AFFeCTInG ImAGe ouTPuT
is imposed. Th e symbols M and Ve denote the allowed mass Th ere are many parameters used to defi ne the virtual heat ratio in forming heat paths and the element volume, respec- transfer system. First, let us consider the eff ects of the dis- tively. For the topology optimization setup, the element con- tance between a heat source S and a sink E on generated ductivity coeffi cient ke needed in forming is interpolated images (Fig. 5a). Here, we only consider the case of a single
228 Kim et al., A Variational Art Algorithm for Image Generation
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021 l l l Fig. 4. (a) The variation of the color triplet Ce for ρe (e and l: element and layer numbers). The value of C here is arbitrarily chosen.
(b) Superimposed images based on the proposed color mixing rule for nL = 3 (j: iteration number). (© Yoon Young Kim)
Fig. 5. Effects of several parameters on generated images. The canvas region is discretized by 400×400 elements (j: iteration number, Ωs:
heat source, ΩE: heat sink). (a) Results with varying distances between source and sink. (b) Effects of varying the number of heat sinks for the case of a single source. (c) The effects of varying the number of heat sources for a single sink. (d) Experiments with different numbers of sinks and sources: (NS,NE ) = (25,25), (100,100), (200,200) (top to bottom). (e) Effects of the penalty exponent ρ. The ρ value is ρ = 1 for an upper image and ρ = 3 for a lower one. (f) The effects of the iteration number. (g) Effects of the mass constraint ratio M (if M = 0.03, 30% of the canvas will be colored). (© Yoon Young Kim)
Kim et al., A Variational Art Algorithm for Image Generation 229
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021 source and a single sink. All parameters other than distance image (see Fig. 1b) of a mannequin face. In this case, the
remain unchanged regardless of the distance. The optimized nominal conductivity ke may be set to be proportional to the path widens as the distance between the sink and the source brightness of the image at the corresponding element. (Note
decreases and narrows as the distance increases. The effects that the uniform value of ke = 1 is assigned everywhere when
of the numbers of heat sinks are similar to those of heat no base image is used.) In fact, the value of ke controls how sources as examined in Fig. 5b,c. Figure 5b shows the images well the color is spread out spatially in the virtual system. In obtained for a single source with a varying number of sinks, Color Plate Cb, each figure shows 2 sets of the images pro- and Fig. 5c shows the reverse case. Experiments varying the cessed from a mannequin face (the base image), with some total numbers of sinks NE and sources NS (with NE = NS ) are variations in controlling parameters. shown in Fig. 5d. The more numbers of sinks and sources used, the finer the obtained images are. Con clusions We also briefly cover here the parameters related to the By transforming the topology optimization method—an topology optimization algorithm. The effects of the so-called engineering design method—into the variational art algo- penalty exponent p in the interpolation of the element con- rithm, we have developed an automated algorithm to gener- e = p ductivity coefficient k ke ρe are examined in Fig. 5e. Dis- ate images that may be artistically interesting. In doing so, tinct images are obtained for ρ 1 while blurred images we propose the notion of a virtual physical system. Images are obtained when ρ = 1. Although blurred images should of various impressions are possible because the parameters be avoided in engineering applications, they can give an to control the virtual physical system and the optimization impression of ink-and-wash painting in images. In fact, the algorithm can be arbitrarily selected. With this freedom, im- generation of such blurred images may be an advantage of ages of various impressions, including those of ink-and-wash the proposed variational art algorithm. It is also found from painting, may be obtained. Some links between brushstrokes Fig. 5f that distinct images appear as the iteration number j and the process of the proposed algorithm were mentioned increases. The influence of the mass constraint ratio is shown mainly because technical aspects may be more effectively ex- in Fig. 5g. The result is obvious: The greater the allowed mass, plained. Actually, the drawing by the variational algorithm the thicker the connecting line becomes. Using three col- is two-dimensional and thus differs from mimicking brush- ors with different parameter values, our algorithm obtains strokes or realizing various stroke effects. The artistic virtue images as shown in Color Plate Ca. Although not explicitly of the images created by the variational art algorithm may analyzed in this study, the profound effects of parameter con- be debatable, but this work suggests a new possibility for trol making a variety of heat paths can be seen from these extending the topology optimization method, thus far a pure images. engineering method, beyond the engineering realm to an The variational algorithm can also be applied to the base art/design realm.
Acknowledgments 7 E. Heijer and A.E. Eiben, “Comparing aesthetic measures for evolu- tionary art,” Applications of Evolutionary Computing Lecture Notes This research was supported by the BK21+ Transformative Training in Computer Science, Vol. 6025, 311–320 (2010). Program for Creative Mechanical and Aerospace Engineers and Com- mercializations Promotion Agency for R&D Outcomes; funded by the 8 S. Todd and W. Latham, Evolutionary Art and Computers (London: Ministry of Science, ICT & Future Planning, and Korea (Grant No. Academic Press, 1992). 2015K000357); and contracted through the Institute of Advanced Ma- chines and Design at Seoul National University. 9 A. Bastanfard and H. Mansourifar, “A novel decorative Islamic star pattern generation algorithm,” in Proceedings of International Confer ence on Computational Science and Its Applications (2010). References 10 R. Alexander, Mycelium. Available:
230 Kim et al., A Variational Art Algorithm for Image Generation
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021 15 N. Chu and C.-L. Tai, “MoXi: Real-time ink dispersion in absorbent Glossary paper,” in Proceedings of SIGGRAPH, 504–511 (2005). finite element method—a numerical analysis method used to transform 16 K. Washizu, Variational Methods in Elasticity and Plasticity (Oxford: a continuous problem for which it is hard to obtain exact solutions Pergamon Press Ltd, 1982), 3rd ed. into a discrete problem for which approximate solutions can be ob- tained. 17 Y.Y. Kim, J.C. Ryu, and H. Kim, “Computational Expressionism by Topology Optimization: Painting beyond Engineering Design,” in heat path—a path transferring heat from a heat source to a heat sink. Proceedings of 9th World Congress on Structural and Multidisciplinary Optimization (2011). topology optimization method—a method used to find an optimal to- pological layout that minimizes or maximizes a given design objec- 18 Ryu et al. [12]. tive such as structural compliance or eigenfrequency. It is typically implemented with the finite element method so that the optimal 19 M.P. Bendsoe and O. Sigmund, Topology Optimization: Theory, layout at convergence is determined by the material presence in finite Methods and Applications (Berlin: Springer-Verlag, 2003). elements composing a design domain. variational principle—a principle using calculus of variations to find functions or equations by minimizing or maximizing the value of a certain quantity of interest.
Manuscript received 9 December 2013.
C a LL f o R P a pers
The Role of Artists and Scientists in Times of War
section editor: Michele Emmer
We are living in an age of conflict, tension, wars and terrorism. Every day we see the slaughter of innocents, kidnapping and murder; we hear of conflicts whose causes have never been resolved and which continue to generate more conflicts and misery.
What is the role of artists and scientists in times of war and conflict? Viewed in terms of ethics, the role of scientists, artists and intellectuals is irreplaceable. Despite the difficulty of imagining any possibility of changing the world, many of us believe that the role of art in the widest sense is essential.
This is why we renew the Leonardo call for papers from artists, scientists and concerned thinkers worldwide to send manuscript proposals to Leonardo on the subject of “Artists and Scientists in Times of War.”
Author guidelines:
Kim et al., A Variational Art Algorithm for Image Generation 231
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021 COLOR PLATE C Color Plate C: A Variational Art Algorithm COLOR PLATE A for Image Generation
Yoon Young Kim created these images with a variational art algorithm using three colors with different parameter values. The images in (a) were created without a base image, and those in (b) were created with a base image of a mannequin face. Each of the 4×4 images in (b) was obtained separately, with slight parameter variations from each other, and then arranged in the 4×4 format. (© Yoon Young Kim) (See article in this issue by Yoon Young Kim et al.)
243
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021