T e c h n i c a l n o T e A Variational Art Algorithm for Image Generation Y o o n Y o u n G K I m , J A e C h u n R Y u , e u n I l K I m , h Y o u n G K e e K I m a n d B Y u n G s e o n G A h n The authors propose a variational art algorithm: a virtual system-based approaches have been developed, such as the parametric optimization algorithm developed for generating images. Observing method [9] and algorithms to simulate fungal hyphae growth T that the topology optimization method used for multiphysics system s [10]. In the literature described above, images were mostly design can produce two- or three-dimensional layouts without baselines, ABRACT the authors propose to expand it beyond engineering applications for generated by simulating biological or physical phenomena. generating images. They have devised a virtual physical system---a heat- Unlike the aforementioned approaches to simulate physi- path system---that “interprets” the optimization-based process of image cal phenomena, we are proposing an alternative method of generation as the simultaneous drawing of multiple strokes in a painting. image generation—a topology design optimization-based method—as illustrated in Fig. 1. To create the images in Fig. 1, we used randomly generated multiple starting and ending points, regarded as “heat sources and sinks” in the topology Since computational algorithms for art or aesthetic design optimization algorithm [11] for each of the different colors. were suggested in the early 1960s, there has been a growing This algorithm has been used for structural design optimiza- interest in utilizing them in nonengineering and nonscien- tion, but we suggest using the algorithm for image generation tific fields. Among pioneers in computer and algorithmic art, [12]. To realize this idea, we need to answer a fundamental we note the following as relevant to this discussion: Verostko question: What kind of optimization problems should be [1] presented computer-assisted algorithmic drawing in solved to generate images? In theory, any physical system which pen strokes controlled by his own algorithm created may be considered, but to facilitate the generation of diverse artistic images; Cohen [2] developed the painting program images, we need to come up with a virtual design problem AARON for abstract and representational drawings; Nake [3] for which the process of the topology optimization can be and his colleagues presented pioneering works in the field; intepreted in the context of painting. For a virtual problem, and Mandelbrot [4] reported fractal images that were gen- we are proposing a topology optimization problem to find an erated by computer graphics and mathematical equations. efficient two-dimensional, heat-dissipating structural layout. Recently, McCormack [5], Sims [6], Heijer and Eiben [7], and Interestingly, the following interpretation may be possible in Todd and Latham [8] used evolutionary algorithms for image this case: A brush stroke drawn on a canvas by an artist may generation. Besides evolutionary algorithms, several other be viewed as an optimal heat path connecting a heat source and a heat sink in a two-dimensional plate under a constraint Yoon Young Kim (educator), School of Mechanical and Aerospace Engineering, Institute of Advanced Machines and Design, Seoul National University, 1 Gwanak-ro, imposed on the mass used to form the path. The interpreta- Gwanak-gu, Seoul 08826, Korea. Email: <[email protected]>. tion is schematically illustrated in Fig. 2, where the starting Jae Chun Ryu (student), School of Mechanical and Aerospace Engineering, point of the brushstroke is interpreted as a heat source, and Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, Korea. Email: <[email protected]>. the end point is interpreted as a heat sink. In solving the Eunil Kim (student), School of Mechanical and Aerospace Engineering, virtual problem, one can use multiple heat sources (starting Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 151-744, Korea. points) and sinks (end points) simultaneously. This inter- Email: <[email protected]>. pretation and freedom to choose the numbers and strengths Hyoungkee Kim (researcher), School of Mechanical and Aerospace Engineering, Institute of Advanced Machines and Design, Seoul National University, 1 Gwanak-ro, of starting and ending points allow us to produce nontrivial Gwanak-gu, Seoul 08826, Korea. Email: <[email protected]>. (nonstraight, thickness-varying) multiple brushstrokes, Byungseong Ahn (student), School of Mechanical and Aerospace Engineering, forming various images. Because the employed thermal Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, Korea. Email: <[email protected]>. problem is artificial, one can also distribute the sources and See <www.mitpressjournals.org/toc/leon/49/3> for supplemental files associated sinks randomly for image diversification. Furthermore, one with this issue. can also vary the value of the mass constraint (interpreted as 226 LEONARDO, Vol. 49, No. 3, pp. 226–231, 2016 doi:10.1162/LEON_a_00914 ©2016 ISAST Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021 Fig. 1. Two cases of image generation by the proposed algorithm: (a) without and (b) with a base image (j: iteration number). (© Yoon Young Kim) Fig. 2. A conceptual similarity between (a) brushstrokes in real painting and (b) the topology optimization problem to fi nd an effi cient heat path on a two-dimensional plate. (© Yoon Young Kim) the amount of paint or ink) quite arbitrarily and even juxta- VARIATIonAl ART AlGoRIThm pose multiple solutions (images) if they are properly colored. As explained in the previous section, the variational art algo- While there were interesting studies related to brush- rithm is based on the topology optimization method, which strokes (e.g. [13–15]), our method does not aim to develop it employs to fi nd effi cient heat-dissipating paths on a two- brushing techniques or to investigate brush stroke eff ects. dimensional structure for a given set of heat sinks, sources Brushstrokes are only used to construct a conceptual similar- and allowable mass usage. Because the detailed procedure ity between the topology optimization and painting. In sub- to perform the heat-path topology optimization has been sequent discussions, the proposed algorithm will be called developed by Kim et al. [17], our main intent here is the ex- the variational art algorithm, because the topology optimiza- planation of the underlying procedure and focus on unique tion method solves a minimization problem formulated by aspects that need to be addressed for image generation. Th e the variational principle [16]. virtual physical system used for image creation is schemati- Kim et al., A Variational Art Algorithm for Image Generation 227 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_00914 by guest on 28 September 2021 Fig. 3. (a) A virtual system of heat transfer that is used in the variational art algorithm. (b) Discretized model of the virtual physical system for the fi nite element analysis. (© Yoon Young Kim) cally shown in Fig. 3. Note that only steady-state conduction as a function of ρe. In this study, a polynomial interpolation e = p and normal convection will be considered in this system. of k ke ρe is employed to express the element conductiv- Following Ryu [18] and referring to Fig. 3, we begin with ity depending on material presence, where ke is the nominal the governing diff erential equation conductivity and ρ the penalty exponent. To solve the topol- ogy optimization using a gradient-based optimizer such as k(x, y)2θ(x, y) − h(x,y)θ(x, y) + Q(x, y) = 0 the optimality criterion (OC) algorithm (see e.g. Bendsoe for a temperature fi eld θ(x, y) distributed on a two-dimen- and O. Sigmund [19]), the sensitivity of the objective func- sional plate . Th e symbols k(x, y) and h(x, y) are the coef- tion with respect to ρe is calculated. Referring to Fig. 1, fi cients of thermal conductivity and convective heat transfer. ρe = M is used everywhere at j = 1 and updated at every itera- Heat sources are denoted by Q(x, y) and prescribed as Q = tion using the sensitivity. Q* in S while the coeffi cient of convective heat transfer is used to model heat sinks such that h = h* in E. ColoRInG TeChnIQue To solve the virtual problem by using the fi nite element To use several colors in generating images, we propose the method, the region where heat transfer takes place for following technique. Let us defi ne the number of colors as given sources S and sinks E (shown in Fig. 3a) is dis- the number nL of layers in the variational algorithm. For each cretized by two-dimensional fi nite elements (i.e. pixels or layer, the topology optimization is solved separately by the voxels) as illustrated in Fig. 3b. By following the standard formulation given above. Aft er topology optimization prob- fi nite element procedure, one can construct an element-level lems are solved n times, the converged images obtained for e e L stiff ness matrix ( ) and nodal vectors (temperature θ and n layers are then superimposed to make a fi nal image. Th e e L load F ) and then form a system matrix equation θ = F color of each layer is represented by the triplet RGB color where , θ and F denote the system-level counterparts. l l l l l l model C such that C = [r , g , b ] (l = 1, 2, ∙∙∙, nL), where r , With these variables, the virtual design optimization prob- g l, and bl take on integer values between 0 and 255.