Noname manuscript No. (will be inserted by the editor) A deep Convolutional Neural Network for topology optimization with strong generalization ability Yiquan Zhanga· Bo Penga · Xiaoyi Zhoua · Cheng Xianga · Dalei Wanga* Received: date / Accepted: date Abstract This paper proposes a deep Convolutional which can be traced back to Michell (1904), have made Neural Network(CNN) with strong generalization abil- tremendous progress. A variety of numerical methods ity for structural topology optimization. The architec- have sprung up later, including SIMP (Bendsoe, 1989; ture of the neural network is made up of encoding and Zhou and Rozvany, 1991; Rozvany et al., 1992), evo- decoding parts, which provide down- and up-sampling lutionary approaches (Xie and Steven, 1993), moving operations. In addition, a popular technique, namely morphable components (Guo et al., 2014; Zhang et al., U-Net, was adopted to improve the performance of the 2018), level-set method (Wang et al., 2003; Allaire et al., proposed neural network. The input of the neural net- 2004; Du et al., 2018), and others. However, the com- work is a well-designed tensor where each channel in- putational cost is still one of the main obstacles pre- cludes different information for the problem, and the venting the adoption of topology optimization methods output is the layout of the optimal structure. To train in practice, in particular for large structures (Sigmund the neural network, a large dataset is generated by a and Maute, 2013). conventional topology optimization approach, i.e. SIMP. With the recent boost of machine learning algo- The performance of the proposed method was evaluated rithms and advances in graphics processing units (GPU), by comparing its efficiency and accuracy with SIMP on machine learning (ML), especially the deep learning, a series of typical optimization problems. Results show has been seen to make many successful stories in vari- that a significant reduction in computation cost was ous fields, including automatic drive, image recognition, achieved with little sacrifice on the performance of de- natural language processing, and even art. It may shed sign solutions. Furthermore, the proposed method can light on accelerating the adoption of topology optimiza- intelligently give a less accurate solution to a problem tion in more design practices. Recently, a few attempts with the boundary condition not included in the train- have been seen to apply ML on topology optimizations ing dataset. (Lei et al., 2018; Sosnovik and Oseledets, 2017; Banga Keywords Topology optimization · Deep et al., 2018; Yu et al., 2018), microstructural materi- learning · Machine learning · Convolutional neural als design (Yang et al., 2018) and additive manufactur- network · Generalization ability ing (Nagarajan et al., 2018). Theoretically, the optimal layout of the material is a complicated function of the initial conditions based on the optimization objective 1 Introduction and constraints. The neural network is good at fitting a complicated function and this characteristic makes it Since the seminal paper of Bendsoe and Kikuchi (1988), possible for the neural network to fit a target function studies on structural topology optimization problems, which can directly give us a good structure without any Dalei Wang iteration and effectively reduce computational time. E-mail: [email protected] Sosnovik and Oseledets (2017) first introduced the * Corresponding Author deep learning model to topology optimization and im- a Department of Bridge Engineering, Tongji University, 1239 proved the efficiency of the optimization process by for- Siping Road, 200092, Shanghai, China. mulating the problem as an image segmentation task. 2 Yiquan Zhanga et al. His deep neural network model could map from the in- In this study, a deep CNN model with strong gener- termediate result of the SIMP method to the final struc- alization ability is proposed to solve topology optimiza- ture of the design, which effectively decreased the total tion problems. The input of our network is a multi- time consumption. However, his work did not consider channel array with each channel representing differ- the initial conditions for topology optimization, and the ent initial conditions and the output is a segmentation accuracy of the result heavily relies on the first few it- mask with each element represents the probability of erations. Banga et al. (2018) proposed a deep learning reservation. The evaluation result shows that the pro- approach based on a 3D encoder-decoder Convolutional posed method can predict a near-optimal structure in Neural Network architecture for accelerating 3D topol- negligible time. The main novelty of this work is the ogy optimization and to determine the optimal com- strong generalization ability of the proposed deep CNN. putational strategy for its deployment. Elaborating as It can give a less accurate solution to the topology op- Sosnovik and Oseledets (2017), this method also takes timization problems with the different boundary condi- the intermediate result of the conventional method as tions even though the CNN was trained on one bound- the input of the neural network. It could cut down some ary condition. The proposed method will be useful in time needed because of the reduction of iterations, but the preliminary design stage as the well-trained neural it can not completely replace the traditional method. network is able to deliver a rough result which provides the designers with a general idea within a very short Lei et al. (2018) developed a ML driven real-time time. topology optimization paradigm under the Moving Mor- phable Component-based solution framework. Their ap- proach can reduce the dimension of parameter space 2 Overview of neural networks and enhance the efficiency of the ML process substan- tially. The ML models used in the approach were the 2.1 Artificial neural networks and convolutional neural supported vector regression and the K-nearest-neighbors. networks Rawat and Shen (2018) proposed a new topology design procedure to generate good-performance structures us- Artificial neural networks (ANN), inspired by animal ing an integrated Generative Adversarial Network (GAN) nervous systems, is one of the most prevalent and suc- and CNN architecture. But only volume fraction, penalty cessful algorithms in machine learning. The basic com- and radius of the filter are changeable in this method. ponent of the neural network is a neuron, which is a All other initial conditions including force and bound- mathematical approximation of real neurons. In neural ary conditions must be fixed. Guo et al. (2018) proposed systems, a neuron could accept signals from other neu- an indirect low-dimension design representation to en- rons, and if the aggregated signals exceed the thresh- hance topology optimization capabilities, which can si- old, the neuron is fired and then sends a signal to the multaneously improve computational efficiency and so- related ones. In ANNs, a neuron is an abstract compu- lution quality. Yu et al. (2018) also used the GAN and tation unit, receiving inputs from the former neurons CNN to propose a deep learning-based method, which and returning an output to other neurons, which can can predict an optimized structure for a given boundary be mathematically expressed as: condition and optimization setting without any itera- tion. However, the neural network model trained by a y = f (z) = f (wT x + b) (1) large dataset in his research could work under only one boundary condition. Therefore, it is impractical to work where x is the n dimensional input vector, w is the n in the real world because the time needed for prepar- dimensional weight factor vector, and b is the bias ing the dataset and training the model is much longer vector. The reason why ANNs have powerful fitting ca- than the traditional SIMP method. To reduce time on pabilities lies in f (z), which is known as an activation preparing data for training, Cang et al. (2019) proposed function and is usually a non-linear function. There are a theory-driven learning mechanism which uses domain- several commonly used activation functions, including specific theories to guide the learning, thus distinguish- the sigmoid function, the hyperbolic tangent activation ing itself from purely data-driven supervised learning. function tanh(x) and the ReLU function (Nair and Hin- Oh et al. (2019) proposes an artificial intelligent (AI)- ton, 2010). They map a linear input wT x + b to a non- based deep generative design framework that is capa- linear form to strengthen the expression capabilities of ble of generating numerous design options which are a network. not only aesthetic but also optimized for engineering The prevalence of neural networks in recent years performance. mainly stems from the wide application of convolutional A deep Convolutional Neural Network for topology optimization with strong generalization ability 3 neural networks. Early in the nineties of the last cen- convolution operation can be deemed as a feature ex- tury, Lecun et al. (1998) introduced the convolution op- tractor which could discover proper features and re- eration to ANNs for handwritten numeral recognition, duce man-made recognition bias for the segmentation and with the growth of data and computation capa- tasks. Shelhamer et al. (2014) firstly introduced CNN bilities, CNNs are widely applied in computer vision, in the field of semantic segmentation and named their natural language processing and other relevant fields. segmentation network as fully convolutional networks In CNNs the inner product part wT x + b is replaced by (FCN) since they discarded all the dense layers in their convolution operation and for 2-D images convolution network. Badrinarayanan et al. (2017) trained a neu- is defined as: ral network based on the encoder-decoder architecture and proposed an ”unpooling” operation for upsampling a b low-resolution images. To improve the performance of w(x, y) ∗ f (x, y) = w(s, t)f (x + s, y + t) (2) encoder-decoder architecture, Ronneberger et al.