Zhang et al. / Front Inform Technol Electron Eng in press 1 Frontiers of Information Technology & Electronic Engineering www.jzus.zju.edu.cn; engineering.cae.cn; www.springerlink.com ISSN 2095-9184 (print); ISSN 2095-9230 (online) E-mail: [email protected] Visual knowledge guided intelligent generation of Chinese seal carving∗ Kejun ZHANG†1, Rui ZHANG†1, Yehang YIN1, Yifei LI2, Wenqi WU1, Lingyun SUN1, Fei WU1, Huanghuang DENG1, Yunhe PAN†‡1 1State Key Lab of CAD & CG, Zhejiang University, Hangzhou 310027, China 2School of software Technology, Zhejiang University, Hangzhou 310027, China †E-mail: [email protected]; [email protected]; [email protected] Received Feb. 22, 2021; Revision accepted June 20, 2021; Crosschecked Abstract: We digitally reproduced the process of resource collaboration, design creation, and visual presentation of Chinese seal-carving art. We developed an intelligent seal-carving art-generation system (Zhejiang University Intelligent Seal-Carving System: http://www.next.zju.edu.cn/seal/; The website of the seal-carving search and layout system: http://www.next.zju.edu.cn/seal/search_app/) to solve the difficulties using a visual knowledge- guided computational art approach. The knowledge base in this study is the Qiushi seal-carving database, which consists of open datasets of images of seal characters and seal stamps. We proposed a seal character-generation method based on visual knowledge and guided by the database and expertise. Further, to create the layout of the seal, we proposed a deformation algorithm to adjust the seal characters and calculate layout parameters from the database and knowledge to achieve an intelligent structure. Experiments show that this method and system can effectively solve the difficulties in the generation of seal-carving. Our work provides theoretical and applied references for the rebirth and innovation of seal-carving art. Key words: Seal-carving; Intelligentunedited generation; Deep learning; Parametric modeling; Computational art https://doi.org/10.1631/FITEE.2100094 CLC number: 1 Introduction became a popular form of art among the literati (Gu, 2013). Seal-carving is the art of engraving Chinese The art of seal-carving adheres to classical con- characters on seals, and it has a 3000-year history. ventions, and the seal script is the major script style The seal was initially used as a practical tool that used for seal-carving. The correctness of the char- performs credibility authentication in political and acters’ orthography is traditionally an important as- economic activities. The discovery and populariza- pect of the seal-carving technique (Li, 2009), and the tion of ophicalcite enabled the literati to self-seal and commonly used seal script typologies are very differ- carve. Due to the rise of seals for signing on art col- ent from modern Chinese characters. Chinese char- lections, the function of seal-carving began to change acters’ structural and stroke features have changed from practical to artistic appreciation. Over time, it so much that the seal script can no longer be con- ‡ Corresponding author sidered a style of modern Chinese characters (Wang, * Project supported by the National Key Research and 1980). Seal script information processing technol- Development Program of China (No. 2017YFB1402104) and the Fundamental Research Funds for the Central Universities ogy is slowly evolving, including industry standards (No. 2020QNA5023) for encoding (Unicode Consortium, 2020), recogni- ORCID: Kejun ZHANG, https://orcid.org/0000-0002-0778- 2303 tion, and glyph production. Seal script, on the other c Zhejiang University Press 2021 hand, can still be seen in historical locations, cultural 2 Zhang et al. / Front Inform Technol Electron Eng in press artifacts, and antiques, as well as in books and cal- of a standard script. The database of seal charac- ligraphy works (Qiu et al., 2000). Previous studies ters and seal stamps offers essential visual knowl- of Chinese seal-carving art have primarily focused edge, which guides the generation of seal characters on the identification and authentication of the en- and the layout of a seal. Fig. 1 depicts the flow chart tire seal (Fan and Tsai, 1984; Chen, 1995, 1996; Su, of the intelligent age of seal-carving. Our study can 2007a,b), but there have been few studies on the also be applied to other forms of art and design of production of seal characters. Compared with Chi- Chinese characters, thereby serving the cultural and nese character calligraphy and other art forms, seal- creative industry as well as inheriting and promoting carving art remains time-consuming, economically the traditional culture. costly, and laborious. Therefore, exploring the intel- ligent generation of seal-carving art could improve 2 Related studies the efficiency and quality of seal-carving art creation and assist in reviving this ancient art form. It would 2.1 Generation of characters also lower the cost of seal-carving, enhance the qual- ity of seal products, and make seal-carving art more 2.1.1 Based on interaction accessible to people. Most of the early studies conducted on charac- The intelligent generation of seal-carving art is ter generation were based on interactive methods. complex and challenging, requiring multiple disci- Artists undertook this type of study, and comput- plines across computer science, such as data science, ers offer assistance in improving human design and computer vision, computer graphics, and human- artistic creation efficiency (Zhang, 2019). According computer interaction. However, “visual knowledge” to the study content, it can be roughly divided into (Pan, 2019), as a new form of knowledge represen- two types: Chinese calligraphy generation and font tation, can guide many tasks on computer vision, design. including the generation of seal-carving. This paper Researchers mainly study the digital modeling proposes a method to generate a seal with a specific of calligraphy tools for Chinese calligraphy genera- style and appropriate layoutunedited from a simple format tion, including virtual brush modeling and ink dif- Final seal Generation of Layout of seal seal characters Layout of standard script Guide Guide 000 1-001 .png 000 1-002 .png 000 1-003 .png 000 1-011 .png 000 1-012 .png 000 1-015 .png 03-005-1.png 03-005-2.png 03-005-3.png 03-005-4.png 000 5-002 .png 000 5-003 .png 000 5-004 .png 000 2-004 .png 000 2-006 .png 000 2-007 .png 000 5-001 .png 000 5-011 .png 000 5-014 .png 000 5-019 .png 000 6-001 .png 000 8-003 .png 03-006-2.png 03-006-3.png 03-006-4.png 03-007-3.png Seal characters Seal stamps Database and knowledge Fig. 1 Flow chart for intelligent generation of Chinese seal-carving Zhang et al. / Front Inform Technol Electron Eng in press 3 fusion simulation. Wang and Pang (1986) proposed professional knowledge and still requires a lot of user a computer-based Chinese calligraphy system. They work. simulated writing brushes, built a library of strokes, and used human-computer interaction to combine 2.1.2 Based on graphics characters. In the same year, Strassmann (1986) Glyph generation based on graphical methods studied virtual brush modeling. The author di- generally starts at the font component level (such vided the functions of the brush-writing process into as radicals, strokes, and skeletons). Then, it au- the brush, stroke, dip, and paper and parameter- tomatically generates glyphs through various tech- ized them separately. After that, more researchers niques, such as parameterized representation, com- adopted various methods of modeling. For virtual ponent mapping, and statistical models (Zhang, brush modeling, some researchers start from the con- 2019). Moreover, the objects of glyph generation are tact shape of the brush and paper and employ the mostly modern characters, such as regular, cursive, scattered point set (Yu et al., 1996), ellipse (Wong and running scripts, and even personalized hand- and Ip, 2000), raindrop (Mi et al., 2002; Bi and Tang, writing. Unfortunately, there are very few studies of 2003), etc. The user specifies the crucial locations glyph generation of seal script. for the brush’s movement trajectory to make calli- Xu (2007) and Xu et al. (2009) introduced a graphic characters; however, the parameter settings calligraphy creation method based on analogical and are extremely complex. Xu et al. (2002) and Gir- integrated reasoning. A matching model based on shick (2004) developed a generation model based on strokes was created using a hierarchical parameter the movement of the mouse. (Lu et al., 2013) pro- representation of characters. Furthermore, the spe- posed a data-driven calligraphy and painting system, cific process disassembles the characters to be gen- RealBrush, which can synthesize more colorful and erated and obtains a new character; the matching texture effects. Some researchers also applied me- model chooses similar strokes to match the corre- chanical methods to model brushes from a physi- sponding topological structure. Shi et al. (2014) cal viewpoint (Lee, 1999; Saito and Nakajima, 1999; modeled Chinese character components, constructed Baxter et al., 2001)). In addition, Chu and Chiew- a dynamic Bayes model, and adjusted character gen- Lan Tai (2004) and Chu and Tai (2005) installed eration by applying the condition equation. Based sensors on the brush to simulate the movement of on texture mapping, Yu and Peng (2005) proposed the brush in three-dimensional space. For the ink a method for generating cursive script, and Markov diffusion effect, Guo and Kuniiunedited (1991) considered interpolation was used for texture synthesis. Dong the paper fiber structure to simulate the dynamic et al. (2008) considered a calligraphy simulation diffusion of ink, and Lee (1999) proposed a Chinese based on statistical models. Li et al. (2014) proposed art paper model using a network structure and ink a WF-histogram to measure a characterąŕs topolog- diffusion algorithm.