Jiuge: a Human-Machine Collaborative Chinese Classical Poetry Generation System

Jiuge: A Human-Machine Collaborative Chinese Classical Poetry Generation System

Zhipeng Guo1∗ , Xiaoyuan Yi1∗, Maosong Sun1† , Wenhao Li1, Cheng Yang1, Jiannan Liang1, Huimin Chen1, Yuhui Zhang1, Ruoyu Li2 1Department of Computer Science and Technology, Tsinghua University, Beijing, China Institute for Artiﬁcial Intelligence, Tsinghua University, Beijing, China State Key Lab on Intelligent Technology and Systems, Tsinghua University, Beijing, China 26ESTATES PTE LTD, Singapore

Abstract Zhang et al., 2017; Yi et al., 2017). Besides the research value of exploring human writing mech- Research on the automatic generation of po- anism and computer creativity, these models and etry, the treasure of human culture, has lasted for decades. Most existing systems, however, systems could also benefit electronic entertain- are merely model-oriented, which input some ment, advertisement, and poetry education. user-specified keywords and directly complete However, the recently released Chinese poetry the generation process in one pass, with lit- generation systems are mainly model-oriented, tle user participation. We believe that the which take some user inputs and directly complete machine, being a collaborator or an assistant, the generation in one pass, resulting in poor user should not replace human beings in poetic cre- participation. Moreover, these systems generate ation. Therefore, we proposed Jiuge, a human- machine collaborative Chinese classical po- poetry in fewer styles and genres, and provide lim- etry generation system. Unlike previous sys- ited options for users. For example, the Daoxi- tems, Jiuge allows users to revise the unsatis- angju system1 requires the user to determine the fied parts of a generated poem draft repeatedly. rhyme, which creates a barrier for beginners. The According to the revision, the poem will be dy- Oude system2 simplifies the user’s choices and namically updated and regenerated. After the only allows the input of a few options and genres. revision and modification procedure, the user The Microsoft Quatrain3 provides limited candi- can write a satisfying poem together with Ji- dates of a theme and each line, but it only supports uge system collaboratively. Besides, Jiuge can accept multi-modal inputs, such as keywords, the generation of quatrains. plain text or images. By exposing the options Due to the lack of user participation, the above of poetry genres, styles and revision modes, Ji- systems are mainly designed for entertainment. uge, acting as a professional assistant, allows We argue that the leading role in literary creation constant and active participation of users in should not be a machine, or at least not only a ma- poetic creation. chine, because it is difficult for machines to handle 1 Introduction the complex expressions of one’s emotion and the use of images in poetic creation. Language is one of the most important forms of Rather than completely replace humans, a better human intelligence, among different genres, po- way is to utilize the system to assist human cre- etry is a beautiful, poetic and artistic genre which ation. The human-machine collaboration mech- expresses one’s emotions and ideas with relatively anism in Jiuge system can not only improve the fewer words. Across various countries, nationali- emotions and semantics of generated poems but ties, and cultures, poetry is always fascinating, im- also guide and teach beginners to understand the pacting profoundly on the development of human poetic creation process. civilization. In summary, the contributions of our Jiuge sys- Recently, researchers have worked on automatic tem are as follows: poetry generation. Meanwhile, neural networks have proven to be powerful on this task (Zhang • Multi-modal input. Jiuge can accept multi- and Lapata, 2014; Wang et al., 2016; Yan, 2016; 1http://www.poeming.com/web/ ∗ indicates equal contribution 2https://crl.ptopenlab.com:8800/poem/index † Corresponding author: M.Sun([email protected]) 3http://duilian.msra.cn/jueju/ Collaborative Revision Module

Keywords: plane, blue sky

Plain Text: There is a plane in the blue sky. Automatic Reference Recommendation Revision Modes Picture: Static/Local Dynamic/Global Dynamic The swan goose is flying outside the clouds.

Extract Expand The heavy mist almost make the Main Framework Transform ferry invisible. Keywords: fly, blue sky, swan goose, vast Working Memory Model The vast road has extended to Pattern Checking thousands of miles away. Genere Control Re-Ranking Unsupervised Style Control It's so remote that it seems like it reaches the sky. Quatrain, Acrostic, Iambic Genre: Acrostic Poetry Generation Style: Standard, Sadness about seasons, …

Input Preprocessing Module Generation Model Postprocessing Model Final Poetry Figure 1: The architecture of Jiuge system. modal input such as keywords, plain text, and and removes the ones that do not conform to struc- even images. For modern concepts in the in- tural and phonological requirements. At last, the put, Jiuge utilizes a knowledge graph to map collaborative revision module interacts with the them into relevant keywords in classical Chi- user and dynamically updates the draft for several nese poetry. times according to the user’s revision, to collaboratively create a satisfying poem. • Various styles and genres. Unlike previous We detail each module in the following parts. systems, Jiuge provides more than twenty options of genre and ten options of style, and 2.2 Input Preprocessing Module can generate more diverse poems. Keyword Extraction. Jiuge allows multi-modal input to meet the needs of generating poetry ac- • Human-machine collaboration . Jiuge sup- cording to keywords, tweets or photos. ports human-machine collaborative and inter- For plain text, we first use THULAC4 (Li and active generation. The user can revise the un- Sun, 2009) to conduct Chinese word segmentation satisfied parts of a generated poem. In terms and compute the importance r(w) of each word w: of the revision, Jiuge will dynamically update and re-generate the poem. During this pro- r(w)=[α∗ti(w)+(1−α)∗tr(w)], (1) cess, Jiuge also offers candidate words and where ti(w) and tr(w) are the TF-IDF (Term human-authored poetry as references for be- Frequency-Inverse Document Frequency) value ginners. and TextRank (Mihalcea and Tarau, 2004) score 2 Architecture calculated with the whole poetry corpus respectively. α is a hyper-parameter to balance the 2.1 Overview weights of ti(w) and tr(w). Afterwards, we se- We show the overall architecture of Jiuge system lect top K words with the highest scores. in Fig.1, which mainly consists of four modules: For each image, we use the Aliyun image recog- 5 1) input preprocessing, 2) generation, 3) postpro- nition tool , which gives the names of five rec- cessing and 4) collaborative revision. Given the ognized objects with corresponding probability user-specified genre, style, and inputs (keywords, s(w). Then we select top K words with the high- plain text or images), the preprocessing module est s(w) · r(w). extracts several keywords from the inputs and then Keyword Mapping. The extracted or recog- conducts keyword expansion to introduce richer nized keywords could be some modern concepts, information. Jiuge also transforms the words in such as airplane and refrigerator. Since these modern concepts, which are incompatible with words never occur in the classical poetry corpus, classical Chinese poetry (written in ancient Chi- the generation module will take them as a UNK nese language), such as refrigerator and airplane, symbol and generate totally irrelevant poems. to appropriate relevant ones, e.g., airplane → fly. To address this problem, we build a Poetry With these preprocessed keywords, the generation Knowledge Graph (PKG) from Wikipedia data, module generates a poem draft. The postprocess- 4http://thulac.thunlp.org/ ing module re-ranks the candidates of each line 5https://ai.aliyun.com/image PKG PWCG (fly) (egret) ! = 0.33 !"#$% = 25.3 (airplane) (fly) ! = 0.08 !"#$% = 27.1 ! = 0.13 !"#$% = 20.7 (wing) (swan goose) (wind) (butterfly)

(a) (b) Figure 2: (a) A sampled subgraph of PKG. (b) A sampled subgraph of PWCG. which contains 616, 360 entities and 5, 102, 192 relations. 40, 276 of these entities occur in our po- Figure 3: The simplified structure of the working mem- etry corpus. Before keywords extension and selec- ory model, which mainly comprise an encoder, a de- tion, we first use PKG to map the modern concepts coder and there memory components. xi is the i-th line and x is the j-word in the i-th line. Please refer to (Yi to its most relevant entities in poetry, to guarantee i,j et al., 2018b) for more details. both quality and relevance of generated poems. For a modern concept word wi, we score its ory model (Yi et al., 2018b), which takes at most each neighbor word wj by: K preprocessed keywords as input. The encoder maps each word or line into vector representa- N p(wj ) g(wj )=tfwiki(wj |wi) · log( ) · arctan( ), 1+df(wj ) τ tions, and the decoder generates each line word- (2) by-word. The topic memory stores keywords ex- where tfwiki(wj|wi) is the term frequency of wj plicitly and independently, which can learn a flex- in the Wikipedia article of wi, df(wj) is the num- ible order and form of keywords expression. The ber of Wikipedia articles containing wj, N is the history memory and local memory are dynami- number of Wikipedia articles, and p(wj) is the cally read and written to improve the context co- word frequency counted in all articles. We give an herence of generated poems. example of mapping the modern word “airplane” Genere Control. Chinese classical poetry in- in Fig.2(a). volves various genres, and each genre strictly de- Keyword Extension. The generation module fines the structural and phonological pattern of a can handle multi-keywords input. More keywords poem, such as the length of each line, the tone could lead to richer contents and emotions in gen- of each word, and the number of lines. We use erated poems. Therefore, if the number of ex- our designed genre embedding (Yi et al., 2018b) tracted keywords is less than K, we further con- to disentangle the semantic content and the genre duct keywords extension. To this end, we build pattern. The genre embedding indicates the line a Poetry Word Co-occurrence Graph (PWCG) as length, word tone, and rhyme, which is fed to the shown in Fig.2 (b). This graph indicates the co- decoder. By this way, we can train one model with occurrence of two words in the same poem. The all genres of poems and control the genre of gen- weight of the edge between two words is calcu- erated poems by specifying a pattern. lated according to the Pointwise Mutual Informa- Training patterns are automatically extracted tion (PMI) as follows: from the corpus. For generating, we make the p(wi, wj ) genre as a user option. However, the selection of PMI(wi, wj ) = log , (3) p(wi) ∗ p(wj ) rhyme may be difficult for users without relevant where p(wi) and p(wi, wj) are the word frequency literature knowledge. Therefore, we train a classi- and co-occurrence frequency in poetry corpus. For fier (implemented with a feedforward neural net- a given word w, we get all its adjacent words wk work) to predict an appropriate rhyme in terms of in PWCG and select those with higher values of the keywords. log p(wk) ∗ PMI(w, wk) + β ∗ r(wk) where β is Unsupervised Style Control. Besides genres, a hyperparameter. there are also diverse styles in Chinese poetry such as battlefield, romantic, pastoral, etc. For certain 2.3 Generation Module contents or topics, creating different styles of po- As shown in Fig.3, the core component of the etry is one main user requirement. Since the la- generation module is our proposed working mem- belled data is quite rare and expensive, we use our proposed style disentanglement model (Yang Pattern Checking. The genre embedding intro- et al., 2018) to achieve unsupervised style control. duced in Sec. 2.3 cannot guarantee that generated This method disentangles the style space into M poems perfectly adhere to required patterns. Thus, different sub-spaces by maximizing the mutual in- we further remove the invalid candidates accord- formation between the style distribution and the ing to the specified length, rhythm, and rhyme. generated poetry distribution. Re-Ranking. Our preliminary experiments It is noteworthy that this method is transparent show that the best candidate may not be ranked as to model structures which can be applied to any the top 1 because the training objective is Maxi- generation model. In this stage, we employ it for mum Likelihood Estimation (MLE), which tends the generation of Chinese quatrain poetry (Jueju), to give the generic and meaningless candidates which will be extended to more genres in the fu- lower costs (Yi et al., 2018a). To automatically ture. We set the number of styles M = 10. Af- select the best candidate, we adopt the automatic ter training, we manually annotate each style with rewarders we proposed in (Yi et al., 2018a), in- some descriptive phrases, such as sorrow during cluding a fluency rewarder, a context coherence drinking and rural scenes, to indicate the theme of rewarder, and a meaningfulness rewarder. Then the corresponding style. The style selection is also the candidate with the highest weighted-average set as a user option. rewards given by them will be selected. Acrostic Poetry Generation In Chinese po- 2.5 Collaborative Revision Module etry, there is another special genre called acrostic poetry. Given a sequence of words seq = We call the poem generated by the generation module in one pass the draft, since the user may (x0,0, x1,0, ··· , xn,0), which could be someone’s name or a blessing sentence, the author is re- revise it for several times to collaboratively create a satisfying poem together with the machine. quired to create a poem using each word xi,0 as We implement such collaboration with a revision the first word of each line xi and the created poem should also conform to the genre pattern and con- module. vey proper semantic meanings. Revision Modes. Define a n-line poem draft The input for this genre is the sequence seq. as X = (x1, x2, ··· , xn), and each line containing l words as x = (x , x , ..., x ). At ev- As our generation module takes keywords as in- i i i,1 i,2 i,li put, we first use pre-trained word2vec embeddings ery turn, the user can revise one word in the draft. (Mikolov et al., 2013) to get K keywords related Then the revision module returns the revision in- to seq according to the cosine distance of each formation to the generation module which updates keyword and the words in seq. Then we directly the draft according to the revised word. We implement three revision modes in terms of the updating feed each xi,0 into the decoder at the first step. way: static, local dynamic, and global dynamic. To alleviate the disfluency caused by this constraint, we generate the second word with • Static updating mode. The revision is re- the conditional probability: pgen(xi,1|xi,0) = quired to meet the phonological pattern of the pdec(xi,1|xi,0) + δ ∗ plm(xi,1|xi,0), where pdec and draft and the draft will not be updated except plm are probability distributions of the decoder the revised word. The rhythm and rhyme in- and a neural language model respectively. formation is given to the user together with If the length of the input sequence is less than the generated draft and the invalid revision n (the number of lines in a poem), we also use the will be alerted. During the beam search pro- language model to extend it to n words. cess, we also store top 10 candidate words in each position as recommendations. 2.4 Postprocessing Module • Local dynamic updating mode. If the Jiuge takes a line-to-line generation schema and user revises word xi,j, then Jiuge will generates each line with beam search (beam re-generate the succeeding subsequence, size=B). As a result, we can get B candidates for xi,j+1, ··· , xi,li , in the i-th line by feeding each line. We design a postprocessing module to the revised word to the decoder for the re- automatically check and re-rank these candidates, vised position. and then select the best one, which is used for the generation of subsequent lines in a poem. • Global dynamic updating mode. If the user revises word xi,j, Jiuge will re-generate all succeeding words,

xi,j+1, ··· , xi,li , ··· , xn,ln . In terms of the revision position, e.g., the rhymed positions, a new phonological pattern may be adopted. Thanks to this collaborative revision-updating process, the user can choose a mode and gradually revise the draft until she/he feels satisfied. Automatic Reference Recommendation. For poetry writing beginners or these lacking professional knowledge, it is hard to revise the draft ap- propriately. To aid in the revision process, we Figure 4: The initial page of Jiuge System. implement an automatic recommendation compo- In order to help the user collaboratively revise nent. This component searches several human- the generated poem, Jiuge provides some high- authored poems, which are semantically similar quality human-authored poems which are seman- to the generated draft, for the user as references. tically similar to the generated one, as the refer- Then the user could decide how to make revision ences. The user can click the button to change with these references. the references. At this time, the user can select In detail, define a n-line human-created poem as an unsatisfying word to be revised in the draft Y = (y1, ··· , yn) and a relevance scoring func- and then Jiuge will give some recommended re- tion as rel(xi, yi) to give the relevance of two vision candidates. Besides these candidates, other lines. Then we return N poems with the highest choices are also allowed. After selecting the revi- relevance score rs(X,Y ), calculated as: sion mode introduced in Sec. 2.5, Jiuge will update n X or re-generate the draft according to the revised rs(X,Y )= rel(xi, yi) + γ ∗ I(Y ∈ Dmaster), (4) word. Through several turns of collaborative revi- i=1 sion, the user and Jiuge work together to create a where D is a poetry set of masterpieces, I master satisfying poem, as in Fig.5 (b) 8. is the indicator function, and the hyper-parameter In addition, Jiuge also supports picture sharing. γ is specified to balance the quality and relevance By clicking the “Share Poetry” button, Jiuge will of searched poems. For more details of the rele- print the created poem on a beautiful picture so vance scoring function, we refer the reader to our that the user can share it with others. previous work (Liang et al., 2018). 3 Demonstration 4 Conclusion and Future Work We implement a webpage version of Jiuge6 which We demonstrate Jiuge, a human-machine collab- allows users to create diverse poems and share orative Chinese classical poetry generation sys- with the others conveniently. The initial page pro- tem. Our system accepts multi-modal input and vides some basic options: multi-modal input and allows deep user participation in the choice of var- the selection of genre and style, as shown in Fig.4. ious styles and genres, as well as in collaborative Jiuge is easy to use. After selecting the in- revision. With an easy-to-use web interface, the put mode and providing corresponding content, user can collaboratively create a satisfying poem the user can further choose a favourite genre and together with the system. Instead of being a simple style7. As shown in Fig.5 (a), the user inputs two entertainment software, Jiuge takes a step towards keywords, desert and cavalry, and chooses to gen- professional AI assistant for poetic education. erate a quatrain with the normal style. Then the We collect a large number of human usage user clicks the “Generate Poetry” button. After a records from the interface, laying the foundation few seconds, the system returns the processed key- for enhancing the collaborative creation method words and a poem draft. in the future. We will also continue to integrate more styles and extend the collaborative creation 6Our system is available at https://jiuge. thunlp.cn/ . 8The poems in Fig.5 are manually translated for better 7Temporarily only the Jueju genre supports multiple demonstration. In practice, we utilize a neural translation styles. model to conduct automatic translation. (a) (b) Figure 5: The collaborative creation page: an example of revision. (a) The poem draft generated in one pass. (b) The poem after four turns of revision and updating. to more genres. Rui Yan. 2016. i,poet:automatic poetry composition through recurrent neural networks with iterative pol- 5 Acknowledgements ishing schema. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelli- gence, pages 2238–2244, New York, USA. We would like to thank Cunchao Tu, Yunqiu Shao and anonymous reviewers for their insightful com- Cheng Yang, Maosong Sun, Xiaoyuan Yi, and Wenhao ments. It is supported by Natural Science Foun- Li. 2018. Stylistic chinese poetry generation via unsupervised style disentanglement. In Proceedings of dation of China (NSFC) Grant 61532001 and the the 2018 Conference on Empirical Methods in Nat- NExT++ project, the National Research Founda- ural Language Processing, pages 3960–3969, Brus- tion, Prime Ministers Office, Singapore under its sels, Belgium. IRC@Singapore Funding Initiative. Xiaoyuan Yi, Ruoyu Li, and Maosong Sun. 2017. Gen- erating chinese plassical poems with rnn encoder- decoder. In Proceedings of the Sixteenth Chinese References Computational Linguistics, pages 211–223, Nan- jing, China. Zhongguo Li and Maosong Sun. 2009. Punctuation as implicit annotations for chinese word segmentation. Xiaoyuan Yi, Maosong Sun, Ruoyu Li, and Wenhao computational linguistics. Computational Linguis- Li. 2018a. Automatic poetry generation with mu- tics, 35(4):505–512,. tual reinforcement learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Jiannan Liang, Maosong Sun, Xiaoyuan Yi, Cheng Language Processing, pages 3143–3153, Brussels, Yang, Huimin Chen, and Zhenghao Liu. 2018. Neu- Belgium. ral network-based jiju poetry generation. In Pro- ceedings of the Seventeenth China National Con- Xiaoyuan Yi, Maosong Sun, Ruoyu Li, and Zong- ference on Computational Linguistics, Changsha, han Yang. 2018b. Chinese poetry generation with China. a working memory mode. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pages 4553–4559, Stock- Rada Mihalcea and Paul Tarau. 2004. Textrank: Bring- holm, Sweden. ing order into text. In Proceedings of the 2004 Con- ference on Empirical Methods in Natural Language Jiyuan Zhang, Yang Feng, Dong Wang, Yang Wang, Processing. Andrew Abel, Shiyue Zhang, and Andi Zhang. 2017. Flexible and creative chinese poetry gener- Tomas Mikolov, Kai Chen, Greg Corrado, and Jef- ation using neural memory. In Proceedings of the frey Dean. 2013. Efficient estimation of word 55th Annual Meeting of the Association for Compu- representations in vector space. arXiv preprint tational Linguistics, pages 1364–1373. Association arXiv:1301.3781. for Computational Linguistics.

Zhe Wang, Wei He, Hua Wu nad Haiyang Wu, Wei Xingxing Zhang and Mirella Lapata. 2014. Chinese Li, Haifeng Wang, and Enhong Chen. 2016. Chi- poetry generation with recurrent neural networks. nese poetry generation with planning based neu- In Proceedings of the 2014 Conference on Empiri- ral network. In Proceedings of COLING 2016, cal Methods in Natural Language Processing, pages the 26th International Conference on Computational 670–680, Doha, Qatar. Linguistics:Technical Papers, pages 1051–1060, Os- aka, Japan.