Learning to Select, Track, and Generate for Data-to-Text Hayate Iso† Yui Uehara♠ Tatsuya Ishigaki♠♦ Hiroshi Noji♠ Eiji Aramaki†♠ Ichiro Kobayashi♠❤ Yusuke Miyao♠♣ Naoaki Okazaki♠♦ Hiroya Takamura♠♦ †NAIST ♠Artificial Intelligence Research Center, AIST ♦TITech ❤Ochanomizu Univ. ♣UTokyo TEAM H/V WIN LOSS PTS REB AST FG PCT FG3 PCT ... The Milwaukee Bucks defeated the New York Knicks, 105-104, at Madison Square Garden on Wednesday. The Summary KNICKS H 16 19 104 46 26 45 46 ... Knicks (16-19) checked in to Wednesday’s contest looking BUCKS V 18 16 105 42 20 47 32 ... to snap a five-game losing streak and heading into the fourth quarter, they looked like they were well on their way to that goal. ... Antetokounmpo led the Bucks with 27 points, 13 ✓ Tracking entity states while summarizing table information PLAYER H/V PTS REB AST BLK STL MIN CITY ... rebounds, four assists, a steal and three blocks, his second consecutive double-double. Greg Monroe actually checked CARMELO ANTHONY H 0 37 NEW YORK ... 30 11 7 2 in as the second-leading scorer and did so in his customary DERRICK ROSE H 15 3 4 0133NEW YORK ... improves adequacy while preserving fluency RG benchCS role, posting 18 points,CO along with nine boards, four COURTNEY LEE H 11 2 3 1138NEW YORK ... Method assists, three steals and a block. JabariBLEU Parker contributed GIANNIS ANTETOKOUNMPO V 27 13 4 3 139MILWAUKEE ... # P% P%15 R%points, four F1%rebounds,DLD%three assists and a steal. Malcolm ✓ Incorporating the writer information is also helpful for GREG MONROE V 18 9 4 1 3 31 MILWAUKEE ... Brogdon went for 12 points, eight assists and six rebounds. JABARI PARKER V 15 4 3 0137MILWAUKEE ... Mirza Teletovic was productive in a reserve role as well, MALCOLM BROGDONPuduppullyV 12 et6 al. (820190038M) 33.06 83.17ILWAUKEE33.06... 43.59 37.60 16.97 13.96 selecting appropriate data and writing better summary generating 13 points and a rebound. ... Courtney Lee MIRZA TELETOVIC V 13 100021MILWAUKEE ... + w in stage 1 28.43 84.75 45.00checked 49.73 in with 47.2511 points,22.16three assists,18.18two rebounds, a JOHN HENSON V220 0 014MILWAUKEE ... steal and a block. ... The Bucks and Knicks face off once ... ... ... ... ... ... ... + w in stage 2 35.06 80.51 31.10again 45.28 in the 36.87 second game16.38 of the home-and-home17.81 series, with + w in stage 1 & 2 28.00 82.27 44.37the 48.71 meeting 46.44 taking place Friday night in Milwaukee. (a) Box score: Top contingency table shows number of wins and 22.41 18.90 losses and summary of each game. Bottom table shows statistics Data-to-Text Generation PROPOSED(a) Box-score 39.05 94.38 35.77(b) 52.05 NBA basketball 42.40(b) game 19.38Gold summary:summary16.15 Each summary of each player such as points scored (PLAYER’s P TS), and total rebounds (PLAYER+ w’s R EB). 30.25 92.00 50.75consists 59.03 of game 54.58 victory25.75 or defeat20.84 of the game and highlights of valuable players. Task: Given a set of records, the system should produce a fluent Table 4: Effects of writer information. w indicates that WRITER embeddings are used. Numbers in bold are the Table 1: Example of input and output data: task defines box score (1a) used for input and summary document of largest among the variants ofGenerated each method. summary and an adequate summary. game (1b) used as output. Extracted entities are shown in bold face. Extracted values are shown in green. The Milwaukee Bucks defeated the New York Knicks, 105-104, at Problem: t The Milwaukee199 Bucks 200defeated the New 201 York Knicks 202, 105- 203Madison 204 Square Garden 205 on Saturday. 206 The Bucks 207(18-16 208) checked in 209 104, at Madison Square Garden on Wednesday evening. The Y Jabari Parker contributed 15 pointsto Saturday’s , contest four with a well, rebounds outscoring ,the Knicks three(16-19) by assists ‣ Template system could generate the adequate text but not fluent t Bucks (18-16) have been one of the hottest teams in the league, a margin of 39-19 in the first quarter. However, New York by just a having won five of their last six games, and they have now won Z 1101001001025-foot lead at the end of the first quarter, the Bucks were able to pull t six of their last eight games. The Knicks (16-19) have now ‣ JABARI JABARI JABARI away, as they outscoredJABARI the Knicks by a 59-46 marginJABARI into the second. NN based system could generate the fluent text but not adequate Et won six of their last six games, as they continue- to battle for the -- -- - PARKER PARKER PARKER 45 points in theP thirdARKER quarter to seal the win for NewP YorkARKER with the eighth and final playoff spot in the Eastern Conference. Giannis At FIRST NAME LAST NAME -PLAYER PTS --Prest of the startersLAYER to sealREB the win.--P The Knicks wereLAYER led by AGiannisST - Antetokounmpo led the way for Milwaukee, as he tallied 27 , who tallied a game-high points, to go along with Nt ---0--1--1-Antetokounmpo 27 Adequacy Fluency points, 13 rebounds, four assists, three blocked shots and one 13 rebounds, four assists, three blocks and a steal. The game was steal, in 39 minutes . Jabari Parker added 15 points, four re- a crucial night for the Bucks’ starting five, as the duo was the most Table 2:bounds, Runningthree assists, exampleone steal of and ourone model’sblock, and 6 generation-of-8 from process.effective shooters, At every as they time posted step Milwaukeet, model to go onpredicts a pair of eachlow random Template ✔ × long range. John Henson added two points, two rebounds, one low-wise (Carmelo Anthony) and Malcolm Brogdon. Anthony added variable.assist, Modelthree steals firstly and one determinesblock. John whetherHenson was to the refer only to data11 rebounds, recordsseven (Ztassists=1 and) ortwo notsteals (Z tot his=0 team-high). If random scoring total. variable other player to score in double digits for the Knicks, with 15 Jabari Parker was right behind him with 15 points, four rebounds, Zt =1, model selects entity Et, its attribute At and binary variables Nt if needed. For example, at t = 202, model NN × ✔ points, four assists, three rebounds and one steal, in 33 min- three assists and a block. Greg Monroe was next with a bench-leading predictsutes. random The Bucks variable were led byZDerrick202 =1 Roseand, who then tallied selects15 the18 entitypoints,J alongABARI with ninePARKERrebounds, andfour assists its attribute and three steals.PLAYER PTS. points, assists, rebounds and steal in minutes. four three one 1533 Brogdon posted 12 points, eight assists, six rebounds and a steal. Ours ✔ ✔ GivenWilly these Hernangomez values, modelstarted outputsin place of tokenPorzingis andfrom finished selectedDerrick data Rose record.and Courtney Lee were next with a pair of 11 / 11 with points, assists, rebounds and steal in { } 15 four three one 33 -point efforts. Rose also supplied four assists and three rebounds, while minutes. started in place of Jose Calderon Willy Hernangomez Lee complemented his scoring with three assists, a rebound and a steal. ( knee ) and responded with rebound and block. The ENT ‣ For document-level generation, the system has an influence of the our training, development,one test datasetone are respec- Johnthat Henson haveand beenMirza Teletovic referredwere to. nexth with a pairis also of two used/ two to up- Knicks were led by their starting backcourt of Carmelo An- { } -point efforts. LM also registered points, and he added a re- tivelythony 2,714,and 534,Carmelo and Anthony 500., but On combined average, for just each13 points sum- date hTeletovic, meaning that13 the referred data records writer’s bias. on 5-of-16 shooting. The Bucks next head to Philadelphia to bound and an assist. Jason Terry supplied eight points, three rebounds mary hastake on 384 the Sixers tokens on Friday and night, 644 while data the Knicks records. remain Each andaffect a pair of steals.the text The Cavs generation. remain in last place in the Eastern home to face the Los Angeles Clippers on Wednesday. Conference’s Atlantic Division. They now head home to face the match has only one summary in our dataset, as TorontoOur Raptors model on Saturday decides night. whether to refer to x, which far as we checked. We also collected the writer data record r x to be mentioned, and how to ex- Entity Tracking 2 of each document.(a)(a) PuduppullyPuduppully Our dataset et al. contains ((2019)2019) 32 differ- press a number.(b)(b) The Our Our selected model data record is used to ENT entTable writers.✔ Fluent 5: Example Thetext most summaries prolific generated writer in with ourPuduppully dataset✔ Fluent etupdate al. ( text2019h)’s model. Formally, (left) and we our use model the four (right). variables: Names Idea: Equipping an NN-based generation system with the wrotein ×bold 607Many face documents. incorrectare salient realizations There entities. (orange-colored areBlue also numbers writers) whoare correct✔ Correct relations realization derived & appropriate from input content data records selection but are not wroteobserved× lessMany in than improper reference ten content documents. summary. selectionsOrange On (blue-colored average, numbers each) are× incorrectDifficult1. Zt: to binaryrelations. capture variable complexGreen relation thatnumbers determines (underlinedare correct) whether relations the entity-tracking module enables capturing of the saliency and writermentioned wrote in 117 reference documents. summary. We call our new model refers to input x at time step t (Zt =1). the coherence while preserving the fluency.