Grammar Introduction Into Markov Chain Text Generation

ICS 661: Adanced AI Final Project

Grammar Introduction into Markov Chain Text Generation

Curran Meek December 12, 2019

1 Contents

1 Introduction 3

2 Technical Discussion 4 2.1 Markov Chain Implementation ...... 4 2.2 Grammar Rule Implementation ...... 5 2.3 Data ...... 7

3 Analysis 7 3.1 Results ...... 7 3.2 Performance ...... 8

4 Conclusion 8 4.1 Potential Improvements ...... 9

Appendices 11

A Code 11

B Data - Star Wars Dialog 14

C Test Runs 26 C.1 10 Sentence Data Runs - No Rules ...... 26 C.2 10 Sentence Data Runs - Some Rules ...... 27 C.3 10 Sentence Data Runs - All Rules ...... 27 C.4 Full Dialog Runs - No Rules ...... 28 C.5 Full Dialog Runs - Some Rules ...... 29 C.6 Full Dialog Runs - All Rules ...... 29

2 1 Introduction

The computational power in today’s world is seemingly limitless. With cloud computing, fast processors and graphics cards, it is relatively cheap and easy to train neural networks or process lots of data in parallel. All while being accomplished in a timely manner. This is not always the case in the robotic domain. Often times robots are sent to places of limited communication availability. Moreover, a robot can only have so much computational power without sacrificing other on-board systems. The limitations of robots require simple, low complexity solutions for algorithms they employ. These constraints have motivated me to find a text generation approach that could be easily em- ployed on a robot. A potential application could be the robot sending generated text on what it has observed instead of entire image files. With these concepts in mind, the goal of this project is to improve upon a low complexity text generation algorithm by using simple English grammar rules. Utilizing grammar in the generation of text will potentially make a small text dataset more robust, providing more coherent generated sentences. Many current robust text generation methods rely on neural networks. For instance, a recurrent neural network (RNN) can be used to generate coherent cooking instructions using a checklist to model global coherence [1]. RNN and generative adversarial networks (GAN) have been shown to have comparable performance in text generation [2]. Despite their success, neural networks often require large amounts of training data to have a generalized model. Large datasets are not always readily available for specific domain applications and can be very time consuming to generate. Furthermore, neural networks require large computational power to use, even during training. The straightforward algorithm explored is Markov Chain text generation. The Markov Chain model gives structure to how random variables can change from one state to the next [3]. Each variable has a related probability on what state will occur next, this can be graphically seen in Figure 1. The Markov Chain makes a powerful assumption that to predict the next state, only the current state is relevant [3]. This assumption makes the model simplified but looses past information that can be useful. The assumption can be mathematically expressed as

P (qi = a|q1 . . . qi−1) = P (qi = a|qi−1)

where qi is the state variable in the sequence, while a is the value taken in that state [3]. Regardless of the simplicity, Markov Chain text generation can produce believable comprehensive text. Internet users believed generated texted, using Markov Chain, was written by a human approximately 20-40% of the time [4]. The results varied based on the background of the individuals, but demonstrates how this method of text generation is capable of producing understandable text [4]. Markov Chain text generation is expanded upon by introducing simple grammatical rules in the training data. The rules add additional words to the training data based on the grammatical rule. To accurately implement these rules the Hunpos tagger is used to tag the text [5]. Using this methodology the original and modiﬁed text generators are tested and evaluated, as discussed in the following sections.

3 Figure 1: The Markov Chain model is a framework for how to predict a sequence of random variables based on related probability. These variables can be words or symbols to represent a number of phenomenon [3]. The weather for each day could be predicted as shown in (a), or the next word in a sentence could be predicted as shown in (b). The circle represent the current value of the variable, while the lines show the probability the variable will be the another state next in the sequence. Figure from [3].

2 Technical Discussion

2.1 Markov Chain Implementation To generate text using the Markov Chain model requires a set of probabilities of a word appearing after another. In order to do this a dictionary model was used in Python. This dictionary contains start and end words that signify the beginning and end of a sentence. While the words that follow are accumulated in the dictionary by analyzing some text data. Figure 2 shows a simple two sentence training set. In this case two start words and two end words are generated. The probability that a word follows another is represented as the number of times a word appears in the dictionary as the following word [6]. For example, as shown in Figure 2, eat is followed by apples or oranges giving a 50% chance to each. In the dictionary this would be represented as {...,0 eat0 :[apples, oranges],...}. Using this representation text data can be broken down into this dictonary model in the training portion.

Figure 2: A transition diagram depicting how a Markov Chain probabilistic predictions work on a sequence of words, based on two diﬀerent sentences. [6].

4 To preform the training in the code a standard logic ﬂow was used. As described in [6], each line, or sentence, is iterated through saving each word and the word that follows it. Once the end of the sentence is reached the word is saved as an end word, then the next line is analyzed. Any word that has been previously seen has the word following it added to its portion of the dictionary to represent the probability that a word follows it [6]. As discussed if ’best’ follows ’the’ three times the dictionary for ’the’ would be {...,0 the0 :[best, best, best],...}. Figure 3 shows the code used to train the model [6].

Figure 3: Python code used in training the Markov Chain model. The code extracts start words, iterating through the sentence capturing each word, then extract a stop word. These are all saved in a dictionary format that is used throughout the program. This code used the same logic as [6]

Generating text is straightforward with the dictionary created. Once again following the methodology from [6], the program begins by choosing a random start word. Then the next word is generated by selecting a random word in the dictionary for the current word. Once an end word is generated the sentence is ended [6]. Figure 4 shows the code used when generating a sentence [6].

2.2 Grammar Rule Implementation These grammatical rules are based on simple rules as shown in Figure 5 [3]. The rules are generated by expanding out the rules shown in Figure 5 and using the most common occurrences, with the addition of common use of adjectives. Since the model predicts based on the current word, the grammar rules were simplified to include what type of word is followed by another. It was found that the following rules were the more common occurrences: Det → Noun, JJ → Noun, JJ → JJ, NNP/PRP → VB, where JJ represents an adjective, NNP is a proper noun, and PRP is a pronoun. Using these rules the dictionary created during training is modified to include additional words based on the rules. In this case, the probability of words that fit the grammar rule is increased by adding a repeat word in the dictionary. Utilizing the earlier example, if ’the’ is followed by ’best’ and ’dog’ the resulting dictionary entry, with the rules applied, would be {...,0 the0 :[best, dog, dog],...}. The additional occurrence of ’dog’ results from ’the dog’ fitting the rule: Det → Noun. In order to correctly determine each words grammatical role, the Hunpos PoS tagger was used [5]. Hunpos provides a very simple accurate method for PoS tagging. In an analysis of various

5 Figure 4: Python code used in generating text from the dictionary created during training. The code begins with a random start words and chooses words randomly until an end word is encountered. This code is from [6].

Figure 5: Simple English grammar rules from [3].

6 Figure 6: Comparative graph of accuracy and computation time of various part of speech taggers. Hunpos can be seen to have high accuracy with low computational time required. Figure from [7].

PoS tagging methods Hunpos was found to be very accurate with orders of magnitude of less time needed for use and training [7]. This can be clearly seen in Figure 6. Having prior experience in using Hunpos made the integration into the code simple.

2.3 Data The data used for testing was lines of dialog from Star Wars Episode VI. This data was obtained from [8] which was reformatted to separate each line of dialog on separate lines. The text was then modiﬁed by removing the names of characters and removing one word or two word sentences. The text ﬁle used can be found in Appendix B.

3 Analysis

3.1 Results To evaluate the performance of adding grammar rules human evaluation was used. The program was run with varying testing data size, as well as with or without the grammar rules. Each run produced 10 sentences. These sentences were evaluated for resembling something someone might say it. Overall contextual comprehension was not considered, nor were two word or less sentences. The sentences were graded on a 0 or 1 scale, 0 meaning the sentence was not adequate, and 1 meaning the sentence was adequate. The results of ﬁve runs of 10 sentences each are shown in Table 1 and Table 2. Each score represents how many sentences, out of 10, were adequate in the run. As seen in Table 1, adding some rules, not all, improved sentence coherence by about 10%. While when adding even more rules caused a degradation in performance. This is most likely to over constraining the words and producing erroneous combinations. Using a larger dataset increased overall coherence of sentences generated, as seen in Table 2. The introduction of grammar rules were run twice to help ensure

7 Table 1: 10 Sentence Data-set Results No Rules Rules Added (JJ and DT) Rules Added (JJ, DT, NNP) 6 4 3 2 6 6 2 2 5 3 4 2 3 3 1 3.4(Avg) 3.8(Avg) 3.4(Avg)

Table 2: Full Dialog Sentence Data-set Results No Rules Rules Added (JJ and DT) Rules Added (JJ, DT, NNP) 3 6 5 3 4 7 5 6 6 3 6 4 6 5 6 6 7 6 3 6 5 5 5 4 3 5.4(Avg) 5.5(Avg) 4.3(Avg) repeatably of the results. The same behavior is seen where adding a few rules is a slight increase in performance while adding all the rules degrades the performance.

3.2 Performance

The goal of this project was to improve text generation with the addition of grammar rules using small datasets. It was found that the introduction of grammar rules in Markov Chain text generation does improve sentence coherence. The grammar rules that are added need to be carefully selected as seen throughout the testing. It is important that only the most common of grammar rules are added to prevent degradation in performance. This decline could be due to adding to many incorrect word combinations, that are unable to be accounted for because of the Markov assumption. All in all, it would seem that adding some grammar rules can improve the performance of text generation with small datasets. These results were compared to sentences generated from a Markov chain with a very large dataset. The sentences generated from this larger dataset had similar coherence to the text generation with no rules added. A more rigorous comparison could provide additional insight, but it does seem larger training data increases performance.

4 Conclusion

Using Markov Chain as a model for text generation is a great introduction into the domain. This type of model provides a simple low complexity solution for generating text. The performance of this approach was improved by introducing simple grammar rules during the training of the model. These rules showed a raise in performance but if not selected carefully can degrade the sentence output.

8 4.1 Potential Improvements Several concepts could be used to improve the methodology developed in this project. Firstly, the human evaluation of the results could be improved. For example, the rating of each sentence could be expanded using a Linkert scale. Furthermore, evaluating more data could give better indication of performance. Secondly, the grammar rules were chosen using simpliﬁed English grammar. This could be improved by ﬁnding the most statistically relevant grammar rules found in everyday speech. Finally varying the type of text data used could provide another potential improvement.

9 References

[1] C. Kiddon, L. Zettlemoyer, and Y. Choi, “Globally coherent text generation with neural checklist models,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 329–339, 2016.

[2] P. Kawthekar, R. Rewari, and S. Bhooshan, “Evaluating generative models for text generation,” 2017.

[3] C. Parsing, “Speech and language processing,” 2009.

[4] R. M. Everett, J. R. Nurse, and A. Erola, “The anatomy of online deception: What makes automated text convincing?,” in Proceedings of the 31st Annual ACM symposium on applied computing, pp. 1115–1120, ACM, 2016.

[5] P. Hal´acsy, A. Kornai, and C. Oravecz, “Hunpos: an open source trigram tagger,” in Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions, pp. 209– 212, Association for Computational Linguistics, 2007.

[6] R. Alami, “How i generated inspirational quotes with less than 20 lines of python code.” https://codeburst.io/ how-i-generated-inspirational-quotes-with-less-than-20-lines-of-code-38273623c905. Accessed 12/3/2019.

[7] T. Horsmann, N. Erbs, and T. Zesch, “Fast or accurate?- a comparative evaluation of pos tagging models.,” in GSCL, pp. 22–30, 2015.

[8] G. Sanchez, “Star wars.” https://github.com/gastonstat/StarWars. Accessed 12/3/2019.

10 Appendices

A Code

Listing 1: Text Generation Code import n l t k import re import sys import getopt import random from n l t k . tag import HunposTagger

#get file input and output from arguments def inputs ( ) : i f len ( sys . argv ) < 2 : print ( ”Use options −i −o ”) e x i t ( 0 ) else : arg = {} options = getopt.getopt(sys.argv[1:] , ’i:o’) for item in options [0]: i f ( item ) : arg[ item[0]] = item[1]

outFile = arg[’−o ’ ] inFile = arg[’−i ’ ]

return (inFile , outFile)

#Generate a dictionary of words with start/end words and next words #def markovData(inFile , dictType = { ’START ’ : [ ] , ’END ’ : [ ] } ):

#for i, element in def genData(inFile ):

#Dictionary is of style as defined by model model = { ’START’ : [ ] , ’END’ : [ ] }

#Open training data with open(inFile) as dataset: for l i n e in dataset : text = line.lower().split() #make characters lowercase and split words

for i , word in enumerate( t e x t ) : i f i == len ( t e x t ) −1: model[ ’END’ ]. append(word) #If at end of line, store end word else : i f i == 0 : model[ ’START’ ]. append(word) #If at start of line #store as start word i f word in model : model[word].append(text [ i+1]) #If word is in dictionary, add #following word as relation

11 #to known word else : model[word] = [text[i+1]] #If word is not in dictionary #add to next word #print model return model def genTags(inFile ):

data = open( i n F i l e ) d = data.read() text = nltk.word tokenize(d) #Tokenize text for hunpos #Load Hunpos model location hpt = HunposTagger( ’/home/curran/Documents/AdvAI WS/a3/hunpos −1.0− linux/english .model’) #Array of words and corresponding tags tags = hpt.tag(text)

return tags def genText(model): generate = []

while True : i f not generate : words = model[ ’START’] e l i f generate [ −1] in model[ ’END’ ]: break else : words = model[generate[ −1]] generate .append(random. choice(words))

return generate def modDict(model, tags):

seen = [ ]

for word in model : #iterate through dictionary for i in range ( len ( tags ) ) : #iterate through array of tags i f word == tags[i ][0]:

i f word not in seen :

#track words that have been observed to prevent repeats seen .append(word)

#If statements to add word entries based on simplified L1 grammar ’’’ if ’VB’ in tags[i][1]: for j in range(len(tags)): if ’NN’ in tags[j][1]: if tags[j+1][1] == ’.’: model[word].append(tags[j ][0]. lower()+’.’) e l s e :

12 model[word].append(tags[j ][0]. lower()) ’’’ i f ’DT’ in tags[i][1]: for j in range ( len ( tags ) ) : i f ’NN’ in tags [ j ] [ 1 ] and ”’” not in tags[j][0]: i f tags[j+1][1] == ’.’: model[word].append(tags[j ][0]. lower()+’. ’) e l i f tags[j+1][1] == ’,’: model[word].append(tags[j ][0]. lower()+’ , ’) else : model[word].append(tags[j ][0]. lower()) e l i f ’JJ’ in tags [ j ] [ 1 ] and ”’” not in tags[j][0]: i f tags[j+1][1] == ’.’: model[word].append(tags[j ][0]. lower()+’. ’) e l i f tags[j+1][1] == ’,’: model[word].append(tags[j ][0]. lower()+’ , ’) else : model[word].append(tags[j ][0]. lower()) e l i f ’JJ’ in tags[i][1]: for j in range ( len ( tags ) ) : i f ’NN’ in tags [ j ] [ 1 ] and ”’” not in tags[j][0]: i f tags[j+1][1] == ’.’: model[word].append(tags[j ][0]. lower()+’. ’) e l i f tags[j+1][1] == ’,’: model[word].append(tags[j ][0]. lower()+’ , ’) else : model[word].append(tags[j ][0]. lower()) ’’’elif ’JJ’ in tags[j][1] and ”’” not in tags[j][0]: if tags[j+1][1] == ’.’: model[word].append(tags[j ][0]. lower()+’.’) elif tags[j+1][1] == ’,’: model[word].append(tags[j ][0]. lower()+’,’) e l s e : model[word].append(tags[j ][0]. lower()) elif ’PRP’ in tags[i][1]: for j in range(len(tags)): if ’VB’ in tags[j][1] and ”’” not in tags[j][0]: if tags[j+1][1] == ’.’: model[word].append(tags[j ][0]. lower()+’.’) elif tags[j+1][1] == ’,’: model[word].append(tags[j ][0]. lower()+’,’) e l s e : model[word].append(tags[j ][0].lower()) ’’’

return model def main ( ) :

#load in training text dictionary = genData(”/home/curran/Documents/AdvAI WS/ AdvAI Final Proj /SW Dialog. txt”) #Generate grammar tags with Hunpos rules = genTags(”/home/curran/Documents/AdvAI WS/ AdvAI Final Proj /SW Dialog. txt”) #Add grammar rules as additional probabilities to dictionary newDict = modDict(dictionary , rules)

#Iterate to generate different sentences

13 for i in range ( 1 0 ) : quote = genText(newDict) print ” ”.join(quote) main ( )

B Data - Star Wars Dialog

Command station, this is ST 321. Code Clearance Blue. We’re starting our approach. Deactivate the security shield. The security deflector shield will be deactivated when we have confirmation of your code trans- mission. Stand by You are clear to proceed. We’re starting our approach. Inform the commander that Lord Vader’s shuttle has arrived. Lord Vader, this is an unexpected pleasure. We’re honored by your presence. You may dispense with the pleasantries, Commander. I’m here to put you back on schedule. I assure you, Lord Vader, my men are working as fast as they can. Perhaps I can find new ways to motivate them. I tell you, this station will be operational as planned. The Emperor does not share your optimistic appraisal of the situation. But he asks the impossible. I need more men. Then perhaps you can tell him when he arrives. The Emperor’s coming here? That is correct, Commander. And he is most displeased with your apparent lack of progress. We shall double our efforts. I hope so, Commander, for your sake. The Emperor is not as forgiving as I am. Of course I’m worried. And you should be, too. Lando Calrissian and poor Chewbacca never returned from this awful place. Don’t be so sure. If I told you half the things I’ve heard about this Jabba the Hutt, You’d probably short-circuit. Artoo, are you sure this is the right place? I better knock, I suppose. There doesn’t seem to be anyone there. Let’s go back and tell Master Luke. Tee chuta hhat yudd! Goodness gracious me! Artoo Detoowha bo Seethreepiowha ey toota odd mischka Jabba du Hutt. I don’t think they’re going to let us in, Artoo. We’d better go. Artoo, I really don’t think we should rush into all this. Artoo, wait for me! Just you deliver Master Luke’s message and get us out of here. Die Wanna Wanga! Die Wanna Wauaga. We bring a message to your master, Jabba the Hutt. Gift, what gift? Nee Jabba no badda. Me chaade su goodie.

14 He says that our instructions are to give it only to Jabba himself. I’m terribly sorry. I’m afraid he’s ever so stubborn about these sort of things. Artoo, I have a bad feeling about this. The message, Artoo, the message. Greetings, Exalted One. Allow me to introduce myself. I am Luke Skywalker, Jedi Knight and friend to Captain Solo. I know that you are powerful, mighty Jabba, and that your anger with Solo must be equally powerful. I seek an audience with Your Greatness to bargain for Solo’s life. With your wisdom, I’m sure that we can work out an arrangement which will be mutually beneficial and enable us to avoid any unpleasant confrontation. As a token of my goodwill, I present to you a gift: these two droids. What did he say? Both are hardworking and will serve you well. This can’t be! Artoo, you’re playing the wrong message. There will be no bargain. I will not give up my favorite decoration. I like Captain Solo where he is. And he’s still frozen in carbonite. What could possibly have come over Master Luke. Is it something I did? He never expressed any unhappiness with my work. You are a protocol droid, are you not? I am See-Threepio, human-cy. Yes or no will do. How many languages do you speak? I am fluent in over six million forms of communication. We have been without an interpreter since our master got angry with our last protocol droid and disintegrated him. This protocol droid might be useful. Fit him with a restraining bolt and take him back to His Excellency’s main audience chamber. Artoo, don’t leave me! You’re a feisty little one, but you’ll soon learn some respect. I have need for you on the master’s Sail Barge. And I think you’ll fit in nicely. Na Chuba negatorie Natoota. I have come for the bounty on this Wookiee. At last we have the mighty Chewbacca. Oh, uh, yes, uh, I am here, Your Worshipfulness. The illustrious Jabba bids you welcome and will gladly pay you the reward of twenty- five thousand. I want fifty thousand. Uh, oh but what, what did I say? Uh, the mighty Jabba asks why he must pay fifty thousand. Because he’s holding a thermal detonator. This bounty hunter is my kind of scum. Fearless and inventive. Jabba offers the sum of thirty- five. And I suggest you take it. Just relax for a moment. You’re free of the carbonite. You have hibernation sickness. I can’t see. You eyesight will return in time. Where am I? Who are you? Someone who loves you. I gotta get you out of here. What’s that? I know that laugh. Look, Jabba, I was just on my way to pay you back, but I got a little sidetracked. It’s not my fault. It’s too late for that, Solo. You may have been a good smuggler, but now you’re Bantha fodder. Take him away! Jabba I’ll pay you triple! You’re throwing away a fortune here. Don’t be a fool! Bring her to me. We have powerful friends. You’re gonna regret this. Ohhh, I can’t bear to watch.

15 Chewie? Chewie, is that you? I can’t see, pal. What’s goin’ on? Luke? Luke’s crazy. He can’t even take care of himself, much less rescue anybody. A Jedi Knight? I’m out of it for a little while, everybody gets delusions of grandeur. I’m all right, pal. I’m all right. I must speak with Jabba. You will take me to Jabba now! You serve your master well. And you will be rewarded. Master Luke’s come to rescue me. Luke Skywalker, Jedi Knight. I told you not to admit him. I must be allowed to speak. He must be allowed to speak. You weak-minded fool! He’s using an old Jedi mind trick. You will bring Captain Solo and the Wookiee to me. Your mind powers will not work on me, boy. Nevertheless, I’m taking Captain Solo and his friends. You can either profit by this or be destroyed! It’s your choice. But I warn you not to underestimate my powers. Master Luke, you’re standing on. There will be no bargain, young Jedi. I shall enjoy watching you die. Bring me Solo and the Wookiee. They will all suffer for this outrage. Are you all right? Together again, huh? Wouldn’t miss it. How are we doing? The same as always. That bad, huh? Where’s Leia? His High Exaltedness, the great Jabba the Hutt, has decreed that you are to be terminated immediately. Good, I hate long waits. You will therefore be taken to the Dune Sea and cast into the pit of Carkoon, the nesting place of the all-powerful Sarlacc. Doesn’t sound so bad. In his belly, you will find a new definition of pain and suffering, as you are slowly digested over a thousand years. On second thought, let’s pass on that, huh? You should have bargained, Jabba. That’s the last mistake you’ll ever make. I think my eyes are getting better. Instead of a big dark blur, I see a big light blur. There’s nothing to see. I used to live here, you know. You’re gonna die here, you know. Just stick close to Chewie and Lando. I’ve taken care of everything. Soon you will learn to appreciate me. Oh, I’m terribly sorry Artoo! What are you doing here? Well, I can see you’re serving drinks, but this place is dangerous. They’re going to execute Master Luke and, if we’re not careful, us too! I wish I had your confidence. Victims of the almighty Sarlacc: His Excellency hopes that you will die honorably. But should any of you wish to beg for mercy, the great Jabba the Hutt will now listen to your pleas. Threepio, you tell that slimy piece of worm-ridden filth he’ll get no such pleasure from us. This is your last chance. Free us or die.

16 Move him into position. Put him in. Chewie, you okay? Where is he? I’m okay, pal. Boba Fett?! Boba Fett?! Lando, grab it! Grab me, Chewie! Almost You almost got it! Now easy, easy. Hold me, Chewie. Chewie, give me the gun. Don’t move, Lando. I thought you were blind! It’s all right. A little higher! Just a little higher! Chewie, Pull us up! Okay up, Chewie, up! We gotta get out of here quick. Not my eyes! Get the gun! Point it at the deck! Point it at the deck! Artoo, where are we going? I couldn’t possibly jump. And don’t forget the droids. We’re on our way. I don’t know. All I can see is a lot of blowing sand! That’s all any of us can see. Then I guess I’m getting better. I’ve got to hand it to you, kid, you were pretty good out there. I had a lot of help. Think nothing of it. No, I’m thinking a lot about it. That carbon freeze was the closest thing to dead there is. And it wasn’t just sleepin’. It was a big wide awake nothing! I’ll see you back at the fleet. Why don’t you leave that crate and come with us? I have a promise I have to keep first to an old friend. Guess I owe you some thanks, too, Lando. Figured if I left you frozen like that you’d just give me bad luck the rest of my life, so I might as well get you unfrozen sooner or later. Come on, let’s get off this miserable dust ball. I’ll meet you back at the fleet. The Alliance should be assembled by now. Hey, Luke, thanks. Thanks for comin’ after me. Now I owe you one. That’s right, Artoo. We’re going to the Dagobah system. I have a promise to keep to an old friend. Rise, my friend. The Death Star will be completed on schedule. You have done well, Lord Vader. And now I sense you wish to continue your search for young Skywalker. Yes, my Master. Patience, my friend. In time he will seek you out. And when he does, you must bring him before me. He has grown strong. Only together can we turn him to the dark side of the Force. As you wish. Everything is proceeding as I have foreseen.

17 That face you make. Look I so old to young eyes? No of course not. I do, yes, I do! Sick have I become. Old and weak. When nine hundred years old you reach, look as good you will not. Soon will I rest. Yes, forever sleep. Earned it, I have. Master Yoda, you can’t die. Strong am I with the Force but not that strong! Twilight is upon me and soon night must fall. That is the way of things the way of the Force. But I need your help. I’ve come back to complete the training. No more training do you require. Already know you that which you need. Then I am a Jedi? One thing remains: Vader. You must confront Vader. Then, only then, a Jedi will you be. And confront him you will. Master Yoda is Darth Vader my father? Mmm rest I need. Yoda, I must know. Your father he is. Told you, did he? Unexpected this is, and unfortunate. Unfortunate that I know the truth? Unfortunate that you rushed to face him that incomplete was your training. Not ready for the burden were you. Well, I’m sorry. Remember, a Jedi’s strength flows from the Force. Anger, fear, aggression. The dark side are they. Once you start down the dark path, forever will it dominate your destiny. Luke do not underestimate the powers of the Emperor, or suffer your father’s fate, you will. Luke, when gone am I , the last of the Jedi will you be. Luke, the Force runs strong in your family. Pass on what you have learned, Luke there is another Sky walker. I can’t do it, Artoo. I can’t go on alone. Yoda will always be with you. Why didn’t you tell me? You told me Vader betrayed and murdered my father. You father was seduced by the dark side of the Force. He ceased to be Anakin Skywalker and became Darth Vader. When that happened, the good man who was your father was destroyed. So what I have told you was true from a certain point of view. A certain point of view! Luke, you’re going to find that many of the truths we cling to depend greatly on our own point of view. I don’t blame you for being angry. If I was wrong in what I did, it certainly wouldn’t have been for the first time. You see, what happened to your father was my fault. Anakin was a good friend. When I first knew him, your father was already a great pilot. But I was amazed how strongly the Force was with him. I took it upon myself to train him as a Jedi. I thought that I could instruct him just as well as Yoda. I was wrong. My pride has had terrible consequences for the galaxy. There’s still good in him. I also thought he could be turned back to the good side. It couldn’t be done. He is more machine now than man. Twisted and evil. I can’t do it, Ben. You cannot escape your destiny. I tried to stop him once. I couldn’t do it. Vader humbled you when first you met him, Luke but that experience was part of your training. It taught you, among other things, the value of patience. Had you not been so impatient to defeat Vader then, you could have finished your training here with Yoda. You would have been prepared.

18 But I had to help my friends. And did you help them? It was they who had to save you. You achieved little by rushing back prematurely, I fear. I found out Darth Vader was my father. To be a Jedi, Luke, you must confront and then go beyond the dark side - the side your father couldn’t get past. Impatience is the easiest door - for you, like your father. Only, your father was seduced by what he found on the other side of the door, and you have held firm. You’re no longer so reckless now, Luke. You are strong and patient. And now, you must face Darth Vader again! I can’t kill my own father. Then the Emperor has already won. You were our only hope. Yoda spoke of another. The other he spoke of is your twin sister. But I have no sister. To protect you both from the Emperor, you were hidden from your father when you were born. The Emperor knew, as I did, if Anakin were to have any offspring, they would be a threat to him. That is the reason why your sister remains safely anonymous. Leia’s my sister. Your insight serves you well. Bury your feelings deep down, Luke. They do you credit. But they could be made to serve the Emperor. When your father left, he didn’t know your mother was pregnant. Your mother and I knew he would find out eventually, but we wanted to keep you both as safe as possible, for as long as possible. So I took you to live with my brother Owen on Tatooine and your mother took Leia to live as the daughter of Senator Organa, on Alderaan. The Organa household was high-born and politically quite powerful in that system. Leia became a princess by virtue of lineage no one knew she’d been adopted, of course. But it was a title without real power, since Alderaan had long been a democracy. Even so, the family continued to be politically powerful, and Leia, following in her foster father’s path, became a senator as well. That’s not all she became, of course she became the leader of her cell in the Alliance against the corrupt Empire. And because she had diplomatic immunity, she was a vital link for getting information to the Rebel cause. That’s what she was doing when her path crossed yours for her foster parents had always told her to contact me on Tatooine, if her troubles became desperate. But you can’t let her get involved now, Ben. Vader will destroy her. She hasn’t been trained in the ways of the Jedi the way you have, Luke but the Force is strong with her, as it is with all of your family. There is no avoiding the battle. You must face and destroy Vader! Well, look at you, a general, huh? Oh, well, someone must have told them about my little maneuver at the battle of Taanab. Well, don’t look at me, pal. I just said you were a fair pilot. I didn’t know they were lookin’ for somebody to lead this crazy attack. I’m surprised they didn’t ask you to do it. Well, who says they didn’t. But I ain’t crazy. You’re the respectable one, remember? The Emperor has made a critical error and the time for our attack has come. The data brought to us by the Bothan spies pinpoints the exact location of the Emperor’s new battle station. We also know that the weapon systems of this Death Star are not yet operational. With the Imperial Fleet spread throughout the galaxy in a vain effort to engage us, it is relatively unprotected. But most important of all, we’ve learned that the Emperor himself is personally overseeing the final stages of the construction of this Death Star.

19 Many Bothans died to bring us this information. Admiral Ackbar, please. You can see here the Death Star orbiting the forest Moon of Endor. Although the weapon systems on this Death Star are not yet operational, the Death Star does have a strong defense mechanism. It is protected by an energy shield, which is generated from the nearby forest Moon of Endor. The shield must be deactivated if any attack is to be attempted. Once the shield is down, our cruisers will create a perimeter, while the fighters fly into the superstructure and attempt to knock out the main reactor. General Calrissian has volunteered to lead the fighter attack You’re gonna need it. We have stolen a small Imperial shuttle. Disguised as a cargo ship, and using a secret Imperial code, a strike team will land on the moon and deactivate the shield generator. I wonder who they found to pull that off. General Solo, is your strike team assembled? Uh, my team’s ready. I don’t have a command crew for the shuttle. Well, it’s gonna be rough, pal. I didn’t want to speak for you. Uh, General count me in. I’m with you, too! What is it? Ask me again sometime. Hi, Han Chewie. Exciting is hardly the word I would use. I want you to take her. I mean it. You need all the help you can get. She’s the fastest ship in the fleet. All right, old buddy. You know, I know what she means to you. I’ll take good care of her. She-she won’t get a scratch. I got your promise now. Not a scratch. Look, would you get going, you pirate. You got her warmed? Yeah, she’s comin’ up. I don’t think the Empire had Wookiees in mind when they designed her, Chewie. Hey, are you awake? Yeah, I just got a funny feeling. Like I’m not gonna see her again. Come on, General, let’s move. Chewie, let’s see what this piece of junk can do. Here we go again. All right, hang on. What is thy bidding, my Master? Send the fleet to the far side of Endor. There it will stay until called for. What of the reports of the Rebel fleet massing near Sullust? It is of no concern. Soon the Rebellion will be crushed and young Skywalker will be one of us! Your work here is finished, my friend. Go out to the command ship and await my orders. Yes, my Master. If they don’t go for this, we’re gonna have to get outta here pretty quick, Chewie. We have you on our screen now. Shuttle Tydirium requesting deactivation of the deflector shield. Shuttle Tydirium, transmit the clearance code for shield passage. Now we find out if that code is worth the price we paid. Vader’s on that ship. Now don’t get jittery, Luke. There are a lot of command ships. Keep your distance though, Chewie, but don’t look like you’re trying to keep your distance. I don’t know. I’m endangering the mission. I shouldn’t have come.

20 It’s your imagination, kid. Let’s keep a little optimism here. Where is that shuttle going? Shuttle Tydirium, what is your cargo and destination? Parts and technical crew for the forest moon. Do they have a code clearance? It’s an older code, sir, but it checks out. I was about to clear them. Shall I hold them? Leave them to me. I will deal with them myself. As you wish, my lord. They’re not goin’ for it, Chewie. Shuttle Tydirium, deactivation of the shield will commence immediately. Follow your present course. I told you it was gonna work. Oh, I told you it was dangerous here. Shall we try and go around? It’ll take time. This whole party’ll be for nothing if they see us. Chewie and I will take care of this. You stay here. Quietly, there might be more of them out there. Hey it’s me. Go for help! Two more of them! I see them. Jam their comlink. Get alongside that one! Keep on that one! I’ll take these two! Oh, General Solo, somebody’s coming. What? She didn’t come back? I thought she was with you. We got separated. Hey, we better go look for her. Take the squad ahead. We’ll meet at the shield generator at 0300. Come on, Artoo. We’ll need your scanners. Don’t worry, Master Luke. We know what to do. And you said it was pretty here. Cut it out! I’m not gonna hurt you. Well, looks like I’m stuck here. Trouble is, I don’t know where here is. Well, maybe you can help me. Come on, sit down. I promise I won’t hurt you. Now come here. You want something to eat? Look, it’s a hat. It’s not gonna hurt you. You’re a jittery little thing, aren’t you? What is it? Come on, get up! Go get your ride and take her back to base. Come on, let’s get outta here. The Emperor does not wish to be disturbed at the moment. The Emperor will see me, now! The Emperor will see you, now. I told you to remain on the command ship. A small Rebel force has penetrated the shield and landed on Endor. Yes, I know. My son is with them.

21 Are you sure? I have felt him, my Master. Strange, that I have not. I wonder if your feelings on this matter are clear, Lord Vader. They are clear, my Master. Then you must go to the Sanctuary Moon and wait for them. He will come to me? I have foreseen it. His compassion for you will be his undoing. He will come to you and then you will bring him before me. As you wish. Oh, Master Luke. There’s two more wrecked speeders back there. And I found this. I’m afraid that Artoo’s sensors can find no trace of Princess Leia. I hope she’s alright. What, Chewie? What? Chewie! Hey, I don’t get it. Nah, it’s just a dead animal, Chewie. Chewie, wa-wait! Don’t! Always thinking with your stomach. Will you take it easy? Let’s just figure out a way to get out of this thing. Han, can you reach my lightsaber? Artoo, I’m not sure that’s such a good idea. It’s a very long dro-o-op! Point that thing someplace else. It’ll be all right. Chewie, give ’em your crossbow. Oh, my head. Oh, my goodness! Do you understand anything they’re saying? Oh, yes, Master Luke! Remember that I am fluent in over six million forms of communication. What are you telling them? Hello, I think I could be mistaken. They’re using a very primitive dialect. But I do believe they think I am some sort of god. Well, why don’t you use your divine influence and get us out of this? I beg your pardon, General Solo, but that just wouldn’t be proper. It’s against my programming to impersonate a deity. He’s an old friend of mine. I have a really bad feeling about this. What did he say? I’m rather embarrassed, General Solo, but it appears you are to be the main course at a banquet in my honor. Your Royal Highness. But these are my friends. Threepio, tell them they must be set free. Somehow, I got the feeling that didn’t help us very much. Threepio, tell them if they don’t do as you wish, you’ll become angry and use your magic. But Master Luke, what magic? Just tell them. You see, Master Luke; they didn’t believe me. Just as I said they wouldn’t. Put me down! Somebody, somebody, help! Master Luke, Artoo! Do something, somebody! I never knew I had it in me. I was just coming to that. What’s going on? I don’t know. We are now a part of the tribe. Just what I always wanted. Well, short help is better than no help at all, Chewie. He says the scouts are going to show us the quickest way to the shield generator. How far is it? Ask him. We need some fresh supplies, too. And try and get our weapons back. And hurry up, will ya? I haven’t got all day. Luke, what’s wrong? Leia do you remember your mother? Your real mother? Just a little bit. She died when I was very young.

22 What do you remember? She was very beautiful. Kind, but sad. Why are you asking me all this? I have no memory of my mother. I never knew her. Luke, tell me. What’s troubling you? Vader is here now, on this moon. How do you know? I felt his presence. He’s come for me. He can feel when I’m near. That’s why I have to go. As long as I stay, I’m endangering the group and our mission here. I have to face him. He’s my father. It won’t be easy for you to hear it, but you must. If I don’t make it back, you’re the only hope for the Alliance. Luke, don’t talk that way. You have a power I don’t understand and could never have. You’re wrong, Leia. You have that power too. In time you’ll learn to use it as I have. The Force is strong in my family. My father has it. my sister has it. It’s you Leia. Somehow I’ve always known. Then you know why I have to face him. Luke, run away, far away. If he can feel your presence, then leave this place. I wish I could go with you. No, you don’t. You’ve always been strong. But, why must you confront him? Because there is good in him. I’ve felt it. He won’t turn me over to the Emperor. I can save him. I can turn him back to the good side. I have to try. Hey, what’s goin’ on? I - just want to be alone for a little while. Nothing? Come on, tell me. What’s goin’ on? I can’t tell you. Did you tell Luke? Is that who you could tell? This is a Rebel that surrendered to us. Although he denies it, I believe there may be more of them, and I request permission to conduct a further search of the area. He was armed only with this. Good work, Commander. Conduct your search and bring his companions to me. Yes, my Lord. The Emperor has been expecting you. I know, father. So, you have accepted the truth. I’ve accepted the truth that you were once Anakin Skywalker, my father. That name no longer has any meaning for me. It is the name of your true self. You’ve only forgotten. I know there is good in you. The Emperor hasn’t driven it from you fully. That is why you couldn’t destroy me. That’s why you won’t bring me to your Emperor now. I see you have constructed a new lightsaber. Your skills are complete. Indeed, you are powerful, as the Emperor has foreseen. Come with me. Obi-Wan once thought as you do. You don’t know the power of the dark side. I must obey my master. I will not turn and you’ll be forced to kill me. If that is your destiny. Search your feelings, father. You can’t do this. I feel the conﬂict within you. Let go of your hate. It is too late for me, son. The Emperor will show you the true nature of the Force. He is your master now. Then my father is truly dead.

23 The main entrance to the control bunker’s on the far side of that landing platform. This isn’t gonna be easy. Hey, don’t worry. Chewie and me got into a lot of places more heavily guarded than this. What’s he saying? He says there’s a secret entrance on the other side of the ridge. Admiral, we’re in position. All fighters accounted for. Proceed with the countdown. All groups assume attack coordinates. Don’t worry, my friends are down there. They’ll have that shield down on time or this’ll be the shortest offensive of all time. All craft, prepare to jump to hyperspace on my mark. Back door, huh? Good idea. It’s only a few guards. This shouldn’t be too much trouble. Well, it only takes one to sound the alarm. Then we’ll do it real quiet-like. Uh, Princess Leia! I’m afraid our furry companion has gone and done something rather rash. There goes our surprise attack. Not bad for a little furball. There’s only one left. You stay here. We’ll take care of this. I have decided that we shall stay here. Welcome, young Skywalker. I have been expecting you. You no longer need those. Guards, leave us. I’m looking forward to completing your training. In time you will call me Master. You’re gravely mistaken. You won’t convert me as you did my father. Oh, no, my young Jedi. You will find that it is you who are mistaken about a great many things. Ah, yes, a Jedi’s weapon. Much like your father’s. By now you must know your father can never be turned from the dark side. So will it be with you. Soon I’ll be dead and you with me. Perhaps you refer to the imminent attack of your Rebel fleet. Yes I assure you we are quite safe from your friends here. Your overconfidence is your weakness. Your faith in your friends is yours. It is pointless to resist, my son. Everything that has transpired has done so according to my design. Your friends up there on the Sanctuary Moon are walking into a trap. As is your Rebel fleet! It was I who allowed the Alliance to know the location of the shield generator. It is quite safe from your pitiful little band. An entire legion of my best troops awaits them. Oh I’m afraid the deflector shield will be quite operational when your friends arrive. The fleet will be here any moment. Come on, come on! They’ll be captured! Wait, come back! Artoo, stay with me. You Rebel scum. All wings report in. Red Leader standing by. Gray Leader standing by. Green Leader standing by. Lock S-foils in attack positions. May the Force be with us. We’ve got to be able to get some kind of a reading on that shield, up or down. Well, how could they be jamming us if they don’t know if we’re coming. Break off the attack! The shield is still up. I get no reading. Are you sure? All craft pull up! Take evasive action! Green Group, stick close to holding sector MV-7. Admiral, we have enemy ships in sector 47. It’s a trap! Fighters coming in. There’s too many of them! Accelerate to attack speed! Draw their fire away from the cruisers. Copy, Gold Leader. See for yourself. From here you will witness the final destruction of the Alliance, and the end of your insignificant Rebellion. You want this, don’t you? The hate is swelling in you now. Take your Jedi weapon. I am unarmed. Strike me down with it. Give in to your anger. With each passing moment, you make yourself more my servant. It is unavoidable. It is your destiny. You, like your father, are now mine! All right, move it! I said move it! I say, over there! Were you looking for me? Bring those two down here! Well, they’re on their way. Artoo, are you sure this was a good idea? Stand back, Artoo. The code’s changed. We need Artoo! Here’s the terminal. Artoo, where are you? We need you at the bunker right away. Going? What do you mean, you’re going. But but going where, Artoo? Oh, this is

24 no time for heroics. Watch yourself, Wedge! Three from above! Red Three, Red Two, pull in! Three of them coming in, twenty degrees! Cut to the left! I’ll take the leader! They’re heading for the medical frigate. Only the fighters are attacking. I wonder what those Star Destroyers are waiting for. We’re in attack position now, sir. We’re not going to attack? I have my orders from the Emperor himself. He has something special planned for them. We only need to keep them from escaping. As you can see, my young apprentice, your friends have failed. Now witness the firepower of this fully armed and operational battle station. Fire at will, Commander. That blast came from the Death Star! That thing’s operational! Home One, this is Gold Leader. We saw it. All craft prepare to retreat. You won’t get another chance at this, Admiral. We have no choice, General Calrissian. Our cruisers can’t repel firepower of that magnitude. Han will have that shield down. We’ve got to give him more time. Oh, Artoo, hurry! Artoo, why did you have to be so brave? Well, I suppose I could hotwire this thing. I’ll cover you. I said closer! Move as close as you can and engage those Star Destroyers at point- blank range. At that close range, we won’t last long against those Star Destroyers. We’ll last longer then we will against that Death Star and we might just take a few of them with us. She’s gonna blow! Your fleet has lost. And your friends on the Endor moon will not survive. There is no escape, my young apprentice. The Alliance will die as will your friends. I can feel your anger. I am defenseless. Take your weapon! Strike me down with all your hatred, and your journey towards the dark side will be complete. Get him off of there! I think I got it. I got it! Oh, Princess Leia, are you all right? It’s not bad. I love you. Get down here! I got an idea. Use your aggressive feelings, boy! Let the hate flow through you. Obi-Wan has taught you well. I will not fight you, father. You are unwise to lower your defenses. Your thoughts betray you, father. I feel the good in you the conflict. There is no conflict. You couldn’t bring yourself to kill me before, and I don’t believe you’ll destroy me now. You underestimate the power of the dark side. If you will not fight, then you will meet your destiny. I’m on it, Gold Leader. Good shot, Red Two. Now come on, Han, old buddy. Don’t let me down. It’s over, Commander. The Rebels have been routed. They’re fleeing into the woods. We need reinforcements to continue the pursuit. Send three squads to help. Open the back door. Throw me another charge. You cannot hide forever, Luke. I will not fight you. Give yourself to the dark side. It is the only way you can save your friends. Yes, your thoughts betray you. Your feelings for them are strong. So you have a twin sister. Your feelings have now betrayed her, too. Obi-Wan was wise to hide her from me. Now his failure is complete. If you will not turn to the dark side, then perhaps she will. Your hate has made you powerful. Now, fulfill your destiny and take your father’s place at my side! I’ll never turn to the dark side. You’ve failed, Your Highness. I am a Jedi, like my father before me. So be it Jedi. The shield is down! Commence attack on the Death Star’s main reactor. We’re on our way. Red Group, Gold Group, all fighters follow me. Told you they’d do it! If you will not be turned, you will be destroyed. Young fool only now, at the end, do you understand. Your feeble skills are no match for the power of the dark side. You have paid the price for your lack of vision. Now, young Skywalker you will die. I’m going in. Here goes nothing. Now lock onto the strongest power source. It should be the power generator. And stay alert. We could run out of space real fast.

25 Split up and head back to the surface. See if you can get a few of those TIE fighters to follow you. Copy, Gold Leader. That was too close. We’ve got to give those fighters more time. Concentrate all fire on that Super Star Destroyer. Sir, we’ve lost our bridge deflector shield. Intensify the forward batteries. I don’t want anything to get through. Intensify forward firepower! It’s too late! Luke, help me take this mask off. But you’ll die. Nothing can stop that now. Just for once let me look on you with my own eyes. Now go, my son. You’re coming with me. I can’t leave you here. I’ve got to save you. You already have, Luke. You were right about me. Tell your sister you were right. Father I won’t leave you. There it is! All right, Wedge. Go for the power regulator on the north tower. Copy, Gold Leader. I’m already on my way out. Move the fleet away from the Death Star. Wedge, I don’t think we’re going to make it. You’ll make it. Just follow me Gold Leader. I promised to return his ship without a scratch. I sure hope that old pirate forgives me. They did it!

C Test Runs

Each test run shows the sentences generated with a 1 or 0 grading the sentence coherence. These were added and results demonstrated in Table 1 and Table 2. Note any .5 grading was counted as 0

C.1 10 Sentence Data Runs - No Rules obi-wan was wise to the dark side. 1 i’ll never turn to the dark side, then perhaps she will. 1 i am a twin sister. 1 obi-wan was wise to the dark side. 0 if you powerful. 0 i’ll never turn to hide her from me. 1 you’ve failed, your feelings have now betrayed her, too. 1 obi-wan was wise to the dark side, then perhaps she will. 0 your hate has made you powerful. 0 obi-wan was wise to the dark side. 1 you’ve failed, your feelings have now betrayed her, too. 1 you’ve failed, your feelings have now betrayed her, too. 0 your father’s place at my father before me. 0 your father’s place at my father before me. 0 now betrayed her, too. 0 obi-wan was wise to the dark side, then perhaps she will. 0 now his failure is complete. 0 now betrayed her, too. 0 your highness. 1 your hate has made you will not turn to the dark side. 0 you’ve failed, your destiny and take your hate has made you will not turn to the dark side, then perhaps she will. 0 your feelings have a jedi, like my side! 1 obi-wan was wise to the dark side, then perhaps she will. 0 you’ve failed, your highness. 0 you’ve failed, your feelings have a jedi, like my

26 side! 0 obi-wan was wise to hide her from me. 0 now, fulﬁll your feelings have a twin sister. 1 your destiny and take your highness. 0 your feelings have a jedi, like my father before me. 1 i am a jedi, like my father before me. 0 obi-wan was wise to hide her from me. 0 i’ll never turn to the dark side, then perhaps she will. 1 your father’s place at my father before me. 0 your father’s place at my side! 1 now, fulﬁll your destiny and take your highness. 1 now betrayed her, too. 0 i am a jedi, like my father before me. 0 your hate has made you will not turn to the dark side. 0 your feelings have now betrayed her, too. 0 so you powerful. 0 you’ve failed, your father’s place at my father before me. 0 now his failure is complete. 0 now his failure is complete. 0 your hate has made you will not turn to the dark side, then perhaps she will. 0 i am a twin sister. 1 if you will not turn to the dark side. 1 you’ve failed, your father’s place at my side! 1 you’ve failed, your father’s place at my father before me. 0 obi-wan was wise to hide her from me. 0 now his failure is complete. 0

C.2 10 Sentence Data Runs - Some Rules so you have now betrayed her, too. 1 your hate has made you powerful. 0 now betrayed her, too. 0 your feelings have now his failure is complete. 0 so you have a complete. 1 i am a jedi, like my side! 1 i’ll never turn to the hate has made you have now betrayed her, too. 0 i am a obi-wan was wise side. 0 i’ll never turn to the sister. 1 your hate has made you powerful. 0 i’ll never turn to the powerful. 1 you’ve failed, your hate has made you have now his failure is complete. 0 so you will not turn to the feelings have a destiny and take your feelings have now his failure is complete. 0 i am a side. 1 i am a failure is complete. 0 obi-wan was wise highness. 1 you’ve failed, your father’s place at my father before me. 0 i’ll never turn to hide her from me. 1 now, fulﬁll your father’s place at my side! 1 i’ll never turn to the side. 1 obi-wan was wise feelings have a side, then perhaps she will. 0 your father’s place at my father before me. 0 if you will not turn to hide her from me. 1 now betrayed her, too. 0 your feelings have now betrayed her, too. 0 you’ve failed, your destiny and take your highness. 0 you’ve failed, your father’s place at my father before me. 0 now, fulﬁll your hate has made you have a failure is complete. 0 now betrayed her, too. 0 so you have now betrayed her, too. 1 so you will not turn to the wise side, then perhaps she will. 1 so you powerful. 0 your highness. 1 obi-wan was wise father before me. 0 obi-wan was wise hate has made you powerful. 1 obi-wan was wise to hide her from me. 0 if you powerful. 0 you’ve failed, your father’s place at my side! 1 so you have a jedi, like my side! 1 your hate has made you have now betrayed her, too. 0 your father’s place at my father before me. 0 i’ll never turn to the obi-wan was wise sister. 0 your hate has made you have a jedi, like my father before me. 1 you’ve failed, your destiny and take your father’s place at my side! 0 now his failure is complete. 0 i’ll never turn to hide her from me. 1 i am a wise failure is complete. 0 i am a jedi, like my side! 1 i am a side, then perhaps she will. 0 i am a complete. 0

C.3 10 Sentence Data Runs - All Rules so you powerful. 0 your failed, your failed, your hide her from me. 0 your feelings have a dark obi- wan was wise side, then perhaps she hide her made you will not turn to the jedi, like my betrayed her, too. 0 i am a dark father before me. 1 you’ve failed, your father’s place at my side! 1 if you have a jedi, like my have now his turn to hide her have a twin side, then perhaps she made you failed, your has made you has made you was wise place at my is complete. 0 you’ve failed, your

27 have now betrayed her, too. 1 now, fulfill your hide her hide her from me. 0 your made you take your am a highness. 0 so you fulfill your failed, your have a destiny and take your betrayed her, too. 0 now, fulfill your destiny and take your am a twin obi-wan was wise dark father before me. 0 if you have now betrayed her, too. 1 you’ve failed, your highness. 1 so you was wise father before me. 1 your take your fulfill your have a complete. 0 you’ve failed, your failed, your hate has made you will not turn to the jedi, like my turn to the side. 0 you’ve failed, your betrayed her, too. .5 i am a dark sister. 1 i am a highness. 1 now, fulfill your highness. 1 i am a hate has made you powerful. 0 your highness. 1 you’ve failed, your made you will not turn to hide her fulfill your hate has made you hide her turn to hide her is complete. 0 i am a highness. 1 now his failed, your feelings have a highness. 0 if you hide her fulfill your turn to the side. 1 i’ll never turn to the powerful. 1 i am a complete. 0 i’ll never turn to the failure is complete. 0 your father’s place at my side! 1 so you betrayed her, too. 1 so you betrayed her, too. 0 if you powerful. 0 i’ll never turn to the dark side, then perhaps she have now his turn to the wise twin side, then perhaps she am a obi-wan was wise wise place at my made you made you powerful. obi-wan was wise father before me. 0 your father’s place at my take your failed, your have now betrayed her, too. 0 obi-wan was wise obi-wan was wise twin failure is complete. your destiny and take your feelings have now his have now betrayed her, too. 0 you’ve failed, your turn to hide her am a twin jedi, like my have now betrayed her, too. 0 i am a side. 1 if you fulfill your feelings have now his betrayed her, too. 0 if you have now betrayed her, too. 1 your made you am a obi-wan was wise hate has made you betrayed her, too. 0 obi-wan was wise destiny and take your feelings have now betrayed her, too. 0 so you have a wise obi-wan was wise jedi, like my fulfill your turn to the sister. 0 your turn to the twin failure is complete. 0 now betrayed her, too. 0 so you has made you was wise obi-wan was wise failure is complete. 0 now betrayed her, too. 0 now betrayed her, too. 0

C.4 Full Dialog Runs - No Rules father was about it. 1 that shield generator. 1 you telling them? 1 accelerate to completing your lack of things i’ve felt it. 0 artoo, are to the code’s changed. 0 lando calrissian and head back to get oﬀ this protocol droid and that i left you are attacking. 0 here goes our weapons back. 0 it was seduced by now a way to appreciate me. 0 go on time or this’ll be turned from the exact location of the shield is relatively unprotected. 0 guess i’m afraid he’s ever so i also know your friends up there will be turned, you will destroy vader! 0 the forest moon and the mighty chewbacca. 1 well, it’s your feelings, boy! 1 the emperor hasn’t driven it is good idea. .5 i don’t know you are waiting for. 1 you’re the master’s sail barge. 1 the alliance should be, too. 1 but i see me, pal. 1 watch yourself, wedge! 1 i don’t want to you, a funny feeling. 0 there is. 0 exciting is to defeat vader again! 1 why he must fall. 1 that’s why don’t do they designed her, too. 0 they’re going in. 1 commence attack 0 you will therefore be rewarded. 1 allow me and destination? 0 i was high-born and take your choice. 1 you’ll learn some kind of a trap! 1 it was with them to show us if any attack 0 yes, forever sleep. 1 we have accepted the death star orbiting the death star destroyers are powerful, as will therefore be completed on the ﬁrst to beg your training. 0 i must know that you? 0 take the pit of it was a feisty little furball. 0 it checks out. 1 i like your feeble skills are you take it is with them if i could go for solo’s life. 0 a strong defense mechanism. 1 because she became the

28 things i’ve accepted the endor moon and await my own father. 1 ask you the right away. 1 bury your training. 1 chewie, is swelling in my design. 1 you can feel when gone and will gladly pay you can’t repel firepower of this station will always been expecting you. 0 i think we shall stay here. 1 when i don’t think i was pregnant. 1 hi, han will be mutually beneficial and could never knew i beg your overconfidence is my mark. 0 take it. 0 instead of command station, this thing. 1 we’re not that we have come. 0 yes, i said you rushed to pay fifty thousand. 1 you’re the other he found out of the nearby forest moon are now come on, let’s pass on tatooine, if we’re going on? 0

C.5 Full Dialog Runs - Some Rules there it upon me solo and take care of the shall we can never returned from the such fire away from your destiny and deactivate the strong. 0 we’re going to be with a luke but i got her path crossed yours for me. 0 and get it. 0 it was the power generator. 1 not sure na chuba negatorie natoota. 1 so according to that. 1 luke, you must know. 1 in his companions to save you. 1 just sleepin’. 0 earned it, i need it. 1 lord vader, this companions to bring me again sometime. .5 oh, master luke. 1 i felt his compassion for them about this destiny and will gladly pay you on tatooine and murdered my programming to keep your work out there. 0 follow your father has done well, i had terrible captain solo must be proper. 0 well, i’m looking for once anakin were once thought she didn’t want fifty approach. 0 i told them myself. 1 oh, master luke and, if you must. 1 he does, you can turn to speak with you. 1 point of the scum. 1 that’s the hutt will serve your real free emperor himself is st 321. .5 it is! 0 the father couldn’t be done. 1 i warn you know, father. 1 you were lookin’ for your apparent shield down with jabba. .5 it’s just take good darth vader my own draw their way. 0 intensify the friend. 0 na chuba negatorie natoota. 0 i don’t believe they didn’t help my life, so old father. 1 obi-wan was just follow your hate. 1 this machine now i said you start down the father was high-born experience was dangerous eyesight will take time. 0 cut it taught you achieved little bridge deflector shield. 0 you are you? 1 why you must bring us in, artoo. 1 the luck the father. 0 i owe you understand. .5 that super star will be a dune sea and await my goodness! 1 i could be so impatient hutt. 1 well, don’t let go for yourself. 1 uh, i can’t tell luke? luke’s message and your father has made a woods. 1 split up my head. 1 luke and, if they found on the message, artoo, don’t know the scratch. 1 old shield is that strong! 1 his ship in him. 0 you on alderaan. 1 take these outta here you can stop him you triple! 1 chewie, but now his presence. 0 there’s two more position. 1 it was a very primitive parts and will find new jedi knight and take it should have come. 0 come back? 0 i have come on, let’s move. 1

C.6 Full Dialog Runs - All Rules so you powerful. 0 your failed, your failed, your hide her from me. 0 your feelings have a dark obi- wan was wise side, then perhaps she hide her made you will not turn to the jedi, like my betrayed her, too. 0 i am a dark father before me. 1 you’ve failed, your father’s place at my side! 1 if you have a jedi, like my have now his turn to hide her have a twin side, then perhaps she made you failed, your has made you has made you was wise place at my is complete. 0 you’ve failed, your have now betrayed her, too. 1 now, fulﬁll your hide her hide her from me. 0 your made you take your am a highness. 0 so you fulﬁll your failed, your have a destiny and take your betrayed her, too. 0

29 now, fulfill your destiny and take your am a twin obi-wan was wise dark father before me. 0 if you have now betrayed her, too. 1 you’ve failed, your highness. 1 so you was wise father before me. 1 your take your fulfill your have a complete. 0 you’ve failed, your failed, your hate has made you will not turn to the jedi, like my turn to the side. 0 you’ve failed, your betrayed her, too. .5 i am a dark sister. 1 i am a highness. 1 now, fulfill your highness. 1 i am a hate has made you powerful. 0 your highness. 1 you’ve failed, your made you will not turn to hide her fulfill your hate has made you hide her turn to hide her is complete. 0 i am a highness. 1 now his failed, your feelings have a highness. 0 if you hide her fulfill your turn to the side. 1 i’ll never turn to the powerful. 1 i am a complete. 0 i’ll never turn to the failure is complete. 0 your father’s place at my side! 1 so you betrayed her, too. 1 so you betrayed her, too. 0 if you powerful. 0 i’ll never turn to the dark side, then perhaps she have now his turn to the wise twin side, then perhaps she am a obi-wan was wise wise place at my made you made you powerful. obi-wan was wise father before me. 0 your father’s place at my take your failed, your have now betrayed her, too. 0 obi-wan was wise obi-wan was wise twin failure is complete. your destiny and take your feelings have now his have now betrayed her, too. 0 you’ve failed, your turn to hide her am a twin jedi, like my have now betrayed her, too. 0 i am a side. 1 if you fulfill your feelings have now his betrayed her, too. 0 if you have now betrayed her, too. 1 your made you am a obi-wan was wise hate has made you betrayed her, too. 0 obi-wan was wise destiny and take your feelings have now betrayed her, too. 0 so you have a wise obi-wan was wise jedi, like my fulfill your turn to the sister. 0 your turn to the twin failure is complete. 0 now betrayed her, too. 0 so you has made you was wise obi-wan was wise failure is complete. 0 now betrayed her, too. 0 now betrayed her, too. 0