ITAT 2013 Proceedings, CEUR Workshop Proceedings Vol. 1003, pp. 2–9 http://ceur-ws.org/Vol-1003, Series ISSN 1613-0073, c 2013 M. Lopatková, M. Plátek

Formalization of Word-Order Shifts by Restarting Automata∗

Markéta Lopatková and Martin Plátek

Charles University in Prague, Faculty of Mathematics and Physics Malostranské nám. 25, 118 00 Prague, Czech Republic [email protected], [email protected]

Abstract: The phenomenon of freedom plays a number of word order shifts within the analysis by re- an important role in syntactic analysis of many natural lan- duction. This interplay was exemplified on a sample of guages. This paper introduces a notion of a word order Czech sentences with and non-projectivities (long shift, an operation reflecting the degree of word order free- distance dependencies, see esp. [10, 4]). This approach is dom of natural languages. The word order shift is an op- based upon the method of analysis reduction – a stepwise eration based upon word order changes preserving syntac- simplification of a sentence preserving its correctness. The tic correctness, individual word forms, their morphologi- analysis by reduction had been applied in a relatively free cal characteristics, and their surface dependency relations manner with no constraints on the projectivity of its indi- in a course of a stepwise simplification of a sentence, a so vidual steps. Our investigation focused on a set of syntac- called analysis by reduction. tically analyzed Czech sentences from the Prague Depen- The paper provides linguistic motivation for this opera- dency Treebank (PDT) [2]. The experiment on treebank tion and concentrates on a formal description of the whole data showed that with only basic constraints on the analy- mechanism through a special class of automata, so called sis by reduction, see [6], the minimal number of necessary restarting automata with the shift operation enhanced with shifts for any sentence from the test set did not exceed one. a structured output, and their computations. This number actually confirmed the initial assumption for The goal of this study is to clarify the properties of com- Czech as a language with relatively high degree of word- putations needed to perform (enhanced) analysis by reduc- order freedom; nevertheless, we have continued the search tion for free word order languages. for more complicated sentences. This lead to the discov- ery of a special construction requiring (at least) two shifts, as it is discussed in Section 2.1. 1 Introduction The analysis by reduction has served as a motivation for The phenomenon of word order freedom plays an impor- new a family of automata, so called restarting automata, tant role in syntactic analysis of many natural languages. see esp. [14, 13]. As a first step, analysis by reduction A relatively free word order combined with additional lin- was modelled as a formal string to string translation using guistic constraints constitutes a big challenge for a for- a suitable model of restarting automata [9]. Then a class of mal description of syntactic structures.Here we introduce enhanced restarting automata with an output consisting of a notion of a word order shift, an operation reflecting the a single DR-tree was introduced, [16]. Finally, the model degree of word order freedom of natural languages. The was further enhanced – a model of restarting automata that word order shift is a word order change preserving syntac- assign a set of dependency structures (DR-structures) to tic correctness, individual word forms, their morphologi- every reduction of an input sentence was discussed, [15]. cal characteristics, and their surface dependency relations DR-structures can capture a set of dependency trees repre- in the course of a stepwise simplification of a sentence un- senting sentence on different layers of linguistic descrip- der investigation, referred to as an analysis by reduction. tion in a parallel way. Here we further enrich the perfor- Section 2 provides a linguistic motivation and informal de- mance of these automata with the shift operation. scription of the process. Section 3 concentrates on a formal description of the whole mechanism through a special class of automata, 2 Analysis by Reduction so called restarting automata with the shift operation en- hanced with structured output, pRL -automata, and their e Let us first describe the main ideas behind the method used computations. for sentence analysis. Analysis by reduction (AR) is based on a stepwise simplification of an analyzed sentence, see Previous Work. The paper [6] introduced a specific ap- [8]. It defines possible sequences of reductions (deletions) proach to the problem of measuring the word-order free- in the sentence – each step of AR is represented by deleting dom. It focused upon an interplay of two phenomena re- a sentence member, possibly accompanied by rewriting a lated to word order: a (non-)projectivity of a sentence and word of the sentence (and thus its shortening); in specific

∗The paper reports on the research supported by the grant of GACRˇ cases, deleting is accompanied by a shift of a word to an- No. P202/10/1333. other word order position. Formalization of Word-Order Shifts by Restarting Automata 3

Let us stress the basic constraints imposed on the anal- We can notice that the order of reductions reflects the ysis by reduction: dependency relations in the corresponding dependency tree. Informally, the words are ‘cut from the bottom of the (i) the obvious constraint on preserving individual words tree’ (in other words, the ‘pruning’ operation is applied on (word forms), their morphological characteristics the tree); i.e., a governing node must be preserved in a sim- and/or their surface dependency relations; plified sentence until all its dependent words are deleted.1 AR (ii) the constraint on preserving the correctness (i.e., a In other words, makes it possible to identify the de- grammatically correct sentence must remain correct pendency tree for sentence (1): after its simplification); bojí Z  JJZ (iii) moreover, the application of the shift operation is lim- Petr se o . ited only to cases when a shift is enforced by the JJ correctness preserving principle of AR, i.e., a simple otce deletion would result in an incorrect word order and The analysis by reduction has been designed to model a thus violate the principle (ii) above. human who is able to check the correctness of the analysis. However, it is possible to model correct reductions on the As a consequence of AR, it is possible to derive for- basis of large corpora of natural language sentences, see mal dependency relations between individual sentence [11]. members – pairs of governing and dependent nodes – based on the possible order(s) of reductions, as it is de- scribed in [8, 15], see ex. (1) below. Compare also with 2.1 The role of shifts. The above mentioned experiment other approaches aiming to determine dependency rela- on a set of non-projective sentences from PDT [2] (namely tions summed up esp. in [1] and [12]. on the sentences with a non-projectivity given by a modal Example: , see [3]) constituted only the first step. (1) Petr se bojí o otce. As a follow-up experiment, we have decided to supple- ‘Peter – REFL – fears – for – father’ ment the data with ‘suspicious’ types of sentences identi- ‘Peter fears for his father.’ fied in previous work on Czech word order freedom, esp. The analysis by reduction can be summarized in the fol- in [4]. It allowed to include also less frequent phenom- lowing scheme: ena into our investigations. As a result, we found a well- Petr se bojí o otce. formed Czech sentence that requites at least two shifts in the course of AR, see the following example (2). delete Z delete = Z~ Let us stress that the role of shifts in the course of the Petr se bojí. * se bojí o otce. analysis by reduction was limited to cases where a shift delete shift ? ? ‘corrects’ word order of a simplified sentence – this con- * se bojí. bojí se o otce. cerns primarily the cases when an input sentence contains shift delete a (as in sentences (1)-(2)). ZZ~ = bojí se. Example: Example (1) can be simplified in two ways (for simplicity, (2) S tˇežkýmse bála pomoci úkolem. we do not make distinction between upper and lower case ‘with – difficult – REFL – (she) was afraid – to help – task’ letters at the beginning of the sentence): ‘With a difficult task, she was afraid to help.’ (i) Either by simple deletion of the prepositional group Although the sample sentence is rather strange, the split- o otce ‘for father’ (following the constraint on correctness ting of the prepositional group is a grammatical construc- of the simplified sentence, the pair of word forms must be tion in Czech allowing to put a strong stress on an adjec- deleted in a single step; see the left branch of the scheme). tive modifying the and not on the whole prepositional (ii) Or by deleting the Petr (the right part of the group. scheme); however, this simplification results in an incor- Due to the dependency relations present in the sentence rect word order variant starting with the clitic se (such po- there is only one possibility how to reduce it, the reduction sition of a clitic is forbidden in Czech). Thus the shift op- of the tˇežkým ‘difficult’. Unfortunately, it results in a syntactically incorrect word order: eration correcting the word order is enforced shi ft bojí → * s se bála pomoci úkolem. se o otce. →delete As these possible reductions are independent of each other, This situation can be corrected in two possible ways, let us we can conclude that the words Petr and o otce are inde- focus on one of them: pendent – in other words, both of them depend on / modify 1Let us note that some relations – e.g., the relation between a prepo- a word remaining in the simplified sentence, i.e., the verb sition and its ‘head’ noun as well as a verb and some clitic – are rather AR and its clitic bojí se. technical (not only) from the point of view of : such pairs must be reduced in a single step (despite being represented as two nodes in a Now, we can proceed in a similar way until we get the dependency tree); here we adhere to the practice used in the PDT anno- minimal correct simplified sentence bojí se. tation. → 4 M. Lopatková, M. Plátek

s úkolem se bála pomoci. (A shift of the noun We have chosen a nondeterministic model to enable vari- →shi ft úkolem ‘task’ next to the preposition.) ous sequences of reductions. This can serve for the verifi- * se bála pomoci. (Unfortunately, the next reduc- cation of the (in)dependency between the individual parts →delete tion must remove the prepositional group s úkolem ‘with of a sentence, as was sketched in Section 2. task’ making the sentence again ungrammatical due to the A pRL-automaton M is (in general) a nondeterministic clitic on the sentence first position.) machine with a finite alphabet Γ, a head q (window of size bála se pomoci. (Now we can repair the sentence 1) working on a flexible tape delimited by the left sentinel →shi ft by shifting the verb bála to the left.) c and the right sentinel $ (c,$ Γ), and two finite sets 6∈ bála se. (The reduction of the word pomoci ‘help’ of restarting meta-instructions R(M) and accepting meta- →delete ends the process.) instructions A(M), respectively. Such an automaton makes Similarly for the second sequence of reductions and shifts. use of k pebbles p , p . 1 ··· k Regardless of the selected sequence, we must apply at least Let us first describe informally the way how an pRL- two shifts in the course of AR. automaton works. Each of its finite computations consists of certain phases called cycles, and a last – halting – phase Remark: Note that we limit our observations to simple sentences, i.e., to sentences with a single clause. The rea- called a tail. In each cycle, the automaton performs two son is obvious, complex sentences might blur the whole passes through the tape: during the first pass, it marks se- picture by bringing additional phenomena into the play. lected symbols of a processed sentence with pebbles; then For example, a coordination of clauses might completely during the second pass, it performs its editing operations R(M) change the total number of shifts in a complex sentence re- described by meta-instructions from ; the operations gardless of the phenomena contained in individual clauses. can be applied (only) on the symbols marked by pebbles. In general, each (coordinated) clause can induce a shift in In each (accepting) tail, the automaton marks all symbols A(M) the course of AR, thus the overall number of shifts cannot by pebbles – according to a meta-instruction from –, be bounded. halts, and accepts the analyzed sentence.

3.1.1 Restarting meta-instructions. Each cycle of the 3 Restarting Automaton with a Structured pRL-automaton M is controled by a single restarting meta- instruction I R(M) of the form Output R ∈ IR = (E0,I1,...Is;Op; Restart) (1) The crucial issue of the application of a word-order shift in the process of AR is actually identifying WHAT should where: – each I is an instruction of the form (a ,E ),1 i s, be shifted WHERE. In order to model the whole process i i i ≤ ≤ by means of a special kind of an automaton, we have to – each Ei is a regular language over Γ; Ei is interpreted as use special markers – pebbles, which mark the possible the i-th context of IR (the contexts are usually represented targets of shift locations. For this purpose we are going by regular expressions); – each instruction I marks the symbol a Γ from the tape to define a restarting automaton with a finite set of special i i ∈ meta-instructions using pebbles – an pRL-automaton. The with the pebble pi within the first pass; – O = o , ,o is an ordered set of editing operations, automaton with pebbles was originally introduced in [15] p { 1 ··· p} o dl[i],wr[i,b],sh[i,l] 1 i,l s,i = l,b Γ , in order to define dependency output structures, and we j ∈{ | ≤ ≤ 6 ∈ } are going to enhance its abilities with the shift operation. – o j is the j-th editing operation performed in the second We proceed in several steps. First, a restarting automa- pass: ton with pebbles is enhanced with a shift operation (Sec- — if o j = dl[i] then it deletes the symbol ai, tion 3.1). Second, a structured output is added (Section — if o j = wr[i,b] then it rewrites ai by b, 3.2). Finally, a computation of such enhanced automaton — if o j = sh[i,l] then it shifts ai to the position behind is described (Section 3.3). al. We require a ‘uniqueness’ of operations on a single sym- bol, i.e., if dl[i] O then O does not contain the opera- ∈ p p 3.1 Restarting Automaton with the Shift Operation tions wr[i,b],sh[i,l] for any b,l. Operating on Strings Performing a meta-instruction: For an (input) sentence w Γ∗, we define the tape inscription cw$, and a reduction ∈c Here we present a class of nondeterministic pRL-automata w w0. When trying to execute a cycle C by performing `IR that are suitable for modeling AR – these automata allow the meta-instruction IR, it starts with the tape inscription for checking the whole input sentence (and marking se- cw$, M will get stuck (and so reject) if w does not permit a lected words with pebbles) prior to any changes. It re- factorization of the form w = v0a1v1a2 ...vs 1asvs (vi Ei − ∈ sembles a linguist who can read the whole sentence first, for all i = 0,...,s). On the other hand, if w permits factor- and then can reduce the sentence in a correct way. Thus izations of this form, then one of them is (nondeterminis- he/she makes correct steps of the analysis by reduction. tically) chosen. Formalization of Word-Order Shifts by Restarting Automata 5

The automaton processes the input word in two passes: serve for building the structured output, as it is described 1. During the first pass of C, M puts the pebble pi on in the following sections. each a (this corresponds to the instruction I , for each i, The notation u c v (a reduction of u to v by M) denotes i i `M 1 i s). a reduction performed during a cycle of M that begins with ≤ ≤ 2. During the second pass of C controlled by the or- the tape inscription cu$ and ends with the tape inscription dered set of operations O , the tape inscription cw$ is cv$; the relation c∗ is the reflexive and transitive closure p `M transformed (reduced) into a new tape inscription w = of c . By REM(M) we denote the set of reductions per- 0 `M v0r1v1 vs 1rsvs, where 1 i s: formed by M. ··· − ≤ ≤ A string w Γ is accepted by M, if there is an accept- ∈ ∗ In the second pass of C, M performs its operations in ing computation which starts with the tape inscription cw$ • the prescribed order o1,o2, ,op. and ends by executing an accept operation. By L(M) we ··· denote the language consisting of all words accepted by r = λ if there is a deleting operation o O such 2 • i j ∈ p M; we say that M recognizes (accepts) language L(M). that o j = dl[i]; i.e., ai is deleted during the cycle C; Note that we limit ourselves only to such combinations of operations that are adequate (and necessary) for known r = b if there is a rewriting operation o O such linguistic examples. • i j ∈ p that o j = wr[i,b]; Example: Let us illustrate the notions of restarting and i.e., ai is rewritten with b during the cycle C; accepting meta-instructions on the analysis of sentence (1) from Section 2. This example formalizes the AR of the r = λ and r = a a if there is a shifting operation • i j l i sentence Petr se bojí o otce. ‘Peter fears for his father., o O such that o = sh[i,l]; j ∈ p j namely the left branch of its AR (see the scheme in Section i.e., ai is shifted behind the symbol marked by the 2). The respective pRLe-automaton M is described by two pebble pl during the cycle C; restarting meta-instructions IR1 and IR2 , and one accepting ri = ai if there is no operation o j Op applicable on meta-instruction IA, namely: • ∈ 1 1 1 1 Restart ai. IR1 = (E0,I1 ,I2 ,I3 ;O2; ) 1 (Note that such an ai may – together with symbols E0 = Petr se , I1 = ( bojí , λ ) 1 { } 1 { } where some of above mentioned operation is applied O2 = dl[2],dl[3] , I2 = ( o , λ ) { } 1 { } – create the output structure, see the following sec- I3 = ( otce , ) 2 2 2 2 Restart {·} tion, and thus they must be marked with pebbles.) IR2 = (E0,I1 ,I2 ,I3 ;O2; ) E = λ , I2 = ( Petr, λ ) Restart 0 { } 1 { } At the end of each cycle, the operation is per- 2 = [ ], [ , ] 2 = ( , ) • O2 dl 1 sh 2 3 , I2 se bojí formed: all the pebbles are removed from the new { } 2 { } I3 = ( bojí , λ ) tape inscription w , and the head is placed on the left { } 0 IA = ( bojí , se, . , Accept) sentinel. The computation consists of two cycles. Within the first

cycle described by the meta-instruction IR1 , the instruc- Of course, we can delete, rewrite or shift neither the left 1 1 1 tions I1 , I2 and I3 put pebbles p1, p2 and p3 on the sym- sentinel c nor the right sentinel $. It is required that during bols containing the words bojí (with the left context Petr a cycle, M executes at least one delete operation. se and empty right context), o, and otce, respectively; the If no further cycle is performed, each finite (accepting) operations dl[2] and dl[3] delete the word o (marked with computation necessarily finishes in a tail performed ac- the pebble p2) and otce (marked with the pebble p3), re- cording to an accepting meta-instruction. spectively. Then the automaton restarts (in particular, all pebbles are removed). Similarly in the second cycle (meta- instruction I ), the instructions I2, I2 and I2 mark the re- 3.1.2 Accepting meta-instructions. Tails of accepting R2 1 2 3 computations are described by a set accepting meta- spective words and then operation dl[1] deletes the word instructions A(M), each of the form: Petr (marked with p1) and the operation sh[2,3] shifts the word se with the pebble p2 behind the word bojí (with the pebble p ). The accepting instruction I just marks the IA = (a1,...,as,Accept), (2) 3 A remaining words bojí and se with pebbles. where ai are symbols from Γ. The following property – obviously ensured by pRL- Performing a meta-instruction: The tail performed by automata – has a crucial role in our applications of restart- the meta-instruction IA starts with the inscription on the ing automata. tape cz$; if z = a a , then M marks each symbol a 1 ··· s i pRL with the pebble p , and accepts z (and the whole compu- Weak Correctness Preserving Property. Any -auto- i M weakly correctness preserving tation as well). Otherwise, the computation halts with re- maton is , i.e., (for all jection. Note that only operations distributing pebbles are 2Note that language L(M) accepted by the automaton M is the same allowed within an accepting meta-instruction – they will language as LC(M) in [15]. 6 M. Lopatková, M. Plátek

u,v Γ ) U contains the root of G of the form [i,a ] (unless the ∈ ∗ • p i v L(M) and u c∗ v imply that u L(M). root is already contained in U ). ∈ `M ∈ w Our goal is to model the analysis by reduction by automata 2. An edge (u,v) H represents: with the following stronger property: ∈ pRL Correctness Preserving Property. A -automaton either deleting: if dl[i] Op then the edge has the M is correctness preserving, if (for all u,v Γ ) u • ∈ ∈ ∗ ∈ form u = [i,ai], and v = [ j,a j] for some j = i ( j is in- L(M) and u c∗ v imply that v L(M). 6 `M ∈ terpreted as a position of respective governing node); [ , ] 3.2 Restarting Automaton with the Shift Operation or rewriting: if wr i b Op then the edge has the • = [ , ] =∈ [ , ] Enhanced with Structured Output form u i ai , and v i b (i.e., u represents the original symbol ai and v the rewritten) In this section, we introduce enhanced restarting automata with structured output – the so-called pRLe-automata. During their enhanced computations, these automata build 3.2.2 Enhanced accepting meta-instruction IA. – with the help of pebbles – so called DR-structures, i.e., Let IA be an accepting instruction of the form IA = Accept graphs consisting of items representing individual occur- (a1,...,as, ) (i.e., as it is introduced in Section 3.1, rences of symbols (words) in the input sentence and di- X = A). Then a corresponding enhanced accepting meta- rected edges between them. instruction has a form IEA = (IA,G), where G = (U,H) ,..., In order to represent a structured output by an pRLe- is an enhancing graph; the symbols a1 as are pebbled and they correspond to the nodes [1,a ],[2,a ]...,[s,a ] automaton, the items of flexible tape (representing just { 1 2 s } of U. For each edge h = (u,v) H, we suppose that sentinels and symbols (= words) of a processed sentence) ∈ u = [a ,i], v = [a , j] (1 i, j s and i = j). are enriched with two integers denoting horizontal and ver- i j ≤ ≤ 6 tical position of a respective word in the resulting DR- Example: Let us return to the analysis of sentence (1) structure. Petr se bojí o otce. ‘Peter fears for his father.’, see Section An pRLe-automaton is specified in two steps: First, in- 2. In Section 3.1, we have already presented two restart- pRL structions of an -automaton are enhanced with graphs ing instructions IR1 and IR2 , and one accepting instruction (Sections 3.2.1 and 3.2.2). Second, an application of these IA. Here we enrich these instructions with a graph de- enhanced instructions (i.e., its computations) are specified scribing the (part of) resulting DR-structures – the respec- DR with the help of -structures (Sections 3.2.3 and 3.3). tive pRLe-automaton Me is described by enhanced instruc- pRL A e-automaton Me = (Γ,c,$,ER(Me),EA(Me)) tions IER1 = (IR1 ,G1), IER2 = (IR2 ,G2), and IEA = (IA,GA), consists of an alphabet Γ, sentinels c,$ Γ, a set of en- namely: 6∈ hanced restarting meta-instructions ER(Me), and a set of G = (U ,H ) U = [1, bojí ],[2,o],[3,otce] 1 1 1 1 { } enhanced accepting meta-instructions EA(Me). H = ([2,o],[1, bojí ]),([3,otce],[2,o]) 1 { } An enhanced meta-instruction is a pair IEX = (IX ,G), G = (U ,H ) U = [1,Petr],[3, bojí ] 2 2 2 2 { } where IX is a (restarting or accepting) meta-instruction H = ([1,Petr],[3, bojí ]) 2 { } of a pRL-automaton, and G is a directed acyclic graph G = (U ,H ) U = [1, bojí ],[2,se],[3, ] A A A A { · } G = (U,H). The graph G represents the required structure HA = ([2,se],[1,bojí ]),([3, ],[1,bojí ]) of symbols processed during the application of the meta- { · } instruction I . In this paper we request G to be a rooted X DR tree. (This restriction ensures linguistically adequate out- 3.2.3 -structures. DR put structures.) In this paragraph, we introduce so-called -structures (Delete Rewrite structures), which serve for the represen- tation of output structures of restarting automata. A DR- 3.2.1 Enhanced restarting meta-instruction IER. structure captures syntactic units (words and their cate- Let IR be a restarting meta-instruction of the form (1) intro- gories used in an input sentence and also in its reduced duced in section 3.1.1. We define the corresponding graph forms) as nodes of a graph and their mutual syntactic re- G = (U,H) in the following way: lations as edges; moreover, word order is represented by 1. The set of nodes U consists of three disjunct subsets Un, means of total ordering of the nodes. Uw, and Up: A DR-structure is a directed acyclic graph D = (V,E), where V is a finite set of nodes, and E V V is a finite Un = [i,ai] dl[i] Op or wr[i,b] Op, 1 i s . ⊂ × • { | ∈ ∈ ≤ ≤ } set of edges. A node u V is a tuple u = [i, j,a], where a is We can see that Un covers all (original) words which ∈ a symbol (word) assigned to the node, and i, j are natural are deleted or rewritten words within I (a Γ is the R i ∈ numbers, i represents the horizontal position of the node word marked with a pebble p ). i u (i.e., the word order in an input sentence), j represents U = [i,b] wr[i,b] O , 1 i s . The set U the vertical position of u (it is equal to a number of rewrit- • w { | ∈ p ≤ ≤ }} w represents words resulting from the rewritten opera- ings of a word on a particular horizontal position). Each tions within I . edge h = (u,v) E, where u = [i , j ,a] and v = [i , j ,b] R ∈ u u v v Formalization of Word-Order Shifts by Restarting Automata 7

for some non-negative integers i ,i , j , j and a,b Γ, is continues by performing several cycles, and it ends by a u v u v ∈ either tail. A cycle means an application of an enhanced restart- oblique edge iff i = i ; • u 6 v ing meta-instruction on a given configuration, resulting in such edges are interpreted as representing syntactic a new configuration. During each cycle of C, some items relations between respective symbols (words), or of the tape content are deleted, rewritten, or shifted (as in a cycle of the original automaton without structures); fur- vertical edge iff i = i and j = j + 1; • u v v u ther, some edges and (rewritten) nodes are added to the such edges are interpreted as representing a rewriting DR-structure of the corresponding configuration, in accor- of a symbol (word) on the respective position i . u dance with the graph attached to the respective enhanced We say that D = (V,E) is a DR-tree if the graph D is a meta-instruction (see below for more formal description). rooted tree (i.e., all maximal paths in D end in its single A tail consists in an application of an enhanced accept- root). ing meta-instruction – it results in an acceptance of a con- figuration; further, some edges are added to the current It is important to say that a DR-structure pre- Remark: DR-structure in accordance with a graph attached to this serves word order of an original sentence, i.e., the order of enhanced meta-instruction. nodes in DR-structure (corresponding to words and their Let us stress that during a computation, some nodes and categories) is not affected by (possible) shifts. However, edges may be added to the DR-structures of the computa- shifts affect the way how a DR-structure corresponds to tion, and never removed. At the end of each computation, individual reductions of the automaton – an information we obtain a DR-tree covering the whole input sentence. of both original and changed positions of individual words (and their categories) must be recorded during the compu- tation. 3.3.1 Application of an enhanced restarting meta- instruction IER. Let IER = (IR,G) be an enhanced restarting instruction in 3.3 Computations and (Structured) Languages by the previously defined form, with G = (U,H) (see Section pRL e-automata 3.2). An application of IER on an enhanced configuration C = (T ,D ) of a word w results in a new configuration Based on the definition, it is clear that for a pRL - w w w e C = (T ,D ). The application consists in the following automaton M , a reduction relation can be defined in w0 w0 w0 e `Me steps: the same way as the reduction relation M of the corre- pRL ` sponding -automaton M; further, Me accepts the same 1. Choosing a factorization of w of the form w = DR language as the M (leaving aside output -structures), v0a1v1a2 ...vs 1asvs such that vi Ei for all i = REM − ∈ and we can see that a set of reductions (Me) = 0,...,s (see the form of a restarting meta-instruction REM (M), as well. (1)). Let us consider that the items from T contain- pRL w Similarly as in [15], computation of an e-automaton ing the symbols a , ,a are pebbled. 1 ··· s Me is characterized by sequences of its configurations. A If w does not permit any factorization of this form, configuration is a pair of the form Cw = (Tw,Dw) , where then IER cannot be applied on the configuration Cw. Tw (so called tape content of Cw) is a string of items rep- resenting current sentence w on the tape and Dw = (V,E) 2. Transforming the tape content Tw containing the sen- DR is a -structure (see above). The items of Tw create a tence w into the tape Tw0 containing the sentence subset of V such that Tw contains just all roots of individ- w0 = v0r1v1 vs 1rsvs ; ri (1 i s), are described DR ··· − ≤ ≤ ual components (trees) of the -structure Dw (thus each in section 3.1). The transformation of Tw to Tw0 keeps item [i, j,xi] of Tw consists of a word position i (i.e., the the following rules: horizontal index), a symbol x on this position (word of i each item v T (1 i s) which does not the original sentence), and the number of its rewritings j). • i ∈ w ≤ ≤ undergo an editing operation is copied to Tw , Moreover, Tw contains at most one item for each horizontal 0 no deleted item from T is transferred to T , position i. • w w0 Note that – contrary to [15] – the items of Tw are or- and dered; however, their order need not reflect the original each item [k , j ,a ] from T that corresponds to • i i i w word order of an input sentence as the items (triples) may a rewritten symbol a (hence o = wr[i,b], b Γ, i i ∈ be re-ordered by a shift operation. 1 i s ) is replaced by [k , j + 1,b] in T ; ≤ ≤ i i w0 An (accepting) computation C of Me on an input sen- the shift of an item [i, j,a ] is performed in such in in in • i tence w starts in the initial configuration Cw = (Tw ,Dw ): a way that the indices i, j remain unchanged in for the given sentence w = x1 ...xn (x j Γ, 1 j n), T . ∈ ≤ ≤in w0 the initial tape content consists in the string of items Tw = [1,0,x ] [n,0,x ], and the DR-structure Din = (V in, /0), 3. All edges and nodes from the DR-structure D are 1 ··· n w w w where V in = [1,0,x ], ,[n,0,x ] . The computation transferred into the new DR-structure D . w { 1 ··· n } w0 8 M. Lopatková, M. Plátek

4. For each edge e H, a new edge is inserted into the Let AC(Me) denote the set of all accepting computations DR ∈ = ( , ) pRL DR new -structure Dw0 V 0 E0 : of the e-automaton Me. Then the -language of Me is the set DR(Me) = DR(C ) C AC(Me) . if e = ([i,a ],[r,a ]) is a deleting edge from { | ∈ } • i r From the linguistic point of view, DR(M ) H (1 r s), and [k , j ,a ],[k , j ,a ] T Remark. e ≤ ≤ i i i r r r ∈ w should be considered as a set of (shallow) dependency- are the items covering the pebbled symbols a i based syntactic trees / analyses computed by the automa- and ar, respectively, then an oblique edge eo = ([ , , ],[ , , ]) ton Me. ki ji ai kr jr ar is inserted into Dw0 (i.e., into E0). Example: Let us return to the example (1), section 2, Petr se bojí o otce. ‘Peter fears for his father.’ In if e = ([i,ai],[i,bi]) is a rewriting edge and pRL • Sections 3.1.2 and 3.2.2, the e-automaton Me – de- [k , j ,a ] T is the item covering the pebbled i i i ∈ w scribed by two restarting meta-instructions and one ac- symbol a , then a vertical edge e is inserted into i v cepting meta-instruction together with the graphs enhanc- E : e leads from the item [k , j ,a ] to the item 0 v i i i ing these meta-instructions – was presented. The sequence [k , j + 1,b ]. i i i S = I ,I ,I performs the left branch of the analysis At the same time the node [k , j +1,b ] is added ER1 ER2 EA i i i by reduction. In the course of the corresponding computa- to the set V 0. tion, the DR-tree D3 is gradually constructed. Now we can present the accepting computation C = We say that the configuration Cw0 can be reached in one (restarting) step from the configuration C by M (using C0,C1,C2,C3 on the sentence (1) determined by the se- w e in in in the instruction I ) and denote it by C =IER C , or simply quence of meta-instructions S. Let C = (T ,D ), C1 = ER w | Me w0 = (T1,D1), C2 = (T2,D2), C3 = (T3,D3), C0 is the initial con- by Cw Me Cw0 . | figuration of C , C3 is the accepting configuration. Further in in we gradually describe T ,T1,T2,T3, and D ,D1,D2,D3, 3.3.2 Application of an enhanced accepting meta- where D3 = DR(C ). instruction IEA. T in = [1,0,Petr][2,0,se][3,0, bojí ][4,0,o][5,0,otce][6,0, ] · Let IEA = (I,G) be an enhanced accepting meta- Din = (V in, /0) Accept instruction, where I = (a1,...,as, ) and G = (U,H). V in = [1,0,Petr],[2,0,se],[3,0, bojí ],[4,0,o],[5,0,otce][6,0, ] { · } Let Cw = (T,D) be an enhanced configuration of Me with —————– the tape content T = [i , j ,a ], ,[is, j j,as] covering the 1 1 1 ··· T1 = [1,0,Petr][2,0,se][3,0, bojí ][6,0, ] word w= a a . Then the application of I on the con- in · 1 s EA D1 = (V ,E1) ··· = ( , ) figuration Cw results in a new configuration Cw0 T 0 D0 , E = ([4,0,o],[3,0, bojí ]),([5,0,otce],[4,0,o]) 1 { } where D0 = (V 0,E0): —————– T2 = [3,0, bojí ][2,0,se][6,0, ] 1. T 0 = [i1, j1,a1], ,[is, j j,as] = T; · ··· D = (V in,E E ) 2 1 ∪ 2 2. all edges and nodes from the DR-structure D are E = ([1,0,Petr],[3,0, bojí ]) 2 { } transferred into the new DR-structure D0; —————– T3 = T2 3. moreover, for each edge e = ([r,ar],[s,at ]) H the in ∈ D3 = (V ,E1 E2 E3) new edge ([i , j ,a ],[i , j ,a ]) is inserted into D . ∪ ∪ r r ir t t it 0 E = ([2,0,se],[3,0, bojí ]),([6,0, ],[3,0, bojí ]) 3 { · } Similarly as in the previous case, we say that the con- We can see that the presented example is very simple – the context languages from the instructions contain always figuration Cw can be reached in one (accepting) step from 0 one string (sometimes the empty one λ). Moreover, we the configuration Cw by Me using the instruction IEA, and IEA have not used the operation wr[i] at all. That caused that denote it by C =M C0. | e all items used in C have their vertical index equal to 0. We can finally define an accepting computation of an We can notice that D is a formalization of the depen- pRL DR 3 enhanced e-automaton and subsequently also a - dency tree from the example (1). (Let us note that a com- DR language consisting from the output -structures. putation performing the another branch of AR should cre- Let C = C0,C1,...,Ck be a sequence of configurations ate the same DR-tree.) such that Ii – C is the initial configuration for a word w,– C = C + 0 i | Me i 1 holds for all i = 0,...,k 1, Observations. Let us finally (and just informally) formu- − – I are restarting instructions for all i = 0,...,k 2, late some observations concerned the presented notions. i − – Ik 1 is an accepting instruction. 1. The classes of introduced languages, sets of reductions, − DR Then we say that C is an accepting computation of Me, and -languages create infinite hierarchies with respect the word w is accepted by Me, and the DR-structure Dk to the number of used pebbles. We are able to prove ex- (from the last configuration Ck = (Tk,Dk)) is the output actly such propositions. DR-structure of the computation C . We write DR(C ) = 2. On the other hand, we can observe that very few peb- Dk. bles (six, according to our preliminary hypothesis) suffice Formalization of Word-Order Shifts by Restarting Automata 9

to analyze adequately any sentence from the Prague De- References pendency Treebank (see [2]). It means that the complexity of analysis by reduction of natural languages is not too [1] Gerdes, K., Kahane, S.: Defining dependencies (and con- hight; however, it is not too trivial. stituents). In: Gerdes, Hajicová,ˇ Wanner (eds.) Proceedings of Depling 2011. pp. 17–27. Barcelona (2011) 3. In a similar way, we can see that the number of used [2] Hajic,ˇ J., Panevová, J., Hajicová,ˇ E., Sgall, P., Pajas, P., shifts in one cycle does not need to exceed one. Štepánek,ˇ J., Havelka, J., Mikulová, M., Žabokrtský, Z., 4. As far as we can foresee, the use of rewritings within Ševcíková-Razímová,ˇ M.: Prague Dependency Treebank the analysis by reduction is forced only by two linguistic 2.0. Linguistic Data Consortium, Philadelphia (2006) phenomena: coordinations and noun groups with numer- [3] Hajicová,ˇ E., Havelka, J., Sgall, P., Veselá, K., Zeman, D.: als. Moreover, the number of rewritings in one cycle does Issues of Projectivity in the Prague Dependency Treebank. not need to exceed one. The Prague Bulletin of Mathematical Linguistics 81, 5–22 5. Further, it not hard to see that the shift operation can- (2004) [4] Holan, T., Kubon,ˇ V., Oliva, K., Plátek, M.: On Complex- not be simulated by rewritings within the class of pRLe- automata, due to the limited number of rewritings in en- ity of Word Order. Les grammaires de dépendance – Traite- ment automatique des langues 41(1), 273–300 (2000) coded in meta-instructions. [5] Kubon,ˇ V., Lopatková, M.: A case study of a free word 6. Finally, the measure of word-order complexity from [4] order (manuscript) is not bounded by any natural number for DR-languages pRL [6] Kubon,ˇ V., Lopatková, M., Plátek, M.: Studying formal of e-automata. properties of a free word order language. In: Youngblood, G., McCarthy, P. (eds.) Proceedings of the FLAIRS 25 Con- Conclusion and Perspectives ference. pp. 300–305. AAAI Press, Palo Alto (2012) [7] Kunze, J.: Abhängigkeitsgrammatik, Studia Grammatica, The main novelty of the work presented in the paper is an vol. XII. Akademie Verlag, Berlin (1975) enhancement of a formal system in a way that makes it [8] Lopatková, M., Plátek, M., Kubon,ˇ V.: Modeling Syntax possible to capture adequately relations between linguistic of Free Word-Order Languages: Dependency Analysis by notions in the course of (enhanced) analysis by reductions, Reduction. In: Matoušek, V. et al. (ed.) Proceedings of TSD namely it describes dependency structures on the one side, 2005. LNCS, vol. 3658, pp. 140–147. Springer (2005) and (complexity of) word order on the other side (for the [9] Lopatková, M., Plátek, M., Sgall, P.: Towards a Formal clarity reason, we do not distinguish between input sym- Model for Functional Generative Description: Analysis by bols and their categories in this paper). However, the in- Reduction and Restarting Automata. The Prague Bulletin vestigation of these notions is complete neither from the of Mathematical Linguistics 87, 7–26 (2007) linguistic point of view, nor from the point of view of the [10] Marcus, S.: Sur la notion de projectivité. Zeitschrift fur formal automata theory. Mathematische Logik und Grundlagen der Mathematik 11(1), 181–192 (1965) As for the future work, we plan to continue the research in the direction of combining more types of linguistic phe- [11] Marecek,ˇ D., Žabokrtský, Z.: Exploiting reducibility in un- supervised dependency parsing. In: Proceedings of EMNL- nomena which influence the degree of word order freedom CoNLL 2012. pp. 297–307. ACL (2012) for natural languages as well as enhancing the proposed [12] Mel’cuk,ˇ I.A.: Dependency in language. In: Gerdes, Ha- formal system so that it would capture the added phenom- jicová,ˇ Wanner (eds.) Proceedings of Depling 2011. pp. 1– ena. Further research of the presented type of automata 16. Barcelona (2011) should present its properties in an exact way, as well. AR [13] Otto, F.: Restarting Automata and Their Relation to the Further, we plan to propose a refinement of allow- Chomsky Hierarchy. In: Ésik, Z., Fülöp, Z. (eds.) Proceed- ing to set constraints on possible reduction steps, namely a ings of DLT 2003. LNCS, vol. 2710, pp. 55–74. Springer, constraint on projective reductions only, as the additional Berlin (2003) experiments with a different or modified set of constraints [14] Otto, F.: Restarting Automata. In: Ésik, Z., Martin-Vide, applied on the shift operation will probably provide new C., Mitrana, V. (eds.) Recent Advances in Formal Lan- measures of word order freedom. An interesting research guages and Applications, Studies in Computational Intelli- direction in this respect would be the investigation of pro- gence. vol. 25, pp. 269–303. Springer-Verlag, Berlin (2006) jectivization of sentences prior to the individual reductions [15] Plátek, M., Mráz, F., Lopatková, M.: (In)Dependencies of AR, which might completely change the picture and al- in Functional Generative Description by Restarting Au- low for a more detailed investigation of an interplay be- tomata. In: Bordihn, H. et al. (ed.) Proceedings of NCMA tween clitics and non-projective sentences, see also [5]. 2010. [email protected], vol. 263, pp. 155–170. Österreichis- Further investigation of the shift operation might also che Computer Gesellschaft, Wien, Austria (2010) help to describe the relationship between different levels of [16] Plátek, M., Mráz, F., Lopatková, M.: Restarting Automata formal description of natural languages in the Functional with Structured Output and Functional Generative Descrip- Generative Description, namely between the tectogram- tion. In: Dediu A. et al. (ed.) Proceedings of LATA 2010. matical and analytical layer. This would strengthen the LNCS, vol. 6031, pp. 500–511. Springer, Berlin Heidelberg (2010) formal background of the theory.