AYJ

FORGOTTEN ISLANDS OF REGULARITY REGULAR UNIVERSE OF LANGUAGE MODELS AND ITS CONTINUING EXPANSION ANSSI YLI-JYRÄ Markov Kleene chains closure, union, caten. LOCALITY & REGULARITY

18/12/2017 Moorehttps://upload.wikimedia.org/wikipedia/commons/1/1f/Moore-Automat-en.svg RNN transition logicfinite-stateoutput machine logic input state memory output S Σ 0 S n+1 S S n T G Λ R

clock reset

https://upload.wikimedia.org/wikipedia/commons/1/1f/Moore-Automat-en.svg 1/1 TRIVIAL FINITE-STATE PHONOLOGY

GENERATIVE PHONOLOGY (Chomsky & Halle 1968)

Universal computing (Chomsky 1963, Johnson 1972, Ristad 1990)

Computes PARTIAL functions

Problematic as a Theory (Popper 1959, Johnson 1972)

NAIVE FINITE-STATE PHONOLOGY

right linear derivation α → βγ → ββ’γ’ →…

based on a limited view of regularity - not linguistically intriguing TRUE FINITE-STATE PHONOLOGY

NON-ITERATED FUNCTIONAL RULES (Johnson 1972)

Generative phonological rules have context conditions: α → β / γ _ γ’

Practical grammars with simultaneous and linear application modes

Test contexts with a bi(directional) machine (Schützenberger 1961)

Surprisingly reduced linguistically interesting rule into FS machines

LIMITATION

No cyclic rules, but composition (Schützenberger 1961) FINITE-STATE UNIVERSAL MODELS

•BRAIN COMPATIBLE •PRACTICAL DECIDABLE •EFFICIENT •GOOD THEORY •ALGORITHMIC •ADEQUATE REGULAR

•SAFE AYJ (4) just backs up until it has tested the precondition. In our example, the precondition is just the sux[C][y][T]:

(4.1) !!!!!!!!$ # g l o s s y T $ $ $ $ (4.2) (4) (4.3) !!!!!!s i e s t # $

With this change, long are produced in a zigzag style (5) where every rule application may back up some letters. (4) just backs up until it has tested the precondition. In our example, the precondition is just the sux[C][y][T]: !! - (4.1) !!!!!!!! , $ # g l o s s y T $ $ $ $ !! - (4.2) (4) (5) , (4.3) !! !!!!!!s i e s t # $ ... - With this change, long words are produced in a zigzag style (5) where every , rule application may back up some letters. !! Since the union of the ax-rules is applied repeatedly to its own output, the !! - standard two-part regularity condition of phonological grammars does not apply. , However, as long as the derivation deletes and appends new material only at the !! - BUT (5) , right end of the string, the resulting process is linear and, intuitively, a regular ITERATED!! DERIVATION... grammar. In addition, IN theHUNSPELL moves taken by the TM can now be deterministic because- the machine does not completely rewind the tape at any point but , GOES BEYONDalways!! KAPLAN makes relative moves& KAY that allow (1994) it to remember its previous position. Since the union of the ax-rules is applied repeatedly to its own output, the standard two-part regularity condition of phonological2.3 grammars Linear does Encoding not apply. However, as long as the derivation deletes and appends new material only at the right end of the string, the resulting process is linearAlthough and, intuitively, the grammar a regular represented by a lexicon does not satisfy the grammar. In addition, the moves taken by the TM can now be deterministic classical two-part condition of finite-state phonology, it is equivalent to a finite- becauseLászló the Németh, machine Viktor does not Trón, completely Péter rewind Halácsy, the András tape at any Kornai, point András but Rung, and István Szakadát. always makes relative moves that allow it to rememberstate transducer its previous position. when restricted to the suxrules. Leveraging the open source ispell codebaseThere for are minority now some language methods analysis. to compile hunspell lexicons to finite-state 2.3Proceedings Linear Encoding of the SALTMIL Workshoptransducers. at LREC 2004 Early experiments on compilation are due to Gyorgy Gyepesi (p.c., Although the grammar represented by a hunspell2007)lexicon and does others not satisfy in Budapest. the The author developed his solution (Yli-Jyr¨a, classical two-part condition of finite-state phonology,2009) it is using equivalent a variant to a finite- of Two-Level Morphology (Koskenniemi, 1983). This state transducer when restricted to the suxrules.method viewed the lexicon as a collection of constraints that described linearly There are now some methods to compile hunspellencodedlexicons backing to finite-state up and suxation in derivations. The method included an e- transducers. Early experiments on compilation are due to Gyorgy Gyepesi (p.c., 2007) and others in Budapest. The author developedcient one-shot his solution compilation (Yli-Jyr¨a, algorithm to compile and intersect several hundreds 2009) using a variant of Two-Level Morphologyof thousands(Koskenniemi, of 1983). lexical This context restriction rules in parallel as if the lexical contin- method viewed the lexicon as a collection of constraintsuations that (morphotaxis) described linearly were phonological constraints. A similar method, finally encoded backing up and suxation in derivations.implemented The method included by his an colleagues, e- Pirinen and Lind´en (2010), separated the lexi- cient one-shot compilation algorithm to compilecal and continuations intersect several hundredsfrom the phonological changes at morpheme boundaries and of thousands of lexical context restriction rules in parallel as if the lexical contin- uations (morphotaxis) were phonological constraints.used A a similar three-step method, approach finally where the final step composed the lexicon with the implemented by his colleagues, Pirinen and Lind´en (2010), separated the lexi- cal continuations from the phonological changes at morpheme boundaries and used a three-step approach where the final step composed the lexicon with the 6

6 GOAL:

CHARACTERIZE ALL REGULAR GRAMMARS AND LANGUAGE MODELS BOUNDS OF REGULARITY AYJ People’s Daily Online © CEN REGULARITY MEANS FINITE PARALLELISM REGULARITY MEANS LINEAR BOUNDED SPACE REGULARITY MEANS FINITE COMPOSITION VISIBLE TRACES

CC-BY-SA Xvazquez AND … WRITING HEAD BOUNDED NUMBER OF SPIDER WEBS OUTER LINKS Kornai & Tuza 1992: Narrowness, Path-width and their application in NLP REGULARITY À LA HENNIE (1965)

BOUNDED LTIME ONE-TAPE TURING MACHINE

O(k) CONTROL STATES (BOUNDED PARALLELISM)

O(n) TAPE CELLS MSO Definable String Transductions 247 • (BOUNDED SPACE)

O(n) TIME STEPS (BOUNDED TIME)

CAN BE NONDETERMINISTIC Fig. 8. Track for a3b2aba , Example 9. ⊢ ⊣ (TADAKI ET AL. 2010) Finally, the first visiting sequence of a computation should start with a visit ( , q , ϵ, α), and exactly one visiting sequence should end with a visit ∗ in + (−ϵ, q f , , λ). Since∗ the number of visits to each position is bounded, the visiting sequences come from a finite set, and we can interprete these sequences as symbols from a finite alphabet. Each k-visiting computation is specified by a string over this alphabet, and we will call these strings k-tracks, e.g., the 3-track in Figure 8 specifies the computation of the Hennie machine of Example 9 on input a3b2aba. It should be obvious from the above remarks that the language of such spec- ifications is regular (e.g., see Lemma 2.2 of Greibach [1978c], or Lemma 1 of Chytil and Jakl´ [1977]). For instance, it is the heart of the proof in Hopcroft and Ullman [1979, Theorem 2.5] of the result that two-way finite-state automata are equivalent to their one-way counterparts [Rabin and Scott 1959; Shepherdson 1959].

PROPOSITION 23. Let be a Hennie machine, and let k be a constant. The k-tracks for successful k-visitingM computations of form a regular language. M 7.2 Characterizations Using Hennie Machines From Proposition 23, using standard techniques (e.g., see Chytil and Jakl´ [1977, Lemma 1]) we obtain the following decomposition of nondeterministic Hennie transductions. Note that this decomposition already features in Theorem 20 as characterization of NMSOS.

LEMMA 24. NHM MREL 2DGSM NMSOS. ⊆ ◦ = PROOF. Let be a Hennie machine, finite-visit for constant k; each pair (w, z) in the transductionM realized by can be computed by a k-visiting com- putation. M We may decompose the behavior of on input w as follows. First, a rela- beling of w guesses a string of k-visitingM sequences, one for each position of the input⊢ tape,⊣ such that the first symbol of each visiting sequence matches the input symbol of the corresponding tape position. Then, a 2DGSM verifies in a left-to-right scan whether the string specifies a valid computation, a track, of for w, cf. Proposition 23. If this is the case, the 2DGSM returns to the left end markerM and simulates on this input, following the k-visiting computation previously⊢ guessed. M

ACM Transactions on Computational Logic, Vol. 2, No. 2, April 2001. REGULARITY KEEPS SURPRISING

A. HENNIE (1965) GOES BEYOND THE CLASSICAL FS. PHONOLOGY

1. Restricted Application JOHNSON (1972); KK (1994)

2. Iterated application IN HUNSPELL (NÉMETH ET AL 2004)

B. HENNIE (1965) IS RELEVANT TO REPRESENTATION OF

3. Syntax (Nederhof & YJ 2017; Y-J 2017a, 2017b)

4. Semantics and Pragmatics (Gordon & Hobbs 2017; Kornai 2017 manus.) AYJ 5. RNN, including backpropagation Table 1: The coverage of UD v.2 data with depth bounded weak edge bracketing

lang N depth 0 depth 1 depth 2 depth 3 depth 4 depth 5 depth 6 depth 7 Arabic 26722 4.42% 20.93% 65.09% 94.39% 99.64% 99.99% +0.011% (3) Catalan 14832 1.27% 19.01% 70.39% 96.07% 99.62% 99.99% +0.007% (1) Czech 102660 10.60% 43.11% 86.77% 98.47% 99.89% 99.99% +0.010% (10) German 14917 2.06% 43.11% 87.52% 98.56% 99.91% 99.97% +0.027% (4) English 19785 16.61% 53.59% 91.10% 99.21% 99.96% 100.00% Spanish 31546 1.59% 24.61% 77.27% 97.24% 99.79% 100.00% +0.003% (1) Finnish 30437 30.03% 77.46% 96.54% 99.55% 99.96% 99.99% +0.007% (2) French 19294 2.86% 37.87% 88.24% 98.89% 99.94% 99.99% +0.005% (1) Greek 28478 16.47% 67.77% 95.75% 99.73% 99.98% 100.00% Hebrew 5725 1.61% 24.63% 80.61% 98.72% 99.93% 100.00% Hindi 14963 0.45% 37.19% 82.74% 98.04% 99.90% 99.98% +0.020% (3) Croatian 8289 2.23% 33.78% 86.39% 98.90% 99.93% 100.00% Hungarian 1351 1.11% 22.95% 68.91% 94.89% 98.96% 99.78% +0.148% (2) +0.074% (1) Italian 14992 4.93% 47.62% 88.22% 98.63% 99.92% 100.00% Japanese 7675 1.98% 48.60% 96.73% 99.96% 100.00% 100.00% Latin 33166 20.14% 61.14% 90.57% 98.86% 99.94% 100.00% Latvian 3054 16.50% 57.50% 88.93% 98.23% 99.80% 99.97% +0.033% (1) Dutch 19891 18.32% 60.86% 93.81% 99.56% 99.98% 100.00% Polish 7127 13.03% 75.28% 98.58% 99.94% 99.99% 100.00% Portuguese 19765 4.61% 32.02% 81.33% 97.98% 99.83% 99.99% +0.005% (1) Romanian 8795 0.80% 25.78% 84.51% 98.53% 99.87% 99.99% +0.011% (1) Russian 59827 12.57% 73.86% 97.03% 99.76% 99.98% 100.00% +0.002% (1) Slovenian 9349 17.82% 57.60% 94.34% 99.68% 99.99% 100.00% Scandinavian 47574 12.46% 55.27% 92.52% 99.35% 99.96% 100.00% Turkish 4660 22.08% 71.20% 92.68% 98.88% 99.89% 99.98% + 0.021% (1) Chinese 4497 0.00% 19.64% 70.49% 94.53% 99.51% 99.96% + 0.044% (2) other UD 71147 12.55% 57.43% 91.58% 99.21% 99.96% 100.00% 630518 11.07% 50.38% 88.38% 98.66% 99.91% 99.994% + 0.005% (33) + 0.0003% (2)

Athat REGULAR a finite-state approximation MODEL does not FOR contain UNIVERSALanced bracketing. ForDEPENDENCIES example, if we combine the the correct analysis graph or digraph. In order edge brackets in [ ][ ] and [ ], we Anssi Yli-Jyrä. Bounded-Depth High-Coverage Search Space for Noncrossing Parses{}{}. Proceedings{} of the{} 13th{}{} Conference to learn about the probability of the out-of-the- obtain [ [ ][ ]] that encodes a slightly dif- on Finite-State Methods and Natural Language Processing, Umeå, Sweden.{} 2017.{} ACL.{} search-space event, we used the UD treebanks as ferent set of edges. This method works also with the first proxy to find out how often a given nesting reduced bracketing where the opening and clos- depth is exceeded in gold trees. ing superbrackets cancel one another. The result- ing code string preserves the number of edges but Our current encoding can handle only noncross- some ends of these edges may change. The re- ing trees such as projective trees. However, the sult may be a cyclic or disconnected graph but trees in UD version 2 treebanks do not have this it preserves noncrossing parts of the graph intact restriction. It is well known that the propor- because there the brackets match the two ends of tion of non-projective, and thus crossing, trees each edge. is relatively high for some languages (Gomez-´ Rodr´ıguez, 2016). If all nonprojective analyses The total number of sentences in the sample were discarded, most of the long sentences would was 630 518. This includes both the training and have been excluded, with significant effect on our the development sections of the UD v.2 treebanks. experiments. The data was encoded with our new encoding scheme and automatically converted to noncross- For the experiments, we had to enforce non- ing undirected graphs before the nesting depth of crossing structure to the data. The standard ap- each sentence was computed.6 proach is to perform lift transformations that move Table 1 describes how the nesting depth corre- crossing edges higher in the dependency tree. Al- sponds to coverage in the UD2 data set. It indi- though there are methods to minimize the num- cates that the depth 5 is a good compromise be- ber of lifts either heuristically or exactly, the out- tween depth and coverage. Under this depth, the put of such a transformation is not uniquely de- search space contains 99.994% of all the trees in termined. The second way to projectivise trees is the treebanks. Only 35 sentences require weak to keep the dominance tree intact but reorder the bracketing whose nesting depth is more than 5. nodes of the tree into a canonical order that main- Thus the flat out-of-the-search-space failure rate tains the relative order of the immediate depen- is 0.006% of the sentences only. dents of each head. The third approach, actually used by us, views the trees as undirected trees and 6The code for the encoding script is in https:// takes advantage of the algebraic properties of bal- github.com/amikael/depconv. Generic Axiomatization of Families of Noncrossing Graphs in Dependency Anssi Yli-Jyra¨ (University of Helsinki) and Carlos Gomez-Rodr´ ´ıguez (Universidade da Coruna)˜

Abstract Generic Representation for the Subfamilies of Digraphs

1. We develop a simple linear encoding supporting general noncrossing digraphs. 2. We show that the encoded noncrossing digraphs form a context-free language. NC-DIGRAPH 3. We present an latent encoding that can be used to characterize various families of digraphs by forbidden local patterns. UNAMBS CONNW ORIENTED NC-GRAPH (INV) 4. This can be used to enable generic context-free parsers that produce different families of noncrossing graphs with the same set of inference rules. unamb.or. OUT ACYCU w.c.unamb. w.c.or. ACYCD

ACYCU CONNW (forest) (w.c. graph) Noncrossing Digraphs as Code Strings out oriented out m-forest mixed tree w.c.unamb.or. multitree w.c.dag NC-DIGRAPH Enc : LNC-DIGRAPH out mixed tree or.forest w.c.multitree $ tree

w.c. out oriented out or.forest polytree

out or.tree

1 2 3 4 5 6 7 All elements of the LNC-DIGRAPH ontology are unambiguous and closed under intersection. /

<[ ]// > < / > / > {} {} {} {} {} {} Enumeration Experiment per n Nodes + Simplification to the brackets Name Sequence prefix for n = 2,3,... Example Name Sequence prefix for n = 2,3,... Example Axioms and Forbidden Patterns+ Treatment in Noncrossing of crossing Digraphs edges digraph (KJ): 4,64,1792,62464,2437120,101859328 weakly projective 4,36,480,7744,138880,2661376 hlat(D55 Gn Reg ) digraph hlat(D55 Gn Reg PW ) \ \ lat 1 2 3 4 5 \ \ lat \ 1 2 3 4 5 Bounded treewidth MSO properties become LOGSPACE decidable (by Courcelle’s theorem) w.c.digraph 3,54,1539,53298,2051406,84339468 w.p. w.c.digraph 3,26,339,5278,90686,1658772 hlat(D55 Gn Reg CW ) hlat(D55 Gn Reg PW CW ) ) \ \ lat \ 1 2 3 4 5 \ \ lat \ \ 1 2 3 4 5 unamb.digr. 4,39,529,8333,142995,2594378 w.p. unamb.digr. 4,29,275,3008,35884,453489 hlat(D55 Gn Reg US) hlat(D55 Gn Reg PW US) INV OR. OUT \ \ lat \ 1 2 3 4 5 \ \ lat \ \ 1 2 3 4 5 u v u v u v u v y u v y u v y m-forest 4,37,469,6871,109369,1837396,32062711 w.p. m-forest 4,29,273,2939,34273,421336 hlat(D55 Gn Reg AU ) hlat(D55 Gn Reg PW AU ) A REGULAR MODEL FOR UNIVERSAL DEPENDENCIES \ \ lat \ 1 2 3 4 5 \ \ lat \ \ 1 2 3 4 5 out digraph 4,27,207,1683,14229,123840,1102365 w.p. out digraph 4,21,129,867,6177,45840,350379 Anssi Yli-Jyrä. Bounded-Depth High-Coverage Search Space for Noncrossing Parses. Proceedings of the 13th Conference ACYC ACYC hlat(D55 Gn Reg Out) hlat(D55 Gn Reg PW Out) D on Finite-State Methods and Natural Language Processing, Umeå, Sweden. 2017. ACL.U \ \ lat \ 1 2 3 4 5 \ \ lat \ \ 1 2 3 4 5 u v y u v y u v u v y or. digraph 3,27,405,7533,156735,3492639,77539113 w.p. or.digraph see w.p.dag hlat(D55 Gn Reg O) hlat(D55 Gn Reg PW O) see w.p.dag no arc no arc no arc no arc \ \ lat \ 1 2 3 4 5 \ \ lat \ \ CONNW PROJW dags (A246756): 3,25,335,5521,101551 w.p. dag 3,21,219,2757,38523, 574725, 8967675 hlat(D55 Gn Reg AD) hlat(D55 Gn Reg PW AD) ... v y ...... v ... u v y u v y \ \ lat \ 1 2 3 4 5 \ \ lat \ \ 1 2 3 4 5 w.c. dag (KJ): 2,18,242,3890,69074,1306466 w.p. w.c. dag 2,14,142,1706,22554,316998,4480592 hlat(D55 Gn Reg AD CW ) hlat(D55 Gn Reg PW AD CW ) \ \ lat \ \ 1 2 3 4 5 \ \ lat \ \ \ 1 2 3 4 5 UNAMBS multitree 3,19,167,1721,19447,233283,2917843 see oriented forest w.p. multitree 3,17,129,1139,11005,112797,1203595 hlat(D55 Gn Reg AD US) hlat(D55 Gn Reg PW AD US) u v y u v y u v y u v y u v x y u v x y \ \ lat \ \ or w.c. multitree \ \ lat \ \ \ 1 2 3 4 5 or.forest 3,19,165,1661,18191,210407,2528777 w.p. or.forest 3,17,127,1089,10127,99329,1010189 hlat(D55 Gn Reg AD AU ) hlat(D55 Gn Reg PW AD AU ) \ \ lat \ [ 1 2 3 4 5 \ \ lat \ \ [ 1 2 3 4 5 w.c. multitree 2,12,98,930,9638,105798,1201062 w.p. w.c. multitree 2,10,68,538,4650,42572,404354 Latent Edge Types hlat(D55 Gn Reg AD US CW ) hlat(D55 Gn Reg PW AD US CW ) \ \ lat \ \ \ 1 2 3 4 5 \ \ lat \ \ \ \ 1 2 3 4 5 out or.forest 3,16,105,756,5738,45088,363221 w.p. out or.forest (A003169): 3,14,79,494,3294,22952 2-edge-chain hlat(D55 Gn Reg AD Out) hlat(D55 Gn Reg PW AD Out) \ \ lat \ \ 1 2 3 4 5 \ \ lat \ \ \ 1 2 3 4 5 A chain consists of contiguous linear edge brackets, e.g. [ [][][] ][] polytree (A153231): 2,12,96,880,8736,91392 w.p. polytree (A027307):2,10,66,498,4066,34970 hlat(D55 Gn Reg AD CW AU ) hlat(D55 Gn Reg PW AD CW AU ) A loose chain starts immediately after a boundary . 3-edge chain \ \ lat \ \ \ 1 2 3 4 5 \ \ lat \ \ \ \ 1 2 3 4 5 {} z }| { out or.tree (A174687): 2,9,48,275,1638,9996 projective (A006013): 2,7,30,143,728,3876,21318 A local automaton Chains decorates the chains with latent edge types. hlat(D55 Gn Reg AD CW Out) out or.tree hlat(D55 Gn Reg PW AD CW Out) | {z } \ \ lat \ \ \ 1 2 3 4 5 \ \ lat \ \ \ \ 1 2 3 4 5 graph (A054726): 2,8,48,352,2880,25216 connected graph (A007297): 1,4,23,156,1162,9192 hlat(D55 Gn Reg I) hlat(D55 Gn Reg I CW ) ’ from the initial state 0 for a non-loose chain; \ \ lat \ 1 2 3 4 5 \ \ lat \ \ 1 2 3 4 5 . from the initial (and only) state 1 for a loose chain; forest (A054727): 2,7,33,181,1083,6854 tree (A001764,YJ): 1,3,12,55,273,1428,7752 hlat(D55 Gn Reg I AU ) hlat(D55 Gn Reg I AU CW ) I a bidirectional chain: u (v )y; \ \ lat \ \ 1 2 3 4 5 \ \ lat \ \ \ 1 2 3 4 5 $ $ 0A a primarily bidirectional forward chain: u v y; $ ! F a forward chain: u v y; The correctness was verified against OEIS, the prior art, and procedural enumerate-test algorithms. ! ! VQP a primarily forward chain: u v ( )y; ! $ ···! 1C a primarily forward 1-turn chain: u v y; Application to Generic Parsing ! 2E a primarily forward 2-turn chain: u v x y; ! ! 3Z a 3-turn chain; n 1 I n-Node Digraphs: the set LNC-DIGRAPH Gn where Gn = B⇤( B⇤) . 0afvqp1c2e (analogously for primarily backward chains) \ {}

I ( , ) = ( , ( , )) / Arc-Factored Parsing: Each possible arc i j has a positive weight w w S i j . The ij · 0

<./. A = arg maxA L G wi,j 2 family of NC-DIGRAPHs\ n [I’]I’ /F’>F’ /.>. (i,j) A X2 1 2 3 4 5 6 7 I Indexed brackets: Edges in [ [ ] [ ] ] get weights from a D grammar: 1 1{} 2 2{}{} 4 4 2 w12 w13 w14 Local automata Looseness and Covered select the initial state and some further edge subtypes S ✏ ; S [ S] S; S [ S] S; S [ S] S; ! | {} ! 1 2 ! 1 3 ! 1 4 in Chains, respectively. The subtypes of edges are defined according to the chains they cover. w23 w24 w34 S [2S]3S; S [2S]4S; S [3S]4S. The bracket types I, Q, q and the brackets A, >a, >C, c, E, >e indicate arcs that constitute a cycle ! ! ! \ \ \ I Intersection: (D Reg G Constraints) gives a weighted CFG. with the chain they cover and the bracket types V and v indicate arcs that cover 2-turn chains: 55 \ lat \ n \ I Dynamic Programming: The arg max inference reduces to WCFG parsing.

/I’>I’ /I’>I’

/ / I Lexicalized Search Space: The axioms and lexical contraints on the feasible brackets for each

token can be implemented in lexical entries (compare: multi-modal CCG) that refine Gn.

/ / /

/F’>F’ /F>F <0 0 1 <2 2

Deciding Forbidden Patterns in Digraphs via Star-Free Finite State Constraints 1. Linear Encoding (Enc): Noncrossing digraphs encoded bijectively as strings that constitute a context-free subset of D4. The forbidden patterns become star-free (FO local) and decidable in deterministic linear time. 2. Context-Free Axioms: ACYCU CONNW ACYCD UNAMBS PROJW ORIENTED INV OUT The current MSO definable axioms become unambiguous CF languages I axioms become star-free (mostly local) constraints for latent bracketing I cf. linear time and LOGSPACE testability of MSO under bounded treewidth (Courcelle 1990; Elberfeld et al. 2010) 3. Ontology of Digraphs: The axioms generate a semi-lattice containing 12 known categories plus many new ones. 4. Generic parsing: local (strictly locally testable) languages piecewise One parser or enumeration algorithm for all families of digraphs. testable | {z } | {z } I Weighted CF parsing with dynamic programming I Inference with constraint relaxation I Lexical control over digraph properties Three Representations for LNC-DIGRAPH

L is an unambiguous CFL and a subset of D ,aDyck language over letters [, ], , . Contact Information NC-GRAPH 2 { } derivational [email protected] [email protected] S BS S ✏; S BT S; T BS S; B S representation: ! | | 0 ! | ! | ! [ 0] Department of Modern Languages Departamento de Computacion´ {} {} {} University of Helsinki Universidade da Coruna˜ S [S] S 1st morphic ! S S S (D3 Reg) h = ! [0 ]0 representation: \ S S S ! ! S ✏{ } ! T Acknowledgements

There are similar representations for L D having letters [, ], /, >, <, /, , . NC-DIGRAPH ⇢ 4 { } I The Academy of Finland (dec. No 270354 - A Usable Finite-State Model for Adequate Syntactic Complexity) Latent edge types are distinguished in the internal language I The University of Helsinki (dec. RP 137/2013 - SMT based on D-Tree Contraction Grammars and FSTs) L = D (Reg Chains Looseness Covered) I The European Research Council (ERC): Horizon 2020 (agr. No 714150 - FASTPARSE) NC-DIGRAPHlat 55 0 \ \ \ \ I TELEPARES-UDC project (FFI2014-51978-C2-2-R) from MINECO. This give rise to the third representation for LNC-DIGRAPH: 2nd morphic (D Reg ) h = L h representation: 55 \ lat lat NC-DIGRAPHlat lat

[email protected], [email protected] ACL 2017, Vancouver, July 30-August 4, 2017 PARTIAL CHARACTERIZATIONS:

1. REGULAR GRAMMARS (HALLE & CHOMSKY 1968) 2. REG. SUBSET OF G.P. (JOHNSON 1971; K.&K. 1994) 3. LTIME 1-TAPE TMS (HENNIE 1965; TADAKI 2010) 4. N LOG N STRICT TIME BOUND (TRAKHTENBROT 1964)

REGULARITY OF TURING MACHINES IS UNDECIDABLE => THERE IS ALWAYS ANOTHER FAMILY OF FINITE- STATE REDUCIBLE TURING HAPPY MACHINES (AN ISLAND OF REGULARITY)!!! BIRTHDAY! VisitFinland.com