Facult´edes Sciences Appliqu´ees Universit´eLibre de Bruxelles D´epartement CoDE Visibly Pushdown Transducers Fr´ed´eric SERVAIS Th`ese pr´esent´ee en vue de l’obtention du grade de Docteur en Sciences Appliqu´ees Sous la direction du Prof. Esteban Zim´anyi et co-direction du Prof. Jean-Fran¸cois Raskin Visibly Pushdown Transducers Fr´ed´eric SERVAIS Th`ese r´ealis´ee sous la direction du Professeur Esteban Zim´anyi et co-direction du Professeur Jean-Fran¸cois Raskin et pr´esent´ee en vue de l’obtention du grade de Doc- teur en Sciences Appliqu´ees. La d´efense publique a eu lieu le 26 septembre 2011 `a l’Universit´eLibre de Bruxelles, devant le jury compos´ede: • Rajeev Alur (University of Pennsylvania, United States) • Emmanuel Filiot (Universit´eLibre de Bruxelles, Belgium) • Sebastian Maneth (National ICT Australia (NICTA)) • Frank Neven (University of Hasselt, Belgium) • Jean-Fran¸cois Raskin (Universit´eLibre de Bruxelles, Belgium) • Stijn Vansummeren (Universit´eLibre de Bruxelles, Belgium) • Esteban Zim´anyi (Universit´eLibre de Bruxelles, Belgium) A` mes parents iii R´esum´e Dans ce travail nous introduisons un nouveau mod`ele de transducteurs, les trans- ducteurs avec pile `acomportement visible (VPT), particuli`erement adapt´eau traite- ment des documents comportant une structure imbriqu´ee. Les VPT sont obtenus par une restriction syntaxique de transducteurs `apile et sont une g´en´eralisation des transducteurs finis. Contrairement aux transducteurs `a piles, les VPT jouissent de relativement bonnes propri´et´es. Nous montrons qu’il est d´ecidable si un VPT est fonctionnel et plus g´en´eralement s’il d´efinit une relation k-valu´ee. Cependant cette classe de transducteurs n’est pas close par composition, contrairement `ala classe des transducteurs finis. De plus le probl`eme du typage des VPT par rapport aux langages ”visibly pushdown” est ind´ecidable. Nous introduisons une sous classe des VPT, form´ee par les VPT bien imbriqu´es. Cette classe est close par composition et le typage par apport aux langages visibly pushdown est d´ecidable. Nous montrons ensuite que la classe des VPT est ´egalement close par ’look-ahead’. De plus les transformations fonctionnelles sp´ecifi´ees par des VPT peuvent toujours ˆetre d´efinies par des VPT d´eterministes avec look-ahead. Ces propri´et´es t´emoignent de la robustesse de cette classe. Finalement nous montrons qu’il est possible de s’assurer si une transformation d´efinie par un VPT fonctionnel peut ˆetre ex´ecut´ee avec une quantit´ede m´emoire raisonnable quelque soit le document `atraiter. Par raisonnable nous entendons une quantit´ede m´emoire qui ne d´epend pas de la taille du document, mais peut cependant d´ependre du niveau d’imbrication de celui-ci. Cette quantit´eest en effet la quantit´e minimale n´ecessaire pour traiter ce genre de document, car par exemple pour v´erifier que la structure d’un document imbriqu´eest correcte, une quantit´ede m´emoire pro- portionnelle au niveau d’imbrication est n´ecessaire. iv Abstract The present work proposes visibly pushdown transducers (VPTs) for defin- ing transformations of documents with a nesting structure. We show that this subclass of pushdown transducers enjoy good properties. Notably, we show that functionality is decidable in PTime and k-valuedness in co-NPTime. While this class is not closed under composition and its type checking problem against vis- ibly pushdown automata is undecidable, we identify a subclass, the well-nested VPTs, closed under composition and with a decidable type checking problem. Furthermore, we show that the class of VPTs is closed under look-ahead, and that the deterministic VPTs with look-ahead characterize the functional VPTs transductions. Finally, we investigate the resources necessary to perform trans- formations defined by VPTs. We devise a memory efficient algorithm. Then we show that it is decidable whether a VPT transduction can be performed with a memory that depends on the level of nesting of the input document but not on its length. Acknowledgements My first and foremost thanks goes to my advisors Esteban Zim´anyi and Jean- Fran¸cois Raskin for their support and invaluable help in achieving this work. These years have given me the opportunity to highly value your professional and personal qualities. A special thanks goes to the members of the jury for the time and effort they invested in carefully reviewing this work as well as for their useful feedback. This work would not have been the same without Emmanuel Filiot. We worked together for the last couple of years. Manu let me say my most sincere ’thank you’. This work was also made possible by the so-called french connection: Pierre- Alain and Jean-Marc from Marseille, and Olivier from Lille. Working with you all was a great pleasure. My warm thanks goes to each of you. I owe a big thank to Boris, Serge, Ivan, Fr´ed´eric, Olivier, Fran¸cois, Pierre, Rafaella, C´edric, Nicolas, Gabriel, Thierry, Jean, Laurent and Laurent for their support, discussions, and all the time spent together throughout these years at the university. Thank you all! I would also like to thank colleagues from the laboratory WIT: Mohammed, Jean-Michel, Vinciane, Aura, . As well as colleagues from the department of informatics: Jean, Raymond, Pascaline, V´eronique, Maryka, Gilles, Mahsa. My infinite gratitude goes to my friends for being there throughout those, sometimes difficult, moments: Isa, Schou, Fran¸cois, Magda, Marielle, Manu, Kostas, Liv, Ivan, Marloes, Ewa, Oliver and Olivier, Sylviane, Alain, Georges, Marc, Thierry, Samir, and all those that I, unforgivably, fail to mention. Finally I thank my parents and my family for their care and support. And last but certainly not least: Eleonora. v vi Contents 1 Introduction 1 2 Preliminaries 11 3 Finite State and Pushdown Transducers 17 3.1 FiniteStateMachines. 19 3.1.1 FiniteStateAutomata . 19 3.1.2 FiniteStateTransducers . 23 3.2 PushdownMachines ......................... 37 3.2.1 Context-FreeGrammars . 37 3.2.2 PushdownAutomata . 38 3.2.3 Pushdown Transducers . 44 3.3 Conclusion............................... 49 4 Visibly Pushdown Automata 51 4.1 Definitions............................... 52 4.1.1 Structured alphabet and nesting . 53 4.1.2 Visibly pushdown automata . 54 4.1.3 Examples ........................... 56 4.2 ClosureProperties .......................... 57 4.3 DecisionProcedures . .. .. 62 4.4 Reduced VPA ............................. 64 4.5 Conclusion............................... 66 5 Visibly Pushdown Transducers 69 5.1 Definitions............................... 70 5.2 Examples ............................... 72 5.3 Properties ............................... 75 vii viii Contents 5.3.1 Expressiveness. 75 5.3.2 ClosureProperties . 76 5.3.3 DecisionProblems . 78 5.4 Functional VPTs ........................... 79 5.5 k-valued VPTs ............................. 84 5.5.1 Reversal-Bounded Counter Automata . 85 5.5.2 Multiple Morphism Equivalence Problem . 89 5.5.3 Deciding k-valuedness .................... 91 5.6 Conclusion............................... 92 6 Well-Nested VPTs 95 6.1 Definition ............................... 96 6.2 Composition.............................. 98 6.3 Type Checking against VPL .....................100 6.4 Tree Transducers and wnVPTs ....................101 6.4.1 TreesandTreeTransductions . 102 6.4.2 wnVPT onTrees .......................103 6.4.3 Unranked Tree Transducers . 104 6.4.4 UniformTreeTransducers . 107 6.4.5 Top-DownTreeTransducers . 108 6.4.6 Macro Visibly Pushdown Transducers . 111 6.5 Conclusion...............................114 7 Streaming Evaluation 117 7.1 Evaluation Algorithm: Eval .....................120 7.1.1 Overview of Eval ......................121 7.1.2 Algorithm Naive .......................123 7.1.3 Algorithm Naivecompact....................125 7.1.4 Algorithm Eval .......................129 7.2 Bounded Memory Transductions . 134 7.2.1 FiniteStateTransductions . 134 7.2.2 Pushdown Transductions . 135 7.2.3 Visibly Pushdown Transductions . 136 7.3 Height Bounded Memory Transductions . 138 7.3.1 Horizontal Twinning Properties . 140 7.3.2 HTPisDecidable. 140 7.3.3 HBM is Decidable for fVPTs .................143 Contents ix 7.4 Twinned VPTs ............................146 7.4.1 TwinningProperty . 146 7.4.2 Quadratic Height Bounded Memory Evaluation . 147 7.4.3 Twinning Property is Decidable . 149 7.5 TwinnedTransductions. 152 7.6 ConclusionandRemarks . 155 8 Look-ahead 157 8.1 Definitions...............................159 8.2 Expressiveness............................. 162 8.3 Functional VPTs and VPTla .....................167 8.4 DecisionProblems .. .. .. 171 8.5 Discussion on other Definitions of Look-ahead . 173 8.6 Conclusion ..............................175 9 Conclusion 177 A Proof of Theorem 7.5.3 185 Chapter 1 Introduction The present work proposes an abstract model of machines, called visibly push- down transducers, for defining transformations of documents with a nesting structure. Roughly speaking, a document is a sequence of symbols. The term nesting structure for a document refers to a structure that is organized on the basis of layers some of which are contained into the others. Special symbols, like opening and closing parenthesis, can be used to add nesting structure to a document. More generally, adding a set of opening symbols and a set of closing symbols to the set of data symbols of the document, induces a nesting structure. An opening symbol indicates the beginning of a new nesting layer (we say that we are entering an additional level of nesting),