OBTAINING PRACTICAL VARIANTS OF LL (k) AND LR (k) FOR k >1 BY SPLITTING THE ATOMIC k-TUPLE A Thesis Submitted to the Faculty of Purdue University by Terence John Parr In Partial Ful®llment of the Requirements for the Degree of Doctor of Philosophy August 1993 ii ACKNOWLEDGEMENTS I wish to acknowledge my parents (James and Peggy) and siblings (James Jr., Jacqueline, and Josephine) throughout my education, without whose support I would never have persevered. I thank Professor Hank Dietz, my advisor, for his encouragement and tolerance over the past ®ve years during my Masters and Doctorate. I thank Professor Russell Quong for his interest in my research and helping me develop many of the advanced language and parsing ideas in the ANTLR parser generator. I wish to acknowledge Professors Dave Meyer and Jose Fortes for getting me interested in electrical engineering and encouraging me to attend graduate school. Professor Leah Jamieson deserves thanks for taking a chance on me and supporting my continued studies at Purdue. I thank Professors Matt O'Keefe and Paul Woodward at the University of Minnesota for providing me with the fellowship to ®nish my studies. Further, I wish to thank Paul Woodward for giving me the work ``ack.'' I acknowledge Kevin Edgar for whom ``no thanks is too much.'' To the users of the Purdue Compiler-Construction Tool Set and codeveloper, Will Cohen, I extend my gratitude for making all of my research worthwhile. iii TABLE OF CONTENTS Page LIST OF TABLES...........................................................................................................................vi LIST OF FIGURES.......................................................................................................................viii ABSTRACT.....................................................................................................................................xi CHAPTER 1 INTRODUCTION .....................................................................................................1 CHAPTER 2 MOTIVATION..........................................................................................................4 2.1 Terminology............................................................................................................................4 2.2 The Need for k Terminals of Lookahead ................................................................................7 2.2.1 The Effect of Action Placement upon LR (k) and LL (k).................................................8 2.2.2 LALR and SLR Versus LL ..............................................................................................15 2.3 Comparison to Previous Work in LL (k) and LR (k) Parsing................................................17 2.4 Optimal Parsing of Deterministic Languages.......................................................................20 2.4.1 Structure of Parser Comparison Sequences ...................................................................21 2.4.2 Optimal LL (k) Parsing...................................................................................................23 2.4.3 Optimal LR (k) Parsers...................................................................................................27 CHAPTER 3 PARSING ................................................................................................................31 3.1 Grammar Representation ......................................................................................................32 3.2 Heterogeneous Automata in Deterministic Parsing..............................................................35 3.3 Parsing Decisions..................................................................................................................40 3.3.1 C 1(k) Decisions .............................................................................................................41 3.3.2 SLL (k) Lookahead Characteristics ................................................................................42 3.3.3 When Parsers Need Lookahead .....................................................................................44 iv Page 3.3.4 How Parsers Use Lookahead .........................................................................................46 CHAPTER 4 PARSER LOOKAHEAD ........................................................................................49 4.1 Representation.......................................................................................................................50 4.2 Operations .............................................................................................................................53 4.2.1 Full Lookahead Operations............................................................................................53 4.2.2 Linear, Approximate, Lookahead Operations................................................................56 4.2.3 Lookahead Computation Cycles....................................................................................59 4.2.3.1 Example FIRSTk Cycle...........................................................................................60 4.2.3.2 Example FOLLOWk Cycle .....................................................................................62 4.3 Complexity of Lookahead Information Computation ..........................................................63 CHAPTER 5 SLL 1(k)Ð A LINEAR APPROXIMATION TO SLL (k) .......................................67 5.1 SLL 1(k) Decisions ................................................................................................................67 5.1.1 Example SLL 1(k) Grammar...........................................................................................68 5.1.2 Empirical Studies of SLL 1(k) Versus SLL (k) ...............................................................71 5.1.3 Recognition Strength Versus Space Requirements........................................................75 5.1.4 SLL 1(k) Formalisms ......................................................................................................79 5.2 SLL 1(k) Lookahead Computation ........................................................................................82 5.2.1 Example Lookahead Computation.................................................................................83 5.2.2 Algorithms to Compute SLL 1(k) Lookahead ................................................................85 5.3 Testing for the SLL 1(k) Property..........................................................................................91 5.4 SLL 1(k) Parser Construction.................................................................................................94 CHAPTER 6 SLL (k)......................................................................................................................99 6.1 Example SLL (k) Grammar ...................................................................................................99 6.2 SLL (k) Lookahead Computation ........................................................................................102 6.2.1 Example Lookahead Computation...............................................................................102 6.2.2 Straightforward LOOKk Algorithm .............................................................................104 6.2.3 Constrained LOOKk Algorithm ...................................................................................105 6.2.4 LOOKk Algorithm With Caching ................................................................................106 6.3 Testing for the SLL (k) Property .........................................................................................113 6.3.1 Characteristics of SLL (k) Determinism.......................................................................114 6.3.2 Algorithm for Testing for the SLL (k) Property ...........................................................116 6.3.3 Complexity of Testing for the SLL (k) Property ..........................................................119 6.4 SLL (k) Parser Construction ................................................................................................120 v Page 6.4.1 Lookahead Information Compression..........................................................................120 6.4.2 Implementation of Heterogeneous Decision States .....................................................125 6.4.3 Example SLL (k) Parser Constructions ........................................................................129 CHAPTER 7 LALL (k), LL (k), SLR (k), LALR (k), AND LR (k).................................................134 7.1 LALL (k) ..............................................................................................................................134 7.2 LL (k)...................................................................................................................................140 7.3 SLR (k).................................................................................................................................142 7.4 LALR (k)..............................................................................................................................146 7.5 LR (k)...................................................................................................................................149 7.6 LL m(k) and LR m(k).............................................................................................................151 CHAPTER 8 CONCLUSION......................................................................................................155