A Predicated- Багведзж Parser Generator 1

A Predicated- Багведзж Parser Generator 1

SOFTWAREÐPRACTICE AND EXPERIENCE, VOL. 25(7), 789±810(JULY 1995) ANTLR: A Predicated- ¡ £¢¥¤§¦ Parser Generator T. J. PARR University of Minnesota, AHPCRC, 1100 Washington Ave S Ste 101, Minneapolis, MN 55415, [email protected] AND R. W. QUONG School of Electrical Engineering, Purdue University, W. Lafayette, IN 47907, [email protected] SUMMARY 1 Despite the parsing power of ¨ © ¨¨ © algorithms, e.g. YACC , programmers often choose to write recursive-descent parsers by hand to obtain increased ¯exibility, better error handling, and ease of debug- ging. We introduce ANTLR, a public-domain parser generator that combines the ¯exibility of hand-coded parsing with the convenience of a parser generator, which is a component of PCCTS 2. ANTLR has many features that make it easier to use than other language tools. Most important, ANTLR provides predicates which let the programmer systematically direct the parse via arbitrary expressions using semantic and syntactic context; in practice, the use of predicates eliminates the need to hand-tweak the ANTLR output, even for dif®cult parsing problems. ANTLR also integrates the description of lexical and syntactic analysis, accepts ¨ ¨ grammars for with extended BNF notation, and can automatically generate abstract syntax trees. ANTLR is widely used, with over 1000 registered industrial and academic users in 37 countries. It has been ported to many popular systems such as the PC, Macintosh, and a variety of UNIX platforms; a commercial C++ front-end has been developed as a result of one of our industrial collaborations. KEY WORDS Parsing Compiler Parser generator Predicates LL(k) parser INTRODUCTION Programmers want to use tools that employ mechanisms they understand, that are suf®ciently powerful to solve their problem, that are ¯exible, that automate tedious tasks, and that generate output that is easily folded into their application. Consider parser generators. Existing parser generators often fail one or more of these criteria. Consequently, parsers are often written by hand, especially for languages that are context-sensitive or require large amounts of lookahead. Compared to a hand-built recursive-descent parser, table-driven / parsers often do not have enough parsing strength and can be dif®cult to understand and debug. A parser must do much more than just recognize languages. In particular, parsers must interact with the lexical analyzer (scanner), report parsing errors, construct abstract syntax trees, and call user actions. Existing parsing tools have focused mainly on the language recognition strategy, often ignoring the aforementioned tasks. In this paper, we introduce the ANTLR (ANother Tool for Language Recognition) parser generator, which addresses all these issues. ANTLR is a component of the Purdue Compiler Construction Tool Set (PCCTS)2. It constructs human-readable recursive-descent parsers in 0038±0644/95/070789±22$16.00 Received 10 June 1994 c 1995 by John Wiley & Sons, Ltd. Revised 17 November 1994 790 T. J. PARR, R. W. QUONG 3,4 ¨§ © § © © C or C++ from ¢¡¤£¦¥ - grammars, namely grammars, for , that support predicates. Predicates allow arbitrary semantic and syntactic context to direct the parse in a systematic way. As a result, ANTLR can generate parsers for many context-sensitive languages and many § © § © non- / context-free languages. Semantic predicates indicate the semantic validity of applying a production; syntactic predicates are grammar fragments that describe a syntactic context that must be satis®ed before recognizing an associated production. In practice, many ¨§ © ANTLR users report that developing a pred- grammar is easier than developing the § corresponding grammar. In addition to a strong parsing strategy, ANTLR has many features that make it more programmer-friendly than the majority of and parser generators. ANTLR integrates the speci®cation of lexical and syntactic analysis. A separate lexical speci®cation is unnecessary as lexical regular expressions (token descriptions) can be placed in double-quotes and used as normal token references in an ANTLR grammar. ANTLR accepts grammar constructs in Extended Backus-Naur Form (EBNF) notation. ANTLR provides facilities for automatic abstract syntax tree construction. ANTLR generates recursive-descent parsers in C/C++ so that there is a clear correspon- dence between the grammar speci®cation and the ANTLR output. Consequently, it is relatively easy for non-parsing experts to design and debug an ANTLR grammar. ANTLR has both automatic and manual facilities for error recovery and reporting. The automatic mechanism is simple and effective for many parsing situations; the manual mechanism called ªparser exception handlingº simpli®es development of high-quality error handling. ANTLR allows each grammar rule to have parameters and return values, facilitating attribute passing during the parse. Because ANTLR converts each rule to a C/C++ function in a recursive descent parser, a rule parameter is simply a function parameter. Additionally, ANTLR rules can have multiple return values. ANTLR has numerous other features that make it a product rather than a research project. ANTLR itself is written in highly portable C; its output can be debugged with existing source-level debuggers and is easily integrated into programmers' applications. Ultimately, the true test of a language tool's usefulness lies with the vast industrial program- mer community. ANTLR is widely used in the commercial and academic communities. More than 1000 registered users in 37 countries have acquired the software since the original 1.00 release in 1992. Several universities currently teach courses with ANTLR. Many commercial programmers use ANTLR; we list some examples in the Appendix. For example, a major corporation 5 has nearly completed and is testinga uni®ed C/Objective- C/C++ compiler using an ANTLR grammar that was derived directly from the June 1993 ANSI X3J16 C++ grammar. [Preliminary measurements show that this ANTLR parser is about 20% slower, in terms of pure parsing speed, than a hand-built recursive-descent parser that parses ¨§¤ only C/Objective-C, but not C++]. C++ has been traditionally dif®cult for other tools 1 §¤ and -based tools such as YACC . YACC grammars for C++ are extremely fragile with regards to action placement; i.e., the insertion of an action can introduce con¯icts into the C++ grammar. In contrast, ANTLR grammars are insensitive to action placement due to their ¨§ © nature. The following sections illustrate ANTLR's speci®cation language and the features that distinguish it from other parser generators. As this paper is an overview, we have omitted many details. Refer to the current PCCTS/ANTLR release notes for complete usage details. 15/8/1995 18:01 PAGE PROOFS antlr ¡ ANTLR: A PREDICATED- ¨ ¨ PARSER GENERATOR 791 Table I. ANTLR description elements. Item Description Example Token begins with uppercase letter ID Tokclass set of tokens (token class) Operators ÄTokclass complement set of Tokclass ÄKeyword /* match ID or something else */ . wild card token class a : ID | . ; rule name begins with lowercase letter expr label:item label token or rule reference a : t:ID ; <<...>> user-de®ned semantic action <<printf("%s", t->name);>> (...) subrule ("int" | ID | storage class) (...)* closure ID ("," ID)* (...)+ positive closure slist : ( stat | SEMICOLON )+ ; {...} optional {ELSE stat } <<...>>? semantic predicate type : <<is TYPE(str)>>? ID ; (...)? syntactic predicate ((list EQ)? list EQ list | list) DESCRIPTION LANGUAGE An ANTLR description or speci®cation is a collection of rules and actions preceded by a header in which the user de®nes required data types, such as the type of an attribute. We originally borrowed the notation from YACC to reduce the learning curve, but since then, we have added numerous extensions for new ANTLR features such as predicates, speci®cation of lexical analysis, error reporting, and EBNF grouping. Table I summarizes the elements in an ANTLR description. An ANTLR rule is a list of productions or alternatives separated by ª|º: £¦¡¨§©¢ ¥ £ rule : ¢¤£¦¥ £ ¡§©¢¥ £ | ¢£¦¥ . £ ¡§©¢¥ £ | ¢£¦¥ ; where each alternative production is composed of a list of elements; an element is an item from Table I. In this table, the ª...º within the grouping constructs can themselves be lists of alternatives or items. Rules may also de®ne arguments and return values. In the following line, there are § arguments and return values. ¡ ¢ ¡ ¡ £¥ ¢¤£ ¡¤£¥ ¢¤£ rule[ ¢ ,..., ] > [ ,..., ] : ... ; The syntax for using a rule mirrors its de®nition, ¡ ¢ ¡ a : ... rule[ ¢ ,¼, ] > [ ,¼, ] ... ; ¨! Here, the various ! receive the return values from the rule rule, so that each must be an 15/8/1995 18:01 PAGE PROOFS antlr 792 T. J. PARR, R. W. QUONG l-value. We illustrate the major features of ANTLR's description language via a small example. Consider parsing the following simple assembly language. #segment data a ds 42 b ds 13 #segment code load r1, a load r2, b add r1,r2,r3 print r3 Figure 1 contains a complete ANTLR speci®cation for this problem. #header <<#include "charbuf.h">> <<main() {ANTLR(prog(), stdin); }>> #tokclass OPCODE {"add" "store" "load" "call" "ret" "print" } #tokclass REGISTER {"r0" "r1" "r2" "r3" } #token "[\ \t]+" <<zzskip();>> #token "\n" <<zzskip(); zzline++;>> prog: "#segment" "data" (data)* "#segment" "code" (stat)* ; stat: OPCODE operands ; operands : ID | REGISTER | REGISTER "," NUM | REGISTER "," REGISTER "," REGISTER ; data: ID "ds" NUM ; #token NUM "[0-9]+" #token

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    22 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us