Compiling and Interpreting Parsing and java.io.StreamTokenizer z Interpreting a language consists of (at least) three stages z The class StreamTokenizer is designed to help with both ¾ Lexical analysis lexical analysis and parsing • Reading individual characters and creating tokens: number, ¾ Recognizes numbers, words/identifiers, other symbols variable, reserved word, LEFT_PAREN, SEMI_COLON, … ¾ Configurable as to case-sensitive, what’s a comment, is ¾ Parsing end-of-line significant, and so on • Combining tokens into some structure representing a z Not really object-oriented, but parsing might be simpler using construct in the language, e.g., an abstract syntax tree (AST) a combination of OO and non-OO techniques ¾ Executing/Interpreting ¾ Leaves information in data field regarding the type of • Use the AST, execute/interpret the construct represented token parsed, e.g., number, word, EOL, EOF, char/symbol • Differences between execution and interpretation? ¾ Clients can extract number or word as appropriate z We can parse using well-known tools and techniques z The elan.parser examples use a StreamTokenizer with what’s ¾ Recursive-descent/built-by hand (top-down) essentially a one-token look-ahead ¾ Top-down/Bottom-up parser generators: lex/yacc, javacc ¾ getToken() access, nextToken() advances and accesses Software Design 16.1 Software Design 16.2 Recursive Descent for expressions Parsing with tokens z We’ll use BNF (Backus-Naur form) for representing a valid z Each of the functions in ExpressionParser conforms: expression ¾ Precondition: un-processed token accessible Expression ::= Term ¾ Postcondition: token consumed, new token available ::= Term +|- Expression Term ::= Factor z Matching when parsing ::= Factor *|/ Term Factor ::= Number ¾ Often we expect certain Tokens/symbols, use match to ensure parsing works ::= ( Expression ) • Example, in factor what happens after LEFT_PAREN? ::= - Factor • Example, what happens in parsing a repeat expression? z This grammar represents expressions that conform to standard precedence rules. It’s a right-recursive grammar hence useful in top-down parsing z Each BNF rule usually corresponds to a parsing function, elements of grammar correspond to terminal/non-terminal ¾ The BNF almost completely determines parsing rules, see nodes elan.ExpressionParser.java ¾ Example: instructions, expressions Software Design 16.3 Software Design 16.4 Use cases SLOGO use case See http://members.aol.com/acockburn/papers/AltIntro.htm z There is a read-eval-print loop z An actor interacts with system ¾ How else might user interact with environment ¾ Might be a person using the system ¾ What information should be accessible to user ¾ What about display? Part of use case? ¾ Might be a program or process interacting with the system z Use cases are descriptions of what happens when an actor uses the system to achieve a goal z Debugging and error messages, what are issues? ¾ AST -> maps to source, what is this useful for? ¾ Collection of possible sequences of interactions between the system and its actors relating to a goal ¾ Flag more than the first error? Error correction? • Throw exceptions? ¾ Use cases should define all system behavior relevant to actors achieving goals and should not involve other factors z What about design/development/testing z Easy to read, scenario/descriptive ¾ How can graphical display be tested independently of z NOT: UI/GUI, implementation based parser, lexer? Software Design 16.5 Software Design 16.6.