Context Free Grammars

Context Free Grammars https://courses.missouristate.edu/anthonyclark/333/

Some notes adapted from Professor GianLuigi Ferrari at University of Pisa Outline

Topics and Learning Objectives • Formal grammar theory • Context free grammars

Assessments • Context free grammars Picture from “Crafting Interpreters” Formal Grammars Regular Expressions and Interpreter and Push-Down Automata Finite State Machines Put Tokens in a Tree- Group characters like data structure Source Code (Plain Text) into smallest that represents 1. // GCD Program (in C) Lexemes/Tokens 2. int main() { meaningful units semantics of program 3. int i = getint(), j = getint(); int main ( ) { int i = getint ( ) , j = getint ( ) ; 4. while (i != j) { while ( i != j ) { 5. if (i > j) i = i - j; if ( i > j ) i = i - j ; 6. else j = j - i; Lexer/Scanner else j = j - 1 ; Parser } 7. } putint ( i ) ; 8. putint(i); } Self-evident 9. } You’ve already started this Assignments 5 through 7 We’ll create our own simple calculator language “Walk” the tree and evaluate the program given optional input from outside of the program Result Evaluate/Interpret Assignment 8

User input / File Input / Sockets / Etc. Tree-Walking is pretty We’ll build our own representation straightforward Interpreter read Lexer

request … … send token token Parser

request send AST AST I/O Console Tree Walker Chomsky Hierarchy

• Type-0: Turing machine

• Type-1: Linear bounded automaton

• Type-2: Pushdown automaton

• Type-3: Finite state automaton Scanning vs Parsing

• Regular expressions for regular languages recognized by lexer • Context free grammars for context-free languages recognized by parsers

REs cannot “count” • Cannot balance parenthesis • Cannot balance if then else expressions • Etc. Example from: Abstract Syntax Tree Writing An Interpreter In Go

Lexems/Tokens

We now care about syntax! No parentheses, semicolons, braces, etc. Grammar

A grammar is a tool for describing a language A grammar is a set of rules (productions) for creating valid strings

grammar English;

• You can also create syntactically valid strings that do not make sense semantically • “Computers run cheese” • “This am a lies”

• These are valid sentences, but they do not have any real meaning • The same can be true for our programming language rules • We’ll worry about semantics later. def f(): return “hi”

float y = f() + 5; Error types

Invalid lexemes (all languages will catch this problem early) var x = 5 @ “6” # ‘@’ is not a valid operator in most languages

Valid lexemes, invalid syntax (all languages will catch this problem early) x var= 5 5 * # all valid lexemes, but in the wrong order

Valid lexemes, valid syntax, invalid semantics (catch at compile time or runtime) var x = 5 * “6” # many languages will not multiply an integer and a string

Valid lexemes, valid syntax, valid semantics var x = 5 * 6 Formal Grammars

Grammar a set of rules for creating valid strings Nonterminal a grammar symbol that can be replaced by a sequence of symbols Terminal a word in the language (cannot be replaced with something else)

Production a single rule in the grammar (XàY1Y2…) Derivation a sequence of rule applications that produces a valid string

Start Symbol the rule used to start all derivations

Null Symbol the ε symbol is used to say that a nonterminal can be replaced with nothing Example

Write a regular expression for the following regular language. At least 1 zero followed by at least 1 one 0 0* 1 1* Now write a grammar for the same language Example

• Define a regular expression where you have n 0’s followed by n 1’s

• Define a context-free grammar where you have some number of 0’s followed by the same number of 1’s Activity 1. Write three strings that can be generated by the following grammar. S à 1S | 0T | ε T à 1T | 0S What does this recognize?

S à 1S | 0T | ε T à 1T | 0S Grammar rules for creating valid strings of a language