Context Free Languages and Pushdown Automata

COMP2600 — Formal Methods for Software Engineering

Ranald Clouston

Australian National University Semester 2, 2013

COMP 2600 — Context Free Languages and Pushdown Automata 1 Parsing

The process of parsing a program is partly about conﬁrming that a given program is well-formed – syntax.

But it is also about representing the structure of the program so that it can be executed – semantics.

For this purpose, the trail of sentential forms created en route to generating a given sentence is just as important as the question of whether the sentence can be generated or not.

COMP 2600 — Context Free Languages and Pushdown Automata 2 The Semantics of Parses

Take the code if e1 then if e2 then s1 else s2

where e1, e2 are boolean expressions and s1, s2 are subprograms.

Does this mean if e1 then( if e2 then s1 else s2)

or if e1 then( if e2 else s1) else s2

We’d better have an unambiguous way to tell which is right, or we cannot know what the program will do at runtime!

COMP 2600 — Context Free Languages and Pushdown Automata 3 Ambiguity

Recall that we can present CFG derivations as parse trees.

Until now this was mere pretty presentation; now it will become important.

A context-free grammar G is unambiguous iff every string can be derived by at most one parse tree.

G is ambiguous iff there exists any word w ∈ L(G) derivable by more than one parse tree.

COMP 2600 — Context Free Languages and Pushdown Automata 4 Example: If-Then and If-Then-Else

Consider the CFG

S → if bexp then S | if bexp then S else S | prog

where bexp and prog stand for boolean expressions and (if-statement free) programs respectively, deﬁned elsewhere.

The string if e1 then if e2 then s1 else s2 then has two parse trees:

S S RR ss EE ss EERRR ss EE ss EE RRR sy ss EE" sy ss E" RRR if e1 then S if e1 then S else R) S jjjy EE s jjjj yy EE sss ju jjj | yy EE" sy ss if e2 then S y else S if e2 then S s2 s1 s2 s1

COMP 2600 — Context Free Languages and Pushdown Automata 5 Example: If-Then and If-Then-Else

That grammar was ambiguous. But here’s a grammar accepting the exact same language that is unambiguous:

S → if bexp then S | T T → if bexp then T else S | prog

There is now only one parse for if e1 then if e2 then s1 else s2.

This is given on the next slide:

COMP 2600 — Context Free Languages and Pushdown Automata 6 Example: If-Then and If-Then-Else

S s EE sss EE sy ss EE" if e1 then S T jj E jjj y EE jjj yy EE ju jj y| y E" if e2 then T else S s1 T s2 You cannot parse this string as if e1 then ( if e2 else s1 ) else s2.

COMP 2600 — Context Free Languages and Pushdown Automata 7 Reﬂecting on This Example

We have seen that the same language can be presented with ambiguous and unambiguous grammars.

Ambiguity is in general a property of grammars, not of languages.

From the point of view of semantics and parsing, it is often desirable to turn an ambiguous grammar into an equivalent unambiguous grammar.

This generally involves choices that are not driven by the language itself.

E.g. there exists another grammar for the language of the previous slides that is unambiguous and allows the parse if e1 then ( if e2 else s1 ) else s2 but not if e1 then ( if e2 then s1 else s2 ).

COMP 2600 — Context Free Languages and Pushdown Automata 8 What Ambiguity Isn’t

You might wonder if our grammar is still ambiguous, given the production

T → if bexp then T else S

After a derivation that uses this, we have a choice of whether we expand the T or the S ﬁrst. Is this an ambiguity?

No. A context-free grammar gets its name because non-terminals can be expanded without regard for the context they appear in.

In other words, it doesn’t matter if you expand T or S ‘ﬁrst’. From the perspec- tive of the parse tree these expansions are happening in parallel.

This is a reason why a parse tree is a better formalism for presenting a CFG derivation than listing each step.

COMP 2600 — Context Free Languages and Pushdown Automata 9 Inherently Ambiguous Languages

In fact not all context-free languages can be given unambiguous grammars – some are inherently ambiguous. Consider the language

L = {aib jck | i = j or j = k}

How do we know that this is context-free? First, notice that

L = {aibick} ∪ {aib jc j}

We then combine CFGs for each side of this union (a standard trick):

COMP 2600 — Context Free Languages and Pushdown Automata 10 Inherently Ambiguous Languages

The problem with this CFG is that the union we used has a non-empty intersection, where the a’s, b’s, and c’s all have equal number.

The sentences in this intersection are a source of ambiguity:

S S T W B C || BB {{ CC |} | B {} { C! U V X Y B A @ }} BB AA ~~ || @@ }~ } B A ~~ |} | @ a U b c V a X b Y c ε ε ε ε In fact (not proved here!) there is no alternative choice of grammar for this language that avoids this ambiguity.

COMP 2600 — Context Free Languages and Pushdown Automata 11 The Bad News

We would like to have an algorithm that turns grammars into equivalent unambiguous grammars where possible.

However this is an uncomputable problem.

Worse – determining whether a grammar is ambiguous or not in the ﬁrst place is also uncomputable!

Uncomputable problems are everyone in computer science. You will see many more next week in particular.

The best response is not despair; rather we should see what tricks and tech- niques might help us out at least some of the time.

COMP 2600 — Context Free Languages and Pushdown Automata 12 Example: Subtraction

Consider the grammar S → S − S | int

where int could be any integer and the symbol ‘−’ is intended to be executed as subtraction.

This grammar is ambiguous, and this matters:

S S > @ ~~ >> ÐÐ @@ ~~ >> ÐÐ @@ ~ ~ > Ð Ð @ S − 1 5 − S @ > ÐÐ @@ ~~ >> ÐÐ @@ ~~ >> Ð Ð @ ~ ~ > 5 − 3 3 − 1

The left tree evaluates 5 − 3 − 1 to 1; the left evaluates the same string to 3!

COMP 2600 — Context Free Languages and Pushdown Automata 13 Technique 1: Associativity

We can remove the ambiguity of a binary inﬁx operator by making it associate to the left or right. S → S − int | int

Now 5 − 3 − 1 can only be read as (5 − 3) − 1 - this is left associativity.

For right associativity we would use the production S → int − S instead.

Idea: Force one side of our operator to a ‘lower’ level, making sure we avoid loops in our grammar that allow the original non-terminal to be recovered.

Here we force the right hand side of the minus sign to the lowest possible level – a terminal symbol.

COMP 2600 — Context Free Languages and Pushdown Automata 14 Example: Multiplication and Addition

S → S ∗ S | S + S | int

where ∗ is to be executed as multiplication and + as addition.

Again this is obviously ambiguous – 1 + 2 ∗ 3 could evaluate to 7 or 9.

(Note that 1 + 2 + 3 is also a source of ambiguity as it can be produced by different parse trees, even though it is not ambiguous in the interpretation we have in mind.)

If all we care about is resolving ambiguity we can use the same trick as the last slide, making both ∗ and + left (or right) associative.

But this is not the behaviour we expect from these operations: we expect ∗ to have higher precedence than +.

COMP 2600 — Context Free Languages and Pushdown Automata 15 Technique 2: Precedence

S → S + T | T T → T ∗ int | int

Given a string 1+2∗3, or 2∗3+1, we have no choice but to expand to S+T ﬁrst, so that (thinking bottom-up) the + will be last command to be executed.

Suppose we tried to derive 1 + 2 ∗ 3 by ﬁrst doing S ⇒ T ⇒ T ∗ 3. We are then stuck because we cannot send T to 1 + 2!

As with associativity this trick works by forcing ourselves down to a lower level to generate parts of our sentences. Here we have three levels: S, then T , then the integers as non-terminals.

COMP 2600 — Context Free Languages and Pushdown Automata 16 Example: Basic Arithmetic

S → S + T | S − T | T T → T ∗U | T/U | U U → (S) | int

Note that we have brackets available to give a clearly labelled way to loop back to the top – if we want to break the usual rules of arithmetic to get the execution (1 + 2) ∗ 3 then we must indicate this with explicit brackets.

(Note also that the previous slides’ languages were actually regular and so could have been generated by right-linear grammars. The language above is truly context-free because of the need to keep track of bracket balancing to an arbitrary depth.)

COMP 2600 — Context Free Languages and Pushdown Automata 17 Example: Balanced Brackets

The following grammar generates a language where each left bracket is exactly matched by a closing bracket:

S → ε | (S) | SS

This is ambiguous in a rather stupid way: to generate () we could use the derivation S ⇒ (S) ⇒ (), which seems sensible.

But we could also start with S ⇒ SS, then use the left S to derive () as above while sending the right to ε immediately. Or we could have done the opposite. In fact there are inﬁnitely many parse trees for any string in the language!

Edit: There is another source of ambiguity here – ()()() has two parses even without such ‘abuse’ of ε. We ﬁx this on the next slide with left associativity.

COMP 2600 — Context Free Languages and Pushdown Automata 18 Technique 3: Controlling ε

Instead, we ensure there is only one way to access the empty string:

S → ε | T T → TU | U U → () | (T)

The string ε can be derived directly from S, but everything else must go through T .

This is another technique that relies on creating a hierarchy of levels within our grammar.

ε as a source of ambiguity can be easy to miss, so keep your eyes out for it!

COMP 2600 — Context Free Languages and Pushdown Automata 19 Hierarchy of Automata

We have seen that there is a hierarchy of grammars. A language may be generated by grammars at a certain level, but not by less powerful grammars.

We have seen that regular (right-linear) grammars are exactly as expressive as ﬁnite state automata.

Are there notions of automata exactly as expressive as context-free grammars? Context-sensitive grammars? Unrestricted grammars?

Yes - there is a hierarchy of automata.

Just as we needed a general deﬁnition of grammar to deﬁne their hierarchy, we need a general view of automata...

COMP 2600 — Context Free Languages and Pushdown Automata 20 General Structure of Automata

a0 a1 a2 ...... an

read input tape head

Finite Auxiliary State Memory Control

The input tape is a sequence of tokens. Each time a symbol is processed the read head advances.

The ﬁnite state control (FSC) can be in any one of a ﬁnite number of states.

The auxiliary memory is usually a linear organisation (e.g. a stack or array).

COMP 2600 — Context Free Languages and Pushdown Automata 21 General Automata ctd

Each action of the machine may change the FSC state, change the auxiliary memory, or advance to the next input symbol, ‘consuming’ the current one.

The action of the machine depends on the current input symbol, the current FSC state, and the current memory.

The machine starts in some particular start state (q0), with the read head at the ﬁrst input symbol, and the memory containing only the start symbol (Z).

A machine accepts an input string as a sentence of the language if it reaches a ﬁnal state (in F) with the input exhausted

(we sometimes also require that the memory be emptied).

COMP 2600 — Context Free Languages and Pushdown Automata 22 Automata and Grammars

The kind of auxiliary memory in a machine determines the class of languages that the machine can recognise:

Language Class Memory regular none context-free stack context-sensitive tape (bounded by input length) unrestricted unbounded tape

We have already looked at Finite State Automata (automata without memory) and their relation to regular languages.

We now consider Push-Down Automata (i.e. automata with stack memory) and their relation to context-free grammars and languages.

COMP 2600 — Context Free Languages and Pushdown Automata 23 Push-down Automata — PDA

a0 a1 a2 ...... an

read input tape head

Finite State zk Control stack memory z2 z1

COMP 2600 — Context Free Languages and Pushdown Automata 24 PDAs ctd

Each action of the machine may involve change to the FSC state, pushing or popping the stack, and advancing to the next input symbol.

The action of the machine may depend on the current FSC state, the current input symbol, and the current top-of-stack symbol.

The machine accepts an input string if it reaches a ﬁnal state, with the input exhausted, and the stack empty.

ASIDE: Other (equivalent) deﬁnitions of PDAs exist, where

• an input string is accepted if the PDA has an empty stack with the input exhausted

• an input string is accepted if the PDA is in a ﬁnal state with the input exhausted

COMP 2600 — Context Free Languages and Pushdown Automata 25 Example

{anbn | n ≥ 1}

Recall that this language cannot be recognised by a FSA (because there can only be a ﬁnite number of states), but can be generated by a context-free grammar.

It can also be recognised by a PDA.

Ad hoc design:

• phase 1: (state q1) push a’s from the input onto the stack

• phase 2: (state q2) pop a’s from the stack, if there is a b on input

• ﬁnalise: if the stack is empty and the input is exhausted in the ﬁnal state

(q3), accept the string.

COMP 2600 — Context Free Languages and Pushdown Automata 26 Deterministic PDA – Deﬁnition

A deterministic PDA has the form (Q,q0,F, Σ, Γ,Z, δ), where

• Q is the set of states, q0 ∈ Q is the initial state and F ⊆ Q are the ﬁnal states;

• Σ is the alphabet, or set of input symbols;

• Γ is the set of stack symbols, and Z ∈ Γ is the initial stack symbol;

• δ is a (partial) transition function δ : Q × (Σ ∪ {ε}) × Γ → Q × Γ∗ δ : (state,input token or ε,top-of-stack) → (new state,stack string) such that for all q ∈ Q and s ∈ Γ, if δ(q,ε,s) is deﬁned then δ(q,a,s) is undeﬁned for all a ∈ Σ. This condition is vital to maintain determinism.

COMP 2600 — Context Free Languages and Pushdown Automata 27 Notation

Given the transition function

δ : Q × (Σ ∪ {ε}) × Γ → Q × Γ∗ δ : (state,input token or ε,top-of-stack) → (new state,stack string)

we write δ(q,a,s) = q0/σ to show how this function operates.

Then the string σ in the result is the string with which we replace the top-of- stack symbol s. (Doing this makes it simple to specify pushes and pops in a uniform way.)

We will usually indicate ﬁnal states by underlining them, e.g. q.

COMP 2600 — Context Free Languages and Pushdown Automata 28 Two types of PDA transition

• when δ contains (q1,x,s) 7→ q2/σ,

state input stack

the PDA can go from q1 next symbol is x s on top-of-stack

to q2 x is consumed σ replaces s on stack

• when δ contains (q1,ε,s) 7→ q2/σ,

state input stack

the PDA can go from q1 any input, or none s on top-of-stack

to q2 input is ignored σ replaces s on stack

Such transitions do not look at or consume an input symbol.

COMP 2600 — Context Free Languages and Pushdown Automata 29 Example ctd

Recall that the stack starts off containing only the initial symbol Z.

n n PDA to recognise a b (start state q0, ﬁnal state(s) underlined):

δ(q0,a,Z) = q1/aZ ··· push ﬁrst a

δ(q1,a,a) = q1/aa ··· push further a’s

δ(q1,b,a) = q2/ε ··· start popping a’s

δ(q2,b,a) = q2 / ε ··· pop further a’s

δ(q2,ε,Z) = q3/ε ··· accept

Note that δ is a partial function, undeﬁned for many arguments.

COMP 2600 — Context Free Languages and Pushdown Automata 30 Example ctd — PDA Trace

PDA conﬁgurations can be written as a triple (state, remaining input, stack) with the top of stack to the left.

(q0,aaabbb,Z) ⇒ (q1,aabbb,aZ) (push ﬁrst a)

⇒ (q1,abbb,aaZ) (push further a’s)

⇒ (q1,bbb,aaaZ) (push further a’s)

⇒ (q2,bb,aaZ) (start popping a’s)

⇒ (q2,b,aZ) (pop further a’s)

⇒ (q2,ε,Z) (pop further a’s)

⇒ (q3,ε,ε) (accept)

The machine halts in the ﬁnal state with input exhausted and an empty stack, so the string is accepted.

COMP 2600 — Context Free Languages and Pushdown Automata 31 Example ctd — Rejection

The string aaba should be rejected by this PDA:

(q0,aaba,Z) ⇒ (q1,aba,aZ)

⇒ (q1,ba,aaZ)

⇒ (q2,a,aZ) ⇒ ???

No transition applies, and the PDA is “stuck” without reaching a ﬁnal state.

Rejection happens when the transition function is undeﬁned for the current conﬁguration ( that is, state, input symbol and top of stack symbol ).

COMP 2600 — Context Free Languages and Pushdown Automata 32 Example: Palindromes with ‘Centre Mark’

Consider the language

{wcwR | w ∈ {a,b}∗ ∧ wR is w reversed}

This is context-free, and we can design a deterministic PDA to accept it:

• Push a’s and b’s onto the stack as we seem them;

• When we see c, change state;

• Now try to match the tokens we are reading with the tokens on top of the stack, popping as we go;

• If the top of the stack is the empty stack symbol Z, pop it and enter the ﬁnal state via an ε-transition. Hopefully our input has been used up too!

Full formal details are left as an exercise.

COMP 2600 — Context Free Languages and Pushdown Automata 33 Non-Deterministic PDAs

For deterministic PDAs transitions are a (partial) function:

δ : Q × (Σ ∪ {ε}) × Γ → Q × Γ∗ δ : (state,input token or ε,top-of-stack) → (new state,stack string)

with a side-condition about ε-transitions.

For non-deterministic PDAs transitions are a relation

δ ⊆ Q × (Σ ∪ {ε}) × Γ × Q × Γ∗

with no side-condition.

There may be conﬁgurations where there are multiple choices of transition.

COMP 2600 — Context Free Languages and Pushdown Automata 34 Non-Deterministic PDAs ctd.

Recall for ﬁnite state automata, non-determinism can be more convenient, but does not give extra power (thanks to the subset construction).

This is not the case for PDAs - non-deterministic PDAs can recognise languages that deterministic PDAs cannot.

It turns out that non-deterministic PDAs are the more important from the per- spective of Chomsky’s hierarchy.

COMP 2600 — Context Free Languages and Pushdown Automata 35 Example: Even-Length Palindromes

The language of even length palindromes

{wwR | w ∈ {a,b}∗ ∧ wR is w reversed}

is context-free but cannot be recognised by a deterministic PDA, because without a centre mark it cannot know whether it is in the ﬁrst half of a sentence (so should be pushing everything into memory) or the second half (so it should be matching input and stack, and popping).

But a non-deterministic PDA can recognise this language via the transition

δ(q,ε,x) = r/x

where x ∈ {a,b,Z}, q is the ‘push’ state, and r the ‘match and pop’ state.

In other words, we continually ‘guess’ which job we should be doing!

COMP 2600 — Context Free Languages and Pushdown Automata 36 Grammars and PDAs

Theorem

The class of languages recognised by non-deterministic PDA’s is exactly the class of context-free languages.

We will only justify this result in one direction: for any CFG, there is a corre- sponding PDA.

This is the most interesting direction since it is the basis of automatically deriving parsers from grammars.

(The other direction is a bit complicated).

COMP 2600 — Context Free Languages and Pushdown Automata 37 From CFG to PDA

The PDA has three states: q0 (initial), q1 (processing), and q2 (ﬁnal). Its alphabet will be that of the CFG, and its stack symbols will be all the CFG’s terminals and non-terminals.

1. Initialise the process by pushing the start symbol S onto the stack, and

enter state q1:

δ(q0,ε,Z) 7→ q1/SZ

2. If a non-terminal is on top of stack, replace it with the right hand side of a production. For all productions A → α:

δ(q1,ε,A) 7→ q1/α

continued. . .

COMP 2600 — Context Free Languages and Pushdown Automata 38 From CFG to PDA, ctd

3. For all terminal symbols t, pop the stack if it matches the input:

δ(q1,t,t) 7→ q1/ε

4. For termination, add the transition, with ﬁnal state q2:

δ(q1,ε,Z) 7→ q2/ε

In general we get a non-deterministic PDA since there may be several productions for each non-terminal.

COMP 2600 — Context Free Languages and Pushdown Automata 39 Example — Derive a PDA for a CFG

S → S + T | T T → T ∗U | U U → (S) | int

1. Initialise:

δ(q0,ε,Z) 7→ q1/SZ

2. Expand non-terminals:

δ(q1,ε,S) 7→ q1/S + T δ(q1,ε,T) 7→ q1/U

δ(q1,ε,S) 7→ q1/T δ(q1,ε,U) 7→ q1/(S)

δ(q1,ε,T) 7→ q1/T ∗U δ(q1,ε,U) 7→ q1/int

COMP 2600 — Context Free Languages and Pushdown Automata 40 CFG to PDA ctd

3. Match and pop terminals:

δ(q1,+,+) 7→ q1/ε

δ(q1,∗,∗) 7→ q1/ε

δ(q1,int,int) 7→ q1/ε

δ(q1,(,() 7→ q1/ε

δ(q1,),)) 7→ q1/ε

4. Terminate:

δ(q1,ε,Z) 7→ q2/ε

COMP 2600 — Context Free Languages and Pushdown Automata 41 Example Trace

(q0, int ∗ int, Z) ⇒ (q1, int ∗ int, SZ)

⇒ (q1, int ∗ int, TZ )

⇒ (q1, int ∗ int, T ∗UZ)

⇒ (q1, int ∗ int, U ∗UZ)

⇒ (q1, int ∗ int, int ∗UZ)

⇒ (q1, ∗int, ∗UZ)

⇒ (q1, int, UZ )

⇒ (q1, int, intZ)

⇒ (q1, ε, Z)

⇒ (q2, ε, ε) ⇒ accept

COMP 2600 — Context Free Languages and Pushdown Automata 42 A Context-Sensitive Language

In case you’re curious, a brief look at context-sensitive languages.

The following language is not context-free:

{anbncn | n ≥ 1}

Intuitively, we can imagine a CFG generating either the ab pairs or the bc pairs, but this language requires us to keep the generation process in step, in two different points in the sentential forms.

A context-sensitive grammar is on the next slide. Each production has a right- hand side of equal or longer length than their left.

(If we wanted to include the empty word also, we need a special exemption to this requirement.)

COMP 2600 — Context Free Languages and Pushdown Automata 43 A Context-Sensitive Language ctd.

On the left we have a context-sensitive grammar, and on the right an example derivation:

S → aBC S → aSBC S → aSBC → aaBCBC CB → BC → aabCBC aB → ab → aabBCC bB → bb → aabbCC bC → bc → aabbcC cC → cc → aabbcc

The automata that recognise CSGs have a tape memory, of length bounded by a linear function of the length of the input.

COMP 2600 — Context Free Languages and Pushdown Automata 44