Nested Word Automata

Automata Theory

Christian Schilling

June 4th, 2012 Nested Word Automata

Overview

Motivation and background

Nested words and their acceptors

Determinization proof

Conclusion

1 / 30 Nested Word Automata Motivation and background

Overview

Motivation and background Common languages Visibly pushdown languages

Nested words and their acceptors

Determinization proof

Conclusion

1 / 30 q0

q q 1 r 2

Nested Word Automata Motivation and background Common languages Regular language

1 procedure foo () § ¤ 2 { 3 r e t u r n ; 4 }

¦ L1 = {c r} ¥

2 / 30 Nested Word Automata Motivation and background Common languages Regular language

1 procedure foo () § ¤ 2 { q0 3 r e t u r n ; 4 } c

¦ L1 = {c r} ¥ q q 1 r 2

2 / 30 q0 q3

⊥, c, ⊥A A, r, ε A, r, ε

q1 q2 B, r, ε

A, c, AB A, r, ε B, c, BB

Nested Word Automata Motivation and background Common languages (det.) Context-free language

1 procedure bar () § ¤ 2 { 3 i f (*) 4 c a l l bar (); 5 r e t u r n ; 6 }

n n ¦ L2 = {c r | n > 0} ¥

3 / 30 Nested Word Automata Motivation and background Common languages (det.) Context-free language

q0 q3 1 procedure bar () § ¤ 2 { 3 i f (*) ⊥, c, ⊥A A, r, ε A, r, ε 4 c a l l bar (); 5 r e t u r n ; q1 q2 6 } B, r, ε

L = {cn rn | n > 0} ¦ 2 ¥ A, c, AB A, r, ε B, c, BB

3 / 30 / / /

all standard properties not under intersection and complementation all standard problems intersection, inclusion, equivalence undecidable powerset construction not possible

Question: Is there some class of languages in between that is more expressive than regular languages, but keeps their nice properties? Answer (Alur & Madhusudan 2004): yes, at least in some sense

Nested Word Automata Motivation and background Common languages Comparison

regular context-free comparison, constants two variables of numbers closure

decidability

determinize

4 / 30 / /

all standard problems intersection, inclusion, equivalence undecidable powerset construction not possible

Question: Is there some class of languages in between that is more expressive than regular languages, but keeps their nice properties? Answer (Alur & Madhusudan 2004): yes, at least in some sense

Nested Word Automata Motivation and background Common languages Comparison

regular context-free comparison, / constants two variables of numbers closure all standard properties not under intersection and complementation decidability

determinize

4 / 30 /

powerset construction not possible

Question: Is there some class of languages in between that is more expressive than regular languages, but keeps their nice properties? Answer (Alur & Madhusudan 2004): yes, at least in some sense

Nested Word Automata Motivation and background Common languages Comparison

regular context-free comparison, / / constants two variables of numbers closure all standard properties not under intersection and complementation decidability all standard problems intersection, inclusion, equivalence undecidable determinize

4 / 30 Question: Is there some class of languages in between that is more expressive than regular languages, but keeps their nice properties? Answer (Alur & Madhusudan 2004): yes, at least in some sense

Nested Word Automata Motivation and background Common languages Comparison

regular context-free comparison, / / / constants two variables of numbers closure all standard properties not under intersection and complementation decidability all standard problems intersection, inclusion, equivalence undecidable determinize powerset construction not possible

4 / 30 Answer (Alur & Madhusudan 2004): yes, at least in some sense

Nested Word Automata Motivation and background Common languages Comparison

Question: Is there some class of languages in between that is more expressive than regular languages, but keeps their nice properties?

4 / 30 Nested Word Automata Motivation and background Common languages Comparison

Question: Is there some class of languages in between that is more expressive than regular languages, but keeps their nice properties? Answer (Alur & Madhusudan 2004): yes, at least in some sense

4 / 30 • states, initial state, ﬁnal states, stack alphabet, bottom-of-stack symbol (no change here),

• partitioning of the input alphabet: Σ = Σi ] Σc ] Σr , • δ = δi ] δc ] δr , • δi ⊆ Q × Σi → Q • δc ⊆ Q × Σc → (Γ \ {⊥}) × Q • δr ⊆ Q × Σr × Γ → Q

Note: pops occur implicitly, ⊥ never popped, no ε

Nested Word Automata Motivation and background Visibly pushdown languages Visibly pushdown languages (VPLs)

A visibly pushdown language (VPL) is the language accepted by a visibly pushdown automaton (VPA).

A VPA A = hQ, q0, Qf , Σ, Γ, ⊥,δ i is a deterministic PDA with special rules: Determined by the input symbol, only one symbol per push is allowed and reading the stack implies a pop.

5 / 30 • partitioning of the input alphabet: Σ = Σi ] Σc ] Σr , • δ = δi ] δc ] δr , • δi ⊆ Q × Σi → Q • δc ⊆ Q × Σc → (Γ \ {⊥}) × Q • δr ⊆ Q × Σr × Γ → Q Note: pops occur implicitly, ⊥ never popped, no ε

Nested Word Automata Motivation and background Visibly pushdown languages Visibly pushdown languages (VPLs)

A visibly pushdown language (VPL) is the language accepted by a visibly pushdown automaton (VPA).

A VPA A = hQ, q0, Qf , Σ, Γ, ⊥,δ i is a deterministic PDA with special rules: Determined by the input symbol, only one symbol per push is allowed and reading the stack implies a pop. • states, initial state, ﬁnal states, stack alphabet, bottom-of-stack symbol (no change here),

5 / 30 • δ = δi ] δc ] δr , • δi ⊆ Q × Σi → Q • δc ⊆ Q × Σc → (Γ \ {⊥}) × Q • δr ⊆ Q × Σr × Γ → Q Note: pops occur implicitly, ⊥ never popped, no ε

Nested Word Automata Motivation and background Visibly pushdown languages Visibly pushdown languages (VPLs)

A visibly pushdown language (VPL) is the language accepted by a visibly pushdown automaton (VPA).

• partitioning of the input alphabet: Σ = Σi ] Σc ] Σr ,

5 / 30 Note: pops occur implicitly, ⊥ never popped, no ε

Nested Word Automata Motivation and background Visibly pushdown languages Visibly pushdown languages (VPLs)

A visibly pushdown language (VPL) is the language accepted by a visibly pushdown automaton (VPA).

• partitioning of the input alphabet: Σ = Σi ] Σc ] Σr , • δ = δi ] δc ] δr , • δi ⊆ Q × Σi → Q • δc ⊆ Q × Σc → (Γ \ {⊥}) × Q • δr ⊆ Q × Σr × Γ → Q

5 / 30 Nested Word Automata Motivation and background Visibly pushdown languages Visibly pushdown languages (VPLs)

A visibly pushdown language (VPL) is the language accepted by a visibly pushdown automaton (VPA).

• partitioning of the input alphabet: Σ = Σi ] Σc ] Σr , • δ = δi ] δc ] δr , • δi ⊆ Q × Σi → Q • δc ⊆ Q × Σc → (Γ \ {⊥}) × Q • δr ⊆ Q × Σr × Γ → Q Note: pops occur implicitly, ⊥ never popped, no ε

5 / 30 We construct a VPA for L2.

Partitioning: Σi = ∅,Σc = {c},Σr = {r}

δc = { (q0, c, A, q1), (q1, c, B, q1) } δr = { (q1, r, A, q3), (q1, r, B, q2), (q2, r, A, q3), (q2, r, B, q2) }

Nested Word Automata Motivation and background Visibly pushdown languages

L2 as VPL

n n Consider again L2 = {c r | n > 0}.

q0 q3

⊥, c, ⊥A A, r, ε A, r, ε

q1 q2 B, r, ε

A, c, AB A, r, ε B, c, BB

6 / 30 δc = { (q0, c, A, q1), (q1, c, B, q1) } δr = { (q1, r, A, q3), (q1, r, B, q2), (q2, r, A, q3), (q2, r, B, q2) }

Nested Word Automata Motivation and background Visibly pushdown languages

L2 as VPL

n n Consider again L2 = {c r | n > 0}. We construct a VPA for L2.

Partitioning: q q 0 3 Σi = ∅,Σc = {c},Σr = {r}

⊥, c, ⊥A A, r, ε A, r, ε

q1 q2 B, r, ε

A, c, AB A, r, ε B, c, BB

6 / 30 Nested Word Automata Motivation and background Visibly pushdown languages

L2 as VPL

n n Consider again L2 = {c r | n > 0}. We construct a VPA for L2.

Partitioning: q q 0 3 Σi = ∅,Σc = {c},Σr = {r}

c, A r, A δc = { (q0, c, A, q1), r, A (q1, c, B, q1) } q1 q2 δr = { (q1, r, A, q3), r, B (q1, r, B, q2), (q2, r, A, q3), c, B r, B (q2, r, B, q2) }

6 / 30 (Alur & Madhusudan 2006): no, with some further treatment of the input → nested words (NWs) • automaton model: nested word automata (NWAs) • nested word languages (NWLs) and VPLs have same power → NWAs deterministic PDAs • main idea: call and return symbols are matched in the input

Nested Word Automata Motivation and background Visibly pushdown languages From VPAs to NWAs

• main diﬀerences between VPAs and PDAs: • closed under determinism • partitioning of the alphabet • very limited use of the stack • Do we really need the stack?

7 / 30 • nested word languages (NWLs) and VPLs have same power → NWAs deterministic PDAs • main idea: call and return symbols are matched in the input

Nested Word Automata Motivation and background Visibly pushdown languages From VPAs to NWAs

7 / 30 • main idea: call and return symbols are matched in the input

Nested Word Automata Motivation and background Visibly pushdown languages From VPAs to NWAs

• main diﬀerences between VPAs and PDAs: • closed under determinism • partitioning of the alphabet • very limited use of the stack • Do we really need the stack? (Alur & Madhusudan 2006): no, with some further treatment of the input → nested words (NWs) • automaton model: nested word automata (NWAs) • nested word languages (NWLs) and VPLs have same power → NWAs deterministic PDAs

7 / 30 Nested Word Automata Motivation and background Visibly pushdown languages From VPAs to NWAs

7 / 30 Nested Word Automata Nested words and their acceptors

Overview

Motivation and background

Nested words and their acceptors Nested words Nested word automata

Determinization proof

Conclusion

8 / 30 i i

Examples:

i i i i i

Note: Every sequence has a unique well nesting.

Nested Word Automata Nested words and their acceptors Nested words Well nested sequences

A sequence of symbols is well nested if calls and returns are matched without crossing, i.e., for any diﬀerent call-return-pairs (ci , ri ), (cj , rj ), ci < cj < ri < rj is forbidden.

9 / 30 i i

Note: Every sequence has a unique well nesting.

Nested Word Automata Nested words and their acceptors Nested words Well nested sequences

A sequence of symbols is well nested if calls and returns are matched without crossing, i.e., for any diﬀerent call-return-pairs (ci , ri ), (cj , rj ), ci < cj < ri < rj is forbidden. Examples:

i c i c i i r r i

9 / 30 i i

Note: Every sequence has a unique well nesting.

Nested Word Automata Nested words and their acceptors Nested words Well nested sequences

A sequence of symbols is well nested if calls and returns are matched without crossing, i.e., for any diﬀerent call-return-pairs (ci , ri ), (cj , rj ), ci < cj < ri < rj is forbidden. Examples:

iciciirri

9 / 30 Note: Every sequence has a unique well nesting.

Nested Word Automata Nested words and their acceptors Nested words Well nested sequences

A sequence of symbols is well nested if calls and returns are matched without crossing, i.e., for any diﬀerent call-return-pairs (ci , ri ), (cj , rj ), ci < cj < ri < rj is forbidden. Examples:

iciciirri

r c r r c i c i

9 / 30 Note: Every sequence has a unique well nesting.

Nested Word Automata Nested words and their acceptors Nested words Well nested sequences

A sequence of symbols is well nested if calls and returns are matched without crossing, i.e., for any diﬀerent call-return-pairs (ci , ri ), (cj , rj ), ci < cj < ri < rj is forbidden. Examples:

iciciirri

rcrrcici

9 / 30 Nested Word Automata Nested words and their acceptors Nested words Well nested sequences

A sequence of symbols is well nested if calls and returns are matched without crossing, i.e., for any diﬀerent call-return-pairs (ci , ri ), (cj , rj ), ci < cj < ri < rj is forbidden. Examples:

iciciirri

rcrrcici

Note: Every sequence has a unique well nesting.

9 / 30 Nested Word Automata Nested words and their acceptors Nested words Nested words

A relation ⊂ {−∞, 1, 2, . . . , `} × {1, 2, . . . , `, ∞} of length ` ≥ 0 is a matching; relation if the following holds:

I if i j, then i < j (monotone)

II if i1; j and i2 j, then i1 = i2 (left-unique) if i ;j1 and i ;j1, then j1 = j2 (right-unique) III if i1; j1 and i2; j2, then we have not i1 < i2 < j1 < j2 ; ; (well nested) Explanation: I not r c, not reﬂexive ex post note: (−∞, ∞) 6∈ , II not c c r, not c r r ±∞ excluded from uniqueness; III not ccrr

10 / 30 A nested wordn over Σ is a pair (a1 ··· a`, ), where ai ∈ Σ and is a matching relation of length `. ; ;

Nested Word Automata Nested words and their acceptors Nested words Nested words

A relation ⊂ {−∞, 1, 2, . . . , `} × {1, 2, . . . , `, ∞} of length ` ≥ 0 is a matching; relation if the following holds:

I if i j, then i < j (monotone)

II if i1; j and i2 j, then i1 = i2 (left-unique) if i ;j1 and i ;j1, then j1 = j2 (right-unique) III if i1; j1 and i2; j2, then we have not i1 < i2 < j1 < j2 ; ; (well nested)

If i j, i is a call position and j is a return position. All the rest is an internal; position. If i 6= −∞ and j 6= ∞, they are well-matched, otherwise pending. e ∈ is a nesting edge. ;

10 / 30 Nested Word Automata Nested words and their acceptors Nested words Nested words

A relation ⊂ {−∞, 1, 2, . . . , `} × {1, 2, . . . , `, ∞} of length ` ≥ 0 is a matching; relation if the following holds:

I if i j, then i < j (monotone)

II if i1; j and i2 j, then i1 = i2 (left-unique) if i ;j1 and i ;j1, then j1 = j2 (right-unique) III if i1; j1 and i2; j2, then we have not i1 < i2 < j1 < j2 ; ; (well nested)

If i j, i is a call position and j is a return position. All the rest is an internal; position. If i 6= −∞ and j 6= ∞, they are well-matched, otherwise pending. e ∈ is a nesting edge. ; A nested wordn over Σ is a pair (a1 ··· a`, ), where ai ∈ Σ and is a matching relation of length `. ;

; 10 / 30 Nested Word Automata Nested words and their acceptors Nested words Example 1

iciciirri 1 2 8 9

4 7 3

5 6 adapted from [1]

Here: 2 8, 4 7 and the whole word is well-matched. ; ;

11 / 30 Nested Word Automata Nested words and their acceptors Nested words Example 2

rcrrcici 4 5

1 2 3 6

8 adapted from [1]

Here: −∞ 1, 2 3, −∞ 4, 5 ∞, 7 ∞ and only 2 3 is well-matched.; ; ; ; ; ;

12 / 30 • Q ﬁnite set of linear states,

• q0 ∈ Q initial linear state,

• Qf ⊆ Q set of linear ﬁnal states, • P ﬁnite set of hierarchical states,

• p0 ∈ Q initial hierarchical state,

• Pf ⊆ P set of hierarchical ﬁnal states,

• δi ⊆ Q × Σ → Q internal transition function,

• δc ⊆ Q × Σ → Q × P call transition function,

• δr ⊆ Q × P × Σ → Q return transition function

acceptance via both Qf and Pf as VPAs: at return implicitly go to hierarchical state before matching call

Nested Word Automata Nested words and their acceptors Nested word automata Deﬁnition of NWAs

A = hQ, q0, Qf , P, p0, Pf ,δ i ,δ c ,δ r i over alphabet Σ

13 / 30 • P ﬁnite set of hierarchical states,

• p0 ∈ Q initial hierarchical state,

• Pf ⊆ P set of hierarchical ﬁnal states,

• δi ⊆ Q × Σ → Q internal transition function,

• δc ⊆ Q × Σ → Q × P call transition function,

• δr ⊆ Q × P × Σ → Q return transition function

acceptance via both Qf and Pf as VPAs: at return implicitly go to hierarchical state before matching call

Nested Word Automata Nested words and their acceptors Nested word automata Deﬁnition of NWAs

A = hQ, q0, Qf , P, p0, Pf ,δ i ,δ c ,δ r i over alphabet Σ • Q ﬁnite set of linear states,

• q0 ∈ Q initial linear state,

• Qf ⊆ Q set of linear ﬁnal states,

13 / 30 • δi ⊆ Q × Σ → Q internal transition function,

• δc ⊆ Q × Σ → Q × P call transition function,

• δr ⊆ Q × P × Σ → Q return transition function

acceptance via both Qf and Pf as VPAs: at return implicitly go to hierarchical state before matching call

Nested Word Automata Nested words and their acceptors Nested word automata Deﬁnition of NWAs

A = hQ, q0, Qf , P, p0, Pf ,δ i ,δ c ,δ r i over alphabet Σ • Q ﬁnite set of linear states,

• q0 ∈ Q initial linear state,

• Qf ⊆ Q set of linear ﬁnal states, • P ﬁnite set of hierarchical states,

• p0 ∈ Q initial hierarchical state,

• Pf ⊆ P set of hierarchical ﬁnal states,

13 / 30 acceptance via both Qf and Pf as VPAs: at return implicitly go to hierarchical state before matching call

Nested Word Automata Nested words and their acceptors Nested word automata Deﬁnition of NWAs

A = hQ, q0, Qf , P, p0, Pf ,δ i ,δ c ,δ r i over alphabet Σ • Q ﬁnite set of linear states,

• q0 ∈ Q initial linear state,

• Qf ⊆ Q set of linear ﬁnal states, • P ﬁnite set of hierarchical states,

• p0 ∈ Q initial hierarchical state,

• Pf ⊆ P set of hierarchical ﬁnal states,

• δi ⊆ Q × Σ → Q internal transition function,

• δc ⊆ Q × Σ → Q × P call transition function,

• δr ⊆ Q × P × Σ → Q return transition function

13 / 30 Nested Word Automata Nested words and their acceptors Nested word automata Deﬁnition of NWAs

A = hQ, q0, Qf , P, p0, Pf ,δ i ,δ c ,δ r i over alphabet Σ • Q ﬁnite set of linear states,

• q0 ∈ Q initial linear state,

• Qf ⊆ Q set of linear ﬁnal states, • P ﬁnite set of hierarchical states,

• p0 ∈ Q initial hierarchical state,

• Pf ⊆ P set of hierarchical ﬁnal states,

• δi ⊆ Q × Σ → Q internal transition function,

• δc ⊆ Q × Σ → Q × P call transition function,

• δr ⊆ Q × P × Σ → Q return transition function

acceptance via both Qf and Pf as VPAs: at return implicitly go to hierarchical state before matching call 13 / 30 We can also use hierarchical states for acceptance.

Nested Word Automata Nested words and their acceptors Nested word automata

L2 as NWA

n n Consider again L2 = {c r | n > 0}. 0 n n We construct an NWA for L2 := {(hc) (ri) | n > 0}.

q0 q3

c, A r, A r, A

q1 q2 r, B

c, B r, B

14 / 30 We can also use hierarchical states for acceptance.

Nested Word Automata Nested words and their acceptors Nested word automata

L2 as NWA

n n Consider again L2 = {c r | n > 0}. 0 n n We construct an NWA for L2 := {(hc) (ri) | n > 0}.

q0 q3

hc/p0 ri/p0 ri/p0 P = {p0, p1}, Pf ⊆ {p0} q1 q2 ri/p1

hc/p1 ri/p1

14 / 30 Nested Word Automata Nested words and their acceptors Nested word automata

L2 as NWA

n n Consider again L2 = {c r | n > 0}. 0 n n We construct an NWA for L2 := {(hc) (ri) | n > 0}. We can also use hierarchical states for acceptance.

hc/p1 P = {p0, p1}, Pf = {p0} q1 q2 ri/p1

hc/p1 ri/p1

14 / 30 {(ha)n(bi)n | n > 0} vs. {anbn | n > 0}

possibly exponentially more states for deterministic NWAs • not all sets of NWs acceptable by NWAs

• nondeterministic NWAs: Q0 ⊆ Q, P0 ⊆ P, δ

Nested Word Automata Nested words and their acceptors Nested word automata Remarks

• no stack anymore, but structure on the input word

15 / 30 {(ha)n(bi)n | n > 0} vs. {anbn | n > 0}

• not all sets of NWs acceptable by NWAs

possibly exponentially more states for deterministic NWAs

Nested Word Automata Nested words and their acceptors Nested word automata Remarks

• no stack anymore, but structure on the input word

• nondeterministic NWAs: Q0 ⊆ Q, P0 ⊆ P, δ

15 / 30 {(ha)n(bi)n | n > 0} vs. {anbn | n > 0}

• not all sets of NWs acceptable by NWAs

Nested Word Automata Nested words and their acceptors Nested word automata Remarks

• no stack anymore, but structure on the input word

• nondeterministic NWAs: Q0 ⊆ Q, P0 ⊆ P, δ possibly exponentially more states for deterministic NWAs

15 / 30 Nested Word Automata Nested words and their acceptors Nested word automata Remarks

• no stack anymore, but structure on the input word

• nondeterministic NWAs: Q0 ⊆ Q, P0 ⊆ P, δ possibly exponentially more states for deterministic NWAs • not all sets of NWs acceptable by NWAs {(ha)n(bi)n | n > 0} vs. {anbn | n > 0}

15 / 30 Note: Equivalence and inclusion problem are exptime-complete for nondeterministic NWAs. Implication: determinization ∈ Ω(exptime) if at all possible

! ! ! % ! ! % ! ! ! % % Nlogspace ptime ptime ptime Nlogspace ptime undecidable decidable Nlogspace ptime undecidable undecidable

Nested Word Automata Nested words and their acceptors Nested word automata Comparison of properties

DFA DNWA PDA DPDA pre-/suﬃx ! ! ! ! ∪, ·, ∗ complement ∩ emptiness equivalence inclusion

16 / 30 Note: Equivalence and inclusion problem are exptime-complete for nondeterministic NWAs. Implication: determinization ∈ Ω(exptime) if at all possible

Nlogspace ptime ptime ptime Nlogspace ptime undecidable decidable Nlogspace ptime undecidable undecidable

Nested Word Automata Nested words and their acceptors Nested word automata Comparison of properties

DFA DNWA PDA DPDA pre-/suﬃx ! ! ! ! ∪, ·, ∗ ! ! ! % complement ! ! % ! ∩ ! ! % % emptiness equivalence inclusion

16 / 30 Note: Equivalence and inclusion problem are exptime-complete for nondeterministic NWAs. Implication: determinization ∈ Ω(exptime) if at all possible

Nested Word Automata Nested words and their acceptors Nested word automata Comparison of properties

DFA DNWA PDA DPDA pre-/suﬃx ! ! ! ! ∪, ·, ∗ ! ! ! % complement ! ! % ! ∩ ! ! % % emptiness Nlogspace ptime ptime ptime equivalence Nlogspace ptime undecidable decidable inclusion Nlogspace ptime undecidable undecidable

16 / 30 Nested Word Automata Nested words and their acceptors Nested word automata Comparison of properties

Note: Equivalence and inclusion problem are exptime-complete for nondeterministic NWAs. Implication: determinization ∈ Ω(exptime) if at all possible

16 / 30 Nested Word Automata Determinization proof

Overview

Motivation and background

Nested words and their acceptors

Determinization proof Intuition Construction

Conclusion

17 / 30 • state of automaton A for nested word n with position k: deterministic NWA (DNWA): (qk , pk )

NNWA: one of (qk1 , pk1 ), ... ,(qki , pkj )

• ﬁnite automata: call the states {qk1 ,..., qki } • NWAs: also need information about hierarchical states → powerset construction over nesting edges hierarchical states = nesting edges + call symbol so far • handle hierarchical proceeding when reading return symbols

Nested Word Automata Determinization proof Intuition Idea behind the proof

• goal: determinize a nondeterministic NWA (NNWA)

18 / 30 • ﬁnite automata: call the states {qk1 ,..., qki } • NWAs: also need information about hierarchical states → powerset construction over nesting edges hierarchical states = nesting edges + call symbol so far • handle hierarchical proceeding when reading return symbols

Nested Word Automata Determinization proof Intuition Idea behind the proof

• goal: determinize a nondeterministic NWA (NNWA) • state of automaton A for nested word n with position k: deterministic NWA (DNWA): (qk , pk )

NNWA: one of (qk1 , pk1 ), ... ,(qki , pkj )

18 / 30 • NWAs: also need information about hierarchical states → powerset construction over nesting edges hierarchical states = nesting edges + call symbol so far • handle hierarchical proceeding when reading return symbols

Nested Word Automata Determinization proof Intuition Idea behind the proof

• goal: determinize a nondeterministic NWA (NNWA) • state of automaton A for nested word n with position k: deterministic NWA (DNWA): (qk , pk )

NNWA: one of (qk1 , pk1 ), ... ,(qki , pkj )

• ﬁnite automata: call the states {qk1 ,..., qki }

18 / 30 • handle hierarchical proceeding when reading return symbols

Nested Word Automata Determinization proof Intuition Idea behind the proof

• goal: determinize a nondeterministic NWA (NNWA) • state of automaton A for nested word n with position k: deterministic NWA (DNWA): (qk , pk )

NNWA: one of (qk1 , pk1 ), ... ,(qki , pkj )

18 / 30 Nested Word Automata Determinization proof Intuition Idea behind the proof

• goal: determinize a nondeterministic NWA (NNWA) • state of automaton A for nested word n with position k: deterministic NWA (DNWA): (qk , pk )

NNWA: one of (qk1 , pk1 ), ... ,(qki , pkj )

18 / 30 0 Q×Q • Q := 2 = {S1,..., Si } 0 • q0 := Q0 × Q0 0 0 0 0 • Qf := {S | ∃q, q .(q, q ) ∈ S ∧ q ∈ Qf } 0 0 0 or: S ∈ Qf :⇔ S contains (q, q ) with q ∈ Qf 0 0 0 • P := {p0} ∪ (Q × Σ) 0 • p0 := fresh hierarchical state 0 0 • Pf := P

Nested Word Automata Determinization proof Construction The states: deﬁnition

wlog =P z}|{ Consider the NNWA A = hQ, Q0, Qf , P, P0, Pf , δi , δc , δr i. 0 0 0 0 0 0 0 0 0 We construct the DNWA B = hQ , q0, Qf , P , p0, Pf , δi , δc , δr i:

19 / 30 0 • q0 := Q0 × Q0 0 0 0 0 • Qf := {S | ∃q, q .(q, q ) ∈ S ∧ q ∈ Qf } 0 0 0 or: S ∈ Qf :⇔ S contains (q, q ) with q ∈ Qf 0 0 0 • P := {p0} ∪ (Q × Σ) 0 • p0 := fresh hierarchical state 0 0 • Pf := P

Nested Word Automata Determinization proof Construction The states: deﬁnition

wlog =P z}|{ Consider the NNWA A = hQ, Q0, Qf , P, P0, Pf , δi , δc , δr i. 0 0 0 0 0 0 0 0 0 We construct the DNWA B = hQ , q0, Qf , P , p0, Pf , δi , δc , δr i: 0 Q×Q • Q := 2 = {S1,..., Si }

19 / 30 0 0 0 0 • Qf := {S | ∃q, q .(q, q ) ∈ S ∧ q ∈ Qf } 0 0 0 or: S ∈ Qf :⇔ S contains (q, q ) with q ∈ Qf 0 0 0 • P := {p0} ∪ (Q × Σ) 0 • p0 := fresh hierarchical state 0 0 • Pf := P

Nested Word Automata Determinization proof Construction The states: deﬁnition

wlog =P z}|{ Consider the NNWA A = hQ, Q0, Qf , P, P0, Pf , δi , δc , δr i. 0 0 0 0 0 0 0 0 0 We construct the DNWA B = hQ , q0, Qf , P , p0, Pf , δi , δc , δr i: 0 Q×Q • Q := 2 = {S1,..., Si } 0 • q0 := Q0 × Q0

19 / 30 0 0 0 • P := {p0} ∪ (Q × Σ) 0 • p0 := fresh hierarchical state 0 0 • Pf := P

Nested Word Automata Determinization proof Construction The states: deﬁnition

wlog =P z}|{ Consider the NNWA A = hQ, Q0, Qf , P, P0, Pf , δi , δc , δr i. 0 0 0 0 0 0 0 0 0 We construct the DNWA B = hQ , q0, Qf , P , p0, Pf , δi , δc , δr i: 0 Q×Q • Q := 2 = {S1,..., Si } 0 • q0 := Q0 × Q0 0 0 0 0 • Qf := {S | ∃q, q .(q, q ) ∈ S ∧ q ∈ Qf } 0 0 0 or: S ∈ Qf :⇔ S contains (q, q ) with q ∈ Qf

19 / 30 0 • p0 := fresh hierarchical state 0 0 • Pf := P

Nested Word Automata Determinization proof Construction The states: deﬁnition

19 / 30 0 0 • Pf := P

Nested Word Automata Determinization proof Construction The states: deﬁnition

19 / 30 Nested Word Automata Determinization proof Construction The states: deﬁnition

19 / 30 Invariants

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

Question: acceptance condition of B for n? 0 Answer: Sk+1 ∈ Qf , 0 0 nk+1 0 0 i.e., ∃q, q .(q, q ) ∈ Sk+1 ∧ q → A q ∧ q ∈ Qf

Nested Word Automata Determinization proof Construction The states: semantics

Consider a nested word n with k pending calls. We can write this

n = n1hc1n2hc2 ··· nk hck nk+1

where the ni have no pending calls.

20 / 30 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

Question: acceptance condition of B for n? 0 Answer: Sk+1 ∈ Qf , 0 0 nk+1 0 0 i.e., ∃q, q .(q, q ) ∈ Sk+1 ∧ q → A q ∧ q ∈ Qf

Nested Word Automata Determinization proof Construction The states: semantics

Consider a nested word n with k pending calls. We can write this

n = n1hc1n2hc2 ··· nk hck nk+1

where the ni have no pending calls.

Invariants

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci .

20 / 30 Question: acceptance condition of B for n? 0 Answer: Sk+1 ∈ Qf , 0 0 nk+1 0 0 i.e., ∃q, q .(q, q ) ∈ Sk+1 ∧ q → A q ∧ q ∈ Qf

Nested Word Automata Determinization proof Construction The states: semantics

Consider a nested word n with k pending calls. We can write this

n = n1hc1n2hc2 ··· nk hck nk+1

where the ni have no pending calls.

Invariants

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

20 / 30 0 Answer: Sk+1 ∈ Qf , 0 0 nk+1 0 0 i.e., ∃q, q .(q, q ) ∈ Sk+1 ∧ q → A q ∧ q ∈ Qf

Nested Word Automata Determinization proof Construction The states: semantics

Consider a nested word n with k pending calls. We can write this

n = n1hc1n2hc2 ··· nk hck nk+1

where the ni have no pending calls.

Invariants

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

Question: acceptance condition of B for n?

20 / 30 , 0 0 nk+1 0 0 i.e., ∃q, q .(q, q ) ∈ Sk+1 ∧ q → A q ∧ q ∈ Qf

Nested Word Automata Determinization proof Construction The states: semantics

Consider a nested word n with k pending calls. We can write this

n = n1hc1n2hc2 ··· nk hck nk+1

where the ni have no pending calls.

Invariants

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

Question: acceptance condition of B for n? 0 Answer: Sk+1 ∈ Qf

20 / 30 Nested Word Automata Determinization proof Construction The states: semantics

Consider a nested word n with k pending calls. We can write this

n = n1hc1n2hc2 ··· nk hck nk+1

where the ni have no pending calls.

Invariants

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

Question: acceptance condition of B for n? 0 Answer: Sk+1 ∈ Qf , 0 0 nk+1 0 0 i.e., ∃q, q .(q, q ) ∈ Sk+1 ∧ q → A q ∧ q ∈ Qf

20 / 30 00 0 00 0 {(q, q ) | (q, q ) ∈ Sk+1 ∧ q ∈ δi (q , i)}

Nested Word Automata Determinization proof Construction Internal transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

0 n = n · i = n1hc1n2hc2 ··· nk hck nk+1i

0 δi (Sk+1, i) =

21 / 30 Nested Word Automata Determinization proof Construction Internal transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II S contains the pair (q, q ) iﬀ q → q . nk+1 i i A q q0 q00

0 n = n · i = n1hc1n2hc2 ··· nk hck nk+1i

0 00 0 00 0 δi (Sk+1, i) = {(q, q ) | (q, q ) ∈ Sk+1 ∧ q ∈ δi (q , i)}

21 / 30 {(0, 3)}

b a ⇒ c {(0, 0)} {(0, 1), (0, 2)} {(0, 4)}

Nested Word Automata Determinization proof Construction Example

b 1 3 a 0 a c 2 4

22 / 30 Nested Word Automata Determinization proof Construction Example

{(0, 3)} b 1 3 a b a 0 ⇒ a c c {(0, 0)} {(0, 1), 2 4 (0, 2)} {(0, 4)}

22 / 30 0 (S , (Sk+1, ck+1)), 0 00 00 0 00 0 S = {(q , q ) | (q, q ) ∈ Sk+1 ∧ ∃p ∈ P.(q , p) ∈ δc (q , ck+1)}

Nested Word Automata Determinization proof Construction Call transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

0 n = n · hck+1 = n1hc1n2hc2 ··· nk hck nk+1hck+1

0 δc (Sk+1, ck+1) =

new hierarchical state that keeps track of the old state/symbol

23 / 30 0 00 00 0 00 0 S = {(q , q ) | (q, q ) ∈ Sk+1 ∧ ∃p ∈ P.(q , p) ∈ δc (q , ck+1)}

Nested Word Automata Determinization proof Construction Call transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

0 n = n · hck+1 = n1hc1n2hc2 ··· nk hck nk+1hck+1

0 0 δc (Sk+1, ck+1) = (S , (Sk+1, ck+1)),

new hierarchical state that keeps track of the old state/symbol

23 / 30 Nested Word Automata Determinization proof Construction Call transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II S contains the pair (q, q ) iﬀ q → q . nk+1 i A q q0

ck+1/p

0 00 n = n · hck+1 = n1hc1n2hc2 ··· nk hck nk+1hck+1 q

0 0 δc (Sk+1, ck+1) = (S , (Sk+1, ck+1)), 0 00 00 0 00 0 S = {(q , q ) | (q, q ) ∈ Sk+1 ∧ ∃p ∈ P.(q , p) ∈ δc (q , ck+1)} new hierarchical state that keeps track of the old state/symbol

23 / 30 hc/({(0, 0)}, c) ⇒ {(0, 0)} {(1, 1), (2, 2)}

Nested Word Automata Determinization proof Construction Example

1 hc/p1

hc/p 1 2

24 / 30 Nested Word Automata Determinization proof Construction Example

1 hc/p1 hc/({(0, 0)}, c) 0 ⇒ {(0, 0)} hc/p1 {(1, 1), 2 (2, 2)}

24 / 30 00 0 00 0 {(q, q ) | (q, q ) ∈ Sk+1 ∧ ∃p ∈ P0.q ∈ δr (q , p, r)}

k > 0 subword nk hck nk+1ri, hierarchical state = (Sk , ck ) 0 00 0 δr (Sk+1, (Sk , ck ), r) = {(q, q ) | (q, q ) ∈ Sk ∧ (q1, q2) ∈ Sk+1 0 00 ∧ ∃p ∈ P.(q1, p) ∈ δc (q , ck ) ∧ q ∈ δr (q2, p, r)}

Nested Word Automata Determinization proof Construction Return transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

0 n = n · ri = n1hc1n2hc2 ··· nk hck nk+1ri

We have two cases here: k = 0 no matching call, like internal transition 0 0 δr (Sk+1, p0, r) =

25 / 30 k > 0 subword nk hck nk+1ri, hierarchical state = (Sk , ck ) 0 00 0 δr (Sk+1, (Sk , ck ), r) = {(q, q ) | (q, q ) ∈ Sk ∧ (q1, q2) ∈ Sk+1 0 00 ∧ ∃p ∈ P.(q1, p) ∈ δc (q , ck ) ∧ q ∈ δr (q2, p, r)}

Nested Word Automata Determinization proof Construction Return transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q . q00 0 n = n · ri = n1hc1n2hc2 ··· nk hck nk+1ri r/p

q q0 We have two cases here: nk+1 k = 0 no matching call, like internal transition 0 0 δr (Sk+1, p0, r) = 00 0 00 0 {(q, q ) | (q, q ) ∈ Sk+1 ∧ ∃p ∈ P0.q ∈ δr (q , p, r)}

25 / 30 00 0 {(q, q ) | (q, q ) ∈ Sk ∧ (q1, q2) ∈ Sk+1 0 00 ∧ ∃p ∈ P.(q1, p) ∈ δc (q , ck ) ∧ q ∈ δr (q2, p, r)}

Nested Word Automata Determinization proof Construction Return transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q .

0 n = n · ri = n1hc1n2hc2 ··· nk hck nk+1ri

We have two cases here: k = 0 no matching call, like internal transition 0 0 δr (Sk+1, p0, r) = 00 0 00 0 {(q, q ) | (q, q ) ∈ Sk+1 ∧ ∃p ∈ P0.q ∈ δr (q , p, r)}

k > 0 subword nk hck nk+1ri, hierarchical state = (Sk , ck ) 0 δr (Sk+1, (Sk , ck ), r) =

25 / 30 Nested Word Automata Determinization proof Construction Return transitions

I After reading n, B will be in state Sk+1, where (Si , ci ) will be the hierarchical state for each hci . nk q q0 q00 0 ni 0 II Si contains the pair (q, q ) iﬀ q →A q . ck /p r/p n 0 k+1 n = n · ri = n1hc1n2hc2 ··· nk hck nk+1ri q1 q2

We have two cases here: k = 0 no matching call, like internal transition 0 0 δr (Sk+1, p0, r) = 00 0 00 0 {(q, q ) | (q, q ) ∈ Sk+1 ∧ ∃p ∈ P0.q ∈ δr (q , p, r)}

25 / 30 ri/p0 ⇒ 0 {(0, 0)} {(0, 1), (0, 2)}

Nested Word Automata Determinization proof Construction Example

1 ri/p0

ri/p 0 2

26 / 30 Nested Word Automata Determinization proof Construction Example

1 ri/p0 0 ri/p0 0 ⇒ {(0, 0)} ri/p0 {(0, 1), 2 (0, 2)}

26 / 30 {(0, 2), {(0, 0)} (0, 3)}

⇒ hc/({(0, 0)}, c) ri/({(0, 0)}, c)

{(1, 1)}

Nested Word Automata Determinization proof Construction Example

2 ri/p1 hc/p1 0 1

ri/p 1 3

27 / 30 Nested Word Automata Determinization proof Construction Example

{(0, 2), {(0, 0)} (0, 3)} 2 ri/p1 hc/p1 0 1 ⇒ hc/({(0, 0)}, c) ri/({(0, 0)}, c) ri/p 1 3 {(1, 1)}

27 / 30 • correctness results from invariants 2 2 • complexity: if |Q| = s, then |Q0| = 2s and |P0| ∈ O(2s ) This is succinct, so there exists an example where the DNWA cannot have less states.

Nested Word Automata Determinization proof Construction R´esum´e

• now all components of B deﬁned

28 / 30 2 2 • complexity: if |Q| = s, then |Q0| = 2s and |P0| ∈ O(2s ) This is succinct, so there exists an example where the DNWA cannot have less states.

Nested Word Automata Determinization proof Construction R´esum´e

• now all components of B deﬁned • correctness results from invariants

28 / 30 Nested Word Automata Determinization proof Construction R´esum´e

• now all components of B deﬁned • correctness results from invariants 2 2 • complexity: if |Q| = s, then |Q0| = 2s and |P0| ∈ O(2s ) This is succinct, so there exists an example where the DNWA cannot have less states.

28 / 30 Nested Word Automata Conclusion

Overview

Motivation and background

Nested words and their acceptors

Determinization proof

Conclusion

29 / 30 • visibly pushdown automata and nested word automata as suitable models for this class • no stack, but complexity shifted to the input word • all relevant closure properties, all interesting problems decidable 2 • determinization always possible in O(2s ) • many practical problems describable as nested words • recent concept, time will show the relevance

Nested Word Automata Conclusion

Conclusion

• nested word languages as a (proper) fragment of deterministic context-free languages strictly more expressive than regular languages

30 / 30 • no stack, but complexity shifted to the input word • all relevant closure properties, all interesting problems decidable 2 • determinization always possible in O(2s ) • many practical problems describable as nested words • recent concept, time will show the relevance

Nested Word Automata Conclusion

Conclusion

30 / 30 • all relevant closure properties, all interesting problems decidable 2 • determinization always possible in O(2s ) • many practical problems describable as nested words • recent concept, time will show the relevance

Nested Word Automata Conclusion

Conclusion

30 / 30 2 • determinization always possible in O(2s ) • many practical problems describable as nested words • recent concept, time will show the relevance

Nested Word Automata Conclusion

Conclusion

30 / 30 • many practical problems describable as nested words • recent concept, time will show the relevance

Nested Word Automata Conclusion

Conclusion

• nested word languages as a (proper) fragment of deterministic context-free languages strictly more expressive than regular languages • visibly pushdown automata and nested word automata as suitable models for this class • no stack, but complexity shifted to the input word • all relevant closure properties, all interesting problems decidable 2 • determinization always possible in O(2s )

30 / 30 • recent concept, time will show the relevance

Nested Word Automata Conclusion

Conclusion

30 / 30 Nested Word Automata Conclusion

Conclusion

30 / 30 References

Rajeev Alur and Parthasarathy Madhusudan. Adding Nested Structure to Words. In Journal of the ACM, 2009. Rajeev Alur and Parthasarathy Madhusudan. Visibly Pushdown Languages. In STOC ’04, 2004.