CIT 596 – Theory of Computation 1

Theory of Computation: An Introduction

Traditionally, the study of theory of computation comprises three cen- tral areas:

automata, computability, and complexity.

These areas are linked by the question:

What are the fundamental capabilities and limitations of computers?

In this course, we will study the fundamentals of these three central ar- eas of theory of computation.

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 2

Theory of Computation: An Introduction

The central question of complexity theory is

What makes some problems computationally hard and others easy?.

Remarkably, we still have not been able to answer this question!

An important research achievement, though, was the definition of a scheme to classify problems according to their “level of difficulty” (diffi- culty is precisely expressed in this scheme).

What does a computer scientist do when faced with a “hard” computa- tional problem?

Cryptography: sometimes, hardness is very desirable!

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 3

Theory of Computation: An Introduction The central question of computability theory is

Which problems can be solved by computers and which ones cannot?

Even before the invention of “real computers”, Kurt Gödel, Alan Turing, and Alonzo Church discovered that certain basic problems cannot be solved by computers!

For instance, computers cannot solve the problem of determining whether a mathematical statement is true or false.

The fact that this discovery is older than the birth of real computers sug- gests that we do not have to study semicondutors or integrated circuits in order to understand “computation”.

Actually, we can do that by using “abstract models” of computers.

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 4

Theory of Computation: An Introduction

Automata theory deals with the definitions and properties of “abstract models” of computers.

These abstract models are models of computation that can be defined mathematically.

Some of these models are (believed to be) as powerful as real comput- ers, whereas others can be defined more simply and are less powerful.

The simplest model of computation is the finite automaton or finite- state machine.

This simple model has several applications in computer science, includ- ing text processing, compilers, and interface design.

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 5

Theory of Computation: An Introduction

Another important model is the pushdown automaton, which is more powerful than the finite automaton.

This model provides a natural way of determining whether the sintax of a statement in a high-level programming language is correct, and therefore it is heavily used in compiler construction and computational linguistics.

Finally, there is the Turing machine, named after Alan Turing, the En- glish mathematician who invented it.

There is general agreement that Turing machines are able to carry out any algorithm whatsoever.

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 6

Theory of Computation: An Introduction

How do these models of computation work?

These models actually solve decision problems, i.e., computational problems for which every specific instance can be answered “yes” or “no”.

For instance, “Given a positive integer n, is it a prime?”

The input value n is encoded as a string of digits, and a computation that solves the problem starts with this string and ends with a “yes” or “no”.

This encoding process is no different from what actually happens with the input data in real computers. Everything is converted into strings of 0’s and 1’s, isn’t it?

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 7

Theory of Computation: An Introduction

Not all computational problems are decision problems. However, in the case of a problem requiring an answer more complicated than “yes” or “no”, there is often a decision problem whose solution is comparable to that of the original.

For instance, if f is a function, being able to answer the question “Given x and y, is y = f(x)?” is tantamount to being able to compute f(x) for an arbitrary x.

Furthermore, the more powerful computational models are able to deal with more general kinds of problems.

An extremely interesting point is the relationship between models of computation and languages.

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 8

Theory of Computation: An Introduction

We can think of a decision problem as a language recognition prob- lem.

For instance, for the primality decison problem (“Given a positive integer n, is n a prime?”), we can view ALL strings for which our model answers “yes” as a “language”.

So, our decision problem can be viewed as the one of determining whether an arbitrary string is in the language recognized by the model of computation.

In our example, the recognition problem is to determine whether a given string (enconding a number) belongs to the language of ALL strings rep- resenting prime numbers.

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 9

Theory of Computation: An Introduction

Distinct models of computation recognize (resp. solves) languages (resp. decision problems) of different complexity.

Finite automata recognize regular languages.

Pushdown automata recognize context-free languages.

Turing machines recognize recursively enumerable languages.

Regular languages are simpler than context-free languages, which in turn are simpler than recursively enumerable languages. This seems very natural! Why?

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 10

Theory of Computation: An Introduction

We can also study “languages” from a generation point of view, rather than recognition.

From the generation point of view, we use a formalism known as a gram- mar.

A grammar contains a set of rules that can be applied to generate all strings in a language.

Grammars are extremely useful for specifying and representing pro- gramming languages.

In order to build a compiler for a programming language, we must spec- ify the language in some way. Grammars are pretty handy and often used for that.

c Marcelo Siqueira — Spring 2005 CIT 596 – Theory of Computation 11

Theory of Computation: An Introduction As you may expect, we also classify grammars according the complexity of the languages they can generate.

Regular languages are generated by regular grammars.

Context-free languages are generated by context-free grammars.

Recursively enumerable languages are generated by unrestricted gram- mars.

This hierarchy of grammars and languages was created by Noam Chom- sky, and it is known as Chomsky hierarchy.

Yeah, this is the same Noam Chomsky you may have heard about in the past four years within a completely different context... He is a former PhD student of UPenn!

c Marcelo Siqueira — Spring 2005