sign in sign up
<<
Home , Randomized algorithm

Randomized

Prabhakar Raghavan

IBM Almaden Research Center

San Jose CA

Typ eset byFoilT X E

Deterministic Algorithms

INPUT

OUTPUT

Goal To prove that the

algorithm solves the problem

correctly always and quickly

typically the number of steps

should be p olynomial in the

size of the input

Typ eset byFoilT X E

Randomized Algorithms

INPUT ALGORITHM OUTPUT

RANDOM NUMBERS

In addition to input algorithm

takes a source of random

numbers and makes random

choices during execution

Behavior can vary even on a

xed input

Typ eset byFoilT X E

Randomized Algorithms

INPUT ALGORITHM OUTPUT

RANDOM NUMBERS

Design algorithm analysis

to show that this b ehavior is

likely to be good on every

input

The likeliho o d is over the

random numbers only

Typ eset byFoilT X E

Not to be confused with the

Probabilistic Analysis of

Algorithms

RANDOM INPUT ALGORITHM OUTPUT

DISTRIBUTION

Here the input is assumed

to be from a probability

distribution

Show that the algorithm works

for most inputs

Typ eset byFoilT X E

Monte Carlo and Las Vegas

A runs

for a xed number of steps

and pro duces an answer that is

correct with probability

A always

pro duces the correct answer

its running time is a random

variable whose exp ectation is

bounded say by a polynomial

Typ eset byFoilT X E

Monte Carlo and Las Vegas

These probabilitiesexp ectations

are only over the random

choices made by the algorithm

indep endent of the input

Thus indep endent rep etitions

of Monte Carlo algorithms drive

down the failure probability

exp onentially

Typ eset byFoilT X E

Advantages of randomized

algorithms

Simplicity

Performance

For many problems a randomized

algorithm is the simplest the

fastest or b oth

Typ eset byFoilT X E

Scop e

Numb ertheoretic algorithms

Primality testing Monte Carlo

Data structures Sorting

order statistics searching

Algebraic identities Polynomial

and matrix identity verication

Interactive pro of systems

Typ eset byFoilT X E

Mathematical programming

Faster algorithms for linear

programming Rounding linear

program solutions to integer

program solutions

Graph algorithms Minimum

spanning trees shortest paths

minimum cuts

Counting and enumeration

Matrix p ermanent Counting

combinatorial structures

Typ eset byFoilT X E

Parallel and distributed

Deadlo ck avoidance distributed

consensus

Probabilistic existence pro ofs

Show that a combinatorial

object arises with nonzero

probability among objects drawn

from a suitable probability

space

Typ eset byFoilT X E

Derandomization First devise

a randomized algorithm then

argue that it can be derandomized

to yield a

Typ eset byFoilT X E

Gametree evaluation

9

MAX

93

MIN MIN

79 317

MAX MAX MAX MAX

1 7 9 5 3 2 17 12

Tree with alternating MAXMIN

no des

Each leaf has a real value

Goal To evaluate the ro ot

Typ eset byFoilT X E

Gametree evaluation

9

MAX

93

MIN MIN

79 317

MAX MAX MAX MAX

1 7 9 5 3 2 17 12

Cost The number of leaves

insp ected

Algorithm Computes leaf

insp ection sequence

Typ eset byFoilT X E

Simple sp ecial case

Uniform binary tree height h

h

with n leaves

All values are b o olean so

MAXOR and MINAND

Exercise Every deterministic

algorithm can be forced to read

n leaves

Typ eset byFoilT X E

Randomized algorithm

AND

10

OR OR

01 00

AND AND AND AND

1011 1010

Evaluate recursively a random

child of the current no de

If this do es not determine

the value of the current no de

evaluate the other child

Typ eset byFoilT X E

Analysis of tree evaluation

ANDOR trees of depth k k

levels of AND and k levels of

OR on any path will prove by

induction on k

Exp ected cost for an OR

subtree evaluating to

k  k  k 

Typ eset byFoilT X E

Analysis of tree evaluation

For an OR subtree evaluating

to we must evaluate b oth

of its subtrees incurring cost

k 

Now consider the ro ot AND

a If it is both OR subtrees

must evaluate to exp ected

k

cost paid is twice ab ove

b If it is at least one OR

subtree evaluates to

Typ eset byFoilT X E

Game tree analysis

Exp ected cost for ro ot AND

evaluating to

k  k  k

k

Number of leaves in tree

Thus exp ected cost of randomized



algorithm n

Typ eset byFoilT X E

Lower bounds and the

minimax principle

Basic Idea For a lower

bound on the performance of

all randomized algorithms for a

problem derive instead a lower

bound for any deterministic

algorithm for the problem when

the inputs are drawn from a

probability distribution of your

choice

Typ eset byFoilT X E

Let I denote the set of

instances of the problem and

A the set of deterministic

algorithms for the problem

Any randomized algorithm can

be viewed as a probability

distribution on the algorithms in

A

Typ eset byFoilT X E

Minimax Principle

For all distributions p over I

and q over A

min EC I A max E C I A

p q

AA I I

Typ eset byFoilT X E

Lower b ound for game tree

evaluation

We will sp ecify a probability

distribution on instances

values for the leaves and lower

bound the exp ected running

timeof any deterministic algorithm

Typ eset byFoilT X E

NOR trees instead

Exercise Showthat a balanced

ANDOR tree of even depth is

equivalent to a tree of the same

depth all of whose no des are

NOR no des

We will show the lower b ound

for a NOR tree with n leaves

Typ eset byFoilT X E

The input distribution

p

Let p

Claim The value of any

no de is with probability p

indep endent of all other no des

at the same level

Claim Any deterministic

algorithm mayas well determine

the value of one subtree of a

no de before insp ecting any leaf

of its sibling subtree

Typ eset byFoilT X E

The Analysis

Let W h be the exp ected

number of leaves it insp ects in

determining the value of a no de

at distance h from the leaves

Clearly

W h W hpW h

Letting h log n this gives a





lower b ound of n

Typ eset byFoilT X E

Exercise Why is this lower

bound weak

Typ eset byFoilT X E

The SAT Problem

Given a set of b o olean clauses

in CNF each containing two

literals nd a satisfying assignment

if one exists

x x x x x x

     

Start with any tentative assignmen t

If there is an unsatised

clause pick one of its two

literals at random and it

Typ eset byFoilT X E



If no solution found in n

steps declare none exists

Monte Carlo If a solution

exists will nd it with probability

If not will always declare none

exists

Typ eset byFoilT X E

Random Walk Analysis

What is the probability of

missing a satisfying assignment



in n steps

Fix any particular satisfying

assignment A

Consider the number of literals

on which the algorithms

tentative assignment agrees

with A

Typ eset byFoilT X E

This is a random walk on the

integers that increases with

probability at least at

each step

Exp ected time to nd an



assignment n

Markovs inequality probability



of missing an assignment in n

steps is

Typ eset byFoilT X E

Binary planar partitions

L2 L1 L1 s1 L2 L3 s 3 sss s s 1 3 3 2 2

L3

Given n nonintersecting line

segments in the plane build a

small linear decision tree that

has pieces of at most one

segment in each cell

Typ eset byFoilT X E

Autopartitions

Cho ose the input segments in

random order

Extend the chosen segment if

necessary to any cells

where it is not alone

The exp ected size of the

resulting tree is n nH

n

Typ eset byFoilT X E

Analysis of autopartition size

v 1

u v

2

Dene l u to be the line

containing segment u

u a v denotes l u cuts v in

the constructed tree

indexu v is i if l u intersects

i other segments before

hitting v

Typ eset byFoilT X E

Autopartitions

Pr u a v indexu v

Thus the exp ected size of the

tree constructed is

X X

n Pr u a v

u

v u

X X

n

indexu v

u

v u

Typ eset byFoilT X E

For any u i indexu v i for

at most two v thus the ab ove

sum is n nH

n

It follows that for any n

segments in the plane there

always exists an autopartition of

size O n log n

Typ eset byFoilT X E

Matrix pro duct verication

Supp ose A B and C are n

n matrices with entries from a

nite eld F We wish to verify

whether AB C

Multiplying out takes ab out



n steps this can be cut to



O n steps using sophisticated

algorithms

Typ eset byFoilT X E

Simple randomized

algorithm

Pick an nvector x with

entries randomly drawn from

F

Compute z ABx

If z Cx output AB C

else output AB C



Takes O n steps

Typ eset byFoilT X E

Simple randomized

algorithm

If AB C will always output

AB C

If AB C will output

AB C with probability at

most jF j

Typ eset byFoilT X E

Sources

RM Karp An intro duction to

randomized algorithms Discrete

Applied Mathematics

R Motwani and P Raghavan

Randomized Algorithms Cambridge

University Press

D J A Welsh Randomised

algorithms Discrete Applied

Mathematics

Typ eset byFoilT X E