CHAPTER

MARKOV

APPLIED TO PERFORMABILITY

EVALUATION

y

Ricardo Fricks Miklos Telek

Center for Adv Comp and Comm Dept of Telecommunications

Dept of Electrical Engineering Technical University of Budap est

Duke University Budap est Hungary

Durham NC

y

Antonio Puliato Kishor Trivedi

Ist di Informatica e Telecom Center for Adv Comp and Comm

Universita di Catania Dept of Electrical Engineering

Catania Italy Duke University

Durham NC

Abstract

Signicant advances have been made in performability modeling and

analysis since the early s In this chapter we present two special

classes of continuous time stochastic processes with embedded Markov

renewal sequences that can besuccessful ly employed for performability

analysis Detailed examples il lustrate the solution techniques surveyed

in the introductory sections of the chapter

y

This work was supp orted in part by an NSF grant EEC Brazils Na

tional Council of Research and Development and a CACC core pro ject funded by

NASA Lewis Research Center

FRICKS TELEK PULIAFITO TRIVEDI

INTRODUCTION

Computer and communication systems are designed to meet a certain

sp ecied b ehavior The pro curement of metrics to establish howwell

the system b ehaves that is how closely it follows the sp ecied b ehav

ior is the ob jective of quantitative analysis Traditionally p erformance

and dep endabilityevaluation are used as separate approaches to pro

vide quantitative gures of system b ehavior Performance evaluates

the quality of service assuming that the system is failurefree De

p endability fo cuses on determining deviation of the actual b ehavior

from the sp ecied b ehavior in the presence of comp onent or subsystem



failures



Beaudry prop osed the aggregated measure computation before



failure while Meyer prop osed the term performabilitywhich has b een

used since then Performability analysis aims to capture the interaction

between the failurerepair b ehavior and the p erformance delivered by

the system Its results are fundamental to the analysis of realtime



system p erformance in the presence of failure

Performability measures provide b etter insightinto the b ehavior

of faulttolerant systems Basic metrics used to evaluate faulttolerant

designs are reliabilityandavailability The conditional probabilitythat

a system survives until some time tgiven it is fully op erational at



t is called the reliability Rt of the system Reliability is used to

describ e systems which are not allowed to fail in which the system is

serving a critical function and cannot b e down Note that comp onents

or subsystems can fail so long as the system do es not The instanta

neous availability At of a system is the probability that the system

is prop erly functioning at time t Availabilityistypically used as a

basis for evaluating systems in which functionality can b e delayed or



denied for short p erio ds without serious consequences Reliability and

availability do not consider dierentlevels of system functionality

Performability analysis of real systems with nondeterministic

comp onents andor environmental characteristics results in sto chastic

mo deling problems Several techniques for solving them for transient

and steadystate measures have b een prop osed and later combined un



der the framework of Markov reward mo dels The traditional frame

work allows the solution of sto chastic problems enjoying the Markov

prop erty the probability of any particular futurebehavior of the pro

cess when its current state is known exactly is not altered by additional

MARKOVRENEWAL THEORY



know ledge concerning its past behavior If the past history of the pro

cess is completely summarized in the current state and is indep endent

of the current time then the pro cess is said to b e time homoge

neous Otherwise the exact characterization of the present state needs

the asso ciated time information and the pro cess is said to b e non

homogeneous A wide range of real problems fall in the class of Markov

mo dels b oth homogeneous and nonhomogeneous but problems in

p erformability analysis havebeenidentied that cannot b e adequately

describ ed in this traditional framework The common characteristic

these problems share is that the Markovproperty is not valid if valid

at all at all time instants This category of problems is jointly re

ferred to as nonMarkovian mo dels and can b e analyzed using several

approaches

Phasetypeexpansions when the past history of the sto chas

tic pro cess can b e describ ed by a discrete variable an expanded

continuoustime homogeneous Markovchain can b e used to cap

ture the sto chastic b ehavior of the original system



Supplementary variables when the past history is describ ed

by one or more continuous variables the approach of the supple

mentary variables can b e applied and a set of ordinary or partial

dierential equations can b e dened together with b oundary con

ditions and analyzed



Embeddedpointprocesses when the temp oral b ehavior of

the system can b e studied by means of some appropriately chosen

embedded epochs where the Markov prop erty applies Several

wellknown classes of sto chastic pro cesses such as regenerative

semiMarkov and Markov regenerative pro cesses are based on the

concept of emb edded p oints

Theobjectofthischapter is to present a theory based on the concept

of emb edded p ointpro cesses that encompass semiMarkov and Markov

regenerative pro cesses This theory named Markov renewal theoryis

reviewed in the rst three sections of this chapter and later applied to

several nonMarkovian p erformability mo dels

Our purp ose is to provide an uptodate treatment of the basic

analytic mo dels to study nonMarkovian systems by means of Markov

renewal theory and an accurate description of the solution algorithms

In particular we develop a general framework which allows us to deal

FRICKS TELEK PULIAFITO TRIVEDI

with renewal pro cesses and sp ecically with semiMarkovandMarkov

regenerative pro cesses Wehopethatthischapter will serve as a refer

ence for practicing engineers researchers and students in p erformance

and reliability mo deling Other surveys on Markov renewal theory ap



plied to reliability analysis have app eared in the literature but none

of them as complete or as didactic as the presentone

The rest of this chapter is organized as follows Section intro

duces the basic terminology asso ciated with the theory including the

concepts and distinction b etween semiMarkov pro cesses and Markov

regenerative pro cesses Section presents basic solution techniques for

sto chastic pro cesses with emb edded Markov renewal sequences Markov

regenerativePetri nets useful as a highlevel description language of

these kind of sto chastic mo dels are reviewed in Section and employed

in the analyses of three examples presented in Section Examples are

selected to illustrate the metho dology asso ciated with semiMarkov and

Markov regenerative pro cesses Section concludes the chapter

MARKOV RENEWAL THEORY

Assume we wish to quantitatively study the b ehavior of a given non

deterministic system One p ossible solution would b e to asso ciate a

Z taking values in a countable set F to describ e

t

the state of the system at any time instant t The family of random

variables Z constitutes a sto chastic pro cess Z fZ t R g

t t

successive occurrences of a recurrent phenomenon

time S = 0 S S 0 1 S2 3 θ θ θ 1 2 3

start of observation

Figure A sample realization of a renewal pro cess

Supp ose we are interested in a single event related with the

system eg when system comp onents fail Additionally assume

MARKOVRENEWAL THEORY

the times b etween successive o ccurrences of this typeofeventare

independent and identical ly distributed iid random variables Let

S S S b e the time instants of successiveevents to o c



cur as shown in Figure The sequence of nonnegative iid random

variables S fS S n N g is a renewal process

n n

Otherwise if we do not start observing the system at the exact moment

an event has o ccurred ie S the sto chastic pro cess S is a delayed

renewal process

Contexts in which renewal pro cesses arise ab ound in applied

probabilityFor instance the times b etween successive electrical im

pulses or signals impinging on a recording device are often assumed to

form a renewal pro cess Another classical example of renewal pro cess is



the item replacement problem explored in where S S S S



represent the lifetimes of items light bulbs machines etc that are

successively placed in service immediately following the failure of the

previous one

However supp ose instead of a single event we observe that cer

tain transitions b etween identiable system states j of a subset E of F

EF also resemble the b ehavior just describ ed when considered in

isolation Successive times S at which a xed state j j Eisentered

n

form a p ossibly delayed renewal pro cess In the sample pro cess real

ization depicted in Figure we see that the sequence of time instants

fS S g forms a renewal pro cess while fS S g and fS S g

   

form delayed renewal pro cesses

Additionally when studying the system evolution weobserve

that at these particular times the sto chastic pro cess Z exhibits the

Markov prop erty ie at anygiven moment S n Nwe can for

n

get the past history of the pro cess In this scenario we are dealing

with a countable collection of renewal pro cesses progressing simultane

ously such that successive renewals form a discretetime Markovchain

DTMC The sup erp osition of all the identied renewal pro cesses gives

the p oints fS n Ngknown as Markov renewal moments and to

n

gether with the states of the DTMC denes a Markov renewal se

quence MRS

In this section we review the denitions and some of the concepts

of Markov renewal theory a collective name that includes MRSs

Note that these instants S are not renewal moments as describ ed in renewal

n

theory since the distributions of the time interval b etween consecutivemoments are not necessarily iids

FRICKS TELEK PULIAFITO TRIVEDI

X

k

j

i

time S = 0 SSS S S 0 12 3 45

start of observation

Figure A set of renewal pro cesses progressing concurrently

and two other imp ortant classes of sto chastic pro cesses with embedded

MRSs named semiMarkov pro cesses SMPs and Markov regenera

tive pro cesses MRGPs Our ultimate aim is to study fZ t R g

t

however as a rst step we need to study Markov renewal theoryOur

emphasis in this chapter is how to explore the p ossibilities of this

wealthy theory rather than its technical details or why do es it work

The denitions and terminology mentioned here were inuenced

 

by but the formalism comes fromCinlar We strongly recom

 

mendCinlar and Kulkarni for a more detailed study of Markov

renewal theory Classical references for the other classes of sto chastic



pro cesses mentioned in this chapter are renewal pro cesses and



regenerative pro cesses For the general theory of Markovchains



go o d references are

Historic Overview of Markov Renewal Theory



SemiMarkov pro cesses were indep endently intro duced byPLevy



and WL Smith in Although Smiths work was only pub

lished in its main results were announced in a talk given on

the authors b ehalf by DV Lindley at the International Congress of

Mathematicians held in Amsterdam in Septemb er At the same

congress Levy announced his results concerning semiMarkovian pro

MARKOVRENEWAL THEORY

cesses whichwere identical with the results given by Smith Also at



the same time L Takacs intro duced and applied the same typ e of

sto chastic pro cess to problems in counter theory SemiMarkov pro

cess is a generalization of b oth continuous and discrete time Markov

chains which p ermit arbitrary so journ distribution functions p ossibly

dep ending b oth on the current state and on the next state to b e entered

The term Markov renewal sequence is due to R Pyke who gave



an extensive treatmentofmany asp ects of such pro cesses in

Markov regenerative pro cesses were intro duced byRPykeandR

Schaufele in where they were called semiMarkov pro cesses with



auxiliary paths Most of the theoretical foundations of Markovregen



erative pro cesses were laid out in the work ofCinlar in under



the name of semiregenerative pro cesses Later Kulkarni suggested

the name Markov regenerative pro cess that we use in this chapter

MarkovRenewal Sequence

Dene for each n N a random variable X taking values in a count

n

able set E and a random variable S taking values in R suchthat

n

S S S assuming S The bivariate sto chastic pro



cess X SfX S n Ng is a Markovrenewal sequence if it

n n

satises

PrfX j S S t j X X S S g

n n n n n

PrfX j S S t j X g

n n n n

for all n N j Eandt R Thus XS is a sp ecial case of

bivariate Markov pro cess in which the increments S S S S



are all nonnegative and are conditionally indep endentgiven X X

We will always assume timehomogeneous MRSs that is the

conditional transition probabilities K t where

ij

K t PrfX j S S t j X ig

ij n n n n

are indep endentofn for any i j E t R Therefore we can always

write

K t PrfX j S t j X ig i j EtR

ij

FRICKS TELEK PULIAFITO TRIVEDI

The matrix of transition probabilities Kt fK ti j Et R g

ij

is called the kernel of the MRS

The sto chastic sequence fX n Ng keeps track of the succes

n

sive states visited at Markov renewal moments and forms a discrete

time Markovchain with state space E The onestep transition proba

bilities of this embedded EMC are

p PrfX j j X ig i j E

ij n n

lim K t

ij

t

There are no restrictions regarding the structure of the EMC on

an MRS There is no imp osition that fX n Ngshould b e irreducible

n

for instance Therefore we can start at time S in a state of E that

will not b e reached again at any other Markov renewal moment in the

future evolution of the pro cess

Let the vector of initial probabilities of X b e describ ed by a

P

a a where i a PrfX ig i Eandii a

i i

iE

then wesay that X SfX S n Ngis an MRS completely

n n

determined bya Kt Thus the vector of initial probabilities a

and the kernel matrix Kt completely determine all nitedimensional



distributions of the Markovrenewal sequence

Emb edded MRSs can b e identied asso ciated with semiMarkov

and Markov regenerative pro cesses Hence wenow pro ceed with a

study of such pro cesses

SemiMarkov Pro cesses

Given an MRS X S with state space E and kernel Kt wecan

intro duce the counting pro cess

Nt sup fn S tg t R

n

to countthenumber of Markov renewal moments up to time tbutnot

considering the one at zero Using the counting pro cess just dened

weintro duce the pro cess Y fY t R g dened by

t

Note that K t is a p ossibly defective distribution function so that

ij

lim K t

t ij

MARKOVRENEWAL THEORY

Y X

t

N t

X if S t S

n n n

for all t R called semiMarkov pro cess SMP determined by

a Kt An SMP for a sample realization see Figure is a sto chastic

pro cess whichmoves from one state to another within a countable num

b er of states with the successive states visited forming a discretetime

Markovchain and that the time required for each successivemoveis

a random variable whose distribution function may dep end on the two

states b etween which the move is b eing made The nomenclature semi

Markov comes from the somewhat limited Markov prop erty which Y

has the future of Y is indep endent of its past provided the present



is a Markov renewal moment Note that since we consider S

then the initial condition Y i always means that the SMP has just

entered state i at the time origin Like MRSs an SMP is sp ecied by

its vector of initial probabilities a and the kernel matrix Kt

Y

k

j

i

time S = 0 SSS S S

0 12 3 45

Figure A sample realization of a semiMarkov pro cess

From the SMP denition it should b e observed that the pro cess

only changes state p ossibly back to the same state as shown in Figure

at the Markov renewal moments S The p ossibility of transitions

n

not resulting in real state changes can b e easily veried by insp ect

ing the kernel matrix Kt for nonzero elements on its main diago

nal It follows that the matrix Kt asso ciated with a particular SMP

as well as to an MRS is not necessarily unique There are SMPs

as well as MRSs which can b e describ ed by more than one matrix

FRICKS TELEK PULIAFITO TRIVEDI

Kt A unique kernel matrix can always b e dened by its minimal

representation in which all Markovrenewal moments represent real

state transitions For instance the minimal representation of the SMP

depicted in Figure would preventMarkov renewal moment S and



consequently the time interval b etween consecutive Markov renewal

moments would always represent the so journ time in eachoftheEMC

states The derivation of the minimal representation of Ktisintro



duced and explored in

SMPs represent suciently general and constructive mathemat

ical mo dels of complex multicomponent systems whose states are ran

domly changed by extreme conditions The only essential restriction

is the semiMarkov prop erty that can b e interpreted from twodierent

p ersp ectives actually used to simulate the b ehavior of SMPs

The sequence of system states at times of changes must b e de

scrib ed by a homogeneous DTMC and times to the next Markov

renewal moment only dep end on the current state and next state

of the system

The times to the next Markov renewal moment only dep end on

the current state and the selection of the destination state only

dep end on the current one and the time when the state transition

happ ens

In the subsequent derivations of this pap er we consider an ex

tension of SMPs obtained by attaching rewardrates r to their states

i

i E Having intro duced these new variables we can then compute

the reward accumulated by Y over anyinterval t The accumulated

reward B t is dened by the following integral

Z

t

B t r d

Y

Several imp ortant measures of Y can b e dened by this extension such

as the cummulative time Y sp ends in a subset of states etc These

measures are refered to as reward measures of the SMPandcanbe

characterized bya Kt r where r is the vector of the reward rates

Details on this and other analytical results asso ciated with the reward



concept can b e found in

We call these conditions extreme b ecause we assume that after the condition

app ears or is detected the system state will change instantaneously

MARKOVRENEWAL THEORY

Markov Regenerative Pro cesses

A sto chastic pro cess Z fZ t R g with state space F is called

t

regenerative if there exist time p oints at which the pro cess probabilisti

cally restarts itself Such random times when the future of Z b ecomes

a probabilistic replica of itself are named times of regeneration for Z

This concept maybeweakened by letting the future after a time of

regeneration dep end also on the state of an MRS at that time We

then saythat Z isaMarkov regenerative pro cess

MRGPs are sto chastic pro cesses fZ t R g that exhibit em

t

b edded MRSs XS with the additional prop erty that all conditional

nite distributions of fZ t R g given fZ u S X ig

t S u n n

n



are the same as those of fZ t R g given X i As a sp ecial case

t

the denition implies that

PrfZ j j Z u S X ig PrfZ j j X ig

t S u n n t

n

It also implies that the future of the pro cess fZ t R g from

t

t S onwards dep ends on the past fZ u S g only through X

n u n n

Observe that in the regenerative pro cess this future from S onwards

n

was completely indep endent of the past

Z

l

k

j

i

time S = 0 SSS S S

0 12 3 45

Figure A sample realization of a Markov regenerative pro cess

In contrast to SMPs state changes p ossibly to states outside

E are allowed b etween two consecutiveMarkov renewal moments see

FRICKS TELEK PULIAFITO TRIVEDI

Figure in MRGPs It is p ossible for the system to return to states

in E without these moments constituting Markov renewal moments

For example supp ose we start observing the system when it has just

entered a state j as shown in Figure At that particular instant

the Markov prop erty is applicable since there is no past history of the

pro cess but b ecause of system characteristics weknow this prop erty

will no longer b e valid for that state after the rst state transition not

necessarily to a state in E This situation could b e understo o d if we

consider that although state j beingpartoftheEMCXitdoesnot

communicate with other states of X and hence the Markovchain X

is reducible Although others states of X are p ossibly accessible from

state i this state cannot b e accessed from any other state of X

The structural complexityofanMRGP dep ends on twomain

constituents

the sto chastic pro cess b etween two consecutiveMarkov renewal

moments

the cause of the o ccurrence of a Markov renewal moment

The sto chastic pro cess b etween the consecutiveMarkov renewal

moments usually refered to as subordinatedprocesscanbeanycontinuous

time discretestate sto chastic pro cess over the same probability space



Recently published examples considered sub ordinated CTMCs

 

SMPs MRGPs or a more general sto chastic reward pro cess

The o ccurrence of a Markov renewal moment can b e caused by

the expiration of a random delay

indep endent of the sub ordinated pro cess

some preceeding state transition of the sub ordinated pro cess

results in a new Markov reneval moment

an acummulated reward measure of the sub ordinated pro cess

reaches a random barrier

indep endent of the path of the sub ordinated pro cess

While the preemption p olicy of the reward pro cess preemptive repeat with re

sampling can b e considered by means of one of the rst reason as well other



preemption p olicies preemptive resume or preemptive repeat without resampling



can result in a more complex situation which falls only into reason

MARKOVRENEWAL THEORY

some preceeding state transitions of the sub ordinated pro

cess results in new Markov reneval monent

y

other more complex reason

The sub class of MRGPs with sub ordinated SMPs or CTMCs

and with Markov renewal moments caused by one of the rst tworea



sons is considered in

PROBLEM SOLVING USING MARKOVRENEWAL

THEORY

Let Z fZ t R g b e an MRGP with state space F whose emb ed

t

ded MRS is X SfX S n Ng with kernel matrix Ktover a

n n

countable state space E a subset of F ie EF For such a pro cess

we can dene a matrix of conditional transition probabilities as

V t PrfZ j j Z ig i E j F t R

ij t

In many practical problems involving Markov renewal pro cesses

our primary concern is nding ways to eectively compute V tsince

ij

several measures of interest eg reliabilityandavailability are related

to the conditional transition probabilities of the sto chastic pro cess

In this section we review some of the main techniques to deter

mine transition probabilities The underlying pro cess discussed is an

MRGP

MarkovRenewal Equation

Atany instant t the conditional transition probabilities V tofZ can

ij

 

b e computed as

V t PrfZ j S tj Z ig PrfZ j S t j Z ig

ij t t

PrfZ j S tj Z ig

t

Z

t

X

PrfZ j j Z k gdPrfZ k S ug

tu u

u

k E

Z

t

X

V t udK u PrfZ j S tj Z ig

kj ik t

k E

y

This last reason results in the wides class of MRGPs but we b elievethatthema

jority of the practically interesting cases are captured by one of the former reasons

FRICKS TELEK PULIAFITO TRIVEDI

Z

t

X

PrfZ j S tj Z ig dK uV t u

t ik kj

k E

for all i E j F and t R Ifwe dene matrix Etby

E t PrfZ j S tj Z ig i E j F t R

ij t

then the set of integral equations V tdenesa Markov renewal

ij

equation and can b e expressed in matrix form as

Z

t

V tEt dKuV t u

where the Leb esgueStieltjes integral is taken term by term

To b etter distinguish the roles of matrices Etand Ktin

the description of the MRGP we use the following terminology when

referring to them

We call matrix Et the lo cal kernel of the MRGP since it

describ es the state probabilities of the pro cess during the interval

between successiveMarkov renewal moments

Since matrix Kt describ es the evolution of the pro cess from the

Markov renewal moment p ersp ective without describing what

happ ens in b etween these moments wecallittheglobal kernel

of the MRGP

The Markov renewal equation represents a set of coupled Volterra



integral equations of the second kind and in general are hard to solve

in timedomain The research for eetivenumerical solution metho ds

of this equation has only started recently We b elievethatthebest

numerical approach should b e based on sp ecic features of MRGPs

which are not necessarily captured by the kernel matrices Kt and

Et However metho ds based on the kernel matrices are the most

general so far and will b e the ones exp osed in this chapter

To summarize solving problems using Markov renewal theory

is a two step pro cess

R R

t t

dK uV t u k uV t udu when K t p ossesses a density function

 

dK t

k t

dt

Actually this terminology will b e adopted even when discussing issues related

with SMPs for consistency in our presentation

MARKOVRENEWAL THEORY

y

First we need to construct b oth kernel matrices KtandEt

We then solve the set of Volterra integral equations for the con

ditional transition probabilities V t or for some measure of in

ij

terest

Synthesis of the Kernel Matrices

The construction of kernel matrices can pro ceed by reasoning from par

ticular facts to a general conclusion inductive approach or reasoning

from the general to the sp ecic deductive approach The inductive

approach starts from the analysis of p ossible state transitions and re

lies only on basic to construct b oth kernel matrices

Conversely the deductive approach applies general techniques to solve

for a particular case It is hard compare in general the two alternative

approaches for a particular problem under consideration Only exp e

rience can help in selecting the most suitable technique Hence we

illustrate b oth approaches whenever p ossible in the examples in this

chapter without discussing their particular merits

The inductive approach do es not haveany general formulation

Its application varies from case to case though the contructive pro cess

of the matrices follows approximately a regular pattern We illustrate

this logical pattern stepbystep along the solution of the examples

presented in the end of this chapter Our main interest in this section

is to explore the deductive approach b ecause of its algorithmic nature

The deductive approachprovides a closed form expression in

transform domain for the elements of matrices Etand Ktbased

on the kernels of the sub ordinated pro cesses Due to the generalityof

this approach it provides a robust and widely applicable pro cedure to

construct the kernel matrices and that is whywe use it in the analysis

of subsequent examples as a deductive metho d with comparison of the

inductiveone

Since the rst reason of the o ccurrence of Markov renewal mo

ments can b e considered as the sp ecial case of the second reason a uni



ed approachwas intro duced in to analyze the sub class of MRGPs

i

characterized by the rst two reasons Three matrix functions F t w

i i

D t w and P t w where t denotes the time w denotes the barrier

level and the sup erscript i refers to the initial regeneration state of

y

For the case of SMPs only the global kernel matrix Kt is necessary

FRICKS TELEK PULIAFITO TRIVEDI

the sub ordinated pro cess were intro duced to quantify the dierentoc

casions of the completion of the regeneration p erio d and the internal

i

state probabilities with a xed barrier height F t w refers to the case

when the next regeneration moment is b ecause of the accumulated re

ward measure reached the xed value w of the barrier For the analysis

i

of this case an additional matrix referred to as branching prob

ability matrix is intro duced as well to describ e the state transition

i

subsequent to the regeneration moment D t w captures the case

when the next regeneration momentiscausedby one of the concluding

i

state transitions of the sub ordinated pro cess And P t w describ es

the state transition probabilities inside the regeneration p erio d

i i

Based on the kernel of the sub ordinated SMP Q t fQ tg

k



these functions can b e evaluated by the following equations

i

r Q s vr

k k

k

i

F s v

k

k

s vr

k

X

i i

Q s vr F s v

k

ku u

uR

X

i i i i

s v D s v s vr D s vr Q Q

k k

k u ku kl

v

uR

i

s Q s vr

k

k

i

P s v

k

k

v s vr

k

X

i i

Q s vr P s v

k

ku u

uR

P

i i

where Q t Q t s is the time variable and v is the barrier

k k

level variable in transform domain r is the reward rate asso ciated to

k

state k R is the part of the state space from which an exit results in a

new regeneration moment and the sup erscript refers to Laplace

Stieltjes Laplace transformation

Given that G w is the cumulative distribution function of the

g

random barrier height to reach the next regenerativemoment the el

ements of the ith row of matrices Ktand Et can b e expressed as

i i i

follows as a function of the matrices P t w F t w and D t w

This subsection summarizes the results only for the cases when the cummulative

or reward measure is accumulated according to prd and prs mo dels for pri mo dels



we refer to

MARKOVRENEWAL THEORY

Z

X

i i i

K t F t w D dG w t w

ij g

ik kj ij

i

k R

Z

i

t w dG w E t P

g ij

ij

Solution Techniques of MarkovRenewal Equations

We can classify the existent solution metho ds in two categories



time domain metho ds



LaplaceStieltjes domain metho d

One p ossible time domain solution is based on a discretization

approachtonumerically evaluate the integrals presented in the Markov

renewal equation The integrals are solved using some approximation

rule such as trap ezoidal rule Simpsons rule or other higher order meth

ods

n

X

V t Et a K t V t t

n n i i n i

i

dK x

where K t denotes the derivative evaluated at p oint t In

i i

dx

these equations h is the discretization step and it is assumed constant

and the co ecients a dep end on the integration technique used For

i

h

example when the trap ezoidal rule is used a a and a h i

n i



n Hence at anygiven time t t nh a linear system

n

of the form

n

X

I a K V t Et a K t V t t

n n i i n i

i

needs to b e solved Note that if a K is a diagonal matrix then the

metho d is explicit otherwise it is implicit

A p otential problem with this approachisthattherighthand

side of the ab ove equation can in general b e exp ensive to compute

Nevertheless there exist cases where the generalized Markovrenewal

When the derivative of matrix Kt is dicult to obtain Equation can b e ap

P

n

proximated as V t Et K t K t V t t

n n i i n i

i

FRICKS TELEK PULIAFITO TRIVEDI

equation has a simple form and the timedomain solution can b e carried

out

Another time domain alternative is to construct a system of par

tial dierential equations PDEs using the metho d of supplementary



variables This metho d has b een considered for steadystate analysis

 

in and subsequently extended to the transient case in Uptonow

this metho d has b een elab orated only for the cases when the o ccurrence

of a new Markov renewal moment is due to one of the rst two causes

discussed in the previous section

An alternative to the direct solution of the Markovrenewal equa

tion in timedomain is the use of transform metho ds In particular if we

R R

st st

dene E s e dEtandV s e dV t the Markov

renewal equation b ecomes

V s E sK sV s

I K s E s

After solving the linear system for V s transform inversion is

required In very simple cases a closedform inversion might b e p ossible

but in most cases of interest numerical inversion will b e necessary

The transform inversion however can encounter numerical diculties

esp ecially if V s has p oles in the p ositive half of the complex plane

MARKOV REGENERATIVE STOCHASTIC PETRI

NETS

  

Sto chastic Petri nets of various typ es SPN GSPN ESPN



DSPN etc have b een prop osed as mo del description languages

for analyzing the p erformance and reliability of systems The analyti

calnumerical solution of such mo dels pro ceeds by utilizing mathemati

cal engines based up on the underlying sto chastic pro cesses of eachPetri

net class CTMC for SPNs and GSPNs SMP for a subset of ESPNs

and MRGP for DSPNs In this chapter we use the class of sto chas

tic Petri net named MRSPN to describ e and help solve the sample

problems explored in the next section

Markov Regenerative Sto chastic Petri Nets MRSPNs



to overcome limitations on mo deling p ower no were intro duced in

tably allowing the solution of nonmarkovian mo dels of existing analyt

MARKOVRENEWAL THEORY

ical to ols MRSPNs allow transitions with zero ring times exp onen

tially distributed or generaly distributed ring times The underlying



sto chastic pro cess of an MRSPN is an MRGP and it was proved in

that MRSPNs constitutes a true generalization of all the ab ove classes

With a restriction that at most one generally distributed timed transi

tion is enabled in each marking the transient and steady state analysis

of MRSPNs can b e carried out analyticallynumerically rather than by

discreteeventsimulation Wenow present some of the basic concepts

concerning MRSPNs but to do that we review some of the classical

terminology of Petri nets



A Petri net PN is dened by a set of places drawn as

circles a set of transitions drawn as bars and a set of directedarcs

which connect transitions to places or places to transitions Places may

contain tokens The state of the Petri net called marking is dened

byavector enumerating the number of tokensineach place

The states of a PN can b e used to representvarious entities

asso ciated with a system for example the numb er of functioning re

sources of eachtyp e the numb er of tasks of eachtyp e waiting at a

resource the allo cations of resources to tasks and states of recovery

for each failed resource Transitions representthechanges of states due

to the o currences of simple or comp ound events such as the failure of

one or more resources the completion of executing tasks or the arrival

of jobs

A place is an input to a transition if an arc exists from the place

to the transition If an arc exists from the a transition to a place it

is an output place of the transition A transition is enabled when each

of its input places contains at least one token Enabled transitions

can re by removing one token from each of input place and placing

one token in each output place Thus the ring of a transition may

cause a change of state pro ducing a dierent marking of the PN The

reachability set is the set of markings that are reachable from a given

initial marking The reachability set together with arcs joining the

marking indicating the transition that cause the change in marking is

called the reachabilitygraphof the net

After the original conception some extensions of PNs were pro

p osed Inhibitor arcs were intro duced to increase the fundamental mo d

eling or decision p ower of ordinary Petri nets An inhibitor arc from

a place to a transition has a circle rather than an arrowhead at the

transition The ring rule for the transition is changed such that the

FRICKS TELEK PULIAFITO TRIVEDI

transition is disabled if there is at least one token present in the corre

sp onding inhibiting input place

In sto chastic Petri nets a random ring time elapses after a

transition is enabled until it res Transitions whichhave nonzero ring

times are called timedtransitions and transitions with zero ring times

are called immediate transitions In MRSPN a timed transition can res

according to an exp onential or any other general distribution function

Immediate transitions have priority to re over timed transitions

The markings of a sto chastic Petri net can b e classied into

vanishing markings and tangible markingsInavanishing marking at

least one immediate transition is enabled and in a tangible marking

no immediate transition is enabled For sto chastic Petri nets classes

that avoid immediate transitions the analysis of the emb edded sto chas

tic pro cess can start directly from the reachability graph For instance

the reachability graph of an SPN can b e mapp ed directly into a Markov



chain and then solved for transient and steadystate measures How

ever b efore analysing the underlying sto chastic pro cess of an MRSPN

wehave an extra step after obtaining the reachability graph The re

duced reachabilitygraphis obtained from the reachability graph

by merging the vanishing markings into their successor tangible mark



ings according some rules After constructing the reduced reachability

graph we can start the analysis of the underlying sto chastic pro cess

which is going to b e explored in the next section together with the

develop ed examples

PERFORMABILITY ANALYSIS APPLYING

MARKOV RENEWAL THEORY

The use of Markovrenewal theory for p erformabilityevaluation will b e

shown by its application to three examples of computer system archi

tectures

series system with repair

parallel system with single shared repair facility and

warm standby system with single shared repair facility

We start this section by describing each of the sample cases fol

lowed by the identication of underlying sto chastic pro cesses and con

cluding with the construction of kernel matrices using b oth approaches

MARKOVRENEWAL THEORY

discussed whenever applicable Finally the section is closed with the

numerical solution of the resulting Markovrenewal equations for mea

sures of interest Our emphasis in this chapter is on synthesis of the

kernel matrices rather than on solution of the Volterra equations

Series System with Repair

Consider a series system comp osed of twomachines a and b with con

stant failures rates and Up on failure of either machine the

a b

system fails and is repaired with general repairtime distribution func

tions G tand G t We assume that machines cannot fail while the

a b

system is down and that failure of one machine do es not aect the op erational status of the other

1: 1,1,0,0 P1 P2 f a f b

2:0,1,1,0 3: 1,0,0,1 rfa a f b rb ra rb

1 1

(b) P3 P4 f f (a) a b 231

ra rb

(c)

Figure a Petri net of series system b Reachability graph c

State transition diagram

The overall b ehavior of the system can b e easily understo o d

from the MRSPN illustrated in Figure a Machine a is working

whenever there is a token in place P Transition f ring according

a

to an exp onential distribution with parameter represents the failure

a

pro cess of machine a When machine a fails a token is dep osited in

place P and repair is immediately started Transition r with a gen

 a

erally distributed ring function G t represents the random duration

a

of the repair pro cedure A symmetrical set of places and transitions

describ es the b ehavior of machine b The system has failed whenever

FRICKS TELEK PULIAFITO TRIVEDI

atoken is dep osited in places P or P Thetwo inhibitor arcs imp ose

 

the restriction that no machine can fail while the system is undergoing

repairs

The reachability graph corresp onding to the Petri net is shown

in Figure b Each marking in the graph is a tuple counting the

numb er of tokens in places P to P Figure b also corresp onds to the



reduced reachability graph for the system since there are no vanishing

markings In the graph solid arcs represent transitions ring according

to exp onential distribution functions while dotted arcs denote state

transitions ring according to general distributions

Let a random variable Y b e dened according to the op erational

t

condition of the system at anyinstant ie

if the sy stem is w or k ing at time t

Y if machine a is being r epair ed at time t

t

if machine b is being r epair ed at time t

Note that p ossible values of Y are the lab els corresp onding

t

markings in Figure b We are interested in computing p erformability

measures asso ciated with the system Todosowe need to determine

the conditional transition probabilities V tof fY t R g

ij t

We start the solution pro cedure with the identication of the

typ e of sto chastic pro cess underlying the system b ehavior From the

reachability graph we can conclude that all state transitions corresp ond

to Markov renewal moments S fS n Ng and consequently

n

markings lab eled and dene the emb edded Markovchain X

fX n Ng such that X is the state of the system at time S ie

n n

n

X Y

n S

n

If we assume that at the time origin the system has just entered

a new state ie S then the bivariate sto chastic pro cess XS is

an MRS and consequently fY t R g is an SMP since whenever the

t

system changes state weidentify a Markov renewal moment Having

identied Y we prepare for its solution by constructing the kernel

t

matrices Ktand Et Note that since we are dealing with an SMP

the construction of Et is immediate once Kt is determined

An additional step adopted b efore starting the construction of

the kernel matrices was the construction of a simplied state transition

diagram Figure c shows a simplied version of the reduced reach

ability graph where the markings were replaced by the corresp onding

MARKOVRENEWAL THEORY

state indices We preserved the convention for the arcs and extended

the notation by representing states of the EMC by circles and even

tually others states by squares

The only nonnull elements in matrix Kt corresp ond to the

p ossible transitions in a single step Consequentlywehave the follow

ing structure of the matrix

K t K t

 

K t

Kt



K t



The elements of the matrix can b e constructed by induction

from Figure c

K t PrfX S t j X g



Prfthe sy stem f ail up to time t and machine

a is the causeg

h i

a

t

a

b

e

a b

K t PrfX S t j X g



Prfthe sy stem f ail up to time t and machine

b is the causeg

h i

b

t

a

b

e

a b

K t PrfX S t j X g



Prfr epair of machine a has compl eted by time tg

G t

a

K t PrfX S t j X g



Prfr epair of machine b has compl eted by time tg

G t

b

The pro cedure to construct K tand K tdeserves some

 

further explanation Since the reasoning is similar for b oth elements

FRICKS TELEK PULIAFITO TRIVEDI

we only detail the determination of K t Let the random variables



L and L b e the resp ective timetofailure of the twomachines wecan

a b

compute K tinthefollowing way



K t PrfX S t j X g



Prfthe sy stem f ail up to time t and machine a

is the causeg

PrfL t L L g

a b a

Z

o i n h

t

a

b

d e e

Z

t

a

b

e e d

a

h i

a

t

a

b

e

a b

We can then write the global kernel matrix as

h i h i

t t

a

b

a a

b b

e e

a a

b b

Kt

G t

a

G t

b

We construct the lo cal kernel matrix Et following a similar

inductive pro cedure In this case we are lo oking for the probabilitythat

the system will remain in a given state up to the next Markov renewal

moment This happ ens since fY t R g is an SMP and there are

t

no nonnull elements in the main diagonal of matrix Kt Therefore

elements of Et are the complementary so journ distribution functions

in each state

t

a

b

e

Et G t

a

G t

b

We can always verify our answers by summing the elements in

eachrowofbothkernel matrices Corresp onding rowsums of the two

matrices must add to unity condition that is easily veried to hold in

the example Having completed the construction of the kernel matrices

we can solve a Markov renewal equation for the transient distribution

of Y t

MARKOVRENEWAL THEORY

Parallel System with Single Repair Facility

Two machines a and b are working in a parallel conguration sharing

a single repair facilityworkingonanFCFS schedule As in the previous

case we assume that b oth machines have exp onential lifetime distri

butions with parameters and resp ectively Whenever one of the

a b

machines fails it go es immediately to repair unless the other machine

is still undergoing repair When the repair facility is busy and a second

failure o ccurs the second machine to fail waits in a repair queue until

the rst machine is put backinto service

1: 1,1,0,0,1,0,0

P1 P2 f a f b r r a b 6: 0,1,1,0,1,0,0 7: 1,0,0,1,1,0,0 P5 P P i a i b f a 6 7 f b 2: 0,1,0,0,0,1,0 3: 1,0,0,0,0,0,1

i a i b ra rb P3 P4 f 1 b 1 f a

(a) 4:0,0,0,1,0,1,0 5: 0,0,1,0,0,0,1 r b 5 f a ra rb f a f b 76 2 1 3 (b) ra rb

f 4 b ra

(c)

Figure a Petri net of parallel system b Reachability graph c

State transition diagram

The pro cedure describ ed in the last section can b e rep eated

to solve this case Figure a presents an appropriate MRSPN to

describ e the system b ehavior and the corresp onding reachabilitygraph

is repro duced in Figure b We observe in Figure a the same typ e

of symmetry present in the series system A token in place P represents



the availability of the single repair facility Other extra places P and



P have b een added to mo del the capture of the repair facilityby the



rst machine to fail We asso ciate immediate transitions i and i to

a b

mo del the capture of the repair facility since we assume that starting

a repair takes no time if the facilityisavailable All the other places

FRICKS TELEK PULIAFITO TRIVEDI

and transitions preserve the same semantics as in the series system

Markings in Figure b are tuples due to the addition of the

three extra places Another distinction from the previous system is that

vanishing markings enclosed by dashed ellipses in the diagram also

o ccur in the reachability graph These markings are eliminated when

the reduced reachability graph is constructed not shown and based

on the reduced version we constructed the state transition diagram of

Figure c

Dene the sto chastic pro cess Z fZ t R g to represent the

t

system state at anyinstant where

if both machines ar e w or k ing at time t

if machine a is under r epair w hil e machine b is

working at time t

if machine b is under r epair w hil e machine a is

working at time t

Z

t

if machine a is under r epair w hil e machine b is

w aiting f or r epair at time t

if machine b is under r epair w hil e machine a is

w aiting f or r epair at time t

Analysis of the resultant reduced reachability graph shows that

Z is an MRGP with an emb edded Markovchain dened by the states

and We can observe that transitions to states and do

not corresp ond to Markov renewal moments b ecause they o ccur while

timed transitions ring according nonexp onential distributions are en

abled Furthermore states and do not b elong to the state space of

the DTMC emb edded in the pro cess EMC therefore they are repre

sented by squares in the state transition diagram to show this particular

condition

What makes this example particularly interesting is the fact

that it allows us to demonstrate b oth techniques for the synthesis of

the kernel matrices the inductive and the deductive

Inductive approach Once identied as an MRGP to nd the

distribution of Z we need to construct the kernel matrices Starting

with matrix Kt we can identify its structure directly from Figure c

MARKOVRENEWAL THEORY

K t K t

 

K t K t

Kt

 

K t K t

 

The elements K tand K t are computed in a similar pro

 

cedure as the corresp ondent elements in the series system case Ad

ditionaly since determination of elements K t and K t is quite

 

alike so we will only show the pro cedure to construct K t The



third row is completelly symmetrical to the second so it can b e easily

undesto o d once K t is understo o d



We need some auxiliary variables to help in the explanation

of the construction pro cess of K t Hence we dene the random



variables R and R to represent times necessary to repair machine a

a b

and b The distribution function of R R isG G Using this new

a b a b

variables we can compute K t



K t PrfX S t j X g



Prfr epair of a is f inished up to time t and b

has not f ail ed dur ing the r epair of ag

PrfR t L R g

a b a

Z

t

PrfL gdG

b a

Z

h i

t

b

e dG

a

Z

t

b

e dG

a

The global kernel matrix inducted Ktis

h h i i

t t

a

a b a

b b

e e

a a

b b

R R

t t

b b

e dG e dG

a a

R R

t t

a a

e dG e dG

b b

Note that the global matrix is and always is going to b e a

square matrix In this case with dimensions since wehave states

in the emb edded Markovchain However the lo cal kernel matrix not

necessary is a square matrix since the cardinality of the state space of

FRICKS TELEK PULIAFITO TRIVEDI

Z can b e larger than the cardinality of the state space of the embedded

Markovchain This can b e seen for instance in this system since

the emb edded Markovchain has only states while the system has

p ossible states

The construction of the lo cal kernel matrix for an MRGP re

quires a more elab orate thinking pro cess than for an SMP This hap

p ens b ecause as explained an MRGP can change states b etween two

consecutive Markov renewal moments and we need to capture these

changes through the E matrix Careful analysis of Figure c reveals

the structure of the lo cal kernel matrix

E t

E t E t

Et

 

E t E t

 

E should b e similar to the series systems since the system

can only go from state to the other two states of the EMC exactly as

in the previous case The diculty comes with the induction of E t



and E t complementof E t Once we solve for these wehave

 

the solution for the remaining comp onents of the matrix due to the

symetry of the problem Therefore we explain the induction pro cess

that leads to E t



E t PrfZ S tj X g

 t

Prfr epair of a is not f inished up to time t and b

has not f ail ed until tg

Prfr epair of a is not f inished up to time tg

Prfb has not f ail ed until tg

t

b

G te

a

We can now express the lo cal kernel matrix as

h i

Et E t E t

 where

MARKOVRENEWAL THEORY

t

a

b

e

t c

b

e G t E t

a

t c

a

e G t

b

t c

b

e G t

E t

a



t c

a

e G t

b

c

G t G t

a

a

c

G t G t

b

b

Both matrices can b e veried using the pro cedure describ ed in

the end of the previous section Once again weverify that the answers

attained satisfy the necessary requirement

Deductive approach With reference to the ab ove discussion

here we only evaluate the elementofthekernel matrices related to

the regenerative p erio d starting from state since there is no state

transition during the sub ordinated pro cess starting form state and

the sub ordinated starting from state is symmetrical to the studied

one

The sub ordinated pro cess starting from state is a CTMC in

deed but to emphasize the generality of this approachwe considered

it as a SMP over the reduced state space of the sub ordinated pro cess

ie state R and Rwithkernel Qt

t

b

b

e

s

b

Qt and Q s

Since states and b elongs to Rie the next regeneration

can not b e caused by a state transition out of R the matrix function

Dtw do es not play role ie D t w for i j E in this case

ij

For notational convenience we neglect the sup erscript refer to the initial re

generation state in the subsequent derivations

FRICKS TELEK PULIAFITO TRIVEDI

By applying equations and wehave

s v P s v sv P s v

b

 

s v P s v P s v P s v

b b

  

s v sv s v P



sv

wehave Since P s v



s v

sv

s v P



s v

b

sv

b

s v P



s v s v

b

The same steps results

s v F



s v

b

b

F s v



s v s v

b

th

In time domain the sum of a the i rowofF plus the sum of

th th

a the i rowofD plus the sum of a the i rowofP has to equal

to one for all t and w In this double LaplaceStieltjes and Laplace

transform domain the same sum has to equal to v which holds for

P s v P s v F s v F s v

   

To implement equations and a symbolic inverse Laplace

transformation is necessary with resp ect to v Whenever the sub ordi

nated pro cess is a CTMC with binary reward rates this step can b e



p erformed symb olically

The inverse Laplace transformation with resp ect to v results in

s

s w

b

e P s w



s b

MARKOVRENEWAL THEORY

s

b

sw s w

b

e e P s w



s s

b b

s w

b

F s w e



sw s w

b

F s w e e



Hence by and the nonzero kernel elements are

Z

E s P s w dG w

a

 

w

s

G s

b

a

s

b

Z

E s P s w dG w

a

 

w

s

b

G G s s

b

a a

s s

b b

Z

K s F s w dG w

a

 

w

G s

b

a

Z

F s w dG w K s

a

 

w

G s G s

b

a a

The of the relevantentries of Et and Kt

reached by the inductive metho d results in the same expressions Fi

nally the LST domain description of the kernel matrices are

b a

s s

a a

b b

K s

s s G s G G

b b

a a a

G s G s G s

a a

b b b

h i

E s E s E s

where

FRICKS TELEK PULIAFITO TRIVEDI

s

s

a

b

E s E s



s

G s

a

b

s

a

and

E s

s E



s

a

G s G s

a

b b

s s

a a

Warm Standby with Single Repair Facility

Two statistically identical machines X and Y are working in a

warm standby conguration sharing a single repair facilityworking on

an FCFS schedule If b oth machines are available one of the is active

online while the other one is active oline spare Machine X is the

active one and Y the spare on the initial condition of the system An

active machine has constant while the machine acting

a

as spare has constant failure rate usually If the online

b a b

oline machine fails its repair b egins immediately and completely

restores it during a random repair time having an arbitrary distribution

function G G and the other comp onentcontinues to work online

a b

We assume that the switchover from the active to the spare machine

takes no time Similarly to the parallel case when the repair facility

is busy and a second failure o ccurs the actions on the last machine to

fail are p ostp oned until the other machine is put backinto service

The MRSPN on Figure a describ es the exp ected system b e

havior We can then construct the corresp ondent reachability graph

which is repro duced in Figure b Atoken in place P represents

that the machine working as active is in op erational condition When

the activemachine fails a token is placed in P and then repairs b egin



immediately in the failed machine while a switchover to the spare ma

chine is executed A token in place P indicates that the spare machine



is available for switchover but since we assume that the spare machine

can also fail we included place P to capture this situation Place



P has a token whenever a switchover has just o curred and the failed



machine previous activemachine is undergoing repairs This place

has b een included to capture the dierence in distribution functions

asso ciated with repair of active and spare machines

MARKOVRENEWAL THEORY

1: 1,1,0,0,0 P1 P2 f a f b

6: 0,1,1,0,0 3: 1,0,0,1,0

f a f b rb ra i rb

2: 1,0,0,0,1 1 f a P3 P4 ra 5: 0,0,1,1,0 i P5 f 1 a rb (a) 4: 0,0,1,0,1 6

ra f a f a 6 1 2 4 ra r (b) f b rb rb a

3 5 f a

(c)

Figure a Petri net of warm standby system b Reachability graph

c State transition diagram

Similarly to the previous case we observe the o ccurrence of van

ishing markings enclosed by dashed ellipses in the diagram This

marking o ccurs b ecause of the zero time switchover and is eliminated

when the reduced reachability graph is constructed Based on the re

duced reachability graph we constructed the state transition diagram

of Figure c

Dene the random sequence Z fZ t R g to represent the

t

system state at any instant where

if activ e and spar e ar e w or k ing at time t

if pr ev ious activ e is under r epair w hil e spar e is

working at time t

if spar e is under r epair w hil e activ e is w or k ing

at time t

Z

t

if activ e has f ail ed w hil e the pr ev ious activ e is

under r epair at time t

if activ e has f ail ed w hil e the spar e is under r epair

at time t

FRICKS TELEK PULIAFITO TRIVEDI

Analysis of the resultant reduced reachability graph shows that

Z is an MRGP with an emb edded Markovchain dened by the states

and We can observe that transitions to states and do not cor

resp ond to Markov renewal moments therefore they were represented

by squares in the state transition diagram Markings

and states and in the transition diagram represent

the failure states of the system one machine has failed when the other

is still undergoing repair

Once identied as an MRGPwe need to construct the kernel

matrices to nd the transition probability distributions of Z Starting

with matrix Kt we can identify its structure directly from Figure

c

K t K t

 

Kt K t K t

 

K t K t

 

The computation of all the nonzero elements of the global kernel

matrix pro ceeds similarly to the series and parallel system Therefore

we can induce the following global kernel matrix Kt

h h i i

t t

a

b

a a

b b

e e

a a

b b

R R

t t

a a

e dG e dG

a a

R R

t t

a a

e dG e dG

b b

Likewise we can express the lo cal kernel matrix as

h i

Et E t E t



where

t

a

b

e

t c

a

E t e G t

a

t c

a

e G t

b

t c

a

e G t

E t

a



t c

a

e G t b

MARKOVRENEWAL THEORY

c

G t G t

a

a

c

t G t G

b

b

The sub ordinated pro cess starting from state is similar to the

parallel system example just discussed The sub ordinated pro cess start

ing from state diers a bit b ecause the failure intensity during the

repair of comp onent a is while it was in the parallel system hence

a b

nd

we can reach the elements and all the derivation of the rowofma

trices EtandKtby substituting by in the results derivations

b a

of the previous section

Numerical Results

For completeness of our analysis the Markovrenewal equations cor

resp onding to the selected examples were solved for numerical values

Deterministic repairtime distribution functions were considered in all

three examples ie

G t ut

a a a

G t ut

b b b

where ut is the unitary step function Table summarizes the nu

merical values for parameters used in the computations The units are

hours for repairtime parameters and and hour for the failure

a b

rates parameters and In the standby case the failure rate of

a b

the spare unit was made lower than the failure rate of the active unit

to characterize the warm standby situation

A LaplaceStieltjes transform metho d was adopted to solve the

Markov renewal equation which in LST domain b ecomes

V sE sK sV s

The time domain probabilities were calculated by rst deriving

the matrix V s using a standard package for symb olic analysis eg

MATHEMATICA and then numerically inverting the resulting LST

 expressions resorting to the Jagermans metho d

FRICKS TELEK PULIAFITO TRIVEDI

Table Parameters used in the numerical solutions

example

a b a b

Series System

Paral lel System

Standby System

Figures and rep ort availability and p erformabilityre

sults for all examples under the parameters established Following the



approachusedin we also ploted corresp onding Markovian system

results where each deterministic ring transition was replaced byan

equivalent stage Erlang distribution The Markovian mo dels were



solved using the Sto chastic Petri Net Package SPNP intro duced in

As exp ected the plots reect b etter availability for the warm

standby system followed by the parallel system However when con

sidered from the interval p ower available the situation reverses The

series system with repair provides more available p ower during a given

interval than the parallel or warm standby systems

CONCLUSIONS

An overview of Markov renewal theory was presented to intro duce a

promising alternative for the p erformability analysis of nonMarkovian

mo dels Our emphasis was to clarify the distinction b etween semi

Markov and Markov regenerative pro cesses and to establish a metho di

cal approachonhowtoidentify and prepare for the solution of problems

involving the mentioned sto chastic pro cesses

Although the development of adequate numerical approaches for

the analysis of these mo dels still require further eorts we studied two

approaches for the analytical formulation of systems b ehavior and two

metho ds for their analysis

The ma jor contribution of this chapter is the didactic structure

adopted to present and discuss p erformability analysis in the context

of nonMarkovian systems Essential examples of p erformabilityanal

ysis of series parallel and warm standby systems were elab orated to

show the phases of applications of the intro duced theoretical results

The whole solution pro cess asso ciated with each of the examples was

MARKOVRENEWAL THEORY

describ ed in detail aiming to suggest a metho dological approachwhen

dealing with Markov renewal theoryStochastic Petri nets were used to

supp ort the analyses of the examples and facilitate their understanding

REFERENCES

JF MEYER Performability a retrosp ective and some p ointers

to the future Performance Evaluation

J GRAY AND DP SIEWIOREK HighAvailability Computer

Systems Computer

M BEAUDRY Performance related reliability for computer sys

tems IEEE Transactions on Computers C

JF MEYER On evaluating the p erformability of degrad

able computer systems IEEE Transactions on Computers C

JK MUPPALA S WOOLET AND KS TRIVEDI Real

Time Systems Performance in the Presence of Failures

Computer

KS TRIVEDI Probability and with Reliability

Queuing and Computer Science Applications PrenticeHall

Englewo o dClis

DP SIEWIOREK Architecture of FaultTolerant Computers

An Historical Persp ective Pro cedings of the IEEE

KS TRIVEDI JK MUPPALA SP WOOLET AND

BR HAVERKORT Comp osite p erformance and dep endabil

ity analysis Performance Evaluation

S KARLIN AND HM TAYLOR A First Course in Sto chastic

Pro cesses Academic Press San Diego nd ed

RF BOTTA CM HARRIS AND WG MARCHAL Charac

terizations of Generalized Hyp erexp onential Distribution Func

tions Communications in Statistics Sto chastic Mo dels

FRICKS TELEK PULIAFITO TRIVEDI

MF NEUTS Renewal Pro cesses of Phase Typ e Naval

Research Logistics Quarterly

DR COX The analysis of nonmarkovian sto chastic pro cesses

by the inclusion of supplementary variables Pro ceedings of the

Cambridge Phylosophical So ciety

P FRANKEN D KONIG U ARNDT AND V SCHMIDT

Queues and Point Pro cesses AkademieVerlag Berlin

T ROLSKI Stationary random pro cesses asso ciated with p oint

pro cesses Lecture Notes in Statistics SpringerVerlag

VS KOROLYUK Markov Renewal Sequences in Reliability

Analysis in SemiMarkov Mo dels Theory and Applications

edited by Jacques Jansen Plenum Press New York

P FRANKEN AND A STRELLER Sto chastic Pro cesses with

an Emb edded Point Pro cess and their Application to System Re

liability Analysis in SemiMarkov Mo dels Theory and

Applications edited by Jacques Jansen Plenum Press New

York

DR COX Renewal Theory Methuen Co London

H CHOI VG KULKARNI AND KS TRIVEDI MarkovRe

generativeStochastic Petri Nets Performance Evaluation

ECINLAR MarkovRenewal Theory Adv Appl Prob

ECINLAR Intro duction to Sto chastic Pro cesses PrenticeHall

Englewo o d Clis

VG KULKARNI Mo deling and Analysis of Sto chastic Systems

Chapman Hall London

W FELLER An Intro duction to Probability Theory and its

Applications Vol I I John Wiley Sons nd ed

MARKOVRENEWAL THEORY

WL SIMTH Renewal Theory and its Ramications J Roy

Statist So c Ser B

WL SIMTH Regenerative Sto chastic Pro cesses Pro c Roy

So c London Ser A

WL SIMTH Remarks on the pap er Regenerative Sto chas

tic Pro cesses Pro c RoySocLondon SerA

PLEVY Systemes SemiMarkoviens a au plus une innite

denombrable detats p ossibles in Pro c Int Congr Math Am

sterdam

PLEVY Pro cessus semiMarkoviens in Pro c Int Congr

Math Amsterdam

L TAKACS Some Investigations Concerning Recurrent

Sto chastic Pro cesses of a Certain Typ e Magyar Tud Akad

Mat KutatoInt Kozl

R PYKE Markov Renewal Sequencees Denitions and Prelim

inary Prop erties Ann Math Statist

R PYKE Markov Renewal Sequences with Finitely Many

States Ann Math Statist

R PYKE AND R SCHAUFELE The Existence and Unique

ness of Stationary Measures for Markov Renewal Sequences

Ann Math Statist

MIKLOS TELEK Some advanced reliability mo delling

techniques Phd Thesis Hungarian Academy of Science

R GERMAN AND C LINDEMANN Analysis of determinis

tic and sto chastic Petri nets by the metho d of supplementary

variables Performance Evaluation

A BOBBIO AND M TELEK Markov regenerative SPN with

nonoverlapping activity cycles in International Computer

Performance and Dep endability Symp osium IPDS

FRICKS TELEK PULIAFITO TRIVEDI

G CIARDO R GERMAN AND C LINDEMANN A charac

terization of the sto chastic pro cess underlying a sto chastic Petri

net IEEE Transactions on Software Engineering

A BOBBIO VG KULKARNI A PULIAFITO M TELEK

AND KS TRIVEDI Preemptive rep eat identical tran

sitions in Markov regenerative sto chastic Petri nets in

Petri Net Performance Mo dels

Addison C FROBERG Intro duction to Numerical Analysis

Wesley Reading nd ed

R GERMAN D LOGOTHETIS AND K TRIVEDI Tran

sient analysis of Markov Regenerative Sto chastic Petri Nets A

Comparison of Approaches in Petri Net Performance Mo dels

R GERMAN Transient Analysis of deterministic and sto chas

tic Petri nets by the metho d of supplementary variables in

MASCOT

MK MOLLOY Performance Analysis using Sto chastic Petri

Nets IEEE Trans Comput C

M AJMONE MARSAN G CONTE AND G BALBO A Class

of Generalized Sto chastic Petri Net for the Performance Evalua

tion of Multipro cessor Systems ACM Trans Comput Syst

JB DUNGAN KS TRIVEDI RM GEIST

AND VF NICOLA Extended Sto chastic Petri Nets Appli

cations and Analysis in Performance edited by E Gelembe

Elsevier Science Publishers BV Amsterdam

M AJMONE MARSAN AND G CHIOLA On Petri nets with

deterministic and exp onentially distributed ring times Lecture

Notes in Computer Science

JL PETERSON Petri Net Theory and the Mo deling of

Systems PrenticeHall Englewo o d Clis

MARKOVRENEWAL THEORY

JL PETERSON Petri Nets Computing Surveys

M TELEK A BOBBIO L JEREB A PULIAFITO AND

KS TRIVEDI Steady state analysis of Markov Regenerative

SPN with age memory p olicy in MMB

DL JAGERMAN An Inversion Technique for the Laplace

Transform The Bell System Technical Journal

D LOGOTHETIS AND KS TRIVEDI TimeDep endent

Behavior of Redundant Systems with Deterministic Repair

in Computations with Markov Chains edited by WJ Stewart

Kluwer Academic Publishers Norwell

G CIARDO J MUPPALA AND KS TRIVEDI SPNP

Sto chastic Petri net Package in Pro c Int Workshop on Petri

Nets and Perf Mo dels IEEE Computer So ciety Press Los Alamos

FRICKS TELEK PULIAFITO TRIVEDI

SPNP results:

1.00 2.00

0.98 1.95

instantaneous 0.96 interval 1.90

availability 0.94 interval power

1.85 0.92

0.90 1.80 0 100 200 300 0 100 200 300 time (hours) time (hours)

LST results:

1.00 2.00

0.98 1.95

0.96 instantaneous interval 1.90

availability 0.94 interval power

1.85 0.92

0.90 1.80 0 100 200 300 0 100 200 300

time (hours) time (hours)

Figure Numerical results for the series system

MARKOVRENEWAL THEORY

SPNP results:

1.000 2.00

instantaneous 1.98 interval 0.999 1.96

availability 1.94

0.998 interval power

1.92

0.997 1.90 0 100 200 300 0 100 200 300 time (hours) time (hours)

LST results:

1.000 2.00

instantaneous 1.98 interval 0.999 1.96

availability 1.94

0.998 interval power

1.92

0.997 1.90 0 100 200 300 0 100 200 300

time (hours) time (hours)

Figure Numerical results for the parallel system with single repair

FRICKS TELEK PULIAFITO TRIVEDI

SPNP results:

1.0000 1.0000

0.9995 0.9995 instantaneous interval availability

0.9990 interval power 0.9990

0.9985 0.9985 0 100 200 300 0 100 200 300 time (hours) time (hours)

LST results:

1.0000 1.0000

0.9995 0.9995 instantaneous interval availability

0.9990 interval power 0.9990

0.9985 0.9985 0 100 200 300 0 100 200 300

time (hours) time (hours)

Figure Numerical results for the warm standby system with single repair