<p> Markov models – lesson 6</p><p>Computation of system with repair or backup in general is very complex  Can be simplified for some failure and repair distribution (exponential) using Markov models</p><p>Basic features of Markov models for system reliability:</p><p> function of two random variables (state of the system and time or other variable on which the state depends)  two random variable can be discrete or continuous  4-type of models basic models: discrete time discrete state of the system </p><p>Markov chain </p><p> continuous time discrete state of the system </p><p>Markov process </p><p> Markov model is define as:</p><p> set of probability transitions between states (initial state  consecutive state) S1 p S2</p><p>Where: p depends only on state S1 & S2 (it do not depend on old previous states, sometimes Markov models are called models without memory) p=p(S1,S2) In Markov model are defined mutually exclusive states of the system: Example: 1 component ~ 2 states (failure, correct working) without repair</p><p>Definition: initial state ~ t=0 final states ~ t>></p><p> System of Markov equations ~ description of probability of transition from initial state to final states  System of Markov equations suppose:</p><p>1) Probability of transition from one state to another in time interval t is i(t).t, where i(t) is probability rate of transition (so called hazard) between both states 2) Probability of more than one transition in time t is 0 (negligible error)</p><p>Remark: if i(t)=i  homogenous model</p><p>Example: suppose we have system with one component with two states and without repair.</p><p>The probability PS0(t+t) is the probability that the system will be in state without failure S0 in time t+t. It holds:</p><p>PS0(t+t)= PS0(t).(1-(t).t); where PS0(t) probability of working without failure in time t and (1-(t).t) is probability that the failure was not in time t</p><p> similar probability of failure state S1</p><p>PS1(t+t)= PS0(t).(t).t+ PS1(t); where (t).t is probability of failure in time t PS1(t) is probability that the system was in failure state S1 till time t</p><p>Therefore:  probability of the system failure: (t).t  probability of failure persistence: (~state S1) =1 (absorbing state)</p><p>Previous equations can be converted: P t  t P t S0 S0  tP t t S0 P t  t P t S1 S1  tP t t S0 With limit for t  0 : dP t S 0  tP t  0 dt S 0 (1) dP t S1  tP t dt S 0</p><p> general init condition is: t=0, So PS0(0)=1 and PS1(0)=0 Solution of previous equation system dP t S0  tt PS0t t ln PS0t   tdt  ln c 0 t  tdt 0 PS0t  ce ; where c  PS00  1 t   tdt 0 Rt  PS0t  e</p><p>Solution equation(2), similar (can be computed by PS0(t)+ PS1(t)=1) t   tdt 0 Qt  PS1t  1 e  Markov model graph representation  oriented graph (previous example): 1-t </p><p>t</p><p>S0(=x1) S1(=x1)</p><p>Necessary condition for every state:  probabilities of all transitions from one state = 1 situation for two components without repair: (4 states: X1X2, X1X2, X1 X2, X1 X2 )</p><p>13t </p><p>01t 13t S1(=X1X2) [01+02]t 1 this transition is vanished </p><p>(in practice do not arrive) S0(=X1X2) S3(=X1X2) 23t 02t 23t </p><p>S2(=X1X2) </p><p>The previous situation is described by equations for S0, S1, S2, S3</p><p>S0: PS0(t+t)=[1-(01+02)t] PS0(t)</p><p>S1: PS1(t+t)=01t PS0(t)+(1-13t)PS1(t)</p><p>S2: PS2(t+t)=02t PS0(t)+(1-23t)PS2(t)</p><p>S3: PS3(t+t)=13t PS1(t)+23t PS2(t)+1.Ps3(t) With matrix:</p><p>[PS0(t+t) PS1(t+t) PS2(t+t) PS3(t+t)]=</p><p>1 01  02 t 01t 02 t 0   0 1  t 0  t [P (t) P (t) P (t) P (t)]*  13 13  S0 S1 S2 S3  0 0 1  t  t   23 23   0 0 0 1 </p><p> so: PS(t+t)= PS(t).p matrix of transition probabilities</p><p> vector of probabilities of states in time t+t and t</p><p>Matrix of transition probabilities can be created directly from graph representation of Markov model </p><p>Pkj denotes the probability of transition from state k  to state j p=[Pkj]  rows ~ initial states in time t  columns ~ end states in time t+t</p><p>Remark: element for k=j ~ probability of persist in state zero element ~ transition cannot arrive</p><p> Pkj 1!! ~ sum in rows =1, system has to be in next k step in some state from defined state set (stochastic matrix) Writing in differential type: changing the equation for S0 (from previous example):</p><p>P t  t P t S 0 S 0     P t and limit t 0 we receive: t 01 02 S 0 ˙ PS 0 t  01  02 PS 0 t and similar for S1, S2, S3 ˙ PS1 t  01PS 0 t  13 PS1 t ˙ PS 2 t  02 PS 0 t  23 PS 2 t ˙ PS3 t  13 PS1 t  23 PS 2 t</p><p>Using matrix:</p><p> 01  02  01 02 0   0   0   ˙  13 13  PS t  PS t.z  matrix of transition rate z   0 0  23 23     0 0 0 0   sum in every row is = 0 !!!</p><p> solution of previous equation system for previous example PSi(0)=0 ; i=0…3</p><p>For i = constant we receive:</p><p>01 02 t PS0t  e</p><p>01 13t (01 02 )t PS1t  e  e  01  02  13</p><p>01 23t (01 02 )t PS2t  e  e  01  02  23 PS3t  1 PS1t PS1t PS2t</p><p>Conclusion: probability of states S0…S3 computed independently on structure of the system  it holds for every system configuration with two components without repair So special case: Serial system: R(t)=PS0(t) - both components working without failure Parallel system: R(t)=PS0(t)+ PS1(t)+ PS2(t) -at least one component works without failure System with backup: -t - 02=0, 01=13, PS2(t)=0  R(t)=PS0(t)+PS1(t)=e +te</p><p>Complexity of Markov model</p><p> depends on number of system’s states m  m – differential equations of 1-st order  in general for n components with k-states m=k n !! increase very quickly</p><p> simplification  distinction only states with different number of failure components (for n components with 2 states the previous number of Markov states m=2n decrease to m=n-1)</p><p>Example: Previous system with two components can change by merging states S1 and S2 and introducing probability PS1'(t)=PS1(t)+ PS2(t) Markov graph is:</p><p>1-'01t 1-'12t 1 </p><p>'01t '12t </p><p>S'0(=X1X2) S'1(=X1X2+ X1X2) S'3(=X1X2) No failure One failure Two failures </p><p>Simplification of graph for 13=23 can be different and '01=01+02 , '12=13=23</p><p>Using Markov model for systems with repair</p><p> How the repair changes the Markov graph?</p><p>It introduce the transition from states with w failed components to states with w-1 failed components.</p><p>Example: Markov graph for system with one component with repair (constant failure rate  and repair rate ).</p><p>˙ PS 0 t  PS 0 t  PS1t ˙ PS1t  PS 0 t  PS1t 1-t 1-t t</p><p>S0(=x1) S1(=x1) t</p><p>With initial condition PS0(0)=1 and PS1(0)=0 </p><p>  P t   exp   t S 0         P t   exp   t S1       in time t the system is working without failure = availability factor KP(t)=PS0(t)  The stable value of KP = lim´K p t  t   </p><p>1</p><p>1 2 3 4 5 t               </p><p>1 1 1    K t  ˙ 1 P  Usually the mean values holds   1    Solution of complex systems with repair: . similar to system without repair (introduce the new transition that correspond to repairs  modification of some matrix elements) . for big number of states can simplify by merging some states . constant failure rate and repair rate  homogenous differential equations  simply solution </p>