2.5 0 CV 0.5 2 1 Skewness 1.5 1.5 15 1

0.5 loglog 0

gamma Pareto 10

Kurtosis

lognorm 5

Weibull

0

Figure Kurtosis vs Skewness vs Co ecient of Variation Rotated Ball Bearings

at Skewness

0 0.5 1 1.5 2 2.5 0 CV 0.5 1 1.515 Pareto loglog

gamma 10 lognorm

Kurtosis

5 Weibull

0

Figure Kurtosis vs Skewness vs Co ecient of Variation Ball Bearings at

Kurtosis

14

Pareto

12

Gamma

10

8 Lognormal

6 Log log

4 Weibull

2

0 Skewness

0.5 1 1.5 2 2.5 3

Figure Kurtosis vs Skewness Ball Bearings at Skewness

5

4

Lognormal 3 Pareto

Log Logistic

2 Gamma

1

Weibull CV

0.2 0.4 0.6 0.8 1 1.2

Figure Skewness vs Co ecient of Variation Ball Bearings at 1

0.8

0.6 S(t) 0.4

0.2

0 0 50 100 150 200

t

Figure MLE Survivor Function vs KaplanMeier Pro ductLimit Estimate Ball

Bearings Weibull Fit 3

2.75

2.5

2.25 kappa

2

1.75

1.5

0.01 0.011 0.012 0.013 0.014 0.015 0.016

lambda

Figure Condence Region for MLEs Ball Bearings Weibull Fit Log Likelihood Function

-125 4 -150 -175 3 -200

0 2 kappa 0.005

0.01 1

lambda 0.015 0.02

0

Figure Log Likeliho o d Function with Maximum Value Identied Ball Bearings

Weibull Fit log(-log(S(t)))

log xi 3 3.5 4 4.5 5

-2

-4

y = -9.004 + 2.041 x

-6 lambda0 = 0.012

kappa0 = 2.041

Figure Weibull Plot Ball Bearings Weibull Fit r(p) Reliability Bounds 1

0.8

0.6

0.4

0.2

p

0.2 0.4 0.6 0.8 1

Figure Reliability Bounds for a FourComp onent System 1

3 2

4

Figure Four Comp onent System

AUTHORS

Glen Hartless Department of Statistics GrinFloyd Hall University of Florida

Gainesville FL USA

Glen Hartless received his BS in statistics from Virginia Tech Blacksburg VA

in He is currently pursuing his Masters degree in statistics at the University

of Florida He is a student memb er of ASQC and ASA

Lawrence Leemis Department of Mathematics College of William and Mary Williams

burg VA USA

Lawrence Leemis is a professor in the Mathematics Department at the College

of William Mary He received his BS and MS degrees in Mathematics and his PhD

in Industrial Engineering from Purdue University He has served as an Asso ciate

Editor for the IEEE Transactions on Reliability and Bo ok Review Editor for the

Journal of Quality Technology

REFERENCES

RE Barlow F Proschan Statistical Theory of Reliability and Life Testing Prob

ability Models To Begin With Silver Springs MD

DR Cox D Oakes Analysis of Survival Data Chapman and Hall New York

JF Lawless Statistical Models and Methods for Lifetime Data John Wiley and

Sons New York

S Wolfram Mathematica A System for Doing Mathematics by Computer Second

Edition AddisonWesley

ACKNOWLEDGEMENT

This research was carried out during a Research Exp erience for Undergraduates

sp onsored by the National Science Foundation grant numb er at The College

of William Mary during the summer of We thank the asso ciate editor and

referees for their helpful comments

Uniform NoShapeParameter

Exponential NoShapeParameter

Weibull k

Gamma k

LogLogistic k

Pareto b

LogNormal sigma

Discussion

It should b e kept in mind that the Euclidean distance technique should only b e

used to determine which mo dels would b e obviously inferior It is not a metho d

with which to cho ose a mo del For instance in testing this metho d using randomly

generated samples from a log normal distribution the tted Weibull and gamma

distributions would often b e closer to the sample moments the lognormal is not a clear

cut winner based on distance to the sample moments for example the Kolmogorov

Smirnov go o dnessoft test smaller for the MLE lognormal t than it is for the MLE

Weibull t Hence For the ball b earing data set one should only conclude that the

exp onential Pareto distributions are inferior in describing the distribution For the

Weibull distribution the initial estimate of is close to the MLE which was

see section

The Mathematica implementation is similar to the Weibull maximum likeliho o d

problem the ma jor step is p erformed by a ro otsolver Using Mathematica did lead

us to a new discovery although the kurtosis vs skewness vs co ecient of variation

plot was only marginally useful in and of itself the minimum Euclidean distance to

the sample p oint provides us with some very useful exploratory results

The moment curves given here are the function of at most a single shap e parameter

The value of this parameter when the Euclidean distance is minimized can b e used

as an initial estimate for obtaining its maximum likeliho o d estimate Hence not

only can we see which distributions are p ossible mo dels we can also learn something

ab out their parameter estimates Again due to Mathematicas ro otsolving ability

this minimizer was not dicult to develop Unlike the likeliho o d function in section

the distance functions do not need a particularly accurate initial estimate to converge

Example

Consider again the ball b earing data from section Its moments are

Figures are the skewness vs co ecient of variation

3 4

and kurtosis vs skewness plots for several lifetime distributions gamma log logistic

log normal Pareto and Weibull The exp onential distribution is a sp ecial case of

the gamma and Weibull and has moments Also the uniform

3 4

1 9

p

distribution has moments The sample moments of the

3 4

5

3

ball b earing data are indicated by a p oint on each graph Figures show

the dimensional moment curves from two p ersp ectives Figure lo oks down at

the kurtosis vs skewness plot with depth provided by the co ecient of variation

Figure is a rotated version with the kurtosis height scale reduced As explained

previously the b ehavior is dicult to see

What follows is the results of determining the minimum Euclidean distance b e

tween and each of the moment curves in three dimensions

3 4

In MomentDistancedata

OutMatrixForm

DISTRIBUTION MIN DISTANCE INITIAL ESTIMATE

This larger programming application revealed a downside of using Mathematica

The program syntax can b e quite tedious Slight deviations from the desired command

can pro duce a wealth of syntax errors and warnings andor incorrect results

MODEL SELECTION

We select a parametric mo del to describ e a set of lifetime data In order to limit

the numb er of mo dels sample moments can b e compared to the theoretical moments

of the distribution in question

Two graphical metho ds used for this purp ose are plots of skewness vs co ef

3

cient of variation and kurtosis vs skewness see Cox and Oakes p

4

The rst plot is a visual representation of symmetry and spread while the second is of

p eakedness and symmetry Some distributions such as the exp onential and uniform

distributions reduce to a single p oint on these plots Close proximity of the curve

to the corresp onding sample moments indicates that the distribution is not a bad

p otential parametric mo del

Mathematica Implementation

Since Mathematica is capable of pro ducing a dimensional parametric plot it

o ccurred to us that a plot of would b e of similar use Unfortunately the

3 4

simplicity of the curves makes their proximity to the sample moments dicult to

discern without rotating the dimensional plot One solution to this problem is

to nd the minimal Euclidean distance b etween and the dimensional

3 4

moment curves where is the ratio of the sample standard deviation to the sample

mean and

3 4

n n

X X

s t t t t

i i

4 3

n s n s t

i=1 i=1

kappa lambda

Log Likelihood at the MLE

Observed Information Matrix

Inverse Information Matrix VarianceCovariance Matrix of the MLEs

Two comments with resp ect to the scondence region in Figure follow First

Figure veries that a Weibull distribution is a b etter t than an exp onential dis

tribution since the line is not interior to the scondence region Second

adjusting an argument in ReliabilityFit gives smo other contours at the cost of

larger computation times

Discussion

Mathematicas overall p erformance on this larger more complex problem was

adequate We were pleased with the programs ability to overlay several graphic

images such that the scaling app earance were appropriate We accomplished nearly

everything we had hop ed to although much trial error was often necessary

Another advantage of our Mathematica implementation is that our function has

only three arguments the data set a survivor function and initial parameter es

timates The log likeliho o d function score vector and information matrix are all

determined by Mathematica As such the co de app ears much more like the mathe

matical derivations than the corresp onding co de in an algorithmic language Other

twoparameter distributions could b e tted in a similar fashion

Example

6

The data b elow are ball b earing failure times in revolutions from Lawless

The commands necessary to run our function ReliabilityFit are

In data

In SurvFunctWeibull NE LAM x K

In ReliabilityFitdata WeibullSurvivor InitEstimWeibulldata

The second element of each data pair gives its censoring status right censored

observed and InitEstimWeibull is a function that returns initial estimates of

the Weibull parameters The output of the program includes a scatterplot and tted

regression line Figure a dimensional graph of the log likeliho o d function Figure

and condence regions for the MLEs Figure and a comparison of

the MLE survivor function to the corresp onding KaplanMeier pro ductlimit estimate

Figure The function prints the survivor function maximum likeliho o d estimators

the maximum value of the log likeliho o d function the observed information matrix

and the asymptotic variancecovariance matrix of the MLEs as shown here

k

lam t

Survivor Function E

Maximum Likelihood Estimators

the data There is no diculty in implementing either the regression or the plot in

2

Mathematica

Initial estimates in hand Mathematica calculates the maximum of the log likeli

ho o d function and plots the log likeliho o d function surface The observed information

matrix the asymptotic variancecovariance matrix of the MLEs and the value of the

log likeliho o ds maximum are all printed The latter can b e used to compare the ts

of dierent families of distributions

As a further test of the Weibull t

log L log L

2

and hence we can obtain an approximate scondence region for is asymptotically

2

the parameter estimates This scondence region can b e used to determine whether

the additional parameter in the Weibull distribution compared to the exp onential

distribution without a shap e parameter is warranted If the line is interior

to the scondence region the extra parameter is not ssignicant and the reduced

mo del might b e appropriate Quantiles of the chisquare distribution and those

of many other distributions are available in Mathematica Using the Mathematica

function ContourPlot pro duces a contour plot of the log likeliho o d function which

is the b oundary of an asymptotically valid scondence region for the parameters

The output given in the example that follows represents a balance b etween precision

and computing time Finally a graph of the estimated survivor function is pro duced

along with the Kaplan Meier pro ductlimit estimator of the survivor function

2

Obtaining the Weibull MLEs can b e reduced to a onedimensional search In order to facilitate

the tting of other twoparameter distributions with the same co de however this approach was not

taken Additionally a twodimensional search was a b etter test of Mathematicas abilities

for all t and Most of the derivation shown b elow is given in Cox

and Oakes

For failure times t t t and censoring times c c c the log likeliho o d

1 2 n 1 2 n

function can b e expressed in terms of the hazard function ht and the cumulative

hazard function H t

n

X X

H x log hx log L

i i

i=1 iU

where U is the set of indices corresp onding to uncensored observations r is the numb er

of observed failures and x minft c g for i n

i i i

Maximum likeliho o d estimates can b e obtained in the usual way by taking the

rst partial derivatives of the log likeliho o d function with resp ect to and setting

the resulting equations equal to zero and solving for the parameters

Given the survivor function Mathematica can p erform all of the derivations nec

essary to obtain this system of equations Given suitable initial estimates it

can solve the equations via Newtons Metho d We found that simply guessing the

initial estimates resulted in a rather slow convergence of the ro ot solver Therefore

we used a least squares approach to obtain initial estimates for and For the

Weibull distribution

log log S t log log t

Let Rt b e the set of indices corresp onding to items at risk at or just prior to time

t and let nt jRtj b e the numb er of elements in or the cardinality of Rt

n(t)

Estimating the survivor function with we p erform a simple linear regression on

n+1

i h

n(t)

The slop e of the tted mo del is an estimate of and the log x vs log log

i

n+1

intercept is an estimate of log This is equivalent to a plot of the data on Weibull

pap er and has the side b enet of b eing able to assess mo del adequacy By plotting

the data and the tted regression line we can determine whether the relationship

is linear thereby conrming the appropriateness of the mo del over the region of

In lowerbnd

Out p p

In lowerbnd p

Out

Discussion

Mathematica has more p ower exibility than is really needed to solve this

problem Still the ability to view exp eriment with the upp er and lower b ounds is

an educational to ol since students can easily explore the b ehavior of the inequality

for dierent reliabilities and system congurations

LIFETIME DATA ANALYSIS

Consider tting statistical mo dels to a set of lifetime data In particular we

wish to t the Weibull distribution to a rightcensored data set using maximum

likeliho o d estimation the Weibull distribution was chosen b ecause its MLEs must b e

calculated by numerical metho ds Also we would like to obtain some measures of

mo del adequacy scondence regions for the parameters a visual comparison of the

corresp onding nonparametric mo del and the ability to compare the t to that of

other parametric mo dels

Mathematica Implementation

Recall that the survivor function of the Weibull distribution is

(t )

S t e

Plotlowerbnd upperbnd p PlotLabel

Reliability Bounds AxesLabel p rp

The arguments passed to bound are the set of minimal cut sets and the set of minimal

1

path sets

Example

To illustrate the use of these reliability b ounds on a small system consider the

fourcomp onent system in Figure This system has three minimal path sets

fg f g f g

and two minimal cut sets

f g f g

Once bound b een input the appropriate Mathematica commands to determine

the reliability b ounds for this system are

In mpaths

In mcuts

In boundmpaths mcuts

The upp er and lower b ounds for the system reliability for comp onents with iden

tical reliabilities are plotted in Figure We can request the expression for say the

lowerb ound and evaluate it for p

1

One Mathematica data structure is a list which is a set of numb ers algebraic expressions

functions graphical elements etc

RELIABILITY BOUNDS

The reliability r p of a scoherent system of sindep endent comp onents can b e

b ounded using minimal path sets and minimal cut sets Barlow and Proschan

p where p p p p is a vector of comp onent reliabilities For a s

1 2 n

coherent system of sindep endent comp onents with minimal path sets P P P

1 2 s

and minimal cut sets C C C one such b ound is

1 2 k

s k

Y Y Y Y

p p r p

i i

j =1 j =1

iP iC

j j

or

s k

a Y Y a

p p r p

i i

j =1 iP j =1 iC

j j

The b ounded expression is the probability that the system is op erational at one

particular time ie the system reliability

Mathematica Implementation

Consider a scenario where each comp onent has the same reliability p where

p For various values of p how precise are our b ounds for an arbitrary system

conguration A Mathematica function bound calculates and plots the reliability

b ounds Since the co de for this function is short it is included b elow

boundpathsList cutsList

numpaths Lengthpaths

numcuts Lengthcuts

upperbnd lowerbnd

Doupperbnd upperbnd p Lengthpathsii numpaths

upperbnd upperbnd

Dolowerbnd lowerbnd p Lengthcutsi i numcuts

c c c censoring times

1 2 n

x minft c g time on test for item i i n

i i i

U index set of uncensored observations

r numb er of observed failures

L likeliho o d function

Weibull scale parameter MLE

Weibull shap e parameter MLE

ht hazard function

H t cumulative hazard function

Rt risk set at time t

nt cardinality of Rt

co ecient of variation

3

T

E skewness

3

4

T

E kurtosis

4

BACKGROUND

Mathematica is an environment for numerical symb olic computations with

excellent graphics capabilities It combines the interactive user interface and func

tional programming of a high level language such as SPlus with commands func

tions that supp ort symb olic manipulation We will not emphasize the actual co ding of

our functions but rather the insight gained from applying Mathematica to reliability

problems The Mathematica co de is available from the authors

Each subsequent section is divided into four parts an intro duction to the par

ticular topic the implementation of Mathematica to solve the problem a sp ecic

example to illustrate the implementation and a brief discussion of the merits of the

Mathematica solution

INTRODUCTION

Computational algebra languages are to ols for solving a wide array of mathe

matical problems These interactive frameworks have p owerful symb olic graphical

capabilities that are easily quickly implemented making these languages sup erior

to standard algorithmic languages such as C or FORTRAN for certain applications

In addition they provide lowlevel programming constructs that allow exibility that

is often not available in sp ecic application software packages

This pap er do cuments our application of one such program the Mathematica

system as a to ol In short how would one fare in applying such a language

to some of the diverse computational problems in reliability What features of the

system make it sup erior or inferior to other programming environments Can a

reliability engineer with a mo dest computer programming background make use of

Mathematica to solve problems eciently In order to answer these questions we

chose three problems that test the usefulness of this to ol reliability b ounds lifetime

data analysis and mo del selection

Acronyms

MLE maximum likeliho o d estimator

KMPL KaplanMeier pro duct limit

Notation

r p system reliability

p p p p vector of comp onent reliabilities

1 2 n

P P P minimal path sets

1 2 s

C C C minimal cut sets

1 2 k

S t survivor function

Weibull scale parameter

Weibull shap e parameter

t t t failure times

1 2 n

Computational Algebra Applications in

Reliabil i ty Theory

Glen Hartless

University of Florida

Gainesville

Lawrence Leemis

The College of William Mary

Williamsburg

Key Words Computational algebra language Mathematica Mo del selection

Reliability b ounds Symb olic algebra language Weibull distribution

Summary Conclusions Reliability analysts are typically forced to cho ose

b etween using an algorithmic programming language or a reliability package for

analyzing their mo dels and lifetime data This pap er shows that computational lan

guages can b e used to bridge the gap to combine the exibility of a programming

language with the ease of use of a package Computational languages facilitate the

development of new statistical techniques and are excellent teaching to ols This pap er

considers three diverse reliability problems that are handled easily with a computa

tional algebra language system reliability b ounds lifetime data analysis and mo del

selection