Probabilistic Arithmetic

Rob ert Charles Williamson BE QIT MEngSc Qld

A thesis submitted for the degree of Do ctor of Philosophy

Department of Electrical Engineering University of Queensland August

Statement of Originality

To the b est of the candidates knowledge and b elief the material presented in this

thesis is original except as acknowledged in the text and the material has not b een

submitted either in whole or in part for a degree at this or any other university

Rob ert Williamson ii

Abstract

This thesis develops the idea of probabilistic arithmetic The aim is to replace

arithmetic op erations on numb ers with arithmetic op erations on random variables

Sp ecicallyweareinterested in numerical metho ds of calculating convolutions of

probability distributions The longterm goal is to b e able to handle random prob

lems such as the determination of the distribution of the ro ots of random algebraic

equations using algorithms whichhavebeendevelop ed for the deterministic case

To this end in this thesis we survey a numb er of previously prop osed metho ds

for calculating convolutions and representing probability distributions and examine

their defects We develop some new results for some of these metho ds the Laguerre

transform and the histogram metho d but ultimately nd them unsuitable We

nd that the details on how the ordinary convolution equations are calculated are

secondary to the diculties arising due to dep endencies

When random variables app ear rep eatedly in an expression it is not p ossible to

determine the distribution of the overall expression by pairwise application of the

convolution relations We prop ose a metho d for partially overcoming this problem in

the form of dependency bounds These are b ounds on the distribution of a function

of random variables when only the marginal distributions of the variables are known

They are based on the Frechet b ounds for joint distribution functions

Wedevelop ecientnumerical metho ds for calculating these dep endency b ounds

and showhow they can b e extended in a number of ways Furthermore weshowhow

they are related to the extension principle of fuzzy set theory which allows the

calculation of functions of fuzzy variables Wethus provide a probabilistic interpre

tation of fuzzy variables We also study the limiting b ehaviour of the dep endency

b ounds This shows the usefulness of interval arithmetic in some situations The

limiting result also provides a general law of large numb ers for fuzzy variables In

terrelationships with a numb er of other ideas are also discussed

Anumb er of p otentially fruitful areas for future research are identied and the

p ossible applications of probabilistic arithmetic which include managementofnu

meric uncertainty in articial intelligence systems and the study of random systems

are discussed Whilst the solution of random algebraic equations is still a long way

o the notion of dep endency b ounds develop ed in this thesis would app ear to b e of

indep endentinterest The b ounds are useful for determining robustness of indep en

dence assumptions one can determine the range of p ossible results when nothing is

known ab out the joint dep endence structure of a set of random variables iii

Acknowledgements

Iwould liketoacknowledge the help and supp ort of the following p eople who con

tributed to this thesis in various ways

Most imp ortantly I would like to publicly thank my wife Angharad who has

assisted me greatly in numerous ways over the years Without her help and supp ort

this thesis would not have b een p ossible I am deeply grateful for all the assistance

she has provided

Iwould also liketothank

My sup ervisor Professor Tom Downs for continued supp ort assistance and

encouragement and for b eing a solver of administrative problems rather than a

generator of them

Former head of department Professor Mat Darveniza who help ed me as a b e

ginning graduate student and later assisted in my obtaining a CPRA scholarship in

order to pursue my PhD studies

The head of department Professor Tom Parnell for the use of departmental

facilities in the preparation of this thesis

Dr Phil Diamond of the Department of Mathematics University of Queensland

for useful advice and discussions and for the loan of some pap ers

An anonymous referee of the International Journal of Approximate Reasoning

for a question which led to section of chapter

Professor Brian Anderson of the Australian National University whose simple

question as to whether I had lo oked at any limiting results was part of the motivation

for chapter

Dr Helen McGillivray of the Department of Mathematics University of Queens

land for a discussion on condence intervals and other matters and the loan of a

b o ok

Dr Guy West of the Department of Economics University of Queensland for

discussions and the loan of some pap ers

The following p eople assisted me by sending comments on some questions and

reprints of their pap ers Professor Alice Agogino University of California at Berke

ley Dr Piero Bonissone General Electric Research Labs New York Dr Jo el Bren

ner Palo Alto Professors Didier Dub ois and Henri Prade UniversityofToulouse

I I I Professor Jurgen Garlo Institut fur Angewandte Mathematik Universitat iv

Freiburg Professor Robin Giles Queens UniversityOntario Professor VL Girko

Kiev University Dr Ellen Hisdal University of Oslo Professor M Sambandham

Atlanta University Dr Elie Sanchez Marseille Professor Berthold Schweizer

UniversityofMassachusetts Amherst Professor Eugene Seneta Universityof

Sydney Professor Ross Shacter Stanford University Dr Oskar Sheynin Moscow

Professor Ushio Sumita UniversityofRochester and Professor Klaus Weise

PhysikalischTechnische Bundesanstalt Braunschweig West Germany

Iwould also like to thank the indulgence of the Electrical Engineering librarian

Mrs Barbara Kormendy and the Mathematics Librarian Mrs Freida Kanowski

It is a pleasure to acknowledge a debt to Donald Knuth and Leslie Lamp ort

for the developmentofT X and L T X under which this thesis was prepared and

E E

to James Alexander for his TIb bibliographic prepro cessor These to ols were most

useful

Financial supp ort from a Commonwealth Postgraduate ResearchAward and a

grant from the Australian Research Grants Scheme is gratefully acknowledged

FinallyI would liketoacknowledge an intellectual debt to Karl Popp er who

through his writings has taught me the imp ortance of clarity of exp osition and

honesty of thought

Publications

Most of the material in this thesis has b een or will b e published elsewhere The

following is a list of pap ers either published submitted or nearing submission

which rep ort material on the topic of this thesis Some of these pap ers rep ort

further material which is not included in this thesis

Rob ert C Williamson and Tom Downs Probabilistic Arithmetic and the Dis

tribution of Functions of Random Variables Pro ceedings of the st IASTED

Symp osium on Signal Pro cessing and its Applications Brisbane August

Parts of chapter including further details and examples on

the histogram metho d

Rob ert C Williamson and Tom Downs The Inverse and Determinantofa

Uniformly Distributed Random Matrix Statistics and Probability Letters

Chapter

Rob ert C Williamson and Tom Downs Probabilistic Arithmetic Numeri

cal Metho ds for Calculating Convolutions and Dep endency Bounds accepted

for publication in the International Journal of Approximate Reasoning

Chapter

Rob ert C Williamson An Extreme Limit Theorem for Dep endency Bounds

of Normalised Sums of Random Variables accepted for publication in Infor

mation Sciences Chapter

Rob ert C Williamson The LawofLargeNumb ers for Fuzzy Variables under a

General Triangular Norm Extension Principle under revision for resubmission

to Fuzzy Sets and Systems Chapter

Rob ert C Williamson Interval Arithmetic and Probabilistic Arithmetic

to app ear in the pro ceedings of SCAN IMACSGAMMGI International

Symp osium on Computer Arithmetic and SelfValidating Numerical Metho ds

Basel Octob er A summary of chapters and and a discussion of the

relationship b etween probabilistic arithmetic and interval arithmetic

Rob ert C Williamson and Tom Downs Probabilistic Arithmetic Relation

ships with Other Ideas to b e submitted to the International Journal of Ap

proximate Reasoning Chapter vi

Rob ert C Williamson and Tom Downs Numerical Metho ds for Calculating

Convolutions of Probability Distributions to b e submitted to Mathematics

and Computers in Simulation Chapter

Rob ert C Williamson The QuotientofTwo Normal Random Variables An

Historical Study in preparation Traces the history of a simple problem

related to probabilistic arithmetic

Rob ert C Williamson The Discrete Tconjugate Transform in preparation

To b e submitted to Information Sciences Extends the Tconjugate

transform see section and chapter

Contents

Probabilistic Arithmetic The Very Idea

Motivation

Outline of Results

Dierent Metho ds for Calculating Convolutions and the La

guerre Transform Metho d

The Dep endency Bounds and Numerical Metho ds of Calcu

lating Them

Precursors Multiple Discoveries and Relationships with Fuzzy

Sets

The Inverse and Determinant of a Random Matrix

A Limiting Result for Dep endency Bounds

Thesis Structure

Notational Conventions

Numerical Metho ds for Calculating Convolutions of Probability

Distributions

Intro duction Aim and Analytical Metho ds

History Motivation and Outline

Exact Analytical Results

Sp ecialised Formulae for Convolutions

Integral Transforms

The Hfunction Metho d

Miscellaneous Numerical Metho ds for Calculating Convolutions

Normal Approximations and other Parameterised Distributions

Metho ds Based on Moments

NonLinear Transformations

Direct Sampling of the Densities for Sum and Dierence Con

volutions viii

The SkinnerAckroyd Metho d

Spline Based Metho ds

Laguerre Transforms and Other Orthonormal Transforms

The Laguerre Transform Metho d

Distributions of Pro ducts and Quotients using the Laguerre

and Mellin Transforms

Distribution of Pro ducts and Quotients Calculated with the

Laguerre Transform Directly

Distributions of Pro ducts and Quotients using the Laguerre

Transform and LogarithmicExp onential Transformation of

Variables

Other Typ es of Laguerre Transforms and Related Results

Conclusions on the Laguerre Transform Metho d

The Histogram and Related Metho ds

The Histogram Representation

Combining Histograms to Calculate Convolutions

Kaplans Metho d of Discrete Probability Distributions

Mo ores Histogram Metho d

Interval Arithmetic and Error Analysis

The Standard Interval Arithmetic

Triplex Arithmetic and Dempsters Quantile Arithmetic

Interval Arithmetic and Condence Intervals from the Metrol

ogists PointofView

PermutationPerturbation and Related Metho ds

Numerical Metho ds for Calculating Dep endency Bounds and Con

volutions

Intro duction

The Problem

The Idea of Probabilistic Arithmetic and the Need for Dep en

dency Bounds

Metho ds of Handling the Errors in Probabilistic Arithmetic

Outline of the Rest of This Chapter

Denitions and Other Preliminaries

Distribution Functions

Binary Op erations

QuasiInverses

Triangularnorms

Copulas and Joint Distribution Functions

The Triangle Functions and and convolutions

Dep endency Bounds and their Prop erties

The Dep endency Bounds

Pointwise BestPossible Nature of the Bounds

Examples

ExclusiveUseofLower and Upp er Bounds

Numerical Representation

Duality

Numerical Representation and the Calculation of ldb and udb

Numerical Calculation of Convolutions

Calculation of Moments and other Functionals

Examples

Incorp orating and Up dating Dep endency Information

General Idea

Interpretation of Dep endencies Implied by C W

W in Probabilistic Arithmetic Calculations Use of C

Measures of Dep endence

Use of Mixtures for Nonmonotonic Op erations

Intro duction and General Approach

Complications Arising from the use of Lower and Upp er Prob

ability Distributions

Diculties Intro duced by Using the Numerical Approximations

Numerical Algorithms for Splitting Distributions into Positive

and NegativeParts and Recombining Them

Conclusions on the use of Mixtures for Nonmonotonic Op era

tions

Conclusions

Relationships with Other Ideas

Intro duction

George Bo oles Results on Frechet Bounds and Some RecentExten

sions and Applications

Bo oles Ma jor and Minor Limits of Probabilities

Hailp erins Reformulation and Extension of Bo oles Bounds

Using Linear Programming Metho ds

Application of the Frechet Typ e Bounds to Probability Logic

Further Results on Distributions known only by their Marginals

AForest of GraphTheoretical Ideas

Networks of Probabilities of Events Bayesian Networks Be

lief Networks and Inuence Diagrams

Path Analysis and Covariance Structure Mo delling

Common Sub expression Elimination in Probabilistic Arith

metic byDAG Manipulations

Conclusions The Diculties of Manipulating Graphs of Ran

dom Variables

Other Lower and Upp er Probability Metho ds

The DempsterShafer Theory of Belief Functions

Fines IntervalValued Probabilities

Other LowerUpp er Probability Theories

Fuzzy Arithmetic of Fuzzy Numbers

Numerical Metho ds for Fuzzy Convolutions and the Notion of

InteractiveFuzzy Variables

The Mo dalities Possible and Probable and their Relation

ship to Fuzzy Set Theory and Possibility Theory

The Relationship Between Probability Theory and Fuzzy Set

Theory

Condence Curves Possibility and Necessity Measures and

Interval Fuzzy Numb ers

General Interpretations Positivist vs Realist

Probabilistic Metric Spaces

Philosophical Asp ects of Probabilistic Metric Spaces

General Conclusions

An Extreme Limit Theorem for Dep endency Bounds of Normalised

Sums of Random Variables

Intro duction

Denitions and Other Preliminaries

Distribution Functions

Triangular Norms

Representation of Archimedean tnorms

Copulas

Dep endency Bounds

Tconjugate Transforms

Convergence of Dep endency Bounds of Normalised Sums

Rates of Convergence

Examples

Conclusions

ALaw of Large Numb ers for Fuzzy Variables

Intro duction

Theorem and Direct Pro of

Dep endency Bounds

Pro of of Theorem via Decomp osition

A Direct Pro of Using Bilateral Tconjugate Transforms

Numerical Calculation of supT Convolutions

Increasing Part

Decreasing Part

Discretisation

The Inverse and Determinantofa Uniformly Distributed Ran

dom Matrix

Intro duction

Results

Discussion

Conclusions

Overview

Directions for Future Research

Some General Philosophical Conclusions

References

List of Figures

Illustration of Ackroyds metho d of discretising a continuous proba

bility distribution function The solid line is F x and the twodashed

xand F x resp ectively lines are F

The histogram representation f x approximates f x

Histogram densityof Z X Y where df X df Y U and

The Discrete Probability Distribution representation

Discrete probability distribution of Z X Y df X df Y

U andn

Discrete probability distribution of Z X Y df X df Y

U andn

Illustration for the pro of of theorem

Graph of Lu v x

A p ossible F F when F and F have discontinuities

X Y X Y

C F uF v

X Y

Lower and upp er dep endency b ounds for X Y where b oth X and

Y are uniformly distributed on

Lower and upp er dep endency b ounds for X Y where X is uniformly

distributed on and Y is uniformly distributed on

Lower and upp er dep endency b ounds for X Y where b oth X and

Y are uniformly distributed on

Lower and upp er dep endency b ounds for X Y where X is uniformly

distributed on and Y is uniformly distributed on

Lower and upp er dep endency b ounds for X Y where b oth X and Y

are uniformly distributed on xiii

The copula W

The numerical representation of probability distributions

The quasiinverse representation of a distribution function

Illustration of the condensation pro cedure

Numerical representation of F uniform distribution on with

Numerical representation of F a Gaussian distribution with

and and which is curtailed at

Numerical representation of F where X YZ andY and Z are

as in gures and resp ectively

Numerical representation of an exact triangular distribution centred

at with a spread of

Output of the convolution algorithm for two random variables

uniformly distributed on

Lower and upp er dep endency b ounds for the same calculation as for

gure

Lower and upp er dep endency b ounds for the sum of tworandom

variables each uniformly distributed on when the lo wer b ound

on their connecting copula is given by T for various p

Illustration of quantities used in splitting a distribution into p ositive

and negative parts

Result of the numerical calculation of a convolution of a uniform

and a Gaussian random variable

The lower and upp er dep endency b ounds for subtraction for the same

variables as for gure

An example distribution whichistobesplitinto p ositive and negative

parts

The p ositive part of the distribution in gure

The negative part of the distribution in gure

Directed graph representation of the expansion

Path diagram of the linear mo del describ ed by and

DAG representations for and

Illustration of a convex function f on C X and its conjugate dual

f on the dual space X

Illustration of the optimization problem

The idea b ehind Fenchels duality theorem

Atypical condence curve after Birnbaum

AFuzzy Numb er FN and a Flat Fuzzy Numb er FFN

Memb ership function constructed from histogram data

Lower and upp er probability distributions for which there is no cor

resp onding normal fuzzy number

Illustration for the pro of of lemma

Lower and upp er dep endency b ounds for S X with N

N i

X with N Lower and upp er dep endency b ounds for S

i N

Dep endency b ounds for S where df X is upp er half Gaussian

N i

Dep endency b ounds for the same problem as gure but calculated

using

Comparison b etween the exact and numerical calculation of the de

p endency b ounds gures and

The probability density f

The probability density f s

Chapter

Probabilistic Arithmetic The

Very Idea

Now that you have learnt about adding minussing

multiplying and dividing you can do any sum Even an

atomic scientist only real ly uses these four operations

Mrs S Boal Grade Oakleigh Primary Scho ol

If one looks at the development of the measurement process

during the past century one soon observes that with

increasing frequency the raw data areprobability

distribution functions or frequency functions rather than

real numbers This is so in the physical sciences and in the

biological and social sciences it is the rule rather than the

exception One may thus convincinglyargue that

distribution functions are the numbers of the futureand

that one should therefore study these new numbers and their

arithmetic

Berthold Schweizer

Motivation

This thesis studies the idea of probabilistic arithmetic which is the name we giveto

the idea of calculating the distribution of arithmetic and p erhaps other functions of

random variables It is motivated by the hop e of b eing able to develop pro cedures for

solving random problems based on those already existing for the deterministic case

The fundamental observation to make is that nearly all existing numerical algorithms

are based on the four op erations of arithmetic addition subtraction multiplication

and division For example it may b e p ossible to determine the distribution of the

ro ots of p olynomials with random co ecients by mo dication of existing algorithms

for calculation of the ro ots in the deterministic case Work to date on random

p olynomials has only obtained this for a very restricted class of p olynomials

The rst step necessary in achieving this goal is the development of an ecient

and accurate metho d for determining the convolution of probability distribution

functions We use the word convolution in its general sense describing the op er

ation on distribution functions corresp onding to virtually any op eration on random

variables and not just addition If Z LX Y where X and Y are indep endent

random variables with joint distribution function F then F the distribution of

XY Z

Z isgiven by

F z dF uF v

Z X Y

Lfz g

where Lfz g fu v j u v Lu v zg The ability to calculate is

necessary but not sucient for the construction of a probabilistic arithmetic

The most imp ortant dierence b etween the deterministic and sto chastic cases is

the app earance of sto chastic dep endence This has no counterpart in the determin

istic case We shall see that even if we restrict ourselves to indep endent random

variables at the outset sto chastic dep endencies can arise during the course of a cal

culation due to the o ccurrence of rep eated terms in expressions For example if X

Y and Z are indep endent random variables then V X Y and W X Z are not

necessarily indep endent Wegive the name dependency error to the error incurred

in calculating the distribution of some function of V and W suchas U VW by

assuming that V and W are indep endent

Whilst a general solution to this problem seems imp ossible we can provide a

partial solution in terms of dependency bounds This is the name wegivetolower

and upp er b ounds on the distribution of functions of random variables when only the

marginal distributions are known In other words the dep endency b ounds contain all

the p ossible results due to all the p ossible joint distributions of the random variables

involved If ldb and udb denote the lower and upp er dep endency b ounds and

denotes a binary op eration we write the b ounds on Z X Y as

ldbF F z F z udbF F z

X Y Z X Y

These b ounds can b e calculated explicitly for certain classes of binary op erations

The general approachwehave just outlined has b een studied previously bya

numb er of authors Whilst we defer a more detailed review of previous work to

later chapters let us mention the following now The idea of an algebraofrandom

variables based on the use of integral transforms for calculating the appropriate

convolutions was advocated by Springer in and formed the motiv ation for

Analytical metho ds for determining distributions of functions of random

variables are at the core of applied probability theory an early but obscure

review is given by Halina MilicerGruzewsk a and of course the study of the

distribution of sums in the central limit theorem is the basis for many of the

theoretical results in probability theory Numerical metho ds for calculating

distributions of functions of random variables have b een studied bya number of

authors chapter is a review of available techniques Some of the more recent

techniques are the histogram metho d and the related metho d of discrete

probability distributions and the intricate Hfunction metho d The

problem we study is obviously related to the propagation of errors of measurement

Fuzzy arithmetic deriv es from similar motivations and it is compared

with probabilistic arithmetic in chapter

There are many problems arising in the course of our study of probabilistic

arithmetic and in this thesis we present solutions to some of them However many

of them remain topics for future research and wegive some more details on these in

the nal chapter of this thesis We note here that our idea of probabilistic arithmetic

can b e considered to b e a natural generalisation of interval arithmetic which

works entirely in terms of the supp orts of the distributions of the variables involved

We discuss this connexion in more detail later in this thesis

The rest of this chapter gives an overview of the results contained in this thesis

section a brief decription of the structure of the thesis and suggestions on how

to read it section and a few standard notational conventions section

Outline of Results

Some of the sp ecic results obtained in this thesis the highlights are now describ ed

The general structure of the thesis is describ ed in section

Dierent Metho ds for Calculating Convolutions and the Laguerre

Transform Metho d

In chapter we study a variety of metho ds for calculating convolutions of proba

bility distributions Amongst these are the Laguerre transform metho ds develop ed

by Keilson and Nunn We consider the p ossibility of using this metho d for

op erations other than addition and subtraction of random variables Although we

develop a numb er of new results these turn out to b e of little practical value b ecause

of the computational complexity of the formulae involved

The Dep endency Bounds and Numerical Metho ds of Calculating

Them

The most interesting new techniques develop ed in this thesis are those for numer

ically calculating dep endency b ounds We use the results of Frank Nelsen and

Schweizer who sho w that

ldbF F z sup maxF xF y

X Y X Y

xy z

and

udb F F z inf minF xF y

X Y X Y

xy z

for certain classes of op erations Thenby making use of a duality result to express

these b ounds in terms of the inverses of the distribution functions the quantiles we

develop an ecient and accurate metho d for calculating the dep endency b ounds A

numb er of extensions to the dep endency b ounds are also considered If some infor

mation is known ab out the joint distribution then tighter b ounds can b e calculated

All this material is rep orted in chapter

Precursors Multiple Discoveries and Relationships with Fuzzy

Sets

In the course of reviewing previously published material on the topics covered in

this thesis a numb er of indep endentandmultiple discoveries and precursors were

found For example George Bo ole discussed the analogue of dep endency b ounds

for events in his Investigation of the Laws of Thought These are b ounds on

the probability of conjunction and disjunction of random events They are usually

asso ciated with Frechet who rightly credits their original intro duction to Bo ole

Bo oles work on these b ounds as lower and upp er probabilities has recently seen a

revival of interest in the area of exp ert systems whichhave to deal with uncertain

information In some cases recent authors are unaware of some of the previous work

in the eld We discuss this material in detail in section There are

several other instances of multiple discoveries suchasRuschendorf Makarov

and F rank Nelsen and Schweizer on the dep endency b ounds for random

variables Another example is the duality result we use for numerical calculation of

the dep endency b ounds This has b een presented in various degrees of generality

byFrank and Schweizer Sherw ood and Taylor Hohle Fenchel

see Bellman and Karush Nguyen and Mizumota and T anaka

We discuss this in section Of course these multiple discoveries should not b e

considered surprising esp ecially given the arguments of Lamb and Easton on the

pattern of scientic progress

The Inverse and Determinant of a Random Matrix

Chapter is concerned with a new result on the inverse and determinant of a ran

dom matrix Inverses and determinants of interval matrices matrices with interval

co ecients have b een studied in the literature and so we examined the eect of

interpreting an interval as a uniformly distributed random variable Apart from

some interesting conclusions in this regard the chapter shows the disadvantage of

always seeking analytical results the formula for the density of the determinantis

quite complex and it is necessary to write a computer program in order to determine

its sp ecic values This is one of the arguments of this thesis When sp ecic values

of distributions are required rather than general prop erties it makes more sense

to accept the need for numerical calculations at the outset rather than doggedly

striving for analytical results which are often practically useless

A Limiting Result for Dep endency Bounds

Perhaps the most interesting purely mathematical result in this thesis is that re

p orted in chapter There weshow that the dep endency b ounds of the normalised

X converge to step functions The p osition of the step functions de sum

p ends solely on the supp ort of distribution functions of the random variables fX g

The metho d used to prove this result Tconjugate transforms has further p oten

tial applications These are discussed in chapters and The interpretation of

the result is that in some cases if it is necessary to use dep endency b ounds b e

cause of lack of further dep endence information then no further information will

b e obtained b eyond that obtained by using interval arithmetic on the supp orts of

the distributions

Thesis Structure

Each of the chapters in this thesis is essentially selfcontained and can thus b e

read indep endently of the others although chapter should b e read b efore reading

chapters In fact eachchapter was written as a pap er details of which are given

in the Publications section at the b eginning of the thesis and at the b eginning

of eachchapter For this reason there is some slight rep etition of material in some

places We feel this will in fact b e of b enet to the reader as most of the rep eated

material is concerned with technical denitions which are dicult to rememb er for

the entire length of the thesis

The remaining chapters of the thesis are summarised b elow

Chapter is a survey of dierent metho ds for numerically calculating convolu

tions of probability distributions As well as reviewing previous work we present

some new results on the Laguerre transform metho d and the histogram metho d We

also p oint out some interesting connexions with metho ds used in metrology

Chapter is the technical core of the thesis Most of the other chapters are

motivated by it In it we fully develop the idea of dep endency b ounds and showhow

they can b e calculated numericallyAnumb er of examples are given Furthermore

a metho d for calculating ordinary convolutions is presented This uses the

numerical representation develop ed in order to calculate dep endency b ounds and the

result is that this metho d has a number of advantages over the metho ds describ ed

in chapter not the least of which is its simplicity

Chapter which is the longest in this thesis describ es a large number of inter

relationships b etween our dep endency b ound metho ds and other ideas Amongst

other things we discuss the Bo oleFrechet b ounds graph theoretical metho ds lower

and upp er probabilities and fuzzy arithmetic This latter item is of considerable in

terest We showhow the rules for combining fuzzy numbers are very closely related

to the dep endency b ound formulae

Chapter is devoted to the pro of and exp osition of a limiting theorem for de

p endency b ounds We prove a generalisation of the lawoflargenumb ers under

no indep endence assumptions Weshow that convergence to a wide range of dis

tributions is p ossible under the constraint that the restrictions on the supp orts as

calculated byinterval arithmetic are not violated

Chapter simply represents the result of chapter in terms of fuzzy variables

Thus weshowalawoflargenumb ers for fuzzy variables under a general tnorm

extension principle The meaning of this sentence is explained in the intro duction

to chapter

Chapter presents a new result on the distribution of the inverse and determinant

of a random matrix We derive explicit formulae for p erhaps the simplest case of

a random matrix a matrix with indep endent elements all having a uniform

distribution on Surprisingly enough this result do es not seem to have app eared

in the literature b efore Our motivation for deriving it was to examine the eect

of interpreting an interval in interval arithmetic as a uniformly distributed random

variable

Finally chapter drawsanumb er of conclusions from the work presented here

and providesanumb er of suggestions for future research

Notational Conventions

Equations and sections are numb ered bychapter Thus refers to the th

numb ered equation in section which is the third section of chapter Paren

thesised numb ers always refer to equations Numb ers enclosed in square brackets

such as denote references to items in the reference list in chapter Closed

intervals such as fxj x g are also denoted in this manner but the

context makes clear what is meant

Other mathematical notations are generally standard We list the following which

may otherwise cause some confusion

inf inmum greatest lower b ound

sup supremum least upp er b ound

The set of real numb ers

The set of extended real numb ers f g

df X The distribution function of X

for functions p ointwise inequality

supp The support of a function

Ran The range of a function

Dom The domain of a function

a b The halfop en interval fxj a x bg

jf gj Cardinality of a set

The null set

n Settheoretic dierence

i If and only if

End of pro of

Other notations are intro duced where needed

Several algorithms are presented in this thesis We use the syntax of the C

programming language with the exception that denotes assignmen t and

equality

Chapter

Numerical Metho ds for

Calculating Convolutions of

Probability Distributions

Questions on local probability and mean values areof

course reducible by the employment of Cartesian or other

coordinates to multiple integrals The intricacy and

diculty to be encounteredindealing with such multiple

integrals and their limits is so great that little success could

beexpected in attacking questions directly by this method

MW Crofton

Intro duction Aim and Analytical Metho ds

The problem of calculating convolutions of probability distribution functions arises

in a wide range of applications where distributions of functions of random variables

are required In many cases analytical solutions are intractable and so numerical

metho ds are used This chapter will survey the numerical metho ds that have b een

presented to date This will form a suitable background for chapter where we

present a new metho d for calculating convolutions of distribution functions We

shall see that our new metho d has a numb er of asp ects in common with the metho ds

describ ed in the currentchapter but that it also has a numb er of advantages

We will generally restrict ourselves to the problem at hand and not discuss the

applications in whichconvolutions arise Generally we adopt the viewp ointofvon

Mises pwho to ok the attitude that

the exclusive purp ose of probabilitytheory is to determine from the

given probabilities in a numb er of initial collectives the probabilities in

a new collective derived from the initial one

The determination of the initial probabilities is the province of statistics and wedo

not consider it further

History Motivation and Outline

History of the Problem

It was recognised long ago that if the inputs to some calculation are random then

the nal result is likely to b e random also Sheynin pcites a th cen tury

commentary byGaneza on the th century Indian writing Lilavati byBhascara

On page of this there is a discussion of the determination of an area of

a rectangle when the length and breadth are not known exactly and there is an

implicit recognition that the area will b e only approximate It is suggested there

that mean values should b e used in order to provide a b etter estimate than single

sample values Of course other examples can easily b e found However the statement

of the problem in terms of determining distributions of functions of random variables

could not have o ccurred until last century

The p ointisthatlaws of distributions of functions of random quantities

even the simplest the linear functions could not have b een considered

from a general p oint of view at least until distribution functions them

selves b egan to b e considered per se p

Thus although it could b e hardly doubted that Laplace in his Theorie Analytique

des Probabilites w ould have b een able to transform distributions from one

interval to another and from one argumentto another p one

of the rst explicit transformations in terms of distributions app ears to have b een

due to Poisson in Sheynin p says of a problem considered there

that

elementary as it is this problem seems to b e one of the rst in whic h

densities were treated as purely mathematical ob jects

Another early example which arose in a completely dierentcontext concerns

geometrical probability Sylvesters problem whilst not couched in terms

of functions of random variables nor for that matter solved in such a fashion is

still an example of the sort of problem we are concerned with

The Aim of this Chapter

So much for the history of the problem We will now outline the purp ose of the

presentchapter

Until recently most attempts at the determination of distributions of functions of

random variables haveentailed the search for analytical solutions In other words

a formula for the required distribution was sought Quite apart from the severe

diculties encountered in this approach see section b elow a new formula

needs to b e determined for each new problem Thus whilst for some restricted

classes of problems such as pro ducts and quotients of indep endent random variables with standard distributions the main results can b e tabulated in general

it is not practicable to give a list of such o ccasional results for it is clearly p ossible

to invent further variations at will p A review of a numb er of exact

analytical techniques for determining distributions arising in multivariate statistics

is given in

An alternative to the analytical approachistousenumerical metho ds for cal

culating the required distributions by using computer programs Note that suchan

approach should not necessarily b e considered second b est compared to an exact

analytical result This is b ecause the resulting formulae are sometimes so complex

eg innite series of transcendental functions that a computer program is needed

to calculate the sp ecic values and to determine the b ehaviour of the distribution

Thus we are really no worse o if we decide to use the computer from the outset

The study of these numerical metho ds is the fo cus of the presentchapter

Organisation of the Rest of the Chapter

The rest of this chapter is organized as follows The remainder of this intro ductory

section is devoted to a review of analytical metho ds including integral transforms

and the p ossible direct numerical calculation of these Section summarises a

numb er of dierent metho ds that have b een prop osed including the use of moments

Sections and lo ok in rather more detail at two metho ds whichwould app ear to

b e more promising We examine the Laguerre transform in section and lo ok at

the Histogram or discrete probability distribution metho d in section Section

examines interval arithmetic and metho ds used for the propagation of errors in

metrology

Exact Analytical Results

There is a well known general solution to the distribution of functions of random

variables in terms of the Jacobian of transformation Wewillnow briey present

this result for a function of only tworandomvariables We restrict ourselves to

this sp ecial case b ecause we are mainly interested in functions of only two random

variables such as apply to the four arithmetic op erations and b ecause there are

notational diculties in presenting the full general result carefully There is no real

conceptual diculty in extending the results to functions of N random variables

The General Solution in terms of the Jacobian

Let Z g X Y and W hX Y betwo functions of the two random variables X

and Y whichhavea joint probability density f We wish to determine the joint

density f of Z and W This is given by p

f x y f x y

XY XY k k

f z w

jJ x x j jJ x y j

k k

where x y x y are the k real solutions to the pair of equations

k k

g x y z and hx y w

in terms of z and w where x is the absolute value of x In other words

g x y z and hx y w

i i i i

for i k and x x y y for i j The term J x y is the Jacobian of

i j i j

transformation and is given by

g g

x y

xy xy

J x y det

h h

x y

xy xy

g g

where the notation means that the partial derivative is evaluated at x y

x x

The density f is equal to zero for anyz w such that has no real solutions

The use of Auxiliary Variables

We are often interested in m functions of n random variables with m n In the

presentchapter we are only concerned with m and usually n In order to

use the ab ove metho d in this situation it is necessary to pro ceed as follows

Let m but consider general nWrite

Z g X X

for the function weareinterested in It is necessary to dene n m auxiliary

functions in the following manner

Z X

n n

The use of will give f In order to determine f weneedtocalculate

Z Z Z Z

Z Z

dz f z z f z

j Z Z Z n Z

The integral in is almost invariably the cause of diculties encountered in

analyticallyFor practical problems n can b e quite large see for determining f

example where n and thus a fold integral needs

to b e evaluated Even when the integrals are tractable they can b e exceedingly

tedious see the page calculation in app endix IVwhic h actually contains

an error that simplies the calculation Somewhat more reasonable examples can

b e found in In section weshowhow formulae and result

in the standard convolution integrals for arithmetic functions of random variables

The numerical integration of the Jacobian of transformation has b een considered

byCookandDowns

Historical Remarks

The use of the Jacobian of transformation in the ab ovemannerwas anticipated by

Gauss in in his Theoria Combinationis p where he considered m

and assumed Jalways and thus omitted the jjop eration in The general

solution was rst given by Nasimov in It was subsequently studied by

Poincare and Lammel see the fo otnote on page of

Sp ecialised Formulae for Convolutions

When the function g ofthetwo random variables X and Y is one of the four

arithmetic op erations we obtain the four convolution equations b elow

Z X Y f z f z x x dx

Z XY

f z x x dx Z X Y f z

XY Z

Z X Y f z f z x x dx

Z XY

jxj

Z X Y f z jxjf zx x dx

Z XY

When X and Y are indep endent these reduce to

Z X Y f z f z xf x dx

Z X Y

f z xf x dx Z X Y f z

X Y Z

Z X Y f z f zxf x dx

Z X Y

jxj

Z X Y f z jxjf zxf x dx

Z X Y

These latter equations will b e our main but not exclusive concern in

this chapter We shall refer to them resp ectively as sum dierence pro duct and

quotient convolutions The lesser known pro duct and quotientconvolutions would

app ear to have rst b een published in Note that equations can

b e written in terms of cumulative distribution functions as the Leb esgueStieltjes

integral whichalways exists

dF u v

Lfxg

where Lfxg fu v j u v Lu v xg

The idea of convolution of probability distributions has b een generalised in

anumber of ways For example Urbanik has studied a dieren ttyp e of

convolution to those presented ab ove His generalised convolutions are also briey

mentioned bySchweizer and Sklar

Integral Transforms

Denitions

Wenow consider the relationships b etween and the Fourier and Mellin

transforms The Fourier transform of a function f x is dened by

itx

F f x f xe dx

This is a sp ecialisation of the Laplace transform

L f x f xe dx

where s it The Mellin transform is dened by

M f x f xx dx

The Mellin transform can b e derived from the Laplace transform by a logarithmic

change of variables Note that the Mellin transform is only dened for functions

with domain The multivariate extensions of these transforms are

Z Z

it x

j j

dx e F f x x f x x

j t n n

where t t t and

Z Z

f x x x dx M f x x

n j s n

where s s s

The Calculation of Distributions of Sums and Pro ducts

When these transforms exist they can b e used to determine the distribution of arith

metic functions of random variables as follows If X j n are indep endent

x resp ectivelyandif random variables with densities f

j X

Y X

then

F f y x F f

t Y j t X

If the X are not indep endent and havea joint probability density f x then

j X

F f y F f xj

r Y t X

t t t r

Calculation of dierences can b e accomplished by setting X X If

Y X

then if X are all indep endent

M f y M f x

s Y s X j

whilst if dep endent

M f y M f xj

s Y r X

r r r s

If Y X X then

M f y M f xj

s Y r X

r s r s

and if X and X are indep endent

M f y M f x j M f x j

s Y r X r X

r s r s

Thus if one has a means of inverting the Fourier and Mellin transforms one can

calculate the distributions of sums dierences pro ducts and quotients of random

variables Note that wehave omitted anymention of conditions for the existence of

these transforms and of the uniqueness of the inverse transform More details can

b e found in

We note that the Fourier transform has b een used in statistics for determining

the sums of random variables for quite some time Zolotarev attributes its

intro duction to Lyapunov although it app ears pthat Gauss knew of its

applicability in The widespread use byPoisson and others did not o ccur until

some time later Fourier transforms are usually called characteristic functions in

probability theory a name rst used byPoincare in not as Cupp ens

suggests b yLevy in The Mellin transform has b een used in

probability theory since at least

Diculties in using Integral Transforms

Although the use of integral transforms do es seem promising there remains the

problem of inverting the transform This is usually the most dicult part Indeed

most of the technical diculties encountered by Springer in his b o ok o ccur

in the inversion of the transforms In a numb er of cases the analytical inversion

formulas may b e used These are

f x e L f xds

and

x M f xds f x

However these formulae are of little value for the development of a general numerical

metho d for calculating distributions of functions of random variables In their stead

approximate numerical metho ds which can b e readily implemented on a computer

need to b e considered chapter

The Hfunction Metho d

One wayofavoiding the necessity of calculating the inverse Mellin transform is to

consider the Mellin transform of a very general function which includes as sp ecial

cases all the actual functions to b e encountered The H function whichwas

intro duced byFox contains as sp ecial cases nearly all the sp ecial functions

of applied mathematics The signicance of the H function in the present context

is that multiplication by a suitable constant to make the integral over the domain

of the function equal to allows one to consider H function distributions which

include many classical univariate probability distributions as sp ecial cases These

are all only dened on the p ositive real line and include the gamma exp onential

Chisquare Weibull Rayleigh Maxwell halfnormal uniform halfCauchyhalf

Student F Beta and Bessel distributions Representation by H functions is of use

in the context of probabilistic arithmetic b ecauses the probability densities of pro d

ucts quotients and rational p owers of indep endent H function random variables

are also H function random variables Thus if one can numerically

calculate the H function inversion integral that is the contour integral dening the

H function one can determine the probability distribution of pro ducts quotients

and rational p owers of random variables whichhaveany of the distributions listed

ab ove

Metho ds of numerically evaluating the H function inversion integral are discussed

in chapter of and are based on the w ork of Lovett and Eldred

Eldreds metho d is simplied by Co ok and Barnes whic h is however still

rather complicated By combining their technique of inverting the Mellin transform

with a numerical metho d of inverting the Laplace transform describ ed by Crump

Co ok and Barnes pro duce a metho d for calculating the distribution of the

sum of pro ducts quotients and rational p owers of H function random variables In

they presen ta fortran program implementing the algorithm For successful

op eration this program requires a numb er of technical parameters to b e sp ecied

and the authors give some suggested values They say page that the calculation

of the p df of WX Y where W X and Y were H function random variables

required ab out seconds inputoutput and CPU time on a Cyb er

Other metho ds for the numerical inversion of Laplace and Mellin transforms are

given in The necessit y of using the Laplace transform arises b e

cause the sum or dierence convolution of two or more H function distributions is

not an H function distribution Whilst an analytical expression exists for the convo

lution it app ears to o complicated to b e of any use in probabilistic arithmetic

Springer suggests using Fourier transforms to p erform the convolution Although

the Fourier transform of an H function distribution is known equation

inverting the Fourier transform maybeintractable analytically In section

Springer refers to unpublished work of Carter on a n umerical technique for de

termining the moments of the distribution of the sum of indep endent H function

random variables This would allowapproximations to the distribution to b e ob

tained using standard techniques for determining distributions from moments see

section b elow

Miscellaneous Numerical Metho ds for Calculating Con

volutions

The problem of numerically calculating convolutions is related to the general prob

lem of approximating distributions whicharisesinmany areas where the digital

computer is not involved A go o d survey of the general problem is given byBow

man and Shenton

Normal Approximations and other Parameterised Distributions

Perhaps the simplest and most widespread metho d of calculating the distributions

of certain functions of random variables is to use a normal approximation to the

distributions involved The advantage of doing this is that normal distributions are

closed under sum and dierence convolutions Thus if df X N and

X X

df Y N then df X Y N if X and Y

Y Y X Y

X Y

are indep endent Therefore we could represent the distributions simply by and

for the purp oses of numerical calculation This idea has b een used byPearl

and Sobiera jski and is discussed b y Corsi in Note however that neither

df XY or df X Y are normal in general

When using normal approximations two dierent approaches can b e taken in

order to interpret the nal results The rst approach is simply to assume that

the distributions involved are normal This is often done in error analysis It is an

unjustied assumption in many cases not only in the theory of errors but in other

areas as well The second approach includes the admission that the distributions

are in fact nonnormal but uses normal approximations for the sake of calculation

Whilst this is often acceptable it is certainly not a suitable metho dology for a

general probabilistic arithmetic

One problem in tting a standard distribution which arises more often than is

recognised is describ ed by Greenb erg as follo ws

The analyst thus has great latitude in cho osing a distribution to t his

data naturally he will select one that is convenienttowork with and

easily manipulated The only serious dierences b etween the data and

the selected distribution will probably b e in the tails where relatively

few if any of the observationswill lie Ho wever it is in these tails that

the events of interest o ccur the large delays the long queue lengths etc

Thus anyinvestigation of these events either by analytic means or by

simulation esp ecially if imp ortance sampling is used to obtain a larger

representation of the values in the tail is b ound to b e greatly aected

by the distribution chosen and the distribution must b e chosen with

little or no representation in the region of most interest

The fact that the normal distribution may b e a go o d match to some p opulation

distribution everywhere except in the tails is the sub ject of Bagnolds pap er

See also the reply by Rothschild The eect of the extreme tails on a nal

result is quite pronounced in some cases In chapter weshow that the application of

one metho d for overcoming a problem we call dep endency error see chapter gives

results that dep end entirely on the extreme tail b ehaviour in certain circumstances

Rather than using a normal distribution a more exible parameterised family of

distributions could b e used Recall that wehave already examined section

the use of H function distributions A simpler alternativeistousesay the Pear

sonian curves Whilst these curves provide a much b etter t to a wide

range of distributions compared with the normal distribution their use as a gen

eral metho d for calculating distributions of functions of random variables is severely

restricted by the fact that apart from certain sp ecial cases the distribution

of a function of random variables is not available in terms of the parameters of the

distributions involved Even when the distribution is available it is not necessar

ily Pearsonian In other words the family is not closed under convolutions

General conditions that need to b e satised for a parameterised family to b e closed

under sum convolutions are given by Crow and these are restrictive enough

for us to discard the idea

Thus we reject b oth normal and other parameterised distributions as b eing un

suitable for our purp oses Not only are the assumptions often invalid but we cannot

calculate the convolutions of interest in terms of the parameters involved

Metho ds Based on Moments

Denitions and Basic Approach

Another simple metho d which deserves consideration is representation using mo

ments The nth central moment of a random variable X with distribution function

F is dened by

n n

x dF x E X

assuming the integral exists The method of moments intro duced byTchebyshev

and Markov see app endix I I of entails calculating results in terms of mo

ments of distributions when exact distributional results are unavailable Given the

moments of a distribution one can either t a distribution using the techniques de

scrib ed in or one can use the Tchebyshev inequalities or their generalisations

to determine distributional results Unfortunately there are prob

lems asso ciated with b oth of these metho ds Firstly the moments do not always

determine a distribution exactly so that t wo dierent distributions can have

all moments identical and secondlytheTchebyshev inequalities are often quite lo ose

There are also a numb er of ad ho c metho ds which use moments For example

Broadb ent suggested appro ximating the distributions of pro ducts and quotients

by lognormal distributions tted to the moments This is unsuitable for our needs

b ecause of restrictions on the class of distributions that can b e accommo dated

Even if these problems are ignored there are still substantial diculties in using

moments as a basis for probabilistic arithmetic This is b ecause although there is

a simple and exact formula for the moments of a sum of two indep endent random

variables

n s ns

X Y X Y

see p there are no such simple results for pro ducts and quotients In

deed for the case of the quotient the moments maynoteven exist It is easy to

provethatifZ X Y where X and Y are random variables with densities f

toexistitis and f b ounded and continuous at the origin then in order for

X Y

necessary and sucient for to exist and for f to have a zero of order n at

the origin In some cases the o dd order moments exist as Cauchy principal values

Approximate Formulae for Moments of Functions of Random Variables

Whilst exact formulae for moments of pro ducts and quotients do not exist there

are useful approximations whichhave found wide application in applied probability

theory There are two dierentways of developing approximate formulae The rst

metho d which is based on the binomial expansion is discussed in

and is of little use More useful is the metho d based on partial dierentiation of the

function involved This was studied in great detail byTukey in The basic

idea is to use a truncated Taylor series expansion of the function in question in order

to linearise any nonlinearities ab out exp ected values Tukey says that the results

obtained with only second order expansions are surprisingly accurate Generally

the approximate formulae are more accurate if the random variables have small

co ecients of variation The formulae are known in metrology as the general error

propagation laws The formulae for an arbitrary function of random variables

are given by Hahn and Shapiro pas follo ws Let Z hX X bea

function of n indep endent random variables Then

Z X X

and

n n

X X

h h h

Z X X

i i

X X X

i i

where as usual we write for and the partial derivatives are evaluated at their

X X

exp ected values There are related but more complex formulae in terms of cross

moments for cases when the X are not indep endent Note the following sp ecial

cases If Z hX X X X

Z X X

X Z X

If Z hX X X X

Z X X

Z X X X X X X

If Z hX X X X

X X X X X X

X X X X

There are many p ossible variations on these formulae For example the derivation

of an expression for the variance of pro ducts was the sub ject of the three pap ers

Uses of These Approximate Formulae

The ab ove approximate formulae formed the basis of Metrop oliss signicance arith

metic The name comes from his use of the relationship

j j

s log

in order to determine the numb er of signicant digits s in a calculated result

Metrop olis stated that

an imp ortant observation is that do not dep end on the de

tailed structure of the distribution function asso ciated with each op erand

apart from the natural assumption that the rst and second moments ex

ist p

He also noted that the formulae are only usable when the co ecients of variation

are small

Although the ab oveapproximate formulae are not entirely suitable for a com

pletely general probabilistic arithmetic they are very useful in sp ecic applications

and have b een used widely See the examples in Moment based metho ds

have found wide application in p ower systems analysis The use of cumulants which

are directly related to moments has b ecome p opular in p ower systems analysis

Representation of distributions by their cumulants has the ad

vantage that sum convolutions can b e calculated by the simple p ointwise addition of

cumulants The resulting distribution is then calculated by using a GramCharlier

expansion see section b elow Whilst this metho d may seem elegant there

are many problems even within the restricted application area envisaged for it by

its develop ers Apart from a numb er of sp ecic problems caused by the area of

application whichwould b e encountered by all metho ds of probabilistic arithmetic

there is the diculty of representing distributions that are far from normal In gen

eral there is an unknown approximation error In some cases the tted distribution

can dier considerably from the measured distributions esp ecially in the tails gure

A moment based metho d has also b een suggested byPetkovic who de

velop ed an idea he calls probable intervals These are dened for some random

variable X with density f having nite supp ort a b as the ordered pair q

X X

The parameter qischosen suchthatq b a Petkovicshowed that

if f is continuous and unimo dal then r r r f and

X X X X

X X

r where r b a He suggests that these results can b e used in order

to cho ose q Once q is chosen then the probable intervals can b e combined using

interval arithmetic see section esp ecially subsection

NonLinear Transformations

Another metho d for calculating distributions of functions of random variables is to

use a nonlinear transformation to convert some arbitrary distribution into saya

normal distribution The sp ecial prop erties of the normal distribution can then b e

used to determine the distribution of the appropriate function of the transformed

random variables The desired result is then calculated by using an inverse trans

formation Transformation to normalitywas originally used as an aid to quadrature

It has since b een widely used in reliability assessment although not in a very

rigorous manner pxxi

A recent application of the transformation idea is rep orted byPetrina et al in

Using the probabilityintegral transform to convert a random variable X with

distribution function F into a random variable Y with distribution function F

X Y

ie Y F F X as their starting p oint Petrina et al pro ceed as follows

Hastings approximation to the normal distribution function is used to ev aluate

F They then determine an approximate overall transforming function by tting

alow degree p olynomial to a discrete mapping from X to Y determined by using

sample values of X Once X has b een transformed to Y and in practice there are

several X s and several Y s the original problem can b e solved in terms of normal

random variables For the application considered byPetrina et al a third degree

p olynomial approximation app ears to give quite accurate results The actual de

tails of this approach are more intricate than indicated here Whilst the technique

app ears quite useful in a numb er of circumstances there seems little hop e of devel

oping a general probabilistic arithmetic in terms of it The idea of using p olynomial

transformations to normality is also discussed in section

Direct Sampling of the Densities for Sum and Dierence Convo

lutions

Allan et al in an examination of probabilistic p ower systems analysis consid

ered the distribution of the sum of n indep endent continuous Gaussian distributed

random variables added to the sum of n discrete random variables with binomial

distributions They used the results mentioned earlier for the sum of Gaussian ran

dom variables and found the distribution of the sum of the discrete random variables

bynumerically convolving their distributions using discrete convolution algorithms

The combination of these two results is a convolution of a Gaussian distribution

with a series of Dirac delta distributions This is simply a sup erp osition of Gaussian

distributions

The evaluation of the discrete convolution arising from the addition or subtrac

tion of discrete random variables can also b e accomplished by means of fast Fourier

transform algorithms Indeed any of the wide variety of fast discrete convo

lution algorithms can b e used These fast algorithms are not applicable

outside the case where the random variables can only takevalues on an equispaced

grid A trivial example of howthistechnique is used is given in

The case of continuous convolutions is more dicult A naive approachwould b e

to approximate the continuous density f xby the sampled discrete representation

f kT where T is the sample spacing This is the approachusedbyAckroyd

By simply sampling a continuous density with a nite numb er of samples approxi

mations have to b e made There are twotyp es of approximation errors The rst

whichwe call sampling error is due to the density not b eing bandlimited and thus

the conditions under which the sampling theorem holds are not satised

The second form of approximation error is caused by the requirement of using only a

nite numb er of samples This makes the exact rep esentation of densities with in

nite or semiinnite supp ort imp ossible We call this truncation error An analysis

of sampling and truncation errors for general functions not necessarily probability

densities is given byPap oulis An analysis with particular reference to proba

bility densities is given byWidrow in his consideration of amplitude quan tized

sampled data systems As well as discussing the well known Shannon sampling the

orem Widrowshows that if the probability density is bandlimited to

W that is if the supp ort of the characteristic function or Fourier transform is con

tained within WW then the set of samples spaced W apart allows recovery

of all the moments but not the densitywhich requires samples only W apart

The SkinnerAckroyd Metho d

While it is p ossible as mentioned ab ove to derive estimates for the error incurred

in sampling the distribution function it is preferable to provide strict upp er and

lower b ounds on the error automatically as the calculation is p erformed This has

b een done byAckroyd and Kanyangarara They mo died techniques presented

in b y using an idea originally prop osed by Skinner They sample the

cumulative distribution rather than the probability density The signicance of this

is that upp er and lower b ounds on the error caused by the sampling can b e readily

derived

Given a cumulative distribution F of a random variable X two discrete ap

proximations are formed which are lower and upp er b ounds on F

F x F x F x x

X X

and F are dened by The b ounds F

xF kT kT x k T F

X X

Figure Illustration of Ackroyds metho d of discretising a continuous probability dis

tribution function The solid line is F x and the two dashed lines are F xand F x

resp ectively

and

F x F kT k T xkT

X X

and can b e understo o d by consideration of gure The corresp onding probability

densities f and f are obtained by dierencing and are related by

f k T kT f

Denoting nfold convolution by a sup erscript n Ackroyd and Kanyangarara

showed that

n n

F F x F x x

X X

for all x For the sp ecial case of sumconvolutions it is p ossible to use the

relationship

f kT f k nT

in order to sp eed the calculation of these b ounds by calculating

k k n

X X

mT F x mT kT x k T f f

X X

m m

The tightness of the b ounds dep ends up on the size of T and the shap e of F There

is no consideration in this scheme of the truncation error caused by representing a

distribution with innite supp ort although this error can b e made arbitrarily small

by increasing the numb er of samples In chapter wedevelop a metho d similar

to this where the quantiles F are uniformly sampled and which has considerable

advantages over the metho d describ ed here

Of all the techniques based on converting a continuous convolution to a dis

crete convolution by sampling those which can b e p erformed eciently with fast

Fourier transform algorithms use uniform equispaced sampling A dif

ferent metho d of sampling which has b een applied to the numerical calculation of

characteristic functions is given by Jones and Lotwick This has b een applied

to a metho d of nonparametric densit y estimation presented bySilverman

Their metho d reduces the error incurred by the sampling pro cess by using a

dierent metho d of assigning the sample values We note that for some distributions

nonuniform sampling is b etter although the practical v alue of this requires

further investigation

Spline Based Metho ds

Yet another approach to computing sum convolutions of probability densities was

intro duced byCleroux and McConalogue Their idea is based on the piece

wise representation of the cumulative distribution function of always p ositive random

variables by cubic splines Cubic splines are used as they havewell b ehaved rst

derivatives the probability density In a fortran program is presented which

can b e used to approximate convolutions of densities that are b ounded analytic and

have supp ort only on the p ositive real line The algorithm app ears to give accurate

results although estimates of the errors have not b een determined a useful error

analysis is not practicable p The distribution function is represented by

m equally spaced samples supplemented by m spline co ecients Approximations to

convolutions of distributions so represented are obtained in terms of the representing

values The details of this are quite messy and are omitted here

One of the restrictions on the class of distributions to which the technique can

b e applied is removed in a generalisation presented in This generalisation

overcomes problems with cubic spline approximations to innite singularities or

where the function b eing approximated and all its derivatives vanish at the origin

A discussion of the original metho d this generalisation and a comparison with three

other metho ds which are often used in solving renewal equations the application

for which this technique was originally develop ed is given by Baxter in

Nevertheless this generalised technique is still quite restricted esp ecially since

it deals only with sum convolutions and b ecause we do not knowhow accurate

the nal results are The use of splines in conjunction with the histogram metho d

is mentioned briey in section b elow A relationship b etween splines and

convolutions was also the sub ject of Sakais pap er

Laguerre Transforms and Other Orthonormal Trans

forms

Orthonormal expansions are logical candidates for representing probability distribu

tions so that convolutions can b e calculated numerically In this section we will lo ok

at one particular orthonormal expansion that has b een widely studied in this regard

and is based on Laguerre p olynomials Laguerre p olynomials have b een used in a

numb er of areas such as signal pro cessing and system identication as well as b eing

used as a general means for representing continuous functions on a digital com

puter The Laguerre

transform we examine was develop ed by Keilson and others for calculating

the sum convolutions and other op erations of probability densities The original

motivation was to push back the dreaded Laplacian curtain p This

is the name given to the fact that many results in queueing theory can only b e

expressed in terms of Laplace transforms The technique has b een used success

fully in a numb er of applications In section b elowwe will briey outline the

technique and showhow it can b e used to calculate the sum and dierence convo

lutions No work has b een presented to date on using Keilsons Laguerre transform

to calculate pro duct and quotientconvolutions Accordingly in sections

we examine three p ossible approaches to achieve this We obtain a numb er of new

results but ultimately nd that the metho ds app ear intractable In section

we briey lo ok at some other Laguerre transforms and consider their convolution

relations Our conclusions on the suitability of orthonormal expansions particularly

Laguerre transforms as a metho d of representing probability densities suitable for

numerically calculating convolutions are given in section

Orthonormal transforms other than Laguerre transforms have b een used widely

in other areas Wenow briey consider some whichmight b e useful for calculat

ing convolutions Hermite p olynomials form an orthonormal set on with weight

function expx They have b een used to dene Hermite transforms

Debnath proved a complicated convolution formula for the Hermite

transform of o dd order The convolution in question has no relation to the sum or

pro duct convolutions weareinterested in and is thus of no use to us The convo

lution structure of orthogonal transforms based on Jacobi p olynomials orthogonal

on the interval has also b een investigated but these to o are

of no use to us The convolutions are not of the form Other or

thonormal systems such as the one recently discussed by Christov are also

inapplicable to our area of interest Nonorthogonal expansions whichhave some of

the desirable features of orthogonal expansions without the undesirable features

are also p ossible candidates for calculating convolutions See the consideration of

the Mellin transform and pro duct convolution on pages of

The Laguerre Transform Metho d

Denitions

The Laguerre transform metho d is based on the use of the asso ciated Laguerre

functions

xe L x

n n

whichprovide an orthonormal basis in L Here L x is the Laguerre p oly

nomial of degree n which is dened by the Ro drigues formula

x n x

L x e x e

n dx

The p olynomial L x has the explicit form

x L x

k k

An extended set of asso ciated Laguerre functions which allows the representation

of the probability densities of random variables that are not always p ositive can b e

dened by

xU x n

h x

xU x n

where U xforx and U x forx The set fh xg is an

orthonormal basis of L Thus for any f L there is a unique

representation

f x f h x

where the equalityisinthe L sense The co ecients ff g are known as the Laguerre

dagger coecients and are given by the Laguerre transform

f h xf x dx

If jf j pointwise convergence is guaranteed for all x nfgTheinverse

transform is unique if f is continuous

Calculation of Sum and Dierence Convolutions

In order to see howconvolutions are calculable with Laguerre transforms wedene

the two generating functions

n y

f u T f

n u

and

n y

f u uT f f T

u n u

The Laguerre sharp coecients ff g are related to the dagger co ecients by

f f xh x dx f f

n n

where h xh x h x The inverse relation is

n n n

n k

X X

lim f f lim f

j j

k k

j n

j k

e f x dx for the Laplace transform of f then Recalling our notation L f

it can b e shown that

L h e h x dx

s n n

n Res

s s

Equation gives a relationship b etween the sharp generating function and the

bilateral Laplace transform

f T f L

Using this relationship the sum convolution

f g x f x y g y dy

can b e calculated as follows The convolution theorem for Laplace transforms states

that L f g L f L g Therefore T f g T f T g This is equivalent

s s s

u u u

to the discrete convolution

f g f g

n m

which can b e readily calculated on a computer Dierence convolutions can b e

calculated by simply setting g xg xbyswapping the roles of g and g for

n and then calculating a sum convolution In practice it is necessary to

y y

truncate the series ff g and fg g to a nite length This will intro duce errors but

n m

they can b e analysed and controlled

Extensions and Applications

The assumption that f the densityisin L is not always satised If f L then

the Laguerre dagger transform dened by do es not exist However one can

dene a sharp transform which do es exist if f L provided that f L It is

equivalent to the dagger transform when f L

The Laguerre transform metho d has b een successfully applied to a number of

problems in applied probability that require the evaluation of sum convolutions

Some of the applications include a study of the approach to normality in the central

limit theorems and the evaluation of renewal densities The Laguerre

co ecients for a numb er of widely used distributions are given by Sumita in

although not all are given in a closed form We note in passing that whilst it is

p ossible to derive some formulae for the Laguerre transforms of H function random

variables by using the results in these do not app ear to b e of anyvalue

b ecause they are very complicated Further details on the Laguerre transform and

its applications can b e found in

Distributions of Pro ducts and Quotients using the Laguerre and

Mellin Transforms

We will now consider the p ossibility of using the Laguerre transform for evaluat

ing the density of the quotient or the pro duct of tworandomvariables in terms of

the Laguerre co ecients of the op erands Because of the success with whichithas

b een used for sum convolutions a metho d of using it for calculating pro duct and

quotientconvolutions would make the technique rather more complete We should

exp ect that the determination of the quotient or pro duct convolutions will b e rather

more dicult compared to the sum or dierence convolutions b ecause pro duct and

quotient are nonlinear op erations Also whereas the sum convolution has the nice

prop erty that the sum convolution of anytwo b ounded continuous distributions is

b ounded and continuous the same is not true of the pro duct convolutions For ex

ample the pro duct of two standardised Gaussian random variables has a probability

density f x K x where K is the Bessel function of the second kind of

purely imaginary argument This has a singularity at the origin

There do es not app ear to b e anywork on using the Laguerre transform to cal

culate pro duct and quotient convolutions in the literature Sumita and Kijima have

considered the simpler problem of nding the Laguerre transform of the pro duct

of two functions in terms of the Laguerre co ecients of the functions That

is given the Laguerre co ecients of f xandg x they determine the Laguerre

co ecients of f xg x

Drawing on an analogy with the metho d of determining the relationship for the

Laguerre transform co ecients for the sum and hence dierence of two random

variables by considering the Laplace transform of the asso ciated Laguerre functions

we will now examine the unilateral Mellin transform of x If we write xas

n n

x k

x e x

k k

then we can determine the Mellin transform termwise Using equations and

of w ehave

h i

ax k sk

e x a s k M s

Setting a gives

h i

s k M

s n

k k

s j we obtain Noting that s k s

n k

X Y

M e s s j

s n

k k

where c ln Unfortunately this is nowhere near as simple as the Laplace

transform of h x and thus there is no apparent simple relation b etween the

Laguerre co ecients for the pro duct or quotientoftworandomvariables In section

we will briey discuss some other relationships b etween Laguerre p olynomials

and Mellin transforms that have app eared in the literature

Distribution of Pro ducts and Quotients Calculated with the La

guerre Transform Directly

Another p ossible approach to determining the Laguerre co ecients of pro duct and

quotient convolutions is to examine the convolutions directlyWe consider the quo

tient here and weletZ X Y b e the quotientoftwo random variables The

functions f g and h are the densities of X Y and Z resp ectively We assume

X and Y and hence Z are always p ositive thus f xg x hx for

all x This do es not sacrice any generalityaswecanalways determine the

quotientoftwo random variables that are not always p ositiveby considering the

p ositive and negative parts separately and then combining the results in a mixing

op eration

The General Approach

The convolution equation we are studying is

hy jxjf xy g x dx

xf xy g x dx x y

P P P

y y

Let hy h y f x f xandg x g x where

k n m

k k n n m m

y y

fh g ff g and fg g are the resp ective onesided Laguerre dagger co ecients Then

n m

X X

y y xy x

hy f x g e L xy e L x dx

n m

y y

g then the Laguerre expansion converges p ointwise uniformly g and fg If ff

m n

almost everywhere see theorem in the following subsection Assuming this

we can write

X X

y y xy

f hy g xe L xy L x dx

n m

and we need to evaluate the integral

I xe L xy L x dx

n m

Evaluation of the Integral

Gradshteyn and Ryzhik eq giv e

e x L xL x dx

n m

m n bb b b

F m n m n

mn b b b

for Re Reb where F z is the h yp ergeometric function

dened by

z z F z

The generalised Laguerre p olynomial L x is dened by

n x

L x

n m m

Observing that L xL x L x eq we can write the

n n

integrand of I as

h ih i

xy xy

xe L xy L xxe L xy L xy L x L x

n m

If weset b y y and wehave from equation

B y xe L xy L x dx

n m

n mn

y y m n

m n m n F

mn y y

The hyp ergeometric function F z terminates if or are negativeintegers

Since m n and m n we can write

minmn

X Y

z j mj n

F m n m n z

j m n i

i j

We also have when maximinn m

j mj n

j m n

mm m i nn n i

m n m nm n m n i

i i

mm m i nn n i

m n m nm n m n i

m nm n i

m i n i m n

Therefore

n mn

mn y m n

B y

mn m n y

minmn

y m n i

m in ii y

mni

minmn

y m n i

y m in ii y

Recalling equation wehave

I B y B y B y B y

mn mn mn mn

and therefore

X X

y y

hy f g B y B y B y B y

mn mn mn mn

n m

The Result

y y

Equation gives h in terms of ff g and fg gHowever what we really wantis

n m

y y

to determine fh g directly in terms of ff g and fg gWehave

k n m

h hy y dy

X X

y y

B y B y B y B y y dy g f

mn mn mn mn k

m n

n m

where B y isgiven by Even if the inner series in is uni

formly convergent a fact which is not readily apparent this is obviously going

a complicated sum of integrals of the form to b e a rather messy expression for h

r y

e y

dy So although Gradshteyn and Ryzhik eq giv e an ex

pression for this integral in terms of Whittakers function this do es not result in

a simple expression for h Diculties remain even if wenumerically compute the

integral b eforehand This can b e seen by considering

X X

y y

h f g K

mnk

k n m

n m

where K is the precomputed integral If we truncate the Laguerre series by

mnk

using only co ecients then in order to calculate fh g for k in

excess of op erations would b e required

A derivation of the Laguerre co ecients of the product of tworandomvariables

gives a similar result to that obtained ab ove a complicated sum of Whittaker func

tions and is not considered further

The fact that the results wehave obtained are so complicated is surprising when

we consider Feldheims result that the Laguerre p olynomials are the only or

thogonal p olynomials p y with a multiplicative theorem of the form

n nj

p y A p y

n nj j

where A are constants The signicance of Feldheims result can b e seen by com

paring with the pro duct convolution see equation where f xy isa

term in the integrand Nevertheless this is not as simple as equation

Z Z

x x

L tL x t dt L t dt L x L x

m n mn mn mn

whichwas the original motiovation for Keilsons Laguerre transform

Other expansions for pro ducts of Laguerre p olynomials have app eared in the

literature but these are all in terms of Laguerre p olynomials L x

of dierent orders Pro ducts of other orthogonal p olynomials are discussed in

Niukkanen has presen tedavery general result giving the

pro duct of two Laguerre p olynomials in terms of a series of Laguerre p olynomials

These are also presented in terms of Laguerre p olynomials of dierent orders

The co ecients for the Laguerre p olynomials in his expansion are very complicated

expressions in terms of his generalised hyp ergeometric series and do not app ear

to b e of any use here

Distributions of Pro ducts and Quotients using the

Laguerre Transform and LogarithmicExp onential

Transformation of Variables

Instead of determining the distributions of pro ducts and quotients directlyitmay

b e p ossible to use logarithmic and exp onential transformations It can b e shown

that if X is an almost surely p ositive random variable with a probability density

y y

f e The inverse f x then the random variable Y logX has a density g y e

transformation is f xxg log x If we can determine the Laguerre co ecients

for g in terms of the co ecients for f thenwe can calculate the probability density

of pro ducts and quotients of random variables in terms of the Laguerre co ecients of

the op erands by calculating the eect of the logarithmic transformation p erforming

a sum or dierence convolution and then transforming back again In this section

we will derivea formula for the Laguerre co ecients of the logarithm of a random

variable with a given density in terms of the Laguerre co ecients of the densityIt

turns out to b e surprisingly dicult to do this As a consequence this subsection

is rather longer and more intricate than the others

The General Approach

y y y y

Let f x f x where f f x x dx and let g y e f e

n n

n n n

y y y y

g y y dy We require fg g in terms of ff gObvi g y where g

n n

n n n n n

ously

y y y

g e f e y dy

y y y

Let z e y log z andso e and hence dz e dy Then

g f z log z dz

m n

p k

X X X

m n

y z p log z k

f e z e log z dz

p k p k

p m

m n

p k y

X X X

m n f

z k p

e log z z dz

p k p k

p m

If the innite series in is uniformly convergenttoanintegrable function then

wecaninterchange the order of integration and summation and integrate termwise

Equation is equivalentto

y y

f z log z dz g

m n

Sumita page giv es the following theorem regarding the uniform convergence

of the extended Laguerre expansion

Theorem Let f x L have an extendedLaguerreexpansion

h x x f f x

y y

and dene the partial sum S x f h x If ff g then S x

N n N

nN n n

converges to f x pointwise uniformly almost everywhereasN

y y

This theorem implies that if ff g then f x f h xiscontinuous

n n n

for all x b ecause when a sequence of continuous functions suchas S x

converges to f uniformlythen f is continuous The class of functions f for which

the co ecients ff g has not b een determined However wedoknow that if f

is not continuous for all x then ff g page We will assume

that ff g from nowon

It is required that

S x f z log z

m n

N m

converges uniformly to an integrable function for almost all z and for

n This is true if ff g b ecause S xconverges to f z log z

m N

which is continuous and b ounded and S z f z converges uniformly

N m

m m

to f z Regarding the integrability condition we observe that

log z k

log z e log z

k k

z O log z

which approaches as z for any given n and therefore f z log z as

z b ecause f z asz Now f z isintegrable on it is a

probability density and f z for all z Also log z isboundedforall

z Thus f z log z isintegrable on

Evaluation of the Integral

We can nowinterchange the order of integration and summation in giving

n m

k p

X X X

n m

y y z k p

g f e log z z dz

n m

k k p p

m p

Wenowneedtoevaluate the integral

z k p

dz I e log z z

Gradshteyn and Ryzhik eq giv e

x m

x e log x dx

Re Re m

where x is the incomplete gamma function dened by

x e t dt

If wesetm k p p and then is equivalent to

Therefore

I p

Let p p then

I p

We require an expression for

e a x

where x p and c ln Firstly consider

e a x

Erdelyi et al eq p giv e

x x

e a x a e a x

If we write

cx cx x

e a x e e a x

and use the pro duct rule along with we obtain

cx cx cx

e a x c e a xa e a x

e a x There do es not app ear to We actually require an expression for

b e one in the literature Thus wenowprove the following theorem by induction

Theorem

n i

X Y

cx ni cx

e a x c a j e a i x

x i

i j

Proof In order to make the pro of less cumb ersome wedene Gae a x

and D Wehave already seen equation that

D Ga c Gaa Ga

The theorem to b e proved is

n i

X Y

ni n

c Ga i D Ga a j

i j

Thus the induction hyp othesis is

n i

X Y

D c a j Ga i

i j

n i

X Y

c a j Ga i

i j

Firstly observe that for arbitrary co ecients fq g

X X X

D q Ga i q c Ga i a i q Ga i

i i i

This fact along with equations and gives

n i

X Y

c a j Ga ic

i j

n i

X Y

a i Ga i c a j

i j

n i

X Y

c a j Ga i

i j

n i

X Y

Ga i c a j

i j

i n

Y X

c a j Ga i

j i

n i

X Y

Ga i c a j

i j

Ga c a j

c a j Ga n

If weseti k in the second sum so k i this can b e written as

n i

X Y

c a j Ga i

i j

X Y

c a j Ga k

n n

c Ga a j Ga n

n i

X Y

Ga ic a j

i j

a j Ga i c

c Ga a j Ga n

i n

Y X

n n

a j Ga ic

i i

j i

c Ga a j Ga n

i n

Y X

a j Ga ic

j i

c Ga a j Ga n

which equals

The Result

The result we require a closed form for equation is obtained by substituting

into We nd that

k i

X Y

k i p

p ln j i p

i j

Substituting into and gives

n n

k p

X X X

n m

y y

g f

n m

k p k p

m p

k i

X Y

p k i

ln j i p

i j

This can b e written as

n n

X X

g F k g F n

k k

y y

where F k f Gk mand Gk m do es not dep end on f

m m m

i k m

p k

Y X X

m k

k i p

ln j i p Gk m

p i k p

j i

Thus Gk m could b e tabulated for k m NwhereN is the length at

y y

g g and fg whichwe truncate the innite series of Laguerre dagger co ecients ff

n m

These tabulated values could then b e used with equation and the given

y y

Laguerre dagger co ecients ff g to calculate the set fg g

m n

There are problems with this scheme which reduce its practicability Firstly

if N is of the order that has b een used in most of the pap ers on the Laguerre

transform technique cited earlier to then the table of values of Gk m

will contain to entries Secondly the values of Gk m for some

m and k app ear to b e very large and b eyond the range of most mo dern digital

computers which can usually only representnumb ers up to ab out This second

p ointwas discovered up on programming using a sp ecial package of extended

oating p oint arithmetic subroutines whic h is similar to the extendedrange

arithmetic used for calculating Legendre p olynomials in and the lev elindex

system describ ed in and a computational algorithm for the incomplete gamma

function The expression is unfortunately to o complicated to derive

rough asymptotic values for Gk mtoverify the programs correctness We note

that Sumita has encountered problems of excessive magnitudes in the calculation of

the Laguerre co ecients of Gamma densities p

Reducing the Extent of the Laguerre Co ecients

The problems asso ciated with and would b e less severe if the La

guerre co ecients were of smaller extentie if N the numb er of co ecients was

smaller Sumita has sho wn that the extent of the co ecients dep ends heavily

on the concentration ab out zero and the extent of the tails Scaling and exp onen

tial weighting of the density to b e represented can reduce this extent considerably

Scaling involves setting g xf cx for some p ositive constant candweighting

involves setting g xe f x for some p ositive constant Sumita says that a

general pro cedure to nd the Laguerre co ecients of g xfrom the co ecien ts

of f xhas y et to b e develop ed p The following theorem solves this

problem

x y

Theorem Let g xe f cx c f x f xand

n n

y y

g x g x Then if ff g

m m n

minmn

i ni mi

X X

m n i c c

y y n

g f

m n

mni i

m i n i ic c

Proof Wehave

Z Z

y y x

g f g x x dx e cx x dx

m n m

m n

If ff g then the Laguerre expansion in the square brackets in converges

uniformly see theorem and so

y y

f e g L cxL x dx

n m

c and use equation we obtain If weletb

n m

c c

m n

y y

f g

n m

m n

c c

m n m n F

c c

n m

m n c c

m nc

c c

F m n m n

c c

Substituting the expression for the terminating Hyp ergeometric series gives

Note the following sp ecial cases

minmn

n mni i

X X

c m n i

y y

g f

m n

c m i n i i c

minmn

mni i

X X

m n i

y y

f c g

n m

mni

m i n i i

y y y

c g f f

m n m

theorem is a sp ecial case of Sumitas result for

minmn

i ni mi

X X

m n i c

n y y

f g

n m

mni i

m i n i i c

The ith term of the inner sum equals zero unless i m b ecause of the term

If i m then n m b ecause of the upp er limit of the inner sum and so

m nm

m n m c

y n y

f g

n m

mnm m

n m m c

y nm nm m

f c c

nm m y

c c f

This is Sumitas result

Other Uses of Logarithmic Transformation

The use of a logarithmic transformation with the Laguerre transform has b een sug

gested elsewhere In section of Sumita considers the problem of deter

mining the distribution of the pro duct of indep endent Beta variates He nds the

distribution of the logarithm of the variates analytically rather than in terms of the

Laguerre co ecients and then uses the Laguerre transform to nd the distribution

of the sum of the transformed variates From this he obtains the distribution of the

pro duct bytheinverse transformation Sumitas approach is similar to that used by

Ramsey et al describ ed b elow

Furmanski and Petronzio in tro duced the idea of a logarithmic transforma

tion to solve a sp ecial class of problems encountered in Quantum ChromoDynamics

The metho d is describ ed by Ramsey in He considers the solutions of integro

dierential evolution equations of the form

dF x t

P xy F y t dy

dt y

Comparing the right side of with the expression for the probability density

of the pro duct of two random variables in terms of their densities suggests that it is

a closely related problem If the tworandomvariables are indep endent and always

p ositivewehave from

f zxf x dx f z

X Y Z

Metho ds that have b een used to solve are similar to those suitable for

solving Two examples are Mellin transforms and brute force

numerical metho ds evaluating the integral Ramsey says that the brute

force numerical metho ds tend to b e inecient and prone to instability The

Mellin transform requires a numerical inversion which can b e dicult The metho d

presented by Ramsey is more stable and more accurate than other metho ds in

that it allows one to deal with the functions F x t directly rather than with their

integral transforms p It in volves a change of variables in via a

logarithmic transformation followed by a sum convolution whichisevaluated bya

Laguerre transform Of course more than this is required in order to solve

but this is not our concern here

Ramsey discusses the eect of truncation error and roundo error p ointing out

that there is an inherent tradeo b etween them making the truncation error smaller

by using more co ecients to represent the functions will comp ound rounding errors

by requiring more arithmetic op erations For his particular application Ramsey

found that the round o error b ecame signicant greater than with double

precision arithmetic when N thenumb er of co ecients used was greater than

In general he suggests values b etween and should b e used His values of N

are less than those used by Keilson and Nunn pwho used or Sumita

pwho used and found that when programmed with double precision

arithmetic there was no evidence of numerical problems for simply calculating

the convolution p

Ramseys metho d diers from the metho d wehavedevelop ed ab ove in the manner

of determining the Laguerre co ecients of the logarithmically transformed function

Whereas wehave attempted to determine these from the Laguerre co ecients of the

function to b e transformed Ramsey assumes the Mellin transform of the original

function is known He shows on page of ho w to derive the required Laguerre

co ecients in terms of these transforms This approachisobviously no go o d for our

application Ramsey also suggests page the idea of expanding the function by

apower series and presents a metho d for determining the Laguerre co ecients in

terms of the co ecients of the p ower series expansion It is not clear to the present

author how this metho d can work

Other Typ es of Laguerre Transforms and Related Results

Before leaving the topic of Laguerre transforms we briey p oint out some other

work on convolutions and Laguerre transforms which is not as widely known A

numb er of unexp ected connections have b een considered in see esp ecially the

editors preface which contains a pap er on the use of Laguerre p olynomials for

convolutions on the Heisenb erg group

Hirschmann dev elops an inversion formula for the transform dened for any

real function f nby

f mL x f x

His formula is related to the PostWidder inversion formula for the Laplace transform

page

Debnath in tro duced a transform in which is more closely related to

Keilsons It is a generalisation of McCullys transform whic hwas used for

solving certain dierential equations Debnaths transform of a function F xis

dened by

x a a

f n e x L xF xdx a

In Debnath sho ws that if f nandg n are the DebnathLaguerre transform

a a

of the functions F xand Gx resp ectively then the DebnathLaguerre transforms

of the convolution F x Gx exists when Rea and is equal to f ng n

D a a

The convolution is dened by

Z Z

n a

t a xt cos

F x Gx e t F t e

J xt sin sin

xt cos dt d Gx t

xt sin

where J z is the Bessel function of the rst kind of order nThisismuch more

complicated than equation and is obviously of no use to us for the purp oses of

probabilistic arithmetic Debnaths transform has b een further studied by Glaeske

and F enyo In Glaeskeshows that

Z Z

sin z s

xs a

f z e x F x dx ds

sin z

M L x F x

This relationship seems to b e of even less use than equation Note that the

Laguerre p olynomials in the ab ove transform are dened for nonintegral degrees

x L x

where is the degenerate hyp ergeometric function section Equation

is equivalent to when is an integer

The Laguerre transform closest in spirit to Keilson et als whichwe studied

ab ove is that discussed byVerma He obtains a real inversion formula for a

general Laguerre transform of the form

stf t dt s e st L f

n n

His inversion formula is

s stf L f tsst

n n

He also gives the following convolution formula for this transform Dening a

sp ecial case of the transform equivalent to Debnaths

V f ts e L stf t dt

n n

he shows that

V g ts V f tsV f g ss

n r

where is the ordinary sum convolution of two distributions and f and g are

distributions with supp orts b ounded on the left This is a generalisation of the

result of Genin and Calvez that

g ts f g s V f tsV V

r n

Noting that L L can b e seen to b e equivalent to

Other Laguerre transforms are studied in Various issues relating

to the convergence of certain Laguerre series expansions are discussed in

Laguerre series expansions have also b een used byAckroyd to

invert a Poisson transform Laguerre p olynomials have b een used by Tsamas

phyros et al for the numerical inversion of Mellin transforms although

unfortunately this do esnt help with the problem discussed in section They

have also b een used for the numerical calculation of Fourier transforms and

evaluation of certain Hankel transforms The probabilistic origin of Laguerre

and other classical orthogonal p olynomials is investigated by Cooper et al in

Laguerre series representations of certain sp ecial probability densities can b e found in

Laguerre p olynomial expansions also app ear in probability theory under the

name of GramCharlier expansions If the successive derivatives of the Gaus

sian density function in the ordinary typ e A GramCharlier series are replaced by

derivatives of the Gamma density then a representation of a density f x of the

following form is obtained

L x p x f x

n m

x m

In equation p x e x m for m The co ecients f g are

m n

given by

f xL x dx

n n

The fact that there is no exp onential weighting function in means that

f g will only represent f using if all the moments of f exist Equation

has the prop erty that if only k terms of the innite series are used then the

rst k moments of the series representation of f will b e correct theorem

Further details and applications can b e found in

Conclusions on the Laguerre Transform Metho d

Although wehavecovered a lot of detail in the analysis of the Laguerre transform

metho d in the hop e of using it for pro duct and quotientconvolutions there are many

problems whichwehave not mentioned Many of these have already b een examined

to some extentby Sumita and others and do not just o ccur in the attempt to

calculate pro duct and quotientconvolutions Wehave already mentioned section

how the shap e of the density can aect the numb er of Laguerre co ecients

required for an accurate representation This is clearly an undesirable prop erty

There are other metho ds whichwe discuss elsewhere for representing distributions

and calculating convolutions which do not suer from this problem Other problems

asso ciated with the Laguerre transform metho d include the handling of dep endencies

but see and the manner in which one can obtain the Laguerre co ecients

to start with If the densitys analytical formula is known then one can calculate

the co ecients analyticallyHowever if the density is derived from sample values

this is not p ossible

To sum up then we can say that while orthogonal series in general and the

Laguerre transform in particular seem at rst sighttobegoodchoices for our goal

it turns out that this is not the case The Laguerre transform can not b e used to

calculate pro duct and quotientconvolutions anywhere near as easily as it can b e

used to calculate sum and dierence convolutions Thus although a useful to ol for

sum and dierence convolutions it is not suitable as the basis for a more general probabilistic arithmetic

Figure The histogram representation f x approximates f x

The Histogram and Related Metho ds

The histogram metho d is a way of representing probability distributions and cal

culating their convolutions It was develop ed byColomb o and Jaarsma

based on ideas in and has also b een examined b y Keey and Smith In this

section we will examine this metho d in some detail explaining how it is used and

discussing the various diculties that are encountered We present the algorithms in

question explicitly they were only outlined by Colomb o and Jaarsma in We

will also compare the histogram metho d with two other similar metho ds Kaplans

DPD metho d and Mo ores generalisation of Interval Arithmetic

The Histogram Representation

Denitions and Notation

The histogram representation is discussed with reference to gure The proba

bility density f of a random variable X is approximated by f This is dened by

the set of ordered pairs

H fx p x p x p x g

X n n n

where p and x x for i n Wehave

i i i

f x p for X x x

i i i X

In the equiprobable histogram which is the fo cus of our attention here we also have

Z Z

x x

i i

p f x dx f xdx

i X

x x

i i

for i n The zero in the last ordered pair in allows a neater

denition of the histogram and it is used to indicate the end of the data structure

in computer implementations

Alternative Choices of p

An alternative to the equiprobable histogram is the equispaced histogram where

x x is constant Colomb o and Jaarsma discuss the relativ e merits of

i i

the equiprobable versus the equispaced histograms They presentanargumentin

favour of equiprobable intervals in terms of the choice of class intervals in tests

and they compare the accuracy of the results obtained in using b oth

techniques Their conclusion is that equiprobable intervals are generally b etter

Note that optimal representations of the form such as those that are

obtained by minimising the L distance or disparit y

F x F x dx

H H

between the distribution functions F and F corresp onding to the densities f

X X

and f inordertocho ose a histogram representation are not appropriate for our

purp oses Not only is the exact distribution F required to construct these approxi

mations F but one can not easily maintain the optimality when histograms are

combined

Choice of the NumberofIntervals and the End Points x and x

Henceforth we shall only consider the equiprobable histogram There thus arises the

question of the choice of n the number of intervals and the choice of the endp oints

x and x The number of intervals aects the accuracy of the convolutions and

the time taken to compute them We shall see b elow that the combination of two

histograms of n bins has a computational complexityof O n This restricts n to

b e normally less than

Regarding the end p oints x and x ifF has nite supp ort then they can b e

n X

simply set to inf supp F and sup supp F resp ectively If supp F then there

X X X

are two alternatives one can either truncate f in some way or one can use an

extended numb er system that allows and as p ossible values Colomb o and

Jaarsma argue but not v ery convincingly that a reasonable rule is simply to

make the width of the end intervals some constant amount larger than the width

of the p enultimate interval contiguous to it That is

x x x x and x x x x

n n n n

They suggest that should b e around to as long as the long tailed distribu

tions are regularly shap ed The alternative metho d using the extended

real numb er system f g has b een used by several authors in interval

arithmetic This alternative metho d was used to calculate the

example results presented b elow It has the advantage of not relying on sometimes

unprovable assumptions ab out the distributions in question

Combining Histograms to Calculate Convolutions

Wenow consider how the equiprobable histograms can b e combined in order

to calculate convolutions of the probability distributions which they represent

The Basic Combining Rule

Supp ose wehavetwo histograms

H fx p x p x p x g

X n n n

and

H fy q y q y q y g

Y m m m

which representtwo random variables X and Y with probability densities f and

f resp ectivelyWe assume that X and Y are indep endent If is some arithmetic

op eration and Z X Y then H isan unsorted histogram which approximates

the density f Itisgiven by

z r z z r g H fz

nm nm

where

z min fx y x y x y x y g

i j i j i j i j

imj

z max fx y x y x y x y g

imj i j i j i j i j

and

r p q

imj i j

for i n and j m

The rationale b ehind and is to consider all p ossible pairs of in

tervals and combine them together using the rules of interval arithmetic The

upp er and lower b ounds of the resultantinterval are the maximum and minimum of

all combinations of the endp oints of the interval op erands Equation says that

the probability that Z will takeona value within a given interval z z is

imj

the pro duct of the probabilities that X and Y will takeonvalues within the intervals

x and y y x

This basic combining rule is not entirely adequate b ecause the histogram H is

unsorted and some of its constituentintervals mayoverlap This can b e seen from

the following example Let

H f g X

and

H f g

Then if Z X Y

f g H

Observe that the four intervals in H and are not

disjoint

Construction of the Disjoint Histogram and Condensation

Our nal aim is to approximate f by a histogram of the form of The

following pro cedure can b e used in order to convert H to this form

Sorting step Sort the nm tuples of H into an order sp ecied by the

relation to give H The relation is dened by

H H

z z p z z p

i i H j j

i j

z z z z or z z and

i j

i j i j

This can b e p erfomed using a standard sorting algorithm such as quicksort

by using an appropriate comparison function

Construction of the disjoint histogram H The next step is to construct

a disjoint histogram H where

H fw w s w w s w w g

and

w w w w for all i j

i j

This will not necessarily b e an equiprobable histogram ie there mayexist

an i and j i j suchthat s s although it will have the prop ertythat

i j

for i nm We will use the ab ove redundant tuple w w

representation of the elements of H for clarity The pro cedure used to form

S D

is as follows from H H

Z Z

for i i nm if

w z w z s

i i

i i i

for j i j j f

if z w and z w f

j i

i j

z if w z and w

i j

s r z w z z

i j j j

i j

z else if w z and w

i j

s r w w z z

i j i j

i j

g g

The purp ose of the inner for lo op is to determine for eachinterval z z

whether it overlaps the interval w w The lo op starts at j i b ecause there

cannot b e anyoverlap of relevance at jidue to the sorting p erformed in

w z step and the assignments w z and

i i i

z w w When this o ccurs we The rst if statement is true when z

j i

have to determine howmuchtheyoverlap and hence howmuch probabilityto

include in the interval w w due to the probabilit yinz z This is what

i j

the next two statements do

Regarding the inner two if and else if statements note that the cases w

z and w z and w z and w z cannot o ccur b ecause w z

i i j

j j i j i i

w z for j i and by the sorting step z z or z z and z

i i

i i ip i ip

z when p

Construction of the equiprobable histogram H via condensation In

order to construct an equiprobable histogram H from the disjoint histogram

H we pro ceed in the following manner If H is to comprise intervals ie

H fv t v t v t v g

then t for i and the v are determined by the following

i i

algorithm The variable u keeps trackofhowmuch probabilitywehave used

fori j i nm if

u s

whileu f

w w v w u w w

i j

i i i i

w v

The histogram H is the result we required Wehave condensed the histogram

H of nm intervals to the histogram H of intervals We usually

have nm This condensation is necessary if we are to p erform a series

of arithmetic op erations on a numb er of histograms for otherwise the nal

histogram would have an enormous number of intervals

Computational Complexity and Examples

The computational complexity of the ab ove algorithm is dominated by step the

construction of the disjoint histogram H This has time complexity O nm

b ecause for eachofthenm bins in H wehavetocheck whether any of the other Z

Figure Histogram densityofZ X Y where df X df Y U and N

nm bins overlap The sorting in step O nm log nm saves some time but only

by a constant factor We will see b elow that when the histogram bins intervals

are replaced by single p oints then the complexity is dominated by the time taken

for sorting

The ab ovecombination algorithm was implemented in the form of a computer

program and was used to calculate the following examples Figure shows the

calculated histogram densityofZ X Y where X and Y are indep endent random

variables with uniform distributions on All the distributions were represented

by histogram bins The calculation to ok minutes of CPU time on a Microvax

I I minicomputer Figure shows the result of the same calculation when only

bins were used to represent the distributions involved In this case the CPU time

was only minutes Figure shows the histogram representation of Z when

Z X Y Again X and Y were indep endent and uniformly distributed on

and bins were used in the histogram representation

Figure Histogram densityofZ X Y where df X df Y U and N

Figure The Discrete Probability Distribution representation

Kaplans Metho d of Discrete Probability Distributions

The DPD Representation

A metho d for representing and combining probability distributions similar to the

histogram metho d was develop ed by Kaplan He used simple discretisations

of probability distributions and called his metho d the DPD Discrete Probability

Distributions metho d Instead of approximating a continuous probability density

by a sequence of histogram bins as in gure a sequence of delta functions is

used see gure Thus a random variable X is represented by a set of ordered

pairs

D fx p x p g

X n n

where p is the probabilitythatthevariable will takeonthevalue x Whilst this

i i

seems to b e really little dierent to Colomb o and Jaarsmas histogram metho d it

do es in fact dier in twoways Firstly Kaplan considers a DPD to be our state of

knowledge of the random variable in question and so the question of the accuracy

of representation do es not arise

We allow ourselves total freedom to select the x and p anywayatallsave

i i

only that the set fx p g adequately represents our state of knowledge

i i

and that it b e suited to the numerical pro cedures wehave in mind p

Secondly the combination rules and condensation pro cedure are simpler than those

for the histogram metho d b oth in a conceptual and computational complexity sense

We will briey examine this b elow

Nonuniqueness of the DPD Represenatation

Firstly let us note that under Kaplans denition and interpretation there are an

innite numb er of DPDs which can describ e the same random variable This caused

consternation for Nelson and Rasmussen They were unhappy with the fact

that

f g

and

f g

b oth describ ed the same random variable As Kaplan do es not sp ecify any con

straints suchas x x for i j this is not surprising Of course one can represent

i j

a given random variable with the minimal numb er of tuples if this condition is

imp osed Kaplans reply p oin ts this out The op eration required to convert

into is analogous to the condensation op eration p erformed on the his

tograms describ ed earlier but is simpler here b ecause weonlyhave tuples instead

of tuples This condensation needs to b e p erformed in anycaseaswe shall see

b elow

Combination and Condensation of DPDs

Two DPDs are combined in the following obvious manner If X and Y are two ran

n m

dom variables represented by D fx p g and D fy q g resp ectively

X i i Y j j

i j

then we calculate D fz r g where

Z k k

z x y

ijm i j

and

r p q

ijm i j

for some op eration This is analogous to the combination rule for the histogram

metho d

Since D has nm p oints a condensation op eration is necessaryWehave used

the following algorithm in the calculation of the examples presented b elow Let

b e the DPD pro duced by the combination pro cedure Assume we D fz r g

Z i i

c c c

g r fz want to condense D to p oints Let us call the condensed DPD D

i i i Z

Then the following algorithm will determine D from D

i j

spacing z z

v z spacing

whilei f

whilez v and i f

r r i

z v spacing

v spacing

Recently Kaplan has demonstrated an improved condensation pro cedure It is

similar to that whichwe use in the metho d wedevelop in chapter

Kaplan psuggested that up on obtaining the DPD of the desired result

that it could b e smo othed in order to make it lo ok b etter He do es not make

any suggestions as to how this could b e carried out although p erhaps the work of

Scho enb erg de Bo or and others on splines and histograms could b e

used Of course there is absolutely no probabilistic justication for such a pro cedure

We shall see later that when working with lower and upp er b ounds on distribution

functions the need for smo othing is absent

Applications and Examples

Kaplan shows how one can dene probabilistic functions based on the DPD metho d

His approach is analogous to the extension principle used for fuzzy numb ers see

chapter In fact the fuzzy extension principle was originally motivated by the

probabilistic counterpart He uses the idea of a probabilistic function as a basis for

a metho d of seismic risk assessment Other uses are in probabilistic risk as

sessment see and for some general bac kground and in probabilistic

fracture mechanics

Using the algorithm detailed ab ovewe calculated the distribution of Z X Y

and X Y for the same parameters as the examples for the histogram metho d

Figure shows the output for Z X Y with nthenumb er of p oints used equal

to Figure shows the output for Z X Y with n The irregularityin

gure and the dip in the middle of gure are b oth due to the condensation

pro cedure Note that the cumulative distribution formed from these two discrete

densities would b e quite accurate Whilst the simple DPD condensation algorithm

has the disadvantage of pro ducing these irregularities it has the advantage of b eing

much faster than the histogram metho d In fact the complexity of the DPD metho d

is dominated by the time taken for sorting Sorting is only implicit in our ab ove

algorithms but is necessary in a practical implementation The CPU times for the

Figure Discrete probability distribution of Z X Y df X df Y U and

two examples here were and seconds resp ectively Compare these with the

and seconds for the histogram metho d

Mo ores Histogram Metho d

The histogram metho d presented ab ove has b een rediscovered by RE Mo ore

who develop ed it in the context of interval analysis see and section b e

low Mo ore sho ws how to determine the cumulative distribution function of

an arithmetic op eration on random variables when the random variables are repre

sented by the union of disjoint but contiguous intervals with a certain probability

of o ccurence within eachinterval Mo ore uses only a small numb er of sub divisions

or He still obtains results which corresp ond closely with results obtained by

Monte Carlo simulation in terms of visual insp ection of the graph of the cumulative

distribution This may b e due in part to the fact that cumulative distributions do

tend to lo ok b etter than the corresp onding densities b ecause of the smo othing eect

of the integration

More imp ortanthowever is Mo ores idea of the elasticity of a function with

resp ect to a given input variable This is dened as the limit of the ratio of variation

of the function to the variation of the input variable as the input variation b ecomes

small In other words if the function is f x x the elasticityof f with

resp ect to x is the logarithmic derivative This can b e approximated

f x

X suc h that x X and calculating by considering some interval X X

i i i i

widthf widthX

midf midX

X widthX X X In equation wehavemidX X and

i i i i

i i

X Mo ores idea is to sub divide widthf is the range of f as x ranges over X

i i

nely only those X with a large elasticity b ecause increasing the sub division of those

with a small elasticitywillhave a negligible eect on the accuracy of the answer

This idea would decrease the computational complexity of the histogram metho d

considerably if some of the variables have a larger elasticity than the rest and should

certainly b e taken into account in the development of a system for determining dis

tributions of complex functions of random variables The notion of sub dividing only

those intervals with high elasticity is equivalent to the screening pro cedure men

tioned in Mo ore gives an example application an extended discussion

of which app ears in

Interval Arithmetic and Error Analysis

We will now consider a number of techniques which determine limited information

ab out the distribution of functions of random variables It seems natural to group

these techniques together here more for their commonality of motivation than of metho d One of the most widely used metho ds for determining information ab out

distributions of functions of random variables is interval arithmetic In this sec

tion as well as describing the basic ideas of interval arithmetic we will consider a

number of variations whichhave b een prop osed that provide some extra informa

tion ab out the distribution of values within an interval We will then compare the

interval arithmetic approach to the standard metho ds used by metrologists for the

propagation of the eects of uncertainties in the results of physical measurements

The Standard Interval Arithmetic

Interval arithmetic was develop ed by Mo ore although the original idea w as

prop osed by Sunaga in The original motivation was to develop an au

tomated way of handling rounding error in numerical computation However it can

also b e used to propagate uncertainties in the input variables and this explains our

consideration of it here

The basic idea of interval arithmetic is to work in terms of closed intervals of

real numb ers Wedene

I fx xj x x x x g

to b e the set of real intervals The rules of interval arithmetic which are rules for

combining elements of I follow immediately from the convolution relations

It is simply necessary to observethataninterval X I which represents

some random variable X is given by

X inf supp f sup supp f

X X

I I

where f is the densityof X IfX Y I then for some binary op eration

I I I

Z X Y is given by

z fxy j x X y Y g Z z

If is monotonic then it suces to examine the end p oints of X and Y and we

can write z min and z max where

fx y xy xy xy g

I I I I

Note that according to if Z X Y and Y thenZ or is

undened if we do not allow the values and When equation is

also known as Minkowski addition after Hermann Minkowski who considered it in

It has applications in areas other than interval arithmetic p

When is one of the four arithmetic op erations Z can b e written explicitly

Addition and subtraction are describ ed by

I I

X Y x y x y

and

I I

X Y x y x y

Pro ducts and quotients of intervals are more complicated b ecause these op erations

are not monotonic if zero is contained in one of the intervals It turns out that

there do es not exist a single co ordinatebased representation of intervals such that

b oth sums and pro ducts can b e calculated directly in terms of the co ordinates ie

the whole intervalhastobetaken into account There are at least three

b o oks on interval arithmetic and interval analysis and a comprehensiv e

bibliographyby Garlo whic hcontains items

We shall see b elow that the denition of the interval arithmetic op erations is

only a small part and the easiest part of developing interval algorithms It turns

out that one can not simply apply the standard interval arithmetic op erations to

the standard algorithms of numerical analysis The reason for this is dependency

width or dependency error This manifests itself in the subdistributive property If

I I I I I I I I I I

X Y Z I then X Y Z X Y X Z This fact was rst p ointed out

byYoung in and has b een considered in detail b yRatschek and Spaniol

The idea of a general function of intervals go es back at least to Burkhill

Triplex Arithmetic and Dempsters Quantile Arithmetic

We will now turn to an examination of generalisations of interval arithmetic which

aim to provide some information ab out the distribution of the variables within the

interval Ecker and Ratschek ha ve considered intervals probabilistically in

an attempt to understand the phenomena of sub distributivity and inclusion mono

tonicity They also suggested a joint representation of distributions and intervals

and studied some prop erties of Dempsters quantile arithmetic whichwe examine

b elow Ahmad is supp osed b y Mo ore page note to ha velooked at

the arithmetic of probability distributions from the p oint of view of interval arith

metic However Ahmads pap er is solely concerned with nonparametric estimators

of probability densities and he has nothing to say ab out probabilistic arithmetic

Triplex Arithmetic

Triplex arithmetic w as develop ed by Nickel and others to overcome a p er

is an ordered ceived inadequacy of ordinary interval arithmetic A triplex number X

triple x x xwhere x is the lower bound on X x is the main value of X and x is

the upper bound on X If thex is discarded then we are left with ordinary interval

arithmetic The purp ose of the main value which is treated like a single real

number in Triplex arithmetic calculations is to provide some information ab out the

distribution of X within X This is particularly imp ortantwhen f is concentrated

within a small region but has very wide tails It turns out that the provision of this

main value can help the convergence and accuracy of iterative algorithms

Problems which are unstable using ordinary interval arithmetic can b e made stable when triplex arithmetic is used

Quantile Arithmetic

Dempster has dev elop ed a metho d called quantile arithmetic which

can b e considered as a variation on triplex arithmetic It is p erhaps worthwhile to

quote Dempsters motivation for quantile arithmetic b efore we describ e it in detail

While the supp ort of the error distribution of some nite compu

tations on interval random variables may b ecome arbitrarily large the

error distribution itself may b e concentrated in a small interval with prob

ability close to one In such a case the error distribution of the result

has low disp ersion but long tails in which little probability is massed It

is up on consideration of this p ossibility that quantile arithmetic is based

pp p pp

Again we consider a random variable X with density f and distribution function

F Thequantile number X represents X by the approximation of f Q to f

X X

if x F

f Q x

if x F

otherwise

Note that f is a density while f Q is a discrete frequency function

Q Q Q Q Q

Two quantile numb ers X and Y are combined to give Z X Y via the

rule

f Q xf Q y for z xy

X Y

f z

otherwise

Wehave assumed here that b oth f Q and f Q for only one choice of x and

X Y

y such that z xy for a given z In order to convert the nine p oint frequency

function f into a three p oint frequency function f Q Dempster uses the following

condensation algorithm Let z z z b e the nine values of z suchthat

q q

f z and let q f z Then f Q is given by

Z i Z i

if z x

f z

if z x

otherwise

where

x z for the largest i such that q

i j

q x z for the smallest i such that

j i

x z for the smallest i such that q

i j

j i

The choice of the parameter is somewhat arbitrary likethechoice of q in Petcovics

probable intervals see section Dempster gives a few suggestions but these

are mainly just common sense Dempster has applied his quan tile arithmetic

to the approximate solution of the distribution problem in linear programming

Whilst sup ercially interesting and seemingly an improvementover interval

arithmetic it turns out that quantile arithmetic has an imp ortant disadvantage

compared with interval arithmetic In quantile arithmetic under estimation of the

spread of a result can o ccur Standard interval arithmetic often over estimates the

spread sometimesby a large amount but the result is still correct in the sense that

the true interval is contained within the broader one In contrast quantile arith

metic will sometimes give results whichare wrong in that the calculated interval is

narrower than the true interval

Interval Arithmetic and Condence Intervals from the Metrolo

gists Point of View

Wehave seen that ordinary interval arithmetic uses only information ab out the

supp ort of the distribution of the random variables involved An alternative metho d

is to use interval arithmetic to combine condence intervals The standard metho d

can thus b e considered to combine condence intervals Dierent condence

intervals should then give further information ab out the distribution of the nal

result

This idea has b een considered by metrologists in the context of the theory of

errors and the propagation of uncertainties arising from physical measurements We

will now examine what they have had to say on this topic Metrologists have consid

ered the use of condence intervals b ecause of a shortcoming of the standard metho d

of the statement and propagation of exp erimental uncertainty Note that the term

exp erimental error is out of favour nowadays the uncertainty in former times

frequently called error p The standard metho d is to state uncer

tainties as standard deviations and to use the general law of error propagation

to propagate the errors through subsequen t calculations A go o d recentre

view can b e found in This general law is simply the linearisation of nonlinear

functions by truncated rst order Taylor series expansions ab out exp ected values

see section Sometimes higher order expansions or the more complicated

expressions taking accountofthecovariances are used The shortcom

ing of this metho d arises in the determination of appropriate values of uncertainty

to use for sub jectively estimated systematic errors and the dicultyinconverting

a nal uncertainty statement in terms of standard deviations into an interval result

An interval statement is often required for calibration or legal purp oses We shall

consider these two diculties in turn

Sub jectiveInterval Estimates of Systematic Error and Standard Deviations

In recentyears the distinction b etween random and systematic errors rst

explicitly prop osed by D Bernoulli in palthough implicitly adopted

by Newton as early as p has b een called into question

The p oint is that there has never b een an entirely satisfactory criterion

for deciding which category to use in any particular instance Muller p

quotes Vigoureux One has to rememb er that some errors are random for one

p erson and systematic for another Similarly what is a systematic error for one

exp erimentmay not b e for anotherMuch of the skill in exp erimental work comes

from eliminating sources of systematic error p The choice is imp ortant

b ecause the standard ortho dox theory propagates random and systematic errors

through calculations dierently Systematic errors are added arithmetically whereas

random errors are added in quadrature Indeed as Muller has p ointed out this is

precisely the source of the diculty in classication the traditional classication

of uncertainties dep ends up on the further use weintend to make of them and in

general this can not b e known in advance p

The new randomatic theory avoids the problem of categorization by consid

ering al l errors to b e random However it do es distinguish b etween ob jective

statistical estimates and sub jective guesstimates p The sub jective

uncertainties are often given in terms of an interval This has to b e converted into

a standard deviation in order to propagate it through any subsequent calculations

Anumb er of admittedly somewhat ad hoc metho ds for doing this can b e found in

The sub jective uncertainties are usually considered to b e indep endent

p although there seems to b e no go o d a priori reason for this to b e so It could

well b e argued though that any error arising from an ad hoc handling of sub jec

tive uncertainties should usually b e negligible compared with the variability due to

the ob jective statistical estimates Arguments against the randomatic theory of

errors and prop osals for an improved ortho dox theory based on a more careful dis

tinction b etween the two classes of errors can b e found in the closely argued pap er

of Colclough

Determination of a Condence Interval for the Final Result

A more serious and older problem is the conversion of a nal uncertaintystate

ment in terms of a standard deviation into an interval statement Wehave already

observed section that the Chebyshev inequality or its generalisations

can b e used to determine a condence in terval in terms of means and

standard deviations However in general these will b e very lo ose p essimistic in

tervals Alternative approaches whichgivetighter less p essimistic intervals require

assumptions ab out the underlying distributions

The oldest metho d is the assumption of normalityWehave already discussed

this in section It has never found universal acceptance and had strong critics

even some years ago I reject then the Gaussian theory of error without qualication and with the utmost p ossible emphasis and with it go all theoretical

grounds for adopting the rules that are based on it p We need add

nothing further to this

An alternative which is similar in its general approachtothelower and upp er

b ounds on the distributions we consider in chapter has b een considered inde

p endently by Kuznetsov and W eise Assuming that the true error

distribution is symmetrical unimo dal and has nite supp ort Kuznetsov cal

culated a condence interval in terms of a variance by using the mean distribution

derived from lower and upp er distribution functions satisfying the distributional as

sumptions Weise somewhat more ambitiously considered a whole class of

distributions D He then used a distribution whichwas a mean of all p ossible dis

tributions over D in order to determine a condence interval in terms of a given

variance Obviously the choice of D and the fact that the mean distribution is

used will aect the result dierentchoices will give dierent results Nevertheless

this approach seems to b e a promising wayofinvestigating the eects of dierent

distributional assumptions and of handling dierent degrees of optimism

Direct and Exclusive Use of Condence Intervals

Although they are preferable to assuming normality the metho ds of Kuznetsov and

Weise are still not entirely satisfying An alternativeistoavoid completely the use of

standard deviations as a measure of uncertainty and to use condence intervals only

from the outset Muller and others ha ve argued against this idea on the grounds

of diculties in combining condence intervals The general consensus amongst

metrologists seems to b e describ ed by the DIN standard part Basic Concepts

of Measurements Treatment of Uncertainties in the Evaluation of Measurements

a summary app ears in This suggests a careful but fairly straigh t forward

application of the general law for error propagation in terms of standard deviations

No distinction is made b etween random and systematic errors apart from the

metho ds of initially estimating the numerical value to b e attributed There are

still diculties in using standard deviations when rep eated measurements are not

indep endent but these seem to b e manageable given some idea of the sp ectrum

of the error sequence

Recently Rowe has presen ted some very interesting results which could lead

to a partial combination of the two techniques intervals and standard deviations

Using two separate approaches Rowe has determined lower and upp er b ounds for

and in terms of X X or limited order statistic information for a

Y Y X X

class of transformations Y g X Rowes approach is preferable to the simple

rst order Taylor series approximations b ecause lower and upp er b ounds for

g X

and are given These allow a more rigorous propagation of uncertainty Also

g X

when X and X or other distributional information is available the tighter b ounds can b e obtained

Condence Curves and Fuzzy Numb ers as a Generalisation of Interval Arith

metic

Finally to conclude this somewhat discursive exploration of the metrological uses

and signicance of interval arithmetic we can mention the idea of fuzzy arithmetic

This has b een suggested as a natural generalisation of interval arithmetic and

error propagation techniques b ecause under the standard supmin com bination

rules see chapter fuzzy numb ers can b e combined in terms of interval arithmetic

on the level sets of their memb ership functions We examine fuzzy arithmetic and

p oint out some similarities to the idea of condence curves intro duced byCox

and develop ed byBirnbaum and Mau in c hapter These condence

curves are made up of nested sets of condence intervals at dierent condence

levels and they may provide a useful generalisation for the purp oses of the theory

and propagation of errors

PermutationPerturbation and Related Metho ds

The original motivation for interval arithmetic was the automatic control of rounding

errors in numerical calculations There are several others approaches p ossible for

this and some are of interest for our goals as well

The most widely known is Wilkinsons analysis of particular algorithms to de

termine the accuracy of the result that can b e exp ected More explicitly

sto chastic metho ds have b een adopted in recentyears See for example

Another metho d which has b een presented several times by Vignes et al

is called the PermutationPerturbation metho d or cestac Con

trole et Estimation Sto chastique des Arrondis de Calcul The basic idea is to con

sider the perturbations ofaresultofan n stage numerical computation obtained

by p erturbing each op eration by an appropriate amount p ositively or negatively

This is combined with permutations of the order of computation which will not

always give the results that would b e obtained over b ecause of the failure of as

so ciativity and distributivity under oating p oint arithmetic Vignes argues that

only a few two or three of this large numb er of results need b e considered He

randomly p erturbs and p ermutes the calculation and then estimates the accuracy

of the result Applications of the metho d can b e found in

Stummel has also considered the eect of p erturbations on in termediate

results in the computation of arithmetic expressions by using the computational

graph concept He has obtained a numb er of results for the condition number

of an algorithm and has applied his results to a careful analysis of the numerical

solution of a linear system of equations and the analysis of some in terval arithmetic algorithms

Chapter

Numerical Metho ds for

Calculating Dep endency Bounds

and Convolutions

In my view thereisacentral obligation to facesquarely

what we do and do not know and to study robustness of

conclusion against mistakes in a priori assumptions of

independence conditional or not

William Kruskal

In this chapter we present a new and general numerical metho d for calculating

convolutions of a wide range of probability distributions An imp ortant feature

of the metho d is the manner in which the probability distributions are represented

We use lower and upp er discrete approximations to the quantile function the quasi

inverse of the distribution function This results in any representation error b eing

always contained within the lower and upp er b ounds This metho d of representation

has advantages over other metho ds previously prop osed The representation ts in

well with the idea of dep endency b ounds

Sto chastic dep endencies which arise within the course of a sequence of op erations

on random variables are the severest limit to the simple application of convolution

algorithms to the formation of a general probabilistic arithmetic We examine the

error caused by this eect dep endency error and showhow dep endency b ounds

are a p ossible means of reducing its eect Dep endency b ounds are lower and upp er

b ounds on the distribution of a function of random variables whichcontain the

true distribution even when nothing is known of the dep endence of the random

variables They are based on the Frechet inequalities for the joint distribution of

a set of random variables in terms of their marginal distributions Weshowhow

the dep endency b ounds can b e calculated numerically when using our numerical

representation of probability distributions Examples of the metho ds we develop

are presented and we briey describ e relationships with other work on numerically

handling uncertainties

The presentchapter is a very slightly mo died version of the pap er It

is essentially self contained and thus there is some slight rep etition of material

presented elsewhere in this thesis

Intro duction

In order to develop automated systems for dealing with uncertainty it is necessary

to b e able to calculate the basic op erations of an uncertainty calculus numerically

Amongst the many dierent uncertaintycalculinowavailable ordinary probability

theory is the oldest Surprisingly though there has b een little detailed examination

of numerical metho ds for calculating the distribution of arithmetic op erations on

random variables Although there havebeenanumber of schemes prop osed so far

there have b een none that meet the following simple criteria

The metho d should allow the calculation of the distribution of all four arith

metic functions of random variables and not just addition and subtraction

There should b e no restrictions or only very slight restrictions on the class

of random variables that can b e handled

There should b e a careful treatment of all the errors arising in the calculation

particularly those due to the numerical representation adopted

The metho d should b e computationally tractable and the algorithms should

b e describ ed explicitly

The metho d should b e simple to understand and implement

The presentchapters goal then is to develop a metho d satisfying these criteria

for what we will call probabilistic arithmetic

In this intro ductory section we will dene the problem to b e studied more pre

cisely section and intro duce the notions of dep endency error and dep endency

b ounds sections and Since we b elieve it is more worthwhile developing

a metho d with a rigourous foundation rather than an ad ho c technique whichworks

for some applications we will b e concentrating on the foundations rather than the

applications of probabilistic arithmetic Using the description of the necessary lay

ering of uncertain reasoning systems due to Bonissone wecouldsay that we

are concentrating on the representation and inference layers but ignoring the control

layer

It is worthwhile to compare our metho ds with other probabilistic metho ds as

well as metho ds based on other uncertainty calculi such as the theory of fuzzy

sets While we p ostp one a detailed examination of this to chapter we do p oint

out now that our metho d has some similarity to Jains metho d for combining fuzzy

numb ers Jains metho d was subsequently criticised by Dub ois and Prade

although there own metho d LR fuzzy numb ers is not without

drawbacks either More recently Dub ois and Prade ha ve examined the re

lationship b etween Mo ores probabilistic arithmetic itself an outgro wth of

interval arithmetic and fuzzy arithmetic bydrawing on some results from the

DempsterShafer theory of evidence In chapter see also chapter section

we will show that the normal combination rules for fuzzy numb ers are in fact

equivalent to our dep endency b ounds and that our limiting result chapter can

b e used to derivea law of large numb ers for fuzzy variables under a general extension

principle Perhaps the work which is closest in spirit to that presented here is that of

Grosof Grosof has taken much the same approachaswehave in analysing the

probabilities of events and combinations of events rather than arithmetic op erations

on random variables He has shown that a sp ecial case of his interval conditional

probability logic is actually formally identical to the DempsterShafer theory of ev

idence Several other authors ha ve recently adopted an approach similar

to that outlined in the presentchapter viz calculation of lower and upp er b ounds

on probabilities when limited dep endence information is available

The Problem

Consider the following problem Let X and Y be two random variables with distribu

tion functions F and F resp ectivelyLetZ X Y where is some arithmetic

X Y

or other op eration Then what is F the distribution of Z For any given

if the joint distribution F is known then a solution to the problem in terms of

an integral can b e written down The appropriate integral is determined from the

Jacobian of transformation In many cases closed form solutions to the integral do

not exist Whilst series solutions can generally b e obtained the resulting formulae

are often very complex and are of little value for an automated system

If the joint distribution of X and Y is not known ie only the marginals F

and F are known then one can not even in principle calculate F One can

Y Z

however calculate lower and upp er b ounds on F aswas recently shown byFrank

Nelsen and Schweizer

The presentchapter develops numerical algorithms to solve b oth of these prob

lems Algorithms are develop ed for calculating lower and upp er b ounds on F when

it is known that X and Y are indep endent or when there is no knowledge of the

dep endency structure of X and Y at all The techniques develop ed are quite gen

eral and can b e used for almost all distributions The metho d of representing the

distributions and the results obtained for the convolutions are b etter than other

numerical techniques that have b een presented to date The algorithms can b e used

to calculate lower and upp er b ounds in the manner of Frank Nelsen and Schweizer

foramuch larger class of distributions than can b e managed analytically All the al

gorithms are describ ed explicitly and are computationally ecient They have b een

implemented on a minicomputerandhave b een used to calculate some example

results which are included in this chapter

The Idea of Probabilistic Arithmetic and the Need for Dep endency

Bounds

The problems mentioned ab ove arise naturally in the consideration of probabilistic

arithmetic the name is due to Kaplan The goal of probabilistic arithmetic is

to replace the usual arithmetic op erations on numb ers by the appropriate op erations

on random variables which are represented by their distribution functions This

is akin to several other ideas that have app eared in the literature most notably

interval arithmetic and fuzzy arithmetic The similarities and

connexions with these other ideas are not considered here but are examined in

some detail in chapter One of the goals of probabilistic arithmetic is to solve

random algebraic equations numerically problems for which the metho ds of solution

available at present are still rather limited see for a review of the a vailable

metho ds and known results There are numerous other p ossible applications if a

successful probabilistic arithmetic can b e develop ed

Among the problems that need to b e considered are the errors that can o ccur in

a probabilistic arithmetic calculation and how they can b e handled Wewillshow

how one typ e of error dep endency error is the most severe restriction on the simple

application of convolution algorithms to the formation of a workable probabilistic

arithmetic The errors in probabilistic arithmetic can b e classied into vetyp es

Representation Error This is the error caused by the approximation of a func

tion dened on an uncountable subset of by a nite numb er of p oints or

co ecients

Truncation Error This error which can b e distinguished from general represen

tation error for some representations is caused by the necessity for some

distributions of truncating the supp ort of the distributions to a union of nite

intervals

Calculation Approximation Error This is caused by approximations made in

developing the formulae used to implement the probabilistic arithmetic For

example if a Taylor series expansion sayforthevariance of the pro duct of two

indep endent random variables in terms of their moments is truncated

after a nite numb er of terms then the rule itself will only b e approximate

This will intro duce errors into the calculated results

Rounding Error This is simply the error caused by p erforming numerical com

putations on machines with a nite wordlength This will not b e considered

further here b ecause in the absence of illconditioning this can easily b e made

arbitrarily small In any case it is a problem asso ciated with nearly all nu

merical algorithms not only those for probabilistic arithmetic

Dep endency Error Dep endency error is the most imp ortanttyp e of error in prob

abilistic arithmetic It arises in much the same manner as spurious correlation

It is explained by considering the following sequence of op erations where

all the quantities are random variables Assume that V X andY are inde

p endent Then calculate

A X Y B X V C A B

The problem is that even though the three inputs are indep endent A and B

are not b ecause they b oth dep end on X

Although such dep endencies can b e handled in principle bytechniques based

on the Jacobian of transformation it is impractical to contemplate the use of

such techniques for handling sequences of computations of the typ e commonly

employed in the deterministic case Thus in order to carry out sequences

of op erations on random variables one is usually obliged for tractability

to assume indep endence in cases where dep endencies such as the one in the

ab ove equations exist A ma jor aim of the presentchapter is to investigate

the question of handling the error that arises when indep endence is assumed

ie dep endency error

The ab ove classication is useful for comparing the dierent approaches to rep

resenting and calculating with distribution functions This is true even though no

precise denition of the error involved has b een given There do es not seem to b e

any one b est measure of error b etween two distributions as the b est measure will

dep end to a large extent on what the distribution b eing calculated will b e used for

Metho ds of Handling the Errors in Probabilistic Arithmetic

We only concern ourselves here with representation and dep endency errors Trun

cation error and approximation error are discussed in chapter and rounding errors

are mentioned for a sp ecic problem arising in section In section it will b e

shown how a natural representation inspired by the metho d wedevelop to combat

dep endency error can essentially remove all the problems of representation error

The basic idea is to use lower and upp er approximations to the desired distribution

rather than one single approximation Any representation error is contained within

these lower and upp er b ounds

Our metho d of handling dep endency error is to use the results of Frank Nelsen

and Schweizer men tioned ab ove This allows the calculation of lower and upp er

b ounds on a required distribution even if the distribution itself can not b e calculated

b ecause it is not known that the variables involved are indep endent or b ecause the

joint distribution is not available There are several other p ossible approaches to

handling dep endency error For instance to calculate the distribution of

X Y Z XV

it is only necessary to rewrite it as

X Y Z V

and the problem of dep endency error intro duced by rep eated o ccurrences of the

variable X disapp ears This can b e considered as solving the problem in Bonissones

control layer as wehavechanged the order in which the lower level computations are

carried out This rearrangement of expressions has b een used in interval arithmetic

and is discussed in more detail in chapter

There are various extensions to the ideas mentioned in the previous paragraph

which are worth studying One of these which is discussed briey in section is

to use some measure of dep endence b etween the random variables and to mo dify the

combination rules to take this extra information into account With resp ect to this

it is of interest to note a forgotten pap er of Kapteyn it app ears to have b een cited

no more than three times since publication in In Kapteyn considers

problems such as the correlation b etween X and Y where X A B C D

Y A E F G and all the AG are indep endent Another pap er that

deserves mention here is that of Manes see also Manes sho ws that

the rep eated o ccurrence of a variable in an expression causes problems in a wide

range of fuzzy vague or imprecise theories including the theory of fuzzy sets and

so the problem is not p eculiar to probability theory

Outline of the Rest of This Chapter

The rest of this chapter is organised as follows Section contains concise deni

tions of concepts which are needed later on In section the dep endency b ounds

of Frank Nelsen and Schweizer are derived and explained We extend their results

byproving the p ointwise b est p ossible nature of the dep endency b ounds for op era

tions other than addition and subtraction Some examples are calculated by directly

using the formulae for dep endency b ounds We show that a b etter way to calculate

dep endency b ounds numerically is to use the numerical representation of probabil

ity distributions develop ed in section along with sp ecial discrete versions of the

dep endency b ound formulae Algorithms for calculating ordinary convolutions in

terms of this numerical representation are also develop ed in section Sections

and contain some extensions to the basic results of sections and and

include suggestions and directions for further research Finally section contains

a summary of the contributions of this chapter and some conclusions

Denitions and Other Preliminaries

Wenow briey presenta numb er of denitions we need later on Most of the material

here is covered in more detail in

Distribution Functions

Denition Let X bearandom variable on Then its distribution function

F is denedbydf X F x P fXxg for x

X X

The corresp onding density when it exists is denoted f

Denition The supp ort of F denoted supp F is the set of x such

X X

that f xF x exists and is non zero

Denition The set of al l distribution functions that areleftcontinuous on

wil l be denoted The subset of distribution functions in such that F wil l

be denoted

Binary Op erations

Denition A binary op eration on a nonempty set S is a function T from

S S into S

Denition Let T be a binary operation on S An element a of S is a left

null element of T if T a x a for al l x S itisarightnull element of T if

T x aa for al l x S and it is a null element of T if it is both a left and right

nul l element of T

A binary op eration can haveatmostonenull element

QuasiInverses

Quasiinverses are generalisations of the inverse of a function which are dened

even when the function has jump discontinuities In this chapter we are only con

cerned with quasiinverses of nondecreasing distribution functions Let F be a

nondecreasing function on a closed interval a bLety a bThenF y is the

set fxj F xy gIfF has no jump discontinuities then the cardinalityof F y

is one and we simply write F y x If the cardinalityof F y is not one then

wehave to somehowcho ose b etween the various elements of F y

Denition Let F beanondecreasing function on a closed interval a b

Then QF is the set of functions F known as quasiinverses of F denedon

F aFb by

F F a a and F F b b

If y Ran F then F y F y

If y Ran F then F y supfxj F x yg inffxj F x yg

All F QF are nondecreasing and coincide except on an at most a denumerable

set of discontinuities There is a unique function F QF whichisleftcontinuous

QF whichisrightcontinuous on on F aFb and a unique function F

F aFb These are given by

F y sup fxj F x yg

and

F y inf fxj F x yg

For all F QF F F F on F aFb If F and G are nondecreasing

on a b then F G F G and F G Ifa b f g

the extended reals then F Thisintro duces technical diculties and

F inf supp F so we adopt the convention that F

Triangularnorms

Denition A binary operation T on a set S is asso ciative if

T T x y z T x T y z x y z S

Denition A triangular norm or tnorm is an associative binary operation

on that is commutative nondecreasing in each place and such that T a a

for al l a

Copulas and Joint Distribution Functions

The most imp ortant notions for the presentchapter are those of the copula and the

Frechet b ounds

Denition A twodimensional copula C is a mapping C

such that

C a C a and C a C a a for al l a

C a b C a b C a b C a b for al l a a b b such

that a a and b b

All copulas satisfy

W a b C a b M a b

for all a b where the tnorms W and M which are also copulas

are given by

W a b maxa b

and

M a bmina b

Copulas link joint distributions with their marginals Let H be a two dimensional

distribution function with marginals F and G Then there exists a copula C such

that

H u v C F uGv

for all u v The inverse relation is

C u v H F uG v

where F QF and G QG The copula C contains all the dep endency infor

mation of H IfC u v u v uv then the random variables are indep endent

Equation is sometimes referred to as the uniform representation

Combining gives b ounds on the joint distribution in terms of the

marginals

maxF u Gv H u v minF uGv

These are known as the Frechet bounds A copula is a tnorm if and only if it

is asso ciative A tnorm T is a copula if and only if it is increasing that is if

T a b T a b T a b T a b T a b for all a a b b

suchthat a a and b b see p of

Denition Let C beacopula The dual of C is the function C denedby

C x y x y C x y for al l x y

The dual copula should not b e confused with the conorm of a tnorm T given by

T x y T x y The dual of W is W u v minu v Schweizer

and Sklar use the notation C for a dual copula We use the overbar notation

for a dierent purp ose b elow

The Triangle Functions and and convolutions

The following three op erations are of great imp ortance in the sequel The op erations

and are intro duced bySchweizer and Sklar b ecause of their prop erties as

triangle functions in the theory of probabilistic metric spaces They are known

elsewhere as the supremal and inmal convolutions

Denition Let C beacopula and let L be a binary operation from

onto which is nondecreasing in each place and continuous on except

possibly at the points and we shal l cal l this class of functions L in

the sequel Then is the function on whose value for any F G

is the function F G denedon by

F Gx sup C F uGv

Luv x

Denition Let C L F and G be as in denition Then F G

is the function denedby

F Gx inf C F uGv

Luv x

Denition Let C L F and G be as in denition Then F G

is the function denedby

F Gx dC F uGv x

Lfxg

where Lfxg fu v j u v Lu v xg

R R

In the sequel we will write for The function is the distribution of

Luv x Lfxg

LX Y where X and Y are random variables with joint distribution F u v

C F uGv This op eration is called a convolution for the op eration L the

signies the additive prop erties of the integral in contradistinction to the inmum

and supremum op erations in the inmal and supremal convolutions The well

known convolution for C and L Sum given by

Z Z

F Gx dF uGv F x t dGt

uvx

is a sp ecial case of For each of the three typ es of convolution and

L is sometimes written as an inx op erator as in Z X Y Explicit formulae

for the other arithmetic op erations are given by

The and op erations can actually b e dened on the whole of

CL CL CL

for appropriate LWe use these extended denitions in the sequel

Dep endency Bounds and their Prop erties

The dep endency b ounds for the four arithmetic op erations of addition subtraction

multiplication and division are nowderived and their prop erties examined Unless

otherwise stated the random variables considered are almost surely p ositive In

other words their distribution functions are in Bounds on the distribution of

Z LX Y are derived where L L It is then shown how to apply these b ounds

to subtraction and division of random variables Following this the p ointwise b est

p ossible nature of these b ounds for general L L is shown This was not proven in

although our pro of is a fairly straightforward generalisation of the pro of there

for L SumFinallyanumb er of examples whichhavebeennumerically calculated

are presented

The Dep endency Bounds

Let X and Y b e random variables on with df X F and df Y F such

X Y

be that F F and let Z LX Y with df Z F where L LLetC

X Y Z

alower b ound on the copula C Then F dep ends on the joint distribution of X

XY Z

and Y and will b e contained within the b ounds

ldb F F Lz F z udb F F Lz z

C X Y Z C X Y

XY XY

W we will simply write ldb and udb These stand for lower dep en When C

dency b ound and upp er dep endency b ound resp ectively

Theorem When C W the functions ldb and udb are given by

ldbF F Lx F F z sup W F uF v

X Y WL X Y X Y

Luv x

and

udbF F Lx F F z inf W F uF v

X Y WL X Y X Y

Luv x

Figure Illustration for the pro of of theorem

Sometimes the notation F or F is used for ldbF F L or udb F F L

Z X Y X Y

resp ectively when F F and L are clear from the context

X Y

Proof With reference to gure and to section it is clear that for any

given copula C and anypairofpoints u v u v on the line Lu v x

W F u F v C F u F v

X Y X Y

dC F uF v

X Y

F F x

CL X Y

dC F uF v

X Y

F u F v C F u F v

X Y X Y

C F u F v

X Y

W F u F v

X Y

The pro of is completed by observing that is simply the greatest value of

W F u F v where u v is on the line Lu v x Similarly is the

X Y WL

smallest value of W F u F v where u v is on the line Lu v x

X Y

and By a similar argument one can show that the more general b ounds ldb

udb are given by and with W simply replaced by C Notethat

since C W always the b ounds of theorem will always hold However as

we shall see in section the b ounds for C W are tighter and thus provide

W in this more information ab out F We will only concern ourselves with C

Z XY

section Note that knowing an upp er b ound on C other than M do es not allow

one to construct tighter b ounds

Theorem can b e used to b ound the distribution of all four arithmetic op

erations on almost surely p ositiverandomvariables The condition of p ositivityis

necessary b ecause pro duct and quotient are only monotonic on or and not

over all All the results are collected together in the following theorem

Theorem Let X and Y be almost surely positive random variables with dis

tributions F and F and let Z X Y where is one of the four arithmetic

X Y

operations f g Then the lower and upper dependency bounds for F

the distribution of Z are given by

ldbF F x sup maxF uF v

X Y X Y

uv x

udbF F x inf minF uF v

X Y X Y

uv x

ldbF F x sup maxF u F v

X Y X Y

uv x

udbF F x inf minF u F v

X Y X Y

uv x

ldbF F x sup maxF uF v

X Y X Y

uv x

udbF F x inf minF uF v

X Y X Y

uv x

ldbF F x sup maxF u F v

X Y X Y

uv x

udbF F x inf minF u F v

X Y X Y

uv x

Proof The Sum and Pro duct cases follow directly from theorem The

Dierence and Quotient cases can b e seen by noting that on these are b oth

monotonic op erations they are increasing in their rst argument and decreasing in

their second They can b e converted to Sum and Pro duct as follows For Dierence

Z X Y let Y Y Then F y F y Substitution into and

Y Y

some slight rearrangement yields For QuotientZ X Y let Y Y

Then F y F y Substitution into yields

Y Y

The b ounds for addition and subtraction actually hold for F F since

X Y

addition and subtraction are monotonic over all

Pointwise BestPossible Nature of the Bounds

The b ounds in theorem are the p ointwise b est p ossible The exact meaning of

this is given by the theorem b elowwhich is a generalisation of theorem of

Theorem generalises theorem of in t woways it is for general L L

not just L Sum and it is for general C and not just C W The pro of is

XY XY

based on that in but is sucien tly mo died to warrant inclusion here

Theorem Let F and F be distribution functions in let x letL L

X Y

and let C be a lower bound on the copula C W C C Then

XY XY

There exists a copula C dependent only on the value t of F F at

C L X Y

x such that

F F x F F x t

X Y C L X Y

C L

if not both F and F are discontinuous at u and v respectively such that

X Y

Lu v x

F F x There exists a copula C dependent only on the value r of

L X Y C

such that

F F x r F F x

L X Y X Y C

C L

if not both F and F are discontinuous at u and v respectively such that

X Y

Lu v x

This theorem says that for any dep endency b ounds there will always b e a copula

such that the true convolution meets the b ound at a given p oint In other words

one can not construct b ounds anytighter than and Thus these

dep endency b ounds are the p ointwise b est p ossible

Proof The conditions on the continuityof F and F will b e ignored for nowand

X Y

their eect discussed later Consider part of the theorem rst It will b e shown

that if C a b is the copula dened by

maxt C a b a b t

C a b

mina b otherwise

a b for all a b then F F xt Note that C a b C

X Y

C L

Since L is continuous and nondecreasing in each argument for any x the curve

Lu v x is continuous and nonincreasing in the uv plane see gure

Dene A and B to b e the regions of the extended plane ab oveandbelow the line

x x

Lu v x

A fu v jLu v xg

B fu v jLu v xg

Firstly observe that in view of theorem F F x implies that

C L X Y

F F x Thus it is necessary to show that for t

C L X Y

F F x dC F uF v t

X Y X Y

C L

The condition is t and not t to cop e with discontinuous F and F if

X Y

F and F are discontinuous then F F is also and thus the situation

X Y X Y

C L

B depicted in gure could arise In order to show note that for u v

the closure of B

F uF v F F Lu v F F xt C

X Y C L X Y C L X Y

XY XY

maxt C F uF v t

X Y XY

Figure Graph of Lu v x

Figure A p ossible F F when F and F have discontinuities

X Y X Y C

Thus using the denition of C in

C F uF v minF uF v t

X Y X Y

From this it is obvious that C F uF v for all u v B and so

X Y x

F F x

X Y

C L

It is now only necessary to consider t Let

u supfujF u tg

for t We showthatu is nite Since lim F u u

u X

Regarding whether u supp ose to the contrary that u and thus that

F u t for all nite uNowletL be the two place function dened by

Lu L x u x

That is if Lu v xthenL x u v L is strictly increasing in its rst

Dierence argument and strictly decreasing in its second and if L Sum then L

For nite x and u

F L x u

Since for anycopulaC C a a see section and since C is nondecreasing

in each argument for any C a a Combining these facts gives

F uF L x u F u t C

X Y X

But

F F x sup C F uF L x u

L X Y C X Y

and since u in is arbitrary but nite

F F x F u t t

L X Y X C

whichisacontradiction and thus u is nite

It is next shown that F v t whenever vL x u Supp ose to the contrary

that there exists a v L x u such that F v t Dene L to b e the two

place function suchthat

L Lx v v x

ie Lu v x implies Lx v u and L is strictly increasing in its rst argu

ment and strictly decreasing in its second Since Lx v u F Lx v t

Thus for u Lx v

C F uF L x u F u F Lx v t

X Y X X

and for u Lx v

C F uF L x u F L x u F v t

X Y Y Y

and again

F F x t t

L X Y C XY

Figure C F uF v

X Y

This is a contradiction and so F v t whenever vL x u

A nal fact which is needed is that F u t b ecause F u t for u u

X X

and F is leftcontinuous and thus F u t for u u Also F v t for

X X Y

vL x u

Collecting all the facts in the ab ove three paragraphs and substituting into

the denition of C gives

minF v t uu

F u uu vL x u

C F uF v

X Y

minF uF v uu vL x u

X Y

These values of C F uF v are indicated on gure It now b ecomes fairly

X Y

easy to evaluate F F

X Y

C L

Recall that

F F x dC F uF v

X Y X Y

C L

Luv x

Following Frank Nelsen and Schweizer the u v plane is divided into the ve

regions R R given by

R u v v

R u v

R u u v

R B fu v jvv g

R B fu v ju u g

where If

I dC F uF v

k X Y

then

I F u min F u F v

X X Y

I minF u F v

X Y

I minF v t minF u F v

Y X Y

Both I and I are zero b ecause b oth F u and minF v t are constant in one

X Y

direction of the u v plane The value stated for I is obvious and those for I and

I followby consideration of the b oundaries of R and R resp ectively see gure

Since L is nonincreasing in its second argument and continuous

F F x lim I I I

X Y

C L

F u minF v t min F u F v

X Y X Y

t if F v t

maxF u F v t if F v t

X Y Y

There are several p oints worth noting regarding this theorem

Since v L x u the case that F v t in the ab oveformula implies that

F F xhasa jumpatx Lu v This is explained b elow The

X Y

C L

two cases F v t and F v t are considered separatelyIfF v t

Y Y Y

then maxF u F v t and so the b ound has b een met If F v t

X Y Y

then F has a jump at v This can b e seen by noting that F v t for

Y Y

vL x u and therefore F v t for vv But F v t and so

Y Y

F v t for vv Thus F v tand F v t for vv ie there is

Y Y Y

ajumpat v

If F u tmaxF u F v t and the b ound is still met Assume

X X Y

then that F u t Then by a similar argument F has a jump at u

X X

NowifF has a jump at u and F ajumpat v thensinceL is continuous

X Y

F F has a jump at Lu v and this is why

X Y

C L

F F Lu v t

X Y

C L

The b ounds for dierences and quotients constructed earlier are p ointwise b est

p ossible also This is obvious as the dierence is determined in terms of a sum

and the quotient in terms of a pro duct If one wanted to construct an explicit

copula analogous to the C in the ab ove pro of one could use the fact see

theorem ii of that if g is a nonincreasing function then

C u v u C u v

XgY

The second part of theorem can b e proved in an entirely analogous

manner Frank et al giv e some details The reason for having x rather

than x in part of the theorem is due to F and F b eing leftcontinuous but

X Y

not necessarily rightcontinuous

Note that the F F and F F op erations are not distributions of

W X Y W X Y

some function of X and Y see

Examples

Some examples illustrating the lower and upp er dep endency b ounds are nowpre

sented All of the examples have b een calculated numerically but involve no ap

proximation apart from rounding error which is negligible and an error due to

the termination of the search for the inmum or supremum in calculating the

and op erations Frank Nelsen and Schweizer presen t a few analytical re

sults obtained using the metho d of Lagrange multipliers In general it seems rather

dicult to obtain exact dep endency b ounds using this technique In any case as

was mentioned in section since it is the numerical values that are ultimately of

interest there is no real advantage in deriving analytical results if the formulae are

so complicated that a computer program has to b e written to calculate their sp ecic

values

In order to calculate the dep endency b ounds numerically it is useful to rewrite

the formulae in the following form

ldbF F x supF u F x u

X Y X Y

udbF F x inf F uF x u

X Y X Y

ldbF F x supF u F u x

X Y X Y

udb F F x inf F u F u x

X Y X Y

ldbF F x supF u F xu

X Y X Y

udbF F x inf F uF xu

X Y X Y

ldbF F x sup F u F ux

X Y X Y u

udbF F x inf F u F ux

X Y X Y

where x maxx and x minx From these formulae it b ecomes

straightforward to calculate lower and upp er dep endency b ounds if one has a sub

routine to calculate F x and F y The calculation is simply a searchfora

X Y

maximum or minimum of a simple function of F and F for a given xThisisnot

X Y

completely trivial to do numerically as the F and F are dened on a continuum

X Y

The techniques develop ed in section are b etter suited for numerical calculation

All of the examples are for random variables with uniform distributions U given

x a

x a b U x

x b

The examples are presented in gures and details are provided in the gure

W are given in section captions Some examples of dep endency b ounds when C

Some p oints to note ab out these gures are

Comparison of gures and reveals the eect of the spread of the two

random variables on the distance b etween the lower and upp er dep endency

b ounds In fact if one of the random variables has zero disp ersion a distribu

tion function equal to a unit step then the two b ounds are identical

The fact that the lower and upp er b ounds in gures and are themselves

of the form U is not really signicant it is just a p eculiarity of the ldb

and udb formulae for sums and dierences of uniformly distributed random

variables Figure shows the dep endency b ounds for a pro duct It can b e

seen that these are not uniform

A comparison of gures and indicates the eect of identical distributions

for numerator and denominator on the dep endency b ounds of their quotient

If the random variables are almost surely equal then the distribution of their

quotientwillbea unitstepat x This p ossibilityiscontained within the

b ounds of gure

The b ounds for the following two cases are identical to those in gures and

resp ectively

a Z X Y with X distributed U and Y distributed U

b Z X Y with X distributed U and Y distributed U

Examples of dep endency b ounds for random variables with nonuniform distri

butions are given in section

Figure Lower and upp er dep endency b ounds for X Y where b oth X and Y are

uniformly distributed on

Figure Lower and upp er dep endency b ounds for X Y where X is uniformly dis

tributed on and Y is uniformly distributed on

Figure Lower and upp er dep endency b ounds for X Y where b oth X and Y are

uniformly distributed on

Figure Lower and upp er dep endency b ounds for X Y where X is uniformly dis

tributed on and Y is uniformly distributed on

Figure Lower and upp er dep endency b ounds for X Y where b oth X and Y are

uniformly distributed on

Exclusive Use of Lower and Upp er Bounds

We will nowshowhow it is p ossible to work only with the dep endency b ounds them

selves without worrying ab out the distributions within these b ounds the precise

meaning of this statement is given by the following theorem

Theorem Let F F and F F betwopairs of distributions functions

X Y

such that F x F x and F y F y for al l x y and let be non

X Y

decreasing in each argument Then for any F F F and any F F F

X X Y Y

X Y

F F ldb F F ldb

C X Y C

X Y

XY XY

and

udb F F udb F F

C X Y C X Y

XY XY

Proof This follows directly from lemma of

The only thing that remains to b e determined is explicitly how to use the F

and F But this is very simple The following formulae are derived by considering

the increasing or decreasing nature of the op erations for f g we only

consider random variables on

F ldbF F

Z X Y

F udbF F

Z X Y

F ldbF F

Z X

F udbF F

Z X

F ldbF F

Z X Y

F udbF F

Z X Y

ldbF F F

Z X

F udbF F

Z X

These formulae provide a consistent and neat metho d of handling the dep endency

error arising in probabilistic arithmetic In section the numerical implementation

of these b ounds will b e discussed

Numerical Representation

As the examples in section show it is p ossible to use the formulae

to calculate the lower and upp er dep endency b ounds numericallyHowever

there are diculties asso ciated with this scheme For example the search for the

supremum and inmum is over a continuum of p oints without even a b ounded

range In this section new techniques for b oth determining the lower and upp er

dep endency b ounds the and op erations and the calculation of the ordi

WL WL

nary convolutions will b e develop ed The main to ol used is a duality theorem

relating the op erations and to functions in terms of quasiinverses of the

WL WL

distribution functions involved The convolutions can also b e calculated in terms

of the quasiinverses As well as b eing computationally simpler for all these calcula

tions this approachprovides a neat solution to the problem of representation error

in the numerical calculation of convolutions Whilst most other techniques for

the numerical calculation of convolutions of probability distributions allowone

to control the error in the sense that it approaches zero as the numb er of p oints

used to represent the distribution function approaches innity they do not allow

the calculation of rigorous b ounds on the error in anygiven calculation In contrast

the metho d presented here provides lower and upp er b ounds b etween which the

probability distribution in question must lie

The present section is arranged as follows In subsection wemake use

of a duality theorem to express the quasiinverses of dep endency b ounds in terms

of quasiinverses of the distribution functions In subsection weintro duce

our numerical representation and develop explicit formulae using the representa

tion for calculating dep endency b ounds Subsection contains details on how

convolutions can b e calculated using this same numerical representation In sub

section it is shown how functionals of distribution functions such as moments

can b e easily calculated in terms of the numerical representation and in subsection

some examples are presented which demonstrate the use of all the techniques

develop ed here

Duality

The following duality theorem has b een presented in a varietyofcontexts and in

anumb er of dierent forms It is the basis of the levelset or cut formulae for

implementing the fuzzy number convolutions see chapter Frank and Schweizer

ha ve considered the duality in some detail Their results are summarised in

theorem of whic h is restated b elow

Denition For any F in let F be the left continuous quasiinverse of F

denedinsection Then r is the set fF jF g

Denition For any two place function L and any copula C is the func

r r r given by tion

F G x inf LF uG v

C uv x

is the func Denition For any two place function L and any copula C

r r r given by tion

F G x sup LF uG v

C uv x

Theorem Theorem of Let L be a function L

with as its nul l element and which is continuous everywhere Let C bea

copula Then for any F G

F G F G

The advantage of using the duality theorem b ecomes apparent when a sp ecic

L is considered For example if L Sum then the quasiinverse representation of

the lower and upp er dep endency b ounds can b e calculated in terms of the quasi

inverses of the distribution functions of the random variables in question by using

the following formulae Let Z X Y and let ldb F and udb F denote

Z Z

C C

XY XY

the quasiinverses of ldb F F and udb F F resp ectively Then

C X Y C X Y

XY XY

F F F uF x inf v ldb

X X Y Y C

uv x C

F u F v udb F F x sup

X Y C X Y

uv x C

The standard dep endency b ounds with C W follow with the appropri

ate substitution Equations and are simply the maximum and mini

F F and mum of the p ointwise sum of quasiinverses The functions ldb

X Y C

udb F F are quasiinverses of the lower and upp er dep endency b ounds

C X Y

and not lower and upp er b ounds on the quasiinverses That is

udb F F ldb F F

C Y Y C X X

XY XY

is the consequence of

F F udb F F ldb

X Y C X Y C

XY XY

Note that for L Sum or Dierence theorem holds for any F G and

not just b ecause in these cases the p oint x has no sp ecial signicance

When C W the formulae are particularly simple and this gives us a very

simple way of calculating lower and upp er dep endency b ounds Recalling that

W u v maxu v and W u v minu v wehave

inf F uF x u if x

X Y

ldb F F x

X Y

v if x u F inf F

Y X

Since F and F are nondecreasing the case for x b ecomes

X Y

ldb F F F F

X Y X Y

Similarly

sup F uF x u if x

X Y

udb F F x

X Y

F F if x

X Y

The restrictions on the range of the supremum and inmum op erations arise from

the fact that v b ecause dom F Likewise wehave

x u if x u F inf F

Y X

x F ldb F

Y X

F F if x

X Y

and

sup F u F x u if x

X Y

udb F F x

X Y

F F if x

X Y

Analogous formulae for ldb and udb for f g can b e determined in a

manner similar to that used to derive and Consider the quotient rst

Let Y Y IfF has a quasiinverse F what is the quasiinverse of F Itis

Y Y

x y x F y andF xF x If x F y known that F

Y Y Y Y

xy But x F y Therefore F xy and so then F

Y Y

F x y

F x

Using this and it can b e shown that

inf F uF u x if x

X Y

ldb F F x

X Y

F F if x

X Y

and

sup F uF u x if x

X Y

udb F F x

X Y

F F if x

X Y

Similarly it can b e shown that if Y Y then F xF x and so using

Y Y

and

u x if x u F inf F

Y X

x F ldb F

Y X

if x F F

Y X

and

sup F u F u x if x

X Y

x F udb F

Y X

F F if x

X Y

These formulae for ldb and udb can b e seen to b e slightly simpler than the

corresp onding formulae for ldb and udb The one disadvantage is the

Figure The copula W

Figure The numerical representation of probability distributions

sp ecial case for x forand orforx for and The reason these

sp ecial cases are necessary b ecomes apparent up on insp ection of gure which

shows the copula W When W u v x the values u and v can take are

constrained by a linear relationship However when W u v u and v can take

anyvalues within the crosshatched region When a numerical representation of the

quasiinverses is used these sp ecial cases actually disapp ear

Numerical Representation and the Calculation of ldb and udb

In order to use equations to calculate lower and upp er dep endency

b ounds numerically it is necessary to have a discrete approximation to probability

distribution functions dened on a continuum The approach taken is illustrated in

gure The distribution F is approximated bylower and upp er discrete approx

imations denoted F and F resp ectively These are formed by uniformly quantising

F along the vertical axis see gure This denes the p oints x andx For a

x for i n We retain the two sets of numbers single valued F x

g and fx g for clarityInterval valued distributions ie lower and upp er b ounds fx

on F can result in x x The end p oints are

inf supp F x

sup supp F x

The ab oveapproximation of F motivates the following numerical approximation

to F Both in anticipation of the exclusive use of upp er and lower b ounds to

contain b oth dep endency and representation error and to make the ideas clearer

and F are presented in gure In the discrete approximations F and F to F

the following discussion it is not assumed that F F although the same results

do apply to this situation which is often the case at the b eginning of a calculation using probabilistic arithmetic

Figure The quasiinverse representation of a distribution function

The discrete approximations F and F are dened by

F p p p p F p

i i i

p p p p F pF

i i i

for i N Whether the range in these denitions is p p orp p

i i i i

do esnt really makeany dierence as long as one choice is used consistently there

are no problems N is the number of points required to represent either F or

F In gure N Note that p The quantisation is uniform and so

and F suitable p iN for i N Explicit array representations of F

for computer implementation are given by

F i F p F

and

F iF p F

for i N Notice the dierentconventions used for dening F and F

and in terms of F F This is necessary b ecause of the nature of the approximation

Two sp ecial values worth noting are

F p F F

and

F N F p F

These twovalues corresp ond to the p oints a and b in gure Regarding F

recall the convention of redening the value of F at the end p oints of the range

see section This convention resulted in F inf supp F and so F

is set equal to this

An imp ortantadvantage of this representation is that any representation error

dierence b etween F and F or b etween F and F is rigorously contained within the

F F or F F b ounds That is it is p erfectly correct to state that F F

although this do es not b ound F as tightly as stating F F F By always

p erforming any necessary rounding approximations in a manner such that the width

between F and F is made larger outwardly directed rounding this prop ertyis

preserved This is referred to b elow as the preservation of the representation error

containment prop erty The representation error for any given approximation can

b e made arbitrarily small by increasing N The metho d of approximation prop osed

here is b etter than simply using a single function F to approximate F for which

appr ox

jF x F xj is small for most x

appr ox

Nowthata numerical representation has b een chosen it is necessary to derive the

appropriate formulae for ldb and udb the lower and upp er dep endency b ounds

in terms of the numerical representation We explicitly derive the four cases of the

lower and upp er dep endency b ounds for sums and dierences

i ldb F F

X Y

Equations and give

F x inf F uF x u x ldb F

X X X Y

Let x and u Then

N N

j j

i i i

F F ldb F inf F

X Y X Y

N N N N N

j ij N

F i inf F

X Y

N N

j iN

Using the corresp ondences and this can b e written as

i j N i j F i inf F F ldb F

Y X Y X

j iN

Since it is only required to calculate ldb ifor i N the sp ecial case

for x in has b een avoided as wementioned was p ossible at the end of

section

ii udb F F

X Y

Equations and give

udb x sup u x u x F F F F

X Y X Y

and u Then Let x

N N

j j

i i i

udb F F sup F F

X Y X Y

N N N N N

h i

N N

Using the corresp ondences and gives

F F i sup F j F i j i N udb

X Y

j i

for i N and the sp ecial case x hasbeenavoided

iii ldb F F

From and

u F ldb F F x inf F u x x

X X Y Y

and u Then Set x

N N

j j

i i i

F F F ldb F inf

h i

Y Y X X

N N N N N

N N

Using the corresp ondences and

ldb F F i inf F j F j i i

X X

Y Y

j iN

which is all that is required for i N

F iv udb F

Using and

udb F F F x sup u F u x x

X Y X Y

Setting x and u gives

N N

j j

i i i

F F sup F F udb

X X Y Y

N N N N N

h i

N N

and thus

j F j i N i N F F i sup F udb

Y Y

X X

j i

which holds for i N

The analogous formulae for pro duct and quotient are

ldb F F i inf F j F i j N i

X Y X Y

j iN

udb F F i sup F j F i j i N

X Y X Y

j i

ldb F F i inf F j F j i i

X X

Y Y

j iN

udb F F i sup F j F j i N i N

Y Y

X X

j i

All these hold for i N The use of lower and upp er b ounds alone instead

of distributions within the b ounds and whether F or F should b e used in a

particular instance is discussed at the end of section

Two signicantpoints to note ab out the ab oveformulae for ldb and udb

are their low computational complexity and their lackofapproximation error To

calculate a dep endency b ound with N points in terms of two N point discrete

approximations requires only O N op erations The results are free from error in

the two senses of any representation error b eing rigorously b ounded by F and F

see ab ove and the supremum and inmum op erations are exact compared with

the numerical implementation of the formulae develop ed in section for which

approximation is required

The nal p ointwhich needs to b e settled is how to generate F and F from

p a given exact distribution F This is in fact quite simple One just sets F

F pF p for p and uses the denitions and The

X X

only diculty is nding F p given a formula for F x While for some simple

distributions such as the uniform and triangular distributions an exact formula for

F p can b e derived in general one has to p erform a numerical search This will

give F ptoany desired accuracy This metho d is describ ed along with various

techniques for numerically calculating F for some common distributions in chapter

of Some examples are given later When the distribution F has unb ounded

supp ort it is necessary to curtail the distribution so that F and F N are

nite

Numerical Calculation of Convolutions

Wenow examine the calculation of convolutions in terms of our numerical repre

F and F F F sentation First we determine whether for all F F

X Y Y X

X Y

F F F F F F

X Y L X Y L L

X Y

It will b e seen that this is indeed the case for arithmetic op erations dened on

To see that holds for addition and multiplication it suces to insp ect

the convolution relations which can b e written

F x t dF t F x

X Y Z

and

F xt dF t F x

X Y Z

resp ectively If F is considered as xed then since F is always greater than

Y X

it is apparent that

F x x t dF t F

Z X

is always less than or equal to F Likewise for upp er b ounds F andby symmetry

Z X

for F as well In the same manner it can b e seen that

F x F xt dF t

Z X

is always less than or equal to F if F is considered xed Again by symmetry

Z Y

this holds for upp er b ounds and for F as well Similar arguments could b e used to

show that analogous results hold for dierence and quotient convolutions In these

cases it is necessary to use the lower or upp er b ounds on F and F in a manner

X Y

similar to equation The details are omitted As well as showing that only

the lower and upp er b ounds on a distribution need b e considered the ab ove result

comes in useful in the condensation pro cedure which is part of the algorithm for

calculating F F

L X Y

The metho d of calculating F F is b est presented in terms of the dis

L X Y

crete frequency functions corresp onding to F andF Thus for the moment we

X Y

will ignore the use of lower and upp er distributions and we will consider the calcu

F F Other op erations L and the complications of using lower lation of

X Y

and upp er distributions are considered later on in this subsection

Given F and F of N p oints each there are corresp onding discrete frequency

X Y

functions f and f given by

X Y

if x F ifor k dierent i

f x

otherwise

if x F ifor dierent i

f x

otherwise

The discrete frequency function of Z X Y is obviously given by

if x F i F j for m dierent pairs of i and j

Y Y

f x

otherwise

This formula for F follows directly from the laws of probability and has b een

the basis for a numb er of metho ds for calculating convolutions in terms of discrete

frequency functions

expressed in Wethus have the following simple algorithm for calculating F

the syntax of the C language with the exception mentioned in section

fori iN if

forj jN j f

j i F i j N F F

Y X Z

E U

F sort F

Z Z

Figure Illustration of the condensation pro cedure

U E

The array F is unsorted hence the U and is of size N The array F is the

Z Z

result of sorting F into increasing order the E stands for Exact Observethat

E E

it is quite p ossible for F i F j for i j This do es not matter at all it

Z Z

simply results in a bigger jump in the corresp onding F An imp ortant fact to notice

is that while F and F have only N p oints F has N This will obviously

X Y Z

cause severe diculties if weintend to p erform a sequence of op erations the output

of each op eration will b e an array considerably larger than its input What is

required is a metho d for reducing the size of F to N without intro ducing any

error into the result The pro cedure used to do this is called condensation

If F or F were simply represented by a single discrete version F then it

Z Z

would b e imp ossible to approximate F byanarrayofN points without error

However as b oth lower and upp er approximations are available directed rounding

that is not in error in the sense can b e used to pro duce an approximation to F

that the quasiinverse of the distribution is contained within the b ounds That is

it will b e true that

E E

F F F F F

Z Z Z Z Z

where the inequalityis and not b ecause we are talking ab out quasi

inverses The metho d of determining F and F is indicated in gure Only

the pro cedure for the lower b ound is shown there The rule for condensation is

simply

F i F i N N

Z Z

The analogous condensation with upward rounding for the upp er b ound is

F i F i N

Z Z

The overall algorithm for the distribution of the sum of random variables b ecomes

algorithm see b elow The algorithm for pro ducts is identical apart from

fori iN if

forj jN j f

F i j N F i F j

Z X Y

j i F i j N F F

Y X Z

E U

F sort F

Z Z

E U

F sort F

Z Z

fori iN if

F i F i N N

Z Z

F i F i N

Z Z

Algorithm

a replacement of the on the righthandsideoflines and bya For

subtraction lines and b ecome

F i j N F i F j

Z X

j F i j N F i F

Z X

The algorithm for quotients is obtained by replacing the subtraction in the ab ove

two lines by a division Some examples using these algorithms are presented in

section

Calculation of Moments and other Functionals

Wenow consider the calculation of moments and other functionals of F in terms of

F and F Using the change of variables result in lemma of w ehave

Z Z

k k

m F x dF x t dt

X X

and

Z Z

x dF x t dt F

X X X

where is the mean and m These equations can b e used to calculate

X X

moments in terms of F and F by replacing the integral by the appropriate

summation and by realising that we will only b e able to calculate lower and upp er

b ounds on the moments Regarding the lower and upp er b ounds it is easy to see

that if

M i min F F i i F i F i

X X

X X X X

X X

M imax F i F i F i F i

X X

X X X X

X X

h i

and M i M i M i then

M i

will b e lower and upp er b ounds on for k The op erators and are

the interval arithmetic lower and upp er b ounds for exp onentiation byanintegral

W then power If W W

W W if W or k is o dd

W W W and k is even if

jW j if W and k is even

j jW j where jW j maxjW

The use of these equations is discussed in section with reference to some

examples Functionals other than moments could b e calculated in an analogous

manner

Examples

As an illustration of the ideas develop ed ab ovewenow present some examples In all

of these N the numb er of sample values of the lower or upp er quasiinverses is

All the results presented here are outputs of computer programs which implement

the formulae derived ab ove

Figures and show the distributions arising from the numerical represen

tation of Y and Z resp ectively where Y is uniformly distributed on and Z has

a Gaussian distribution with curtailed to These were generated

by using the pro cedure describ ed in section Note that in each case the lower

and upp er distributions toucheach other at the fty sample p oints Figure

shows the distribution of X YZ calculated using the algorithm for derived

earlier Notice that the lower and upp er distributions no longer touch This is due to

the outwardly directed rounding in the condensation pro cedure The magnitude of

this eect can b e seen from gures and where we present the distribution

calculated directly from the exact quasiinverse of a triangular distribution centred

at with spread of and that of the distribution obtained using the algorithm

Figure Numerical representation of F uniform distribution on with N

The moments as calculated by are

and

Figure Numerical representation of F a Gaussian distribution with and

and which is curtailed at It is represented with N The values of

the moments as calculated by are

and

Figure Numerical representation of F where X YZ andY and Z are as in

gures and resp ectively This was calculated using the convolution algorithm

Figure Numerical representation of an exact triangular distribution centred at with

a spread of

Figure Output of the convolution algorithm for two random variables uniformly

distributed on This is exactly the same as gure apart from the eect of the

representation error in the inputs and the outwardly directed rounding in the condensation pro cedure

Figure Lower and upp er dep endency b ounds for the same calculation as for gure

The values of the moments as calculated by are

and

for for the sum of two uniformly distributed random variables on The

dierence b etween this representation width and the dep endency width can b e seen

by examining gure where we present the lower and upp er dep endency b ounds

for the same calculation as for gure

The eect of the dierence b etween lower and upp er distributions on the tight

ness of the b ounds for the moments can b e seen by comparing the various values

presented For cases such as that presented in gure the b ounds on the mo

ments are very lo ose esp ecially for the higher order moments This is to b e exp ected

as one can t a very wide range of distributions b etween the lower and upp er b ounds

Thus the b ounds for moments would app ear to b e of little value

Incorp orating and Up dating Dep endency Information

General Idea

So far haveshown how to calculate lower and upp er b ounds on the distribution

of some arithmetic op eration on random variables when it is either known that

they are completely indep endent or there is no knowledge of their dep endency

at all However there are situations b etween these two extremes These are now

considered It will b e seen that the algorithms rapidly b ecome more complicated

as more information is taken into account The metho ds involved can b e organised

according to the following classication

Using the and op erations

Pairwise b ounds on copulas or pairwise measures of dep endence

Pairwise joint distributions

Either joint distributions of all the variables or combination of all the variables

at once rather than pairwise combinations

So far in this pap er wehave only considered typ e In this section typ e and

in less detail typ e will b e considered Typ e is generally intractable but see the

section on graph theory in chapter

The fact that a lower b ound on a copula C other than W can b e used to

calculate dep endency b ounds was shown in section The lower b ound C

could describ e b ounds on the dep endency of some input random variables or it

could arise in the pro cess of a probabilistic arithmetic calculation Dep endency

W are closer together than those based on W In this b ounds based on C

section we will briey examine dierentlower b ounds on copulas and their eect on

the dep endency b ounds We will also consider the determination and interpretation

of lower b ounds other than W

Interpretation of Dep endencies Implied by C W

The eect of a C other than W is illustrated in gure where lower and upp er

T are presented Here X dep endency b ounds for the sum of X and Y with C

and Y are b oth uniformly distributed on and T is the parameterised tnorm

discussed on pp of and giv en by

p p p

T x y maxx y p

T x y xy

for p This tnorm is related to W and M by T W T and

lim T M

p p

Whilst it is p ossible to calculate these more general dep endency b ounds it is

obviously imp ortantthattheyhave a useful interpretation and that the dep endency

Figure Lower and upp er dep endency b ounds for the sum of two random variables

each uniformly distributed on when the lower b ound on their connecting copula is

given by T for various p p

induced by C W can b e understo o d The family of copulas fT j p g is

in fact only one of a numb er of parameterised copulas that provide an innite number

of copulas b etween W and M All these parameterised copulas are in fact t

norms They have b een also used in fuzzy set theory as generalised intersection

op erators Thus the rst question we should ask is what probabilistic

interpretation can b e ascrib ed to copulas that are also tnorms ie what do es

asso ciativity of a copula imply ab out the dep endence of random variables Schweizer

and Sklar asked this question in problem of A simple argument presented

in chapter shows that if a copula C is asso ciative and satises C a a

XY XY

a a ie it is Archimedean then it is an increasing function of a joint

distribution of independent random variables C x y hF x y U and

XY UV

V are indep endent However it is not apparent what this means probabilistically

Nevertheless such parameterised copulas have b een used in practice See for example

We are actually interested in these parameterised copulas for their role as lower

bounds on the actual unknown copula We can understand their eect to an extent

by making use of the following result whichisgiven by Jogdeo

Theorem Let F and G be two bivariate distribution functions such that

F x y Gx y x y Then for every pair of nondecreasing functions f

and g denedon

cov f X gY cov f X gY

G F

where cov f X gY is the covarianceoff X and g Y given that X and Y

have joint distribution F

This says that sto chastic ordering of F and G implies sto chastic ordering of the

asso ciated covariances For our purp oses it allows us to say that

cov X Y cov X Y

C C

and so cov X Y isalower b ound on the covariance of X and Y By normalising

the covariance appropriately we can calculate lower b ounds on the correlation co e

This gives us an intuitive feel for the dep endence cientof X and Y implied by C

implied by C

The lower b ounds on covariance can b e calculated as follows Wehave the

formulae

Z Z

covX Y F u v F uF v du dv

XY X Y

Z Z

C u v uv dF u dF v

X Y

see If we just consider uniform marginals F F U then the

X Y

lower b ound on the correlation co eecient r x y isgiven by

Z Z

r X Y u v uv du dv C

D X D Y

p r X Y

Table Lower b ounds on the correlation co ecient implied bylower b ounds of the form

T on the connecting copula for various p

where D X is the standard deviation of X For F F U D X

X Y

This will b e a lower b ound on r only for uniform marginals Other marginal distri

butions will give dierent results This is a failing of the correlation co ecentas

an index of dep endence it is not invariant under transformations of the marginals

T If C

Z Z

p p p

maxx y xy dx dy p X Y r

where

Z Z

p p p

I maxx y dx dy

with the aid of The integral I can b e calculated exactly for p

For example with p wehave x y x y and so

I J y dy

Using of w e obtain

J y dx

x y

y y

lny

y y

The integral I can then b e evaluated using and of to give

I andthus r X Y the other cases are determined similarly

and the results are summarised in table Bounds on other indices of dep endence

can also b e calculated For example Genest and Mackay ha veshown that

is for T Note that the interpretation of C Kendalls is given by

particularly simple it says that X and Y are positively quadrant dependent see

Statistical tests for p ositive quadrant dep endence are considered in

Use of C W in Probabilistic Arithmetic Calculations

In order to use lower b ounds on the connecting copula to calculate tighter dep en

dency b ounds it is necessary to calculate the dep endency arising through the course

of a probabilistic arithmetic calculation That is given random variables W X and

Y with Z X Y for some arithmetic op eration calculate C C and C

ZX ZY ZW

cf If there were other variables U V etc it in terms of F F and C

X Y

and C etc Since the calculations would b e would b e necessary to calculate C

ZU ZV

of the same form as those for C this is not considered further We will consider

addition here The details for other op erations are similar

The joint distribution of Z and X is given by

dF x y F z w

XY ZX

where D is the region in the x y plane suchthatx yz and x w see

pp Thus

F z w dC F xF y

ZX XY X Y

and so

C u v dC F xF y

ZX XY X Y

F uF v

Z X

This is very similar to a convolution Using an argument along the lines of that

used to prove theorem it is clear that

u v sup C F xF y C

X Y

ZX XY

xy F u

y F v

It is p ossible to use C rather than C by an argument similar to that used to

prove theorem Equation is not as complex computationally as it seems

For example given a value of t C u v it is easy to calculate C u v by

ZX ZX

u v max sup F xF y C u v C C

X Y

ZX XY ZX

F v y F v

X X

u xy F

Obviously C can b e calculated in a similar manner

The calculation of C is rather more complicated The joint distribution of Z

and W is given by

ZZ Z

F z w dF x y v

ZW XY W

xy z

v w

F F v z dv

X Y

v w

where F F v z is simply F F z for a given v ie for W v Thus

X Y X Y

t dv F F v F C t u

X Y ZW

v F u

and

C F F v F t dv t u ldb

X Y C

v F u

F F v is the lower b ound on F F v and is given by where ldb

X Y X Y C

ldb F F q x F q F q x

C X Y C

X Y

XY XY

F F q C F F q x C

XW X W YW Y W

sup C C F uF q C F v F q

XY XW X W YW Y W

uv x

Up on substituting with a few changes of variables one obtains

sup dv C t u C F aF v C F bF v C

ZW XY XW X W YW Y W

abF t

u v F

Again this is not as computationally complex as it lo oks b ecause

C dv t u C

ZW ZW

F uv F u

W W

where the term in the parenthesis is the same as that in the previous equation

The numerical implementation of the ab oveformulae will intro duce further com

plications For instance a numerical representation of C that ts in neatly with

the representation already adopted for distribution functions will have to b e found

Further investigation is required to determine the feasability of the approach out

lined here

Measures of Dep endence

Wehave seen that dierentlower b ounds on C induce or imply dierentvalues

of indices of dep endence eg the correlation co ecient The converse is not nec

essarily true dierentvalues of r do not imply unique corresp onding lower b ounds

on the copula Nevertheless b ecause of the complexity of the approach outlined in

the previous subsection it seems worthwhile to examine what eect knowledge of

the values of dierent measures or indices of dep endence can have on the result of

some op eration combining two ormorerandomvariables We simply p oint to some

of the literature here

Measures of dep endence based on copulas are discussed in Other

typ es of dep endence measures are numerous See for example Explicit

consideration of the eects of convolution on various measures of dep endence can

b e found in this is not an exhaustiv e list So far wehave b een

unable to develop an appropriate metho d of dealing with op erations on random

variables when some dep endence information is available

Use of Mixtures for Nonmonotonic Op erations

Intro duction and General Approach

If the random variables to b e combined under a division or multiplication op era

tion are not sign denite ie the distributions are not suchthat F or

then the required convolution can not b e calculated using the ab ove techniques

or simple mo dications thereof In such a case a new approach is called for and it

is this which is the sub ject of the present section Because of the intricacy of the

results so far obtained and b ecause of some remaining problems the presentsec

tion gives less detail and is more tentative than other sections in this chapter The

general idea of handling nonmonotonic op erations is to split the op eration up into

monotonic segments p erform the op eration and then combine the results together

again In the case under consideration here this entails splitting the random vari

ables involved into p ositive and negative parts combining the various parts under

the split op eration and then recombining them The basic concept we use for this

is a mixture

The pro cess is easily explained if we restrict consideration to single valued rather

than lower and upp er probability distributions which are dened on The nu

merical approximations are considered later Let F and F b e the probability

X Y

distributions of tworandomvariables X and Y neither of which is sign denite

Do the following for b oth X and Y expressions are only given for X Split X into

p ositive and negativepartsIfp is the probabilitythat X is p ositive then

p F

X X

and the distributions of the parts are

F x F F x

X X

F x F F x

X X

for x From now on it will b e assumed that F and F are continuous at x

X Y

and so F F There is no loss of generality in doing this as it is always

X X

p ossible to assume that any jump at x is in fact due to the random variable

X b eing a mixture of two random variables one without the jump and the other

consisting solely of the jump cf the Leb esgue decomp osition theorem p

by and F The distribution F can easily b e reconstructed from F

X X

p F x p x

X X

F x

p p F x x

X X X

In order to calculate the result of a convolution or dep endency b ound in terms

of the p ositive and negative parts it is simply necessary to observe that the p ositive

part F is for Z X Y or Z X Y solely determined by F F and

Z X Y

F F and that the negative part F is solely determined by F F and

X Y Z X Y

Let p denote the probabilitythat Z is p ositive Then F F

Y X

p p p p p

Z X Y X Y

The parts F and F are given by

Z Z

h i

F z p p F F z p p F F z

X Y X Y

Z X Y X Y

h i

p p F F z p p F F F z z

X Y X Y

X Y X Y Z

for zand f g Substituting Z for X in gives the formula for

creating F from F and F

Z Z

Complications Arising from the use of Lower and Upp er Proba

bility Distributions

The ab oveformulae are quite straightforward However things b ecome more com

plicated when the analogous formulae for lower and upp er probability distributions

are considered The corresp onding formulae for and are re

sp ectively

p F

x F F x F

X X

F x F x F

x F F F x

X X

F x F F x

for x and

x p F p x

X X

F x

p F x x p

X X

x p F p x

X X

F x

p p F x x

X X

F xF xas Substitution of into gives F x F xand

X X

one would exp ect

It is apparent that it is p ossible to calculate the dep endency b ounds in terms of

the p ositive and negative parts in the usual manner What is not apparenthowever

Figure Illustration of i quantities used in splitting a distribution into p ositiveand

negative parts and ii the reason why p F is not always necessarily asso ciated with

nor p with F

X X

is whether such results can b e combined together using equation to givea

p Rememb ering and meaningful result Let us rst examine how to calculate p

the viewp oint adopted regarding the lower and upp er distributions F and F lower

and upp er approximations to a single xed but unknown distribution F it seems

and reasonable to dene p p by

mint t t t p

p maxt t t t

where

t p p p p

X Y X Y

t p p p p

X Y X Y

t p p p p

Y Y

X X

p p p p t

X X

Y Y

p are used in any one p ossible choice p is uniquely and That is not both p

dened it is just that its valueisnotknown

In order to combine the various s together to give F F F F it and

Z Z Z Z

must b e realised that p is not necessarily asso ciated with F This is b ecause the

situations depicted in gure can o ccur Thus it is necessary to calculate F

F F F by and

Z Z Z

i h

z c p p F p p p p p p min

Y X X Y

X Y X Y

p p p p where F F F F z p p z

X Y X Y X Y

X X Y Y

h i

p F c max p p p p p p p z

Y X X Y

X Y X Y

where p p p p F F z p p F F z

X Y X Y X Y

X Y X Y

i h

p p p p p min p p p F z c

Y X X Y Z

X Y X Y

where p p p p F F F F z p p z

X Y X Y X Y

X X Y Y

h i

F c max p p p p p p p p z

Y X X Y

X Y X Y

z p p z In all F F where p p p p F F

X Y X Y X Y

Y Y X X

of these formulae c is a normalising constantsuch that F The and

functionals are such that the appropriate lower or upp er comp onentofits

arguments is chosen see sections and Some information is lost in this

pro cedure b ecause it is not known that p is necessarily asso ciated with F etc

and thus the b ounds so obtained would not b e p ointwise b est p ossible even ignoring

the ordinary approximation error Nevertheless they would app ear to b e the b est

p ossible given the approach that has b een adopted The question whether

can b e used to calculate F and and F from F F F F will b e examined

Z Z Z Z

b elow

Diculties Intro duced by Using the Numerical Approximations

Unfortunately further diculties are intro duced when the ab oveformulae are imple

mented in terms of the numerical approximations of the distribution functions For

instance just the splitting of the probability distribution into p ositive and negative

parts causes problems b ecause of the need to maintain b oth the equispaced quan

tisation and the representation error connement prop erty One could either form

p ositive and negative parts of N points each from an N point initial distribution or

one could keep the same p oints values of F i resulting in negative and p ositive

parts of N and N points resp ectively with N N N N The rst

choice results in approximations b eing necessary in the form of further outwardly

directed rounding The second choice has b een adopted here although as will b e

seen b elow it is not without problems either Having done this it is necessary to

generalise our algorithms for calculating lower and upp er dep endency b ounds and

convolutions to handle inputs of diering sizes ie a dierentnumb er of p oints

used in the approximations The appropriate equations and algorithms which are

presented b elow were derived in a fairly straightforward but longwinded manner

and so the derivations are omitted

Let N be the number of points with which F is to b e represented Let M

be the number of points used for F and let P b e the numb er of p oints used for

M P

F This is explicitly indicated in the formulae b elowby writing F F and

Y X Y

ldb or F The dep endency b ounds are given by

M P

ldb F F i

X Y

h j ki

P M

P iM j N NM

j F min P F inf

Y X

j M

b c

L M PN

for i iM if

forj jP j f

P M M PU

j i F i P j F F

Y X Z

M PU M P

F i P j F i F j

Z X Y

M PE M PU

F sort F

Z Z

M PE M PU

F sort F

Z Z

for i iN if

N M PE

F i F i L L

Z Z

N M PE

F i F i L

Z Z

Algorithm

M P

udb i F F

X Y

h l mi

M P

P iM jN

sup F j F max

X Y

d e

P M

i F ldb F

h l mi

P M

P N j M i

inf F j max F

iM N

j M

b c

M P

udb F F i

ki h j

P M

P jN iM NM

j F min P F sup

Y X

e d

for i N The symbols and mean that the formulae hold for

either or or or resp ectively Equations reduce to equations

when M P N It is imp ortant to calculate the complex index

expressions in the way they are written here Otherwise roundo error can b e

a problem b ecause of the o or and ceiling op erations The only oating p oint

op eration necessary if the formulae are programmed as written here is a division

As long as oating p oint division of an integer by one of its integer divisors gives an

exact integer as a result there should b e no problems

The algorithm for calculating convolutions needs to b e mo died also The

resulting algorithm for f g is presented as algorithm This algo

rithm will only work if N jMP ie if L is an integer Otherwise the condensation

Figure Result of the numerical calculation of a convolution of X and Y The

variable X is uniformly distributed on and the variable Y has a Gaussian distribution

with and and is curtailed to In b oth cases N The thin line

is for N and the thick line for N

X X

pro cedure will have to b e considerably more complex and b ecause of necessary ap

proximations it would b e less accurate more outwardly directed rounding would

b e required The condition N jMP do es not matter however b ecause as will b e

shown b elow it is natural to adopt the convention that N minM P and in this

case the divisibility condition always holds The mo dications to the convolution

algorithm for f g are exactly the same

The ab ove formulae and algorithms give results which are not at all surprising

The two examples presented in gures and show that the representation

error containmentproperty has b een preserved

Numerical Algorithms for Splitting Distributions into Positive and

NegativeParts and Recombining Them

Having disp ensed with the ab ove preliminaries our attention can now b e fo cussed

on the details of implementing As was the case with the original

convolution algorithm section it will b e found simpler to work directly with

the algorithms rather than attempting to determine appropriate formulae rst

Figure The lower and upp er dep endency b ounds for subtraction for the same vari

ables as for gure Again N and the two cases presented are for N and

Y X

N X

First the positive partisconstructed

fori F i i

p iN

fork iN F k F i

N k

fori F i i

p iN

forj jN N i F j

whilejN F j F i

X X

Now the negative part is constructed

fori F i i

N i

fork i F k F i

X X

i i fori F

forj jN i F j

whilejN F j F i

Algorithm

Algorithm implements and Let N be the numb er of p oints

and F in F and let N and N b e the resulting numb er of p oints in F

X X

Then algorithm splits F into F and F An example calculated with this

X X

algorithm is shown in gures The random variable with the distribution

given in gure was split into p ositive and negative parts The two parts are

shown in gures and This algorithm results in p ositive and negative

parts that havea number of points equal to the number of points used to represent

the p ositive and negativevalues of the original distribution That is if i is the

min

i then N N i andifj is the minimum value of i such that F

min max

maximum value of j such that F j then N j Itisoftenthecasethat

max

N N N b ecause of the dierence b etween p p and

Implementing and using the numerical representation is rather

more complicated Let us rst consider howmanypoints should b e used to represent

the results of the various convolutions Because a smaller numb er of p oints used

to represent an input distribution results in a greater distance b etween the lower

and upp er output distributions see the examples presented in gures and

it makes sense to restrict the numb er of p oints used to represent the output to at

most the minimum of the numb er used to represent the inputs This is henceforth

adopted as a convention It means we do not use an unnecessarily large number of

p oints to represent the output of a convolution calculation

Having done this equations and can b e considered to b e simply

Figure An example distribution which is to b e split into p ositive and negative parts

Figure The p ositive part of the distribution in gure

Figure The negative part of the distribution in gure

f aN g bN t k i j

whilekNf

whilet N f

ifF i G j f

x F i tf

elsef

x G j t g

H k x tN

Algorithm

of the form

H z aF z bGz

Ignoring for the moment the complications intro duced byhaving all the ab ove quan

tities interval valued see equations the general approach used to cal

culate is outlined in algorithm Here it is assumed that H F and G

are all represented by N p oints The idea b ehind the algorithm can b e pictured in

terms of a graph showing b oth F and G Moving along the xaxis from the left to

the right at each step that is encountered b e it in F or G an appropriate amountis

added to H The variable k keeps trackofhowhighH is and the whilet N

lo op accumulates sucientstepsin F and G to corresp ond to a step in H This

algorithm is only approximate b ecause of the implicit directed rounding asso ciated

with this while lo op Algorithm gives the correct directed rounding for the

lower approximation to H

When all the additional complications of a and b b eing interval valued and F

and G being interval valued with a dierentnumber of points used to represent them

are incorp orated the algorithm b ecomes rather more complicated The details are

omitted here as to o much space would b e required The main idea used is that

in in order to obtain the minimum or maximum at any given z itisnot

necessary to consider the minimum or maximum obtained at some previous z This

allows the calculation to pro ceed p ointbypoint

The diculties mentioned in the ab ovetwo paragraphs are relatively straight

forward to solve when compared with those encountered in attempting to calculate

numericallyAt rst sight it is surprising that this is the case as seems

rather simple The diculties arise b ecause of the need to represent F and F by

the same numb er of p oints and b ecause of the p ossibility of a mismatch either

side of the x line while all the time attempting to preserve the representation

error containment prop erty No nal solutions to this are oered here It should

b e remarked however that this op eration of combining the p ositive and negative

parts of each of the lower and upp er distributions need only b e p erformed if either

the combination of random variables b eing p erformed is the last one in an overall

probabilistic arithmetic calculation or if the following op eration requires the whole

distribution The latter p ossibility is the case when the following op eration is either

a subtraction or an addition If a given op eration is to b e followed bya multiplica

tion say then there is no need to combine the p ositive and negative parts as the

overall distribution would only have to b e split up again

Conclusions on the use of Mixtures for Nonmonotonic Op erations

The use of mixtures for splitting random variables into p ositive and negative parts

for calculating dep endency b ounds for nonmonotonic op erations has b een consid

ered and many of the details worked out Because of our desire to maintain the

representation error containment prop erty the algorithms have b een more complex

than they would have b een otherwise Further work is needed to see whether the use

of mixtures can solve all the diculties asso ciated with nonmonotonic op erations

Conclusions

The idea of probabilistic arithmetic whichwould allow one to workwithrandom

variables in the same way that one works with ordinary numb ers was intro duced

in section It was seen that the rst step in developing such an arithmetic was

to examine ways of numerically calculating convolutions of distribution functions

The phenomenon of dep endency error and a suggested manner of handling it de

p endency b ounds led to a numerical representation of distribution functions This

representation was shown to b e also suitable for numerically calculating ordinary

convolutions The main dierence b etween the representation adopted in this chap

ter and other metho ds that have b een suggested is that the present metho d allows

the representation error always to b e b ounded By the representation error contain

ment prop erty of the lower and upp er approximations one always knows that the

true distribution is contained within the lower and upp er b ounds

Other metho ds in comparison not only have unknown errors although the or

der of magnitude may b e known but often the combination rules are rather more

complex than those used in this chapter An example of this is the Hfunction

based metho d describ ed in This metho d is based on the use of rules

expressing the distribution of certain convolutions of random variables whose distri

bution functions are Hfunctions in terms of other Hfunctions Not only is this

metho d restricted in the typ e of distributions it can handle the standard parametric

distributions but its combination rules are very complex and even when an answer

is calculated in terms of Hfunctions a computer program needs to b e used to cal

culate the p ointvalues of the distribution b ecause its expression is so complicated

The metho d prop osed in the presentchapter is b etter than the metho ds discussed

in chapter

Whilst some of the groundwork for a useful probabilistic arithmetic has b een laid

a go o d deal of further work is required As well as the sp ecic p oints mentioned earlier measures of dep endence based on copulas the use of mixtures for non

monotonic op erations and the development of algorithms for implementing other

op erations such as log and exp it will b e necessary to examine metho ds of using

the convolution and dep endency b ound algorithms to calculate the sort of results we

are after solutions of random equations The appropriate p oint of departure for

this is the consideration of algorithms whichhave b een used successfully in interval

arithmetic Since interval arithmetic can b e considered as a very crude form

of probabilistic arithmetic where the only information known ab out a distribution is

the smallest closed interval containing its supp ort it is hop ed that ideas that have

b een useful in interval arithmetic will also b e of use for more general probabilistic

arithmetic

Another asp ect which deserves consideration is the acquisition of lower and upp er

approximations to distribution functions from sample data A natural idea is to con

sider the lower and upp er b ounds as forming a condence band However there are

many problems which arise with this scheme such as whether the condence region

should b e considered p ointwise or overall and how one should combine the con

dence levels of two distributions which are combined in the course of a probabilistic

arithmetic calculation Further investigation is required here Also worth pursuing

is the relationship b etween the ideas examined in this chapter and recent ideas on

uncertaintysuch as fuzzy set theory and generalisations of probability theory eg

DempsterShafer theory This is done in the following chapter

It seems that the phenomenon of dep endency error may turn out to b e the biggest

obstacle to a successful probabilistic arithmetic It may b e that probabilistic arith

metic will b e no b etter in terms of accuracy and computational complexity than

Monte Carlo simulations followed by statistical estimation of the resultant proba

bility distributions This p ossible intractability of all metho ds other than Monte

Carlo simulation has b een considered byseveral authors in the general context of

probabilistic theories of physics esp ecially quantum mechanics In

Feynmann argues that the only way to simulate some probabilistic systems

is to use a probabilistic computer This is b ecause determining all the distribution

functions and then integrating over those we are not interested in is intractable in

many cases

For example supp ose there are variables in the system that describ e the

whole world x x the variables x youre interested in theyre

A B A

around here x are the whole result of the world If you wanttoknow

the probability that something around here is happ ening you would have

to get that byintegrating the total probability of all kinds of p ossibilities

over x Ifwehad computed this probabilitywewould still havetodo

the integration

P x P x x dx

A A A B B

which is a hard job But if wehave imitated the probabilityitsvery

simple to do it you dont havetodoanything to do the integration you

simply disregard what the values of x are you just lo ok at the region

x p A

What Feynmann is saying here is that if one calculated some complex joint dis

tribution analytically then in order to determine not necessarily onedimensional

marginals of this distribution it is necessary to p erform the ab oveintegration If

instead the probabilistic pro cesses have b een simulated imitated one needs only

to lo ok at the quantities of interest it is not necessary to integrate out the other

variables This is true regardless of whether the variables are indep endent or not

The nonlo calityofphysical theories can b e considered to b e analogous to dep en

dency error arising in the course of a probabilistic arithmetic calculation In sucha

calculation one needs to keep track explicitly of all the dep endencies that arise due

to common sub expressions whereas in a Monte Carlo simulation these dep enden

cies take care of themselves The course for an improved probabilistic arithmetic is

thus clear it will b e necessary to develop further metho ds of handling dep endency

error making controlled approximations in order to avoid intractability Recalling

Manes result mentioned in section such a course mayprove useful to

other uncertainty calculi as well

In summary then wehavedevelop ed new metho ds for calculating convolutions

and dep endency b ounds for the distributions of functions of random variables which

are b etter in several resp ects than previously available metho ds Wehave presented

examples which show that the metho ds are feasible and readily implemented We

have suggested that these metho ds might form the basis of a probabilistic arith

metic suitable for calculating the distribution of functions of random variables We

have seen that the biggest obstacle to such an application is the phenomenon of

dep endency error but wehave also shown that the concept of dep endency b ounds

can b e used to reduce the eect of this error The prop erties of these dep endency

b ounds are explored further in chapters and

Chapter

Relationships with Other Ideas

The only essential know ledge pertains to the

interrelatedness of things

Jorge Luis Borges

My subject does not exist because subject matters in

general do not exist Therearenosubject matters no

branches of learning or rather of inquiry thereare

only problems and the urge to solve them

Karl Popp er

Intro duction

The techniques develop ed in chapter haveanumber of interesting connexions with

other ideas some new and others surprisingly old This chapter is devoted to the

study of these connexions In summarywe will discuss the following issues

We shall see that simple cases of the analogue of our dep endency b ounds for

random events were considered by George Bo ole in his Investigation of the

Laws of Thought published some years ago and we shall see how these

results have b een recently extended section

The prosp ects of using graphtheoretic techniques for probabilstic arithmetic

calculations will b e considered and we will review a numb er of ideas related

to this section

Dierent theories of intervalvalued probabilities will b e examined and con

trasted with the probability b ounds arising in chapter In particular we will

discuss the DempsterShafer theory of evidence It will b e shown that none

of the metho ds presented to date apart from those discussed in section

derive from the same motivation as our metho ds section

We shall show that the dep endency b ounds haveanintimate connexion with

the metho ds of calculating with fuzzy numb ers and thus we will provide a

probabilistic interpretation of fuzzy set theoretic op erations section

Finally we will discuss two dierent approaches to probabilistic arithmetic the

approachwehave taken and that taken in the study of probabilistic metric

spaces We will show that the latter is a more p ositivist notion and we will

argue that our approach is more meaningful but that it engenders more severe

mathematical diculties section

This chapter b ecame rather larger than the author exp ected b ecause of the sur

prising number of interconnexions that were found However for ease of reading

and comprehension each of the sections can b e read separately and there is only a

small amount of crossreferencing b etween sections

George Bo oles Results on Frechet Bounds and Some

Recent Extensions and Applications

Boole means something that no one has understood yet the

world is not ready to understand him

Stanley Jevons

Booles treatment of probability baed his contemporaries

and I do not pretend to understand him ful ly

Glenn Shafer

Never clearly understood and considered anyhow to be

wrong Booles ideas on probability were simply bypassedby

the history of the subject which developed along other lines

Theo dore Hailp erin

The dep endency b ounds we examined in chapter can b e traced backtoFrechet

Salvemini Gini see and eventually to George Bo ole

Bo ole considered problems where no dep endency information is known in his b o ok

An Investigation of the Laws of Thought on which are founded the Mathematical

Theories of Logic and Probabilities In this section we will examine Bo oles

results on these problems and we will describ e some recentwork which generalises

his results Some of these new results suggest directions for further researchonour

dep endency b ounds for functions of random variables All of these results are

for probabilities of events and not for random variables We shall see that Bo oles

work on probability is not quite as imp enetrable or useless as some authors have

suggested In fact we shall nd that it contains the groundwork for a p otentially

powerful technique of use in Articial Intelligence systems that have to manage

uncertain knowledge and inferences

We are considerably aided in our study of Bo oles work by Hailp erins admirable

book Bo oles Logic and Probability Hailp erin has presented Bo oles results

in mo dern notation and has b een foremost in extending his results using the mo dern

techniques of linear programming His work is discussed in detail b elow

Bo oles Ma jor and Minor Limits of Probabilities

Bo ole was unhappy with the need to makemany assumptions in the normal appli

cation of the calculus of probability As he puts it in a letter to De Morgan dated th August

The grand diculty in the common theory is to knowwhathyp otheses

you maylawfully make and what you cannot p

He was particularly concerned with problems where it was necessary to assume in

dep endence of the constituent probabilities in order to arrive at a denite answer

Interestingly and confusingly Bo ole is not consistent in his approach to suchdif

culties For example on page of he sa ys

When the probabilities of events are given but all information resp ecting

their dep endence withheld the mind regards them as indep endent

Hailp erin padds that in the face of a sp ecic problem with material

content Bo ole app ears to backdown from his p osition We shall see b elowthatit

is quite the opp osite view which seems appropriate for interpreting Bo oles minor

and ma jor limits

The ab ove quoted p osition of Bo ole came under criticism by Wilbraham

subsequently replied to by Bo ole and more recently byJaynes Wilbra

hams critcism and Bo oles reply are discussed in detail by Hailp erin pp

Jaynes criticism is by far the most p owerful and as seems invariably the case

in arguments ab out the foundations of statistics not a little acrimonious He is quite

contemptuous of Bo ole who he takes to task for illfounded criticism of Laplaces use

of prior distributions in Bayesian typ e inference Bo ole was actually inuenced

quite a lot by Laplace MacHale psa ys Laplace played a signicantpart

in Bo oles Development Jaynes notes with ironythat

Bo ole after criticizing Laplaces prior distribution based on the principle

of indierence then invokes that principle to defend his own metho ds

against the criticisms of Wilbraham

He go es on to saythat

Bo oleso wn work on probability theory contains ludicruous errors far

worse than any committed by Laplace While Laplace considered real

problems and got scientically useful answers Bo ole invented articial

scho olro om typ e problems and often gave absurd answers pp

Bo ole made a numb er of mistakes confusing probabilities of conditionals with con

ditional probabilities His work on minor and ma jor limits is essentially sound in

principle though if not in all its details Nevertheless it seems to have b een largely

ignored up until very recently

Surprisinglyhowever Bo ole has b een given very little credit for his contri

butions in this area probability theoryb y presentday probabilitythe

orists and historians of mathematics many textb o oks or even history

b o oks on the sub ject do not even mention his name p

Bo ole rst examined the typ e of problems we are concerned with in a pap er

he wrote around but whichwas not published until after his death when

it was communicated to the Transactions of the Cambridge Philosophical So ciety

by his friend Augustus De Morgan Bo ole was led to examine b ounds on the

probabilities of certain comp ound events for which his general metho d for solving

probability questions gave indeterminate answers Bo ole describ es these cases by

saying that the data given are insucient to render determinantthevalue sought

and thus the nal expression will contain terms with arbitrary constant co ecients

p Under these circumstances one can either obtain new data if p ossible

to render the numerical solution complete or by giving to their constants their

limiting values and determine the limits within which the probability sought must

lie independently of al l experience pemphasis added

Bo ole obtained ma jor and minor limits for the probabilities of certain combi

nations of the events in terms of the unconditional event probabilities On page

of he giv es the following b ounds for the probability of a conjunction

maxP xP y P x y minP xPy

although he do es not use this notation see also p Bo ole also obtained

analogous but rather more complex results for more complicated logical expressions

The ab ove b ounds are a sp ecial case of the more general results considered byFrechet

whic h are given by

Theorem Let x x be events with absolute probabilities of occurenceof

p p respectively Then the bounds

maxp p p P x x x minp p p

n n n

and

maxp p p n P x x x minp p p

n n n

are the best possible given no further information

Proof We will simply prove that the b ounds for n hold The case for

general n follows by induction The b est p ossible nature of the b ounds is proved by

constructing a set of events for anygiven set of fp gsuch that the b ounds are met

The b ounds and follow from the standard rules of probability

theory For events A and B these state that

P A

P A if A

P A P A

P A B P AP B P A B

P A B P AjB P B

First weprove the b ounds for conjunction Wehave P A B P AP B

P A B P A B P AP B P A B But P A B and

so P A B P AP B Wealsohave P A B and therefore

P A B maxPAP B The upp er b ound for P A B follows

from the fact that P A B P AjB P B P B jAP A Since P AjB

and P B jA wehave P A B P B and P A B P A Thus

P A B minP APB

The b ounds for disjunction are derived similarly P A B P AP B

P A B andP A B imply P A B P AP B But P A B and

so P A B minPAP B Wealsohave P A B P A B

P A B minP APB maxP APB

It is instructive to examine Bo oles interpretation of his results on b ounds for the

probabilities of comp ound events One might exp ect given the somewhat strange

motivations for his logic that his interpretation of probabilitywould b e p ecu

liar Bo ole has often b een accused of psychologism in his interpretation of logic

Richards following the reading of Bo ole byVan Evra has argued that

whilst motivated by psychological considerations originallyBoolewas most cer

tainly not a defender of logical psychologism p As already noted Bo ole

was not consistent in his interpretations Perhaps the b est we can do is to quote

the following passage from the the intro ductory chapter to The Laws of Thought

It will b e manifest that the ulterior value of the theory of Probabilities

must dep end very much up on the correct formation of such mediate hy

p othesis where the purely exp erimental data are insucientfor denite

solution and where that further exp erience indicated by the interpreta

tion of the nal logical equation is unattainable Up on the other hand

an undue readiness to form hyp otheses in sub jects which from their very

nature are placed b eyond human ken must react up on the credit of the

theory of Probabilities and tend to throw doubt in the general mind over

its most legitimate conclusions p

Hailp erins Reformulation and Extension of Bo oles Bounds Using

Linear Programming Metho ds

Hailp erin has considerably extended Bo oles results using the mo dern

theory of linear programming His b ounds are for general Bo olean functions of a set

His results do not seem very well known and so we state them in of events fA g

the following two theorems

Theorem Given any Boolean expression A A

Thereare numericalvalued nary functions L and U depending only on the

Boolean structureof such that the inequalities

L a a P A A U a a

n n n

hold in any probability algebra for which P A a i n

i i

Psychologism is the p osition that psychology is the most fundamental branch of science and that all

other discipli nes are sp ecial branches of psychology pp

The bounds in are the best possible and

The functions L and U areeectively determinable by solving a linear pro

gramming problem from the structureof

Hailp erin extends this result to the case where only b ounds are known in place of

the a P A He gives

i i

Theorem Given Boolean polynomial expressions in the vari

ables A A

Therearetwo mary numerical functions L and U depending only on the

structures of such that the inequalities

L a a b b P A A

m m n

U a a b b

m m

hold in any probability algebra for which

a P A A b i m

i i n i

The bounds in are the best possible and

The functions L and U are eectively determinable by solving a linear

programming problem from the structures of

A particularly useful sp ecial case of this theorem o ccurs when A A A

i n i

for i n mInsuch a case theorem says that given b ounds of the

form

a P A b i n

i i i

it is p ossible to determine b ounds on P A A in terms of the intervals

a b P A i n Theorem only has contentwhenf g satises

i i i i

some consistency constraints For anygiven sets f g fa g and fb g the consistency

i i i

is eectively decidable

The pro ofs of these two theorems are essentially contained in We omit

all details of the actual determination of the b ounds in terms of the formulation as

a linear programming problem However we do remark that it was only through

clever use of duality theorems that Hailp erin obtained his results Ursic has

considered the same problems as Hailp erin and used similar techniques linear alge

bra and linear programming in an attempt to solve them He studies the problem

of calculating b ounds which although not the b estp ossible are easier to compute

He describ es how one can trade o tightness of the b ounds against computational

tractability Ursics results are far to o intricate to summarise briey here Further

work is needed to determine the practical value of his results

Adams and Levine ha ve also considered problems similar to those studied by

Hailp erin in his development of probability logic see b elow They to o with Ursic

seem to b e unaware of Hailp erins work in this area Their results are essentially a

subset of Hailp erins and are not discussed further However their interpretation of

the results is interesting They use their results to determine the degree to whicha

given uncertain inference is deductive This allows an ob jective statement of the va

lidity of an inference in terms of uncertain prop ositions As Genesereth and Nilsson

p oin t out this could b e of considerable value in Articial Intelligence systems

dealing with uncertain inference A numb er of other authors such as Grosof

Wise and Henrion Ruspini et al and Driank ov

have also considered the use of lower and upp er b ounds on probabilities along

the lines of those discussed ab ove We will examine their work in the following

subsection

Application of the Frechet Typ e Bounds to Probability Logic

Hailp erin uses the results in theorem to develop a probability logic which allows

inferences in terms of the lower and upp er b ounds for prop ositions For example

generalising material implication If P Ap P A B q then his results

imply pB p q qunder the consistency condition p q Hailp erins

results provide a completely general and rigorous framework for uncertain inference

Not only is no new measure of uncertainty required such as fuzzy sets or b elief

functions but no interpretation problems are caused by the use of these results

The notion of probability is presupp osed as part of the semantics and

questions as to its nature will play no more role than do es the nature of

truth in usual logic p

Hailp erins notion of probability logic seems to b e the thing Popp er had in mind

when he discussed probability logic p

There exists a logical interpretation of the probability calculus which

makes logical derivability a sp ecial case of a relation expressible in terms

of the calculus of probabilityThus I assert the existence of a probability

logic which is a genuine generalization of deductive logic

The idea of a probabilistic logic can b e traced back to Leibniz see p

Since then several authors have examined probabilistic generalisations of logic Let

us just mention Scott and Krauss Supp es Renyi Lewis and

Calabrese

Implications for Dep endency Bounds for Random Variables

Hailp erin presented a numb er of examples of lower and upp er b ounds on the prob

ability of Bo olean functions In many cases these b ounds were obtained using his

Popp er was discussing this in order to refute the idea of an inductive probabili ty logic that allowed

probabilis tic inductive inferences

fullblown linear algebra technique p However the b ounds can often b e

calculated in a simpler manner For instance the results of the example mentioned

ab ove material implication follow easily from application of the b ounds for and

the fact that P A P A given that A B B A This is not always

the case As an example Hailp erin considers the determination of an upp er b ound

for

P A A A A A A A A A

given P A a for i The b est upp er b ound is obtained using the general

i i

linear programming technique as

minfa a a a a a a a a a a a g

Hailp erin pobserv es that this result can not b e obtained by comp osition

of the simple results for and

The reason why the comp osition techniques fail for is quite simple but

imp ortant Equation can not b e rewritten in a form with no rep eated vari

ables This is why the probability b ounds can not b e determined pairwise Observe

that this is very reminsiscent of Manes result men tioned in section The

rep eated variables cause the pairwise b ounds to b e not necessarily the b est p ossible

b ecause not all p ossible combinations of probabilities will b e allowable there will

b e certain constraints which will prevent this The fact that the pairwise b ounds

are quite lo ose when rep eated variables o ccur was the reason for Clearys dissatisfac

tion with the use of and p Even when there are not rep eated

variables the b ounds can rapidly b ecome quite lo ose see the example in p

This should not b e taken as an argument against the probability b ounds technique

though What it is do es show is the danger even in simple problems of assuming

indep endence in order to obtain a unique value at the end

This result has implications for dep endency b ounds for functions of random

variables Obviously the pairwise comp osition of b ounds will give the b est p ossible

b ounds when there are no rep eated variables This will not necessarily b e the

case when rep eated variables o ccur Whether or not in a particular instance the

pairwise b ounds are b est p ossible is an op en problem In such cases the prosp ects

for determining the b est p ossible b ounds are quite daunting Whereas Hailp erins

problem is one of nding functions over the determination of dep endency b ounds

for random variables entails nding functions over This will b e in general very

dicult and it would seem that the eort would rarely b e justied One mayas

well calculate the actual distribution of the function of random variables in question

byintegrating out the Jacobian of transformation It was due to the complexityof

this that we adopted the use of the pairwise dep endency b ounds in the rst place

Rep eated variables do not imply that the pairwise b ounds will not b e the b est p ossible the o ccurence

of rep eated variables is a necessary but not sucient condition for this to b e the case

However see our discussion of Hanevelds results on sto chastic programming problems b elow

Bo oleFrechet Bounds in UncertaintyinArticialIntelligence

The use of the Bo oleFrechet b ounds for uncertain deduction has b een prop osed

bya number of workers in a eld which has come to b e known as Uncertaintyin

Articial Intelligence Two collections of pap ers have b een recently published

in the eld

Quinlan w as p erhaps the rst in the Articial Intelligence communityto

discuss the use of in his Cautious Approach to Uncertain Inference

It has most recently b een considered by Grosof who has dev elop ed Nilssons

ideas further Nilsson p has rederived Hailp erins result on un

certain modus ponens Hewas aware like Hailp erin of the need for consistency

conditions Nilsson says

In principle the probabilistic entailment problem can b e solved by linear

programming metho ds but the size of problems encountered in proba

bilistic reasoning is usually much to o large to p ermit a direct solution

Nilsson was unaware of Hailp erins use of duality results which reduce the com

plexity considerably Grosof who only considers the calculation of probability

b ounds for disjunction and conjunction makes the following p oints ab out the use

of in a probability b ounds logic

The use of probability b ounds gives a closed and consistent uncertain inference

system

Underdetermined information can b e represented

The direct use of lower and upp er probabilities gives a more general system

than the DempsterShafer theory see section b elow

No unjustiable indep endence assumptions are required

Lower and upp er probabilities can b e empirically acquired via standard con

dence interval pro cedures

The combination rules of fuzzy set theory are a sp ecial case of the Bo oleFrechet

b ounds see section b elow

App elbaum and Ruspini and Mon togomery ha ve also prop osed the

use of Bo oleFrechet b ounds They to o psa y that the use of linear pro

gramming in order to nd the b est p ossible b ounds is impractical They argue

that the probability b ounds approach is preferable to the use of the principle of

insucient reason assumption of indep endence given no information to the con

trary b ecause the probability b ounds approach is rigorous the result will always

b e correct Following MartinClouaire and Prade App elbaum et al p

pha ve considered more general lower and upp er b ounds on the probability

of conjunction and disjunction based on tnorms and tconorms Neither Martin

Clouaire and Prade nor App elbaum et al provide a go o d justication for these

Their discussion is in terms of valuations v and they set

v a v a

a b T v avb v

v a b T v a v b

v a b T v avb

v a b T v a v b

App elbaum et al used T and M in their exp eriments with these formulae and

found that the upper b ounds were of little practical value A more realistic proba

bilistic approachwould b e in analogy with the dep endency b ounds for C W

along the lines

v avb v a b v a b T v a v b T

and a similar relationship for v a b in terms of T The lower and upp er b ounds

on T whensettoW and M would give the standard Bo oleFrechet b ounds If

or T M then tighter b ounds would b e obtained Other authors such W T

as Bonissone and Go o dman ha ve also considered the use of general t

norms for the combination of uncertain evidence Their metho ds are more sub jective

and nonprobabilistic and are of little value to us

Wise and Henrion ha ve considered the use of the Bo oleFrechet

inequalities in a manner similar to that describ ed in the previous paragraph In

they consider three separate rules for conjunction and disjunction

Maximum Correlation

pA B minpApB

pA B maxpApB

Indep endence

pA B pApB

pA B pApB pApB

Minimum Correlation

pA B maxpApB

pA B minpApB

Wewould suggest they b e used as in thus allowing one to write if one knew

events A and B were never negatively dep endent

pApB pA B minpApB

pApB pApB pA B maxpApB

Wise and Henrion palso consider probabilistic modus ponens in the

same manner as Hailp erin They also note that the Maximum Correlation rules

are those of fuzzy set theory see section b elow Henrion pmak e

the p oint with whichwe agree that the representability of some notion of dep en

dence is a more imp ortant feature of uncertainty calculi than the ne details of

representation of single events or prop ositions He also makes a numb er of other

p oints whichwe will b e discussing elsewhere use of graph mo dels and lo cal event

groups section and MonteCarlo metho ds or logic sampling Co op er

has also used the idea of probability b ounds and has combined this with the use of

a graph to represent conditional dep endence information He is aware of the linear

programming formulation of the problem but is unaware of Hailp erins work on

using duality results to simplify this

Grosof and man y other authors eg Hailp erin and Shafer studying lower

and upp er probabilities have made use of the relationship

pA pA

which comes from the fact that pApA Grosof suggests that

allows one to work with lower probabilities only rather than lower and upp er

if one knows the lower probabilityofevery event then one automatically has an

upp er probabilityforevery event Wemention this b ecause it is in fact of no use

when random variables are considered To see this consider

F xP fXxg P A

Thus AX x Assume weknow F x the lower b ound on the dis

tribution function of X Now P A P X x F x However

F xS x the survival function of X Thus we can use lower probabilities

X X

only if we carry around the lower distribution function and the lower survival func

tion This is equivalent to carrying b oth the lower and upp er distribution function

and so no advantage is gained

Further Results on Distributions known only by their Marginals

Wewillnow review some further recentwork on problems where only marginal

distributions are known We will briey lo ok at the problem of compatibilityof

joint distributions and then consider some problems arising in pro ject planning

Perhaps the most surprising feature of some of these problems is that the more

general and more useful problem where only the marginal distributions are known

is easier to solve than the problem where indep endence is assumed

Compatibility of Marginal Distributions and the Existence of Joint Distribu

tions

One problem which has attracted a lot of attention is that of the compatibility of marginal distribution functions esp ecially higher dimensional marginals This

is related to the general problem of the existence of multivariate distributions with

given marginals Let us just mention some of the literature on the topic

Schweizer and Sklar discuss the compat

ibility problem in terms of copulas in The extension of bivariate results to

higher dimensions has proved surprisingly dicult For example the analogue of

the copula W is in fact the b est lower b ound on multidimensional distribution

functions with uniform marginals but it is not itself a copula A go o d but some

what dated review of the compatibility problem is given by DallAglio This

problem do es not concern us directly and is not discussed any further It would

b e of considerable imp ortance however if tighter b ounds on comp ound expressions

involving more than twovariables were to b e studied further

Pro ject Planning and Other Network Problems

Problems formulated in terms of sets of random variables known only by their

marginals arise naturally in network problems Let I fng b e a set of

no des and let I I b e subsets of I such that I I andsonotwo I are

k j j

ordered by inclusion fI g is known as a clutter over I The blocking clutter to

fI g is a clutter J J suchthat I J rs and J are minimal sets with

j l r s j

this prop ertyGiven a DAG Directed Acyclic Graph I is known as a system and

k l

fI g and fJ g are the paths and cuts of the system Eachnodei has a weight

j j

X asso ciated with it

The problems we will discuss are various optimal value functions of fX g These

include

X over the clutter of paths PERT Critical Path M max

j k

Maximum Flow L min X over the clutter of cuts

j l

Shortest Route S min X over the clutter of paths

j k

Reliability System Lifetime T maxmin X minmax X

j i

iI iJ

j k ij l

j j

Two sp ecial cases are the pureparal lel k n I fj g j ninwhich case

M S T max X and L X and the pure series k inwhich

iI i i

case M S X and L T min X

i iI i

Lai and Robbins ha ve considered the determination of EM where M

n n

maxX X They were actually interested in determining EM when all the

n n

X are independent Whilst it is known that in this case

EM n xF xdF X m

where F is the common marginal distribution of X isingeneralvery

dicult to evaluate In the maximal ly dependent case it can b e shown that

EM a n F xdx m

n n n

a n

where a inffxj F x g Lai and Robbins study how close m and m are

n n

They turn out to b e surprisingly close We note that their constructions would b e

a lot simpler if the concept of a copula was used

More interesting to us with regard to probabilistic arithmetic is the evaluation

of the distribution of M in the pure series case Ruschendorf has used a

general duality result obtained in and discussed in in order to obtain

lower and upp er b ounds on P fM xgRuschendorf s results in are similar

to our dep endency b ounds for sums In describing these b ounds he simply writes

sup F x Gt x and inf F x Gt x obviously implicitly including

the max and min op erations required to keep the result within His results were

published contemp oraneously with Makarovs the precursor to F rank Nelsen

and Schweizers pap er Rusc hendorf obtains one explicit result for b ounds on

X when df X U the uniform distribution on These are a df

i i

sp ecial case of Alsinas results

U U U

W ab cd

acmaxadbc

and

U U U

W ab cd minadbcbd

Ruschendorf also gives some results on

X tg t sup P f

i n

F HF F

where F df X fori n and F is the joint distribution function If G

i i

is a subadditive strictly isotone function with G F x x for all x

then t x x Gt If G is a sup eradditive isotone function with G F

for all x then t Gt Ruschendorf s results should b e compared

to those in chapter of this thesis where we examine the dep endency b ounds for

X as N Ruschendorf did not use copulas and seems unaware of df

the workofSchweizer et al in this area Nevertheless he has found the viewp oint

of copulas or uniform representations useful recently

Klein Hanevelds use of Duality Results

Klein Haneveld made use of sev eral duality results and rearrangement tech

niques in order to determine distributions of pro ject completion times in

PERT networks He assumes the marginal distributions of the subpro ject comple

tion times are known but not the joint distribution He calculates the worst case

results over all p ossible joint distributions His pap er contains a large number of

detailed results and we can not do it justice in the space available here

We note that Klein Hanevelds inner problem was of the following form De

termine ht where

ht sup E f t

H n

H HF F

f xf x M x t x

t n

M x max x

j k

x the pro ject completion time and E denotes the exp ectation over H

This calculates the cost of a promised completion time of t If indep endence of

the random variables is assumed it is p ossible to obtain b ounds for the exp ected

value of M x or its distribution function The problems dened

by have also b een studied byCambanis et al They have

determined b ounds for E k X Y where H HF F and k is quasimonotone

H X Y

ie increasing like a copula Whilst these problems are somewhat dierentto

the typ e of problems wehave b een concerned with they are of interest b ecause the

same basic approach has b een taken determining lower and upp er b ounds assuming

no dep endency information

One of Klein Hanevelds most surprising results is his analysis of worstcase

marginals Assume we only knowvery limited information ab out the marginal

distributions Then what are the worst p ossible marginals in terms of pro ject

completion time satisfying the known information Klein Haneveld solves this

problem for the situation where the supp ort or the supp ort and mean or mo de

of the marginals are known He notes that the case of knowing the supp ort and

mo de is the most interesting since it contains precisely the information whichis

usually supp osed to b e known in PERTnetworks p Using results of

Zackova and Dupa cova Klein Haneveld manages to solve this problem

in an elegant manner We will briey return to a discussion of this in section

b elow where we consider fuzzy PERTnetworks Duality in marginal problems was

also the sub ject of Kellerers dicult pap er

AForest of GraphTheoretical Ideas

Structure between variables has receivedtoo little attention

in statistics

SL Lauritzen and DJ Spiegelhalter

Probability is not real ly about numbers it is about the

structureofreasoning

Glenn Shafer

The use of graphs for represen ting structural relationships conditional in

dep endence b etween random quantities is a p owerful technique with apparently

considerable p otential for probabilistic reasoning and probabilistic arithmetic In

this present section we will review the applications of graph theory to sto chastic

problems and showhow it can b e used to understand problems arising in proba

bilistic arithmetic Our aim is to explore the p ossibility of using graphtheoretical

techniques to aid probabilistic arithmetic Wehave already seen section the

study of probabilistic PERT whichisaprobleminvolving random variables on a

seriesparallel graph

In subsection we will see how graphs have b een used in studying networks

of events Most of this work has app eared in the articial intelligence literature

Subsection examines analogous ideas for networks of random variables These

metho ds have app eared mainly in the statistical biological and so ciological liter

ature They are used in studying causal models These metho ds can in fact b e

considered as a rather weak and restricted form of probabilistic arithmetic A graph

representation aids the understanding of the control layer for probabilistic arithmetic

calculations In subsection we will consider how techniques used by computer

scientists in the construction of optimizing compilers might b e protably employed

in probabilistic arithmetic This is a quite dierent use of graphtheoretical tech

niques We shall see that whilst certainly providing a clear understanding of the

problems unfortunately these metho ds do not manage to pro duce solutions for the

situations we are concerned with Finally in subsection we will draw some

conclusions on the use of graph theoretical ideas for probabilistic arithmetic

The one theme that is rep eated through this section is the desirabilityofhaving a

graphical structure which is a tree Wehave already mentioned section Manes

general result whic h shows this eect for a variety of uncertainty calculi

Networks of Probabilities of Events Bayesian Networks Belief

Networks and Inuence Diagrams

Bayesian networks or causal networks or b elief networks are a way of representing a

set of interdep endent probabilities by use of directed graphs Pearl

has b een foremost in dev eloping these metho ds see also the recent review by

Co op er The basic idea is to use a directed graph to represent the conditional

indep endence prop erties of the probabilities of a set of ev ents This is b est

explained by an example

Given a joint probability P x x of n events we can always write

P x x P x jx x P x jx x P x jx P x

n n n

This expansion can sometimes b e simplied For example wemay b e able to write

P x x P x jx P x jx x P x jx x P x jx P x jx P x

Equation can b e expressed graphically as in gure which describ es some

conditional independence relations For example x is independent of x given x

But x may not b e indep endentof x given x and x because x dep ends on x which

in turn dep ends on x and x It is p ossible pto determine conditional

indep endence relations solely in terms of the graph mo del

The main advantage of this graphical representation is computational it allows

advantage to b e taken of conditional indep endence in order to reduce the com

putational complexity Whilst theoretically this is of course p ossible without the

graphical representation it is much harder to do so The graphical representation is

easily understo o d in an intuitive manner Pearl has shown ho w given a set

The review of Pearls work b elowwas written b efore the author could read This b o ok provides

a detailed examination of the material summarised b elow plus a lot of other ideas It is now the preferred reference for this material x1

x2 x3

x5 x 4

Figure Directed graph representation of the expansion

of probabilities with a tree structure which is unknown it is p ossible to determine

the actual tree structure solely from the pairwise dep endencies of the leaves This

can b e done in O n log n time where n is the number of leaves Whilst of interest

this is not of enormous practical use b ecause the metho d is extremely sensitiveto

errors in the estimates of the correlation co ecients Furthermore it do es not solve

the still op en problem of determining go o d tree structured approximations to sets

of probabilities that are not in fact tree structured

Given a network which is not a tree there are several approaches whichcanbe

taken Brute force metho ds can always in principle b e used to propagate proba

bilities but these are intractable for realistic problems Instead one can attempt

to transform the graph into a tree Pearl discusses t woways of doing this

clustering and conditioning

Conditioning reasoning by assumptions entails instantiating a no de setting it

to a xed value and then computing the probabilities Onethensetsthenode

to a dierentvalue and recalculates The largest probability is then chosen The

eectiveness of this metho d dep ends to a large extent on the top ological prop erties

of the network It is necessary to instantiate an entire cycle cutset to make the

network singly connected This results in a complexity exp onential in the size of the

cyclic cutsets p More promising is the clustering approach This entails

forming comp ound events lo cal event groups such that the resulting network of

clustered events is a tree As noticed by Barlow p this is similar to Chat

terjees mo dularisation of fault trees It is also reminiscentofDowns Co ok

In the same volume as Chatterjees pap er there are several other pap ers the results of whichmaybe

protably applied to b elief networks These include Mazumdar on imp ortance sampling this would

increase the eciency of sto chastic simulations of b elief networks Rosenthal on the NPcompleteness

of nding the reliabil ity of the top event when the fault tree is not in fact a tree common cause events

this is an analogue of the NPcompleteness results known for general b elief networks and Lamb ert

on measures of imp ortance in fault trees and cutsets this might b e useful in deciding which no des

in a b elief network could b e discarded in order to reduce computational complexity without aecting the

and Rogers partitioning approach for the statistical analysis of interconnected sys

tems One metho d of clustering Pearl has studied in some detail is based

on the formation of cliquetrees using an algorithm of Tarjan and Yannakakis

These transformation metho ds are in fact relevant to a wide variety of problems on

graphs we return to this shortly b elow

A metho d of handling graphs which are not trees whichdoesnotinvolve trans

forming the graph is to use sto chastic simulation This uses random samples

Monte Carlo metho ds in order to estimate propagated probabilities The metho d

has b een suggested by several authors Pearl giv es a brief review of early

attempts Pearl improves the eciency of the simple application of the metho d by

taking some account of the structure of the graph involved Recall that wehave

already discussed section the use of Monte Carlo simulations to p erform prob

abilistic arithmetic typ e calculations

Lo ops in Networks in Constraint Satisfaction Problems

As well as o ccurring in probabilistic problems the presence of lo ops in a network

nontree structure causes diculties in any problem where globally dened solu

tions are pro duced by lo cal computations b e it probabilistic deterministic logical

numerical or hybrids thereof p Examples of such problems include the

class of Constraint Satisfaction Problems CSP A numb er of prob

lems arising in articial intelligence can b e formulated in this manner Constraint

Satisfaction problems are easy if they are backtrackfree p This is equiv

alent to their graph b eing a tree In P earl shows how the propagation of

probabilities in a b elief network can b e considered as a CSP In he describ es

an algorithm for rearranging a graph in order to make it easier to handle The algo

rithm uses the ideas of triangulating a graph identication of maximal cliques and

the notion of a dual graph Dierent CSPs soluble by these metho ds are describ ed

Constraint propagation formulations of problems similar to those wehavebeen

studying in this thesis were examined byDavis The typ e of problems he

considers are explained by the following example Consider three no des with the

labels X Y and Z com bined with the constraints X Y Z

and Y X Davis shows how application of the Waltz algorithm tigh tens the

lab els to b ecome X Y and Z This idea has b een applied

to the analysis of electrical circuits and geometrical reasoning Davis

encountered exactly the same sort of problems as wehave b een discussing in this

section When the graph representing the structure of the problem is not a tree then

the standard metho ds give p o or results p and Nonlinear constraints

also cause severe diculties In fact Davis concludes rather negatively on the whole

enterprise Wehave not found any arguments that the partial results computed by

nal results greatly

Constraint propagation can b e considered to b e simply a programming style see Constraint

programming languages are declarative and nonpro cedural and seem to b e a natural and simple wayof

formulating many problems arising in articial intelligence

lab el inference should b e adequate for the purp oses of AI p

Inuence Diagrams

Inuence diagrams are similar to Pearls b elief networks with the addition of de

cision information Inuence diagrams were develop ed some time ago as an aid

to automated decision analysis and ha ve b een promoted recently by Shachter

and others The v alue of inuence diagrams is that they allow a graphi

cal means for manipulating and exploiting conditional indep endence structure For

example whilst the fact that

P x y z P xP y jxP z jy

is equivalentto

P x y z P y P xjy P z jy

can b e readily determined using the rules of the probability calculus similar trans

formations on larger sets of variables are rather more dicult The rules for inuence

diagram transformations are simply a way of p erforming such transformations in a

manner that is easier to use

Apart from b eing a suitable basis for the construction of exp ert systems which

deal with uncertain information and representing semantic mo dularity

for plausible reasoning inuence diagrams have b een used for analysis purp oses

Barlow has suggested them as an alternativ e to the decision tree for

Bayesian decision analysis He describ es an application of this idea calibration of

a measuring instrument in

The mathematical basis of inuence diagrams or b elief networks is presented most

clearly by Smith in where all the relev ant theorems are rigorously proved A

go o d recent and general review which do es not go into to o many details is

Another recentandvery detailed pap er with considerable discussion at the end

is Lauritzen and Spiegelhalters At the end of their pap er they suggest the

following extensions as suitable goals for further research

The incorp oration of imprecision in probabilites lower and upp er b ounds

These b ounds could then b e propagated through the system in order to provide

lower and upp er b ounds for the nal result

Consideration of the metalevel of control The p oint is that global propa

gation may b e unnecessary b ecause of high level restructuring

Extension to no des representing continuous measurements

Item ab oveisobviously reminiscent of our probability b ounds Item corresp onds

to Bonissones con trol layer Item was solved to an extentby the recent

pap er whic hintegrates b elief networks with covariance structure mo dels These

are graphical mo dels which describ e interrelationships b etween continuous random

variables They are the topic of the next subsection Another p ossible extension

suggested by Smith pp is the depiction of weak relationships

ie relationships which are almost conditionally indep endent This is related to

the use of correlation co ecients in path mo dels see b elow and our preliminary

attempts at just this problem in section

Path Analysis and Covariance Structure Mo delling

Path analysis is a graphical technique for statistical mo delling which Moran

phas suggested should b e regarded solely as a shorthand aid to the correct use

of the regression equations such as and

E X jW V U x W

i i i j j

where the x are constants and

E X X jW E X jW E X jW

i i i

X X

x W x W

k k j j

x x W W

j k j k

Nevertheless it is a very useful aid and one that is continuing to nd wide application

in the biological and so cial sciences

The technique involves making the following assumptions

All the variables in the system under consideration have a joint probability

distribution with nite rst and second moments Normality is not assumed

though

The conditional exp ectations are linear functions of the other variables

The variates are related by a conditional indep endence structure Thus when

conditioned on the appropriate variates the conditional distributions are in

dep endent of all the other variables whichwehave not conditioned on

Path analysis can b e explained simply in terms of a covariance algebra

chapter Writing C X Y for the covariance b etween X and Y the rules of this

are that

C X k

C kX Y kC X Y

C X X V X the variance

C X Y Z C X Y C X Z a XX13 c 12 e b X d

2 X 4

Figure Path diagram of the linear mo del describ ed by and

If all the variates are standardised zero mean unit variance bychange of units

then we can represent linear mo dels as in the following example

X aX bX

and

X cX dX eX

This can b e represented as in gure The quantities a b c d e are known as

path coecients and is the correlation b etween X and X which equals the

covariance b ecause of the standardisation The two main to ols of path analysis are

the rst law and the tracing rule

The rst lawsays

YZ YX X Z

i i

where p is the path co ecient or causal parameter from X to Y is the

YX i X Z

i i

correlation b etween X and Z and fX g is the set of all variables that are causes

i i

of Y Thislaw follows from rules and of the covariance algebra

The tracing rule which only applies to hierarchical mo dels without lo ops is as

follows The correlation b etween X and X equals the sum of the pro duct of all the

i j

paths obtained from each of the p ossible tracings b eween i and j The set of tracings

includes all the p ossible routes from X to X given that a the same variable is not

i j

entered twice and b a variable is not b oth entered through an arrowhead and exited

through an arrowhead For the ab ove example this rule says that a b The

whole p oint of path analysis in practice is to help determine the causal parameters

the ps although this is sometimes imp ossible pp

Path analysis is not intended to b e a blackbox pro cedure for accomplishing

the imp ossible task of deducing causal relations from the values of the correlation

co ecients pbut see our men tion of the recently develop ed tetrad

program b elow The theory was originally develop ed by the geneticist Sewall Wright

and has since found wide application in the so cial and

biological sciences

The advantages of using standardised variables and co ecients are explained in detail byWrightin

There are of course many asp ects wehave not mentioned in this very concise

overview of path analysis and covariance structure mo delling These include the dif

ferenttyp es of mo dels recursive versus nonrecursive diculties encountered

when the numb er of equations is far greater than the number of coecients to b e

determined p the problem of estimating the parameters of the mo dels

and the problem of interpreting the mo dels Further details

can b e found in the b o oks on the sub ject which include A go o d concise

review is given in A more recent review is

From our p oint of view investigating probabilistic arithmetic the most serious

problem is the diculty in handling pro duct terms XY Whilst the relationships

in standard path analysis do not havetobeentirely linear they have to b e linear

over the range of values considered p which of course amounts to the

same thing The topic of interaction as it is called has received scant attention

p The term interaction is used b ecause the situation considered is

Y a bZ X instead of Y aX Here Z is another random variable In his

original presentation of path diagrams pW right recognized multiplying

factors as amongstthe most imp ortan t He suggests that if the co ecients of

variation are small then the approximate formula

X X

X Z X Z

XZ Z X

could b e used see section

Pro duct terms have b een considered in more detail in and ratio terms

are examined in Glass has used P earsons formula for spurious

correlation see section and in order to study eects of pro duct terms in

path mo dels Cohen has generalised Glasss results and pro vides a discussion

of the interpretation of pro duct terms as interactions He also considers p owers

of variables and comp ound pro duct and p olynomial terms Nevertheless all these

eorts will only work well for small co ecients of variation b ecause the formulae

used are all based on the linearisation of a nonlinear function byaTaylor series

expansion In general nonlinear functions are handled quite p o orly Whether the

more sophisticated approach outlined in section of will prove b etter is a matter

for further research

Finally let us just mention that graphtheoretical mo dels have received increased

attention in recentyears with the development of sophisticated computer programs

suchaslisrel and tetrad whichallow more complex mo dels to b e examined

lisrel is describ ed in It is a metho d of estimating the parameters of co

variance structure mo dels The more complex task of estimating the structure has

also received attention tetrad is a program designed to do just this

Such a task is obviously at the core of scientic researchandthus not surprisingly

there is considerable debate ab out the philosophy b ehind this approach There is a

wide literature on the philosophy of probabilistic causality Some of the more recent

works include

There is obviously ro om for a lot more work to b e done in extending the metho ds

of path analysis to more general op erations and relationships b etween random vari

ables Perhaps eventually it will b e p ossible to integrate the techniques discussed

here with the copula based metho ds discussed in section Use of dierentmea

sures of asso ciation instead of the correlation co ecient would b e the rst step in

this direction

Common Sub expression Elimination in Probabilistic Arithmetic

byDAG Manipulations

A rather dierent use of graphtheoretical ideas for probabilistic arithmetic arises in

the elimination of common sub expressions from arithmetic expressions An example

of this is can b e found in chapter where weevaluate by hand the distribution

of the elements of the inverse of a random matrix The entries of the inverse

matrix in terms of the entries of the original matrix can b e written in the form

AD BC

It is not p ossible to evaluate the distribution of X in terms of pairwise applications

of the convolution formulae b ecause the A term app ears twice However wecan

see that dividing through by A gives

which can b e analysed in terms of pairwise convolutions It is the purp ose of the

present subsection to study when and howsuch transformations can b e p erformed

and to consider the prosp ects for an algorithmic application of these transformations

This goal has arisen b efore in the eld of co de generation for compilers The

problem there is given an expression such as determine a sequence of

machine instructions which will evaluate preferably in the shortest time

p ossible It is common to use a DAG Directed Acyclic Graph representation of

expressions in order to analyse this problem We can represent and

by the DAGs in gure Note that the direction of the arcs is implicit in these

diagrams by the ordering vertically down the page We can immediately see that

the prop ertyofhaving no common terms or sub expressions corresp onds to the

DAG b eing a tree For completeness a straightline program implementing is

given by

r B

r r C

r r A

r D r

r r

Straightline programs play an imp ortant role in computer science not only practically but theoretically

as well Lynch has suggested a numb er of reasons why straightline program length is a go o d measure

of algorithmic complexity Straightline programs have also b een used as a metho d for representing and

manipulatin g p olynomial s in symb olic algebra systems see also on the optimal evaluation of

expressions in such algebra systems We return to this asp ect in the following subsection 1

ADBC A

B C

Figure DAG representations for and

where r and r are registers

In generating co de within a compiler it is usual to not assume that the algebraic

laws such as distributivity hold b ecause of the rounding error in oating p oint

arithmetic Nevertheless the problem of optimal co de generation for arbitrary ex

pressions with common sub expressions is NPcomplete there are still dicul

ties in determining the evaluation order over the DAG The problem is relatively

easy for trees or collapsible graphs T aking account of asso ciative and

commutative prop erties of op erators is easy on trees but is dicult otherwise

The general problem whichisofinterest to us where b oth algebraic laws are

assumed to hold and common sub expressions or rep eated terms can o ccur has re

ceived little attention in the literature p The only pap ers explicitly

addressing this issue are and The second reference simply rep orts the

application of a metho d similar to Horners rule p Gonzalez and JaJa

ha ve made a detailed and intricate study of the problem They consider only

multiplications and additions and assume that the distributivelaw holds They de

velop an algorithm which will transform an expression DAGinto a tree whenever

this is p ossible and another which determines whether an expression calculated by

the new DAG is the same as the original one Gonzalez and JaJa prove a theorem

their theorem whichcharacterises when an expression can b e transformed into

a tree It essentially says that this can o ccur only when there are no rep eated terms

when expressed in a standard canonical form This is hardly surprising They also

presenta numb er of complexity results including the fact that even if the arithmetic

expressions are of degree degree of the expression when viewed as a p olynomial

then determination of an equivalent expression with the minimal numb er of arith

metic op erations is NPhard Their results are not extendable to the more

general case we are interested in arbitrary expressions including subtraction and

division op erations

More complex results have b een obtained recently by Reif Lewis and Tarjan

They consider more general program structures not just straightline

programs and more general arithmetic op erations Their original motivation was

the symb olic analysis of programs This involves constructing for each expres

sion in a program a symb olic expression whose value is equal to the value of the

program expression for all p ossible executions of the program Whilst reasonably

ecient algorithms for a restricted class of programs can b e develop ed the

general problem even over the integers is not only dicult but it is provably un

decidable Whilst this result only shows the imp ossibility of general global ow

problems it certainly removes some of the hop e wemayhave had for the p ossibility

of automatic rearrangementofeven simple programs for the purp oses of probabilis

tic arithmetic Note that whilst there exist a numb er of to ols and results on more

general graphical structures for representing programs with control ow such as the

program dep endence graph these are unlikely to b e of value for

our purp oses

Whilst the ab ove results do not help us greatly in solving the problem of rep eated

variables in probabilistic arithmetic they do provide go o d understanding of the

diculties involved Manes has made use of a general seman tics of owgraphs

in order to pro ve his p owerful metatheorem on the eect of tree structures

in uncertainty calculi There is still some hop e for useful results in this area but

see our comments in the following subsection

Conclusions The Diculties of Manipulating Graphs of Random

Variables

Having trekked through such forests of trees what can we conclude Unfortunately

very little which can not b e summarised in the motto Trees are trivial Nets are

not Wehave seen that whether it b e b elief networks covariance structure mo dels

or arithmetic expressions in compiler construction the existence of a tree structure

makes the problem feasible Whilst there are other classes of graphs which tend to

b e relatively easy such as seriesparallel graphs in general nontreelike structures

cause severe diculties

Ironically the relationship b etween graph structures and probability theory from

the p oint of view of rearranging expressions to remove common sub expressions may

well b e in the opp osite direction to that whichweenvisaged In Freeman et

al have considered the use of straightline programs as a representation of p olyno

mials in symb olic algebra systems They often obtain large greater than line

programs in the course of computations and haveinvestigated ways of optimizing

In order to prove this imp ortant result Reif and Lewis p make use of Matijasevics

negative solution to Hilb erts tenth problem whichsays that determining whether a p olynomial

QX X X hasarootover the integers is recursively unsolvable

Recently a very wide variety of easy problems on graphs have b een unied by the notion of tree

decomp osibil i ty These problems extend past the implementation of uncertainty calculi

Sahner and Trivedi study probabili sti c problems very similar to ours they consider addition

max min and selection of k th largest op erations on random variables on seriesparallel graphs Their

metho d of solution p erhaps not surprisinglyentails decomp osing the graph into a tree using the algorithm in

these programs in the manner of A very ecient and clever metho d they have

develop ed to do this is their StraightOpt pro cedure whichworks as follows

All instructions in the program are evaluated at random values for the

indeterminates mo dulo a random prime Then a binary search tree is

built to sort their values If a value of a new instruction is found in an

existing leaf the corresp onding instructions are assumed to compute the

same function and the new one is eliminated from the program The al

gorithm is Monte Carlo and its running time is that of sorting essentially

O l log l where l is the length of the straightline program since with

high probability the search tree is well balanced p

Application of StraightOpt to a line program pro duced by a factorisation

algorithm resulted in a saving in instructions

Whilst StraightOpt is Monte Carlo and thus can b e wrong the probabilityof

this o ccurring could b e made arbitrarily low Thus again we see the advantage

of Monte Carlo algorithms Such algorithms have b een used for multidimensional

integration for some time now and have recently found applications in primality

testing optimization and other hard problems It is our contention that

ultimately it will only b e Monte Carlo based metho ds which will b e able to handle

complex systems of interacting uncertainties

Other Lower and Upp er Probability Metho ds

Our results indicate that an objectivist frequency or

propensityoriented view of probability does not necessitate

an additive probability concept and that intervalvalued

probability models can represent a type of indeterminancy

not captured by additive probability

Peter Walley and Terrence Fine

My preference is for intervals because they can bebasedon

objective know ledge of distributions and because this

compatibility is demonstrable

Henry Kyburg

The use of dep endency b ounds or Bo oleFrechet b ounds leads to lower and upp er

probability distributions or probabilities Since there have b een a numb er of pro

p osed systems utilising lower and upp er or intervalvalued probabilities in recent

years it seems worthwhile to compare these mo dels with our use of lower and upp er

probabilities We will see that intervalvalued probabilities havebeenintro duced in

a wide varietyofways Some authors have explored epistemic sub jective proba

bilites and credal states in this manner Others notably Fine and his coworkers

have prop osed an ob jective frequentist interpretation of intervalvalued probabili

ties and have shown how such eects can arise in real physical systems We will

examine these and other systems b elow We do not go into these in much detail In

some cases esp ecially Fines work there are a lot of very technical results which

it is imp ossible to summarise in a couple of pages Instead we will present a fairly

nontechnical overview

The DempsterShafer Theory of Belief Functions

Idonotbelieve in belief

Karl Popp er

First we will examine the DempsterShafer theory of b elief functions whichhas

attracted a lot of attention in recentyears It is based on Shafers exten

sions of Dempsters The metho d has found a numb er of appli

cations and has b een recommended as b eing suitable for uncertainty

propagation in exp ert systems It has recently b een extended byYager to enable

representation and arithmetic op erations on b elief functions representing imprecise

numb ers We shall see b elow that the lower and upp er probabilities arising in

the DempsterShafer theory are quite dierent to those arising from our dep endency

b ounds The main dierence is that we consider the b ounds to b e secondary to the

primary notion of a single valued probabilityLower and upp er probabilities arise

in DempsterShafer theory b ecause of imprecise assignment of the probabilities in

the sample space probabilities are assigned to subsets of the sample space rather

than single elements of it Thus they are more of a primary notion

The Elements of the DempsterShafer Theory of Evidence

The following brief summary is based on that app earing in Let X b e the

universe of discourse akin to the sample space WeassumeX fx x x g

is nite although most results can b e extended to innite spaces Let m be a

measure on the p ower set of X suchthat

mA A X

The measure m is called a basic probability assignment bpa function A focal

element is any subset A of X suchthatmA If all the fo cal elements are

singletons then the b elief structure is called a Bayesian belief structure and all the

results reduce to the standard rules of probabilitytheory The union of all the fo cal

elements of a b elief function is called the core

The Belief and Plausibility functions dened by

mA A B X BelB

Yagers work can b e considered to b e a straightforward analogue of the histogram or DPD metho ds for

calculating functions of random variables in terms of discrete representations of their distribution functions

see section Yager do es not consider metho ds of condensation of the resultant b elief structures

although this would not b e dicult to do In this sense Yagers results are trivial and since he has not

describ ed any applications of the metho d it seems to b e of little interest to us here Of course this is not

to say that it do es not havethepotential to b e useful Indeed this is whywe are examining the basis of

the DempsterShafer theory

and

mA A B X PlB

are imp ortant in the theory and b ehavelikelower and upp er b ounds on P Awe

always have BelA P A PlA Writing the complementof B as B X n B

we can state an imp ortant prop erty of Bel and Pl as

BelB PlB

B PlB Bel

This can b e rewritten as

BelB Bel B

PlB Pl B

If the b elief structure is Bayesian then BelA P APlA Equations

and are why the theory is called nonadditive Shafer sets considerable

store by this arguing that it is a more accurate mo del of human reasoning under

uncertainty Berres pphas sho wn how the degree of nonadditivity of

a b elief function can b e measured in terms of an integral dened over the b elief

function

The p ointoftheabove constructions is to allow a more uncertain assignmentof

probabilities to events Instead of sp ecifying an exact probabilityofanevent one can

assign some probability to a subset A X without sp ecifying how the probability

mass is to b e distributed within A It is argued that this gives a more realistic

representation of total ignorance than the assignment of a uniform or noninformative

prior in the ordinary Bayesian probabilitymodel

Combination of Belief Functions

When wehavetwo pieces of evidence represented by b elief functions wemay wish

to combine them in to one piece of evidence The usual metho d of doing this

is to use Dempsters rule of combination although alternative rules do exist

Dempsters rule is also called the orthogonal sum of b elief functions The

op erator is essentially anding indep endent b elief structures Let m and m b e the

bpas of the b elief functions Bel and Bel with cores fA A g and fB B g

p q

resp ectivelyThen

mA m B

i j

A B A

i j

if A m m A

mA m B

i j

A B

i j

jX j

and m m Since p and q are of the order of this rule can havea

very high computational complexityVery recently approximations to which

reduce its complexityhave b een considered Dempsters combination rule can

b e represented in terms of the theory of Markovchains Norton has used this

representation to study the limiting eect of up dating b elief functions with

m His results can b e summarised bysaying That is he studies lim

i n

that evidential supp ort tends to converge to single fo cal elements or small groups

of them The actual details dep end on the sp ecics of the corresp onding Markov

mo del used

The assumption of indep edence of the separate pieces of evidence is imp ortant

for to b e valid This assumption has b een criticised bya numb er of authors

Williams has argued that not only should indep endence not b e

assumed for epistemic probabilities but that the very notion of indep endence is

not deneable within the terms of the theory given its view of the nature of evidence

p Errors resulting from rep eated no des common cause events in a fault

tree when analysed using DempsterShafer theory have b een rep orted by Guth

His suggested solution to this amounts to the computation of a global b elief

function akin to working with the complete joint distribution However as he

confesses in a fo otnote this scheme is impractical computationally for even a

small example it was necessary to resort to Monte Carlo simulation Dempster and

Kong ha ve made some progress in applying the ideas of Lauritzen Spiegelhalter

and Pearl see the previous section on graph theoretical techniques for handling

complex probability structures to the DempsterShafer framework They use the

idea of trees of cliques in order to b e able to indep endently combine subgroupings

of no des Nevertheless they to o conclude We exp ect however that realistic

practice will quickly driveustoMonte Carlo approximations the study of which

is just b eginning pF urther work on more complex structures of b elief

functions has b een rep orted by Kohlas and Shafer and Logan

Further Problems with the DempsterShafer Theory and its Relationship to

Classical Probability Theory

As well as the diculties of indep endence in combinations rep orted ab ove the

DempsterShafer theory has b een criticised in terms of the empirical acquisition

of b elief functions Whilst intro duced by Dempster in the context of sampling in

ference there are many diculties still to b e resolved Lemmer

has said that there is

strong supp ort for the conjecture that DempsterShafer theory cannot

in principle b e empirically interpreted as a calculus applicable to sam

ple spaces if the b elief functions ab out these sample spaces arise from

the real world byany sort of reasonably error free pro cess of observation

Thus though the numbers produced by this theory are often termed prob

abilities they cannot b e interpreted as probabilities ab out sample spaces

Pearl has compared the DempsterShafer theory with Bayesian probability structures on trees

and has argued in favour of the latter

Statistical asp ects of DempsterShafer theory have recently b egun to receive further

attention

It is tempting to ask whether the problems asso ciated with DempsterShafer

theory are simply a result of it masquerading as ordinary probability theory In

anycaseitisinteresting to examine to what extent the pro cedures of the theory

can b e derived from the standard basis of probability theory based on Bayesian

b elief structures Baron has examined DempsterShafer theory in the ligh t

of secondorder probabilities probabilities of probabilities and has shown that

Dempsters rule of combination is a sp ecial case of a more general rule for combining

second order probabilities This general rule is derived using just the usual basis

for probability Baron considers b elief functions to b e a sp ecial case of secondorder

probabilities

Falmagne has studied the em b edding of b elief functions in higherorder

probability spaces and has shown that the values of the b elief function have an

interpretation in terms of events in the emb edding space involving random vari

ables p He proves a theorem his theorem which gives a representation

of a b elief function Bel over a nite set X in terms of a collection f j X g of

jointly distributed random variables In this representation the b elief in B X is

interpreted as the random event

fmaxf j B g maxf j X n B gg

Falmagne explains how can b e understo o d in terms of a random utilitymodel

Kyburg Supp es Zanotti and the Relationship Between Dep endency Bounds

and Belief Functions

Of more interesttousisthework of Kyburg and Supp es and Zanotti

Kyburg proves the following theorem

Theorem Let m beaprobability mass function denedoveraframe of dis

cernment Let Bel be the corresponding belief function BelX mA

Then there is a closed convex set of classical probability functions S dened over

the atoms of such that for every subset X of

BelX min P X

P S

This theorem says that any b elief function can b e represented as a lower probability

The converse is not true without a further restriction That is not every lower

probability is a b elief function Sucient conditions are given by Kyburgs theorem

The question of whether DempsterShafer theory is better than ordinary probabili ty theory for some

problem is a dierent matter altogether We will briey discuss this b elow in the context of other uncer

tainty calculi as well For now just note that Smets has constructed an example problem whichhe

solves with b oth theories The reader is then invited to cho ose which answer he likes b est and thus cho ose the appropriate theory

Theorem If S is a closedconvex set of classical probability functions dened

over the atoms of and for every A A

jI j

A min P A A min P

i n

I fng

then thereisamassfunction m dened over the subsets of such that for every X

in the corresponding Bel function satises

P X BelX min

P S

This says that for a lower probability function to b e a b elief function it must b e a

capacity of innite order The righthand side of equation describ es

a capacity of order nwhich is the highest order p ossible here Not all lower

probabilities satisfy this For example Supp es and Zanotti pp oin tout

that if P is some arbitrary but nonempty set of probability measures then

P A inf P A

P P

is not a capacity of innite order and thus cannot b e generated by a random

relation on a probability space

The result of this is that b elief functions are sp ecial typ es of lower probabilities

Lower probabilities dened by and the analogous upp er probabilites dened

P A sup P A are not b elief and plausibility functions Since we feel that by

P P

it is these lower and upp er probabilities that are of the most interest they are how

the dep endency b ounds arise the theory of b elief functions has little to contribute

to the theory of dep endency b ounds and the lower and upp er probabilities generated

by them

DempsterShafer Theory as an Alternative Uncertainty Calculus

Although DempsterShafer theory is of no apparentvalue for our dep endency b ound

studies it do es havea numb er of other advantages for the purp oses of a general cal

culus of uncertaintyWedonothave the space or the motive to review this here

Instead we simply refer the reader to some recent reviews on this topic Bonissone

and Tong and P ang et al ha ve b oth presented brief reviews of dierent

metho ds Two b etter reviews giving a lot more detail are These pa

p ers compare Bayesian probability DempsterShafer theoryFuzzy set theory and

the mycin certainty factors The conclusions drawn include the fact that each

metho d has advantages and disadvantages and thus no one metho d is b est in all

circumstances in some applications it do es not seem to matter muchwhich is used

although in others it do es and all four metho ds are quite closely related see es

p ecially p Relationships b etween b elief functions and other uncertainty

calculi have also b een explored in

Fines IntervalValued Probabilities

Fine and his cow orkers have develop ed a theory of

intervalvalued probability which has some elements similar to the DempsterShafer

theoryHowever Fines motivation was rather dierent to Shafers Whereas Shafers

theory was develop ed from the p oint of view of epistemic sub jective probability

judgements Fine has worked closer to the ob jectiveinterpretations of probability

This is not to say he discards epistemological uncertainties In fact he suggests

that his mo del can handle b oth ontological and epistemological indeterminancies

Elsewhere Wolfenson and Fine ha ve considered the decision theoretic

viewp oint instead of the inferential

From an engineering p ointofviewoneofthemostinteresting applications of

the theory is the construction of sto chastic mo dels for empirical pro cesses that are

stationary but have uctuating diverging time averages These can not b e mo d

elled by a standard probability mo del by the Ergo dic theorem but they can b e

mo delled byintervalvalued probability mo dels An example of this is the

mo delling of icker noise in quartz crystal oscillators Wedonothave the space to

provide anytechnical details on Fines work here A very go o d recent review article

has b een published by Fine In any case the main p ointisthatheconsiders

lower and upp er probabilities which arise in circumstances quite dierent to those

we study which arise from dep endency b ounds However there seems to b e no

reason why these two approaches could not b e integrated although further research

is needed in order to achievethis What we feel is the most surprising asp ect of

Fines theories is that they do not app ear to have b een adopted and studied further

by other authors It is not just a matter of developing a nice theory for its own sake

As Fine has said

Overall our ob jective is not the creation of yet another anemic math

ematical theory Rather we wish to come to grips through selected

probabilitylike structures with certain features of naturally o ccurring

nondeterministic phenomena so that we can b etter understand and make

use of these phenomena

Other LowerUpp er Probability Theories

There havebeenawidevariety of other lowerupp er probability theories prop osed

over the years We just mention some of them briey

Epistemic theories of intervalvalued probabilities have b een studied by Ko opman

Smith and Supp es Higherorder probabilities the

probability of a probability are discussed by Skyrms Cyranski and Go o d

Mo dels of comparative probability instead of numerical probability have

also b een suggested as b eing more suitable for mo delling human judgement

Issues regarding making decisions based on probabilityboundsorlowerupp er

probabilities are discussed in There is in fact no great diculty in making

decisions based on intervalvalued probabilities Some go o d general discussions of

the idea of intervalism use of lower and upp er probabilities in preference to single

valued ones can b e found in Levis discussion in section

is particularly relev ant with a regard to our earlier remarks comparing the two

approaches of taking probabilityintervals as primary or secondary ob jects Levi

takes convex sets of credal states and derives lower and upp er probabilityenvelop es

He argues page that this is preferable to b eginning with an intervalvalued

probability measure and then developing other notions in terms of it

Because of the amount of literature on the topic wehave not b een at all exhaus

tive in our ab ove citations we do not makeany further attempt to synthesise or

summarise what are in many cases quite divergent opinions Wefeelhowever that

the main aim of the section has b een achieved to show that there is a wide varietyof

lowerupp er probability theories but in all cases the lowerupp er probability arise

in quite a dierent manner to our dep endency b ounds

Fuzzy Arithmetic of Fuzzy Numb ers

The concept of fuzzy numbers is expected to play important

roles in solving problems describing ambiguous systems such

as mathematical programming and deterministic systems as

is applied to fuzzy language probability fuzzy programs

fuzzy neurons etc

Masaharu Mizumoto and Kokichi Tanaka

Algebraic calculus on real fuzzy sets is devoted sic to play

acentral role in the development of the theory of fuzzy sets

and its applications to systems science

Didier Dub ois and Henri Prade

Agreat amount of work has already been accomplished

However the ability to apply fuzzy concepts to practical

problems requires a somewhat deeper understandingofthe

specicity of Zadehs theory

Didier Dub ois and Henri Prade

It is stil l tooearly to utter denite judgements about the

actual benet of introducing fuzzy numbers in topology or

random variable theories as wel l as the usefulness of this

concept in engineering applications

Didier Dub ois and Henri Prade

Fuzzy arithmetic is a topic rather similar to probabilistic arithmetic whichhas

received considerable attention in recentyears In this section we will compare these

two ideas in some detail We will study the dierences b etween the two ideas in terms

of their philosophical basis and interpretations practical computation metho ds and

practical ecacy We will show that the fuzzy arithmetic combination rules are

in fact almost identical to the dep endency b ounds we develop ed in the previous

chapter This allows a probabilistic interpretation of fuzzy set op erations Much

of what wehavetosaymay b e considered controversial to some but we feel our

arguments are sound and there is a only a very small amount of sp eculation and

this is clearly indicated as such We b egin with an outline of fuzzy sets

Given some universe U afuzzy set F on U is an imprecise subset of U

The set F is describ ed by its memb ership function whichtakes values on

Thus if U then for some x U if x then x is denitely in F if F

x then x is denitely not in F The value of x expresses the degree

F X

of memb ership of x in F As long as there is at least one x such that x

then F is said to b e normal A fuzzy set is usually considered to represent linguistic

vagueness rather than sto chastic uncertaintyHowever this is not always the case

Fuzzy sets can b e combined under the normal settheoretic op erations as follows

Intersection x min x x

F G F G

Union x max x x

F G F G

Complement x x

Alternativecombination rules dened in terms of tnorms and tconorms have also

b een considered We will discuss these b elow Many other op erations can b e p er

formed on fuzzy sets but as we shall see there is considerable dispute ab out the

correct denitions and interpretations for many of these Of most interest to us

is the situation where the fuzzy set represents an imprecise numb er In this case

one can dene rules for arithmetic op erations on these fuzzy numb ers This is the

sub ject matter of fuzzy arithmetic and is the fo cus of the present section

There are twomaintyp es of fuzzy numb ers whichwe will discuss Whilst

Dijkman et al talk of sev eral dierenttyp es they can all b e considered to

be variations of the twotyp es we will consider The rst typ e studied byHohle

Klement Lowen and others considers a fuzzy numb er to b e dened

by a nondecreasing memb ership function which lo oks like a probability distribution

function This approachisvery similar to the use of distribution functions in

probabilistic metric spaces We will briey return to this idea of a fuzzy

numb er b elow where we compare it with the other more widespread notion

Fuzzy numb ers are considered by Dub ois and Prade and others

to b e a fuzzy generalisation of the notion of an in terval in interval analysis

they are fuzzy sets of real numb ers Wesay X is a fuzzy numb er if its fuzzy

memb ership function is normalised sup x and is pseudoconvex or

X X

quasiconvex y min x z y x z Note that

X X X

pseudoconvexity otherwise known as unimo dality has sometimes erroneously b een

called simply convexity in the fuzzy set literature Of course convexity is inti

mately related to unimo dality see

By an analogy with the metho d of calculating functions of random variables

the extension principle pis used to calculate the mem b ership function

of Z f X X where X are fuzzy numb ers with memb ership functions

n i

in This says that

x x z sup min

X n Z X

z f x x

Manes p has said that Mathematicians do not usually feel that the existence of such

formulas deserves to b e called a principle He go es on to p ointoutanumber of severe deciencies in

the mathematical basis of fuzzy set theory and contrasts it with Top os theory and his general distributio nal

set theories see We do not have space here to describ e Manes substantial results and we simply

remark that his work should b e classed amongst the ma jor contributions of the eld

Some authors have given derivations of the extension principle but these are quite dierent

to Maness and their interpretations of fuzzy sets haveanumb er of shortcomings

When n and f wehave the supmin convolution

z sup min x x

Z X X

x x z

The min op erator is the fuzzy set intersection op erator and it can b e replaced

by a general tnorm T We then obtain the more general extension principle for

n as

z sup T x y

X Y

f XY

f xy z

This is obviously very similar to the op erations wehave encountered in studying

dep endency b ounds The dierence is mostly in interpretation although there is

the minor dierence that is unimo dal whereas F is monotonic this leads to

X X

only slightchanges in the pro cedures used to numerically calculate The

theory of fuzzy numb ers outlined ab oveisnowadays describ ed in terms of p ossibility

theory and p ossibility distributions We shall return to this topic after wehave

examined the general interpretation of the mo dal terms p ossible probable and

necessary A review of the theory of fuzzy numb ers including work up to is

given by Dub ois and Prade in

The rest of this section will cover the following material

Numerical metho ds for calculating the fuzzy number combinations describ ed

by the extension principle We will also consider the notion of interactivityof

fuzzy numb ers which is an analogue of dep endence for random variables

A general critical discussion of the interpretations of the mo dal terms p ossi

ble probable and necessary This was undertaken b ecause of the recent

app eal by fuzzy set theorists to a theory of p ossibility on which fuzzy num

b ers are based Weshow that the mo dal logic interpretations of p ossible

which they use to supp ort their development of p osibility theory are not the

only valid ones and that a probabilistic semantics for mo dal terms is preferable

for a numb er of reasons

We also present a general discussion of the relationship b etween fuzzy set

theory and probability This contentious issue is as old as the theory of fuzzy

sets We aim to restrict ourselves to a concise review of the arguments on this

topic and try to add something of our own

The relationships b etween interval fuzzy numb ers esp ecially in terms of p os

sibility and necessity measures and condence curves is then examined Con

dence curves are an old but rarely used statistical technique whichwe show

are very similar in their form and intuitiveinterpretation to fuzzy numb ers

Finally we summarise what we see as the relationships b etween fuzzy numbers

and random variables in the light of the dep endency b ound op erations

This section is rather longer than most in this chapter and thesis solely for the

reason that there is now an enormous literature over pap ers on the theory of

fuzzy sets and thus there are many dierentpoints to discuss

In a single sentence our conclusions could b e put as follows The theory of

fuzzy numbers as developed to date does essential ly nothing new compared to the

theory of random variables when proper account is taken of missing independence

information and would appear to be of little value in engineering applications

Numerical Metho ds for Fuzzy Convolutions and the Notion of

InteractiveFuzzy Variables

There are a variety of metho ds available for numerically calculating fuzzy arithmetic

convolutions They can b e classied into three basic typ es

Trap ezoidal triangular and LR fuzzy numbers

These are all parametric metho ds The idea is that a fuzzy numb er is repre

sented by or parameters and the arithmetic op erations are implemented

in terms of op erations on the parameters The implicit assumption in these

metho ds is that these few parameters are adequate to capture the fuzzy

uncertainties in human intuition p The formulae for pro ducts and

quotients are necessarily approximate The only reason the sum and dier

ence formulae are exact is b ecause the tnorm Min is used and so there is a

shap epreservation eect We discuss this in detail later on

The basic idea of LR fuzzy numb ers is to represent the memb ership function

of a fuzzy number by

m x

L x m

x m m

R x m

mis the core and m m is the support of The functions here m

L and R from onto are kno wn as shapefunctions Fuzzy number

arithmetic op erations can b e determined sometimes exactly in terms of m

m and for the fuzzy numb ers involved

Sampling along the xaxis to represent x

This simple technique whichentails only using values x for some set

X i

fx g is related to Baekeland and Kerres piecewise linear fuzzy quantities

and the DPD metho d for calculating con volutions of probabil

ity densities and suers from the same sorts of problems It should b e said

however that the problems can b e circumvented in similar ways For example

one of Dub ois and Prades criticisms of Jains metho d can b e

solved by using the condensation pro cedure as used by Kaplan in or b y

us in chapter

Interval Arithmetic on the Level Sets

This is p erhaps the most interesting metho d We examine it b elow and show

that

It is particularly simple for T M for a go o d reason and it can b e

extended to other tnorm intersection op erators

The apparent success and simplicity of the metho d are not sucient jus

tication for the claim that fuzzy arithmetic is the natural generalisation

of interval arithmetic

The fundamental result that allows calculation of fuzzy numb er convo

lutions in terms of level sets is a sp ecial case of a duality result that has

b een known for some time

Let us write F for the cut or level set of some fuzzy set F with memb ership

function That is F fx j x g Dub ois and Prade p

F F

state the following fundamental result Let M and N be two fuzzy intervals

with upp er semicontinuous memb ership functions and assume that M and

N for Let f b e continuous and orderpreserving that is

u u v v f u v f u v Then for all

f M N f M N

This means that f M N can b e calculated in terms of interval arithmetic on the

cuts of M and N Dub ois and Prade credit this result to Nguyen who sa ys

the original idea was due to Mizumota and Tanaka

Fenchels Duality Theorem

We will nowshow that is in fact a sp ecial case of a more general duality result

whic hwemadeuseofinchapter to calculate dep endency b ounds numerically

in an ecient manner Let p q rs denote the set of all nondecreasing functions

from p q in to rssatisfying pr and q s where pq rsand

p q rs f g Recall the denition of the quasiinverse F of a

function F p q rs

F y supfxj F x yg

Dene the binary op eration on the space of quasiinverses of the elements of

p q rsby

F G x inf LF uG v

C uv x

Theorem Let C C L L F G p q rs Then

F G F G

Up on letting C M it can b e seen that says the same thing as the funda

mental result This is b ecause the inmum in will always o ccur at

u v x since L F and G are nondecreasing In other words we can write

F GxLF G x

When L Sum reduces to

F GxF G x

This was proved directly by Sherwood and Taylor in see their prop osition

on page and used byKlement in who w as only aware of the result for

T M and L and could not see how it could b e extended Equation

can b e used for calculating fuzzy numb er convolutions by decomp osing memb ership

functions into an increasing part and a decreasing part and op erating on two parts

separately see chapter b elow In fact this duality result can b e traced backtoa

result of Fenchel in the theory of con vex functions

Fenchels duality theorem is b est explained with the aid of some diagrams This is

not done in nor in F enchels original pap er We shall followLuenb ergers

presentation whic hdoescontain diagrams The main idea we require is that

of a dual functional on the dual space of some linear vector space We consider

a convex function f over a convex domain C The dual space C and the dual

functional f are given by

C fx X j sup hx x if x g

and

f x sup hx x if x

where hx x i denotes the value of the functional x corresp onding to x Note that

x is a functional even when x is a p oint The space X is called the dual space

of a linear vector space X It comprises the linear functionals on X and is itself a

linear vector space

This should b e much clearer up on consideration of gure which depicts a

convex region C with a convex function f dened over it The conjugate function

f is a functional dened over the space X of dual functionals In the example

we consider f x isahyp erplane In other words for each point f xthere

corresp onds a line f x hx x ir We refer to the set f C as the epigraph

of f over C The set f C is itself con vex if f and C are One can similarly dene

conjugate concave functionals of concave functionals of concave functions g on a

concaveset D Wehave

D fx X j inf hx x ig x g

and

g x infhx x ig x

These conjugate functions are useful in solving optimization problems

Consider determining

inf f x g x

C D

where f is convex over C and g is concaveover D Examination of gure shows

this entails nding the length of the shortest dashed line in that gure ie the

smallest vertical distance b etween f C and g D The Fenchel duality theorem

allows this problem to b e solved in terms of the conjugate functionals f and g r r=-f* (x* ) [f,C]

-r=f (x* )

on the Figure Illustration of a convex function f on C X and its conjugate dual f

dual space X

[f,C]

[g,D]

x Figure Illustration of the optimization problem r

[f C]

[g,D]

Figure The idea b ehind Fenchels duality theorem

Theorem Assume that f and g are respectively convex and concave func

tionals on the convex sets C and D in a normedspace X Assume that C D

contains points in the relative interior of C and D and that either f C or g D

has a nonempty interior Suppose further that inf f x g x is nite

xC D

Then

inf f x g x max g x f x

xC D x C D

where the maximum on the right side is achieved for sone x C D If the

inmum on the left is achieved by some x C D then

maxhx x if x hx x if x

and

minhx x ig x hx x ig x

The idea b ehind this theorem is shown in gure The minimum vertical dis

tance b etween f and g is the maximum vertical distance b etween the two parallel

hyp erplanes separating f C and g D

The relationship b etween this theorem and theorem is only sketched here

Firstly consider the more general optimization problem

x inf f u g v

uv x

v D

Setting v x u and assuming the conditions u C and v D implicitly this can

b e written as

xinff u g x u

If wenowsethxg x so h is concave and apply an exp onential transfor

f x hx

mation letting F xe and H xe we obtain

x infF uH x u

Finallyby use of the multiplicative generator representation of Archimedian tnorms

or tconorms this can b e transformed into something of the form

x inf T F uHx u

This is the dep endency b ound formula and the fuzzy number convolution formula

The Fenchel duality theorem is very closely related to the Maximum transform

develop ed by Bellman and Karush They develop ed their results indep en

dently of Fenchel see p The maximum transform is the basis for the

Tconjugate transform studied in chapter

We exp ect that the duality theorems discussed ab ove will proveusefulinanum

b er of dierentways We also feel there are several new results p ossible One idea

whichwe briey examine in chapter is the discrete Tconjugate transform which

we think will enable an even more ecientnumerical calculation of dep endency

b ounds and fuzzy numb er convolutions

The duality theorem theorem presented ab ove is for general tnorms

Whilst not as simple for the case of T min it is still useful The theory of

convolutions and T conjugate transforms develop ed byMoynihan app ears

to oer some promise of improved metho ds for numerically calculating fuzzy number

convolutions under general tnorm intersection op erators A dierent gen

eralisation of is given byHohle Hohles prop osition p which

we will not quote here can b e viewed as a generalisation of to the scop e

of Brouwerian lattices whose duals are also Brouwerian Interpreting his results

Hohle says

There always exists a sto chastic mo del in which Lfuzzy quantities admit

an interpretation of abstract valued random variables and the binary

It is only of the form as there are a numb er of complications wehave glossed over here Further

details can b e found in chapter and

op erations corresp ond to the usual multiplication of random vari

ables Thus contrary to the widespread opinion there exists a p ointof

view from which the theory of fuzzy concepts can b e regarded as a

part of probability theory p

Hohles and op erations are generalisations of and to completely distribu

T T

tive complete lattices As Dub ois and Prade p oint out Hohles approachto

fuzzy numb ers is quite dierent to theirs see also and p W e will

compare Hohles more p ositivist approach with ours and others in more detail in

section

Shap e Invariance of Fuzzy Numb er Addition under Min Intersection and its

Eects

A feature of the supmin fuzzy numb er addition that is presented as an advantage

is its shap e invariance

Addition of fuzzy numb ers is remarkably shap einvariantcontrary to ran

dom variable convolution adding triangular distributions yields trian

gular distributions adding parab olic distributions yields parab oles etc

Dub ois and Prade conclude from this that fuzzy arithmetic is b etter than a

sto chastic approach for nding the shortest path of a graph

As so on as we allow distances b etween vertices to b e random we are

faced with many diculties among which are the intricate dep endency

of paths the necessity of p erforming rep eated convolutions of random

variables

Viewing the supmin convolution as the convolution of completely p ositively de

p endent random variables provides the understanding of what is happ ening here

The pairwise dep endence problem is ignored b ecause all the variables are com

pletely p ositively dep endent up on one another and therefore the variables can b e

combined simply pairwise Similar arguments have b een given byYazenin

The ironic thing is that whilst the problem of determining sto chastic shortest

routes when all the variables are indep endent is in fact dicult the more general

and more useful problem of determining b ounds on the shortest routes when the joint

dep endence structure is unknown is computationally simpler This is explained in

detail in Klein Hanevelds excellent pap er see the discussion of this in section

ab ove Thus the claimed advantages of using fuzzy numb ers for uncertain

network problems are overstated The more dicult but more useful sto chastic

problem can b e solved using appropriate techniques We feel that Klein Hanevelds

work in particular demonstrates that the sto chastic approach to op erations research

problems is feasible and that the b enets of the fuzzy approach

ha ve b een exaggerated but see the review by Zimmerman esp ecially

his discussion of the lack of duality results for fuzzy programming p

Relationships with Interval Arithmetic

We turn now to a brief examination of the relationship b etween fuzzy arithmetic

probabilistic arithmetic and interval arithmetic Our main p oint is that the fol

lowing p oint of view due to Dub ois and Prade is at b est quite misleading and

in our opinion completely wrong Dub ois and Prade con tend that fuzzy

arithmetic is the natural generalisation of interval arithmetic

The allornothing nature of interval analysis in contrast to probability

theorywhich admits gradations intro duces an asymmetry b etween them

whichonewould like to remove It is clear that the latter do es not

generalize the former since a function of a uniformly distributed random

variable the probabilistic counterpart of an error interval do es not in

general itself have a uniform distribution One of the ma jor contributions

of this b o ok is to prop ose a canonical generalization of interval analysis

that admits of appropriate gradations p

Our argument runs as follows The shap epreservation prop erty see ab ove which

holds when T M should b e considered as a p oint against fuzzy arithmetic If the

shap es of the initial memb ership functions are preserved so well then one mayaswell

disp ense with the function all together and simply work with the supp ort interval

The fact that adding two indep endent uniformly distributed random variables results

in a triangular distribution shows that wehave somehow included some information

ab out the addition op eration The most imp ortantpoint though is the trivial fact

that random variable addition does naturally generalise interval arithmetic if one

considers the supp ort of the distributions of the random variables involved In

fact one can think of the probability distributions describing the distribution of

values within the support interval Consider two random variables X and Y with

distribution functions F and F having supp ort u and u resp ectiv ely

X Y X X Y Y

If F df Z X Y then

supp F u u

Z X Y X Y

This is simply the interval arithmetic addition of supp F and supp F Thus it

X Y

is incorrect to maintain that a probabilistic approach to extend interval

additionw ould fail p See also our discussion of the relationship b etween

probabilistic arithmetic and interval arithmetic in We return to the topic of

intervals in section where we examine nested sets of condence intervals

Noninteraction of Fuzzy Variables

One of our dissatisfactions with fuzzy arithmetic is that there has b een little careful

consideration of dep endencyWehave already mentioned the connection b etween

the use of min intersection op erators in the extension principle and complete p ositive

dep endence of random variables Like Supp es we feel that the concept of indep en

dence and hence dep endence is one of the most profound and fundamental ideas

not only of probability theory but of science in general pand th us de

serves careful consideration in any uncertainty calculus Manes has sho wn that

the phenomenon of dep endency error will arise in a large class of uncertain ty

calculi Smith see esp ecially section of has argued that the notion

of conditional independence explicated with the use of directed graphs is useful for

anumb er of dierent uncertainty calculi Wehave examined the use of graphs in

probability theory in section ab ove For nowwe shall explore the fuzzy set ana

logues of probabilistic indep endence We shall see that not only is the analogue of

independence still p o orly interpreted but the analogues of dependence and measures

of dep endence have barely b een considered

The analogue of probabilistic indep endence in the theory of fuzzy variables is

called noninteractivity or unrelatedness Noninteractivityhas

b een assumed implicitly in our discussion of extension principle in the intro ductory

comments to this section ab ove Noninteractivity is dened in terms of a joint

memb ership function If

x y min x y x y supp supp

XY X Y X Y

or the equivalent statement in terms of p ossibility distributions then X and Y are

noninteractive The interpretation of noninteractivity is rather more problematic

than its denition Dub ois and Prade argue as follows

Noninteraction plays the same role in p ossibility theory as indep endence

in probabilitytheory but they do not convey the same meaning at all

interaction simply means a lack of functional links b etween the variables

while in the frequentistic view of probabilityanevent A is said to b e

indep endent from B if asymptotically A is observed as often whether it

B p simultaneously o ccurs with B or

This passage would seem to mean that noninteraction is a muchweaker notion

than indep endence It is certainly not an entirely satisfactory explanation of how

noninteraction is to b e understo o d Perhaps the main diculty is that there is no

analogue of a fuzzy event although some authors have tried to dene this

In probability theory the idea of a functional link is something whichmay cause

sto chastic dep endence but is not considered to b e equivalent to it see studies on

probabilistic causality and section on graphical ideas Nonin teraction of

fuzzy variables is also discussed in

Statistical independence arises in the theory of fuzzy variables when memb ership

functions or p ossibility distributions are determined from statistical exp eriments

Possibility measures usually refer to nonstatistical typ es of imprecision

such as the one p ervading natural language and sub jectiveknowledge

Recentlyhowever a statistical interpretation of p ossibility measures has

b een prop osed in the framework of random exp eriments yielding impre

cise outcomes however the concept of statistical indep endence under

imprecise measurements has yet to b e dened p

Since the notion of indep endence is logically prior to that of a statistical exp eriment

the ab ove deciency is a serious problem

Four Typ es of Interaction of Fuzzy Variables

There are at least four typ es of interaction of fuzzy variables whichhave b een dis

cussed in the literature to date We will now examine each of these in turn and we

will note their relationships with each other and with the statistical concepts from

which they were motivated

The rst typ e we consider is more accurately describ ed as a generalisation of

the non interactivitydenedby Two fuzzy variables X and Y are weakly

noninteractive or T noninteractive or independent if

x y T x y x y supp supp

XY X Y X Y

where T is some tnorm other than min Whilst it is easy to dene very little com

putation seems to have b een done with weakly noninteractive fuzzy variables The

extension principle for T noninteractive fuzzy variables is the general form

Dub ois and Prade ppha ve derived some prop erties of additions of

T noninteractive LR fuzzy numb ers for T Z W and but have not made much

use of them It is in fact p ossible to calculate arbitrarily go o d approximations to

the supT convolutions using the discretisation of the quantiles and the additiveor

multiplicative generator decomp osition of an Archimedean tnorm We will study

T noninteractive fuzzy additions further in chapter

The idea of strong nonfuzzy interaction is to restrict the jointmemb ership fuc

tion as follows

x y min x y x y D supp supp

XY X Y X Y

This can b e trivially extended to higher dimensions Dub ois and Prade

have used this idea to determine sums of fuzzy probabilities under the restric

tion that probabilities always must sum to one Strong nonfuzzy interaction is

equivalent to the recently intro duced statistical idea of regional dependence

Given two random variables U and V with F df U andF df V the

U V

idea of regional dep endence is where the joint distribution F is only equal to

the pro duct of the marginals on some limited region F u v F uF v for

UV U V

u v D supp F supp F Thus U and V are indep endenton D Regionally

U V

dep endent random variables can exhibit manyofthecharacteristics of indep endence

such as zero correlation constant regression functions and zero lo cal dep endence

see Strongly nonfuzzily interactive fuzzy variables have also b een studied

by Dong and Wong who ha vevery briey considered howthevertex metho d

a generalisation of simple in terval arithmetic on the level sets of member

ship functions can handle suchinteractions which arise in the solution of certain

equations

Strong fuzzy interaction is a further generalisation of strong nonfuzzy interaction

where the region D is a fuzzy set In this case the jointmemb ership function is given

x y min x y x y x y supp supp

XY X Y D X Y

The fuzzy set is called a fuzzy relation Although Dub ois and Prade p

pdiscuss the p ossibilit y of calculating functions of strongly fuzzily interac

tive fuzzy variables using a generalisation of the extension principle suchas

z sup min x y x y

X Y D

f XY D

z f xy

they say that the study of the prop erties and even more the calculation of

f X Y D are in general very dicult p We are unaware of any ex

amples of such calculations in the literature although very recently Sarna has

considered the fast calculation of the related but simpler formula

sup min x y x y

X Y D

where and are rectangular or triangular fuzzy numb ers and

X Y

minx y

x y

maxx y

with His results app ear to b e of little value for our purp oses

The fourth typ e of interaction is a further generalisation of whichhas

only really b een noted in passing Two fuzzy variables are weakly fuzzily interactive

pif

x y T min x y x y

XY X Y D

No examples using this have b een presented

Before we leave the topic of interactive fuzzy variables let us briey examine

the idea of measures of association By analogy with measures of asso ciation in

probability theory such as the correlation co ecient Buckley has considered the

use of a single or p erhaps interval parameter describing the degree of asso ciation

between two fuzzy variables Buckley and Siler have used probabilistic analogies

rather freely pto dev elop an ad hoc measure of asso ciation b etween

two fuzzy sets not fuzzy numb ers Their measure of asso ciation R is dened

implicitly pb y a strongly probabilistic analogy They consider the lower

and upp er Frechet b ounds for AND and OR op erations and use R as a parameter

taking on values within to giv e a mixture of the extreme rules This is

very similar to the parameterised families of bivariate probability distributions

Our criticism of Buckley and Silers approach is that they have eectively put

the cart b efore the horse They have suggested a wayofusing R to vary the

combination rules without saying where the values of R aretocome from Of

course an app eal could always b e made to sub jectiveintuitive judgement a

common panacea of sub jective metho ds Buckley has used these ideas in an exp ert

Their unsupp orted statement that The prop osed measures of asso ciation do es sic seem to oer a

convenientway to estimate prior asso ciations p notwithstanding

system He has more recently turned his attention to interactive fuzzy numbers

where he just denes the correlation co ecien t but do es not use it and

considers the eects of strong nonfuzzy interaction on sums and pro ducts of fuzzy

numb ers He considers the region D to b e dened bytheintersection of supp

supp and a quadrilateral region His results seem to b e of little value

Our conclusion up on examining this material can b e put as follows The ideaof

dependence or interaction is moreimportant in uncertain reasoning than the purely

distributional eects In other words dep endence or interactivity is something

arising in uncertain reasoning which has no real counterpart in the deterministic case

Until acceptable theories of dep endence are available the theory of fuzzy variables

will remain decient This is true even for purely sub jectiveinterpretations Under

these interpretations there are still further diculties in considering what is meant

by indep endencedep endence See the discussion of this matter with regard to

sub jective theories of probabilityinPopp ers Realism and the Aim of Science

the idea of indep endence turns out to b e contradictory under suchaninterpretation

The Mo dalities Possible and Probable and their Relationship

to Fuzzy Set Theory and Possibility Theory

It is natural to admit degrees of possibility and of necessity

as for probability

Didier Dub ois and Henri Prade

Therecan bedegrees of probability but not of possibility

Alan White

The use of fuzzy numb ers has lately b een advo cated in terms of their basis in

p ossibility theory which is said to b e quite dierent to probabilitytheoryWe will

now present an analysis of the notions p ossible and probable and will suggest

that it is incorrect to talk ab out degrees of p ossibility in the manner of the advo cates

of fuzzy set theory The following review of interpretations of the mo dalities p ossible

and probable was motivated by the apparentlack of balance in fuzzy set theorists

discussions of the topic

Our main thesis is that the discussion to date on the interpretation of p ossibility

has b een quite confused and mostly wrong We will argue that the sub jective empha

sis is misplaced or overrated and that clear and consistent ob jectiveinterpretations

of p ossibility are available Furthermore suchinterpretations clear up the distinction

between p ossibility and probability and make clear the fact that p ossibilitydoesnot

admit degrees That is it is p ointless to talk of degrees of p ossibility The main

p ointtokeep in mind is that the split b etween epistemic and physical p ossibilities

closely mirrors that b etween epistemic and physical probabilities We do not try to

argue for one or the other here such arguments in the past have not resolved the

ongoing debate but rather p oint out some advantages and disadvantages of these

two approaches Our own preference is for the physical p oint of view

All Kinds of Possibilities

Prop onents of fuzzy set based p ossibility theories often allude to the p ossibility

of mo dal logic stressing the distinction b etween these two concepts Thus

Zadeh in his well known pap er Fuzzy Sets as a Basis for a Theory of Possibility

says The interpretation of the concept of p ossibility in the theory of p ossibility

based on fuzzy setsis quite dieren t from that of mo dal logic Four years later in

a pap er presented appropriately enough at the Sixth International Wittgenstein

symp osium he says

The p ossibility theory which serves as a basis for testscore semantics is

distinct from but not totally unrelated to the p ossibility theories

related to mo dal logic and p ossible world semantics p

Dub ois and Prade ha ve app ealed to mo dal logic and in particular Aris

totles denition of necessity in terms of p ossibility see b elow They do not makea

clear distinction b etween their notion of p ossibility based on fuzzy sets and that of

ordinary mo dal logic However they do distinguish b etween epistemic and physical

p ossibilityWe examine this distinction further b elow

Our purp ose here is not to answer the question What is p ossibility or What

is the true meaning or essence of p ossibility LikePopp er we dismiss

such essentialist questions as b eing irrelevant In any case as Morgan has argued

To ask for the true meaning of necessity and p ossibility is parallel to

asking for the true meaning of negation Such notions havenouniversally

constant meaning other than the minimal way they interact with our

acceptance of prop ositions of various sorts p

Instead we will examine the distinctions b etween a numb er of dierent concepts of

p ossibilityHacking distinguishes b et ween many dierent kinds of p ossibility

A dierent classication is given by Lacey The main distinction which

is of medieval origin is b etween de re mo dalities and de dicto mo dalities This

division which is still controversial is explained as follows De re mo dalities refer to

prop erties of things whereas de dicto mo dalities refer to prop erties of propositions

Thus we could talk of physical mo dalities de re andepistemic de dicto mo dalities

Plantinga has discussed this distinction in some detail in

The notion of p ossibilityisinterpreted in mo dal logic by means of Kripkean

p ossible w orlds semantics This amounts to p ostulating an innityof

p ossible worlds and explaining mo dal terms with resp ect to identity and common

prop erties or accessibility pp across all p ossible worlds An obvious

criticism of this as Loux recognises runs as follows

Appropriately enough b ecause Wittgenstein develop ed a philosophy based on the meaning of words

Recall fuzzy set theorys linguisti c basis His famous blue b o ok op ens with the question What is

the meaning of a word A numb er of philosophers the most eminent b eing Karl Popp er have explicitly

argued against such a conception of philosophy

The trouble with p ossible worlds wewanttosay is that they repre

sent an exotic piece of metaphysical machinery the armchair invention of

a sp eculativeontologist lacking what Bertrand Russell called a robust

sense of reality

Another diculty is that interpreting p ossibility in terms of possible worlds comes

very close to b eing a circular argument We need not concern ourselves with p ossible

worlds further here but see Morgans probabilistic semantics of mo dal terms b elow

and we simply refer the reader to for the history of the sub ject The

idea of p ossible worlds has b een combined with the idea of fuzzy sets byForb es in

chapter of Noting what app ears to us to b e a devastating selfcriticism by

Forb es pThe role of mo dal logic is more to mak e the theses absolutely

precise than to facilitate any substantial consequences from them we simply

state that Forb es attempts to use the idea of degree of memb ership in the p ossible

worlds setting to examine some sorites typ e mo dal paradoxes asso ciated with vague

transworld identity relations We state his conclusions in his own words as we are

unable to follow his reasoning suciently to makethemany clearer

Thus while every de dicto mo dal thesis ab out identity has the same truth

value in the present framework as it has in the classical framework a

dierence emerges over the de re not b ecause identity somehow b ecomes

fuzzy but b ecause de re sentences intro duce a new fuzzy relation that

of counterpartho o d whichinturngives rise to degrees of p ossibility

The Relationship b etween Possibility and Probability

Fuzzy set theorists often talk ab out the dual notions of p ossibility and necessity

We shall see b elow that their idea of the interrelationship b etween p ossibility and

necessity is not shared byeveryone The duality they refer to is due to Aristotle and

is widely accepted in mo dal logic It says that a prop osition is necessary as so on as

the converse prop osition is not p ossible We will only concern ourselves here with

p ossibility

The notion of p ossibility is hardly new in a probabilistic context Recall the

equip ossible denitions of probability It is interesting to note that whilst

the Greeks eg Aristotle did have a notion of empirical as opp osed to logical

p ossibility and thus b elieved in the existence of real contingency p

they did not develop a notion of the probable nor did they observe the long run

stability of relative frequencies pp This is quite surprising given their

p enchant for gambling and given the similarities of their views otherwise Sambursky

notes that the Stoics who w ere like Laplace rigid determinists interpreted

p ossibility in terms of equal ignorance Hacking recalls a remark of Boudet

that the p erennial question ab out probabilit y is whether it is de re or de dicto

p Thus we can distinguish b etween de dicto epistemic probability and

de re physical probability

The mapping of the distinction b etween p ossibilities to probabilities and the

widespread distinction b etween the twotyp es of probability is considered by Hacking

in Such a carrying across to probability of the distinctions b etween p ossibilities

makes sense if one can in fact dene probabilities in terms of p ossibilities However

as Hacking and others have observed this is ultimately a circular exercise to say

that some events are equip ossible is simply to say that they are equally probable

All this reinforces our view that it is p ointless to talk of degrees of p ossibilityIf

probabilityisinterpreted as relative frequencies or in terms of prop ensities

such a circularity do es not arise The probability so dened is denitely

physical de re The role of p ossibilityinsuch a context is discussed b elow Hacking

observes that Laplace the champion of the epistemic view of probabilitydoesin

fact makethe de re de dicto distinction

When he Laplaceneeds a w ord to refer to an unknown physical char

acteristic he picks on p ossibility using it in the old de re sense This

was the language of his early pap ers When he wants to emphasise the

epistemological concept which nally captivated him he uses p ossibil

ity in what he makes clear is the de dicto epistemological sense

We will now consider the place of p ossibility within ob jective theories of prob

ability Three authors views on the matter are examined in some detail We will

progress from the less to the more formal and rigorous b eginning with White

Whites Problematic and Existential Possibilities

White actually distinguishes b et ween existential and problematic probability

p He distinguishes these two concepts in terms of the p ossibility of can and

that of may

Existential p ossibility It is p ossible for X to V can

Problematic p ossibility It is p ossible that X Vs may

He argues that it is problematic may p ossibility whichisrelevant to proba

bility Whites views on the relationship b etween p ossibility and probability are

summarised bysaying

Probabilityenters at this stage whichisintermediate b etween the exclu

sion of the p ossibility of something and the exclusion of the p ossibilityof

its opp osite There can b e degrees of probability but not of p ossibil

ity Something can b e highly probable or extremely improbable but not

highly p ossible or extremely imp ossible One thing can b e more or less

probable but not more or less p ossible than another The probabilitybut

not the p ossibility of something can increase or decrease Its p ossibility

can only app ear or vanish pp

Mo dal Language Exact Probabilistic Language

x is p ossible There is a scientic theory in which Prx

x is contingent There is a scientic theory in whichPrx

x is necessary There is a scientic theory in which Prx

x is imp ossible There is no scientic theory in which Prx

x is almost imp ossible There is a scientic theory in which Prx

x is almost necessary There is a scientic theory in which Prx

Table Bunges probabilistic interpretation of mo dal terms

Whites interpretation of probability is ob jective and he rejects b oth the epistemic

degrees of b elief interpretation and the Keynesian prop ositional interpretation

White says The relation of probability to p ossibility is parallel with that of

condence to b elief Furthermore Just as there can b e degrees of probability

but not of p ossibility so one can have degrees of condence but not of b elief

p The notion of p ossibility is not for White the opp osite of necessity but rather

it is the opp osite of certainty White also examines the de re de dicto distinctions

and argues that de dicto mo dalities do not exist

In the sense discussed there is no such thing as mo dality de dicto As

wesaw in detail a wide variety of things can b e qualied bydierent

mo dals but it is a varietywhich can all b e classed as de re The danger

of the thesis that mo dalityis de dicto is that it tempts one particularly

with such mo dalities as necessity p ossibility probability and certaintyto

embrace sub jective theories of mo dality according to which mo dalityis

acharacteristic of thought rather than that which can b e thought ab out

Bunges Probabilistic Degrees of Possibility and Necessity

Bunge has views quite similar to those of White Bunge argues that Aristotles

denition of necessity in terms of p ossibility is not applicable to physical p ossibility

b ecause it ignores the comp onent of circumstance p Bunge interprets proba

bility in terms of prop ensities but diers with Popp er in some resp ects He

says that Probability exacties p ossibility but not everything p ossible can b e as

signed a probability Ignoring a number of points Bunge makes in his discussion of

the interpretation of p ossibility and probabilitywe can present his view simply by

repro ducing table Bunge notes with resp ect to this table that Whereas in mo dal

logic there is a gap b etween p ossibility and necessity in a probabilistic context there

isacontinuum b etween them pp He also remarks on what is essen

tially Aristotles statistical interpretation of mo dality although he do es not call it

this see This is the simple fact that What is merely p ossible in the short

run for a small sample or a short run of trials may b ecome necessary or nearly so

in the long run p Finally note that real p ossibility cannot b e given an

op erational denition say in terms of frequency b ecause whatever is measured is

actual and not just p ossible Probability as a degree of p ossibilitywas also consid

ered by Kattso who however used a Keynesian prop ositional logic

framework Hart considered relativ e frequencies across p ossible worlds to give

degrees of p ossibility Since this is based on the Kripkean p ossible worlds semantics

see ab ove it is of little interest to us

Morgans Probabilistic Semantics for Mo dal Terms

Morgan has dev elop ed a probabilistic semantics for prop ositional mo dal log

ics His motivation for doing this was a dissatisfaction with the standard p ossible

worlds intepretation which apart from b eing circular interpreting p ossibility

in terms of possible worlds do es not admit a rigorous quantitative foundation

Morgans starting p ointisPopp ers work on conditional probability see app endices

iiiv of and also W e will not attempt to present Morgans

rather technical results in the limited space wehaveavailable here Let us just say

that his probabilistic mo del of standard mo dal logic is provably sound and complete

and is a strict generalisation of any p ossible worlds mo del Morgan also discusses

the mechanisms of b elief up dating and the choice of alternative logics Although

arguing that his probabilistic semantics has considerable advantages over the p os

sible worlds semantics Morgan do es not present his mo del as the only p ossible

solution As well as having a greater computational complexity the probabilistic

mo del as currently develop ed only allows the changes in the b elief of the necessity

of a prop osition to o ccur in rather large jumps p In any case Mor

gan has said elsewhere pthat a single unique and correct in terpretation of

mo dality is unlikely

Our confusion and uncertainties with regard to many mo dal prop osi

tions and many arguments containing the mo dalities is certainly strong

evidence in favour of the semantically underdetermined character of ne

cessity and p ossibility

Possibility as a Degree of Eort

An alternative ob jectiveinterpretation of p ossibility is due to von Mises On page

of V on Mises says how ordinary sp eech recognises dierent degrees of

p ossibility He interprets p ossibility in terms of the varying degrees of eort

involved in pro ducing a particular outcome This seems to b e the sort of thing

Zadeh had in mind when he sp ok e of the p ossibility of squeezing a certain

numb er of tennis balls into a b ox The larger the numb er of balls the lesser is the

p ossibility of doing it the greater the degree of eort is required Unfortunately

there do es not seem to b e muchwe can do with suchaninterpretation b ecause the

degree of eort do es not seem to b e adequately formalizable Nevertheless it do es

seem to b e the concept of p ossibility that has b een adopted by some prop onents of fuzzy set theory

With a probabilistic typ e mo del we are answering a question ab out what

p ercentage of the numb er of times we p erform an exp eriment will a given

outcome o ccur Whereas with p ossibilitywe are addressing questions

ab out how easy it is for a particular outcome to o ccur p

Yager do es not dene or explain his notion of p ossibilit y apart from saying

that In many instances the information with resp ect to the p ossibility distribution

asso ciated with a variable can b e inferred from information conveyed via natural

language In other words the degree of p ossibility is to b e determined in terms of

memb ership functions of fuzzy sets

Taking all the ab oveinto account it still seems that von Mises interpretation

of p ossibility as the degree of ease is probability in disguise Consider howwe

think it is hard requires a lot of eort to throw bullseyes in a game of darts

Considered in terms of scatter prop erties this is just another wayofsaying that

with the given exp erimental arrangement the probabilityofachieving this situation

is low Thus the degree of ease interpretation seems quite useless for any practical

purp oses

Some Fuzzy Set Theorists Views

Wenow examine some fuzzy set theorists views on the mo dalities of p ossible and

probable Dub ois and Prade distinguish b et ween physical

and epistemic p ossibility

The former p ertains to whether it is dicult or not to p erform some

action ie questions such as is it p ossible to squeeze eight tennis balls

in this b ox p

This obviously parallels the von Misian interpretation mentioned ab ove Dub ois and

Prade are really only interested in epistemic sub jectiveinterpretations and agree

with Zadeh that epistemic p ossibility can b e related to imprecise verbal state

ments Although they are ultimately interested in the sub jective p ossibilities they

do argue that natural language statements are not the only reasonable source of

knowledge ab out the p ossibility of o ccurence of events Thus epistemic p ossibility

and statistical data are not completely unrelated although as we shall see b elow

their supp osed relationship is by no means clear Dub ois and Prade go on to con

sider this relationship in more detail Perhaps their clearest statementofhow they

view the two concepts of p ossibility and probability is the following

P A denotes the probabilityof A understo o d as how frequent A is A

denotes the p ossibilityof A ie anumb er assessing someones knowledge

ab out this p ossibility in rely to the question may A o ccur

Later they say p

Possibilityisthusaweak notion of evidence In particularwhat is

probable must b e p ossible but not converselysothatwemay require

grades of probabilitytoactaslower b ounds on grades of p ossibility

We shall discuss Dub ois and Prades p ossibility and necessity measures b elowin

section This idea of p ossibility as an upp er b ound for probability has b een

develop ed further by Giles Wewillnow examine his arguments

Giles Interpretation of Possibility as an Upp er Bound on Probability

Giles has made a careful study of the relationship b et ween p ossibility

probability and necessity in fuzzy set theory and has develop ed an interpretation

along the same lines as ours in that he makes use of lower and upp er probabilities

Giles motivation for his work is that the ordinary approach to fuzzy sets

gives no indication of how one is to decide what particular numerical value

to assign to a grade of memb ership p ossibility etc in a given situation

or of how one should use suchvalues in for instance decision making

As a result the grounds for application of the resulting theory are to say

the least very insecure p

He go es on to develop a denition of grades of memb ership in terms of test

pro cedures A grade of memb ership is dened in terms of b ets on the outcome

of these testpro cedures

Giles b etting interpretation which is based on the Bayesian ideas of de Finetti

Lindley and Savage is more general than their metho ds b ecause we are not obliged

to retain the assumption that every rational agent should b e a Bayesian agent

Thus Giles interpretation partially answers the criticisms of the b etting interpre

tation of probability put forward byPopp er p Giles also admits

pthat his in terpretation is not the only correct or valid one

Giles interpretation is of particular interest to us b ecause of his consideration of

p ossibility in terms of lower and upp er probabilities He intro duces these ideas as

follows

Any rational agent b ehaves as though he b elieves that each prop osition

A has some true probability but he is not himself aware of its value

knowing only that it lies in the closed interval p Ap A

Perhaps the b est exp osition of Giles ideas is his pap er In this he says how

weinterpret the assignment of a degree of p ossibilityto A as an assignmentofan

upp er b ound on the probabilityof A Most interesting for us are Giles results

characterising p ossibility functions Without presenting all the denitions and in

tro ductory material necessary for a complete understanding of his results wecan

give an idea of their avour p

Theorem If f j i I g is a set of p ossibility functions then where

for every prop osition Aa supf ji I g is also a p ossibilityfunc

tion

Example Let P be any nonempty set of probability measures on some

Bo olean algebra B

A sup fpAj p P g A B

then P is a p ossibility function This exampleis v ery imp ortant for

every p ossibility function arises in this way

Theorem If is any p ossibility function then there exists a nonempty

set P of probability measures on T a totally disconnected compact Haus

dor space suchthatB is isomorphic to the Bo olean algebra of all closed

and op en subsets of T suc h that where P is dened by

He interprets theorem as saying that

every rational agent behaves as though he b elieves each prop osition A has

some ob jective probability pA of b eing true the probability assignment

phowever not b eing known precisely but known only to b e in a subset

of P of all probability assignments For he who has such a b elief will oer

to b et only if he would not exp ect to lose no matter where p lies in P

This use of lower and upp er b ounds on probabilities is identical to that whichwe

advo cated in chapter Note that one can adopt this interpretation without neces

sarily talking of a b etting interpretation of probability The probabilityofanevent

whichwe assume to b e an ob jectivequantity is unknown Weknow b ounds on

this though and wework with these b ounds in much the same manner that we use

ordinary interval arithmetic for calculating with simple quantities when we only

knowlower and upp er b ounds on the quantities

An imp ortant departure from Giles interpretation o ccurs however when we con

sider the combination op erations Wehave already seen that in fuzzy set theory

p ossibility measures are combined using the op erations These corresp ond to our

lower dep endency b ounds The upp er b ounds are combined and calculated with

the op erations However we use these op erations without intro ducing the idea

of p ossibility Furthermore for the oft used sp ecial case of T M wehave the

fact that theorem In this case our lower and upp er dep endency

T T

b ounds are identical

Giles has also briey mentioned the more general idea of b elief structures set

of all acceptable b ets of rational agents corresp onding to closed convex sets of

probability measures p This idea has b een discussed in more detail in

section where we discuss the work of Kyburg and others on lower and upp er probabilities

Some Possible Conclusions or The Necessity of Probable Possibility

What can we conclude from this general discussion of the mo dal terms p ossible

and probable Firstly it is apparent that there is no consensus as to how these

terms are related and what role the various notions should play in the examination

of uncertainty Secondly the degrees of p ossibility that fuzzy set theorists talk

ab out only make sense if p ossibility and probability are considered in an epistemic

sense Wehave also seen that the epistemicob jectivede dictode re distinction

applies equally well to p ossibility and probability Furthermore if an ob jective

interpretation is accepted then probability is the degree or measure of p ossibility

The notion of degree of p ossibility as somehow corresp onding to a degree of eort or

diculty do es not seem tenable Nor do the p ossible worlds semantics seem very

valuable in this regard Wehave also seen attempts Giles to consider p ossibility

and probability in terms of b ounds Wehave examined this idea elsewhere sections

and where we review work along the same lines as our interpretation of the

fuzzy set theoretic op erations in terms of dep endency b ounds

The Relationship Between Probability Theory and Fuzzy Set The

ory

Ultimately we may think of bridging the gap between fuzzy

interval arithmetic and the calculus of random variables

ie embedding both into a unique setting

Didier Dub ois and Henri Prade

Articial Intel ligence is philosophical explication turned

into computer programs

Clark Glymour

Wewillnow attempt to briey review various arguments on the topic of proba

bility theory versus fuzzy set theory and their general interrelationship This is

after twentyveyears still a contentious topic Wehave certainly not aimed for

completeness that would require far more space but wehave aimed to b e rea

sonably representative Although our preference should quickly b ecome apparent

anyway let us explicitly state our opinions here We feel that fuzzy set theory is

of little value in engineering applications Our reasons for this conclusion are sev

eral but the main two are the philosophical basis whichwe b elieve to b e confused

and wrong and the practical ecacy apparently close to zero when reasonably

compared with probability theory Since we do not have space to fully develop

our arguments we do not exp ect to convert many p eople in the following text

Nevertheless we feel the issue to o imp ortanttopassover in silence

A lot of the debate b etween prop onents of probability and fuzzy set theory is

essentially philosophical Whilst the immediate reaction of the engineer is to avoid

this just get on with building something that works it turns out that this is neither

desirable nor p ossible This is esp ecially true given that one of the main application

As many p eople have observed the question is really less a matter of probability versus fuzzy set

theory than a balanced comparison of their merits The p ointisthatthetwo approaches can b e and have

b een combined in a numb er of dierentways Nevertheless since it is the purp ose of the present section

to highlight the dierences asking the question in the ab ove form is reasonable

areas for the metho ds wehave b een discussing is Articial Intelligence We agree

with Glymour who says that articial intelligence is philosophy He argues that

Since AI is philosophy the philosophical theory a program implements

should b e explicit Any claim that a program solves some wellstudied

problem but do esnt sayhow should b e disb elieved p

We aim to show inter alia that the philosophical basis of fuzzy set theory is inad

equate for engineering and articial intelligence problems

One of the diculties in critically discussing fuzzy set theory was explained by

Cheeseman as follows

Unfortunately this the comparison b etween fuzzy and nonfuzzy theories

is not as easy as it sounds b ecause the fuzzy approach is itself fuzzy

there are fuzzy sets fuzzy logic p ossibility theory and various higher

order generalisations of these eg fuzzy numb ers within fuzzy set theory

This diversity complicates the task of critiquing the fuzzy approach

Toth has recen tly tried to clarify some of the distinctions and to develop a

more rigorous foundation for fuzzy settheory The reason wemention Cheesemans

complaint is that b ecause of the variety of dierent views it is dicult to know

which to criticise any criticism can b e deected bychanging ground slightlyIn

general we will refrain from attacking the most absurd and the weakest arguments

in favour of fuzzy set theory and we will concentrate on what app ears to b e the

most useful material

To us the two most convincing arguments are as follows

The whole enterprise of fuzzy set theory is based on the inherent imprecision

of natural language This is supp osed to b e an uncertainty of a completely

dierent kind to the uncertainty of probability theory It has its ro ots in sorites

typ e paradoxes Arbib has presented a simple argument against

this He observes that although p eople can certainly draw a degree of tallness

curve if pushed to it this do es not show that our concept of tallness has such

a form p He go es on to note that vague terms are normally context

sensitive a notion fuzzy set theory either ignores or handles in a very p o or

manner and that natural language is not inherently imprecise although it

may b e used imprecisely in some circumstances

Perhaps the most distressing mistake of fuzzy set theorists is to b e

lieve that a natural language like English is imprecise The fact that

Weallow ourselves one irresistable exception namely Goguens argument for the so cial nature of

truth Goguen up on realising the diculties in actually determining grades of memb ership or

degrees of truth suggested that the notion of ob jective truth was not as useful as one based on so cial con

sensus This pap er suggests wemust abandon classical presupp ositi ons ab out truth and view assertions

in their so cial context p Whilst this may app eal to totalitarian governments it has little to recommend it otherwise

many p eople use English badly is no pro of of inherent imprecision

In any case the p oint at issue for practical purp oses is the referentofaword

what the word describ es rather than the word itself A concentration on

linguistic asp ects was the cause for severe diculties in a stream of twentieth

century philosophywhich followed Ludwig Wittgenstein see

Our second argument is more app ealing to the engineer Fuzzy set theory based

methods do not work More precisely it seems generally fair to say that fuzzy

set theory has not b een used to develop any metho ds for any problems that

are demonstrably b etter than probabilistic or nonfuzzy metho ds Although

numerous applications have b een rep orted the fuzziness of the metho ds is

not essential to any success they mayhave Furthermore there has b een very

little hardheaded and honest comparison with nonfuzzy techniques We will

use the example of fuzzy control to illustrate this p ointbelow

Regarding the general applicability of fuzzy set theory Zeleny phas

argued that apart from human decision making and judgement there are no

other areas of application We agree with this but would even question the

applicabilitytohuman decision making

Fuzzy Control

An example of a suggested engineering application of fuzzy set theory is fuzzy con

trol The idea of this which seems to have b een rst studied by Mamdami

is to develop automatic controllers for dynamic plants by using linguistic informa

tion obtained by questioning human op erators of the plant That is one asks the

op erator howhecontrols the plant and then incorp orates these fuzzy rules into

an automatic controller It is suggested that this approachwhich do es still seem

to show some promise is suitable for highly nonlinear plants which it is dicult to

mo del explicitlyAsurvey of fuzzy control is given by Sugeno Fuzzy control

is considered to b e one of the most develop ed and successful applicationsof the

theory of fuzzy sets p Sugeno argues that Fuzzy control is without

doubt one of the most exciting and promising elds in fuzzy engineering

Not only are the fuzzy controllers rarely compared with the classical designs but

when they are it is only with the simplest PID Prop ortional Integral Derivative

controllers and little advantage if any is claimed The main disadvantage of

fuzzy control and this is admitted by Sugeno in his survey p is that there

are no analytical to ols to test the stability of these controllers Furthermore as has

Some authors consider fuzzy set theory as a purely mathematical theory and suggest that it b e judged

on its mathematical merits Whilst we admit that there has b een some very interesting mathematical

work esp ecially byHohle Lowen and others we generally agree with MacLanes assessment see

and the pap ers following that most of fuzzy mathematics is valueless It is b elieved that there are far

to o many mistakes and that most of the results are trivial A cynic mightsay that the same argument

applies to all mo dern mathematics to whichwewould reply p erhaps but it applies more so to fuzzy

mathematics See also Johnstones op en letter to Ian Graham Fuzzy mathematics is NOT an excuse for fuzzy thinking

b een recently shown by Buckley and Ying many fuzzy controllers are more

closely related to linear controllers than previously thought As the numb er of rules

in a fuzzy linear controller is increased its b ehaviour approaches that of a simple

linear controller

We argue that it is more the structure of the fuzzy controllers such as those

discussed in rather than the use of fuzziness per se that contributes

to their p erformance In fact there exists a little known probabilistic analogue to

these fuzzy controllers Black has dev elop ed a metho d based on conditional

expectation arrays whichwould app ear to b e a considerable advance over the fuzzy

controllers for the application areas envisaged The aim of Blacks work is similar

control of plants which can not b e readily mo delled but whichseemtobecontrol

lable bya human op erator Not only do Blacks metho ds app ear to p erform b etter

but they have a sounder basis Rather than implementing what the op erator says he

do es the conditional exp ectation controller observes what he actually do es and im

plements control laws based on these observations Although some fuzzy controllers

which adapt according to the op erators actions have b een rep orted see also

p these are the exception to the rule Czogala has suggested the use

of combined fuzzy and probabilistic control rules Further work is needed in order

to determine the practical value of this

To sum up fuzzy controllers if one of the most successful applications of fuzzy set

theory are not a go o d case for the practical value of fuzzy set theory for engineering

applications Whilst the metho ds do seem to work they are little or no b etter than

alternative approaches and p erhaps more imp ortantly there are no analytical means

to test stability and robustness Admittedly the second p oint applies equally to

Blacks fuzzy exp ectation arraycontrollers at present A more detailed comparison

of fuzzy set based metho ds and probabilistic metho ds for a sp ecic engineering

problem can b e found in

Discussions of the Relationship b etween Fuzzy Set Theory and Probability

Theory in the Literature

Let us now note some of the previous literature which compares fuzzy set theory

with probabilitytheory and which discusses the foundations of fuzzy set theory

There do es not yet app ear to exist a comprehensive and balanced discussion of

these topics

Several authors feel that there is v alue in saying that to an extent

probability and p ossibility fuzzy set theory convey roughly the same information

This is sometimes called the p ossibilityprobability consistency principle Muir

has sho wn that restricting fuzzy set theory in order to savethelaw of excluded

middle results in a probabilistic Bo olean algebra In phe giv es a more

discursive account of these matters He notes as wehave done already see section

that fuzzy set theory do es not provide sucientmechanisms for dealing with

This informal principle may b e translated as the degree of p ossibil ityofanevent is greater than

or equal to its degree of probabili ty whichmust b e itself greater than or equal to its degree of necessity p

relationships dep endence

Ralsecu has argued that probabilit y and fuzzy set theory can b e combined

by considering fuzzy random variables random v ariables taking on inexact

values Go o dman has considered in terconnections b etween fuzzy sets and

random sets This provides a bridge b etween fuzzy sets and probability theoryIt

is along quite dierent lines to that whichwehave pro ceeded

An interesting episo de is the discussion of Coxs theorem by Zadeh and others

Coxs theorem sa ys that under a numb er of reasonable assump

tions including additivity probability is the only measure of uncertainty In other

words given a few reasonable axioms for a measure of uncertainty probabilityis

the only one which satises all of them Lindley argues along similar lines

and concludes that no reasonable scoring rule will lead to the combination laws of

fuzzy sets Zadehs reply ppis of considerable in terest Zadeh says that

he would agree with Lindleys conclusions were it not for the necessary assumption

of additivity There would b e no issue to argue ab out if Professor Lindleys pa

per were Scoring rules and the inevitability of probability under additivity See

also page of Since we and others haveshown that the nonadditive

fuzzy set theoretic combination rules arise naturally in probability theory when one

makes no indep endence assumptions Zadehs comment seems to imply a tacit agree

ment with our conclusion Fuzzy set theory is eectively no dierent to the use of

probability theory when indep endence is not assumed

Some Further Discussions Hisdal Cheeseman and Others

Hisdal is another author who has examined the relationship b etween fuzzy sets and

probability and has studied the interpretation of grades of memb ership

She has develop ed an alternative foundation for fuzzy set theory which she calls the

TEE mo del Threshold Error and assumption of Equivalence This was

develop ed in resp onse to a numb er of p erceived inadequacies of the standard theory

suc hasthead hoc mo dications often necessary to the combination rules

the plethora of combination op erators that have b een prop osed the lack of con

sensus on the interpretation of memb ership functions and questions on the place

of probability in fuzzy set theory Wedonothave space to summarise

Hisdals rather intricate mo del Let it suce to say that Hisdal seems unaware of

the probabilistic Bo oleFrechet b ound interpretation of the standard fuzzy set com

bination op erators In fact she derives to the credit of her TEE mo del combination

formulae dierent to the standard minmax rules

Anumb er of authors have noticed the connexion b etween the Bo oleFrechet

b ounds and the fuzzy set op erators Wehave already discussed this p oint in section

Let us nowjustmake a few additional remarks Cheeseman who

has noticed the connexion has gone on to argue that one can use probabilities

to do things which fuzzy set theorists maintain can only b e done with fuzzy sets

The imp ortantpoint he says is that a strict frequentist view of probabilityisnot

always necessary By admitting sub jective probabilities many fuzzy problems

are solvable by probabilistic means Cheeseman argues that probability theory is

a richer and more p owerful framework than fuzzy set theory b ecause it allows the

rigorous representation of dep endencies He agrees with Stallings that for an y

given problem probability based metho ds seem at least as go o d as or b etter than

fuzzy set based metho ds Zadehs reply to Cheeseman is w eakly argued and

essentially comprises pro of by example and pro of byvehement assertion see

Grosof and Hec kerman ha ve b oth shown how a large number of

measures of uncertainty including fuzzy sets whichhave b een prop osed recently

can b e given straightforward probabilistic intepretations Wise Henrion Ruspini

App elbaum and others have also compared fuzzy set theory with probability See

our discussion of their work in section Henrion has presen ted a closely

reasoned argumentinfavour of probabilistic metho ds for handling uncertainty in ar

ticial intelligence As well as p ointing out that probabilities can b e used in a wider

range of problems than sometimes stated he notes several advantages of probabilis

tic schemes over nonprobabilistic ones including fuzzy set metho ds These include

the abilitytotake account of the nonindep endence of dierent sources of evidence

Whilst there are computational problems with probabilistic metho ds see the con

cluding chapter of this thesis these can b e overcome byMonteCarlo metho ds his

logic sampling This p oints out another disadvantage of fuzzy metho ds There

is no way one can simulate the correct result to check the validity of a prop osed

computation metho d

Concluding Remarks

Fuzzy set theory is based on the misguided premise that People reason with words

not numb ers pthey do neither they reason with ideas Th us the lin

guistic basis is the wrong starting p ointeven for purely human problems Even if

this rst p oint is not admitted then fuzzy sets are still of very little use for engi

neering problems Recall the example of fuzzy control It seems to b e much b etter

engineering to observe what an op erator do es than to ask him to say what he do es

In any case there is still no convincing argument that fuzzy sets are either correct

or necessary in dealing with the inherent imprecision of natural language When

one considers the practical ecacy of fuzzy metho ds the story is the same there is

no evidence for sup eriorityover probabilistic metho ds neither in the breadth of the

p ossible domains of application nor in the p erformance in a given domain Finally

wehave the argumentwhichwe feel clinches the matter the fuzzy set theory combi

nation op erations can b e explained simply in probabilistic terms by considering the

situation where no dep endency information is available The other supp osed advan

tages of fuzzy set theory over probability theory are also untenable For example

it is often said that the frequentist interpretation of probability theory severely re

stricts its domain of application to rep eatable exp eriments This is incorrect Either

the sub jective or the prop ensityinterpretations of probability can assign meaning

to the probability of singular events

We conclude by sp eculating that one of the reasons for the app eal of fuzzy set

theory particularly in the eld of articial intelligence is that naiveintrosp ection

can lead one to b elieve that fuzzy sets do accurately mo del human thinking The

goal of AI at present seems to b e to mimic or implementhuman thinking on a

computer We suggest following Lem in fact the wrong goal for engineering

applications Certainly intelligentmachines are desired but that do es not mean they

havetohaveanything in common with human intelligence Lem has noted in a

retrosp ective review of military history of the st century that articial intelligence

b ecame a force to b e reckoned with precisely b ecause it did not b ecome the machine

embodiment of the human mind pp

Condence Curves Possibility and Necessity Measures and Inter

val Fuzzy Numb ers

There is a little known statistical technique known as condence curves which seems

to b e surprisingly closely related to fuzzy numb ers Wewillnow explore this rela

tionship Our aim is not to show that they are identical they are not but rather

to show that the intuitive information they capture is nearly identical

Condence Curves Their Denition and Application

Condence curves were intro duced byCox in They were develop ed

further by Birnbaum but ha ve received very little attention since The

basic idea of condence curves is to have a set of nested condence intervals for

some parameter at dierent condence levels Coxs motivation for this was that

the ordinary condence intervals do not giveany measure of the informativeness

of a sample

When we write down the condence interval

p p

x n x n

for a completely unknown normal mean there is certainly a sense in which

the unknown mean is likely to lie near the centre of the interval and

rather unlikely to lie near the ends and in which in this case even if

do es lie outside the interval it is probably not far outside The usual

theory of condence intervals gives no direct expression of these facts

Birnbaum denes condence curv es as follows

For each c c let t cand t c resp ectively denote lower

L U

and upp er condence limits for an unknown parameter at the c

level based on the observed value of some suitable statistic t Sucha

pair of estimates also represents a c level condence interval In

the c plane for each c plot the twopoints t cc and

Consultation of the Science Citation Index revealed the only references to Birnbaums two pap ers were

By Birnbaum but not concerned with condence curves Irrelevant said nothing of value with regard

to condence curves or The pap ers by Kiefer and Mau whichwe discuss b elow .5

Figure A typical condence curve after Birnbaum

t cc For c wehave t c t c whichisamedian

U L U

unbiased p oint estimate of represented by the p oint t

We denote this graph or the function of which it represents by c t

In most problems of interest such a graph is continuous and resembles

that in gure p

Birnbaum pgiv es the following simple example of a condence curve Con

sider an estimate of the mean of a normally distributed random variable with unit

variance Then

x x

c x

x x

There are many problems which arise in the use of condence curves particularly

admissibility These are considered byBirnbaum in some detail in

Further Work on Condence Curves for Statistical Inference

We will now examine Kiefer and Maus comments on condence curves This con

stitutes a comprehensive literature review of the topic Kiefer men tioned

condence curves in his criticism of the general theory of condence intervals His

main complaint can b e b est describ ed by the following simple example see

Cho ose b etween twohyp otheses H df X N and H df X N on

the basis of a single observation x of X The standard pro cedure would b e to not

reject H accept if x and accept H if x the probabilitybeing

that a correct decison will follow from this pro cedure Kiefers complaintis

that given two dierent observations x or x the same degree of con

clusiveness is expressed by the test Similar complaints have b een voiced by other

authors and it is often said that condence intervals give good

estimates of the validity of conclusion before the data has b een seen but not after

wards Kiefers solution is to use conditional condence intervals For

the ab ove problem this would involve partitioning the sample space into for ex

ample three sets and The probability of the 1

FN FFN

a b c

Figure A Fuzzy Numb er FN and a Flat Fuzzy Number FFN

correctness of the decision to accept or reject H is now conditional on whichof

these sets the sample value x falls within The probabilities are and

resp ectively Kiefer says pthat the metho d of nested condence sets

condence curves although it do es have the frequentist interpretation of con

dence without conditioning it suers from the the same defect as the example

ab ove This is not necessarily true although considerable further researchisre

quired to resolve the problem completely Birnbaum phas giv en dierent

condence curves for dierent sample values x The intuitively more informative

sample gives a narrower condence curve reecting a more conclusive result It

is surprising that these ideas have received such little attention in the literature

Mau has recen tly examined the use of condence curves he calls them

condence distributions He carefully derives some of the prop erties of condence

distributions and shows how their use quanties the strength of evidence against a

null hyp othesis in the lightofgiven data pHe also dev elops prop erties of

central and symmetric condence distributions One of the most interesting p oints

Mau makes is that condence distribution pro cedures are very similar to Bayesian

pro cedures In fact with one restriction Maus formula for updating a condence

distribution in the pro cess of accumulating data is essentially Bayess formula

This connexion b etween the two sides of statistical inference is very interesting and

deserves further research

Relationship Between Condence Curves and Fuzzy Numbers

Our motivation for examining the relationship b etween condence curves and fuzzy

numb ers comes from comparing gure with gure Apart from the factor

of two dierence in the vertical scaling these two graphical representations are very

similar Furthermore the condence curveisvery directly related to the distribution

function of the quantity in question Recall equation There are several

Birnbaums example is for a two sample test of the dierence b etween two quantities 1

Histogram

Nested

Intervals

Figure Memb ership function constructed from histogram data

dierent approaches for developing a tighter analogy b etween the two concepts We

will examine the p ossibilityof conditional condence curves as an analogy of at

fuzzy numb ers b elow For now let us consider two ideas relating fuzzy numb ers and

statistical data

Dub ois and Prade ppha ve discussed how to determine member

ship functions p ossibility distributions from statistical data They b egin with a

nested set of intervals fI g such that

a b I I I A B

They consider a series of q imprecise exp eriments for which the outcome is an

interval a b k q They imp ose a consistency requirementthat

k k

a b a b

i i

and let A B a b Then they construct a p ossibility distribution or

i i

memb ership function x along the following lines although their notation and

presentation is rather dierent

if x I

jfk j k q a b I n I gj if x I n I

k k j j n x

if x I

The result is a memb ership function which lo oks something like gure Apart

from the maximum value b eing rather than this is identical to the condence

curve pro cedure the vertical scale b eing degree of memb ership rather than

condence Thus at least for statistical data the fuzzy set metho d provides no

more information or insight than the classical probabilistic metho ds Admittedly

this is not the domain of application that prop onents of fuzzy set theory have aimed

for We return to this later

Dub ois and Prade go on to suggest the use of p ossibility and necessity measures

based on the ab ove acquistion pro cedure McCain has also considered a con

dence interval interpretation of fuzzy numbers However his hard to follow pap er

says very little of substance The construction of memb ership functions from statis

tical data has also b een considered byCivanlar and Trussell Their conclusions

are of negligible value

Interval Fuzzy Numb ers Possibility and Necessity Measures Lower and Upp er

Probability Distributions and Higher Order Fuzzy Numbers

We will now examine Dub ois and Prades p ossibility and necessity measures and

their relationship with lower and upp er probability distributions We will follow the

presentation in More detail can b e found in Further work on interval

valued fuzzy numb ers and higher order fuzzy sets a related idea can b e found in

Possibility and necessity measures are condence measures that satisfy

A B maxA B

and

N A B minN ANB

resp ectively for any events A and B They are related by

A N A

Given a p ossibility distribution corresp onding to a memb ership function

this means that

A sup f j ag

and

N A inf f j Ag

The relationship b etween p ossibility distributions and p ossibility measures is con

sidered to b e analogous to the relationship b etween probability densities and prob

A ability distributions Possibility measures have the prop ertythat

sup A Dub ois and Prade show that p ossibility and necessity measures are

related to probabilityby

N A P A A

They present the theory of fuzzy numb ers in terms of p ossibility theory and show

how a fuzzy numb er or as they prefer to say a fuzzy interval describ es lower and 1

Figure Lower and upp er probability distributions for which there is no corresp onding

normal fuzzy number

upp er probability distributions as follows

P u u sup f r j r ug F u

u u q

u q

and

F u P uN u inff r j rug

u q

u u q

The fuzzy quantity Q has a memb ership function with a at in the interval q q

corresp onding to the region b in gure When there is no at q q there

is only one value of x x q such that F x and F x Since the lower

and probability distributions are derived from normal fuzzy numb ers the situation

depicted in gure can never o ccur Admittedlyby sacricing normality one

could stretch the analogy to cover this case We feel that taking the lower and

upp er probability distributions as primary as bounds on some inaccurately known

probability distribution is preferable to the ab ove approach

Finally we note in passing that Dub ois and Prade have used the connexion b e

tween p ossibility and necessity measures and lower and upp er probability distribu

tions in order to dene and study the prop erties of the mean value of a fuzzy

numb er They dene exp ectations of fuzzy numb ers in terms of the exp ecta

tions of random variables whichhave the lower and upp er probability distributions

They also note some connexions with the theory of random sets Dub ois

and Prades psuggestions for further researc h include a study of the de

termination of P M N in terms of P M and P N where

P Q fP jA measurable A P A N Ag

for some fuzzy number Q where denotes the supmin convolution and A and

N A are dened ab ove Our dep endency b ounds in chapter essentially answer

this question for a more general class of probability measures Dub ois and Prade

palso discuss the relationship b et ween their notion of a fuzzy numb er and

Hohles and others see the b eginning of section and section Dub ois and

Prades conclusion is of some interest to us They say that Ultimately wemay think

of bridging the gap b etween fuzzy interval arithmetic and the calculus of random

variables ie emb edding b oth into a unique setting p This has b een our

aim in the present section We conclude bysaying that there is a reasonably close

corresp ondence b etween memb ership functions and condence curves This par

allels the corresp ondence b etween p ossibilitynecessity measures and lowerupp er

distribution functions

General Interpretations Positivist vs Realist

Al l the things in fact that we approached by our senses

reason or intel lect are so dierent from one another that

thereisnoprecise equality between them

Nicolas Cusanus

Elaboration of this idealeads to the concept of a spacein

which a distribution function rather than a denite number

is associated with every pair of elements

Karl Menger

The ob ject of this section is to compare two p oints of view for using distribu

tion functions as a generalisation of numb ers One approach is that whichwehave

adopted This entails taking the view of random variables of ortho dox Kolmogorov

probability theory The random variables are the primary entities and they are ma

nipulated in terms of their distribution functions The other viewp oint is that taken

in the development of probabilistic metric spaces In this case the distribution func

tions are considered primary and op erations are p erformed on distribution functions

without reference to any underlying random variables

We shall see that one of the advantages of the latter viewp oint no longer holds

It was originally adopted for two reasons The rst was the severe mathematical

diculties arising in the random variable viewp oint b ecause of intricate dep endency

relations The second was that there were many functions of distribution functions

whichwere not expressible in terms of random variables The interpretation of

the and op erations as dep endency b ounds changes this b ecause nowthese

TL TL

op erations can be interpreted in terms of random variables

Probabilistic Metric Spaces

Wenow presenta very brief outline of the theory of probabilistic metric spaces

More details can b e found in the excellent book bySchweizer and Sklar See

Historically this statement is not true the reasons why the development of probabili sti c metric

spaces went the way it did are more complex than the few sentences ab ovewould indicate However the

main p oint that the random variable viewp oint did not allow all op erations is true

also the review of the b o ok by Bro oke who men tions some of the p ossible

physical applications of the theory Rosen has also considered the idea

and suggested uses in physics

The original idea of probabilistic metric spaces was prop osed by Menger

who w as motivated byPoincares paradox see also p

Poincare noted that for physical measurements given that A B and B C one

could not necessarily conclude that A C when the equality relation is interpreted

physically Menger suggested that a distribution function b e asso ciated with each

pair of quantities thus providing a generalisation of the notion of a metric space

rst intro duced byFrechet in

The distribution function asso ciated with two elements of a statistical

metric space might b e said to give for every x the probability that the

distance b etween the two p oints in question do es not exceed x

Menger subsequently develop ed his ideas further in where he suggested that

the theory could b e applied to psychophysics and physics b oth microscopic and

macroscopic

Mengers original prop osal was for triangle functions T suchthat inter alia the

triangle inequality

F x y T F xF y

pr pq qr

would hold F is the distance distribution function for the twopoints p and r

Wald suggested that could b e replaced b y

F x F F x

pr pq qr

where denotes convolution This has the simple probabilistic interpretation that

the probability that the distance from p to r is less than x is at least as large as the

probability that the sum of the distances from p to q and from q to r regarded as

indep endent is less than x These Wald spaces have not generated muchinterest or

many results b ecause of the severe mathematical diculties caused by the complex

dep endency structure induced on the underlying random variables

The ma jority of the work on probabilistic metric spaces has taken the original

approach of Menger and studied a variety of dierent triangle functions One of

these is the op eration given by

F Gx sup T F uGv

Luv x

for some tnorm T An imp ortant result due to Schweizer and Sklar is that

apart from L max and T min the function is not derivable from a function

on random variables See also Schweizers recent remark in This means

that for any F and G in there do not exist random variables X and Y where

such F df X andG df Y and a Borel measurable function V

that df V X Y F G This implies that the viewp oint taking distribution

functions as primary is in a sense more p owerful However as wehave seen and in

fact as Schweizer et al were the rst to show the op erations do arise naturally

as dep endency b ounds for functions of random variables The impact of this on the

development of probabilistic metric spaces remains to b e seen It is certainly a topic

which deserves further investigation

Philosophical Asp ects of Probabilistic Metric Spaces

Menger develop ed a numb er of philosophical ideas from his work on probabilistic

metric spaces His general view of the theory was that it was a p ositivist geom

etry Positivism or rather logical p ositivism is a philosophical do ctrine

which can trace its ro ots to Humes empiricism It holds that no prop ositions should

b e considered to b e true unless veried by direct sense exp erience It has b een shown

bya numb er of authors most notably Karl Popp er that the whole scheme

is fundamentally awed It is fair to say that p ositivism is dead Mengers use of

the term was to indicate that the ob jects of geometry whichwe p erceive and can

verify are not the ideal p oints or lines of Platonic geometry but rather the blobs

and fuzzy regions of a probabilistic geometry This seems to b e the appropriate

viewp oint for interpreting currentwork on probabilisitic metric spaces

The viewp ointwe adopt is that random variables are randomly varying quan

tities Wewould prefer the quantities to b e not varying in which case we could

readily calculate the functions of interest Instead the b est we can do is to try

to determine the distribution of the functions of interest given the distribution of

the random variables This viewp oint seems similar to that necessary for interpret

ing a random metric space see chapter of Random metric spaces rst

intro duced by Spacek have b een shown bySchweizer and Sklar to

in fact b e a prop er part of the theory of probabilistic metric spaces Calabreses

investigations in this area seem p otentially useful He shows that when the

standard distribution based approach is examined in terms of random variab es

quite p eculiar eects can arise We feel that there is considerable scop e for further

research in this area esp ecially if the dep endency b ound viewp oint is adopted We

hop e to pursue some of these issues ourselves at a later date

General Conclusions

Wehave seen a surprisingly large numb er of connexions b etween the material we

develop ed in chapter and other ideas Some of these connnexions have suggested

areas for future research and others have shown that some results that have b een

Amongst those whichwe do not discuss here p erhaps the most interesting and promising is his very

careful analysis of the idea of a random variable from the p oint of view of a general theory of variables or

uents He develop ed new improved notation for the concepts of variables and functions

see also which he used in a calculus textb o ok This asp ect of his work has received

little attention in the literature since

It is no accident that the standard probabili stic metric space structure has b een adopted bysome

fuzzy set theorists the goals and techniques are the same b oth sidestep the issue of dep endence See

for example the works byHohle Klement Lowen and others

presented in the literature duplicate earlier work The most useful connexions and

directions for future research are as follows

The extensions by Hailp erin of the Bo oleFrechet b ounds using the techniques

of linear programming suggest the similar application of general mathematical

programming ideas to dep endency b ounds for functions of random variables

This would generalise the b ounds to functions of more than twovariables

Graphtheoretic techniques may b e of use in probabilistic arithmetic There are

two issues the control of calculations using the structure of the graph and

determination and approximation of complex sto chastic dep endencies using

graph theoretic metho ds and the p ossibility of transforming expression DAGs

in order to make determination of distribution functions easier As wesaw

however the prosp ects for the latter are not very good

Our viewp ointtaken in interpreting our b ounds on probability distribution

functions is quite dierenttothattaken in most contemp orary theories of

lowerupp er probabilities However it seems that it might b e p ossible to inte

grate our approach with other theories In particular Fines theories of lower

probabilities deservemoreattention

The theory and practice of fuzzy arithmetic and fuzzy numb ers has a lot in

common with probabilistic arithmetic whichmakes use of dep endency b ounds

The dualitytheorywe used to develop our numerical metho ds encompasses

the results used in calculating op erations on fuzzy numb ers and shows how

extensions are p ossible The intuitive ideas captured by fuzzy numb ers can

b e equally well represented using probabilistic techniques such as condence

curves

Our viewp oint is somewhat dierent to that adopted in the theory of prob

abilistic metric spaces although there is some scop e for integrating the two

approaches

To summarise the chapter as briey as p ossible Thereare numerous connexions with

other results some apparently of no use at al l some of interest some immediately

useful in other areas and some potential ly useful and suitable subjects for further

research

Chapter

An Extreme Limit Theorem for

Dep endency Bounds of

Normalised Sums of Random

Variables

This becomes clearer when you restrict your considerations

to the maximum and minimum of quantity

Nicolas Cusanaus

This chapter presents a new result on the b ehaviour of dep endency b ounds of

iterated normalised sums We show that the dep endency b ounds converge to step

functions as the numb er of summands increases The step functions are p ositioned

at p oints which dep end only on the extremes of the supp orts of the summands

distribution functions With very minor dierences this chapter will app ear in

Information Sciences with the same title as this chapter In order to make this

chapter selfcontained there is some reiteration of material covered in chapter

Since most of this material is not widely known this do es no harm

Intro duction

Dep endency b ounds are lower and upp er b ounds on the distribution of functions

of random variables when all that is known ab out the random variables is their

marginal distributions They have b een recently studied byFrank Nelsen and

Schweizer who sho wed that Makarovs solution to a question originally

p osed by Kolmogorovfollows naturally from the theory of copulas Dep en

dency b ounds arise in the development of probabilistic arithmetic see chapter

and it is in this context whichwe study the question of limit results for dep endency

b ounds of normalised sums

If we write ldbS and udbS for the lower and upp er dep endency b ounds

N N

X and fX g is a sequence of random variables with distribution of S

i i N

functions F thenweshowthatasN approaches innityldbS andudbS

i N N

approach unit step functions The p osition of the step functions dep ends only on the

supp ort of the F and this is whywe refer to our result as an extreme limit theorem

Note that our result is actually more analogous to the law of large numb ers than

the central limit theorem The central limit theorem describ es the b ehaviour of

We prove our result in a fairly straightforward manner by making use of the

prop erties of T conjugate transforms These transforms whichplay a role analogous

to the LaplaceStieltjes transform in the central limit theorem for sums of random

variables have b een develop ed byRichard Moynihan Apart from

b eing mathematically interesting our limit theorem has a practical interpretation

with regard to the suitabilityofinterval arithmetic for certain problems We show

that if one has to calculate the dep endency b ounds for the sum of a large number

of random variables then one can use interval arithmetic from the outset using

just the endp oints of the supp orts of the distributions b ecause one will lose little

information in doing so

The reason whyonewould want to calculate dep endency b ounds rather than

ordinary convolutions can b e explained by considering probabilistic arithmetic The

aim of this is to replace the ordinary arithmetic op erations on numb ers by appropri

ate convolutions of probability distribution functions of random variables However

when one do es this the phenomenon of dep endency error o ccurs This is caused

by sto chastic dep endencies arising b etween intermediate results of a calculation In

order to avoid errors in the calculations one has to take cognizance of these dep en

dencies when they exist If as seems to b e the case practically with probabilistic

arithmetic one can only determine whether or not two quantities are indep endent

and not any measure of dep endence if they are not then one has to assume the

worst and calculate dep endency b ounds

The rest of this chapter is organised as follows Section contains all the

preliminary information we require in order to prove our main result in section

We formally dene the notions of dep endency b ounds copulas tnorms the and

op erations and T conjugate transforms as well as discussing the representation

of Archimedean tnorms and the characterisation of asso ciative copulas Section

is devoted to the pro of of our main result theorem and section gives an

explicit formula for ldbS for a sp ecial case This enables us to examine the rate of

convergence for theorem In section we present some examples illustrating

the results of sections and These examples are calculated using the numerical

representations and algorithms develop ed in chapter Finally section draws

some general conclusions from the results of this chapter

Denitions and Other Preliminaries

Wenowintro duce the notation and results needed to prove our result in section

in sucient detail to make this chapter self contained The general references for

this section are We will often write inequalities b etween two functions

F and G as FG Thisistobeinterpreted as meaning F x Gx for all x in

the common domains of F and GAconvex function f dened on some set A is

one which satises

f x x f x f x

for all x x A and all If the ab ove inequalityisreversed then f is

concaveWe write i for if and only if

Distribution Functions

The distribution function of a random variable X is denoted df X and is given by

df X F x P fXxg

and is a left continuous function from onto I I The set of all distribution

functions is denoted The subset dened by

fF j F g

is the set of distribution functions of random variables that are almost surely p ositive

The support of a distribution function is dened by

supp F u

F F

where

inf fxj F x g

and

u supfxj F x g

Three subsets of are dened by

fF j g

L F

fF j u g

U F

and

LU L U

We also have

fF j u g

fF j g

and

LU L U

The step function isdenedby

x a

Triangular Norms

A tnorm triangular norm T is a two place function T I I I which is sym

metric asso ciative nondecreasing in each place and has as a unit ie T a a

T a a a I An Archimedean tnorm is one which satises T a a a a

A strict tnorm is one which is continuous on I and is strictly increasing in

each place on All tnorms T satisfy Z T M where

x x I y

y x y I

Z x y

x y

and

M x y minx y x y I

Two other tnorms we will use are W andgiven by

W x y maxx y

and

x y xy

These four tnorms have the following prop erties

ZisArchimedean but not continuous

WisArchimedean and continuous but not strict

Miscontinuous but neither Archimedean nor strict

iscontinuous strict and Archimedean

Any strict tnorm is Archimedean We dene T by

T fT j T is a continuous tnormg

and T by

T fT TjT is Archimedeang

If T is a tnorm then T dened by

T x y T x y

is known as a tconorm see section of

Representation of Archimedean tnorms

Archimedean tnorms are of sp ecial interest b ecause of the following representation theorems

Theorem A tnorm T is continuous and Archimedean i

T x y f g xg y

T T T

where

g isacontinuous strictly decreasing function from I into with g

T T

f is a continuous function from onto I such that it is strictly decreasing

on g and such that f x x g

T T T

f is a quasiinverse of g seesection of this thesis

T T

The functions f and g are known as the outer and inner additive generators of T

T T

and are unique up to a multiplicative constant If we set h x f log xand

T T

k x expg x we obtain a multiplicative analogue

T T

Theorem A tnorm T is continuous and Archimedean i

T x y h k xk y

T T T

where

k is a continuous strictly increasing function from I into I with k

T T

h is a continuous function from I onto I that is strictly increasing on k

T T

and such that h x x k

T T

h is a quasiinverse of k

T T

The functions h and k are known as the outer and inner multiplicative generators

T T

of T and are unique up to an exp onentiation T is strict i h k and k

T T

For any T T h and k satisfy

A T T

h k x x x I

T T

k h x maxk x x I

T T T

If T W wehave h x max log x and k x e

W W

Copulas

A copula or just copula isamappingC I I such that is the null element

the unit and C is increasing

C a b C a b C a b C a b

for all a a b b I such that a a and b b The set of all copulas is

denoted C All copulas satisfy W C M and are continuous

Copulas link joint distribution functions with their marginals If H I is a

two dimensional joint distribution function corresp onding to the random variables

X and Y ie H x y P fX x Y y g and F df X and G df Y are

the marginals then

H x y C F xGy

for some copula C known as the connecting copula for X and Y The copula

contains the dep endency information relating the random variables X and Y If

C then X and Y are indep endent

The dual of a copula written C is dened by

C x y x y C x y

For any C C the op erations and from onto are dened p ointwise

C C

F Gx sup C F uGv

uv x

and

F Gx inf C F uGv

uv x

These op erations have b een studied b ecause of their prop erties as triangle functions

when C T in the theory of probabilistic metric spaces If C T then

and are asso ciative They are also nondecreasing in each place lemma of

The only additional requirementon C C for C to also b e in T is asso ciativity

In fact under one weak condition C must b e Archimedean

Theorem Let T T C besuchthatT x y minx y x y

Then T T

Proof This follows at once from theorem of and the fact that C M

for all C C

We dene the set of T that satises the condition of theorem as T T C

Since we will restrict ourselves to asso ciative copulas in this chapter it is of

interest to have a probabilistic characterisation of them This is part of problem

of Archimedean copulas have b een studied by Genest and MacKay

who havelooked at sto chastic orderings and sequences of copulas and have used

their results to develop new parameterised families of copulas satisfying certain

conditions Schweizer and Sklar presen t the following two theorems their

theorems and

Theorem A tnorm T is a copula i

T c b T a b c a a b c I with a c

Theorem Let T T then T C i either f or g the additive generators

A T T

of T isconvex

Restricting C to b e asso ciative restricts the typ e of dep endency structure two ran

dom variables X and Y can have Noting that if C is a copula it can b e considered

as the joint distribution function of X and Y where X and Y have uniform marginal

distributions on we can pro ceed as follows If C is Archimedean then

C x y h k xk y

C C C

where is the joint distribution function of two indep endent random variables with

uniform marginals on I Therefore

k xk y P fXk xY k y g

C C C C

P fk X xk Y yg

C C

F x y

where F is the joint distribution function of U and V U h X and V

UV C

h Y Wethus have

C x y h F x y

C UV

Noting that U and V are indep endentbecauseX and Y are wehave shown

that if C is asso ciative then it is an increasing function of a joint distribution of

indep endent random variables This demonstrates the restriction on the typ es of

dep endencies an asso ciative copula can represent but is not a wholly satisfying

characterisation b ecause there seems to b e no natural probabilistic interpretation of

such a dep endence structure

Dep endency Bounds

Wenow formally dene dep endency b ounds and showhow they are related to the

and op erations We rstly consider sums of tworandomvariables and then

C C

examine when we can extend the results to sums of N random variables

Let X and Y be tworandomvariables with distribution functions F df X

and G df Y Let their joint distribution function b e given by H x y

C F xGy where C is the connecting copula for X and Y Then the lower

and upp er dep endency b ounds ldb and udb on df X Y are suchthat

C C

ldb F G df X Y udb F G

C C

W The function C is the lower b ound on the connecting copula for all C C

C The crucial result we need is given by

Theorem

ldb F G F G

C C

and

udb F G F G

C C

and these bounds are the pointwise best possible

Proof See chapter and

An analogue of this theorem holds for op erations other than addition on random

variables See chapter We will b e interested in the case that C T in which

case we will write T instead of C

These dep endency b ounds can b e extended to sums of the form

X S

i N

where fX g is a set of random variables with distribution functions F df X

i i i

and such that for all i j i j N

C T

X X

i j

where C is the connecting copula for the pair of random variables X X

X X i j

i j

In such a case b ecause addition is asso ciative and commutative and b ecause any

T T is also we nd that and are as well and we can write p

C T T

F F df S ldb F F

N N T N

F F

udb F F

T N

N N

The N place functions and are given by

T T

F G F G

N N N

F F F F F F F F

N T N T N N

T T T

and

F G F G

N N N

F F F F F F F F

N T N T N N

T T T

We will generally drop the N sup erscript as no confusion can arise when the

arguments are explicitly stated

The condition is not equivalentto

C T

X X

where C is an N copula and T is an N iterate of a tnorm

X X

N N

T x x T T x x x

N N N

The problem of relating high order copulas to their lower order marginals is still

op en See the problems at the end of chapter of It is known that all

N copulas C satisfy

N N

W C M

N N N

that and M are copulas but that W is not p The fact that

W is not a copula for N can b e understo o d simply as follows For N if

C W is a connecting copula for X and Y then X and Y are decreasing functions

of each other However it is imp ossible given three or more variables X X

for each X to b e a decreasing function of all of the others Nevertheless the lower

b ound in cannot b e improved

Tconjugate Transforms

Before intro ducing T conjugate transforms we note how T can b e represented in

terms of h and k

T T

Theorem Let T T and F Dene k F by

A T

k F x

k F x x

Then for al l F and G in both F G and k F k G and in

T T T

fact

F Gh k F k G

T T T T

This follows directly from theorem

Denition Let T T and F Then the T conjugate transform of F

denoted C F is denedby

C F z sup e k F x z

T T

where k F is given by

The study of T conjugate transforms is due to Moynihan who extended

the Pro dconjugate transform C F by using theorem Note that C F

C k F The C transform has a numb er of prop erties in relation to analogous

T T T

to the Laplace transforms prop erties with resp ect to ordinary convolutions The

essential one is

Theorem Theorem of Let T T and let F G Then

C F Gz maxfk C F z C Gz g z

T T T T T

If T is strict becomes

C F G C F C G

T T T T

We also have theorem

Theorem Let

A f j is nonincreasing positive

continuous and logconvex g f g

where z z Then for al l T T if

A f Ajz k z g

T T

then A fC F j F g Thus A A and equality holds i T is strict

T T T

The inverse Tconjugate transform C is dened by

Denition Let T T and let A Then

A T

C x h inf e z x

is normalisedtobe leftcontinuous and C

If F thenwesay F is logconcave if log F is concaveon and that

F is Tlogconcave if k F is logconcave We dene

fF j F is T logconcaveg

The logconcaveenv el ope denoted F ofF is dened by

F x x

F x supfp log F x q log F x j p q log

p q xx and x px qx g x

F F

In other words the graph of log F is the upp er b oundary of the concavehull of the

graph of log F on The Tlogconcave envelope of F denoted F isin

and is given by

F h k F

T T

With this notation we state the following prop erties of conjugate transforms T

logconcaveenvelop es and functions whichwe shall require later Pro ofs can b e

found in

Theorem For al l T T and al l F G

C F F C

T T

F F and F F i F

T T

F G i C F C G

T T

C C F C G F G

T T T

T T

F G F G with equality i T is strict

T T

GxF ax a x C Gz C F za z

T T

C F lim k F x

T x T

T T

F G F G

T T

In order to prove our limit results we also need the following theorem theorem

of which follows by induction from theorem

Theorem Let T T and F i N Then for al l z

A i

C F F z max k C F z

T T N T T i

Finally we dene the notion of weak convergence in order to state theorem

b elow

Denition Let fF g and F bein Then we say fF g converges weakly

i i

to F and write

F F

if F x F x at each continuity point x of F

This top ology is in fact metrizable see section of The relationship b e

tween weak convergence of probability distributions and convergence of Tconjugate

transforms is given by theorem of

Theorem Let T T and let fF g beasequenceofF Then for

A i i

F F i C F z C F z z

T i T

Convergence of Dep endency Bounds of Normalised Sums

This section is devoted to a pro of of our main result whichis

Theorem Let fX g beasequenceofrandom variables with distribution func

tions F df X and let T T Then

i i LU C

ldb X

T i

u and u is as in as N where lim

F F N

i i

i N

This theorem has the following two corollaries

Corollary Let fX g fF g and T beasabove Then

i i

udb X

T i

and is as in Note that in as N where lim

F F N

i i

the ab ovetwo results if u u and for all ithen u and

F F F F F F

i i

Corollary Let fX g bea sequenceofrandom variables with distributions

and let T T Then F df X

C i i

X ldb

a i T

and

udb X

T i b

N N

Q Q

N N

u and b lim as N where a lim

F F N N

i i i i

In order to prove theorem weprove the following lemmata The pro of of

the theorem follows from their conjunction

Lemma Let F and let T T Then there exists a z such that

A u

for al l z z

u z

C F z e

where u is given by

Proof Recall that

C F z sup e k F x z

T T

However k is strictly increasing and continuous and so k F is increasing Also

T T

k and so k x for x Therefore

T T

x u

k F x

x u

Now since the slop e of e can always b e made as close to zero as desired by making

for small enough z the supremum z smaller it can b e seen that for any F

in must o ccur at x u see gure If for a given z the supremum

F u

o ccurs at x u thenobviously it will o ccur at x u for all zz

F F u Figure Illustration for the pro of of lemma

Lemma Let z maxfk e g for z where k is

T T

the inner multiplicative generator of some T T Then the inverse transform of

is given by

C x x x

Proof Weknow theorem vithat C z maxfk e g for

T T

all z We also know theorem part that for all T T and for

C F F The lemma then follows directly from the fact that all F C

Lemma Let T T and let fX g beasequenceofrandom variables with

C i

distribution functions F df X where F for i Then

i i i

z maxfk e g X ldb lim C

T i T T

u where lim

F N

i i

Proof Theorem and the discussion following it tells us that ldb X

i i

F F where F df X If weset X X N then obviously

T i

N i i i

F xF Nxfor i and for N Wealsoknow theorem that

C F F z max k C F z

T T T T

N i

and that C F z C F zN theorem part Thus

T T i

N N

X Y

X z max k C F zN C ldb

i T T i T T

i i

However since F lemma tells us that

C F xexpu x for x z

T i F

for i N where z is the z of lemma for a given F If welet

u i

z min fz g we can then write

u i

N N

X Y

z C ldb X z max k zN exp u

u T T i T F

i i

Now for any z there exists an integer N such that for all NN zN

m m

z Therefore for all z holds for suciently large N Ifwenowobserve

that

N N

Y X

exp u zN exp u z e

F F

i i

i i the pro of is completed

T T

Lemma Let H ldb S be the Tlogconcave envelopeofldb S

T N T N

where S X T T and fX g is a sequenceofrandom variables with

N i C i

distribution functions F df X Then

i i

as N and is as in lemma

Proof This follows directly from theorem lemma lemma and

theorem part

Lemma Let S T and be as in lemma Then

lim sup supp ldb S

T N

Proof This follows by induction from the case when N Thus wenow prove

that

sup supp ldb X X u u

T F F

Theorem tells us that

ldb X X sup T F uF v

uv x

Since T is an Archimedean tnorm and a copula weknowthat T M and that

T a b implies a b Therefore the minimum x suchthat F uF v

isx u u

F F

T T

Lemma Let H be such that H ldb S as in lemma

T N

Then

T T T

j G H gThen Proof Let H fG

T T

G H H G

T T

theorem part That is H is the maximal elementin H The only

is H H that satises and H

Lemma Let fF g beasequence of distribution functions in and let T

i L

T Then if min f g

A i F

x N F F F x F

T N T

where F x F x and F for i N

i i

Proof For N the result follows directly from the denition of The result

for general N then follows by induction using the iterative construction of given

Proof of theorem Immediate from lemmas and

Proof of corollary We rstly prove a restricted version of corollary

by a metho d whichgives an understanding of whyitworks in a case of sp ecial

interest T W

Corollary Let fX g and fF g beasincorol lary and let T T be such

i i A

that T T see and Then

X udb

i T

as N and lim

F N

Proof We provethisby expressing udb in terms of the lower dep endency b ounds

of some transformed random variables Again we consider N and the general

case follows by induction Let X X Then F x F x Observe that

i i

v uF x sup T F F F

uv x

sup T F u F v

uv x

sup T F uF v

uv x

inf T F uF v

uv x

inf T F uF v

uv x

F F x

udb X X x

Writing this the other way around wehave

x X udb X X x ldb X

T T

will b e reversed and and we can apply theorem noting that the roles of u

F F

i i

The question of when T T has in fact b een solved byFrank who sho wed

that apart from T W and T the only T T satisfying T T are given

x y

s s

T x y log log s

for s s In fact T T

s C

The more general corollary is proved in a dierent manner

Proof of corollary Corollary establishes the result for T W

We also know theorem that for an y copulas C and C suchthat C C

That is

C C

F G F G F G

C C

Since all copulas C satisfy W C wehave

C W

Combining with the fact to b e proved b elow that inf fxj xg

for all C T completes the pro of

In order to see that

inf fxj F F x g

C N

for all F F and any C T we again use induction on N and start

N L C

with N Consider then

F F x inf C F uF v

uv x

d d d

Now C has as a null element That is C x C x for all x I

Therefore in order for C x y to equal one it is necessary only that either x or y

equal one Thus for C F uF v to equal one either F u orF v The

smallest x such that u v x and either F uorF v equals one is x

F F

The rest of the pro of follows by induction noting that wehavetotake accountof

the N term

Proof of corollarySimply let X log X apply the results of theorem

and corollary and exp onentiate the result

Rates of Convergence

Theorem says that ldb S Wenow examine how fast the convergence

T N

is The main to ol we use is theorem of whic hwe state b elow as theorem

Since if T T is not strict then is not closed under page

A T

we dene the set B which is Firstly dene k F by

T T

k F x

k F x x

T F

where is given by Then dene

B fF j k F is logconcaveg

T T

If T is strict B Otherwise B Nowifforany andF B with

T T T

T T

F we dene F by

F x h k F x

T T

and let F we can then state

Theorem

F F F

Theorem Let T T and let fX g beasequenceofrandom variables with

C i

identical distribution functions df X F B Then

i T

h i

ldb k F x x h X

T T T i

is taken Proof The result follows immediately from theorem when the

account of in the manner of the pro of of lemma

A particularly interesting and imp ortant sp ecial case of theorem o ccurs

when T W In this case wehave k x e and so

B fF j F is concaveon g

W F

and

k F x

F x

e x

Recalling that

x e

h x

log x e x

wehave

h i

N N

k F x for k x e h

W w W

The condition here is equivalentto

F x N

e e

If we assume that F has an inverse F then implies

x F

In other words with fX g as in theorem

ldb x for x F X

W i

N N

If xF then h x log x and so

h i

F x

X N F x e x log ldb

i W

Again assuming that F has an inverse we can write

y F X ldb

N N

The convergence to is apparent from up on setting y and y In

these cases wehave

ldb F X

and

F X ldb

N N

Equations and tell us that the rate of convergence is O N

Another interesting sp ecial case is when T in which case

ldb X xF x

which is a particularly simple result The condition T is equivalent to Lehman

ns positive quadrant dependence Tworandomvariables X and Y are p osi

tively quadrant dep endentif

F x y F xF y

XY X Y

Positive quadrant dep endence has b een studied and compared with other measures

of dep endence in

T T

If F B we can use the fact theorem part that F G F G

T T T

for any T T to b ound the rate of convergence For example using wecan

say that for any F

X X

x F sup xj ldb

This b ehaviour can b e seen in gure presented in the next section Rates of

convergence for udb are similar and follow directly using the arguments used to

prove corollaries and

Examples

We will now present some examples illustrating the results of sections and

We restrict ourselves to the case T W and use the algorithms develop ed in chapter

for numerically calculating F G and F G when F and G are represented

W W

by discrete approximations Wecheck the accuracy of the results so obtained in

gure where we compare the W obtained using the numerical approximations with that obtained using

X with N Figure Lower and upp er dep endency b ounds for S

i N

X with N Figure Lower and upp er dep endency b ounds for S

i N

Figures and show the lower and upp er dep endency b ounds for S

X In this case all the X are identically distributed Their distribution is

i i

presented as the two central curves in gures and F and F The reason why

there are twocurves is b ecause they are the output of a condence interval estimation

pro cedure designed to generate the lower and upp er discrete approximations to

probability distribution functions develop ed in chapter In this case we estimated

the distribution of a p opulation consisting of a mixture of samples from N and

N distributions In b oth cases the normal distributions were curtailed at

Figure shows ldb S and udb S for N This was generated

W N W N

by iteratively calculating

ldb F F F

i i

F udb F F

i W i

In order to sp eed up our view of the convergence in gure we used the iteration

ldb F F F

i i i

F udb F F

i W i i

Equation has the eect of doubling N at each iteration Thus gure shows

ldb S and udb S forN It can b e seen that cases for

W N W N

N and are identical to those in gure The convergence of ldb S

W N

and udb S to and is apparent

W N

Figures and are all related to S where df X F for all i and

N i

F is an upp er half Gaussian distribution That is F x x where

is the distribution of a N random variable In this case F is curtailed at

Again we represent F bylower and upp er discrete approximations this

time touching each other and these can b e seen as the central curves in gure

Recalling the symmetry relationship b etween ldb and udb see corollary

W W

we will b e able to observe the eect of b oth a concave and convex F Thelower

and upp er dep endency b ounds for F being a lower half Gaussian distribution can b e

seen by viewing gures upside down and changing the axes appropriately

Figure shows the dep endency b ounds calculated using equation As can

b e seen from gure and by comparing gures and the results agree

very closely with those obtained using the numerical approximations The only

dierence is due to the chording eect apparent in gure This is an artifact

of the numerical representation we use in We approximate F by uniformly

discretising its quantiles Considering the eect of it can b e seen that the

top most discrete levels are stretched down in successive iterations to result in the

straight line segments which app ear in gure The b ounds obtained numerically

are still correct but they are not as tight as the true dep endency b ounds Once

there are straight line segments they remain present in successive iterations This

can b e understo o d in terms of Alsinas result that

U U U

W ab cd minadbcbd

and

U U U

W ab cd

ac maxadbc

Figure Dep endency b ounds for S where df X is upp er half Gaussian

N i

Figure Dep endency b ounds for the same problem as gure but calculated using

Figure Comparison b etween the exact and numerical calculation of the dep endency b ounds gures and

where U is the uniform distribution function on a b

Observe that udb converges to more rapidly than ldb converges to or

W W

equivalently ldb converges to more rapidly for lower half Gaussian F than it

do es for upp er half Gaussian F This is due to the concavity or convexity of F

Lo oking at gure upside down demonstrates the eect mentioned at the end of

section Line misF for F b eing lower half Gaussian and it can b e seen that

X X

sup xj ldb x F

Conclusions

Apart from b eing mathematically interesting the results presentedinthischapter

have a useful practical interpretation Theorem says that in order to determine

dep endency b ounds for the sum of a large numb er of random variables it is only

necessary to consider the values of supp F u In other words dep endency

F F

b ounds reduce to Minkowski sums or pro ducts of intervals and the metho ds of

interval arithmetic could b e used for their calculation The signif

icant p oint is that the shap es of the F within supp F have no eect on the nal

i i

result

Observe that this situtation is quite dierent to the classical central limit theorem

and law of large numb ers for sums where we are lo oking at ordinary convolutions

rather than dep endency b ounds In this case the supp ort of the comp o

nent distributions is irrelevant to the nal limiting distribution apart from some

nondegeneracy conditions Note also that this result has a b earing on statistical

inference It is often considered adequate to t a distribution to some data in such

a manner that the t is close over the central part of the distribution When the

densities are plotted in the usual manner anomalies in the tails do not showup

However as Bagnold has shown p opulations that app ear to have normal dis

tributions when plotted in the usual manner are quite apparently nonnormal in the

tails when a logarithmic vertical scale is used Determination of the tail b ehaviour

is of course the domain of the theory of statistics of extremes and has

signicant practical application to the study of rare events

The results presented in this chapter can also b e applied to fuzzy numb ers In

this case a general law of large numb ers under a general tnorm extension principle

is obtained see the following chapter

Chapter

A Law of Large Numb ers for

Fuzzy Variables

The use of fuzzy numb ers for calculation with imprecisely known quantities has

b een advo cated byanumb er of dierent authors Fuzzy numb ers are combined us

ing extended arithmetic op erations develop ed using the extension principle When

a general tnorm is used for the intersection op erator a general tnorm extension

principle is obtained The purp ose of this chapter is to present a result for the lawof

large numb ers for fuzzy variables when using this general tnorm extension principle

The result is that convergence to a crisp set is obtained for all Archimedean tnorm

intersection op erators This generalises a previous result of Badard who conjectured

that something along the lines of that presented here would b e true The result is

proved by deriving it from a similar result for the lawoflargenumb ers for dep en

dency b ounds Dep endency b ounds arise in probabilistic arithmetic when nothing

is known of the joint distribution of tworandomvariables apart from the marginal

distributions The bridge used to connect dep endency b ounds with fuzzy number

op erations under the tnorm extension principle can b e used to give a probabilistic

interpretation of fuzzy number combination Some remarks on this interconnection

are made in the nal section of the chapter

This chapter after undergoing some revisions will b e resubmitted to Fuzzy Sets

and Systems under the title The LawofLargeNumb ers for Fuzzy Variables under

a General Triangular Norm Extension Principle

Intro duction

Fuzzy numbers or variables are develop ed from the theory of fuzzy sets by using the

extension principle Their prop erties and metho ds of calculating

with them have b een studied byanumb er of authors A more general extension

principle mak es use of a general tnorm intersection op erator When

this is used the sum of two normal fuzzy variables X and Y with memb ership

functions and is given by

X Y

z sup T x y

Z X Y

xy z

where T is some triangular norm or tnorm A natural question to ask is What

is the limiting b ehaviour of

Z X

where fX g are fuzzy variables and we use the supT convolution to calculate

the memb ership function of their sum This is the fuzzy analogue of the law of large

numb ers for random variables

In this pap er wewillshow that for nearly all tnorm intersection op erators the

memb ership function of Z approaches that of a crisp set or interval This result

generalises some sp ecial cases rep orted by Badard and Rao and Rashed

The result was also mentioned by Dub ois and Prade in p The result in

the present pap er is a law of large numb ers for fuzzy variables not fuzzy random

variables whic h are random variables that take on fuzzy values

Weprove the result in three sep erate ways each of whichgives a dierent insight

into the problem The rst is a short and simple direct pro of The second uses a

recently develop ed theorem on a related question for dep endency b ounds of sums of

random variables Wethus incidentally provide a probabilistic interpretation

of the supT convolution used for fuzzy numb er addition A third pro of

only sketched here shows how T conjugate transforms can b e used in the study

of op erations of the form These transforms play a role analogous to that of

the Fourier transform in probability theory

Some numerical examples are presented which illustrate the convergence of mem

b ership functions in calculating These examples are of indep endentinterest

b ecause they are the rst exact calculation of op erations on fuzzy numb ers not

just addition for tnorms other than min Dub ois and Prade worked in terms

of their simple LR representation in and their form ulae are necessarily only

approximate for op erations other than addition or subtraction Wegive details on

the new metho ds used to calculate our examples

The rest of this pap er is organised as follows Section presents the main theorem

and the simple direct pro of Section briey intro duces the notion of dep endency

b ounds This is necessary for the understanding of theorem which is used in the

second derivation of theorem Section is devoted to this derivation and in section

we outline a dierent approachtoproving theorem byintro ducing the bilateral

T conjugate transform Section intro duces a means of numerically calculating

the supT convolutions and some examples using this metho d are presented in sec

tion These examples graphically illustrate the convergence of theorem Finally

in section wemention some interpretation issues raised by the bridge b etween

probabilistic dep endency b ounds and fuzzy variable convolutions

Theorem and Direct Pro of

The fuzzy variables we will b e concerned with are normal and Tnoninteractive A

normal fuzzy number X has a memb ership function such that sup x

X X

If fX g is a set of fuzzy variables with normalised unimo dal continuous mem

b ership functions f g and joint memb ership function thenfX g are T

X X X i

noninteractiveif

x x T x x

X X N X X N

N N

where T is the N th serial iterate of some tnorm T page dened

N N

by T x x T T x x x with T T The notion of T

N N N

noninteractivity is called indep endence by Badard and w eak noninteraction by

Dub ois and Prade T noninteractivity is a generalisation of Rao and Rasheds

minrelatedness

Whilst noninteractivity is often assumed in discussions in the literature the study

of interactive variables has only b egun recently Without clear

semantics for the notion of interaction of fuzzy variables there is little incentive for

pursuing it further

The tnorms we will b e concerned with are Archimedean An Archimedean t

norm is one which satises T a a aa Apart from min nearly all of the

tnorms suggested in the fuzzy set literature are Arc himedean

Theorem Let fX g beasetofT noninteractive fuzzy variables with mem

bership functions i N Assume that T is Archimedean Let

inffxj x g

X X

i i

sup fxj xg

X X

i i

and let

X Z

have a membership function Let and be dened analogously to and

Z Z Z X

Then

lim

Z X

lim

Z X

and

Z Z

as N The symbol in means weak convergenceconvergenceat

every continuity point and denotes the indicator function of the interval a b

denedby

x a b

otherwise

This theorem says that only the values of and haveany eect on the limiting

X X

i i

memb ership function which is that of a crisp nonfuzzy set If for all

X X

i i

i then none of the fuzzy variables have a at in their memb ership functions and

converges to a single delta function

In other words the shap e preservation prop erty of supmin convolutions is a sp ecial

case min is not Archimedean

Proof Using the fact that T is Archimedean and the relationship T b

T b b b whic h holds for all tnorms it can b e seen that z

sup T x y only if there exists x and y such that x y z

X Y

xy z

and x y Up on noting that all Archimedean tnorms are strictly

X Y

less than min then one can use the fact that supmin convolution has the shap e

T M T

z z for all z such that z preservation prop erty to argue that

Z Z Z

Since is an asso ciative op eration inductive application of the ab ove

argument completes the pro of

Dep endency Bounds

This section provides the minimum background necessary to understand the state

ment of theorem and the discussion in section Further details can b e found in

A copula is a function that links multivariate distribution functions to their

marginals It suces to consider the bivariate case If H x y is the joint distri

bution of two random variables X and Y dened on a common probability space

then H x y C F xF y where F xdf X PrfXxg and

XY X Y X

F y df Y PrfYyg are the marginal distributions of X and Y resp ectively

and C is their connecting copula In other words C x y H F xF y

XY XY

X Y

All copulas are increasing C a b C a b C a b C a b for all

a a b b I and satisfy W C M where W x y maxx y and

M x y minx y If C x y x y xy then X and Y are sto chastically

indep endent Copulas are tnorms if they are asso ciative and tnorms are copulas if

they are increasing

Dependency bounds are lo wer and upp er b ounds on the distribution of

functions of random variables when only the marginal distributions are known A

more general form of dep endency b ounds considered in pro vides tighter b ounds

when some lower b ound C W on the connecting copula is known We will

assume that C is a tnorm and use the traditional T to represent it If T W

then we obtain the b ounds of We write the dep endency b ounds as

ldb X Y df X Y udb X Y

T T

where ldb df and udb refer to lower dep endency b ound distribution function

T T

and upp er dep endency b ound resp ectively The main result weneedis

ldb X Y z F F z sup T F uF v

T T X Y X Y

uv z

and

udb X Y z F F z inf T F uF v

T T X Y X Y

uv z

where F df X F df Y and T is the dual copula given by

X Y

T x y x y T x y

This should not b e confused with the tconorm S dened by

S x y T x y

Since the and op erations are asso ciative we can iteratively calculate

T T

ldb X X and udb X X Theorem presented b elow charac

T N T N

terises the b ehaviour of these quantities as N

Theorem Let fY g beasequenceofrandom variables with distribution

functions F df Y such that u isabounded closed interval for i N

i i F F

i i

where inf fxj F x g and u sup fxj F x g Also let T bean

F i F i

i i

Archimedean tnorm and copula Then

N N

X X

w w

and udb Y Y ldb

T i i T

N N

i i

P P

N N

as N where lim u lim and is the

N F N F

i i i i

N N

unit step distribution function at

Proof See

Pro of of Theorem via Decomp osition

Theorem can also b e proved by representing the memb ership functions by

two probability distributions and then applying theorem which is in terms of

probability distributions

We decomp ose the memb ership function of some fuzzy variable X as follows

Let

x x

X X

x X

and

x x

X X

where and are given by and Wewillwork with the related

X X

I D

pair where

X X

I i

x x

X X

but

D d

x x

X X

I D

Note that and are b oth distribution functions continuous nondecreasing

X X

with range

I D I I D D

Wenow need to calculate and in terms of the pairs and

Z Z X Y X Y

resp ectively The key observation to make is that up on examining and noting

that T a T aa for all a we nd that

and

Z X Y Z X Y

Thus wehave

I i i i i i

z z z sup T x y

Z Z X Y X Y

xy z

and

d d d

z sup T x y

Z Z Z

xy z

Substituting into gives

D D D

z sup T x y

Z X Y

xy z

Therefore

D D D

y x z sup T

Y X Z

xy z

D D

y x inf S

Y X

xy z

If in fact S T we obtain

D D D

Z X Y

Therefore we can use and and theorem to prove theorem for all

T such that S T This condition can b e removed by using an argument along

the lines of that used in to pro ve the udb part of theorem for general

Archimedean T One uses the fact that T is increasing in b oth places and a b ound

R on T such that the corresp onding tconorm of R equals its dual copula R

We note in passing that one could also examine the rate of convergence in theorem

by using the results in section of The details are omitted here Note also

that the ab ove decomp osition is of course unnecessary when the notion of a fuzzy

numb er adopted byHohle and others is used

Triangular norms Generators

g x x

W x y maxx y

g x max x

g x log x

x y xy

x expx g

g x x

p Sch

p p

maxx y p

Sch

x y T p

y g x

Sch

x y p

x x g

Yag

Yag p

p p

T x y min x y p

y g x

Yag

h i

i h

g x log

Fra

x y

p p

Fra

h i

T x y log

x log g

Fra

Table Some Archimedean tnorms and their additive generators

A Direct Pro of Using Bilateral Tconjugate Transforms

Theorem was proved in b y using the prop erties of T conjugate transforms

By extending the denition of these transforms it is p ossible to construct

an alternative pro of of theorem with no reference to dep endency b ounds Most

of the details would b e the same as for the dep endency b ound case and they are

omitted here Nevertheless it seems worth mentioning the extension and its p ossible

applications We b egin with the representation of Archimedean tnorms These are

also used in section

A continuous Archimedean tnorm T can always b e written theorem

T x y f g xg y

T T T

where

g an inner additive generator of T is a continuous and strictly decreasing

function from I in to fxj x and x g with g

f the outer additive generator of T isacontinuous function from onto

I that is strictly decreasing on g and suc hthat f x for all x

T T

f and g are quasiinverses of each other and unique up to a multiplicative

T T

constant

If T is strict continuous and strictly increasing in each place on then f

g and g Some examples of tnorms and their additive generators have

b een recently collected by Mizumoto and a selection are giv en in table Note

the following sp ecial cases of the parameterised tnorms Schweizer Frank

Yager

Sch Yag

Fra

W T T limT

Sch

Fra

T limT

Fra

Sch Yag

M lim T lim T T

p p

There is also a multiplicative representation A continuous Archimedean tnorm

T can always b e written

T x y h k xk y

T T T

where

k the inner multiplicative generator isacontinuous strictly increasing func

tion from I into I with k

h the outer multiplicative generator isacontinuous function from I onto I

that is strictly increasing on k and suc h that h x x k

T T T

h is a quasiinverse of k and they are b oth unique up to an exp onentiation

T T

The Tconjugate transform of a probability distribution function F is dened

C F z sup e k F x z

T T

where

k F x

k F x x

k is an inner multiplicative generator of the Archimedean tnorm T and it is

assumed that F There is an inverse transform given by

C x h inf e z x

where h is an outer multiplicative generator of T The signicance of these trans

forms for the study of the convolutions is that the following prop erty is satised

T T

C C F C G F G

T T T

where F is the Tlogconcave envelope of F This means that for certain

classes of F and G it is p ossible to calculate convolutions by p ointwise multipli

cation of the T conjugate transforms followed byaninverse T conjugate transform

This is analogous to the use of the Fourier transform or characteristic function

in probability theory The T conjugate transform is in fact closely related

to Fenchels duality theorem In fact the duality result of Frank and

Schweizer describ ed in the following section is yet another manifestation of Fenchels

theorem

In order to apply T conjugate transforms to the addition of fuzzy variables it is

necessary to extend their domain of denition from probability distributions non

decreasing to more general unimo dal functions The obvious idea of applying the

I D

transform to the two parts and separately turns out to b e equivalenttothe

X X

simple replacement of the condition z in by the condition z We

thus obtain the bilateral Tconjugate transform

C z sup e k x z

T X T X

This is an analogue of the Fourier transform for fuzzy variable addition

Numerical Calculation of supT Convolutions

In order to compute some examples which illustrate the convergence describ ed by

theorem it is necessary to b e able to calculate the supT convolutions

accurately and easily In this section we will present a new metho d for p erforming

this calculation The numerical representation is very similar to that develop ed

in for calculating con volutions and dep endency b ounds when T W The

metho d presented here is for quite general Archimedean T and determines the result

in terms of the inner additive generator g of T

As well as enabling an illustration of theorem to b e calculated the metho d

seems to b e the rst to allow the calculation of Whilst some simple prop

erties of were presented by Dub ois and Prade in terms of LR fuzzy

numb ers these were only for the sp ecial cases of T W and M Incontrast

the present metho d allows the use of arbitrary parameterized Archimedean tnorms

Furthermore op erations other than addition are handled just as easily

We will consider the increasing and decreasing parts of separately

Increasing Part

The main to ol we use is the duality theorem of Frank and Schweizer In order

to do so consider the more general form of dened by

F Gx sup T F uGv

Luv x

where F and G are nondecreasing left continuous functions from onto I

f g and I and L is a binary op eration In other words F and G

are distribution functions We denote the quasiinverse of F by F where

F y supfxj F x yg

Note that if F is continuous F F Theinverse distribution function is known

as the quantile distribution in probability theory and of course corresp onds to the

levelsets or cuts of fuzzy set theoryFrank and Schweizer haveshown that

F G x inf LF uG v

T uv x

When T M the inmum in will o ccur at u v x and reduces to

the well known result

F G x LF xG x

This means that interval arithmetic on the level sets can b e used to calculate the sup

min convolution of fuzzy numb ers Note that Hohle has giv en a generalistion

of when T M to completely distributive complete lattices

If wenow observe that anyArchimedean tnorm can b e decomp osed in terms of

its additive generators see the previous section wecandevelop a simple wayto

calculate supT convolutions in terms of level sets Firstly we rewrite the condition

for the inmum in as

f g u g v x

T T T

for a given Archimedean T Assuming that T is strict and thus that f g we

obtain

g ug v g x

T T T

g x g v u g

T T

and v is constrained to range over x g g x g x Therefore

T T

can b e rewritten as

F G x inf LF g g x g v G v

T T

TL T

v x

Decreasing Part

Considering the decreasing part wehave

d d d

z sup T x y

Z X Y

Lxy z

and so

D D D

y x z sup T

Y X Z

Lxy z

D D D

z sup T x y

Z X Y

Lxy z

Let S x y T x y bethetconorm corresp onding to T We can thus

write

D D D

y x z inf S

Y X Z

Lxy z

Frank and Schweizer ha ve shown that if

F Gx inf S F uGv

Luv x

then

F G x sup LF uG v

S uv x

A tconorm S derived from an Archimedean tnorm T can b e decomp osed by

S x y f g xg y

T T T

where f f x and g x g x The condition S u v x in

T T

can b e rewritten as

g ug v x S u v g

T T

g u g v g x

T T T

u g g x g v

T T

Therefore

F G x sup LF g g x g v G v

T T

SL T

v z

Using and and changing back to the fuzzy set notation wecan

write

I I I

z inf L g g z g v v

T T

Z X T Y

v z

and

D D D

z sup L g g z g v v

T T

Z X T Y

v z

Formulae and provide a general and fast way of calculating supT

convolutions for arbitrary Archimedean tnorms in terms of the level sets of the

memb ership functions of the fuzzy variables in question Note that for op erations

I I

such as subtraction and division the formula for will b e in terms of and

Z X

D D D I

and that for will b e in terms of and Subtraction is thoughtof

Y Z X Y

as negation of the right argumentfollowed by addition

Discretisation

In order to calculate and numericallywe discretise and thus

X Y

We will set i for i P ie P points In fact we

X X

will adopt an approach similar to that used in where w eusedlower and upp er

approximations If is known exactlythen Wethus have

i i i

x x

P P P

i i i

x x

P P P

for x and i PTheadvantage of this representation is that directed

rounding can b e used in order to ensure that the true result is contained within the

z z This can b e seen in equations and b elow interval

We dene the array g by

g i g iP i P

and can then write the formulae

I I I

i inf L hP g g i g j i j

Z X T T Y

j iP

and

D D D

j P j i P i g g hP g i sup L

Y T T T X Z

j i

for i Pwhere

hi bc for

hi de for

I I D D

and the and are lower or upp er approximations as appropriate

X Y X Y

this dep ends on L Note that the rounding is down wards for the upp er approxi

mations b ecause we are dealing with the inverse memb ership function or the level

sets Some examples calculated using and are presented b elow

Chapter

The Inverse and Determinant of a

Uniformly Distributed

Random Matrix

Intro duction

This chapter presents two results on matrices with iid indep endent identically

distributed uniformly distributed elements Theorem gives an expression for the

density of the determinant of a matrix whose elements are iid uniformly on

Theorem gives an expression for the density of the elements of the inverse of this

matrix Both of these results are derived in a straightforward manner but they

do not seem to have app eared in the literature previously The only hint of sucha

result which the author has found is a single sentence on page of the b o ok by

Prohorov and Rozanov

The distribution of the determinant of a random matrixis kno wn

only in two cases if the column vectorsare uniformly distributed on

dimensional unit sphere of the dimensional space and if the column

vectorsare normally distributed with v anishing mean vector and non

degenerate correlation matrix

We make use of the convolution relations for the dierence pro duct and quo

tientoftworandomvariables The original motivation for the results derived

here was to construct an explicit example to show the misleading nature of Szulcs

denition of almost ev erywhere nonsingular matrices see also This

chapter has b een published in Statistics and Probability Letters

under the same title as this chapter

Results

Theorem Let

a a

bea matrix with elements a i j where a areiidrandom variables

ij ij

with density

f x

elsewhere

and let D det AIff x is the probability density of D then

x log x x log x

k i i

k x x

x log x x

f x

i i

f x x

elsewhere

Proof Let p a a q a a and so D p q Obviously p and q are iid

with density

f zxf x dx f z f z

x a q p

jxj

log z z

Now

f w xf w dw f x

p q D

log w xlogw dw x

log w x log w dw x

elsewhere

The two cases are symmetricf xf x and so we will only consider x

D D

Let

I w log w x log w dw x

Integration by parts twice gives

I w w log w w xlogw xlogw x K w

x x

where

log w x

K w dw

x w

Figure The probability density f

and cannot b e expressed as a nite combination of elementary functions eq

We can determine a series expansion though

log w x w x

w x

w wk

This can b e integrated termwise within the region of convergence and so

k k

w x

dw K w

k wk

where the integral can only b e over a range suchthat w x Using a

binomial expansion of the integrand and then integrating termwise we obtain

k i

X X

k w

k k i

K w x log w x

k i i

Evaluating I w j gives A graph of f x calculated using only the

x D

w x

rst terms of the innite series is shown in gure

Theorem Let A be as in theorem and let

a a

p q

a a a a a a a a

a a

r s

a a a a a a a a

Then p q r and s are identical ly distributed although not independent with com

mon density function f x given by

logx

x x x x

f x

x log x

Proof The fact that p q r and s are identically distributed is obvious due to the

interchangability of the a they are iid We rewrite the expression for s as

a a c

a b b e g

a d

and then successively determine the densities of c e g ands We already know

from the pro of of theorem that

log x x

f x

elsewhere

The densities of e and g are determined by the convolution relations for quotient

and dierence see We obtain

log x

f x

and

x x

x x log x

f x

x x log x

The distribution of s g is given by

f x f x

g s

Substituting into gives A graph of f x is presented in gure

Figure The probability density f

Discussion

Observe that lim f jxj and so A is almost surely nonsingular in the

jxj

conventional measure theoretic sense The matrix A would not b e considered to b e

almost everywhere nonsingular using Szulcs denition though Szulc is

concerned with interval matrices and denes an interval matrix A to b e almost

everywhere nonsingular if there exists only a nite numb er of real singular matrices

contained in A The interpretation of an interval matrix as b eing a random matrix

with uniformly distributed elements seems p erfectly natural and so Szulcs denition

is misleading Komlos has studied the singularit y of random matrices and has

shown that if i j are iid with a nondegenerate distribution then

lim P

n n nn

He also settled the problem of the n umb er of singular matrices there are when

the elements can b e either or

There are very few exact results on the distribution of random determinants A

numb er of authors have derived results in terms of moments for a few sp ecial cases

Nyquist Rice and Riordan considered the determinants

of random matrices with indep endentidentically normally distributed elements with

zero means and derived exact densities for the and cases They also

investigated the moments of the determinant in the case of nonnormally distributed

elements with zero means Nicholson studied the case for indep endent

identically distributed elements with normal distributions having nonzero means

and derived a complicated innite series for the cumulative distribution function

of the determinant Alagar has deriv ed an expression for the density of the

absolute value of the determinantofa matrix with indep endentexponentially

distributed elements It is in terms of x log x He also presented a

complex result for the determinant in terms of Gfunctions

Instead of calculating the marginal densities we could calculate the joint density

While this has a fairly simple analytical form and can b e calculated directly for the

sp ecial case weareinterested in or from the general result of Fox and Quirk

it is of limited value Usually one will ultimately b e interested in the densities of

the individual elements and to obtain these from the joint density it is necessary to

p erform multiple integrations with variable limits The joint density itself is hard

to conceptualise it is four dimensional Note that we cannot calculate correlation

co ecients to determine the dep endence of the elements as none of the moments of

f exist s

Chapter

Conclusions

I hate quotationTel l me what you know

Ralph Waldo Emerson

Overview

The main ob jectives of this thesis were to study the idea of probabilistic arithmetic

to identify the ma jor problems and to attempt to provide solutions to some of

these problems It so on b ecame clear that the greatest obstacle in the path to the

development of probabilistic arithmetic is the phenomenon of dep endency error It is

not sucient that all the inputs to a calculation are indep endent since dep endencies

can arise in the course of a calculation b ecause of rep eated o ccurrences of terms

The details of how the probability distributions are represented measured or the

convolutions calculated are of much less signicance

Wehavedevelop ed one approach to the handling of sto chastic dep endencies

in the form of dep endency b ounds These allow the calculation of lower and upp er

b ounds on the distribution of a function of random variables when only the marginal

distributions are known Wehave seen that when some information is known a lower

b ound on their copula greater than W then tighter b ounds can b e determined

However it should b e realised that these b ounds do not provide a complete solution

to the problem of dep endency error in probabilistic arithmetic calculations There

are many other issues including the p ossible rearrangement of expressions in order

to remove rep eated terms whichhave greater p otential value in this regard

Probabilistic arithmetic is similar in its motivations and problems to interval

arithmetic In fact interval arithmetic can b e considered as a sp ecial case of prob

abilistic arithmetic However interval arithmetic is considerably simpler b ecause

dep endency error manifests itself as dep endency width which simply causes a

wider nal result than could otherwise have b een obtained Compare this with

the situation where probability distributions are involved there is no analogous

simple idea of containment of results In order to achieve a similar situation

with probability distributions wehave to somehow incorp orate the notion of con

tainment one probability distribution containing another This is the advantage

of our lower and upp er b ounds on probability distributions Whilst simple problems

for example determining the distribution of the ro ot of a linear equation by calcu

lating the distribution of the quotientofthetwo co ecients are readily solvable

using the metho ds presented in this thesis more complex problems require further

investigation

Nevertheless there are a numb er of p ositive results to come out of this work

The dep endency b ounds and the numerical metho ds for calculating them seem to

b e of indep endentinterest These b ounds can b e used to determine the robustness

of untestable indep endence assumptions The connexion b etween these dep endency

b ounds and the fuzzy number supT convolutions is also of particular interest Not

only do es this provide via our numerical metho d based on the duality theorem

an ecientway of calculating op erations on fuzzy numb ers but it also gives a very

close link b etween fuzzy set theory and probability theory Whilst the connexion

between the Bo oleFrechet formulae for conjunction and disjunction and the fuzzy

set intersection and union op erations was known b efore the knowledge that this

can b e extended to the case of random and fuzzy variables is new and further

strengthens the argument that to a large extent the fuzzy set theory op erations are

b etter considered in terms of probability theory

Other contributions of this thesis include

A detailed review of metho ds for numerically calculating convolutions of

probability distribution functions

The new convolution algorithm develop ed in chapter Whilst this do es

not solve all the problems of probabilistic arithmetic it is still a useful and

necessary to ol Our metho d is computationally ecient accurate and simple

It is b etter than any of the metho ds discussed in chapter

The extreme limit theorem derived in chapter This shows that when one has

no knowledge of the dep endence structure of a set of random variables almost

any distribution consistent with the constraints imp osed byinterval arithmetic

on their supp orts is p ossible for their weighted sum

The result in chapter on the inverse and determinantofa random matrix

although simple is new and demonstrates the strange results obtainable with

just a few op erations on random variables

Many of the issues raised in this thesis are suitable topics for future research In

fact the author already has partial results on some of these which are not included

in this thesis The following section gives a list of p ossible directions for further

investigation

Directions for Future Research

There are many areas for further research suggested by the work presented in this

thesis We will very briey mention some of these Some of the items b elow esp e

cially items and have already b een studied to an extentby the author

Results will b e rep orted elsewhere

Condence Interval Pro cedures The numerical representation of probabil

ity distributions adopted in chapter lower and upp er b ounds on a distribu

tion function obviously suggests the use of condence intervals for acquiring

sample distributions This would allow a consistent representation from mea

surement to the nal result There are a numb er of results available in the

literature concerning condence intervals for quantiles Some of the questions

still to b e answered are Should the condence intervals b e overall for the

entire distribution or should they b e constructed p ointby p oint What dis

tributional assumptions are necessary Can a fast algorithm b e develop ed for

implementing the pro cedure chosen and Given two estimated distributions

with certain condence levels if the two distributions are combined is there

anything that can b e said ab out the condence of the result This last ques

tion lies at the heart of debates on the foundations of statistics We already

have some answers to some of these questions and we hop e to rep ort these

elsewhere

Dep endency Bounds for Functions of More than Arguments As we

mentioned in section it is p ossible to use linear programming techniques

to calculate Bo oleFrechet b ounds on the probability of complex comp ound

events When there are rep eated terms these b ounds are tighter than those

obtained by rep eated application of the pairwise b ounds for conjunction and

disjunction Can similar techniques b e applied to the determination of de

p endency b ounds for more complex functions of random variables This is

obviously a much harder problem for random variables than it is for random

events

Rearrangement Metho ds for Convolutions The convolution algorithm

develop ed in chapter makes essential use of sorting Is there a continuous

analogue of this algorithm Note that the continuous analogue of sorting

a discrete function into monotonic order is an equimeasurable rearrangement

Rearrangement techniques have b een recently applied to some statistical

optimization problems and it seems likely that further results can b e

obtained using these ideas In other words it may b e p ossible to develop

results for convolutions in terms of rearrangements of distribution functions or

probability densities

Bucketing Algorithms for convolutions The convolution algorithm we

develop ed has a computational complexityof O N log N whereN is the

numb er of p oints used to represent the distribution functions This means

that it runs rather slower than the algorithm for dep endency b ounds which

is O N but see item b elow It is p ossible to construct an algorithm for

convolutions which has average case complexity O N by using bucketing

algorithms It would app ear that this can b e further improved by using

adaptive recursive bucketing This would also provide a solution to a computer

science problem known as sorting and selection in multisets whichis

very similar to the problem of calculating convolutions

Discrete Tconjugate Transforms The Tconjugate transform used in chap

ter to prove the extremal limit result can b e converted into a discrete form

Furthermore there exists a fast divide and conquer algorithm for calculating

this This allows the calculation of the Tlogconcaveenvelop es of op era

tions in O N log N time A bilateral version could b e used for the very fast

calculation of the supT convolutions of fuzzy number memb ership functions

This has yet to b e implemented in the form of a computer program

Application of Interval Arithmetic Algorithms There exists a wide range

of sp ecial algorithms develop ed for interval arithmetic It is p ossible

that some of these may b e of use for probabilistic arithmetic calculations They

were develop ed in order to reduce the amount of dep endency width incurred

when calculating things like the inverses of interval matrices using Gaussian

elimination In order to use these algorithms it will b e necessary to develop a

means whereby dierentlower b ounds on copulas are used in the calculation of

the dep endency b ounds so that the lower and upp er b ounds on the distribution

functions do not diverge to o far See the next item

W The dep endency Further Study of Dep endency Bounds for C

b ounds studied in section deserve further attention A rst step would

b e the implementation of equation as a fast means of calculating de

W Wehave had no exp erience yet with the p endency b ounds when C

up dating of copulas in the manner suggested in section The relationship

between the dep endencies induced by these lower b ounds and other typ es of

dep endence also merits further investigation

Condence Curve Pro cedures The condence curves discussed in section

would app ear to deserve further consideration As mentioned in sec

tion there is a p ossibility of condence curves providing a link b etween

Bayesian style and condence interval style pro cedures in statistics Their

interpretation along the lines of fuzzy numb ers also requires a closer lo ok An

other asp ect is the direct combination of condence curves could they b e used

instead of distribution functions as representations of random quantities

Measures of Dep endence Based on Copulas and Dep endency Bounds

As we mentioned in section there are a numb er of measures or indices of

dep endence which can b e dened in terms of copulas These havebeeninves

tigated byWol and others The question is whether they can b e used

to provide tighter dep endency b ounds on distributions of functions of random

variables The eects on measures of dep endence of op erations on random

variables also deserves investigation

Empirical Copulas Very recently QuesadaMolina has suggested the

notion of an empirical copula Whilst this is obviously little more than a trans

formation of the empirical joint distribution function its statistical prop erties

require investigation Given the eect of lower b ounds on copulas on the tight

ness of dep endency b ounds it would also b e worthwhile investigating onesided

lower condence b ounds on the empirical copula

Empirical Multiplicative Generators of Archimedean Copulas A re

lated idea is to use the multiplicative generator representation of an Archimed

ean copula and thus to estimate the multiplicative generator This is a one

dimensional function rather than two dimensional Alsina and Gers result

on the con vergence of tnorms in terms of convergence of their generators

may b e of use here

History of Quotient Normal Random Variables In the course of our in

vestigations of probabilistic arithmetic wehave made a study of the history

of the distribution of the quotientoftwo normal random variables This is

obviously the sort of problem we exp ect our numerical metho ds to b e able

to calculate The history of the analytical attempts on this problem which

b egins with the rst determination of the distribution of a quotient of random

variables by Crofton in p and continues to the presentdaywitha

numb er of rediscoveries is another interesting asp ect we aim to write up one

day

Application to Random Equations and Practical Problems As we

mentioned ab ove the probabilistic arithmetic metho ds develop ed in this thesis

have so far only b een studied in terms of their foundations It remains to b e

seen whether they are of value for practical problems

Use of Graph Representations The use of graph representations was dis

cussed in section There it was noted that further research is needed to

determine whether these metho ds will b e of use in probabilistic arithmetic We

exp ect that this will b e the case if complex calculations are attempted The

rst step would b e to implement a simple expression rearrangement pro cedure

Relationship with Fines Interval Valued Probabilities Although our

lower and upp er b ounds on probability distributions can b e completely un

dersto o d in terms of the standard single valued notion of probability it seems

worthwhile examining whether they can b e usefully integrated with Fines in

terval valued probabilities section

Relationship with Probabilistic Metric Spaces Nearly all of the mathe

matical to ols wehave used in this thesis were originally develop ed in the eld

of probabilistic metric spaces Wehave already discussed the relationship b e

tween our metho ds and those of that eld section However we susp ect

that there may b e further results obtainable by studying this particularly

from the viewp oint of considering the and op erations as dep endency

TL TL

b ounds rather than just as triangle functions

Use of Mixtures for Nonmonotonic Op erations Further work is needed

to determine the practical value of the use of mixtures advo cated in section

for calculating b oth convolutions and dep endency b ounds under nonmonotonic

op erations on random variables Whilst analytically there are no diculties

there still remain some numerical problems to b e solved

Some General Philosophical Conclusions

Phenomena that are statistical ly calculable do not become

statistical ly incalculable suddenly at a wel ldened

boundary but rather by degrees The scholar takes a

position of cognitive optimism that is he assumes that the

subjects he studies wil l yield to calculation It is nicest if

they do so deterministical ly It is not quite so niceif

calculable probability has to substitute for certainty But it

is not nice at al l when absolutely nothing can becalculated

Stansilaw Lem

The work presented in this thesis can b e considered to b e just part of a general

long term trend to constructing probabilistic mo dels of reality General discussions

on the probabilistic view of the world can b e found in This probabilisitic

imp erative forms the background to a large p ortion of recent science and philosophy

of science Popp er has presented a numb er of arguments against determinism

and the deterministic world view is or at least should b e considered dead

However having adopted a probabilistic world view problems arise that were

absent in the simpler deterministic situation For example there is the problem of

acquisition of probabilityvalues This problem seemingly inno cuous at rst leads

one through the tortuous maze of interpretations of probability Nearly years

after Laplace there is still no sign of consensus in the scientic community ab out

this Thus it b ecomes necessary for us to try and makeupourown minds Whilst

wehave managed to avoid the problem in this thesis let us just mention nowour

preference for the prop ensityinterpretation with the rider that w e are

aware of many remaining problems with this

Apart from these philosophical concerns there is the practical problem of how

one pro ceeds to calculate with the probabilities Wehave already seen section

the dierence b etween our viewp oint based on random variables and that adopted

by the ma jorityofworkers in the eld of probabilistic metric spaces Even ignoring

this distinction there remain the severe practical diculties of solving even some

of the most simply stated probabilistic problems For example whilst there has

b een some encouraging progress recently in the study of p olynomials with random

co ecients through the use of Kharitonovs theorem the situation remains

much the same as it did in when Hammersley confessed he was still very far

from having solved the practical problem of determining the distribution of the

ro ots of a random p olynomial

The p oint is that there is a very deep hierarchy of problems in terms of their prac

tical diculty This diculty is not the in principle dicultywhich determinists

were always sure could b e removed by b eing clever enough Nowadays we can state

the diculty more precisely in terms of computational complexity theory Many

problems are insoluble b ecause of their exp onential computational complexity

What wehave seen in this thesis is a further example of this The exact calculation

of the distribution of complex functions of random variables is generally intractable

However it is p ossible to calculate lower and upp er b ounds on the solution by use of

the dep endency b ounds This approach seems preferable to the alternativeofinvok

ing the principle of maximum entropy and assuming indep endence solely in order

to get a single valued probability for a result Nevertheless randomness ultimately

wins and for the time b eing at least MonteCarlo simulation will remain the ma jor to ol for solving complex random problems

Chapter

References

He has overlooked no work in his eld regard less of how

barbarously or tediously written He has expoundedand

criticisedeach work Do you prefer to cal l this madness

or fol ly It does not matter to me merely confess that it is

done with my assistance

Folly Erasmus of Rotterdam

AVariant of the CESTAC Metho d and its Application to Constrained Optimization Math

ematics and Computers in Simulation

M H Ackroyd Computing the Waiting Time Distribution for the GG Queue by Signal Pro cessing

Metho ds IEEE Transactions on Communications

Iterative Computation of the MG Queue Length Distribution via the Discrete Fourier

Transform IEEE Transactions on Communications

MM Transient State Occupancy Probabiliti es via the Discrete Fourier Transform IEEE

Transactions on Communications

MG Queue Analysis via the Inverse Poisson Transform IEE Pro ceedings E

M H AckroydRKanyangarara Skinners Metho d for Computing Bounds on Distribution s and the

Numerical Solution of ContinuousTime Queueing Problems IEEE Transactions on Communications

E W Adams H P Levine On the Uncertainties Transmitted from Premises to Conclusions in

Deductive Inferences Synthese

G Adomian Nonlinear Sto chastic Op erator Equations Academic Press New York

A M Agogino AI in ComputerAided Design Qualitative Reasoning and Symb olic Computation

in The Study of the Design Pro cess MBWaldron ed Pro ceedings of the NSF Workshop in Design

Theory and Metho dology Oakland California February

A M Agogino A Rege IDES Inuence Diagram Based Exp ert Systems Mathematical Mo dellin g

R Ahmad A Distributio nFree Interval Mathematics Analysis of Probabili ty DensityFunctions in

Interval Mathematics Pro ceedings of the International Symp osium Karlsruhe West GermanyMay

K Nickel ed SpringerVerlag Berlin

S Ahmed R E Clark D R Metcalf A Metho d for Propagating Uncertainty in Probabilisti c Risk

Assessment Nuclear Technology

A V Aho S C Johnson Optimal Co de Generation for Expression Trees Journal of the Asso ci

ation for Computing Machinery

A V Aho S C Johnson J D Ullman Co de Generation for Expressions with Common Sub expres

sions in Conference Record of the rd ACM Symp osium on the Principles of Programming Languages

Asso ciation for Computing Machinery

A V Aho J D Ullman Optimization of Straight Line Programs SIAM Journal of Computing

V S Alagar The Distribution s of Random Determinants The Canadian Journal of Statistics

G Alefeld J Herzb erger Intro duction to Interval Computations Academic Press New York

D W Allan Should the Classical Variance b e Used as a Basic Measure in Standards Metrology

IEEE Transactions on Instrumentation and Measurement

R N Allan C H Grigg M R G AlShakarchi Numerical Techniques in Probabili stic Load Flow

Problems International Journal of Numerical Metho ds in Engineering

R N Allan A M Leite A A AbuNasser R C Burchett Discrete Convolution in Power System

Reliabil i ty IEEE Transactions on Reliabil i ty

N Alliot Data Error Analysis in Unconstrained Optimization Problems with the CESTAC Metho d

Mathematics and Computers in Simulation

A Allisy Some Statistical Metho ds Used in Metrology in Metrology and Fundamental Constants

Pro ceedings of the International Scho ol of Physics Enrico Fermi Course LXVI I I Varenna on Lake

Como Villa Monastero thth July A F Milone P Giacomo S Leschiutta eds North

Holland Italian Ph ysical So ciety Amsterdam

C Alsina Some Functional Equations in the Space of Uniform Distribution Functions Aequationes

Mathematicae

OnAFamily of Connectives for Fuzzy Sets Fuzzy Sets and Systems

C Alsina R Ger Asso ciative Op erations Close to a Given One Comptes Rendus Mathematical

Rep orts of the Academy of Science Canada

C Alsina E Trillas L Valverde On Some Logical Connectives for Fuzzy Sets Theory Journal of

Mathematical Analysis and Applications

D F Alwin R M Hauser The Decomp osition of Eects in Path Analysis American So ciological

Review

D Angluin HowtoProve it SIGACT News Asso ciation for Computing Machinery Sp ecial Interest

Group on Automata and Computability Theory

N Ap ostolatos U Kulisch R Krawczyk B Lortz K Nickel H W Wipp erman The Algorithmic

Language TriplexAlgol Numerische Mathematik

L App elbaum E H Ruspini Aries An Approximate Reasoning Inference Engine in Approximate

Reasoning in Exp ert Systems M M Gupta A Kandel J B Kiszka eds Elsevier Science Publishers

BV NorthHolland Amsterdam

M A Arbib Review of Fuzzy Sets and their applicati ons to cognitive and design pro cesses Intro

duction to the theory of fuzzy subsets volume Applications of fuzzy sets to systems analysis

Bulletin of the American Mathematical So ciety

M A Arbib E G Manes Partially Additive Categories and FlowDiagram Semantics Journal of

Algebra

S Arnb org J Lagergren D Seese Problems Easy for TreeDecomp osable Graphs in Automata

Languages and Programming th International Collo quium SpringerVerlag Berlin Lecture

Notes in Computer Science

S J Arnold MorphologyPerformance and Fitness American Zo ologist

L A Aroian The Probabili tyFunction of the Pro duct of Two Normally Distributed Variables

Annals of Mathematical Statistics

R Askey Linearization of the Pro duct of Orthogonal Polynomial s in Problems in Analysis A

Symp osium in Honour of Salomon Bo chner R C Gunning ed Princeton UP Princeton NJ

R Askey G Gasp er Convolution Structures for Laguerre Polynomials Journal DAnalyse Mathematique

R AskeySWainger Mean Convergence of Expansions in Laguerre and Hermite Series American

Journal of Mathematics

A Convolution Structure for Jacobi Series American Journal of Mathematics

R A Askey T H Ko ornwinder W Schempp Sp ecial Functions Group Theoretical Asp ects and

Application s D Riedel Dordrecht

R Badard The Law of Large Numb ers for Fuzzy Pro cesses and the Estimation Problem Information

Sciences

R Baekeland E E Kerre Piecewise Linear Fuzzy Quantities A way to ImplementFuzzy Infor

mation into Exp ert Systems and Fuzzy Databases in Uncertainty and Intelligent Systems Lecture

Notes in Computer Science BBouchon L Saitta R R Yager eds SpringerVerlag Berlin

R A Bagnold The Nature and Correlation of Random Distributio ns Pro ceedings of the Royal

So ciety of London A

W N Bailey On the Pro duct of Two Laguerre Polynomials Quarterly Journal of Mathematics

Oxford

S D Ba jpai An Integral Involving Foxs H function and Whittaker Functions Pro ceedings of the

Cambridge Philosophica l So ciety

G A Baker Transformation of NonNormal Frequency Distribution s into Normal Distribution s

Annals of Mathematical Statistics

J F Baldwin Fuzzy Sets and Exp ert Systems Information Sciences

W Bandler L J Kohout Probabilisti c versus Fuzzy Pro duction Rules in Exp ert Systems Inter

national Journal of ManMachine Studies

N C Barford Exp erimental Measurements Precision Error and Truth AddisonWesley London

J L Barlow E H Bareiss On Roundo Error Distribution s in Floating Point and Logarithmic

Arithmetic Computing

Probabili sti c Error Analysis of Gaussian Elimination in Floating Point and Logarithmic Arith

metic Computing

R E Barlow Using Inuence Diagrams in Accelerated Life Testing and Exp erts Opinions in Re

liabil ity C A Clarotti D V Lindley eds Italian Physical So ciety NorthHolland Amsterdam

Pro ceedings of the International Scho ol of Physics Enrico Fermi Course CI I

R E Barlow R W Mensing N G Smiriga An Application of Inuence Diagrams in Accelerated

Life Testing and Exp erts Opinions in Reliabili ty C A Clarotti D V Lindley eds Italian Physical

So ciety NorthHolland Amsterdam Pro ceedings of the In ternational Scho ol of Physics Enrico

Fermi Course CI I

R E Barlow X Zhang Bayesian Analysis of Insp ection Sampling Pro cedures Discussed byDem

ing Journal of Statistical Planning and Inference

B R Barmish A Generalization of Kharitonovs Four Polynomial Concept for Robust Stability

Problems with Linearly Dep endent Co ecientPerturbations in Pro ceedings of the American

Control Conference

J Baron SecondOrder Probabili ties and Belief Functions Theory and Decision

W W Bartley I I I A Popp erian Harvest in In Pursuit of Truth Essays on the Philosophyof

Karl Popp er on the Occasion of His th Birthday P Levinson ed Humanities Press Inc Atlantic

Highlands NJ

B W Bateman Keyness Changing Conception of Probability Economics and Philosophy

W D Baten Combining Two ProbabilityFunctions Annals of Mathematical Statistics

F L Bauer Computational Graphs and Rounding Error SIAM Journal of Numerical Analysis

L A Baxter Some Remarks on Numerical Convolution Communications in Statistics Simulation

and Computation B

R W Beals P C Greiner J Vauthier The Laguerre Calculus on the Heisenb erg Group in

Sp ecial Functions Group Theoretical Asp ects and Applications RAAskeyTHKoornwinder

W Schempp eds D Riedel Dordrecht

R Bellman A Note on the Mean Value of Random Determinants Quarterly of Applied Mathematics

R Bellman W Karush On a New Functional Transform in Analysis The Maximum Transform

Bulletin of the American Mathematical So ciety

Mathematical Programming and the Maximum Transform Journal of the So ciety for Indus

trial and Applied Mathematics

On the Maximum Transform and Semigroups of Transformations Bulletin of the American

Mathematical So ciety

On the Maximum Transform Journal of Mathematical Analysis and Applications

Functional Equations in the Theory of Dynamic Programming XI I An Application of the

Maximum Transform Journal of Mathematical Analysis and Applications

L S Belobragina V I Rudomanov EquivalentPerturba tion Metho d in Accuracy Evaluation for

Indirect Measurements MeasurementTechniques

G L Bennett L A Swiger R L Preston V R Cahill Evaluation of Urea Space and Ultrasonic

Measurement as Selectip on Criteria for Beef Animal Comp osition Journal of Animal Science

M Berres On a Multipli cati on and a Theory of Integration for Belief and Plausibil i tyFunctions

Journal of Mathematical Analysis and Applications

D P Bertsekas Constrained Optimization and Lagrange Multiplie r Metho ds Academic Press New

York

A T BharuchaReid Random Algebraic Equations in Probabili stic Metho ds in Applied Mathe

matics I I A T BharuchaReid ed Academic Press New York

A T BharuchaReid M Sambandham Random Polynomial s Academic Press Orlando

Bhascara Lilavati in Algebra and Mensuration from the Sanscrit of Brahmegupta and Bhascara

H T Colebro oke ed London

A Birnbaum Condence CurvesAnOmnibusTechnique for Estimation and Testing Statistical

Hyp otheses Journal of the American Statistical Asso ciation

A Unied Theory of Estimation I Annals of Mathematical Statistics

A Bjerhammar Theory of Errors and Generalized Matrix Inverses Elsevier Scientic Publishin g Com

pany Amsterdam

J L Black Identication and Control via the Statistical Prop erties of Input and Output Signals

PhD thesis University of Queensland

J L Black G F Shannon A Learning and Control Strategy for Linear and Nonlinear Systems in

Preprints of SOCOCO th IFACIFIP symp osium on Software for Computer Control thth

April separate pagination

M Black Vagueness An Exercise in Logical Analysis Philosophy of Science

Reasoning with Lo ose Concepts in Margins of Precision Essays in Logic and Language M

Black ed Cornell University Press Ithaca

R E Blahut Fast Algorithms for Digital Signal Pro cessing AddisonWesley Reading Massachusetts

H M Blalco ck Jr ed Causal Mo dels in the So cial Sciences Aldine Publishing Company Chicago

P L Bogler ShaferDempster Reasoning with Applications to Multisensor Target Identication Sys

tems IEEE Transactions on Systems Man and Cyb ernetics

H Bohman Numerical Inversion of Characteristic Functions Scandnavian Acturial Journal

G W Bohrnstedt A S Goldb erger On the Exact Covariance of Pro ducts of Random Variables

Journal of the American Statistical Asso ciation

G W Bohrnstedt G Marwell The Reliabili ty of Pro ducts of Two Random Variables in So cio

logical Metho dology KFSchuessler ed JosseyBass San Francisco

P Bois J Vignes A Software for Evaluating Lo cal Accuracy in the Fourier Transform Mathematics

and Computers in Simulation

An Algorithm for Automatic RoundO Error Analysis in Discrete Linear Transforms In

ternational Journal of Computer Mathematics

L P Bolgiano Jr M J Piovoso Poisson Transform Signal Analysis IEEE Transactions on Infor

mation Theory

L I Boneva D Kendall I Stefanov Spline Transformations Three New Diagnostic Aids for the

Statistical DataAnalyst with discussion Journal of the Royal Statistical So ciety Series B

PP Bonissone Theories of Plausible Reasoning in Reasoning with Incomplete and Uncertain

Information General Electric Corp orate ResearchandDevelopment Schenectady New York

Summarizing and Propagating Uncertain Information with Triangular Norms International

Journal of Approximate Reasoning

PP Bonissone K S Decker Selecting Uncertainty Calculi and Granularity An Exp erimentin

Tradingo Precision and Complexity in Uncertainty in Articial Intelligence LNKanalJF

Lemmer eds Elsevier Science Publishers BV NorthHolland Amsterdam

PP Bonissone R M Tong Editorial Reasoning with Uncertainty in Exp ert Systems Interna

tional Journal of ManMachine Studies

G Bo ole An Investigation of the Laws of Thought on which are founded The Mathematical Theories

of Logic and Probabiliti es Walton and Mab erley London Republished byDover Publication s

Incorp orated New York

On Prop ositions Numerically Denite Transactions of the Cambridge Philosophi cal So ciety

I I Reprinted as pp in R Rhees ed Studies in Logic and Probability by George

Bo ole Watts and Company London

Reply to Some Observations by Mr Wilbraham on the Theory of Chances The Philosophi cal

Magazine series August Reprinted as pp in R Rhees ed Studies in Logic and

Probabili ty by George Bo ole Watts and Company London

C de Bo or App endix to Splines and Histograms byIJScho enb erg in Spline Functions and

Approximation Theory A Meir A Sharma eds Birkhauser Basel

S B BoswellMSTaylor A Central Limit Theorem for Fuzzy Random Variables Fuzzy Sets

and Systems

B Bouchon Fuzzy Inferences and Conditional Possibili ty Distribution s Fuzzy Sets and Systems

P M Boudet Probabili te et logique de largumentation selon Jacques Bernoulli Les Etudes Philosophi ques

K O Bowman L R Shenton Approximations to Distribution s in Encyclop edia of Statistical

Sciences Volume S Kotz N L Johnson eds J Wiley and Sons New York

L Breiman Nail Finders Edices and Oz in Pro ceedings of the Berkeley Conference in Honour of

Jerzy Neyman and Jack Kiefer Volume L M L Cam R A Olshen eds Wadsworth Advanced

Bo oks and Software Monterey California

M A Breuer Generation of Optimal Co de for Expressions via Factorization Communications of

the Asso ciation for Computing Machinery

S R Broadb ent Lognormal Approximation to Pro ducts and Quotients Biometrika

J A Bro oke Review of Probabilistic Metric Spaces byBSchweizer and A Sklar Foundations of

Physics

R H Brown The Distributio n Function of Positive Denite Quadratic Forms in Normal Random

Variables SIAM Journal of Scientic and Statistical Computing

C Brownie J Kiefer The Ideas of Conditional Condence in the Simplest Setting Communications

in Statistics Theory and Metho ds A

J Buckley W Siler Fuzzy Op erators for Possibili tyInterval Sets Fuzzy Sets and Systems

J J Buckley Managing UncertaintyinaFuzzy Exp ert System International Journal of Man

Machine Studies

Possibili stic Linear Programming with Triangular Fuzzy Numb ers Fuzzy Sets and Systems

On the Algebra of InteractiveFuzzy Numbers Fuzzy Sets and Systems To app ear

On the Algebra of InteractiveFuzzy Numb ers The Continuous Case Fuzzy Sets and Systems

To app ear

J J Buckley H Ying Fuzzy Controller Theory Limit Theorems for Linear Fuzzy Controller Rules

Automatica

M Bunge Possibility and Probability in Foundations of Probability Theory Statistical Inference

and Statistical Theories of Science Volume III WLHarperCAHooker eds D Riedel Dor

drecht

Causality and Mo dern Science rd Revised EditionDover Publications New York

A J Buras Gaemers Simple Parametrizations of Parton Distributions with Q Dep endence given

by Asymptotic Freedom Nuclear Physics B

M Burke R Cytron Interpro cedural Dep endence Analysis and Paralleliza tion SIGPLAN Notices

J C Burkhill Functions of Intervals Pro ceedings of the London Mathematical So ciety Series

Derivatives of Functions of Intervals Fundamenta Mathematicae

CGini Di una misura della dissomigli anza tra due grappi di quantita e delle sue applicazioni allo

stadio delle relazioni statistiche Att R Ist Venoto Sc Lettere Art

N Cabibb o R Petronzio TwoStage Mo del of Hadron Structure Parton Distributio ns and their

Q Dep endence Nuclear Physics B

P Calabrese The ProbabilityoftheTriangle Inequality in Probabilisti c Metric Spaces Aequationes

Mathematicae

An Algebraic Synthesis of the Foundations of Logic and Probability Information Sciences

S Cambanis G Simons Probability and Exp ectation Inequalities Zeitschrift fur Wahrscheinli ch

keitstheorie und verwandte Gebiete

S Cambanis G Simons W Stout Inequaliti es for E k X Y when the Marginals are Fixed

Zeitschrift fur Wahrscheinli chkei tstheo rie und verwandte Gebiete

N R Campb ell An Account of the Principles of Measurement and Calculation Longmans Green and

Company Limited London Reprinted by University Microlms Ann Arb or Michigan

Foundations of Science Dover New York Originally Physics The Elements

M Caramanis J Stremel W Fleck S Daniel Probabilis tic Pro duction Costing An Investigation

of Alternative Algorithms Electrical Power and Energy Systems

B D Carter On the Probability Distribution of Rational Functions of Indep endent Hfunction Vari

ates PhD thesis UniversityofArkansas

B D Carter M D Springer The Distribution of Pro ducts Quotients and Powers of Indep endent

Hfunction Variates SIAM Journal of Applied Mathematics

N Cartwright Causal Laws and Eective Strategies Nous

U Cerruti U Hohle An Approach to Uncertainty Using Algebras Over a Monoidal Closed Cat

egory Rendiconti del Circolo Matematico di Palermo Biblioteca Matematica Supplement

Number Pro ceedings of the Second Top ology Conference Taormina

D A Chalker D Mackerras Mo dels for Representing the Acoustic Radiation Imp edance of the

Mouth IEEE Transactions on Acoustics Sp eech and Signal Pro cessing

S Chanas J Kamburowski The Use of Fuzzy Variables in PERT Fuzzy Sets and Systems

W K Chang L R Chow S K Chang Arithmetic Op erations on Level Sets of Convex Fuzzy

Numb ers in Fuzzy Information Knowledge Representation and Decision Analysis E Sanchez ed

IFACPergamon Press Oxford

A Chateauneuf J Y Jaray Derivation of Some Results on Monotone Capacities by Mobius

Inversion in UncertaintyandIntelligent Systems B Bouchon L Saitta R R Yager eds Springer

Verlag Berlin Lecture Notes in Computer Science

P Chatterjee Mo dularizati on of Fault Trees A Metho d to Reduce the Cost of Analysis in Reli

ability and Fault Tree Analysis Theoretical and Applied Asp ects of Systems Reliabil i ty and System

Assessment R E Barlow J B Fussell N D Singpurwall a eds SIAM Philadelphi a

P Cheeseman Probabilisti c versus Fuzzy Reasoning in Uncertainty in Articial Intelligence LN

Kanal J F Lemmer eds Elsevier Science Publishers BV NorthHolland Amsterdam

P M Chirlian C R Giardina C L Suel On the Eective Bandwidth of Sampled Signals IEEE

Transactions on Circuits and Systems

K M Chong N M Rice Equimeasurabl e Rearrangements of Functions Queens Pap ers in Pure

and Applied Mathematics

G Cho quet Theory of Capacities Annales Institute Fourier Grenoble

J H Chou I R Horng State Estimation Using Continuous Orthogonal Functions International

Journal of Systems Science

C I Christov A Complete Orthonormal System of Functions in L Space SIAM Journal

of Applied Mathematics

A Church Review of Kattso Journal of Symb olic Logic

M R Civanlar H J Trussell Constructing Memb ership Functions Using Statistical Data Fuzzy

Sets and Systems

J J Clark M R Palmer PDLawrence A Transformation Metho d for the Reconstruction of

Functions from Nonuniformly Spaced Samples IEEE Transactions on Acoustics Sp eech and Signal

Pro cessing

D G Clayton A Mo del for Asso ciation in Bivariate Life Tables and Its Application in Epidemiologi cal

Studies of Familial Tendency in Chronic Disease Incidence Biometrika

J G Cleary Acquisition of Uncertain Rules in a Probabili stic Logic International Journal of Man

Machine Studies

P R Clement Laguerre Functions in Signal Analysis and Parameter Identication Journal of the

Franklin Institute

Application s of Generalized Laguerre Functions Mathematics and Computers in Simulation

C W Clenshaw F W J Olver LevelIndex Arithmetic Op erations SIAM Journal of Numerical

Analysis

R Cleroux D J McConalogue A Numerical Algorithm for RecursivelyDe ned Convolution Inte

grals Involving Distribution Functions Management Science

J Cohen Partialed Pro ducts are Interactions Partialed Powers are Curve Comp onents Psycholog

ical Bulletin

L Cohen Probability Distributions with Given Multivariate Marginals Journal of Mathematical

Physics

A R Colclough Two Theories of Exp erimental Error Journal of Research of the National Bureau

of Standards

A G Colomb o Uncertainty Propagation in FaultTree Analysis in Synthesis and Analysis Metho ds

for Safety and Reliabil i ty Studies G Ap ostolakis S Garribba G Volta eds Plenum Press New

York

A G Colomb o R J Jaarsma A Powerful Numerical Metho d to Combine Random Variables

IEEE Transactions on Reliabil i ty

A S Co ok T Downs Estimating Manufacturing Yield by Means of Jacobian of Transformation

IEE Pro ceedings G

R D Co ok M E Johnson A Family of Distributio ns for Mo delling Nonelliptical ly Symmetric

Multivariate Data Journal of the Royal Statistical So cietySeriesB

Generalized BurrParetoLogistic Distributions with Application s to a Uranium Exploration

Data Set Technometrics

I D Co ok Jr J W Barnes Evaluation of the Hfunction Integral and of Probability Distribution s

of Functions of Indep endent Random Variables American Journal of Mathematical and Management

Sciences

G F Co op er A Diagnostic Metho d that uses Causal Knowledge and Linear Programming in the

Application of Bayes Formula Computer Metho ds and Programs in Biomedicine

Current Research Directions in the Development of Exp ert Systems Based on Belief Net

works Applied Sto chastic and Data Analysis

R D Co op er M R Hoare M Rahman Sto chastic Pro cesses and Sp ecial Functions On the Prob

abilisti c Origin of some Positive Kernels Asso ciated with Classical Orthogonal Polynomial s Journal

of Mathematical Analysis and its Applications

F Corsi Mathematical Mo dels for Marginal Reliabili ty Analysis Micro electronics and Reliabil i ty

D R Cox Some Problems Connected with Statistical Inference Annals of Mathematical Statistics

R Cox ProbabilityFrequency and Reasonable Exp ectation American Journal of Physics

C C Craig On the Frequency Function of xy Annals of Mathematical Statistics

H Cramer Mathematical Metho ds of Statistics Princeton University Press Princeton

M W Crofton On Mean Value and Probability in An Elementary Treatise on the Integral Calculus

rd edition Longmans Green and Company London Crofton wrote this one chapter in

the treatise by Benjamin Willi amson The chapter do es not app ear in earlier editions of Williamso ns

book

E L Crow A Prop erty of Additively Closed Families of Distributions Annals of Mathematical

Statistics

K S Crump Numerical Inversion of Laplace Transforms using a Fourier Series Approximation

Journal of the Asso ciation of Computing Machinery

M Cugiani A Sto chastic Analysis of Errors in Numerical Pro cesses in Numerical Techniques for

Sto chastic Systems PArchetti M Cugiani eds NorthHolland Amsterdam

R Cupp ens The Arithmetic of Distributio n Functions in A Collection of Pap ers dedicated to Eugene

Lukacs J Gani V K Rohatgi eds Academic Press New York

R Cuykendall Uncontrollabl e Computational Growth in Theoretical Physics International Journal

of Theoretical Physics

J F Cyranski The Probability of a Probability in Maximum EntropyandBayesian Metho ds in

Applied Statistics Pro ceedings of the th Workshop Calgary Canada J H Justice ed

Cambridge University Press New York

E Czogala Probabili sti c Fuzzy Controller as a Generalization of the Concept of a Fuzzy Controller

Fuzzy Sets and Systems

E Czogala T Rawlik Mo dellin g of a Fuzzy Controller with Application to the Control of Biological

Pro cesses Fuzzy Sets and Systems

E Czub er Wahrscheinlichkei tsrechnu ng und ihre Anwendung auf Fehlerausgleichung Statistik und

Leb ensversicherun g TwoVolumesBGTeubner Leipzig

G DallAglio Frechet Classes and Compatibility of Distribution Functions Symp osia Mathematica

I Daub echies A Grossman Y Meyer Painless Nonorthogonal Expansions Journal of Mathemat

ical Physics

B Davies B Martin Numerical inversion of the Laplace Transform a Survey and Comparison of

Metho ds Journal of Computational Physics

R B Davies Numerical Inversion of a Characteristic Function Biometrika

E Davis Constraint Propagation with Interval Lab els Aritcial Intelligence

A PDawid Conditional Indep endence in Statistical Theory Journal of the Royal Statistical So ciety

Series B with discussion

L Debnath On Laguerre Transform Bulletin of the Calcutta Mathematical So ciety

On Hermite Transform Matematicki Vesnik Belgrade

Some Op erational Prop erties of the Hermite Transform Matematicki Vesnik Belgrade

OnFaltung Theorem of Laguerre Transform Studia Universitatis Bab esBolyai Series Mathematica

physica

R DechterJPearl NetworkBased Heuristics for ConstraintSatisfaction Problems Articial

Intelligence

Tree Clustering for ConstraintNetworks Articial Intelligence

M Delgado S Moral On the Concept of Possibili tyProbability Consistency Fuzzy Sets and

Systems

M Delgado J L Verdegay M A Vila A General Mo del for Fuzzy Linear Programming Fuzzy

Sets and Systems

L S Dembo An Interview with Jorge Luis Borges Wisconsin Studies in Contemp orary Literature

Summer

A P Dempster Upp er and Lower Probability Inferences Based on a Sample from a Finite Univariate

Population Biometrika

Upp er and Lower Probabilities Induced by a Multivalu ed Mapping Annals of Mathematical

Statistics

A Generalization of Bayesian Inference Journal of the Royal Statistical So cietySeriesB

Upp er and Lower Probabiliti es Generated by a Random Closed Interval Annals of Mathe

matical Statistics

Upp er and Lower Probabili ty Inferences for Families of Hyp otheses with Monotone Density

Ratios Annals of Mathematical Statistics

A P Dempster A Kong Uncertain Evidence and Articial Analysis Journal of Statistical Plan

ning and Inference

M A H Dempster Distributions in Intervals and Linear Programming in Topics in Interval Anal

ysis E Hansen ed Oxford University Press Oxford

An Application of Quantile Arithmetic to the Distribution Problem of Sto chastic Linear

Programming Bulletin of the Institute of Mathematics and its Applications

Intro duction to Sto chastic Programming in Sto chastic Programming M A H Dempster

ed Academic Press London

M A H Dempster A Papagakipap oul ia s Computational Exp erience with an Approximate Metho d

for the Distribution Problem in Sto chastic Programming M A H Dempster ed Academic Press

London

J B D Derksen On Some Innite Series Intro duced by Tschuprow Annals of Mathematical Statis

tics

L Devroye Lecture Notes on Bucket Algorithms Birkhauser Boston

NonUniform Random Variate Generation SpringerVerlag New York

S Dharmadhikari K JoagDev Unimo dalityConvexity and Applications Academic Press New

York

S W Dharmadhikari K JoagDev The GaussTchebyshev Inequality for Unimo dal Distribution s

Theory of Probability and its Applications

B H Dickman M J Gilman Monte Carlo Optimization Journal of Optimization Theory and

Application s

J G Dijkman H van Haeringen S J de Lange Fuzzy Numb ers Journal of Mathematical Analysis

and Applications

M Dishon G H Weiss Numerical Inversion of Mellin and TwoSided Laplace Transforms Journal

of Computational Physics

J P Dolan The Essential Erasmus Meridian New York

Z Domotor Probabilisti c and Causal Dep endence Structures Theory and Decision

W Dong H C Shah Vertex Metho d for Computing Functions of Fuzzy Variables Fuzzy Sets

and Systems

W Dong F S Wong InteractiveFuzzy Variables and Fuzzy Decisions Fuzzy Sets and Systems

T Downs A S Co ok G Rogers A Partitioning Approach to Yield Estimation for Large Circuits

and Systems IEEE Transactions on Circuits and Systems

D Driankov Inference with Consistent Probabiliti es in Exp ert Systems International Journal of

Intelligent Systems

D Dub ois Possibili ty Theory Searching for NormativeFoundations in Risk Decision and Ratio

nality B R Munier ed D Riedel Dordrecht

D Dub ois H Prade CommentonTolerance analysis using fuzzy sets and A pro cedure for multiple

asp ect decision making International Journal of Systems Science

Op erations on Fuzzy Numb ers International Journal of Systems Science

Fuzzy Real Algebra Some Results Fuzzy Sets and Systems

DecisionMakin g Under Fuzziness in Advances in Fuzzy Set Theory and Applications M

M Gupta R K Ragade R R Yager eds NorthHolland Amsterdam

Fuzzy Sets and Systems Theory and Applications Academic Press New York Math

ematics in Science and Engineering volume

Additions of InteractiveFuzzy Numbers IEEE Transactions on Automatic Control

On Several Representations of an Uncertain Bo dy of Evidence in Fuzzy Information and

Decision Pro cesses MMGuptaESanchez eds NorthHolland Amsterdam

A Class of Fuzzy Measures based on Triangular Norms International Journal of General

Systems

Twofold Fuzzy Sets Fuzzy Mathematics Huazhong China

A Review of Fuzzy Set Aggregation Connectives Information Sciences

Fuzzy Sets and Statistical Data Europ ean Journal of Op erational Research

A SetTheoretic View of Belief Functions Logical Op erations and Approximations byFuzzy

Sets International Journal of General Systems

On the Unicity of Dempster Rule of Combination International Journal of Intelligent Sys

tems

The Mean ValueofaFuzzy Number Fuzzy Sets and Systems

Editorial to sp ecial issue on fuzzy numb ers Fuzzy Sets and Systems

Evidence Theory and Interval Analysis Toward bridging the gap b etween fuzzy arithmetics

and MonteCarlo simulation in Preprints of the Second IFSA Congress

Fuzzy Numbers An Overview in Analysis of Fuzzy Information Volume Mathematics

and Logic J C Bezdek ed CRC Press Inc Bo ca Raton Florida

Possibili ty Theory An Approach to Computerized Pro cessing of Uncertainty Plenum Press

New York and London

O D Duncan Path Analysis So ciological Examples The American Journal of So ciology

Contingencies in Constructing Causal Mo dels in So ciologic al Metho dology EF

Borgatta ed JosseyBass San Francisco

Partials Partitions and Paths in So ciological Metho dology E F Borgatta ed Jossey

Bass San Francisco

Intro duction to Structural Equation Mo dels Academic Press New York

J Dupacova Sto chastic and Dynamic Programming Exp erience in Sto chastic Programming Mo d

els in Survey of Mathematical Programming Pro ceedings of the th International Mathematical

Programming Symp osium Part APrekopa ed Publishing House of the Hungarian Academyof

Sciences Budap est

N Ebrahimi Bivariate Pro cesses with Positive or Negative Dep endent Structures Journal of Applied

Probabili ty

K Ecker H Ratschek Intervallarithmeti k mit Wahrscheinli ckkei tsstruk tur Angewandte Infor

matik Elektron Datenverarb eitung

E Eells Probabili stic Causal Interaction PhilosophyofScience

Probabilis tic Causality Reply to John Depre Philosophy of Science

E Eells E Sob er Probabilisti c Causality and the Question of Transitivity Philosophy of Science

W P Elderton N L Johnson Systems of Frequency Curves Cambridge University Press Cam

bridge

B S Eldred The Application of Integral Transform Theory to the Analysis of Probabilisti c Cash

Flows Do ctoral dissertation College of Engineering UniversityofTexas

F S Ellett Jr D P Ericson Correlation Partial Correlation and Causation Synthese

B Epstein Some Applications of the Mellin Transform in Statistics Annals of Mathematical Statis

tics

A Erdelyi On Some Expansions in Laguerre Polynomials Journal of the London Mathematical

So ciety

A Erdelyi W Magnus F Ob erhettinger F G Tricomi Higher Transcendental Functions Volume

II McGrawHill New York

Tables of Integral Transforms Volume I McGrawHill New York

J D Esary F Proschan Relationshi ps Among Some Concepts of Bivariate Dep endence The

Annals of Mathematical Statistics

J C Falmagne A Random Utility Mo del for a Belief Function Synthese

J PFaye J Vignes Sto chastic ApproachofthePermutationPerturbation Metho d for RoundO

Error Analysis Applied Numerical Analysis

PEFeldheim Une Propriete Caracteristique des p olynomes de Laguerre Commentarii Mathematici

Helvetici

W Feller An Intro duction to Probability Theory and its Application s rd Edition Volume John

Wiley and Sons New York

W Fenchel On Conjugate Convex Functions Canadian Journal of Mathematics

I Fenyo Some Remarks on the Theory of the LaguerrePinney transformation Aequationes Math

ematicae

R PFeynmann Simulating Physics with Computers International Journal of Theoretical Physics

M A Field The Literary Theories of Jorge Luis Borges PhD thesis University of Queensland

E C Fieller The Distribution of the Index in a Normal Bivariate Population Biometrika

T L Fine Lower Probability Mo dels for Uncertainty and Nondeterministic Pro cesses Journal of

Statistical Planning and Inference

P C Fishburn Interval Mo dels for Comparative Probabili ty on Finite Sets Journal of Mathematical

Psychology

M FlenstedJensen T Ko ornwinder The Convolution Structure for Jacobi Function Expansions

Arkiv fur Matematik

G Forb es The Metaphysics of Mo dality Clarendon Press Oxford

G E Forsythe Generation and use of Orthogonal Polynomials for DataFitting with a Digital Com

puter Journal of the So ciety for Industrial and Applied Mathematics

R Fortet Random Determinants Journal of Research of the National Bureau of Standards

C Fox The G and Hfunctions as Symmetrical Fourier Kernels Transactions of the American

Mathematical So ciety

G Fox J Quirk Uncertainty and InputOutput Analysis Jet Propulsion Lab oratory Un

published

M J Frank On the Simultaneous Asso ciativityofF x y and x y F x y Aequationes Mathe

maticae

M J FrankRBNelsenBSchweizer BestPossible Bounds for the Distribution of a Sum A

Problem of Kolmogorov Probability Theory and Related Fields

M J Frank B Schweizer On the Duality of Generalized Inmal and Supremal Convolutions

Rendiconti di Matematica Series VI

M Frechet Generalisations du theoreme des probabili tes totales Fundamenta Mathematica

Sur les tableaux de correlation dont les marges sont donnees Annales de lUniversitede

Lyon A

G N Frederickson D B Johnson The Complexity of Selection and Ranking in X Y and Matrices

with Sorted Columns Journal of Computing and Systems Science

T S Freeman G M Imirzian E Kaltofen L Yagati Dagwo o d A System for Manipulatin g Poly

nomials Given by StraightLine Programs ACM Transactions on Mathematical Software

PVFua Using Probabili tyDensi tyFunctions in the Framework of Evidential Reasoning in Un

certaintyandIntelligentSystems B Bouchon L Saitta R R Yager eds SpringerVerlag Berlin

Lecture Notes in Computer Science

D R Fulkerson Exp ected Critical Path Length in PERTtyp e Networks Op erations Research

W Furmanski R Petronzio A Metho d of Analyzing the Scaling Violation of Inclusive Sp ectra in

Hard Pro cesses Nuclear Physics B

B Gabutti B Minetti A New Application of the Discrete Laguerre Polynomials in the Numerical

Evaluation of the Hankel Transform of a Strongly Decreasing Even Function Journal of Computational

Physics

N GakeLRuschendorf On a Class of Extremum Problems in Statistics Mathematische Op er

ationsforschung und Statistik Series Optimization

On the Existence of Probabili ty Measures with Given Marginals Statistics and Decisions

J Garlo Interval Mathematics A Bibliography Freiburger IntervallBerichte

G Gasp er Linearization of the Pro duct of Jacobi Polynomials I I Canadian Journal of Mathematics

Linearization of the Pro duct of Jacobi Polynomials I Canadian Journal of Mathematics

Positivity and the Convolution Structure for Jacobi Series Annals of Mathematics

G Gasp er M Rahman Nonnegative Kernels in Pro duct Formulas for qRacah Polynomials I

Journal of Mathematical Analysis and its Applications

W Gaul Bounds for the Exp ected Duration of a Sto chastic Pro ject Planning Mo del Journal of

Information and Optimization Sciences

W Gautschi A Computational Pro cedure for Incomplete Gamma Functions ACM Transactions on

Mathematical Software

M R Genesereth N J Nilsson Logical Foundations of Articial Intelligence Morgan Kaufmann

Publishers Inc

C Genest Franks Family of Bivariate Distributio ns Biometrika

C Genest J MacKay The Joy of Copulas Bivariate Distribution s with Uniform Marginals The

American Statistician

C Genest R J MacKay Copules Archimediennes et Familles de Lois Bidimensionn el les Dont les

Marges Sont Donnees The Canadian Journal of Statistics

R Genin L C Calvez Sur quelques proprietes de la transformation de Laguerre Comptes Rendus

Academie des Sciences Paris Series AB e

R Gero ch J B Hartle Computability and Physical Theories Foundations of Physics

P Giacomo On the Expression of Uncertainties in Quantum Metrology and Fundamental Physical

Constants P H Cutler A A Lucas eds Plenum Press New York

R A Gideon J Gurland Series Expansions for Quadratic Forms in Normal Variables Journal of

the American Statistical Asso ciation

R Giles A Logic for Sub jective Belief in Foundations of Probability Theory Statistical Inference

and Statistical Theories of Science Volume Foundations and Philosophy of Epistemic Application s

of Probability Theory W L Harp er C A Ho oker eds D Riedel Dordrecht

Foundations for a Theory of Possibility in Fuzzy Information and Decision Pro cesses M

M Gupta E Sanchez eds NorthHolland Amsterdam

Semantics of Fuzzy Reasoning International Journal of ManMachine Studies

The Concept of a Grade of Memb ership Fuzzy Sets and Systems

H J Glaeske Die LaguerrePinney Transformation Aequationes Mathematicae

On a Class of Integral Transformations Wissenschaftli che Zeitschrift der FriedrichSchi ll er

Universitat Mathematischnaturewissen schaftl ich e Reihe

On a Convolution Structure of a Generalized Hermite Transformation Serdica Bulgaria

H J Glaeske O I Maricev The Laguerre Transform of the Some Elementary Functions Zeitschrift

fur Analysis und ihre Anwendungen

G V Glass Correlations with Pro ducts of Variables Statistical Formulation and Implications for

Metho dology American Educational Research Journal

J Glaz B MK Johnson Probabili ty Inequalitie s for Multivariate Distribution s with Dep endence

Structures Journal of the American Statistical Asso ciation

C Glymour Articial Intelligence is Philosophy in Asp ects of Articial Intelligence JHFetzer

ed Kluwer Academic Publishers Dordrecht

C Glymour R Scheines Causal Mo deling with the TETRAD Program Synthese

C Glymour R Scheines P Spirtes K Kelly Discovery Causal Structure Articial Intelligence

Philosophy of Science and Statistical Mo dellin g Academic Press Orlando Florida

H J Go dwin Generalisatio ns of Tchebyche s Inequality Journals of the American Statistical

Asso ciation

Inequalities on Distributio n Functions Grin London

J Goguen Fuzzy Sets and the So cial Nature of Truth in Advances in Fuzzy Set Theory and Appli

cations M M Gupta R K Ragade R R Yager eds NorthHolland Amsterdam

A Goico echea Exp ert System Mo dels for Inference with Imp erfect Knowledge A Comparative

Study Journal of Statistical Planning and Inference

A S Goldb erger Structural Equation Mo dels An Overview in Structural Equation Mo dels An

Overview A S Goldb erger O D Duncan eds Seminar Press New York

G Gong Pro duction Systems and Belief Functions in Computer Science and Statistics The Inter

face D M Allen ed Elsevier Science Publishers BV NorthHolland Amsterdam

T Gonzalez J JaJa On the Complexity of Computing Bilinear Forms with fg Constants

Journal of Computer and Systems Sciences

Evaluation of Arithmetic Expressions with Algebraic Identites SIAM Journal of Computing

A GonzalezArrogo C Lop ez SecondOrder Contributions to the Structure Functions in Deep

Inelastic Scattering I I I The Singlet Case Nuclear Physics B

I J Go o d Some History of the Hierarchical Bayesian Metho dology in Go o d Thinking The Foun

dations of Probability and its Applications University of Minnesota Press Minneap oli s

Sub jective Probabili ty as the Measure of a NonMeasurable Set in Go o d Thinking The

Foundations of Probabili ty and its Applications Univeristy of Minnesota Press Minneap ol is

Condence Journal of Statistical Computing and Simulation

I R Go o dman Some New Results Concerning Random Sets and Fuzzy Sets Information Sciences

Some Asymptotic Results for the Combination of Evidence Problem Mathematical Mo d

elling

L A Go o dman On the Exact Variance of Pro ducts Journal of the American Statistical Asso ciation

The Variance of the Pro duct of K Random Variables Journal of the American Statistical

Asso ciation

S Gottwald A Cumulative System of Fuzzy Sets in Set Theory and Hierarchy Theory Lecture

Notes in Mathematics M Srebrny A Zarach eds SpringerVerlag Berlin

Set Theory for Fuzzy Sets of Higher Level Fuzzy Sets and Systems

Fuzzy Uniqueness of Fuzzy Mappings Fuzzy Sets and Systems

I S Gradshteyn I M Ryzhik Table of Integrals Series and Pro ducts th Edition Academic

Press New York

I GrattanGuinness Fuzzy Membership Mapped onto Interval and ManyValued Quantities Zeitschrift

fur Mathematische Logik und Grundlagen der Mathematik

I J Greenb erg Distributio nFree Analysis of MG and GM Queues Op erations Research

Y I Grize T L Fine Continuous Lower Probability Based Mo dels for Stationary Pro cesses with

Bounded and Divergent Time Averages Annals of Probability

B N Grosof Evidential Conrmation as Transformed Probability On the Duality of Priors and

Up dates in Uncertainty in Articial Intelligence L N Kanal J F Lemmer eds Elsevier Science

Publishers BV NorthHolland Amsterdam

An InequalityParadigm for Probabilisti c Knowledge The Logic of Conditional Probabili ty

Intervals in Uncertainty in Articial Intelligence L N Kanal J F Lemmer eds Elsevier Science

Publishers BV NorthHolland Amsterdam

R D Gumb Comments on Probabilisti c Semantics in Essays in Epistemology and Semantics H

Leblanc R Stern R Gumb eds Haven Publicatio ns New York

E J Gumb el On the Reliabil i ty of the Classical ChiSquare Test Annals of Mathematical Statistics

Statistics of Extremes Columbia University Press New York and London

MeasurementofRareEvents in Measurement Denitions and Theories CWChurchman

P Rato osh eds J Wiley and Sons New York

D P Gupta Cesaro Summabili ty of Laguerre Series Journal of Approximation Theory

M A S Guth Uncertainty Analysis of RuleBased Exp ert Systems with DempsterShafer Mass

Assignments International Journal of Intelligent Systems

I Hacking Possibility Philosophi cal Review

Equip ossibi l ity Theories of Probabili ty British Journal for the Philosophy of Science

All Kinds of Possibili ty Philosophi cal Review

The Emergence of Probabili ty Cambridge University Press Cambridge

G J Hahn S S Shapiro Statistical Mo dels in Engineering John Wiley and Sons New York

T Hailp erin Best Possible Inequalities for the Probability of a Logical Function of Events American

Mathematical Monthly

Probability Logic Notre Dame Journal of Formal Logic

Bo oles Logic and Probability NorthHolland Amsterdam Second Edition

J M Hammersley The Zeros of a Random Polynomial in Pro ceedings of the rd Berkeley Symp o

sium on Mathematical Statistics and ProbabilityVolume I I J Neyman ed University of California

Press Berkeley and Los Angeles

J F Hanna Probabilisti c Explanation and Probabili stic Causality in PSA Pro ceedings of

the Biennial Meeting of the Philosophy of Science Asso ciation Volume PDAsquithT

Nickles eds Philosophy of Science Asso ciation East Lansing

E R Hansen A Globally ConvergentInterval Metho d for Computing and Bounding Real Ro ots

BIT

F Harary R Z Norman D Cartwright Structural Mo dels An Intro duction to the Theory of

Directed Graphs John Wiley and Sons New York

W Harp er A Conditional Belief Semantics for Free Quantication Logic with Identity in Essays in

Epistemology and Semantics HLeblancRSternRGumb eds Haven Publicatio ns New York

A Hart Knowledge Acquisition for Exp ert Systems Kogan Page London

W D Hart Probability as Degree of Possibili ty Notre Dame Journal of Formal Logic

C Hastings Approximations for Digital Computers Princeton University Press Princeton

M Hauenschild Extended Circular Arithmetic Problems and Results in Interval Mathematics

K Nickel ed Academic Press New York

D Hausman Are there Causal Relations Among Dep endentVariables Philosophy of Science

J W Head W P Wilson Laguerre Functions Tables and Prop erties Pro ceedings of the IEE

A C Hearn Optimal Evaluation of Algebraic Expressions in Algebraic Algorithms and Error

Correcting Co des J Calmet ed SpringerVerlag Berlin Lecture Notes in Computer Science

D Heckerman Probabili stic Interpretations for MYCINs CertaintyFactors in Uncertainty in Arti

cial Intelligence L N Kanal J F Lemmer eds Elsevier Science Publishers BV NorthHolland

Amsterdam

D E Heckerman An Axiomatic Framework for Belief Up dates in Uncertainty in Articial In

telligence J F Lemmer L N Kanal eds Elsevier Science Publishers BV NorthHolland

Amsterdam

D E Heckerman E J Horvitz The Myth of Mo dularity in RuleBased Systems for Reasoning with

Uncertainty in Uncertainty in Articial Intelligence J F Lemmer L N Kanal eds Elsevier

Science Publishers BV NorthHolland Amsterdam

D R Heise Problems in Path Analysis and Causal Inference in So ciological Metho dology E

F Borgatta ed JosseyBass San Francisco

S J Henkind M C Harrison An Analysis of Four Uncertainty Calculi IEEE Transactions on

Systems Man and Cyb ernetics

M Henrion Uncertainty in Articial Intelligence Is Probability Epistemologica ll y and Heuristically

Adequate in Exp ert Judgement and Exp ert Systems JLMumpower L D Phillips O Renn

V R R Uppuluri eds SpringerVerlag Berlin

Propagating UncertaintyinBayesian Networks by Probabilisti c Logic Sampling in Uncer

tainty in Articial Intelligence J F Lemmer L N Kanal eds Elsevier Science Publishers BV

NorthHolland Amsterdam

J R Higgins Five Short stories ab out the Cardinal Series Bulletin New Series of the American

Mathematical So ciety

T W Hill Jr On Determining a Distribution Function Known Only by its Moments PhD thesis

Arizona State University

J Hintakka Aristotle on the Realization of Possibili ties in Time in Reforging the Great Chain of

Being Studies in the History of Mo dal Theories SKnuuttila ed D Riedel Dordrecht

I I Hirschman Jr Laguerre Transforms Duke Mathematical Journal

I I Hirschman Jr D V Widder The Convolution Transform Princeton University Press Princeton

E Hisdal Conditional Possibili ties Indep endence and Noninteraction Fuzzy Sets and Systems

Possibili stic all y Dep endentVariables and a General Theory of Fuzzy Sets in Advances in

Fuzzy Set Theory and Applications M M Gupta R K Ragade R R Yager eds NorthHolland

Amsterdam

InniteValued Logic based on TwoValued Logic and ProbabilityPart Diculties with

the PresentdayFuzzySet Theory and their Resolution in the TEE Mo del International Journal of

ManMachine Studies

InniteValued Logic based on TwoValued Logic and ProbabilityPart Dierent Sources

of Fuzziness International Journal of ManMachine Studies

Are Grades of Memb ership Probabili ties Fuzzy Sets and Systems

Guest Editorial Fuzzy Sets and Systems

The Philosophi cal Issues Raised byFuzzy Set Theory Fuzzy Sets and Systems

U W Ho chstrasser Orthogonal Polynomials in Handb o ok of Mathematical Functions M Abramowitz

I A Stegun eds National Bureau of Standards Applied Mathematics Series

W Hoding Masstabinvariante Korrelationstheorie Schriftenreihe Math Inst und Inst Angew

Math Univeristat Berlin Mathematical Reviews Zbl

U Hohle Probabili stic Uniformization of Fuzzy Top ologies Fuzzy Sets and Systems

Representation Theorems for LFuzzy Quantities Fuzzy Sets and Systems

Fuzzy Real Numb ers as Dedekind Cuts with Resp ect to a Multiple Valued Logic Fuzzy Sets

and Systems

A General Theory of Fuzzy Plausibil ity Measures Journal of Mathematical Analysis and

Application s

Quotients with Resp ect to Similarity Relations Fuzzy Sets and Systems

P W Holland Statistics and Causal Inference Journal of the American Statistical Asso ciation

With discussion

PWHollandYJWang Dep endence Function for Continuous Bivariate Densities Communi

cations in Statistics Theory and Metho ds

Regional Dep endence for Continuous Bivariate Densities Communications in Statistics

Theory and Metho ds

S Horwitz T Reps D Binkley Interpro cedural Slicing Using Dep endence Graphs SIGPLAN

Notices Pro ceedings of the SIGPLAN Conference on Programming Language Design

and Implementation Atlanta Georgia June

G E Hughes M J Cresswell An Intro duction to Mo dal Logic Methuen London

R A Hummel M S Landy A Statistical Viewp oint on the Theory of Evidence IEEE Transactions

on Pattern Analysis and Machine Intelligence

E V Huntington Frequency Distribution of Pro duct and Quotient Annals of Mathematical Statis

tics

C Hwang Y P Shih Parameter Identication via Laguerre Polynomials International Journal

of Systems Science

Laguerre Series Solution of a Functional Dierential Equation International Journal of

Systems Science

G E Ingram E L Walker C R Herrmann Designing for Reliabil ity Based on Probabilisti c

Mo dellin g Using Remote Access Computer Systems in Pro ceedings of the th Reliabili ty and Main

tainabili ty Conference San Francisco July

V V Ivlev Determination of the Parameters of the Generalized Laguerre Polynomials in Reliabil i ty

Problems Engineering Cyb ernetics

P S Jackson A SecondOrder Moments Metho d for Uncertainty Analysis IEEE Transactions on

Reliabil i ty

H Jacobs J W Barnes I D Co ok Jr Applications of the HFunction Distribution in Classifying

and Fitting Classical Probability Distributions American Journal of Mathematical and Management

Sciences

R Jain Tolerance Analysis using Fuzzy Sets International Journal of Systems Science

Outline of an Approach for the Analysis of Fuzzy Systems International Journal of Control

E T Jaynes Condence Intervals vs Bayesian Intervals in Foundations of Probability Theory

Statistical Inference and Statistical Theories of Science Volume I I WLHarper CAHooker

eds D Riedel Dordrecht

A J Jerri Sampling Expansion for a LaguerreL Transform Journal of Research of the National

Bureau of Standards B

The Shannon Sampling Theorem Its Various Extensions and Applications A Tutorial

Review Pro ceedings of the IEEE Corrections

K Jogdeo Dep endence Concepts of in Encyclop edi a of Statistical Sciences N L Johnson S

Kotz eds John Wiley and Sons New York

N L Johnson S Kotz Continuous Univariate Distributio ns Houghton Miin Company Boston

P T Johnstone Op en Letter to Ian Graham Fuzzy Sets and Systems with reply

M C Jones H W Lotwick On the Errors Involved in Computing the Empirical Characteristic

Function Journal of Statistical Computation and Simulation

A Remark on Algorithms AS Kernel Density Estimation Using the Fast Fourier Trans

form Applied Statistics

K G Joreskog A General Metho d for Estimating a Linear Structural Equation System in Structural

Equation Mo dels in the So cial Sciences A S Goldb erger O D Duncan eds Seminar Press New

York

W M Kahane A More Complete Interval Arithmetic Unpublished Lecture Notes for a Summer

Course at the University of Michigan

P Kall Sto chastic Linear Programming SpringerVerlag Berlin

L N Kanal J F Lemmer eds Uncertainty in Articial Intelligence Elsevier Science Publishers

BV NorthHolland Amsterdam

Y Kanjin On Algebras with Convolution Structures for Laguerre Polynomial s Transactions of the

American Mathematical So ciety

M Kaplan T L Fine Joint Orders in Comparative Probabili ty Annals of Probability

S Kaplan On the Metho d of Discrete Probability Distribution s in Risk and Reliabil i ty Calculations

Applications to Seismic Risk Assessment Risk Analysis

Resp onse to Nelson and Rasmusons Letter Risk Analysis

S Kaplan J C Lin An Improved Condensation Pro cedure in Discrete Probability Distribution

Calculations Risk Analysis

S Kaplan H F Perla D C Bley A Metho dology for Seismic Risk Analysis of Nuclear Power

Plants Risk Analysis

J C Kapteyn Denition of the Correlation Co ecient Monthly Notices of the Royal Astronomical

So ciety

L O Kattso Mo dality and Probabili ty The Philosophi cal Review

D Katz Discrete Approximations to Continuous DensityFunctions that are L Optimal Compu

tational Statistics and Data Analysis

D Katz D Z DArgenio Discrete Approximation of Multivariate Densities with Application to

Bayesian Estimation Computational Statistics and Data Analysis

A Kaufmann Fuzzy Subsets Applications in OR and Management in Fuzzy Sets Theory and

Application s A Jones A Kaufmann H J Zimmermann eds D Riedel Dordrecht

A Kaufmann M M Gupta Intro duction to Fuzzy Arithmetic Theory and Applications Van

Nostrand Reinhold New York

R B Keey C H Smith The Propagation of Uncertainties in Failure Events Reliabili ty Engi

neering

J Keilson Return of the Wanderer A Physicist Becomes a Probabilist in The Craft of Probabilisti c

Mo dellin g A Collection of Personal Accounts J Gani ed SpringerVerlag New York

J Keilson W R Nunn Laguerre Transformation as a To ol for the Numerical Solution of Integral

Equations of Convolution Typ e Applied Mathematics and Computation

J Keilson W R Nunn U Sumita The Bilateral Laguerre Transform Applied Mathematics and

Computation

J Keilson D Petrondas U Sumita J Wellner Signican ce Points for Some Tests of Uniformity

on the Sphere Journal of Statistical Computation and Simulation

J Keilson U Sumita Waiting Time Distribution Resp onse to Trac Surges via the Laguerre

Transform Working Pap er Series No The Graduate Scho ol of Management The Universityof

Ro chester

The Depletion Time for MG Systems and a Related Limit Theorem Advances in Applied

Probabili ty

Evaluation of the Total Time in System in a PreemptResume Priority Queue via a Mo died

Lindley Pro cess Advances in Applied Probability

A General Laguerre Transform and a Related Distance b etween Probability Measures Jour

nal of Mathematical Analysis and its Applications

H G Kellerer Funktionen auf Pro duktraumen mit vorgegeb enen MarginalFunkti onen Mathema

tische Annalen

Matheoretische Marginalprob le me Mathematische Annalen

Marginalprob leme fur Funktionen Mathematische Annalen

Verteilungsfunkti onen mit gegeb enen Marginalvertei lu ngen Zeitschrift fur Wahrscheinli ch

keitstheorie und verwandte Gebiete

Duality Theorems for Marginal Problems Zeitschrift fur Wahrscheinli chkei tsthe orie und

verwandte Gebiete

S D Kellog J W Barnes The Distribution of Pro ducts Quotients and Powers of Two Dep endent

HFunction Variates Mathematics and Computers in Simulation

M Kendall A Stuart The Advanced Theory of Statistics Volume Distribution Theory th

Edition Grin London

M G Kendall PAP Moran Geometrical Probability Grin London

M G Kendall A Stuart The Advanced Theory Of Statistics Volume Inference and Relationship

rd Edition Grin London

W J Kennedy Jr J E Gentle Statistical Computing Marcel Dekker Inc New York and Basel

D A Kenny Correlation and Causality John Wiley and Sons New York

B W Kernighan D M Ritchie The C Programming Language PrenticeHall Englewo o d Clis

New Jersey

J M Keynes ATreatise on Probability MacMillan and Company London

V L Kharitonov Asymptotic Stability of an Equilibriu m Position of a Family of Systems of Linear

Dierential Equations Dierential nyeUraveniya

J Kiefer Conditional Condence Statements and Condence Estimators Journal of the American

Statistical Asso ciation with discussion Reprinted without the discussion in LD Brown

I Olkin J Sacks HPWynnedsJack Karl Keifer Collected Pap ers I I Statistical Inference and

Probabili ty SpringerVerlag New York

Conditional Inference in Encyclop edia of Statistical Sciences N L Johnson S Kotz eds

John Wiley and Sons New York

G Kimeldorf A Sampson Uniform Representations of Bivariate Distribution s Communications

in Statistics

G Kimeldorf A R Sampson Positive Dep endence Orderings Annals of the Institute for Statistical

Mathematics A

R E King PNParaskevop oulos Parametric Identication of Discretetime SISO Systems

International Journal of Control

N V Kirianaki Y S Klushin Machine Metho d for Plotting the Probability DensityFunction of a

Resultant Error with two or More Comp onents MeasurementTechniques

L B Klebanov S T Mkrtchyan Estimation of the Closeness of Distribution s in terms of Identical

Moments Journal of Soviet Mathematics

J de Kleer J S Brown A QualitativePhysics Based on Conuences Articial Intelligence

W K Klein Haneveld Robustness against dep endence in PERT An application of duality and

distributio ns with known marginals Mathematical Programming Study

E P Klement Op erations on Fuzzy Sets and Fuzzy Numb ers Related to Triangular Norms in

Pro ceedings of the th IEEE International Symp osium on MultipleValued Logic

D E Knuth Seminumerical Algorithms The Art of Computer Programming Volume Addison

Wesley Reading Massachusetts Second Edition

S Knuuttila Reforging the Great Chain of Being Studies in the History of Mo dal Theories D Riedel

Dordrecht

V V Koblin Sto chastic Programming D Riedel Dordrecht

J Kohlas Conditional Belief Structures Probability in the Engineering and Informational Sciences

J Komlos On the Determinant of Matrices Studia Scientiarum Mathematicarum Hungarica

On the Determinant of Random Matrices Studia Scientiarum Mathematicarum Hungarica

B O Ko opman The Axioms and Algebra of Intuitive Probabili ty Annals of Mathematics

The Bases of Probabili ty Bulletin of the American Mathematical So ciety

T Ko ornwinder A New Pro of of a PaleyWeiner typ e theorem for the Jacobi Transform Arkiv fur

Matematik

B Korel The Program Dep endence Graph in Static Program Testing Information Pro cessing Letters

I Kotlarski On the Generalized Mellin Transform of a Complex Random Variable and its Applica

tions Annals of Mathematical Statistics

E Kreyszig Intro ductory Functional Analysis with Application s John Wiley and Sons New York

S A Kripke A Completeness Theorem in Mo dal Logic Journal of Symb olic Logic

H Krips Popp er Prop ensities and Quantum Theory British Journal for the Philosophy of Science

R Kruse K D Meyer Statistics with Vague Data D Riedel Dordrecht

W Kruskal Miracles and Statistics The Casual Assumption of Indep endence Journal of the Amer

ican Statistical Asso ciation

S Kullback The Distributio n Laws of the Dierence and QuotientofVariables Indep endently Dis

tributed in Pearson typ e I I I laws Annals of Mathematical Statistics

Probability Densities with Given Marginals Annals of Mathematical Statistics

A Kumar T L Fine Stationary Lower Probabiliti es and Unstable Averages Zeitschrift fur

Wahrscheinli chkeitsth eori e und Verwandte Gebiete

F C Kung H Lee Solution of Linear StateSpace Equations and Parameter Estimation in Feed

back Systems using Laguerre Polynomial Expansion Journal of the Franklin Institute

W Kuo V R R Uppuluri A Review of Error Propagation Analysis in Systems Micro electroni cs

and Reliabil ity

R E Kurth D C Cox Discrete Probability Distribution s for Probabilisti c Fracture Mechanics

Risk Analysis

V P Kuznetsov Choice of Co ecient for Calculating the Interval Estimate of Measurement Error

MeasurementTechniques

H Kwakernaak Fuzzy Random Variables I Information Sciences

Fuzzy Random Variables I I Information Sciences

M Kwiesielewicz L Stolc Solving Linear Programming Problems with Nondeterministi c Con

straints using Fuzzy Numb ers Arithmetic International Journal of Systems Science

H E Kyburg Jr The Logical Foundations of Statistical Inference D Reidel Dordrecht

H E Kyburg Jr Bayesian and NonBayesian Evidential Up dating Articial Intelligence

H J Kyburg Jr Knowledge in Uncertainty in Articial Intelligence J F Lemmer L N Kanal

eds Elsevier Science Publishers BV NorthHolland Amsterdam

A R Lacey Mo dality in A Dictionary of Philosophy A R Lacey ed Routledge and Kegan Paul

London

T L Lai H Robbins A Class of Dep endent Random Variables and Their Maxima Zeitschrift fur

Wahrscheinli chkeitsth eori e und verwandte Gebiete

L M Laita Bo olean Algebra and its Extralogical Sources The Testimony of Mary Everest Bo ole

History and Philosophy of Logic

D Lamb S M Easton Multiple Discovery The PatternofScientic Progress Avebury

H E Lamb ert Measures of Imp ortance of Events and Cut Sets in Fault Trees in Reliabili ty and

Fault Tree Analysis Theoretical and Applied Asp ects of Systems Reliabil i ty and System Assessment

R E Barlow J B Fussell N D Singpurwall a eds SIAM Philadelph ia

K C Land Principles of Path Analysis in So ciologi cal Metho dology E F Borgatta ed

JosseyBass San Francisco

S L Lauritzen D J Spiegelhal ter Lo cal Computations with Probabili ties on Graphical Structures

and their Application to Exp ert Systems Journal of the Royal Statistical So cietySeriesB

S L Lauritzen N Wermuth Graphical Mo dels for Asso ciations Between Variables Some of Which

are Qualitative and Some Quantitative Annals of Statistics

S E Laveuve Denition Einer KahanArithmetik Und Ihre Implementierung in Interval Mathe

matics Lecture Notes in Computer Science KNickel ed SpringerVerlag Berlin

H Leblanc Probabili sti c Semantics for FirstOrder Logic Zeitschrift fur mathematische Logik und

Grundlagen der Mathematik

Probabilisti c Semantics An Overview in Essays in Epistemology and Semantics H Leblanc

R Stern R Gumb eds Haven Publication s New York

E L Lehmann Some Concepts of Dep endence Annals of Mathematical Statistics

R Leipnik Integral Equations Biorthonormal Expansions and Noise Journal of the So ciety for

Industrial and Applied Mathematics

W Leler Constraint Programming Languages AddisonWesley Reading Massachusetts

S Lem The UpsideDown Evolution in One Human Minute Harcourt Brace Jovanovich San Diego

J F Lemmer Condence Factors Empiricism and the DempsterShafer Theory of Evidence in

Uncertainty in Articial Intelligence L N Kanal J F Lemmer eds Elsevier Science Publishers

BV NorthHolland Amsterdam

J F Lemmer L N Kanal eds UncertaintyinArticialIntelligence I I Elsevier Science Publishers

BV NorthHolland Amsterdam

K S Leung W Lam Fuzzy Concepts in Exp ert Systems IEEE Computer

I Levi On Indeterminate Probabiliti es The Journal of Philosophy

The Enterprise of Knowledge An EssayonKnowledge Credal Probabili ty and Chance MIT

Press Cambridge Massachusetts

Imprecision and Indeterminancy in Probability Judgement Philosophy of Science

PLevy Calcul des Probabilites GauthierVillars Paris

J S Lew On Some Relations Between the Laplace and Mellin Transforms IBM Journal of Research

and Development

D Lewis Probabiliti es of Conditionals and Conditional Probabiliti es The Philosophica l Review

D V Lindley Scoring Rules and the Inevitability of Probability International Statistical Review

With discussion

M Loeve Probability Theory I SpringerVerlag New York

J S Long Covariance Structure Mo dels An Intro duction to LISREL Sage University Series on

Quantitative Application s in the So cial Sciences Sage Publication s Beverly Hills California

R P Loui Interval Based Decisions for Reasoning Systems in Uncertainty in Articial Intelligence

L N Kanal J F Lemmer eds Elsevier Science Publishers BV NorthHolland Amsterdam

Computing Reference Classes in Uncertainty in Articial Intelligence J F Lemmer L

N Kanal eds Elsevier Science Publishers BV NorthHolland Amsterdam

M J Loux Mo dality and Metaphysics in The Possible and the Actual Readings in the Metaphysics

of Mo dality M J Loux ed Cornell University Press Ithaca

A O Lovejoy The Great Chain of Being Harvard University Press Cambridge Massachusetts

J N Lovett Evaluation of the Hfunction Inversion Integral PhD thesis University of Arkansas

R Lowen Editorial Mathematics and Fuzziness Part Fuzzy Sets and Systems

J D Lowrance Automating Argument Construction Journal of Statistical Planning and Inference

D G Luenb erger QuasiConvex Programming SIAM Journal of Applied Mathematics

Optimization byVector Space Metho ds John Wiley and Sons New York

M K Luhandjula On Possibili sti c Linear Programming Fuzzy Sets and Systems

J Lynch G Mimmack F Proschan Uniform Sto chastic Orderings and Total Positivity The

Canadian Journal of Statistics

N A Lynch StraightLine Program Length as a Parameter for Complexity Analysis Journal of

Computer and System Sciences

D MacHale George Bo ole His Life and Work Bo ole Press Dublin

D P Madan J C Owings Decision Theory with Complex Uncertainties Synthese

G D Makarov Estimates for the distribution function of a sum of two random variables when the

marginal distributio ns are xed Theory of Probability and its Applications

C L Mallows Generalization of Tchebyche s Inequalities with discussion Journal of the Royal

Statistical So ciety Series B

E H Mamdami Application of Fuzzy Algorithms for the Control of Simple Dynamic Plant Pro

ceedings of the IEE

E G Manes Review of Fuzzy sets and systems Theory and application s by Didier Dub ois and

Henri Prade Bulletin New Series of the American Mathematical So ciety

A Class of Fuzzy Theories Journal of Mathematical Analysis and its Applications

E G Manes M A Arbib Algebraic Approaches to Program Semantics SpringerVerlag New York

B Manhire R T Jenkins H R Outhred D Sutanto R J Kaye F J Evans The Cumulant

Metho d of Power System Probabilisti c Simulation Principles Mathematics and Limitations Journal

of Electrical and Electronics Engineering Australia

H B Mann A Wald On the Choice of the Numb er of Class Intervals in the Application of the

Chi Square Test Annals of Mathematical Statistics

M Mares J Horak Fuzzy Quantities in Networks Fuzzy Sets and Systems

R MartinClouaire H Prade On the Problems of Representation and Propagation of Uncertainty

in Exp ert Systems International Journal of ManMachine Studies

Y Masuda J G Shanthikumar U Sumita A General Software Availabili tyReliabilityModel

Numerical Exploration via the Matrix Laguerre Transform Working Pap er Series No QM The

Graduate Scho ol of Management The UniversityofRochester

A M Mathai A Review of the DierentTechniques used for Deriving the Exact Distribution s of

Multivariate Test Criteria SankhyaSeriesA

A M Mathai R K Saxena On the Linear Combinations of Sto chastic Variables Metrika

Generalized Hyp ergeometric Functions with Applications in Statistics and Physical Sciences

SpringerVerlag Berlin Lecture Notes in Mathematics

The H function with Applications in Statistics and Other Disciplin es Halstead Press New

York

G Matheron Random Sets and Integral Geometry Wiley New York

Y Matijasevic Enumerable Sets are Diophantine Doklady Akademii Nauk SSSR

J Mau On Coxs Condence Distribution in Probability and Bayesian Statistics International

Symp osium on ProbabilityandBayesian Statistics Insbruck R Vierti ed Plenum Press

New York

D G Mayo In Defense of the NeymanPearson Theory of Condence Intervals Philosophy of Science

M Mazumdar Imp ortance Sampling in Reliabil ity Estimation in Reliabili ty and Fault Tree Analysis

Theoretical and Applied Asp ects of Systems Reliabili ty and System Assessment REBarlow J B

Fussell N D Singpurwalla eds SIAM Philadelp hi a

R A McCain Fuzzy Condence Intervals Fuzzy Sets and Systems

D J McConalogue Convolution Integrals Involving Probability Distributio n Functions Computer

Journal

Numerical TreatmentofConvolution Integrals Involving Distribution s with Densities having

Singulari ties at the Origin Communications in Statistics Simulation and Computation B

J McCully The Laguerre Transform SIAM Review

D V McDermott E Davis Planning Routes Through Uncertain Territory Articial Intelligence

J L Mehlb erg A Classication of Mathematical Concepts Synthese

K Menger Statistical Metrics Pro ceedings of the National Academy of Science United States of

America

Probabili sti c Geometry Pro ceedings of the National Academy of Science United States of

America

The Theory of Relativity and Geometry in Alb ert Einstein PhilosopherS cientist PA

Schilpp ed Tudor Publishi ng Company New York See also Einsteins reply on page

On Variables in Mathematics and in Natural Science British Journal for the Philosophyof

Science

Calculus A Mo dern Approach Ginn and Company Boston

Random Variables from the Point of View of a General Theory of Variables in Pro ceedings

of the Third Berkeley Symp osium on Mathematical Statistics and Probabili tyII J Neyman ed

University of California Press Berkeley and Los Angeles

Mensuration and Other Mathematical Connections of Observable Material in Measurement

Denitions and Theories CWChurchman P Rato osh eds John Wiley and Sons New York

Variables Constants Fluents in Current Issues in the Philosophy of Science HFeigl G

Maxwell eds New York Reprinted in Karl Menger Selected Pap ers in Logic and Foundations

Dialectics Economics D Riedel Dordrecht

Mathematical Implications of Machs Ideas Positivistic Geometry The Clarication of Func

tional Connections in Ernst MachPhysicist and Philosopher R S Cohen R J Seeger eds D

Riedel Dordrecht Boston Studies in the Philosophy of Science VI

Geometry and Positivism A Probabili stic Microgeometry in Karl Menger Selected Pap ers

in Logic and Foundations Dialectics Economics D Riedel Dordrecht

A Counterpart of Occams Razor in Karl Menger Selected Pap ers in Logic and Foundations

Dialectics Economics D Riedel Dordrecht

N Metrop olis Metho ds of Signicance Arithmetic in The State of the Art in Numerical Analysis

D Jacobs ed Academic Press London

Summation of Imprecise Numb ers Computers and Mathematics with Applications

H MilicerGruzewsk a Larithetique des variables aleatories Cahiers Rho daniens

A C Miller I I I M M Merkhofer R A Howard J E Matheson T R Rice Development

of Automated Aids for Decision Analysis Final technical rep ort DO Stanford Research

Institute Menlo Park California

R von Mises Probability Statistics and Truth nd revised English edition George Allen and Unwin

Ltd London

M Miyakoshi M Shimb o A Strong Law of Large Numb ers for Fuzzy Random Variables Fuzzy

Sets and Systems

M Mizumota K Tanaka The Four Op erations of Arithmetic on Fuzzy Numb ers Systems Com

puters and Controls

M Mizumoto Pictorial Representations of Fuzzy Connectives Part Cases of tnorms tconorms

and Averaging Op erators Fuzzy Sets and Systems

C A Montgomery The Active Information System A DataDriven System for the Analysis of Im

precise Data in Pro ceedings of the th International Conference on Very Large Data Bases Cannes

France IEEE Computer Press Los Angeles

A S Mo ore Interval Risk Analysis of Real Estate Investment A NonMonte Carlo Approach

Freiburger IntervallBerichte

R E Mo ore Interval Analysis PrenticeHall Englewo o d Clis NJ

Metho ds and Applications of Interval Analysis SIAM Philadelp hi a

Risk Analysis without Monte Carlo Metho ds Freiburger IntervallBerichte

PAP Moran Path Co ecients Reconsidered Australian Journal of Statistics

An Intro duction to Probability Theory Clarendon Press Oxford

C G Morgan Probabili sti c Semantics for Prop ositional Mo dal Logics in Essays in Epistemology

and Semantics H Leblanc R Stern R Gumb eds Haven Publications New York

TheTrueb eliever the Unbeliever and a New Mo dal Semantics in Essays in Epistemology

and Semantics H Leblanc R Stern R Gumb eds Haven Publications New York

N Morrison Smo othing and Extrap olation of Time Series by Means of Discrete Laguerre Polynomi

als SIAM Journal of Applied Mathematics

Smo othing and Extraploation of Continuous Time Series using Laguerre Polynomials SIAM

Journal of Applied Mathematics

R Moynihan On the Class of Semigroups of Probabili ty Distribution Functions Aequationes

Mathematicae

Innite pro ducts of Distribution Functions Journal of the Australian Mathematical

So ciety Series A

On Semigroups of Probabili ty Distribution Functions I I Aequationes Mathematicae

Conjugate Transforms and Limit Theorems for Semigroups Studia Mathematica

Conjugate Transforms for Semigroups of Probability Distribution Functions Journal of

Mathematical Analysis and Applications

R Moynihan B Schweizer A Sklar Inequalities Among Op erations on Probability Distribution

Functions in General Inequalities I Pro ceedings of the st International Conference on General

Inequalities EFBet al ed Birkauser Basel

B Muckenhoupt Mean Convergence of Hermite and Laguerre Series I Transactions of the American

Mathematical So ciety

Conjugate Functions for Laguerre Expansions Transactions of the American Mathematical

So ciety

Mean Convergence of Hermite and Laguerre Series I I Transactions of the American Math

ematical So ciety

A Muir Fuzzy Sets and Probability Kyb ernetes London

Relations Between Probability and Fuzzy Theory in The Mathematics of Fuzzy Systems

A DiNola A G S Ventre eds Verlag Tuv Rheinland Koln Interdiscipli nar y Systems

Research ISR

J W Muller Some Second Thoughts on Error Statements Nuclear Instruments and Metho ds

The Assignment of Uncertainties to the Results of Exp erimental Measurements in Precision

Measurement and Fundamental Constants I I BNTaylor W D Phillips eds National Bureau of

Standards United States Sp ecial Publicatio n

E J Muth The Discrete Laguerre Polynomials and their use in Exp onential Smo othing IEE Trans

actions

S Nahmias Fuzzy Variables Fuzzy Sets and Systems

U S Nair The Application of the MomentFunction in the Study of Distributio n Laws in Statistics

Biometrika

I Na jfeld Linear Transformations of Vector Sets and their Applications in Interval Mathematics

KLENickel ed Academic Press New York

I Na jfeld R A Vitale PJDavis Minkowski Iteration of Sets Linear Algebra and its Application s

P S Nasimov Ub er eine Guasssche Abhandlung Nachr Warsh Univ Cited as by

Sheynin Arch Hist Exact Sci

B Natvig PossibilityVersus Probabili ty Fuzzy Sets and Systems

R B Nelsen Discrete Bivariate Distributions with Given Marginals and Correlation Communica

tions in Statistics Simulation

A J Nelson D M Rasmuson A Correction to S Kaplans Metho d of Discrete ProbabilityDistri

butions Risk Analysis

H Nguyen On Random Sets and Belief Functions Journal of Mathematical Analysis and Applica

tions

H T Nguyen A Note on the Extension Principle for Fuzzy Sets Journal of Mathematical Analysis

and Applications

W L Nicholson On the Distribution of Random Normal Determinants Annals of Mathematical

Statistics

K Nickel Zeros of Polynomial s and Other Topics in Topics in Interval Analysis E Hansen ed

Clarendon Press Oxford

TriplexAlgol and its Applications in TopicsinInterval Analysis E Hansen ed Oxford

University Press Oxford

J Nieminen On the Algebraic Structure of Fuzzy Sets of Typ e Kyb ernetika

N J Nilsson Probabili stic Logic Articial Intelligence

A W Niukkanen Generalised Hyp ergeometric Series F x x arising in Physical and Quantum

Chemical Applications Journal of Physics A Mathematical and General

ClebschGordontyp e Linearisation Relations for the Pro ducts of Laguerre Polynomial s and

HydrogenlikeFunctions Journal of Physics A Mathematical and General

J Norton Limit Theorems for Dempsters Rule of Combination Theory and Decision

Y Nurges Y Yaakso o Laguerre State Equations for a Multivariate Discrete System Automation

and Remote Control

H J Nussbaumer Fast Fourier Transform and Convolution Algorithms nd Edition SpringerVerlag Berlin

H Nyquist S O Rice J Riordan The Distribution of Random Determinants Quarterly of

Applied Mathematics

D Oakes A Mo del for Asso ciation in Bivariate Survival Data Journal of the Royal Statistical

So ciety Series B

J K Ord Families of Frequency Distributi ons Grin London

R H Orth A R Ferguson eds The Journals and Miscellaneous Noteb o oks of Ralph Waldo Emer

son Belknap Press Harvard

D Pang J Bigham E H Mamdani Reasoning with Uncertain Information IEE Pro ceedings

A Papamarcou T L Fine A Note on Undominated Lower Probabili ties The Annals of Probabili ty

A Pap oulis Probability Random Variables and Sto chastic Pro cesses McGrawHill New York

Error Analysis in Sampling Theory Pro ceedings of the IEEE

G Parisi N Sourlas A Simple Parametrization of the Q Dep endence of the Quark Distribution s

in QCD Nuclear Physics B

J K Patel C H Kapadia D B Owen Handb o ok of Statistical Distribution s Marcel Dekker New

York and Basel

R S Pathak Summabili ty of the Laguerre Polynomial Expansion of Generalized Functions Journal

of the Institute of Mathematics and its Applications

J Pearl Fusion Propagation and Structuring in Belief Networks Articial Intelligence

On Evidential Reasoning in a Hierarchy of Hyp otheses Articial Intelligence

Evidential Reasoning Using Sto chastic Simulation of Causal Mo dels Articial Intelligence

Distributed Revision of Comp osite Beliefs Articial Intelligence

A ConstraintPropagation Approach to Probabili stic Reasoning in Uncertainty in Articial

Intelligence J F Lemmer L N Kanal eds Elsevier Science Publishers BV NorthHolland

Amsterdam

Probabilistic Reasoning in Intelligent Systems Networks of Plausible Inference Morgan Kauf

mann Los Altos California

J Pearl M Tarsi Structuring Causal Trees Journal of Complexity

E S Pearson Some Problems arising in Approximating to Probability Distribution s using Moments

Biometrika

K Pearson Contributions to the Theory of Evolution On a Form of Spurious Correlation which

may arise when Indices are used in the Measurement of Organs Pro ceedings of the Royal So cietyof

London

M S Petkovic A Representation of Random Variables by the Probable Intervals Freiburger Intervall

Berichte

PPetrina R Sexsmith R N White Safety Analysis of Nuclear Concrete Containment Structures

US Nuclear Regulatory Commision Washington NUREGCR RD

M Pichat All Possible Computed Results in Correct FloatingPoint Summation Mathematics and

Computers in Simulation

A Plantinga De Re et De Dicto Nous

H Poincare Calcul des Probabilites Paris

Science and Hyp othesis The Science Press Lancaster

The Foundations of Science The Science Press Lancaster

S D Poisson Memoire sur la probabili te dutiralacible Memorial de lartillerie

K R Popp er The Prop ensityInterpretation of the Calculus of Probabili ty and the Quantum The

ory in Observation and Interpretation A Symp osium of Philosophers and Physicists SKorner ed

Butterworths Scientic Publication s London

The Prop ensityInterpretation of Probabili ty British Journal for the Philosophy of Science

Ob jective Knowledge An Evolutionary Approach Oxford University Press Oxford

Conjectures and Refutations Routledge and Kegan Paul London

The Logic of Scientic Discovery th EdHutchinson London

The Op en Universe An Argument for IndeterminismRowman and Littleeld Totowa New

Jersey

Realism and the Aim of Science Rowman and Littleeld Totawa New Jersey

B Prabhala R Sethi Ecient Computation of Expressions with Common Sub expressions Journal

of the Asso ciation of Computing Machinery

H Prade Mo dal Semantics and Fuzzy Set Theory in Fuzzy Set and Possibility Theory Recent

Developments RRYager ed Pergamon Press New York

R D Prasad Probability Distributions of Algebraic Functions of Indep endent Random Variables

SIAM Journal of Applied Mathematics

A Prekopa On Random Determinants I Studia Scientiarum Mathematicarum Hungarica

Y V Prohorov Y A Rozanov Probability Theory Basic Concepts Limit Theorems Random

Pro cesses SpringerVerlag Berlin

J J QuesadaMolina Measures of Dep endence Based on the Empirical Copula Aequationes Math

ematicae Rep ort of the Twentysixth International Symp osium on Functional Equations

J R Quinlan Inferno A Cautious Approach to Uncertain Inference The Computer Journal

A L Ralescu D A Ralescu ProbabilityandFuzziness Information Sciences

D Ralescu A Survey of the Representation of Fuzzy Concepts and its Applications in Advances in

Fuzzy Set Theory and Applications M M Gupta R K Ragade R R Yager eds NorthHolland

Amsterdam

Toward a General Theory of Fuzzy Variables Journal of Mathematical Analysis and Appli

cations

A Ramakrishnan A New Concept in Probabili ty Theory Journal of Mathematical Analysis and

Application s

G P Ramsey The Laguerre Metho d for Solving IntegroDierential Equations Journal of Compu

tational Physics

M B Rao A Rashed Some Comments on Fuzzy Variables Fuzzy Sets and Systems

P N Rathie H Kaufman On the Distribution of Pro ducts and Quotients of Indep endent Random

Variables Metron

H Ratschek Die Sub distributi vi tat in der Intervallarithmetik Zeitschrift fur Angewandte Mathe

matik und Mechanik

Representation of Interval Op erations by Co ordinates Computing

N S Rau PToy KFSchenk Exp ected Energy Pro duction Costs by the Metho d of Moments

IEEE Transaction on Power Apparatus and Systems

R L Redmond Egg Size and Laying Date of LongBilled Curlews Numenius Aamericanus Implica

tions for Female Repro ductiveTactics Oikos

A Rege A M Agogino SensorIntegrated Exp ert Systems for Manufacturing and PRo cess Diag

nostics in KnowledgeBased Exp ert Systems for Manufacturing S C Y Lu R Komanduri eds

American So cietyofMechanical Engineers

J H Reif H R Lewis EcientSymb olic Analysis of Programs Journal of Computer and System

Sciences

J H Reif R E Tarjan Symb olic Program Analysis in AlmostLinear Time SIAM Journal of

Computing

A Renyi Foundations of Probability HoldenDay San Francisco

J Richards Bo ole and Mill Diering Persp ectives on Logical Psychologism History and Philosophy

of Logic

H Robbins A Remark on the Joint Distribution of Cumulative Sums Annals of Mathematical

Statistics

P Robillard M Traham The Completion Time of PERTNetworks Op erations Research

G K Robinson Condence Intervals and Regions in Encyclop edia of Statistical Sciences NL

Johnson S Kotz eds John Wiley and Sons New York

R T Ro ckafellar Convex Analysis Princeton University Press Princeton

N Rosen Statistical Geometry and Fundamental Particles Physical Review

Quantum Geometry Annals of Physics

A Rosenthal A Computer Scientist Lo oks at Reliabili ty Computations in Reliabil i ty and Fault Tree

Analysis Theoretical and Applied Asp ects of System Reliabili ty and Safety Assesment R E Barlow

J B Fussell N D Singpurwalla eds SIAM Philadelph ia

L Rothschild The Distributio n of English Dictionary Wordlengths Journal of Statistical Planning

and Inference

N C Rowe Absolute Bounds on the Mean and Standard Deviation of Transformed Data for Constant

SignDerivati veTransformations SIAM Journal of Scientic and Statistical Computation

L Rusc hendorf Sharpness of Frechet Bounds Zeitschrift fur Wahrscheinlichkeits theori e und ver

wandte Gebiete

Inequalities for the Exp ectation of Monotone Functions Zeitschrift fur Wahrscheinli ch

keitstheorie und verwandte Gebiete

Random Variables with Maximum Sums Advances in Applied Probability

Solution of Statistical Optimization Problem by Rearrangement Metho ds Metrika

Construction of Multivariate Distributio ns with Given Marginals Annals of the Institute of

Statistical Mathematics A

MonotonicityandUnbiasedness of Tests via as Constructions Statistics

E H Ruspini Possibili ty Theory Approaches for Advanced Information Systems IEEE Computer

Approximate Inference and Interval Probabiliti es in UncertaintyinKnowledgeBased Sys

tems BBouchonRRYager eds SpringerVerlag Berlin Lecture Notes in Computer

Science

B Russell Vagueness Australasian Journal of Philosophy

R A Sahner K S Trivedi Performance and Reliabil ity Analysis Using Directed Acyclic Graphs

IEEE Transactions on Software Engineering

M Sakai A Characterization of a Class of Polynomial Splines by Convolution BIT

W Salmon Probabili sti c Causality Pacic Philosophi cal Quarterly

T Salvemini Sugli indici di omolia Atti I Riunione della So cieta Italiana di Statistica

S Sambursky The Physical World of the Greeks Routledge and Kegan Paul London

On the Possible and Probable in Ancient Greece Osiris

S K Sarkar W Smith Probability Inequalitie s for Ordered MTP Random Variables Sankhya

O V Sarmanov Z N Brato eva Probabili sti c Prop erties of Bilinear Expansions of Hermite Poly

nomials Theory of Probabili ty and its Applications

M Sarna Fuzzy Relations on Fuzzy and Nonfuzzy Numb ers Fast Computation Formulas Fuzzy

Sets and Systems

S C Saunders Birnbaums Contributions to Reliabil ity Theory in Reliabil ity and Fault Tree Anal

ysis Theoretical And Applied Asp ects of Systems Reliabil ity and Safety Assessment REBarlow J

B Fussell N D Singpurwalla eds So ciety for Industrial and Applied Mathematics Philadelphi a

R Scheines Automating Creativity in Asp ects of Articial Intelligence JHFetzer ed Kluwer

Academic Publishers Dordrecht

I J Scho enb erg Splines and Histograms in Spline Functions and Approximation Theory AMeir

A Sharma eds Birkhauser Basel

K Schuessler Ratio Variables and Path Mo dels in Structural Equation Mo dels in the So cial Sciences

A S Goldb erger O D Duncan eds Seminar Press New York

B Schweizer Distribution Functions Numb ers of the Future in Pro ceedings nd Nap oli Meeting

on The Mathematics of Fuzzy SystemsADiNolaAGSVentre eds Istituto di Matematica

della FacoltadiArchitettura Universita degli Studi di Nap oli

Remark Aequationes Mathematicae Rep ort of the Twentyfth International

Symp osium on Functional Equations

B Schweizer A Sklar Op erations on Distribution Functions not Derivable from Op erations on

Random Variables Studia Mathematica

Probabilisti c Metric Spaces NorthHolland New York

B Schweizer E F Wol On Nonparametric Measures of Dep endence for Random variables

Annals of Statistics

D Scott P Krauss Assigning Probabilitie s to Logical Formulas in Asp ects of Inductive Logic J

HintikkaP Supp es eds NorthHolland Amsterdam

H L Seal The Historical Use of Generating Functions in Probability Theory Bulletin de lAsso cia

tion des Actuaires suisses Reprinted in M Kendall and RL Plackett eds Studies in the

History of Statistics and Probablity Volume I I Charles Grin London

R Sedgewick Quicksort Garland New York

T Seidenfeld Philosophica l Problems of Statistical Inference D Riedel Dordrecht

On AfterTrial Prop erties of Best NeymanPearson Condence Intervals Philosophy of Sci

ence

R P Selke A Rewriting Semantics for Program Dep endence Graphs in Conference Record of the

Sixteenth Annual ACM Symp osium on Principles of Programming Languages Asso ciation for Com

puting Machinery

W Sendler On the Second Moments of Random Determinants Studia Scientiarum Mathematicarum

Hungarica

B Sendov Some Topics of Segment Analysis in Interval Mathematics KLENickel ed

Academic Press New York

R Sethi J D Ullman The Generation of Optimal Co de for Arithmetic Expressions Journal of

the Asso ciation for Computing Machinery

R D Shachter Evaluating Inuence Diagrams Op erations Research

Intelligent Probabilisti c Inference in Uncertainty in Articial Intelligence L N Kanal J

F Lemmer eds Elsevier Science Publishers BV NorthHolland Amsterdam

Probabilis tic Inference and Inuence Diagrams Op erations Research Submitted

G Shafer A Mathematical Theory of Evidence Princeton University Press Princeton

A Theory of Statistical Evidence in Foundations of Probability Theory Statistical Inference

and Statistical Theories of Science Volume I I WLHarperCAHooker eds D Riedel Dordrecht

NonAdditive Probabilities in the Work of Bernoulli and Lamb ert Archive for History of

Exact Sciences

Allo cations of Probability Annals of Probability

Nonadditive Probability in Encyclop edia of Statistical Sciences N L Johnson S Kotz

eds John Wiley and Sons New York

The Combination of Evidence International Journal of Intelligent Systems

Belief Functions and Possibili ty Measures in Analysis of Fuzzy Information Volume

Mathematics and Logic J C Bezdek ed CRC Press Incorp orated Bo ca Raton Florida

G Shafer R Logan Implementing Dempsters Rule for Hierarchical Evidence Articial Intelli

gence

M Shah Some Results on the H function Involving the Generalized Laguerre Polynomials Pro ceed

ings of the Cambridge Philosophica l So ciety

M Shaked A Family of Concepts of Dep endence for Bivariate Distributions Journal of the American

Statistical Asso ciation

S Shao Fuzzy SelfOrganizing Controller and its Application for Dynamic Pro cesses Fuzzy Sets

and Systems

H Sherwo o d M D Taylor Some PM Structures on the set of Distributio n Functions Revue

Roumaine de Mathematiques Pures et Appliquees

O B Sheynin Newton and the Classical Theory of Probability Archive for History of Exact Sciences

Finite Random Sums A Historical Essay Archive for History of Exact Sciences

D Bernoullis work on Probability RETE Strukturgeschichte der Naturwissenschaften

Reprinted in M Kendall and RL Plackett eds Studies in the History of Statistics and

Probablity Volume I I Charles Grin London

Mathematical Treatment of Astronomical Observations A Historical Essay Archive for

History of Exact Sciences

SD Poissons Work in Probability Archive for History of Exact Sciences

CF Gauss and the Theory of Errors Archive for History of Exact Sciences

G E Shilov B L Gurevich Integral Measure and Derivative A Unied ApproachDover New

York

A W Shogan Bounding Distribution s for Sto chastic PERT Network Networks

J A Shohat J E Tamarkin The Problem of Moments American Mathematical So ciety New York

R M Shortt Strassens Marginal Problem in Two or More Dimensions Zeitschrift fur Wahrschein

lichkeitstheo rie und verwandte Gebiete

G E Sigal A A B Pritsker J Solb erg The Sto chastic Shortest Route Problem Op erations

Research

B W Silverman Kernel Density Estimation using the Fast Fourier Transform Applied Statistics

Density Estimation for Statistics and Data Analysis Chapman and Hall London and New

York

H A Simon Spurious Correlation A Causal Interpretation Journal of the American Statistical

Asso ciation

C E Skinner A Numerical Inversion of the PollaczekKhintchi ne Formula in rd National Meeting

of the Op erations ResearchSociety of America San Francisco May Brief summary in

Bulletin of the Op erations Research So ciety of America Supplement B

A Sklar Random Variables Joint Distribution Functions and Copulas Kyb ernetika

B Skyrms Higher Order Degrees of Belief in Prosp ects for Pragmatism Essays in Honour of FP

Ramsey D H Mellor ed Cambridge University Press Cambridge

P Smets Belief Functions versus ProbabilityFunctions in Uncertainty and Intelligent Systems B

Bouchon L Saitta R R Yager eds SpringerVerlag Berlin Lecture Notes in Computer

Science

C A B Smith Consistency in Statistical Inference and Decision Journal of the Royal Statistical

So ciety Series B with discussion

Personal Probabili ty and Statistical Analysis Journal of the Royal Statistical So ciety Series

A with discussion

G C Smith The Bo oleDe Morgan Corresp ondence Clarendon Press Oxford

J M Smith F W J Olver D W Lozier ExtendedRange Arithmetic and Normalized Legendre

Polynomial s ACM Transactions on Mathematical Software

J Q Smith Belief Nets and Inuence Diagrams What they are and howtheywork Rep ort

Department of Statistics UniversityofWarwick to app ear in the Pro ceedings of a Conference on

Inuence Diagrams held at the University of California at Berkeley in May

Inuence Diagrams for Statistical Mo delling Rep ort Department of Statistics Univer

sityofWarwick to app ear in Annals of Statistics

Statistical Principles on Graphs Rep ort Department of Statistics UniversityofWarwick

to app ear in the Pro ceedings of a Conference on Inuence Diagrams held at the University of California

at Berkeley in May

M Sobiera jski Probabili sti c Analysis of Load Flow Equations Archiv fur Electrotechnik

O Spaniol Die Distributivi tat in der Intervallarithmetik Computing

A Spacek Note on K Mengers Probabili stic Geometry Czechoslavak Mathematics Journal

Random Metric Spaces in Transactions of the Second Prague Conference on Information

Theory Statistical Decision Functions Random Pro cesses June

T P Sp eed Probabilisti c Risk Assessment in the Nuclear Industry WASH and Beyond in

Pro ceedings of the Berkeley Conference in Honour of Jerzy Neyman and Jack Kiefer Volume LM

L Cam R A Olshen eds Wadsworth Advanced Bo oks and Software Monterey California

S Spielman A Refutation of the NeymanPearson Theory of Testing British Journal for the Phi

losophy of Science

M D Springer The Algebra of Random Variables John Wiley and Sons New York

H M Srivastava On the Pro duct of Two Laguerre Polynomials Bollettino della Unione Matematica

Italiana

H M Srivastava R G Buschman Mellin Convolutions and H function Transformations Ro cky

Mountain Journal of Mathematics

W Stallings Fuzzy Set Theory Versus Bayesian Statistics IEEE Transactions on Systems Man

and Cyb ernertics

I M StaneuMinasian M J Wets A Research Bibliogra phyinStochastic Programming

Op erations Research

R L Stens Error Estimates for Sampling Sums Based on Convolution Integrals Information and

Control

V Strassen The Existence of Probabili ty Measures with Given Marginals Annals of Mathematical

Statistics

R H Stratz H O A Wold Recursiveversus Nonrecursive Systems Econometrica

J P Stremel R T Jenkins R A Sabb W D Bayless Pro duction Costing Using the Cumulant

Metho d of Representing the Equivalent Load Curve IEEE Transactions on Power Apparatus and

Systems

F Stummel Perturbation Theory for Evaluation Algorithms of Arithmetic Expressions Mathematics

of Computation

Rounding Errors in Numerical Solutions of Two Linear Equations in Two Unknowns Math

ematical Metho ds in the Applied Sciences

Rounding Error Analysis of Interval Algorithms Zeitschrift fur Angewandte Mathematik

und Mechanik

K Subrahmaniam On Some Applications of Mellin Transforms to Statistics Dep endent Random

Variables SIAM Journal of Applied Mathematics

M Sugeno An Intro ductory Survey of Fuzzy Control Information Sciences

Editorial Fuzzy Sets and Systems

U Sumita Numerical Evaluation of Multiple Convolutions Survival Functions and Renewal Func

tions for the OneSided Normal Distribution and the Rayleigh Distribution via the Laguerre Trans

formation Working Pap er Series No The Graduate Scho ol of Management The Universityof

Ro chester

Development of the Laguerre Transform Metho d for Numerical Exploration of Applied Prob

ability Mo dels PhD thesis UniversityofRochester

On Sums of Indep endent Logistic and Folded Logistic Variants Journal of Statistical Com

putation and Simulation

The Matrix Laguerre Transform Applied Mathematics and Computation

The Laguerre Transform and a Family of Functions with Nonnegative Laguerre Co ecients

Mathematics of Op eration Research

U Sumita M Kijima On Optimal Bulk Size of SingleServer BulkArrival Queueing Systems

with Setup Times Numerical Exploration via the Laguerre Transform Working Pap er Series No

QM The Graduate Scho ol of Management The UniversityofRochester

The Bivariate Laguerre Transform and Its Applications Numerical Exploration of Bivariate

Pro cesses Advances in Applied Probability

The Laguerre Transform of a Pro duct of Functions with Applications in Alternating Renewal

Theory Working Pap er Series No QM The Graduate Scho ol of Management The Universityof

Ro chester

Numerical Exploration of a Bivariate Lindley Pro cess via the Bivariate Laguerre Transform

Working Pap er Series No QM The Graduate Scho ol of Management The UniversityofRochester

January

Evaluation of Minimum and Maximum of a Correlated Pair of Random Variables via the

Bivariate Laguerre Transform Working Pap er Series No QM The Graduate Scho ol of Manage

ment The UniversityofRochester June

T Sunaga Theory of Interval Algebra and its Application to Numerical Analysis RAAG Memoirs

Resarch Asso ciation of Applied Geometry

P Supp es Probabilisti c Inference and the Concept of Total Evidence in Asp ects of Inductive Logic

J HintikkaP Supp es eds NorthHolland Amsterdam

A Probabilisti c Theory of Causality Acta Philosophi ca Fennica NorthHolland Amster

dam

Probabilsiti c Metaphysics Filososka Studier utgivna av FilososkaForeningen o ch Filososka

Institutionen vid Uppsala Universitet twovolumes

The Measurement of Belief Journal of the Royal Statistical So ciety Series B

with discussion

Testing Theories and the Foundations of Statistics in Foundations of Probability Theory

Statistical Inference and Statistical Theories of Science Volume I I WLHarper CAHooker

eds D Riedel Dordrecht

Probabilis tic Metaphysics Basil Blackwell Oxford

P Supp es M Zanotti On Using Random Relations to Generate Upp er and Lower Probabiliti es

Synthese

A A Sveshnikov Problems in Probability Theory Mathematical Statistics and Theory of Random

Functions Dover New York

J J Sylvester On a Sp ecial Class of Questions on the Theory of Probabilitie s Birmingham British

Asso ciation Rep ort Reprinted as pp in volume I I of his collected pap ers

G Szego Orthogonal Polynomial s th edition American Mathematical So ciety Collo quium Publica

tion Providence Rho de Island

T Szulc On Two Classes of Interval Matrices Freiburger IntervallBerichte

Sucient Criteria for the Nonsingulari tyofInterval Matrices Freiburger IntervallBerichte

T Takagi M Sugeno Derivation of Fuzzy Control Rules from Human Op erators Control Actions

in Pro ceedings of the IFAC Symp osium on Fuzzy Information Kno eldge Representation and Decision

Analysis E Sanchez ed IFACPergamon Press Oxford

W Y TanSPWong On the RoyTiku Approximation to the Distribution of Sample Variances

from Nonnormal Universes Journal of the American Statistical Asso ciation

On Approximating the Central and Noncentral Multivariate Gamma Distributions Com

munications in Statistics Simulation and Computation B

H Tanaka H Ishibuchi Identication of Possibilisti c Linear Systems by Quadratic Memb ership

Functions of Fuzzy Parameters Submitted to Fuzzy Sets and Systems

K L Tang R J Mulholla nd Comparing Fuzzy Logic with Classical Controller Design IEEE

Transactions on Systems Man and Cyb ernetics

R Tanscheit E M Scharf Exp eriments with the use of a RuleBased SelfOrganising Controller

for Rob otics Applications Fuzzy Sets and Systems

R E TarjanMYannakakis Simple Lineartime Algorithms to Test Chordality of Graphs Test

Acyclicity of Hyp ergraphs and Selectively Reduce Acyclic Hyp ergraphs SIAM Journal of Computing

J R Taylor An Intro duction to Error Analysis The Study of Uncertainties in Physical Measurements

University Science Bo oks Mill Valley California

A H Tchen Inequalities for Distributions with Given Marginals Annals of Probability

P S Theo caris A C Chrysakis Numerical Inversion of the Mellin Transform Journal of the

Institute of Mathematics and its Applications

M L Tiku Laguerre Series forms of NonCentral and F Distribution s Biometrika

R M Tong L A App elbaum Exp eriments with IntervalValued Uncertainty in Uncertaintyin

Articial Intelligence J F Lemmer L N Kanal eds Elsevier Science Publishers BV North

Holland Amsterdam

H Toth From FuzzySet Theory to Fuzzy SetTheory Some Critical Remarks on Existing Concepts

Fuzzy Sets and Systems

G Tsamasphyros P S Theo caris Numerical Inversion of Mellin Transforms BIT

A A Tschuprow Grundb egrie und Grundprobleme der Korrelationstheori e Teubner Leipzig

Principles of the Mathematical Theory of Correlation William Ho dge and Company London

J W Tukey Causation Regression and Path Analysis in Statistics and Mathematics in Biology

O Kempthorne T A Bancroft J W Gowen J L Lush eds Hafner Publishing Company New

York

The Propagation of Errors Fluctuations and Tolerances Basic Generalized Formulas Tech

nical Rep ort No Statistical Techniques Research Group Section of Mathematical Statistics De

partment of Mathematics Princeton University

The Propagation of Errors Fluctuations and Tolerances An Exercise in Partial Dier

entiation Technical Rep ort No Statistical Techniques Research Group Section of Mathematical

Statistics Department of Mathematics Princeton University

The Propagation of Errors Fluctuations and Tolerances Supplementary Formulas Tech

nical Rep ort No Statistical Techniques Research Group Section of Mathematical Statistics De

partment of Mathematics Princeton University

M E Turner C D Stevens The Regression Analysis of Causal Paths Biometrics

V A Ubhaya L Approximation b e Quasiconvex and Convex Functions Journal of Mathematical

Analysis and Applications

K Urbanik Generalized Convolutions Studia Mathematica

A Counterexample on Generalized Convolutions Collo quium Mathematicum

S Ursic Generalizing Fuzzy Logic Probabilis tic Inferences in Uncertainty in Articial Intelligence

J F Lemmer L N Kanal eds Elsevier Science Publishers BV NorthHolland Amsterdam

J V Usp ensky Intro duction to Mathematical Probability McGrawHill New York

J Valdes R E Tarjan E L Lawler The Recognition of SeriesParallel Digraphs SIAM Journal

of Computing

J Van Evra A Reassessment of George Bo oles Theory of Logic Notre Dame Journal of Formal

Logic

T N Verma Real Inversion Formula for the Distribution al Generalised Laguerre Transform Pro

ceedings of the National AcademyofSciencesofIndiaA

T N Verma A B B Lall A Convolution Formula for the Distribution al Laguerre Transform

Indian Journal of Mathematics

J Vignes New Metho ds for Evaluating the Validity of the Results of Mathematical Computations

Mathematics and Computers in Simulation

Review of Sto chastic Approach to RoundO Error Analysis and Its Applications Mathe

matics and Computers in Simulation

F Vo orbraak A Computationally Ecient Approximation of DempsterShafer Theory International

Journal of ManMachine Studies

M Wagenknecht K Hartmann Application of Fuzzy Sets of Typ e to the Solution of Fuzzy

Equations Systems Fuzzy Sets and Systems

S R Wagner On the Quantitative Characterization of the Uncertainty of Exp erimental Results in

Metrology PTBMitteilungen Die PhysikalischTechni sche Bundesanstalt

G Wahba Interp olating Spline Metho ds for Density Estimation I Equispaced Knots Annals of

Statistics

A Wald On a Statistical Generalizatio n of Metric Spaces Pro ceedings of the National Academyof

Science United States of America

PWalley Belief Function Representations of Statistical Evidence Annals of Statistics

PWalley T L Fine Varieties of Mo dal Classifacatory and Comparative Probability Synthese

Towards a Frequentist Theory of Upp er and Lower Probability Annals of Statistics

D Waltz Understanding Line Drawings of Scenes with Shadows in The Psychology of Computer

VisionP Winston ed McGrawHill New York

P Z Wang Fuzzy ContactabilityandFuzzy Variables Fuzzy Sets and Systems

W Warmuth MarginalFrechetBoun ds for Multidimensi on al Distributio n Functions Statistics

H Web er Numerical Computation of the Fourier Transform using Laguerre Functions and the Fast

Fourier Transform Numerische Mathematik

S Web er A General Concept of Fuzzy Connections Negations and Implications Based on Tnorms

and Tconorms Fuzzy Sets and Systems

K Weise Distributi onFree Condence Intervals PTBMitteilungen Die Physikalis ch

Technische Bundesanstalt

ed Preliminary Translation of the German Draft Standard DIN Part th Prop osal

June Basic Concepts of Measurements Treatment of Uncertainties in the Evaluation of Measure

ments Normenausschu Einheiten und Formalgroen Deutsches Institut fur Normung Committee

for Units and Symb ols of the German Institute of Standards Berlin Die PhysikalischTechn ische

Bundesanstalt

Treatment of Uncertainties in Precision Measurements IEEE Transactions on Instrumenta

tion and Measurement

G R West Sensitivity and Probabilisti c Analysis of Regional InputOutput Mo dels PhD thesis

University of Queensland

R J B Wets The Distributio n Problem and its Relation to Other Problems in Sto chastic Program

ming in Sto chastic Programming M A H Dempster ed Academic Press London

A R White Mo dal Thinking Cornell University Press Ithaca

W Whitt Bivariate Distribution s with Given Marginals Annals of Statistics

B Widrow Statistical Analysis of Amplitude Quantized SampledData Systems AIEE Transactions

part I I Application s and Industry

H Wilbraham On the Theory of Chances Develop ed in Professor Bo oles Laws of Thought The

Philosophi cal Magazine series supplement June Reprinted as pp in R Rhees

ed Studies in Logic and Probability by George Bo ole Watts and Company London

J H Wilkinso n Rounding Errors in Algebraic Pro cesses Her Ma jestys Stationery Oce London

Error Analysis Revisited Bulletin of the Institute of Mathematics and its Application s

P W Willi ams On a New Theory of Epistemic Probability British Journal for the Philosophyof

Science

R C Williamson The Law of Large Numbers for Fuzzy Variables under a General Triangular Norm

Extension Principle Fuzzy Sets and Systems submitted

An Extreme Limit Theorem for Dep endency Bounds of Normalised Sums of Random Vari

ables Information Sciences to app ear

Interval Arithmetic and Probabilisti c Arithmetic in Pro ceedings of the IMACSGAMMGI

International Symp osium on Computer Arithmetic and SelfValidating Numerical Metho ds To

app ear Octob er

The Discrete TConjugate Transform In preparation

R C Williamso n T Downs The History of the Problem of the QuotientofTwo Normal Random

Variables In preparation

Probabili sti c Arithmetic Numerical Metho ds for Calculating Convolutions and Dep endency

Bounds International Journal of Approximate Reasoning to app ear

B P Wise Exp erimentall y Comparing Uncertain Inference Systems to Probability in Uncertainty

in Articial Intelligence J F Lemmer L N Kanal eds Elsevier Science Publishers BV North

Holland Amsterdam

B P Wise M Henrion A Framework for Comparing Uncertain Inference Systems to Probability

in Uncertainty in Articial Intelligence L N Kanal J F Lemmer eds Elsevier Science Publishers

BV NorthHolland Amsterdam

L Wittgenstein Preliminary Studies for the Philosophic al Investigations The Blue Bo ok Basil

Blackwell Oxford Reprinted as pp of W Barrett and HD Aiken eds Philosophyin

the Twentieth CenturyVolume Two The Rise of the British Tradition and Contemp orary Analytic

Philosophy Harp er and Row New York

W Woger Condence Limits Resulting from Two Mo dels for the Randomization of Systematic Un

certainties PTBMitteilungen Die PhysikalischTechn isch e Bundesanstalt

Randomizati on of Systematic Errors and its Consequences for the Evaluation of Measure

ments in Quantum Metrology and Fundamental Physical Constants P H Cutler A A Lucas

eds Plenum Press New York

M Wolfenson T L Fine BayesLike Decision Making with Upp er and Lower Probabiliti es Journal

of the American Statistical Asso ciation

E F Wol Measures of Dep endence Derived from Copulas PhD Thesis University of Massachusetts

NDimensional Measures of Dep endence Sto chastica

S Wolfram Undecidabili tyandIntractability in Theoretical Physics Physical Review Letters

M L Wong K S Chen C L Chou Solutions of Integral Equations via Mo died Laguerre

Polynomial s International Journal of Systems Science

S Wright The Nature of Size Factors Genetics

Correlation and Causation Journal of Agricultural Research

The Metho d of Path Co ecients Annals of Mathematical Statistics

The Interpretation of Multivariate Systems in Statistics and Mathematics in Biology O

Kempthorne T A Bancroft J W Gowen J L Lush eds Hafner Publishi ng Company New York

Path Co ecients and Path Regressions Alternative or Complementary Concepts Biometrics

S PYadav Approximation of FourierLaguerre Expansion by its Cesaro Mean in Certain Banach

Spaces Journal of Approximation Theory

R R Yager On a General Class of Fuzzy Connectives Fuzzy Sets and Systems

AnIntro duction to Applications of Possibili ty Theory Human Systems Management

Arithmetic and Other Op erations on DempsterShafer Structures International Journal of

ManMachine Studies

On the DempsterShafer Framework and New Combination Rules Information Sciences

A V Yazenin Fuzzy and Sto chastic Programming Fuzzy Sets and Systems

R C Young The Algebra of ManyValued Quantities Mathematische Annallen

L A Zadeh Fuzzy Sets Information and Control

The Concept of a Linguistic Variable and its Application to Approximate Reasoning Part

I Information Sciences Part I I Information Sciences Part I I I Information

Sciences

Fuzzy Sets versus Probability Pro ceedings of the IEEE

Possibili ty Theory as a Basis for Representation of Meaning in Language and Ontology

Pro ceedings of the Sixth International Wittgenstein Symp osium W Leinfellner E Kraemer J

Schank eds HolderPichlerTempsky Vienna

Is Probability Theory Sucient for Dealing with Uncertainty in AI A Negative View in

Uncertainty in Articial Intelligence L N Kanal J F Lemmer eds Elsevier Science Publishers

BV NorthHolland Amsterdam

J Zackova On Minimax Solutions of Sto chastic Linear Programming problems Casopis pro pestovani

matematikyroc

M Zeleny On the Irrelevancy of Fuzzy sets Theories Human Systems Managemant

H J Zimmermann Application s of Fuzzy Set Theory to Mathematical Programming Information

Sciences

V M Zolotarev MellinStiel tjes Transforms in Probability Theory Theory of Probability and its

Application s

I quickly found it to exercise more than my devotion it

exercised my skil l al l I had it exercisedmypatience it

exercised my friends too for tis incomparably the hardest

taske that ever I yet undertooke

Williams Watts Rector of St Albans Wo o d Street

Sources of Motto es

Page Mrs S Boal Grade Oakleigh Primary Scho ol Class Instruction

reconstructed from memory

Page Berthold Schweizer see also p

Page MW Crofton Probability pp in the Encyclop edia Brittan

ica Volume Adam and Charles Black Edinburgh

Page William Kruskal pp

Page Jorge Luis Borges p cited in p

Page Karl Popp er p

Page Stanley Jevons Quoted byMaryEverest Bo ole Logic Taughtby

Love See fo otnote of

Page Glenn Shafer p

Page Theo dore Hailp erin p

Page SL Lauritzen and DJ Spiegelhalter p

Page Glenn Shafer Quoted byPearl in p

Page Peter Walley and Terrence Fine p

Page Henry Kyburg p

Page Karl Popp er p

Page Masaharu Mizumoto and Kokichi Tanaka pp

Page Didier Dub ois and Henri Prade p

Page Alan White p

Page Didier Dub ois and Henri Prade p

Page Clark Glymour p

Page Nicolas Cusanus Nicolas Cusanus De Do cta Ignoran

tia Of Learned Ignorance BookIChapter Translated by Germain Heron

Routledge and Kegan Paul London

Page Karl Menger pp

Page Nicolas Cusanaus Bo ok chapter of De Do cta Ignorantia Of

learned ignorance translated by Germain Heron Routledge and Kegan Paul Lon

don

Page Ralph Waldo Emerson page of his TU journal May reprinted

on page of volume XI of

Page StansilawLemPageofTheWorld as Cataclysm pp in

One Human Minute Harcourt Brace and Jovanovich San Diego

Page Folly Erasmus of Rotterdam The Praise of Folly c English

translation in p

Page Williams Watts Rector of St Albans Wo o d Street Preface to his

translation of St Augustines Confessions London