Bayesian Analysis of Logistic Regression with an Unknown Change Point

Home , Frequentist inference

Gössl, Küchenhoff: Bayesian analysis of logistic regression with an unknown change point

Sonderforschungsbereich 386, Paper 148 (1999)

Online unter: http://epub.ub.uni-muenchen.de/

Projektpartner

Bayesian analysis of logistic regression with an

unknown change point

Christo Gossl Helmut Kuchenho

University of Munich Institute of Statistics

Akademiestrae D Munc hen

Abstract

We discuss Bayesian estimation of a logistic regression mo del with an

unknown threshold limiting value TLV In these mo dels it is assumed

that there is no eect of a covariate on the resp onse under a certain

unknown TLV The estimation of these mo dels with a fo cus on the TLV

in a Bayesian context by Markovchain Monte Carlo MCMC metho ds is

considered We extend the mo del by accounting for measurement error

in the covariate The Bayesian solution is compared with the likeliho o d

solution prop osed by Kuchenho and Carroll using a data set

concerning the relationship between dust concentration in the working

place and the o ccurrence of chronic bronchitis

Keywords threshold limiting value TLV segmented regression mea

surement error MCMC

Intro duction

In toxicology environmental and o ccupational epidemiology the assessment of

threshold limiting values TLVs is an imp ortant task In a doseresp onse relati

onship the TLV is the dose of the toxin or a substance under which there is no

inuence on the resp onse In many applications there is a controversy ab out the

existence of such a TLV from a substantive point of view In empirical studies

evidence for the existence of a TLV and its estimation is often dicult since

distinguishing between no eect and a small eect can only be done by huge

data sets There are dierent mo dels and metho ds for assessing a TLV see eg

Kuchenho and Ulm In this pap er we concentrate on a fully parametric

logistic regression mo del prop osed by Ulm In this mo del which is a seg

mented regression mo del the TLV is treated as an unknown parameter which

can b e estimated assuming its existence The interval estimates of the TLV give

some evidence ab out its existence since a TLV which is smaller than the smallest

observed dose is equivalent to a non existing TLV While the theoretical and prac

tical problems in maximum likeliho o d estimation and the frequentist treatment

of this mo del has b een discussed by Kuchenho and Wellisch we use a

Bayesian approach In this context no dierentiability assumptions are necessary

and it can be implemented with Markov chain Monte Carlo MCMC metho ds

We apply our metho ds to a study concerning the relationship b etween dust con

centration in the working place and the o ccurrence of chronic bronchitis In this

study the exp osure can only be measured with substantial measurement error

Therefore we also showhow to incorp orate this measurement error in our mo del

Since there are dierent approaches and p ossibilities concerning the MCMC al

gorithm and the assumption of the distribution of the regressor variable we give

a detailed discussion of the bronchitis example The results are compared with

those of a frequentist approach

The pap er is organized as follows In Section we present the mo del and a

Bayesian solution of the problem of estimating the limiting value of a logistic

threshold mo del We prop ose a way to calculate the estimates by means of

MCMC metho ds

The mo deling and the handling of measurement error in the dose covariate of our

mo del is treated in Section

In Section we apply our metho ds to analyze in detail an o ccupational study

regarding the assessing of a TLV for dust concentration in the working place

Further our metho ds are compared with the dierent approaches as investigated

b y Kuchenho and Carroll

A Bayesian Approach to the Logistic Thres

hold Mo del

In the following we fo cus our analysis on the logistic threshold mo del prop osed

by Ulm

P Y jX x Z z Gz x

k k

where Gt exp t

maxx IR and x

k k

Here Y denotes the resp onse variable X is the dose variable Z refers to further

covariates The unknown mo del parameters are and the TLV As can b e seen

from there is no inuence of X on Y if X is smaller than which exactly

reects the concept ofaTLV

In contrast to the classical frequentist inference the parameters of a Bayesian

mo del are not supp osed to be x but at random For each of them exists a

probability function which reects the prior knowledge of their value the so

called priors Now it is p ossible according to the theorem of Bayes to determine

in combination with the likeliho o d function of the data a socalled p osterior of

the parameters This p osterior distribution includes all knowledge relating to the

parameters once from the prior and on the other hand from the likeliho o d

The theorem of Bayes in its simplest form runs

p data

p jdata

pdata

Here data denotes the observed and the unknown parameters and latent va

riables The numerator is the pro duct of the likeliho o d and the priors Note

that in contrast to likeliho o d analysis no further assumptions on p data like

dierentiability in are needed for the analysis

From this p osterior the Bayesian p oint and interval estimates are derived The

median or the mean of the partial densities are dep ending on the used loss

function appropriate estimates for the parameters In this pap er we use the

mean of the p osterior as p oint estimate and probability intervals which can be

regarded as Bayesian equivalents to the classical condencesets

Thus to derive the Bayesian p osterior for our logistic threshold mo del we have

to determine the conditional likeliho o d function and the prior distributions

The conditional likeliho o d of the iid sample y z x i n is according

i i i

to given by

Y jZ X

where G z x

i k i k i

As usual refers to the density or probability of the corresp onding random

variables We assume that the threshold is in the range of our observed data X

and use a uniform prior in the range of the observed data for the threshold

For a at prior is assumed

Now the p osterior density of the parameters is

Y jZ X

jY Z X

Y jZ X d

Based on this density the ab ove mentioned estimates are to be calculated Alt

hough the determination of the numerator is easy in most practical cases it is

not p ossible to evaluate the denominator in an analytic way For the threshold

mo del wesolve this problem by means of MCMC metho ds These metho ds allow

to take a sample from a densityonlyknown up to a normalizing constant which

is in our particular problem the denominator Then on the basis of this sample

of the p osterior the Bayesian estimates can simply b e calculated see eg Gilks

Richardson and Spiegelhalter

For the logistic threshold mo del we use a two step Metrop olis Hastings MH

algorithm with multivariate random walk prop osals in each step We sample the

parameter and the threshold in one step and the parameter vector in

k k

step two The full conditionals are straightforward Since the densities cannot

be analytically determined it is not p ossible to apply the GibbsSampler We

take two steps b ecause of the strong dep endence between the threshold and

The covariance matrices of the prop osals are tuned according to test runs to

acceptance rates from to The starting values are chosen overdisp ersed at

random and the burnin phase has to be determined by comparing several runs

and then discarded Due to high auto correlation and slow convergence in the

MHoutput it is often necessary to thin out the simulated chain by taking only

every k th observation into the sample Wecho ose k such that the auto correlation

decreases to a suciently lowlevel The total extend of the runs dep ends on the

convergence of the Markovchains and can b e determined by comparing the p oint

estimates of several runs Figure shows a tra jectory of such a run and the

b elonging histogram with kernel density estimate for a simulated dataset For

results on real data we refer to Section

Figure

Errors in Variables

In many practical regression problems the regressors can only b e measured with

measurement error Here we want to prop ose a solution for incorp orating mea

surement error of the variable X in our Bayesian mo del A general intro duction

to the measurement error problem in threshold mo dels is given by Kuchenho

and Carroll see also Carroll Rupp ert Stefanski

We assume an additive measurement error mo del ie instead of X the variable

W X U is observed where U is the measurement error which is indep endent

of Y X Z and is normally distributed with E U and V U

Now we split our mo del in three parts

main mo del Y j X Z

error mo del W j X

covariable mo del X j Z

where and are the mo del parameters

Using the indep endence assumption of U and Y W X the likeliho o d of the

whole mo del can be written as

y jx z w jx xjz dx Y WjZ

i i i i

where

For our Bayesian approach the underlying mo del is the threshold mo del as

and nally we assume measurement error mo del we dene W j X N X

X to be indep endent from Z and to have a normal distribution This approach

distributions which is a is extended in Section to a nite mixture of normal

exible mo del for X with few parameters

With resp ect to the priors we prop ose as in Section that no additional informa

tion is available and therefore dene again noninformatives for and For the

parameters and of the covariable mo del we assume similar to the nonin

formatives a normal distribution with mean and a very large variance s and a

highly disp ersed inversegamma distribution with parameters and so that

its exp ectation equals innity The use of prop er priors is here advisable as to

avoid improp er p osteriors see e g Besag et al We further assume that

the error variance is known For the p osterior of the unknown parameters we

get by suitable conditional indep endence assumptions

h i i h

jY Z W Y W jZ

x x x

u x u x u x

As in the previous section it is not p ossible to derive the p osterior analytically

Thus we have again to apply the MHalgorithm In addition we have to cop e

with the fact that the integral of formula is in general not evaluable In this

pap er wewant to solve the latter by means of MCMC metho ds to o For another

way of getting analytically the integral which works with an approximation of

the logit by the probitmo del see Carroll Rupp ert Stefanski For the

foundations of the following metho d we refer to Richardson The idea is

to add the unknown variable X to the parameters with a prior according to the

covariable mo del and sample it from its full conditional The partial densities of

the other parameters will not be aected and the integration of formula is

implicitly carried out by the algorithm

Now we describ e the problem more formally As mentioned ab ove we assume

X to b e indep endent from Z and for simplicity distributed according to a single

normal distribution N We regard the latent variables X as parameters

x i

and decomp ose the likeliho o d as follows

Y W jZ X Y jZ XW jX

u u

With the ab ove priors and X N the p osterior takes the form

XjY Z W Y WjZ X Xj

x x x

u x u x x

Y jZ XW jX Xj

x x

u x x

For the MHalgorithm we use as in Section two Metrop olis steps with random

walk prop osals for the parameters and Furthermore we add a Metrop olis

step for X Thus full conditionals are given by

h i

jY Z W X Y jZ X

k k x

u x

h i

jY Z W X Y jZ X

k k x

u x

h i h i

x jy z w y jz x w jx

i i i i x i i i i i

u x u

h i

x j i n

i x

Due to our choice of normal and inversegamma distributions resp ectively for

the parameters and we can derive their full conditionals as

s s x

jX N

s n s n

x x

jX IG x

i x

Concerning the details of the implementation of the algorithm such as xing the

starting values we refer to Section An application of the algorithm is rep orted

in the following section

Bronchitis Study

In several o ccupational studies conducted by the German research foundation

DFG the relationship b etween average dust concentration in the working place

and the o ccurrence of a chronic bronchitis reaction Y has b een investigated

The disease was measured based on medical examinations like a questionnaire

ab out symptoms chest xrays and lung function analysis

Further covariates were smoking SMK and duration of exp osure DUR We

use the data of Munich workers which were also analyzed by Kuchenho

and Carroll It should be mentioned that we use the quantity

X log dustconcentration in our calculations

P Y G SMK DUR X

Because of the concentration measurements were gained by averaging over se

veral single measurements scattered over the p erio d and raw estimates for earlier

p erio ds it seems to b e appropriate not only to apply the simple threshold mo del

but also the mo del of Section

For the simple mo del without measurement error the algorithm of Section can

directly b e used Only the prop osals of the Markov chain have to b e mo died in

the describ ed manner Apart from that we derived our estimates from one chain

with iterations where we to ok due to the high auto correlations every th

observation in our sample With resp ect to the burnin we discarded the rst

iterations Table shows the estimates and those of Kuc henho and Carroll

gained by the classical metho ds While the estimators are nearly identical

the estimators for the variance are higher for the classical metho ds For example

the estimated variance for the threshold was compared to in the Bayes

analysis

Table

In order to takeinto account the measurement error more complex mo dications

of the presented mo del have to be done The rst is to regard the measured

dustconcentration as the errorexp osed surrogate W The true concentration X

is unknown

Following Kuchenho and Carroll we mo del the distribution of the un

known variable X by a mixture of two normal distributions Assuming an additive

measurement error W X U where U N then W is also a mixture of

normals with the same number of mixing distributions as X So taking two mi

xing distributions is justied by the empirical distribution of W Consequently

for we assume X to b e distributed according to

X MixN with

x x

x x x x

x x

The covariable mo del is given ab ove the underlying mo del the logistic threshold

mo del is straightforward and we supp ose an additive measurement error mo del

where W jX x has a normal distribution ie W jX x N x So the

likeliho o d is complete and is given by

Thus the prior distributions are chosen like in the previous section weonlyhave

to take into account that the covariable mo del has more parameters than in the

last section Again we assume the case of no prior information Accordingly

we dene the priors for the additional parameters for which we use the

and the prior same disp ersed normal and gamma distribution as for

for where we use a uniform distribution on Because the integral of the

likeliho o d is not evaluable we also have to consider the dierent prior of the

unknown variable X according to our covariable mo del

With resp ect to a practical solution we assume the variance of the measurement

alue prop osed in Kuchenho and Carroll error as known and take the v

Because of problems with mo del identication other assumptions

like setting another prior distribution on did not work in our mo del

Therefore the p osterior is of the form

XjY Z W

x x

u x x

Y jZ XW jX Xj

x x x x

u x x x x

As far as the derivation of the ab ove formula is quite simple the adapting of the

MHalgorithm is a far more sophisticated task The main problem results from

the determination of the original distribution of the mixture for the variable X

which is necessary for dening the full conditionals of the according parameters

Here we use a metho d which is given in detail in Rob ert and works in

principle by dening an indicatorvariable m which for all observations of X

states from which distribution of the mixture it comes

In general supp ose for X amixtureofm normal distributions is given with para

meters x N Then weights

m m i j j

m j j

i j

g can be calculated that corresp ond to the contributions of

ij j

the several mixing distributions to the densityofx Here x denotes the stan

dard normal density Thus dening a discrete random variable M on the set of

as probabili distributions fmg with the standardized weights p

ties yields avariable that allo cates each element of the mixture a probability of

having generated the observation x Consequently realizations of this distribu

tion assign every observation one particular underlying distribution

After having sampled an origin for every observation the means variances and

mixing parameters of these distributions can be up dated in familiar Gibbs or

MHsteps according to the priors and the observations that come from this dis

tribution In our case we use as ab ove conjugate normal and inv erse gamma

priors for the means and variances and a noninformative prior for the mixing

parameter so that two Gibbs and one MHstep can be applied

Thus the full conditionals for our particular algorithm with the mixture of two

normal distributions are

jY Z W X Y jZ X

x x

u x x

h i

jY Z W X Y jZ X

x x

u x x

h i h i

jY Z W X Xj

x x x x

u x x x x

i h

x jy z w

x x i i i i

u x x

y jz x w jx Xj

i i i i i x x

u x x

for the MHsteps and for Gibbssampling we get

jX N

n n

j j

jX IG x j

j j

P u p

i i

u jX

i x x

x x

P u p

i i

where p is dened as ab ove n denotes the number of observations x coming

i j i

alues themselves from distribution u j and x are their v

Again we derived our estimates from one chain of the length of iterations

where wetookevery th observation into our sample In contrast to the simple

mo del the auto correlations then still had a value of ab out to But with re

gard to the computation time we accepted this high level and the resulting biases

The discarded burnin extended again to iterations Table shows the Bayes

estimates For comparison the estimate for the threshold derived by Kuchenho

and Carroll under the assumption of a mixture of normal distributions

for X where X M ixN and a variance for the

error mo del of takes a value of

Table

Further the classical metho ds with the same assumptions give dierent estimates

for the parameter dep ending on the metho d see Kuc henho and Carroll

While the likeliho o d estimator is the simexestimator Co ok and Stefanksi

which do es not use the information ab out the distribution of X gives a

result of which is close to our result An interesting point is that variance

estimation is higher for the Bayesian approach than it is for all classical metho ds

where the se varies from to A reason could be that in the Bayesian

approach all sources of variability are mo deled automatically while in the classical

approach this is not the case

As mentioned ab ove and are sampled in the same MHstep b ecause of their

high correlation The estimate for this correlation in the simulated dataset of

Section was for the dustdataset it was In Figure a scatterplot

and a twodimensional kernel density estimate of the p osteriors sample of and

for the simple mo del in the bronchitis study also shows this high correlation

Figure

Discussion

We have shown that the Bayesian analysis for the complicated mo del of a seg

mented regression with errors in the regressors can b e done by MCMC metho ds

We use a mixture of normals for the distribution of the regressor variable which

is a exible parametric mo del In this setting a classical analysis is very dicult

both theoretically and from a practical p oint of view Another imp ortant argu

ment for Bayesian analysis for nding threshold limiting values in epidemiology

is the p ossibility of including knowledge from other studies or from substantive

considerations by selecting a suitable prior distribution for the TLV

A p ossible extension of our mo del would b e to drop the assumption of an existing

threshold but to nd out whether there is a threshold by data analysis Another

point is to treat the number of normals in the mixture distribution as a further

parameter For these problems metho ds prop osed by Green have to be

included into the MCMCalgorithm

Acknowledgements

C Gossls research was supp orted by the SFB form the German research

foundation DFG We would like to thank Leonhard KnorrHeld for useful dis

cussions and valuable comments

References

Besag JE Green PJ Higdon D Mengersen K

Bayesian computation and stochastic systems with discussion Statistical

Science

Carroll RJ Rupp ert D Stefanski LA Nonlinear mea

surement error models Chapman Hall New York

Co ok JR Stefanski LA Simulation extrapolation esti

mation in parametric measurement error models Journal of the American

Statistical asso ciation

Gilks WR Richardson S Spiegelhalter DJ Markov

Chain Monte Carlo in Pr actice Chapman Hall London

Green P Reversible Jump MCMC computation and Bayesian

model determination Biometrika

Kuchenho H Carroll R J Segmented Regression with er

rors in predictors Semiparametric and parametric methods Statistics in

Medicine

Kuchenho H Ulm K Comparison of statistical methods for

assessing threshold limiting values in Occupational Epidemiology Compu

tational Statistics

Kuchenho H Wellisch U Asymptotics for generalizedlinear

segmentedregression models with unknown breakpoint Discussion Pap er Nr

SFB Munc hen

Richardson S Measurement error in Gilks Richardson Spie

gelhalter eds Markov Chain Monte Carlo in Practice Chapman Hall

London

Rob ert CP Mixtures of distributions inferenceand estimation

in Gilks Richardson Spiegelhalter eds Markov Chain Monte Carlo in

Practice Chapman Hall London pp

Ulm K A statistical method for assessing a threshold in epide

miological studies Statistics in Medicine

Figure Tra jectory and histogram with kernel density estimate of a MH

algorithm

Param MLestim Bayesestim BayesVar

Table Likeliho o d and Bayesestimates of the simple mo del

Param

x x

Mean

Var

Table Bayes estimates for the dustdataset for the threshold mo del with errors

in variables

Figure ScatterPlot and twodimensional kernel density estimation of the pa

rameters and of the simple mo del of the Bronchitis study

Address for corresp ondence

Christo Gossl

MaxPlanckInstitut fur Psychiatrie

Kraep elinstr

Munc hen

Germany

Tel

Fax

email go esslmpipsyklmpgde