Health Services & Outcomes Research Methodology 1:2 (2000): 149±164 # 2000 Kluwer Academic Publishers, Boston. Manufactured in The Netherlands.

Measurement Errors in Binary Regressors: An Application to Measuring the Effects of Speci®c Psychiatric Diseases on Earnings

ELIZABETH SAVOCA Department of Economics, Smith College, Wright Hall 221, Northampton, MA 01063 email: [email protected]

Received December 10, 1998; revised January 10, 2000; accepted March 2, 2000

Abstract. This paper presents an overview of the theory of measurement error bias in ordinary regression estimators when several binary explanatory variables are mismeasured. This situation commonly occurs in health- related applications where the effects of illness are modeled in a multivariate framework and where health conditions are usually 0±1 survey responses indicating the absence or presence of diseases. An analysis of the effect of psychiatric diseases on male earnings provides an empirical example that indicates extensive measurement error bias even in sophisticated survey measures that are designed to simulate clinical diagnoses. A corrected covariance matrix is constructed from a validity study of the survey mental health indicators. When ordinary least squares estimators are adjusted by this correction matrix, the estimated earnings effects drop for certain diseases (drug abuse, general phobic disorders) and rise for others (anti-social personality).

Keywords: measurement error; binary variables; multiple regression; psychiatric diseases

1. Introduction

The topic of measurement errors in binary explanatory variables has been treated extensively in the theoretical and applied econometrics literature. Aigner (1973) was the ®rst to demonstrate that, unlike the classical case of a mismeasured explanatory variable, the measurement error in a binary regressor, often referred to as `classi®cation error', may have a non-zero mean and is negatively correlated with the true underlying variable. However, questions about whether these deviations from the classical errors-in-variables model lead to different conclusions about the consequences for ordinary least squares (OLS) estimators and about appropriate econometric solutions have not been fully explored. This issue is particularly relevant to empirical studies in health-related areas, where illness is often measured by the presence or absence of a diagnosis. This paper provides an overview of the theory of measurement error in binary regressors and of econometric methods to correct for measurement error bias. It argues the advantages of `modi®ed least squares', a method ®rst proposed by Johnston (1963), over alternative methods. The arguments are geared toward studies for which auxiliary or extra-sample information about the classi®cation error in the reported diagnosis is 150 SAVOCA available. The paper also provides an application of this method to study the effects of psychiatric diseases on earnings. The paper is divided into ®ve main sections. Section 1 presents the basic errors-in- variables model for a simple regression with a binary 0-1 regressor measured with error and derives the inconsistency in the OLS estimator. This section is essentially a synthesis of three previous econometric studies: Aigner (1973), Marquis et al. (1981), and Freeman (1984). Section 2 uses the theoretical results from the basic model to estimate the measurement error bias in OLS estimates of the effects of mental illness. This section focuses on a widely used psychiatric screening instrument, the Diagnostic Interview Schedule, and considers a wide range of psychiatric diseases. Section 3 extends the theoretical analysis to a multivariate framework where more than one binary regressor is measured with error. Section 4 discusses alternative strategies for dealing with measure- ment error bias. Section 5 illustrates the method of modi®ed least squares with an earnings equation estimated using microdata from the U.S. National Institute of Mental Health Epidemiological Catchment Area Survey.

2. Measurement Error Bias in a Binary Health Regressor: The Simple Regression Framework

Consider a simple regression: y bx e where y is a continuous variable, x is a binary variable indicating the presenceˆ or absence‡ of a disease, and e is a population regression error that follows all of the assumptions of the classical model. The outcome variable y can represent any continuous health outcome, such as medical expenditures or earnings. The regressor (x) is measured with error (u) according to the relationship: X x u where X is the diagnosis according to the survey instrument. Let P represent theˆ proportion‡ of persons in the population truly suffering from the disease. De®ne P~ as the proportion of persons diagnosed as having the disease by the survey. Suppose that the survey instrument incorrectly classi®es persons as not having the disease with probability r1 and incorrectly classi®es persons as having the disease with probability r0 . The probability r1 is referred to as the false negative rate; r0 as the false positive rate. X, x, and u can take on four possible combinations of values:

X x u

00 0 10 1 11 0 01 1 À MEASUREMENT ERRORS IN BINARY REGRESSION 151

A non-zero correlation between x and u is evident from computing the probabilities of u conditional on the values of x:

P u 1 x 0 0 ˆÀ j ˆ †ˆ P u 1 x 1 P X 0 x 1 r ˆÀ j ˆ †ˆ ˆ j ˆ †ˆ 1

P u 0 x 0 P X 0 x 0 1 r ˆ j ˆ †ˆ ˆ j ˆ †ˆ À 0 1 P u 0 x 1 P X 1 x 1 1 r † ˆ j ˆ †ˆ ˆ j ˆ †ˆ À 1

P u 1 x 0 P X 1 x 0 r ˆ j ˆ †ˆ ˆ j ˆ †ˆ 0 P u 1 x 1 0 ˆ j ˆ †ˆ

From here, the joint and marginal probabilities of x and u can be easily computed leading to these expressions for E u , VAR u , and COV x; u 1: † † †

E u r r r P †ˆ 0 À 1 ‡ 0† VAR u r r r P r r r P 2 2 †ˆ 0 ‡ 1 À 0† À‰ 0 À 1 ‡ 0† Š † COV x; u r r P 1 P † ˆ À 1 ‡ 0† À †

From the equations in (2) we see that only in the situation where P r0= r0 r1 will E u 0 and, hence, will the sample proportion be an unbiased estimateˆ of the population‡ † proportion, †ˆ i.e., E X E x P. We also see that the measurement error, u, is negatively correlated with the true†ˆ diagnosis, †ˆ x. These are two important deviations from the classical measurement error model. The least squares regression of y on X : y bX e bu yields a slope estimator ^ ˆ ‡ À † bOLS with probability limit:

COV X; e bu P 1 P 1 r r p limb^ b À † b À † À 0 À 1† 3 OLS ˆ ‡ VAR X ˆ P~ 1 P~ † †  À †

Hence, the OLS slope estimator in a simple regression approaches zero as the sum of the errors in classi®cation approach one. What may be less obvious from (3) is the result that ~ ~ the proportional bias term: P 1 P 1 r0 r1 =P 1 P is always less than one in absolute value (Aigner 1973). Furthermore, À † À ifÀ the† sum ofÀ the† error rates exceeds one then the OLS coef®cient is oppositeÂà in sign to the true coef®cient. This situation would occur if more than half of the population were misclassi®ed by the survey instrument (Bollinger 1996). 152 SAVOCA

3. Evidence of Classi®cation Errors in a Survey Indicator of Mental Illness

In the early to middle 1980s, the U.S. National Institute of Mental Health (NIMH) sponsored the Epidemiological Catchment Area (ECA) project. The project surveyed adults in 5 communities about their use of general medical and mental health services, their demographic background, and their employment situation. The survey also included the NIMH Diagnostic Interview Schedule (DIS), a highly structured interview designed for lay-interviewers as well as clinicians. The DIS questioned participants about the occur- rences of symptoms of psychiatric disorders. These responses were run through computer algorithms to simulate clinical diagnoses of speci®c psychiatric diseases according to criteria established by the American Psychiatric Association (1987) in the Diagnostic and Statistical Manual of Mental Disorder (DSM-IIIR). (See Eaton and Kessler 1985.) Although it has its critics (Jenkins et al. 1997, e.g.), the DIS is often regarded as the `gold standard' against which the validity of many survey instruments have been judged (Weinstein et al. 1989; Berwick et al. 1991). These include the General Health Questionnaire and several abbreviated versions of the RAND Mental Health Inventory, the latter serving as the main mental health assessment in the RAND Health Insurance Experiment. The DIS also forms the basis of the Composite International Diagnostic Interview (Robins et al. 1988), a questionnaire designed for worldwide use in both clinical settings and as a screening instrument in epidemiological surveys, such as the U.S. National Comorbidity Survey. Validation studies of the DIS were conducted at two ECA sites: St. Louis and Baltimore (Anthony et al. 1985, Helzer et al. 1985a). At both sites physicians were asked to reexamine a sample of ECA participants who had recently been given the DIS by lay- interviewers. Physicians were required to follow a structured format and obtain diagnoses according to DSM-III criteria.2 However, they were free to elicit from the subjects any information they deemed necessary to arrive at their diagnoses using their best clinical judgment. This study design served a twofold purpose. It insured that the full range of diseases diagnosed by the DIS could be evaluated. It also minimized variations in physician diagnoses resulting from a physician's failure to notice important clinical details and from a physician's use of highly idiosyncratic criteria in making diagnoses. In clinical settings studies of the agreement between two psychiatrists' assessments, using speci®ed criteria and structured interviews, show the same concordance as that between clinicians interpreting results from objective diagnostic tests such as X-rays and EKGs (Helzer et al. 1985b). Columns 1 and 2 of Table 1 report, for 11 psychiatric diseases, the error rates in the lifetime DIS diagnoses that are implied by the ®ndings of the St. Louis DIS validation study. They are computed for the St. Louis population of working adult men. Since most psychiatric diseases are relatively rare in the population, with the exception of alcoholism, the proportion of healthy individuals receiving positive DIS diagnoses is fairly low. This leaves the impression of a high degree of accuracy in the DIS. However, again with the exception of alcoholism, the DIS has a high failure rate in detecting disorders among persons suffering from psychiatric diseases. This implies that the DIS prevalence rates for this population of men may substantially understate the true rates as determined by a MEASUREMENT ERRORS IN BINARY REGRESSION 153

Table 1. Misclassi®cation rates in diagnoses from the lay diagnostic interview schedule

Misclassi®cation Rates Sample Estimated Prevalence Population False Positive False Negative Rate Proportion OLS Bias1 ~ Disease Rate: r0 Rate: r1 P P (Percent) Drug Dependence=Abuse 2.2% 54.9% 5.3% 7.1% 43.1% À Alcohol Dependence=Abuse 39.7% 16.7% 29.8% 24.3% 61.8% À Anti-Social Personality 3.8% 42.8% 7.6% 7.1% 49.7% À Somatization 0.1% 97.9% 0.2% 3.4% 60.5% À Panic Disorders 0.1% 98.1% 0.2% 4.0% 59.5% À Major Depression 0.6% 93.9% 1.7% 20.5% 45.9% À Agoraphobia 1.0% 83.4% 1.9% 5.5% 56.0% À Social Phobia 0.6% 82.4% 1.0% 4.9% 60.0% À Simple Phobia 2.9% 96.8% 4.1% 8.2% 71.6% À Obsessive-Compulsive Disorder 0.6% 96.8% 0.7% 2.2% 91.8% À Notes: The misclassi®cation rates are derived from the raw data reported in Table 5 of Helzer et al. (1985b). See Appendix A2 for more details. 1The OLS bias is p lim b^ b=b, where b^ is from a simple regression: y bX e and X is a 0±1 dummy OLS†À OLS ˆ ‡ variable indicating the presence or absence of the disorder according to the lay DIS. See equation (3) of the text. physician assessment. The third column of Table 1 reports for each disease the sample proportion of working men receiving positive DIS diagnoses (P).~ Column 4 reports the estimated proportion of the working male population truly suffering from the disease (P).3 The contrast between the two is most striking for major depression whose prevalence rate rises from less than 2% to over 20%, after adjusting for error rates in the DIS. Among the more common DIS diagnoses in this sample (drug abuse, alcohol abuse, and anti-social personality disorders), the bias in the DIS prevalence rates is relatively small. The last column of Table 1 reports the proportional bias in the OLS coef®cient from a simple regression whose explanatory variable is a dummy variable indicating the presence or absence of the disorder according to the lay DIS. Given the high estimated error rates for this subpopulation the downward bias is substantial. Most bias estimates fall within the range of 40±60%, with a few considerably higher and none lower.

4. Assessing Measurement Error Bias in a Multivariate Framework

Section 2 was an illustration designed to provide a sense of how the magnitudes of the misclassi®cation rates in a dummy explanatory variable affect the size of the measurement error bias in the ordinary regression estimator. Of course, an analysis of the earnings effect of psychiatric diseases should be carried out in a multivariate framework controlling simultaneously for all types of disorders as well as for non-health factors. The econometric literature on classical measurement error bias in multiple regression estimation has established that, when more than one regressor is measured with error, the direction of bias is generally indeterminate. 154 SAVOCA

A simple transformation of the measurement error model for a binary regressor (X x u) into a model that follows the classical properties can best illustrate this pointˆ and‡ others that follow. The equations in (1) imply that E u x is a function of x: E u x r r r x. Therefore, the conditional expectation j of† X is: E X x j †ˆ 0 À 1 ‡ 0† j †ˆ a bx where a r0 and b 1 r0 r1. Consequently, X can be expressed as: X‡ a bx v whereˆ v, theˆ expectationÀ À error: v X E X x , has a zero mean, ˆ ‡ 2 ‡ ˆ À4 j † variance sv, and is orthogonal to x. That is, COV x; v 0. Consider a three variable regression model: †ˆ

y b x b x e 4 ˆ 1 1 ‡ 2 2 ‡ † where e follows the classical regression properties and both x1 and x2 are binary regressors measured with error by the variables X1 and X2, respectively. By analogy, X1 and X2 can be written as:

X a b x v and X a b x v 5 1 ˆ 1 ‡ 1 1 ‡ 1 2 ˆ 2 ‡ 2 2 ‡ 2 †

Solving (5) for x1 and x2 and substituting the resulting expressions into (4), gives us the regression equation expressed in observable variables:

y aà bÃX bÃX eà 6 ˆ ‡ 1 1 ‡ 2 2 ‡ † where, bà b =b , bà b =b , aà bÃa bÃa , and eà e bÃv bÃv . 1 ˆ 1 1 2 ˆ 2 2 ˆÀ 1 1 À 2 2 ˆ À 1 1 À 2 2 Let d1 b1 VAR v1 =VAR X1 , d2 b2 VAR v2 =VAR X2 ,andr CORR X ; Xˆ . Then,Á assuming † that v† and vˆareÁ uncorrelated, † the probability† limits ofˆ 1 2† 1 2 the OLS estimators of b1à and b2à can be written as (Maddala 1992):

SD X2 b1Ãd1 b2Ãd2r † ^ À SD X1 p lim bà bà 2 †3 7 1†ˆ 1 À 1 r2 † À 6 7 6 7 4 5

SD X1 b2Ãd2 b1Ãd1r † ^ À SD X2 p lim bà bà 2 †3 8 2†ˆ 2 À 1 r2 † À 6 7 6 7 4 5 Focusing on the bias term for a single coef®cient one can see that, as long as X1 and X2 are correlated r 0 , nothing constrains either of the biases to be positive or negative. This leads to the conclusion6ˆ † that the direction of the measurement error bias cannot be assessed a priori in a multivariate framework. However, taken together the theoretical results do impose some constraints on the direction of the biases in a regression model with 2 mismeasured regressors. Without loss MEASUREMENT ERRORS IN BINARY REGRESSION 155

of generality, assume that b1 > 0, b2 > 0 and that 0 < r < 1. Also suppose that ^ p lim b1à > b1Ã). Thus, from (7), b1Ãd1 < b2Ãd2rSD X2 =SD X1 . Multiplying both sides of † † 2 this inequality by rSD X1 =SD X2 gives us: b1Ãd1rSD X1 =SD X2 < b2Ãd2r . Since 2 † † † † 0 < r < 1, then b2Ãd2r < b2Ãd2. Hence, b1Ãd1rSD X1 =SD X2 < b2Ãd2. Therefore, the † ^ à † second term in (8) is positive so that we can say that p lim b2 < b2Ã. That is, at least one OLS coef®cient has a negative bias. Garber and Klepper (1980) derive a similar result for a regression model with 2 continuous mismeasured regressors.

5. Methods to Correct for Measurement Error Bias

Since measurement error introduces a correlation between the explanatory variables and the regression error, the obvious solution for obtaining consistent parameter estimates is the instrumental variables (IV) technique. Examples in health and labor include Stern (1989) who uses binary variables indicating the presence or absence of health-related work limitations as instruments for self-reported binary indicators of health in a model of male labor force participation. Ettner et al. (1997) use variables re¯ecting family history of psychiatric diseases as instruments for binary indicators of the presence or absence of a current psychiatric diagnosis in employment, hours worked, and income equations. When the mismeasured regressor is a binary variable, however, IV will not eliminate the inconsistency in the regression estimator. This point can be seen with reference to equations (4), (5), and (6). In the context of this model, legitimate instruments for X1 and X2 are variables that are correlated with the true underlying health statuses, x1 and x2, but are uncorrelated with the measurement errors, v1 and v2, as well as with the regression error e. Even if such instruments could be found, at best an IV estimator will provide consistent estimates not of b1 and b2, the parameters of interest, but rather of b1Ã and b2Ã, parameters which may deviate from b1 and b2 substantially if the survey instrument is subject to high misclassi®cation rates. With data on misclassi®cation rates, however, IV estimators can, of course, be rescaled to provide consistent estimates of the population parameters. In the absence of legitimate instruments, misclassi®cation rates can be incorporated into the likelihood function for the regression equation to obtain consistent and asymptotically ef®cient estimates via the method of maximum likelihood. However, the likelihood function involves a linear combination of the density function of the regression equation with the number of terms doubling with each additional mismeasured binary regressor. With the usual assumption of normality in the regression error, this method of estimation becomes intractable. Since information on the validity of survey responses is often unavailable, Klepper (1988) has derived consistent bounds for the true slope coef®cient which do not require precise estimates of the misclassi®cation rates but do require the assumption that the false negative (r0) and false positive (r1) rates equal each other and are each less than 0.5. Bollinger (1996) modi®es Klepper (1988) by providing bounds under less restrictive assumptions about the errors in classi®cation. His empirical application of this bounding method, however, suggests that, under the least restrictive assumptions about misclassi- 156 SAVOCA

®cation rates, the technique produces bounds that are too wide to provide any meaningful information about the true parameter.5 In the same paper Bollinger computes narrower bounds by placing upper limits on the rates of misclassi®cation but these limits are usually exceeded in population-based survey indicators of health conditions.6 The modi®ed least squares estimator (MLS), ®rst proposed by Johnston (1963) for econometric applications, requires extra-sample information on the rates of misclassi®ca- tion. Although less ef®cient than the maximum likelihood estimator it has the advantage in computational simplicity and robustness. Analyses based on this approach can be found in labor market studies. Freeman (1984) corrects OLS estimates of the effect of union status on wages using the employer±employee matched ®les from the 1977 and 1978 Current Population Survey. Rates of misclassi®cation are inferred from the deviations between employer and employee responses to questions about collective bargaining status. Krueger and Summers (1988) use the same data and take a similar approach in accounting for measurement error in industrial classi®cations in a study of inter-industry wage differentials. The estimator is easy to motivate in a general framework. Consider the population regression model Y Xb E; the measurement model: W X U and the correspond- ˆ ‡ ˆ ‡ ing regression based on observable variables: Y Wb E Ub. Y is an N 1 vector of observations on the dependent variable; W isˆ N K‡ matrixÀ of observations on the K explanatory variables in X; E is an N 1 vector of stochastic errors; and U is an N K matrix containing measurement errors in W.  The ordinary regression estimator of the parameter vector b can be written as: ^ 1 1 bOLS I W0W À W0U b W0W À W0E. Assuming that E is independent of X and ˆ À † † ‡ † ^ U, the probability limit of the OLS estimator can be expressed as: p lim bOLS I O b 1 †ˆ À † where O wwÀ wu and where AB denotes the covariance matrix between variables A and B.ˆ Using sampleÁ moments to estimate and external data to estimate , the P P P ww wu ordinary regression estimator can be adjusted to obtain a consistent estimator of b: ^ ^ 1 ^ P ^ P bMM I O À bOLS. An estimate of the standard error of bMM can be computed via the deltaˆ methodÀ (Greene† 1997, page 27).

6. Adjusting for Measurement Error Bias in an Earnings Equation: Results

Table 2 contrasts OLS estimates of an earnings equation to MLS estimates. The latter corrects for measurement error in the diagnostic variables for ten psychiatric disorders.7 The disease indicators are 0±1 dummy variables which take on a value of 1 if the respondent met diagnostic criteria for the disorder at any point in his lifetime according to the lay interview DIS; 0 if not. For the modi®ed least squares estimator, estimates of WU are derived from Helzer et al.'s (1985a) validation study conducted at the St. Louis site. Details of this derivation can be found in the Appendix A3. P The sample is restricted to employed men who participated in the St. Louis ECA.8 The OLS estimates predict that three psychiatric diseases have negative earnings effects. Drug abuse=dependency lowers earnings by 48%; anti-social personality by 29%; and general phobic disorders by 35%.9 After correcting for measurement error bias, however, the MEASUREMENT ERRORS IN BINARY REGRESSION 157

Table 2. Estimates of male earnings equations

OLS Estimation Modi®ed Least Squares Estimation

Coef®cient Coef®cient Explanatory Variable Estimate t-statistic Estimate t-statistic

Psychiatric Diseases Depression 0.084 0.388 0.896 0.964 Alcohol Abuse=Dependency 0.073 1.109 0.129 0.347 a À À Drug Abuse=Dependency 0.647 4.813 0.669 1.273 À a À a Anti-social Personality Disorder 0.338 2.972 1.909 3.656 À À À À Obsessive-Compulsive Disorder 0.042 0.121 0.313 1.478 À À b Agoraphobia 0.319 1.447 0.655 1.720 À À À À Social Phobia 0.510b 1.729 0.028 0.164 a À À General Phobic Disorders 0.429 2.816 0.096 0.712 À À Somatization 0.519 0.757 0.057 0.229 Panic Disorders 0.495 0.740 1.010 0.666 Other Health Conditions Cognitive Impairments 0.028 0.140 0.060 0.301 À À À b À Chronic Medical Conditions (1: present; 0.523 1.534 0.588 1.736 À À À À 0: not present) Non-health Variables Age 0.119a 7.849 0.133a 8.860 Age Squared 0.001a 7.844 0.002a 9.069 À À À À White 0.172a 2.628 0.236a 3.739 Married 0.340a 5.425 0.345a 5.554 Number of children 0.025 1.314 0.004 0.242 À a À À a À Years of Schooling 0.066 5.581 0.040 3.207 Income from Other Family Members 0.004 1.487 0.000 1.231 À À À À ($000s)

Dependent Variable: Log of Annual Earnings. ap 4 0.05; b0.05 < p 4 0.10. Sample size 590. ˆ estimated effects of drug abuse and phobic disorders switch signs and become statistically insigni®cant. The impact of anti-social personality disorders, however, becomes even more negative and retains its statistical signi®cance. Measurement errors in the diagnostic variables also have consequences for the coef®cients of non-health variables. In particular, after adjusting for measurement error, inferences about the impact of race and schooling are substantially altered. The unadjusted OLS estimates imply a 6.6% rate of return to an additional year of schooling and an 18% wage premium for white males; the adjusted estimates raise the premium to 27% and lower the returns to schooling to 4%.

7. Conclusions

This paper provides an overview of the theory of measurement error bias in ordinary regression estimators when several binary explanatory variables are mismeasured. This situation commonly occurs in health-related applications where the effects of illness are 158 SAVOCA modeled in a multivariate regression framework and where health conditions are usually 0±1 survey responses indicating the absence or presence of diseases. Unlike the classical measurement error model for continuous explanatory variables, the classi®cation error in a binary variable is negatively correlated with the true underlying health status of the respondent. This correlation, effectively, rules out the use of instrumental variables as a method to eliminate classi®cation error bias from regression estimators. The paper provides an application of an alternative estimation strategy, modi®ed least squares. This method relies on the use of auxiliary or extra-sample information about the classi®cation errors in the survey health indicators, information which is often collected in epidemiological studies (Anthony et al. 1985; Helzer et al. 1985a; Marquis et al. 1986; USDHHS 1994). The auxiliary information can come from a medical record or a direct assessment by a physician. The modi®ed least squares estimator is used to correct for measurement error bias in the estimated earnings effects of poor mental health. Recent econometric studies of the effect of mental illness on labor market outcomes have relied upon large-scale population surveys, which were designed to estimate the incidence of speci®c psychiatric diseases in the general population (Mullahy and Sindelar 1989, 1993, 1994; Ettner et al. 1997). The surveys, conducted by lay-interviewers, are well known for the sophisticated design of their psychiatric screening instruments. However, validation studies have found large discrepancies between the survey diagnoses and direct physician evaluations. This paper ®nds that when OLS coef®cients are adjusted for these misclassi®cations in the survey diagnoses, the estimated earnings effects drop for some diseases (e.g. drug abuse and general phobic disorders) and rise for others (e.g. anti-social personality). Error rates as high as those reported in Table 1 for the survey instrument analyzed in this paper are not limited to survey diagnoses of psychiatric diseases. Evaluation of the diagnostic reporting in the National Health Interview Survey, a household survey that has been the basis of many studies of health-related behavior, has found that the reported incidences of many common physically disabling diseases, such as arthritis, orthopedic impairments, and asthma, differ signi®cantly from the incidences implied by the medical records of the survey participants.10 Hence the potential for substantial measurement error bias in estimating the effects of speci®c diseases may be widespread.

Acknowledgements

Financial support for this research came from the U.S. National Institute of Mental Health Grant MH49865 and from the VA Connecticut-Massachusetts Mental Illness Research, Education, and Clinical Center (MIRECC). I thank Petya Koeva for her able research assistance. MEASUREMENT ERRORS IN BINARY REGRESSION 159

Appendix

A1. Two Equivalent Formulations of a Measurement Error Model for a Binary Variable

Consider the simple regression of y on X: y bx e, where x is a 0±1 indicator denoting the absence or presence of a disease, y is theˆ outcome‡ variable, and e is an error which conforms to all of the properties of the classical model. The explanatory variable x is measured with error by the survey diagnosis, X (1 positive, 0 negative). E x P, the proportion of the population truly suffering fromˆ the disease,ˆ and VAR x †ˆP 1 P . E X P,~ the proportion of the population receiving positive diagnoses by†ˆ the surveyÀ † instrument, †ˆ and VAR X P~ 1 P~ . The relationship between †ˆ x andÀ † X can be expressed in two ways. Section 1 sets X x u, where u is the residual error. De®ne r as the false positive rate, that is, ˆ ‡ 0 P u 1 x 0 , and r1 as the false negative rate, i.e., P u 1 x 1 . Then, the joint probabilities ˆ j ˆ of† x and u can easily be computed: ˆÀ j ˆ †

P u 1 and x 0 0 ˆÀ ˆ †ˆ P u 1 and x 1 P u 1 x 1 P x 1 r P ˆÀ ˆ †ˆ ˆÀ j ˆ †Á ˆ †ˆ 1 Á

P u 0 and x 0 P u 0 x 0 P x 0 1 r 1 P ˆ ˆ †ˆ ˆ j ˆ †Á ˆ †ˆ À 0†Á À † A1:1 P u 0 and x 1 P u 0 x 1 P x 1 1 r P † ˆ ˆ †ˆ ˆ j ˆ †Á ˆ †ˆ À 1†Á

P u 1 and x 0 P u 1 x 0 P x 0 r 1 P ˆ ˆ †ˆ ˆ j ˆ †Á ˆ †ˆ 0 Á À † P u 1 and x 1 0 ˆ ˆ †ˆ

This leads to these marginal probabilities for u:

P u 1 r P ˆÀ †ˆ 1 Á P u 0 1 r 1 P 1 r P A1:2 ˆ †ˆ À 0†Á À †‡ À 1†Á † P u 1 r 1 P ˆ †ˆ 0 Á À † and to these expressions for E u , VAR u , COV x; u : † † †

E u r r r P †ˆ 0 À 1 ‡ 0† VAR u r r r P r r r P 2 A1:3 †ˆ 0 ‡ 1 À 0† À‰ 0 À 1 ‡ 0† Š † COV x; u r r P 1 P † ˆ À 1 ‡ 0† À † 160 SAVOCA

The probability limit of the slope estimator from a regression of y on X: y bX e 7 bu is: ˆ ‡

COV X; e bu p limb^ b À † OLS ˆ ‡ VAR X † COV X; u b b † ˆ À VAR X † VAR X COV X; u b †À † ˆ VAR X  † VAR x u COV x u; u b ‡ †À ‡ † ˆ VAR X  † VAR x COV x; u b †À † ˆ VAR X  † P 1 P 1 r r b À † À 0 À 1† A1:4 ˆ P~ 1 P~ †  À †

In section 3, for expository purposes, the model is re-written as: X a bx v where ˆ ‡ ‡ a r0 and b 1 r0 r1. From the joint probability distribution derived in (2), we can deriveˆ the jointˆ probabilityÀ À distribution of the random variables X, x, and v:

X x v p X; x; v † 00 a 1 r 1 P À À 0† À † 10 1ar1 P À 0 À † 111a b 1 r P À ‡ † À 1† 01 a b r P À ‡ † 1

Substituting for a and b we have that

E v r 1 r 1 P 1 r r 1 P r 1 r P r 1 r P 0 †ˆÀ0 À 0† À †‡ À 0† 0 À †‡ 1 À 1† ‡ 1 À † 1 ˆ COX x; v E x v E x E v r 1 r P r 1 r P P 0 0 A1:5 †ˆ Á †À †Á †ˆ 1 À 1† ‡ 1 À † 1 À Á ˆ † VAR v r 1 r 1 P r 1 r P †ˆ 0 À 0† À †‡ 1 À 1†

In this version of the measurement error model the regression of y on X can be written as: y aà bÃX e bÃv, where bà b=b and aà bÃa. ˆ ‡ ‡ À ˆ ˆÀ MEASUREMENT ERRORS IN BINARY REGRESSION 161

Following the steps in A1.4, the probability limit of the slope estimator can be derived as:

COV X; e bÃv p limb^ à bà À † OLS ˆ ‡ VAR X † VAR a bx v COV a bx v; v bà ‡ ‡ †À ‡ ‡ † ˆ VAR X  † b2VAR x bà † A1:6 ˆ VAR X †  † b b2P 1 P À † ˆ b P~ 1 P~  À † P 1 P 1 r r b À † À 0 À 1† ˆ P~ 1 P~  À †

A2. Method of Estimating Bias Terms

Helzer et. al (1985a) provide the raw data from a study of a sample of St. Louis ECA participants which compared their lay-DIS diagnoses to a physician assessment. The data is presented for each of 11 psychiatric diseases in a two-by-two table:

DIS Diagnosis

‡À Physician AB ‡ Diagnosis CD À

According to equation (3), to compute the bias estimates reported in Table 1 we need, P,~ the proportion of the population who would receive positive lay-DIS diagnoses; P, the proportion of the population which, in fact, have the disease according to a physician assessment; r0, the DIS false-positive rate; and r1, the DIS false-negative rate. In their comparison study, Helzer et al. oversampled individuals who had received positive DIS diagnoses, i.e., people who fell into cells A and C. Consequently, C= C D ‡ † and B= A B are biased estimates of r0 and r1, respectively. Similarly, A C = A B C‡ D† and A B = A B C D are biased estimates of the population ‡ † proportions ‡ ‡ ‡P~ and† P, respectively. ‡ † The‡ proportion‡ ‡ † P,~ however, can be estimated directly from the ECA sample. I estimate the proportion P indirectly in the following manner. Let f1 denote the probability that a person receiving a negative lay-DIS diagnosis truly has the 162 SAVOCA

disease and let f0 denote the probability that a person who receives a positive lay-DIS diagnosis does not actually have the disease. Then, assuming that for each disease these probabilities are constant across individuals, f1 and f0 can be estimated by B= B D and C= A C , respectively. I then estimate P according to the relationship: ‡ † ‡ † P 1 f P~ f 1 P~ ˆ À 0† ‡ 1 À †

Finally, I estimate r1 and r0 from the relationships:

P~ 1 P~ r f and r À † f 0 ˆ 1 P Á 0 1 ˆ P Á 1 À

A3. Method of Estimating the Covariance Terms for the Modi®ed Least Squares Estimator

~ 1 ~ ~ The modi®ed least squares estimator is bMLS I O À bOLS, where bOLS is the OLS 1 ˆ À † estimator and O WWÀ wu. The matrix, WW, the covariance matrix of the observed explanatory variablesˆ (including those measured with errors), can be estimated from the sample covariance matrix.P P P

An estimate of the matrix Wu, the matrix of the covariances between the explanatory ^ variables and their measurement errors can be derived from ww and the error rates from the validation study in the followingP way. First, consider the ith, jth element of where w x P u and w x u ;x and WW i ˆ i ‡ i j ˆ j ‡ j i xj are the true health status variables; and ui and uj are measurement errors. Extending the arguments from Section 3 we have thatP u c d x v and that u c d x v i ˆ i ‡ i i ‡ i j ˆ j ‡ j j ‡ j where (suppressing subscripts) c r0 and d r0 r1 . The variables, vi and vj are ˆ ˆ À ‡ † 2 2 independent random variables with zero means and variances si and sj , each independent of both xi and xj. These relationships imply the following covariances:

A3:1 cov w ; w cov x ; x cov x ; u cov u ; x cov u ; u i j†ˆ i j†‡ i j†‡ i j†‡ i j† A3:2 cov x ; u cov x ; c d x v d cov x ; x i j†ˆ i j ‡ j j ‡ j†ˆ j i j† A3:3 cov u ; x d cov x ; x i j†ˆ i i j† A3:4 cov u ; u cov c d x v ; c d x v d d cov x ; x : i j†ˆ i ‡ i i ‡ i j ‡ j j ‡ j†ˆ i j i j† Substituting A3.2, A3.3, and A3.4 into A3.1 we have that

A3:5 cov w ; w cov x ; x 1 d d d d i j†ˆ i j† ‡ i ‡ j ‡ i j†

Now consider terms related to the ith, jth element of Wu: A3:6 cov w ; u cov x u ; u covP x ; u cov u ; u i j†ˆ i ‡ i j†ˆ i j†‡ i j† MEASUREMENT ERRORS IN BINARY REGRESSION 163

Substituting A3.2 and A3.4 into A3.6 yields:

A3:7 cov w ; u d cov x ; x d d cov x ; x d d d cov x ; x i j†ˆ i i j†‡ i j i j†ˆ i ‡ i j† i j† Solving for cov x ; x from A3.5 and substituting the solution into A3.7 gives us the ®nal i j† expression for the ith, jth element of Wu:

P d d d cov w ; w A3:8 cov w ; u i ‡ i j† i j† i j†ˆ 1 d d d d ‡ i ‡ j ‡ i j† The ith, jth element of provides an estimate of cov w ; w . The error rates WW i j† reported in Table 1 provide estimates of di and dj. Pe

Notes

1. See Appendix A1 for derivations of the results stated in this section. 2. St. Louis used an augmented version of the DIS; Baltimore used an augmented version of the Present State Examination. See Eaton and Kessler (1985), Chapters 13 and 14 for more details. 3. See the Appendix A2 for details on the derivation of these rates. 4. See Appendix A1 for a demonstration of this result. 5. For example, Bollinger estimates that the wage premium associated with being a union member ranges between 23% and 239%. 6. Bollinger's worst case scenario is a false positive rate as high as 20% and a false negative rate as high as 13% (1996). 7. Cognitive impairment is included as an explanatory variable measured without error. Helzer et al. (1985a) did not provide data on the errors rates for the survey diagnosis of this disease. However, they did report that the ECA screening instrument for dementia was unable to distinguish between dementia and other sources of mental de®ciency such as mental retardation. Since mental de®ciencies should have negative effects on worker productivity, it is included as an explanatory variable. Similarly, a dummy variable for the self- reported presence of a chronic medical condition is also included. 8. Estimates for women were also computed. However, neither the OLS estimates nor the MLS estimates predict that psychiatric illnesses in¯uence earnings in any way. These results are available to the reader on request. 9. The percentage drop in earnings is computed according to the formula: exp b 1, where b measures the †À effect of the disorder on the log of earnings.

10. False positive rates r0 and false negative rates r1 sum to 50%, 88%, and 50% for arthritis, orthopedic impairments, and asthma, respectively. See Table 7, page 45, of U.S. Department of Health and Human Services (U.S.DHHS 1994).

References

Dennis Aigner. ``Regression With a Binary Independent Variable Subject to Errors of Observation.'' Journal of Econometrics 1: pp. 49±60, 1973. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 3rd edition, revised. Washington D.C.: American Psychiatric Association, 1987. James C. Anthony, Marshal Folstein, Alan J. Romanoski, Michael R. Von Korff, Gerald R. Nestadt, Raman Chahal, Altaf Merchant, Hendricks Brown, Sam Shapiro, Morton Kramer and Ernest M. Gruenberg. 164 SAVOCA

``Comparisons of the DIS and a Standardized Psychiatric Diagnosis.'' Archives of General Psychiatry 42 (July): pp. 667±675, 1985. D. Berwick, Jane M. Murphy, Paula A. Goldman, John E, Ware, Arthur J. Barsky and Milton Weinstein. ``Performance of a Five-Item Mental Health Screening Test.'' Medical Care 29: pp. 169±176, 1991. Christopher R. Bollinger. ``Bounding Mean Regressions When a Binary Regressor is Mismeasured.'' Journal of Econometrics 73: pp. 387±399, 1996. William Eaton and Larry Kessler, editors. Epidemiological Methods in Psychiatry: The NIMH Epidemiologic Catchment Area Program. New York: Academic Press. 1985. Susan L. Ettner, Richard G. Frank and Ronald C. Kessler. ``The Impact of Psychiatric Disorders on Labor Market Outcomes.'' Industrial and Labor Relations Review 51(1): pp. 64±81, 1997. Richard B. Freeman. ``Longitudinal Analysis of the Effects of Trade Unions.'' Journal of Labor Economics 2: pp. 1±26, 1984. Steven Garber and Steven Klepper. ``Extending the Classical Normal Errors-In-Variables.'' Econometrica 48: pp. 6, 1980. William H. Greene. Econometric Analysis. Saddle River, NJ: Prentice Hall, 1997. John Helzer, Lee N. Robin, Larry T. McEvoy, Edward L. Spitznagel, Roger K. Stoltzman, Anne Farmer and Ian F. Brockington. ``A Comparison of Clinical and Diagnostic Interview Schedule Diagnoses.'' Archives of General Psychiatry 42: pp. 657±666, 1985a. John Helzer, Roger K. Stoltzman, Ann Farmer, Ian F. Brockington, Dennis Plesons, Burton Singerman and Jane Works. ``Comparing the DIS with a DIS=DSM-III-Based Physician Reevaluation.'' In Eaton, William and Larry Kessler, eds., Epidemiological Methods in Psychiatry: The NIMH Epidemiologic Catchment Area Program. New York: Academic Press, 1985b. R. Jenkins, P. Bebbington, T. Brugham, M. Farrell, B. Gill, G. Lewis, H. Meltzer and M. Petticrew. ``The National Psychiatric Morbidity Surveys of Great Britain-strategy and methods.'' Psychological Medicine 27: pp. 765± 774, 1997. J. Johnston. Econometric Methods. New York: McGraw-Hill, 1963. Steven Klepper. ``Bounding the Effects of Measurement Error in Regressions Involving Dichotomous Variables.'' Journal of Econometrics 27: pp. 343±359, 1988. Alan Krueger and Lawrence Summers. ``Ef®ciency Wages and Inter-Industry Wage Structure.'' Econometrica 56(2): pp. 259±293, 1988. G. S. Maddala. Introduction to Econometrics, second edition. New York: Macmillan: pp. 451±454, 1992. K. H. Marquis, N. Duan, M. S. Marquis and J. M. Polich. Response Errors in Sensitive Topic Surveys: Estimates, Effects and Correction Options. Santa Monica, CA: RAND, 1981. Kent H. Marquis, M. Susan Marquis and J. Michael Polich. ``Response Bias and Reliability in Sensitive Topic Surveys.'' Journal of the American Statistical Association 81: pp. 381±389, 1986. John Mullahy and Jody L. Sindelar. ``Lifecycle Effects of Alcoholism on Education, Earnings and Occupation.'' Inquiry 26 (Summer): pp. 272±282, 1989. John Mullahy and Jody L. Sindelar. ``Alcoholism, Work and Income.'' Journal of Labor Economics 11(3): pp. 494±520, 1993. John Mullahy and Jody L. Sindelar. ``Alcoholism and Income: The Role of Indirect Effects.'' Milbank Quarterly 72(2): pp. 359±375, 1994. Lee N. Robins, John Wing, Hans Ulrich Wittchen, John E. Helzer, Thomas F. Babor, Jay Burke, Anne Farmer, Assen Janblenski, Roy Pickens, Darrel A. Regier, Norman Sartorius and Leland H. Towle. ``The Composite International Diagnostic Interview.'' Archives of General Psychiatry 45: pp. 1069±1077, 1988. Steven Stern. ``Measuring the Effect of Disability on Labor Force Participation.'' Journal of Human Resources 24(3): pp. 361±395, 1989. United States Department of Health and Human Services (USDHHS). Vital Health and Statistics: Evaluation of National Health Interview Survey Diagnostic Reporting, Series 2: Data Evaluation and Methods Research, No. 120. Hyattesville, MD: DHHS, 1994. Milton C. Weinstein, Donald M. Berwick, Paula A. Goldman, Jane M. Murphy and Arthur J. Barsky. ``A Comparison of Three Psychiatric Screening Tests Using Receiver Operating Characteristic (ROC) Analysis.'' Medical Care 27: pp. 593±607, 1989.