STATGRAPHICS – Rev. 9/16/2013
Negative Binomial Regression
Summary ...... 1 Data Input...... 3 Statistical Model ...... 3 Analysis Summary ...... 4 Analysis Options ...... 7 Plot of Fitted Model ...... 8 Observed Versus Predicted ...... 10 Predictions...... 10 Confidence Intervals ...... 11 Correlation Matrix ...... 11 Unusual Residuals ...... 12 Residual Plots...... 13 Influential Points ...... 14 Save Results ...... 16 Calculations...... 16
Summary The Negative Binomial Regression procedure is designed to fit a regression model in which the dependent variable Y consists of counts. The fitted regression model relates Y to one or more predictor variables X, which may be either quantitative or categorical. The procedure fits a model using either maximum likelihood or weighted least squares. Stepwise selection of variables is an option. Likelihood ratio tests are performed to test the significance of the model coefficients. The fitted model may be plotted and predictions generated from it. Unusual residuals are identified and plotted.
This procedure is similar to the Poisson Regression procedure, except that the conditional variance of Y is allowed to be greater than the mean. It is thus useful for counts that are “overdispersed” compared to those from a Poisson process.
Sample StatFolio: Negbin reg.sgp
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 1 STATGRAPHICS – Rev. 9/16/2013 Sample Data The file crabs.sgd contains a set of data from a study of horseshoe crabs, reported by Agresti (2002). The data consist of information on n = 173 female horseshoe crabs. A portion of the data is shown below:
Satellites Width 8 28.3 0 22.5 9 26 0 24.8 4 26 0 23.8 0 26.5 0 24.7 0 23.7 0 25.6 … …
It was desired to relate the number of Satellites (male crabs residing nearby) to the carapace Width of the female crabs.
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 2 STATGRAPHICS – Rev. 9/16/2013
Data Input The data input dialog box requests information about the input variables:
Dependent Variable: a numeric variable containing the n values of dependent variable yi. Y must consist of non-negative integer counts.
(Sample Sizes): optional sample sizes ti corresponding to each count. If not specified, all ti are set equal to 1.
Quantitative Factors: numeric columns containing the values of any quantitative factors to be included in the model.
Categorical Factors: numeric or non-numeric columns containing the levels of any categorical factors to be included in the model.
Select: subset selection.
Statistical Model The statistical model assumed for the data is that the values of the dependent variable Y follow a negative binomial distribution of the form
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 3 STATGRAPHICS – Rev. 9/16/2013 1 Y (Y 1 ) 1 p Y , > 0, 0 (1) 1 1 1 (Y 1)( ) where the mean is the product of , the rate at which events occurs, and the sampling period t according to E(Y) = = t (2)
The variance of Y is given by
Var(Y) = (3)
If = 0, the negative binomial distribution reduces to the Poisson distribution. It is further assumed that the rate is related to the predictor variables through a log-linear link function of the form
log 0 1 X1 2 X 2 ... k X k (4)
Analysis Summary The Analysis Summary displays a table showing the estimated model and tests of significance for the model coefficients. Typical output is shown below: Negative Binomial Regression - Satellites Dependent variable: Satellites Factors: Width Number of observations: 173
Estimated Regression Model (Maximum Likelihood) Standard Estimated Parameter Estimate Error Rate Ratio CONSTANT -4.75395 0.267509 Width 0.22032 0.00940901 1.24648 Alpha 0.566406
Analysis of Deviance Source Deviance Df P-Value Model 24.0164 1 0.0000 Residual 275.024 171 0.0000 Total (corr.) 299.04 172 Percentage of deviance explained by model = 8.03116 Adjusted percentage = 6.69355
Likelihood Ratio Tests Factor Chi-Squared Df P-Value Width 24.0164 1 0.0000 Alpha 195.893 1 0.0000
Residual Analysis Estimation Validation n 173 MSE 165.698 MAE 3.27878 MAPE ME -1.3307 MPE
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 4 STATGRAPHICS – Rev. 9/16/2013
The output includes:
Data Summary: a summary of the input data.
Estimated Regression Model: estimates of the coefficients in the regression model, with standard errors and estimated rate ratios. The rate ratios are calculated from the model ˆ coefficients j by
ˆ rate ratio = exp j (5)
The rate ratio represents the percentage increase in the rate of events for each unit increase in X.
Analysis of Deviance: decomposition of the deviance of the data into an explained (Model) component and an unexplained (Residual) component. Deviance compares the likelihood function for a model to the largest value that the likelihood function could achieve, in a manner such that a perfect model would have a deviance equal to 0. There are 3 lines in the table:
1. Total (corr.) – the deviance of a model containing only a constant term, (0).
2. Residual – the deviance remaining after the model has been fit.
3. Model – the reduction in the deviance due to the predictor variables, (1,2,…,k|0), equal to the difference between the other two components.
The P-Value for the Model tests whether the addition of the predictor variables significantly reduces the deviance compared to a model containing only a constant term. A small P-Value (less than 0.05 if operating at the 5% significance level) indicates that the model has significantly reduced the deviance and is thus a useful for predictor for Y. The P-Value for the Residual term tests whether there is significant lack-of-fit, i.e., whether a better model may be possible. A small P-value indicates that significant deviance remains in the residuals, so that a better model might be possible.
Percentage of Deviance – the percentage of deviance explained by the model, calculated by
, ,..., | R2 1 2 k 0 (6) 0
It is similar to an R-squared statistic in multiple regression, in that it can range from 0% to 100%. An adjusted deviance is also computed from
2 1, 2 ,..., k | 0 2p Radj (7) 0
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 5 STATGRAPHICS – Rev. 9/16/2013 where p equals the number of coefficients in the fitted model, including the constant term. It is similar to the adjusted R-squared statistic in that it compensates for the number of variables in the model.
Likelihood Ratio Tests – a test of significance for each effect in the fitted model, and for the negative binomial parameter . The tests for the effects compare the likelihood function of the full model to that of the model in which only the indicated effect has been dropped. Small P-values indicate that the model has been improved significantly by the corresponding effect. The test for compares the fitted model against a Poisson regression model, corresponding to = 0. A small P-Value for that test indicates that the data are significantly overdispersed (the variance is greater than the mean).
Residual Analysis – if a subset of the rows in the datasheet have been excluded from the analysis using the Select field on the data input dialog box, the fitted model is used to make predictions of the Y values for those rows. This table shows statistics on the prediction errors, defined by
ei yi ˆ i (8)
Included are the mean squared error (MSE), the mean absolute error (MAE), the mean absolute percentage error (MAPE), the mean error (ME), and the mean percentage error (MPE). These validation statistics can be compared to the statistics for the fitted model to determine how well that model predicts observations outside of the data used to fit it.
The fitted model for the sample data is
ˆ exp 4.75395 0.22032Width (9)
The regression explains a little more than 8% of the deviance of a model with only a constant. The P-value for Width is very small, indicating that it is a significant predictor for Satellites.
The mean and variance of Satellites are given by:
E(Satellites) ˆ exp 4.75395 0.22032Width (10)
Var(Satellites) ˆ1 0.566406ˆ (11)
In addition, the P-value for Alpha is very small, indicating that there is significant overdispersion, so that a Poisson model would not be appropriate for this data.
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 6 STATGRAPHICS – Rev. 9/16/2013
Analysis Options
Model: order of the model to be fit. First order models include only main effects. Second order models include quadratic effects for quantitative factors and two-factor interactions amongst all variables.
Include Constant: If this option is not checked, the constant term 0 will be omitted from the model.
Fit: specifies whether all independent variables specified on the data input dialog box should be included in the final model, or whether a stepwise selection of variables should be applied. Stepwise selection attempts to find a parsimonious model that contains only statistically significant variables. A Forward Stepwise fit begins with no variables in the model. A Backward Stepwise fit begins with all variables in the model.
P-to-Enter - In a stepwise fit, variables will be entered into the model at a given step if their P-values are less than or equal to the P-to-Enter value specified.
P-to-remove - In a stepwise fit, variables will be removed from the model at a given step if their P-values are greater than the P-to-Remove value specified.
Max Steps: maximum number of steps permitted when doing a stepwise fit.
Display: whether to display the results at each step when doing a stepwise fit.
Exclude: Press this button to exclude effects from the model. A dialog box will be displayed:
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 7 STATGRAPHICS – Rev. 9/16/2013
Double click on an effect to move it from the Include field to the Exclude field or back again.
Plot of Fitted Model The Plot of Fitted Model displays the estimated mean rate ˆ(X ) versus any single predictor variable, with the other variables held constant.
Plot of Fitted Model with 95.0% confidence limits 15
12
9
6 Satellites 3
0 21 24 27 30 33 36 Width
Confidence limits for X) are included on the plot. The estimated mean number of satellites increases from a low of approximately 1 at small widths to over a dozen at large widths.
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 8 STATGRAPHICS – Rev. 9/16/2013 Pane Options
Factor: select the factor to plot on the horizontal axis.
Low and High: specify the range of values for the selected factor.
Hold: values to hold the unselected factors at.
Confidence Level: percentage used for the confidence limits. Set to 0 to suppress the limits.
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 9 STATGRAPHICS – Rev. 9/16/2013
Observed Versus Predicted The Observed versus Predicted plot shows the observed values of Y on the vertical axis and the predicted mean values ˆ on the horizontal axis.
Plot of Satellites
15
12
9
6 observed 3
0 0 3 6 9 12 15 predicted
If the model fits well, the points should be randomly scattered around the diagonal line.
Predictions The fitted regression model may be used to predict the outcome of new samples whose predictor variables are given. For example, suppose a prediction is desired for a crab with Width = 30.3. A new row could be added to the datasheet with 30.3 in the Width column, but the entry for Satellites would be left blank. The Predictions pane would then display:
Predictions for Satellites
Observed Fitted Lower 95.0% CL Upper 95.0% CL Row Value Value for Prediction for Prediction 174 6.83284 6.44308 7.24619
The table shows the fitted value ˆ i , together with approximate 95% confidence intervals.
Pane Options
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 10 STATGRAPHICS – Rev. 9/16/2013 Display: display All Values (predictions for all rows in the datasheet), or Forecasts Only (predictions for rows with missing values for Y).
Confidence Level: percentage used by the confidence intervals.
Confidence Intervals The Confidence Intervals pane shows the potential estimation error associated with each coefficient in the model, as well as for the rate ratios.
95.0% confidence intervals for coefficient estimates Standard Parameter Estimate Error Lower Limit Upper Limit CONSTANT -4.75395 0.267509 -5.27826 -4.22965 Width 0.22032 0.00940901 0.201879 0.238761
95.0% confidence intervals for rate ratios Parameter Estimate Lower Limit Upper Limit Width 1.24648 1.2237 1.26968
Pane Options
Confidence Level: percentage level for the confidence intervals.
Correlation Matrix The Correlation Matrix displays estimates of the correlation between the estimated coefficients.
Correlation matrix for coefficient estimates CONSTANT Width CONSTANT 1.0000 -0.9961 Width -0.9961 1.0000
This table can be helpful in determining how well the effects of different independent variables have been separated from each other.
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 11 STATGRAPHICS – Rev. 9/16/2013
Unusual Residuals Once the model has been fit, it is useful to study the residuals to determine whether any outliers exist that should be removed from the data. The Unusual Residuals pane lists all observations that have unusually large residuals.
Unusual Residuals for Satellites Predicted Pearson Deviance Row Y Y Residual Residual Residual 3 9.0 2.64948 6.35052 2.47 1.68 15 14.0 2.64948 11.3505 4.41 2.55 34 8.0 2.48003 5.51997 2.26 1.57 56 15.0 4.39778 10.6022 2.71 1.80 121 6.0 1.63176 4.36824 2.47 1.67 131 6.0 1.90386 4.09614 2.06 1.46 134 10.0 2.53527 7.46473 3.00 1.94 146 8.0 2.12558 5.87442 2.71 1.80 149 10.0 1.70527 8.29473 4.53 2.58
The table displays:
Row – the row number in the datasheet.
Y – the observed value of Y.
Predicted Y – the fitted value ˆ i .
Residual – the difference between the observed and predicted values, defined by
ei yi ˆ i (12)
Pearson Residual – a standardized residual in which each residual is divided by an estimate of its standard error:
ei ri (13) ˆi 1ˆi ˆi
Deviance Residual – a residual that measures each observation’s contribution to the residual deviance:
yi 1 1 1 di sgn(ri ) 2yi ln yi ˆ lnyi ˆ /ˆ i ˆ (14) ˆ i
The sum of squared deviance residuals equals the deviance on the Residuals line of the analysis of deviance table.
The table includes all rows for which the absolute value of the Pearson residual is greater than 2.0. The current example shows 9 residuals that exceed 2.5, 2 of which exceed 3.0.
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 12 STATGRAPHICS – Rev. 9/16/2013
Residual Plots As with all statistical models, it is good practice to examine the residuals. The Negative Binomial Regression procedure includes various type of residual plots, depending on Pane Options.
Scatterplot versus Predicted Value This plot is helpful in examining very large residuals.
Residual Plot
5.1
3.1
1.1
-0.9
-2.9 devianceresidual
-4.9 0 3 6 9 12 15 predicted Satellites
Residual Autocorrelations This plot calculates the autocorrelation between residuals as a function of the number of rows between them in the datasheet.
Residual Autocorrelations for Satellites
1
0.6
0.2
-0.2
autocorrelation -0.6
-1 0 2 4 6 8 10 12 lag
It is only relevant if the data have been collected sequentially. Any bars extending beyond the probability limits would indicate significant dependence between residuals separated by the indicated “lag”.
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 13 STATGRAPHICS – Rev. 9/16/2013 Pane Options
Plot: the type of residuals to plot:
1. Residuals – the observed values minus the fitted values. 2. Studentized residuals – the residuals divided by their estimated standard errors. 3. Deviance Residuals – residuals scaled so that their sum of squares equals the residual deviance.
Type: the type of plot to be created. A Scatterplot is used to test for curvature. A Normal Probability Plot is used to determine whether the model residuals come from a normal distribution (normality is not expected in this procedure). An Autocorrelation Function is used to test for dependence between consecutive residuals.
Plot Versus: for a Scatterplot, the quantity to plot on the horizontal axis.
Number of Lags: for an Autocorrelation Function, the maximum number of lags. For small data sets, the number of lags plotted may be less than this value.
Confidence Level: for an Autocorrelation Function, the level used to create the probability limits.
Influential Points In fitting a regression model, all observations do not have an equal influence on the parameter estimates in the fitted model. Those with unusual values of the independent variables tend to have more influence than the others. The Influential Points pane displays any observations that have high influence on the fitted model:
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 14 STATGRAPHICS – Rev. 9/16/2013
Influential Points for Satellites
Row Leverage 115 0.113515 141 0.393344 142 0.0362742 147 0.0966221 Average leverage of single data point = 0.0115607
The table displays all points with high leverage. Leverage is a statistic that measures how distant an observation is from the mean of all n observations in the space of the independent variables. The higher the leverage, the greater the impact of the point on the fitted values yˆ. Points are placed on the list if their leverage is more than 3 times that of an average data point.
The observation with the highest leverage in the sample data is row #141. It is over 30 times the average leverage, which means it has a large influence on the fit.
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 15 STATGRAPHICS – Rev. 9/16/2013
Save Results The following results may be saved to the datasheet:
ˆ 1. Predicted Values – the fitted values iti corresponding to each row of the datasheet. ˆ 2. Lower Limits – the lower confidence limits for iti . 3. Upper Limits – the upper confidence limits for . 4. Residuals – the ordinary residuals. 5. Pearson Residuals – the standardized Pearson residuals. 6. Deviance Residuals – the deviance residuals. 7. Leverages – the leverages for each row.
Calculations
Let i = the estimated mean at the settings of the predictor variables in row i.
Likelihood Function
1 y n y 1 1 i L i i for 0, 0 (15) 1 1 1 i1 yi 1 i i
Deviance
n 1 ˆ yi 1 yi ( |) yi ln yi ln (16) ˆ ˆ 1 i1 i i
Leverage
1 hi diagX iX WX X i wi (17)
p h (18) n
2013 by StatPoint Technologies, Inc. Negative Binomial Regression - 16