Employment and Training Administration  Statistical Model Quick Reference Guide

Performance Peer Learning Group - Statistical Model Quick Reference Guide

Model Types

Simple Regression In simple linear regression, we predict one variable based on the values of a second variable. The variable that we are predicting is referred to as Y and can go by many names depending on the context (e.g., explained variable, predicted variable, outcome variable, target, etc.). The variable we are basing our predictions on is referred to as X and can go by names depending on the context (e.g., explanatory variable, predictor variable, regressor, etc.). When there is only one predictor variable, the prediction method is called simple regression. Multiple Linear Regression Multiple linear regression attempts to model the relationship between two or more explanatory variables and a response variable by fitting a linear equation to observed data. Every value of the independent variable X is associated with a value of the dependent variable Y. Fixed Effects Fixed effects are the characteristics of a unit of observation (i.e., individual participant, local area, state) that either don’t change or change at a constant rate over time. When the data used has multiple observations using that same unit of observation at different periods in time, fixed effects can be used to control for those constant characteristics. For example, the state-level statistical adjustment model uses state fixed effects because there are characteristics of states that don’t change over time and because state-quarter observations are used (i.e., there are multiple observations of the same state). Logistic Regression Logistic regression is often the appropriate regression analysis to conduct when the dependent variable is dichotomous (binary). Logistic regressions are used to describe data and explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables. For example, you may use a logistic regression if you were using individual participant level data to predict a binary outcome, such as employment (employed or not employed). A logistic regression may result in better variable estimates; however, one downside is that the generated estimates may be more difficult to explain to stakeholders.

1 Employment and Training Administration  Statistical Model Quick Reference Guide

Model Parts

Dependent Variable The variable whose values are explained by changes in the independent/explanatory variable. For the purpose of the state-level and local-level statistical adjustment models, the dependent variables for each model are the individual performance indicators for each program. Independent Variables The independent variables are the variables that explain the dependent variable; the term is a synonym for explanatory variable or other terms used depending on the context. For the purpose of the state- level and local-level statistical adjustment models, the explanatory variables are the selected participant characteristic and economic condition variables. Determining which variables are appropriate to include in the models involves many considerations including which variables have large effects on outcomes, are statistically significant, improve the model predictions, are required by statute, etc. Regression Coefficients Regression coefficients are estimates of the unknown population parameters and describe the relationship between the dependent and independent variables. In a linear regression, coefficients are the values that multiply the predictor values. Suppose you have the following regression equation: y = 3X + 5. In this equation, +3 is the coefficient, X is the predictor, and +5 is the constant. The magnitude of a coefficient determines how much a one unit increase in the X effects Y. Model Estimates Estimation represents processes of learning and determining the population parameter based on a model fitted to data. An estimator is an example of a statistic, which becomes an estimate when the formula is replaced with actual observed sample values. For the purpose of the state-level and local- level statistical adjustment models, the model estimates are those model-generated statistics that are used throughout the model cycle (e.g., variable coefficient values, fixed effect values, etc.)

2 Employment and Training Administration  Statistical Model Quick Reference Guide

Model Cycle

Predicted X The predicted X is the value used for each explanatory variable in calculating the pre-PY prediction. It is meant to be the best guess of what the value of that variable will be during the program year. There are multiple methods that can be used to determine this value, but a simple method is to use the last program year data and latest economic data for the observational unit that you are generating a prediction for (e.g., local-area, state, etc.). Actual X The actual X are the values used for each of the explanatory variables in calculating the post-PY prediction. This is the actual participant characteristic and economic condition values for each explanatory variable during the assessed program year. Pre-PY Prediction The pre-PY prediction is the performance outcome that is predicted by the model and used a factor in negotiations. It is calculated by multiplying all the variable coefficients by the associated predicted X values and summing all those calculated values together. Note: if fixed effects are used, those values should be added as well, where applicable. Post-PY Prediction The post-PY prediction is the performance outcome that is predicted by the model using the actual data from the program year and is used to determine the adjustment factor. It is calculated by multiplying all the variable coefficients by the associated actual X values and summing all those calculated values together. Note: if fixed effects are used, those values should be added as well, where applicable. Adjustment Factor The difference between the resulting post-PY prediction and the original pre-PY prediction is the adjustment factor that is applied to the negotiated level of performance to get the adjusted negotiated level of performance. Actual performance outcomes are assessed against the adjusted negotiated level of performance

3