48 Journal of The Association of Physicians of India ■ Vol. 65 ■ April 2017 STATISTICS FOR RESEARCHERS Principles of Regression Analysis NJ Gogtay, SP Deshpande, UM Thatte Introduction relationship between one or more the “dependent” variable. In the independent variables and the second example, since CAD is hile correlation analysis helps dependent variable. A correlation associated with both diabetes and Win identifying associations analysis with a scatter plot and hypertension, CAD would become or relationships between two a regression line1 is however a the dependent variable and diabetes variables,1 the regression technique prerequisite to regression and and hypertension would be the two or regression analysis is used to both analyses are often carried out independent variables. If you will “model” this relationship so as together. notice, diabetes in one example is to be able to predict what will a dependent variable and in the happen in a real-world setting. Let Dependent and other, an independent variable. us understand this with something Independent Variables Thus, what is a dependent and that happens in daily life. As what is an independent variable children, we are often told by our An important first step before needs to be defined à priori by the parents that education is a vital carrying out a regression analysis researcher before carrying out the part of our lives and a “good” is to understand the concept regression analysis. The choice education will help us get “good of independent and dependent of the variables would in turn be jobs” and thus “good wages”! If variables. An independent variable, defined by the research question we were to explore the relationship as the name suggests is a “stand and the hypothesis being explored. between education and wages, alone” variable, and one that Independent variables are often we could think up two questions remains unaffected by other called predictor variables or – a) Does a relationship exist variables that are measured in a exogenous variables and dependent between education and wages? And study. The “dependent” variable is variables are called prognostic or a related but more interesting and the one that is usually of interest to endogenous variables. important question b) for every extra the researcher and alters in response year spent in education, do wages to change/s in the independent variable. Types of Regression commensurately increase [and if Let us understand this concept Essentially in medical research, yes, by how much?]? The latter with the following two research there are three common types of question is moving into the realm of questions – 1) “As age increases, regression analyses that are used prediction. Regression attempts to does the risk of developing diabetes viz., linear, logistic regression and answer these and similar questions as measured by HbA1C or blood Cox regression. These are chosen regarding relationships between sugar levels increase? Or b) “Given depending on the type of variables variables. that a patient has both diabetes and that we are dealing with (Table 1). hypertension, what is his risk of Cox regression is a special type of Correlation Versus developing coronary artery disease regression analysis that is applied Regression- [CAD]”? In the former example, to survival or “time to event “data we are exploring the relationship Understanding the and will be discussed in detail in between age and diabetes and in the next article in the series. Distinction the latter, the relationship between Linear regression can be simple The difference between two variables with a third variable– i.e., diabetes and hypertension linear or multiple linear regression correlation analysis and regression while Logistic regression could be lies in the fact that the former focuses with CAD. In the first example, age would be the independent Polynomial in certain cases (Table on the strength and direction of 1). the relationship between two or variable while diabetes, which is more variables without making any dependent on age would become The type of regression analysis assumptions about one variable being independent and the other dependent [see below], but regression analysis Department of Clinical Pharmacology, Seth GS Medical College and KEM Hospital, Mumbai, Maharashtra Received: 07.01.2017; Accepted: 15.01.2017 assumes a dependence or causal Journal of The Association of Physicians of India ■ Vol. 65 ■ April 2017 49 Table 1: Types of regression duration of surgery, extent of Type of regression Dependent variable Independent variable Relationship between blood loss and Hb levels at and its nature and its nature variables day 7, it is intuitive that all Simple linear One, continuous, One, continuous, Linear are correlated amongst each normally distributed normally distributed other and therefore will lead to Multiple linear One, continuous Two or more, may Linear erroneous results. What should be continuous or be taken as the independent categorical variable is only the duration of Logistic One, binary Two or more, may Need not be linear be continuous or surgery. categorical • The sample size should be Polynomial (logistic) Non-binary Two or more, may Need not be linear adequate [multinomial] be continuous or categorical If the dependent variable is Cox or proportional Time to an event Two or more, may Is rarely linear non-binary and has more than hazards regression be continuous or two possibilities, we use the categorical multinomial or polynomial to be used in a given situation is be quantitative or qualitative. logistic regression. primarily driven by the following Both the independent variables three metrics here could be expressed either Step Wise Regression • Number and nature of as continuous data (blood Stepwise regression is an independent variable/s pressure or HbA1C values) automated tool that can be used or qualitative data (presence • Number and nature of the in the exploratory stages of model or absence of diabetes as dependent variable/s building to identify a useful subset defined by the ADA 2016 or of predictor variables. The process • Shape of the regression line hypertension as defined by JNC systematically adds the most 3,4 A. Linear regression: Linear VIII). A linear relationship significant variable or removes regression is the most basic should exist between the the least significant variable during and commonly used regression dependent and independent each step. The idea behind using technique and is of two types variables. such automated tools is to maximize viz. simple and multiple B. Logistic regression: This type the power of prediction with a regression. You can use Simple of regression analysis is used minimum number of independent linear regression when there when the dependent variable is variables. is a single dependent and a binary in nature. For example, if single independent variable the outcome of interest is death Steps in Conducting a (e.g. for the research question in a cancer study, any patient Regression Analysis described above “As age in the study can have only one increases, does the risk of of two possible outcomes- dead Regression analysis is done in developing diabetes increase or alive. The impact of one or 3 steps: as measured by HbA1C or more predictor variables on this 1. Analyzing the correlation blood sugar levels?”. Both the binary variable is assessed. The [strength and directionality of variables must be continuous predictor variables can be either the data] 2 (quantitative data ) and the line quantitative or qualitative. 2. Fitting the regression or least describing the relationship is a Unlike linear regression, this squares line, and straight line (linear). type of regression does not Multiple linear regression on require a linear relationship 3. Evaluating the validity and the other hand can be used between the predictor and usefulness of the model. when we have one continuous dependent variables. Step 1: This has been described in 1 dependent variable and two or For logistic regression to be the article on correlation analysis more independent variables, meaningful, the following Step 2: Fitting the regression line for example when we want to criteria must be met/satisfied Conventionally, in mathematics, answer the second question • The independent variables the equation of a line is given by mentioned above, “Given that must not be correlated the formula “y = mx+c”, where, a patient has both diabetes and amongst each other e.g. if m is the “slope” or gradient of hypertension, what is his risk the dependent variable is the line and “c” is where the line of developing coronary artery presence (or absence) of post- “cuts” the y-axis, also called as disease (CAD)”. Importantly, operative wound infection and the “intercept”, (Figure 1). In the independent variables could the independent variables are regression, this equation is given by 50 Journal of The Association of Physicians of India ■ Vol. 65 ■ April 2017 Regression Equa�on Table 2: Age and Blood pressure in women (n=30) Sr. No. Age (years) Systolic blood HbA1C (%) HbA1C (Y) = B0 + B1*age (X). pressure (mm Hg) 1 22 131 Y-axis B1 2 23 128 B0 3 24 116 4 27 106 5 28 114 6 29 123 X - axis Age (years) 7 30 117 8 32 122 9 35 99 Fig. 1: Regression analysis exploring the relationship between Age and HbA1C 10 35 121 A scatter plot and regression line of age 11 40 147 12 41 139 versus systolic blood pressure y = 0.889x + 94.61 13 41 171 2 R = 0.292 14 46 137 200 15 47 111 16 48 115 150 17 49 133 18 49 128 100 19 50 183 SBP 20 51 130 50 21 51 133 22 51 144 23 52 128 0 24 54 105 0 10 20 30 40 50 60 70 80 25 56 145 26 57 141 Age 27 58 153 Fig. 2: Scatter plot and regression Line 28 59 157 29 63 155 30 67 176 y=B0 +B1x which are the notations of the model commonly used by statisticians.