DOI: 10.26717/BJSTR.2018.05.001185 Robert Pasnak. Biomed J Sci & Tech Res ISSN: 2574-1241 Opinion Open Access To Pretest or Not to Pretest Robert Pasnak* Department of Psychology, George Mason University, USA Received: April 23, 2018; Published: June 08, 2018 *Corresponding author: Robert Pasnak,George Mason University,Fairfax VA 22030 Abstract Pretests decrease the internal validity of experiments (i. e., render them incapable of completely proving what they are intended to prove). The loss of internal validity may be great in some cases, and little or no loss in others, but just how much has been lost cannot be proven. Pretests are sometimes credited with demonstrating that randomization has produced reasonably equivalent groups, but that is indexed by p, with equal accuracy whether or not pretests are employed. There are usually many sources of differences between subjects which could result in different outcomes on the dependent variable(s). The only valid index of whether an initial inequality of groups has produced a spurious by a failure of randomization to create equality in whatever pretests might measure and any other source of differences. Pretests can increase theresult power is p. Thisof experiments value specifies to detect the probability small differences that differences in outcomes, in the but outcome only if for pretest various and levels posttest of the scores independent are strongly variable(s) correlated. were Conditions produced underKeywords: which Pretests; accepting Internal the confound Validity; introduced External Validity; by pretesting Power is most likely to be profitable are identified. Introduction properly, there will be no confounds whose effects could be mis- There are three great goals for all research: internal validity, ex- takenly attributed to the independent variable. A common cause of ternal validity, and power. An unfortunate misunderstanding that mistakes is the failure of random assignment of participants to lev- has become increasingly prevalent is the conception that pretest- els of the independent variable to accomplish what it is supposed to ing increases internal validity. In fact, pretesting has the opposite accomplish - equality in all causal variables, known and unknown. effect; it reduces internal validity. What pretesting does increase is The extent to which to which a failure of randomization could have power if pretest and posttest scores are strongly correlated. Small produced the differences observed on the dependent variable is effects may be detected if pretests are administered, although there expressed by p. Pretests neither increase nor decrease the valid- ity or accuracy of p as a measure of the failure of randomization displacement of conventional methodology courses by courses in are issues in statistical analysis after pretesting. This may reflect to produce initial equality in the characteristics of participants in advanced correlational methods, which have great value in their educational research. In essence a p of 02 means exactly the same own right and have engaged a generation of young scholars. This thing whether or not there were pretests. brief treatise is an effort to clarify the purpose and interpretation of pretests. To make it easier to follow, the discussion will be in terms of one independent and one dependent variable, although good re- that the difference observed was not due to an initial inequality We can have equal confidence that there is a 2 per cent chance search often involves more than one of each, andthe measurements of the participants, whether there was or whether there was not a pretest. The only gain for a researcher who administered a pre- although the same considerations hold for other measurements. test is that p may be smaller than it would otherwise be, because taken for the dependent variable will defined as participants scores, Internal Validity differences between individuals can be removed. Absent the pre- test, p might have been .03 or even >.05. In the latter case, the effect Internal validity expresses the extent to which we can be certain of the independent variable would not be recognized, by conven- that, within the conditions of the experiment, the effects attributed tional standards. However, the statistical analysis required when to the independent variable are indeed due to that variable and not pretests are administered is less powerful than when they are not. some other. Any variable whose effects might be mistaken for those This is because the pretest and posttest scores are correlated, and of the independent variable is termed a “confound” or confound- accounting for the correlation, which is a statistical necessity, sac- ing variable. Confounds are sometimes inherent in the design of an experiment – as when pretests are employed – or are sometimes scores, rendering the statistical analysis less powerful and there- produced when an experiment is not conducted properly. When rifices degrees of freedom, typically one for each pair of correlated fore less able to detect effect of the independent variable. This is unconfounded designs are used and experiments are conducted Cite this article: Robert Pasnak. To Pretest or Not to Pretest. Biomed J Sci &Tech Res 5(2)- 2018. BJSTR. MS.ID.001185. DOI: 10.26717/ BJSTR.2018.05.001185. 4471 Robert Pasnak. Biomed J Sci & Tech Res Volume 5- Issue 2: 2018 not often understood, because the error term (measure of chance validity. However, such experiments may be useful for testing theo- differences) is smaller. This decrease may or may not be offset by ry, or for identifying variables that affect scores, even if those effects the loss of degrees of freedom, depending on whether pretest and are small. However, pretesting, the main subject of this pretest, in- posttest scores are strongly or weakly correlated. volves a different compromise. Pretests may increase power, but do so at the cost of reducing the internal validity of experiments. That Note that for any data set, applying an analysis that ignore is to say, pretests can improve our ability to detect small effects of correlation between correlated scores will produce a statistical the independent variable, but reduce our ability to be certain that the effects we identify are actually those of the independent varia- advantage of pretesting when there is one is that effects of the inde- analysis that is more significant, but spurious. Note also that the ble, because they create a confound. pendent variable are more likely to be detected, not that spurious Pretest-treatment Interactions actually comes at the cost of an increased likelihood of spurious findings are more likely to be avoided. This advantage in detection that confound long ago: the test-treatment or sensitization-treat- A classic methodology text (Kerlinger 1964, pp. 339) identified ment interaction. The pretest may cause the experimental manip- findings,External as Validity we shall see. ulation to produce an effect it would otherwise not have produced. External validity expresses the extent to which the results of an Or it may cause different levels of the independent variable to have experiment can be generalized beyond the conditions of the exper- effects greater or smaller than they would have had if not preceded iment. This usually means to different populations measured under by a pretest. This is not the effect of testing itself; if all participants - are pretested, the effect of the pretest per se will on average be the tions of the variables. It would logically be impossible to have high- different conditions, sometimes with different operational defini same for all. Pretesting will be a constant like all the other things er external validity than internal validity. In fact, there is little or no held constant in a good experiment, not a variable whose effects - could be mistaken for those of the independent variable. The con- jority of experiments. If a sample were randomly and independent- scientific basis for estimating the external validity of the great ma found is the interaction of the pretest with experimental manip- ulation (the different levels of the independent variable). It is not results to that population, but such sampling is seldom done, except ly drawn from a specified population, one could lawfully generalize in survey research. (Random sampling is not to be confused with more responsive or less responsive to a manipulation. Children pre- random assignment to experimental conditions.) Generalization to difficult to recognize that being pretested could make a participant tested on mathematics, even if given no feedback, may recognize unmeasured populations and conditions is usually done on faith what they need to learn in orderto do better next time, and conse- alone. Fortunately, it seems to work pretty well. If manipulations quently respond well to experimental instruction. “For instance, in designed to improve health, for example, are basically similar, they education a pretest can be the impetus for learning the correct an- usually produce effects in the same direction for 12-year-old city swers to items and thus increase the posttest level of performance” girls as for 12-year-old rural girls, although very often to a greater (Cook and Campbell 1979, pp.102). or lesser extent, and there are plenty of exceptions. This is true whether or not the pretest has two different forms. It is indeed fortunate that many experiments in many disci- Children in a control group, however, have no opportunity to apply - the insight they gained from the pretest. Even though the pretest erally replicable with different populations and subtle or not so plines have sufficient, albeit unknown, external validity to be gen is a constant, it can potentiate responses at different levels of the independent variable differently, because those levels are different. research in these disciplines would have little value. High external subtle differences in experimental conditions; otherwise, scientific That is the nature of an interaction between variables. The inter- validity is often the result of [1] robust experimental manipulations action could also be negative.