Median of the P Value Under the Alternative Hypothesis
Total Page:16
File Type:pdf, Size:1020Kb
Median ofthe p Value Underthe Alternative Hypothesis Bhaskar BHATTACHARYA and Desale HABTZGHI ormoreextreme than, the value actually observed. Thus, if the p valueis lessthan the preferredsigni cance level, then one rejects Dueto absence of anuniversally acceptable magnitude of the H0.Over theyears several authors have attempted to properly TypeI error invarious elds, p valuesare used as a well- explain p values.Gibbons and Pratt (1975) provided interpre- recognizedtool in decision making in all areas of statistical tationand methodology of the p values.Recently, Schervish practice.The distribution of p valuesunder the null hypothe- (1996)treated the p valuesas signi cance probabilities. Discus- sisis uniform. However, under the alternative hypothesis the sions on p valuescan be foundin Blythand Staudte (1997) and distributionof the p valuesis skewed. The expected p value inDollinger, Kulinskaya, and Staudte (1996). However, these (EPV) hasbeen proposed by authorsto beusedas a measureof articlesdo nottreat the p valuesas random. theperformance of thetest. This article proposes the median of The p valuesare based on the test statistics used and hence the p values(MPV) whichis more appropriate for thispurpose. random.The stochastic nature of the p valueswas investigated Weworkout many examples to calculate the MPV’ s directly byDempster and Schatzoff (1965) and Schatzoff (1966) who andalso compare the MPV withthe EPV .Weconsidertesting equalityof distributionsagainst stochastic ordering in themulti- introduced“ expectedsigni cance value.” Recently, Sackrowitz nomialcase and compare the EPV’ s andMPV’ s bysimulation. andSamuel-Cahn (1999) investigated this concept further and Asecondsimulation study for generalcontinuous data is also renamedit as the expected p value(EPV). Underthe null hy- consideredfor twosamples with different test statistics for the pothesis,the p valueshave a uniformdistribution over (0, 1) for samehypotheses. In both cases MPV performs betterthan EPV . anysample size. Thus, under H0, EPV is 1/2 always, and there isno wayto distinguish p valuesderived from largestudies and KEY WORDS: Comparingtest statistics; Expected p value; thosefrom small-scalestudies. Also it would be impossibleun- Median p value;Power; Stochastic order. der H0 todifferentiatebetween studies well powered to detect apositedalternative hypothesis and the underpowered to detect thesame posited alternative value. Incontrast, the distribution of the p valuesunder the alter- 1.INTRODUCTION nativehypothesis is a functionof thesample size and the true parametervalue in the alternative hypothesis. As the p values Thetheory of hypothesistesting depends heavily on thepre- measureevidence against the null hypothesis, it is of interest to specied valueof the signi cance level. T oavoidthe nonunique- investigatethe behavior of the p valuesunder the alternative at nessof the decision of testing the same hypotheses using the varioussample sizes. W ereject H when p valueis smallwhich sametest statistic but different signi cance levels, it is apop- 0 isexpected when H istrue.As notedby Hung,O’ Neill,Bauer, ularchoice to report the p value. The p valueis the smallest 1 andKohne (1997), the distribution of the p valuesunder the levelof signicance at which an experimenterwould reject the alternativeis highlyskewed. The skewness increases with the nullhypothesis on thebasis of theobserved outcome. The user samplesize and the true parameter value under the alternative cancompare his/ herown signi cance level with the p value and reecting the ability to detectthe alternativeby increasingpower makehis/ herown decision. The p valuesare particularlyuseful underthese situations. Hence it ismore appropriate to consider incaseswhen the null hypothesis is well de ned but the alterna- themedian of the value(MPV) insteadof theEPV underthe tiveis not(e.g., composite) so thatT ypeII error considerations p areunclear. In this context to quote Fisher, “ Theactual value of p alternativeas a measureof thecenterof its distributionwhich is obtainablefrom thetable by interpolation indicates the strength themain focus of thisarticle. Applications of thedistribution of ofevidenceagainst the null hypothesis.” the p valuesunder the alternative in thearea of meta-analysisof Wewillconsider tests of theform “Reject H when T c,” severalstudies is consideredby Hunget al.(1997). Studying the 0 ¶ where T isareal-valuedtest statistic computed from datawhen p valueunder the alternative is alsobene cial over the power of atestand is explainedin thenext paragraph. testingthe null hypothesis H0 againstthe alternative H1. The value c isdetermined from theprespeci ed sizerestrictions such Whenseveral test procedures are available for thesame test- that P (T c) = ¬ .Of course,when T isadiscreterandom ingsituation one compares them by meansof power.However, H0 ¶ variable,one needs to adoptrandomization so that all sizes are powercalculations depend on the chosen signi cance levels, possible.If t istheobserved value of T andthedistribution of T andin discrete cases involves randomization. These steps can under H is given by F ( ), then the p valueis givenby 1 F (t) beavoidedby considering MPV’ s. Also as thepower functions 0 0 ¢ ¡ 0 whichis theprobabilityof ndingthe test statistic as extremeas, dependon the chosen signi cance levels, it isdifcult to compare themwhen different power functions use different signi cance levels.On the otherhand, MPV’ s dependonly on thealternative BhaskarBhattacharya is Associate Professor,Department ofMathematics, andnot on the signi cance level. The smaller the valueof MPV , SouthernIllinois University, Carbondale, IL 62901(E-mail: Bhaskar@math. siu.edu).Desale Habtzghiis aGraduate Student,Department ofStatistics, Uni- thestronger the test. The value of an MPV cantell us which versityof Georgia,Athens, GA 30602. alternativean attained p valuebest represents for agivensam- 202The American Statistician, August 2002, V ol.56, No. 3 c 2002American Statistical Association DOI: 10.1198/000313002146 ® 1 plesize. Also, it helpsto knowthe behavior of theMPV’ s for Since F (t) F (t); t,itisseen that F (F ¡ (:5)) :5 which ³ µ 0 8 0 ³ ¶ varyingsample sizes. impliesthat ¬ :5 andequality holds when H is true. The ¤ µ 0 InSection 2, we obtaina generalexpression for theMPV’ s. smallerthe MPV thebetter it istodetectthe alternative. Given Wealsoderive a computationallyfavorable form tobe used thestochastic nature of the p valueunder the alternative, it is laterin thearticle. In Section 3, we considerseveral examples to alsotrue that the MPV issmallerthan the EPV andhence clearly computethe MPV’ s directly.In Section 4, we perform asimu- preferableover the EPV .TheMPV beingsmaller than the EPV lationstudy to compute the MPV’s inthecase of testingagainst producesa smallerindifference region in thesense that higher stochasticordering for multinomialdistributions. W ealsoper- poweris generated closer to the null hypothesis region using form asecondsimulation study for generaldistributions when MPV thanwith using EPV . testingthe same hypotheses with two samples using three test Itis alsopossible to expressthe MPV inanother way. Let T ¹ statisticsof t test,Mann– Whitney– Wilcoxon, and Kolmogorov– F ( ) and,independently, T F ( ).If theobserved value of ³ ¢ ¤ ¹ 0 ¢ Smirnov.In Section 5, we makeconcluding remarks. T is t, then the p valueis simply g(t) = P (T t T = t). The ¤ ¶ j MPV is 2. VALUE ASARANDOM VARIABLE AND ITS p med g(t) = MPV(³ ) = P (T ¤ med T ): (3) MEDIAN ¶ If H0 istrue, then T and T ¤ areidentically distributed, and hence For thetest statistic T withdistribution F0( ) under H0, let theabove probability is .5 for anycontinuous distribution. For 1 ¢ discretedistributions, although the above expressions still hold, F³ ( ) beitsdistribution under H1. Also, let F0¡ ( ) betheinverse ¢ 1 ¢ function of F ( ), so that, F (F ¡ (® )) = ® , for any 0 < ® < 1. MPV willbe slightly higher as P (T ¤ = med T ) > 0. For an 0 ¢ 0 0 Since the p valueis theprobability of observinga moreextreme UMP test,the MPV willbe uniformlyminimal for all ³ values valuethan the observedtest statistic value, as a randomvariable inthe alternative as comparedto theMPV’ s ofanyother test of itcanbe expressedas the same H0 versus H1.For computationalpurposes the above form oftheMPV in(3) isvery useful. X = 1 F0(T ): ¡ 3. EXAMPLES As F (T ) U(0; 1) under H , so is X.Thepower of the test 0 ¹ 0 Thissection considers several examples to calculate the isrelatedto the p value as MPV’s directly. = P³ (X ¬ ) µ Example 1. Let X1; X2; : : : ; Xn bearandomsample of size n 1 2 = 1 F³ (F0¡ (1 ¬ )): (1) from a N(· ; ¼ ) distribution,and we liketo test H : · · ¡ ¡ 0 µ 0 versus H : · > · where ¼ isknown. Using the test statistic Notethe above is also the distribution function of the p value 1 0 T = X anda particularvalue · in H itfollows from (2) that underthe alternative. As T isstochastically larger under the 1 1 alternativethan under the null hypothesis, it followsthat the p pn(· · ) MPV = © 0 ¡ 1 ; (4) valueunder the alternative is stochastically smaller than under ¼ thenull (Lehmann 1986). This explains why the distribution of µ ¶ the p valueis skewedto therightunder the alternative. Hence to where © isthecdf of thestandard normal distribution. It iswell estimatethe center of thedistribution of the p value,the median knownthat for asize ¬ testto achieve power ata specied isa betterchoice than the mean. The median is any value of alternativevalue · 1 thesample size n satis es ¬ = ¬ ¤ whichsatis es 2 2 (z1 ¬ + z