The Rev.: an IBM BASIC Program for Bayesian Test Interpretation

Behavior Research Methods, Instruments, & Computers 1992, 24 (3), 491-492 - PROGRAM ABSTRACTS/ALGORITHS The Rev.: An IBM BASIC program Thorndike (l986a, 1986b) extended this thinking to the for Bayesian test interpretation interpretation of standardized tests. In an assessment sit uation, he proposed combining a priori with current in RONALD D. FRANKLIN and DAVID B. ALLISON formation to form an estimate of the subject's "true" The Johns Hopkins School of Medicine score on some variable. For example, if, when assessing and The Kennedy Institute, Baltimore, Maryland a child's intelligence, knowledge of the child's score on an intelligence test administered 2 years earlier and on The Rev. is a BASIC computer program for IBM-PC an achievement test administered earlier that day, can both compatible systems that provides Bayesian estimates of "true" scores from multiple scores measuring the same be used as estimates of the child's "true" score on an construct. Psychological reports often include test scores intelligence test today? from earlier evaluations without objectively incorporat Disregarding earlier test scores when administering later ing them into current findings. Using Thorndike's for tests not only wastes information-described by Cohen mulas for objectively combining test scores while provid (1990) as a psychologist's most precious commodity ing estimates with reduced standard errors, the Rev. is but may ignore the more valid measure. Many psycholo an interactive program that facilitates test interpretation gists subjectively combine such information from differ by combining information from many test administra ent sources, but Thorndike (l986c) provides formulas for tions. The user provides four easily obtainable pieces of objectively combining test scores. information for each test administration. The output in Thorndike's (1986c) formulas for estimates are repre cludes an estimated "true" score and the standard error of the estimate. sented as: "True" Score The crux of Bayesian thinking is that people have [(Xd(SEmeas)2) + (X2/(SE! est)2) + (Xnl(SEn est)2)] a priori subjective or objective probability distributions for the likelihood of some condition being met. New in [(l/(SEmeas)2) + (l/(SE! est)2) + (l/(SEn est)2)] formation can be added later, producing a new, and and presumably more accurate, posterior probability distri "True" Score SE bution. It should be noted that neither the new nor the old information is discarded in favor ofthe other. Rather, they combine, producing a better probability estimate than either could provide alone (Iverson, 1984). Bayesian statistics provide an alternative measure of where XI is the original, or primary, measure ofthe con probability. Rather than asking the traditional statistical struct and X2 to Xn are secondary measures of the con question, "If 60% of children with Johnny's IQ are struct. SEmeas is the standard error of measurement defined retarded, what is the probability that Johnny is retarded?", as SD, «(l-rll)·5), and SEes! is the standard error of the Bayesian methods ask "IfJohnny scores in the retarded estimate defined as SDn «(l-rl n)·5); where SDn is the stan range on an IQ test, what is the probability he is dard deviation of the nth test, rII is the reliability coeffi retarded?". Stated formally: cient (preferable internal consistency) ofthe primary test, and r I n is the correlation between the primary test and p(HIE) = [P(H) x p(EIH)] P(E), I the nth test (the meaning of a primary test is given be where p(HIE) = the posterior probability of the suspected low). In essence, a weighted average of scores is com result given the specified event, P(H) = the outcome puted, where each score is weighted by the reciprocal of probability when no a priori knowledge of the event ex its error estimate. In addition to providing a better esti ists, p(EIH) = the probability of the event within the out mate than any single score, this method can also provide come group, and P(E) = probability of the event's oc estimates with substantially reduced standard errors. currence in the base population (after Thorndike, 1986c, The Rev. is named for Reverend Thomas Bayes, who p.54). developed the conditional probability statements that formed the foundations of Bayesian statistics. It is an inter active program that makes Thorndike's (1986c) formulas R. D. Franklin is now at South Florida State Hospital, Pembroke Pines, FL. D. B. Allison is at St. Luke's/Roosevelt Hospital, New York, NY. easy to apply in test interpretation. The Rev. can com Requests for reprints and copies of the software should be sent to R. D. bine information from a large number oftest administra Franklin, 7958 Pines Boulevard, Suite 136, Pembroke Pines, FL 33024. tions. The user need only provide four pieces of infor- 491 Copyright 1992 Psychonomic Society, Inc. 492 FRANKLIN AND ALLISON mation for each test administration. Except for the APPENDIX (continued) subject's test score, other required information is gener ally available in most test manuals and includes: (1) the 101 REM A SETS THE DEFAULT ARRAY SUBSCRIPT mean score from the standardization sample, (2) the stan 190 CLS 201 GOSUB 1000 dard deviation from the standardization sample, and 210 LET A% = NT% 300 DIM N$(M) ,TS%(M) ,TM%(M) ,TSD%(A%) .R(A%) (3) either the reliability (internal consistency) ofa primary 400 FOR M = 1 TO NT% 410 LET N$(M)="" measure ofthe desired construct or the correlation ofthe 411 LET TS%(A%)=O 412 LET TM%(A%)=O test with a primary measure of the desired construct. By 413 LET TSD%( M) =0 primary measure, two things are implied: (1) the test is 414 LET R(A%)=O 415 CLS generally accepted to be a measure of some construct, and 419 GOSUB 1100 420 GOSUB 10000 (2) the test is given at the approximate point in time that 550 NEXT A% 600 GOSUB 20000 one wishes to measure the construct. (See Appendix.) 700 GOSUB 50500 998 END 999 Program Description 1000 PRINT "Enter the name of the test sub;ect" The Rev. was written in unmaximized generic BASIC. 1010 INPUT TN$ 1020 PRINT "How many tests will you compare" It was interpreted and debugged on a Zenith 286 system 1030 INPUT NT% 1031 CLS and operates under most interpreters supporting 8080, 1040 RETURN 8086, 80286, or 80386 architectures. It requires 2.2 KB 1050 1100 PRINT "Enter the name of test number U;A% or more of RAM and runs under MS-DOS Version 2 or 1110 INPUT N$(A%) 1200 PRINT "Enter the examinee's Standard Score for the higher. Output is routed to the console or a standard LPT II;N$(A't) configuration. The report includes an estimated "true" 1210 INPUT TS%(A%) 1300 PRINT "Enter the Publisher's Mean score for the score and the standard error of this estimate. II ;N$(A%) 1310 INPUT TM%(A%) 1400 PRINT "Enter the Publisher's standard Deviation for the ";N$(A%) Program Availability 1410 INPUT TSD%IA%) 1500 IF A%=1 THEN PRINT "Enter the Publisher's reli The program source code is available free of charge ability for the 't e s t " 1510 IF A% <> 1 THEN PRINT "Enter the PUblisher's by sending a blank formatted diskette along with a return correlational coefficient between postage-paid mailer to the senior author at 7958 Pines 11 ; N$ ( A%) ; II and II ;N$( 1) Boulevard, Suite 136, Pembroke Pines, FL 33024; 1520 INPUT R(A%) 1600 RETURN 305-432-2157. 1700 : 10000 STOIAI) • (1« TS%IA%) - TM%(A%)) / TSD%IA%) • TSD% 11)) + TM%( 1 ) ) REFERENCES 10100 SE(l) = ITSD%(1) • ( SQR « 1 - IR(l) '2 ») 10150 IF A%>l THEN LET SEIA%) = TSD%(A%) • SQR ( 1 - RIA%) ) COHEN, J. (1990). Some things I have learned (so far). American Psy 10200 RETURN chologist, 45, 1304-1312. 10300 IVERSON, G. R. (1984). Bayesian statistical inference. Beverly Hills: 20000 LET PEST = 0 Sage. 20010 LET TEST = a 20050 FOR K% = 1 TO NT% THORNDIKE, R. L. (1986a). Bayesian concepts and test making. Jour 20100 PEST = PEST + ( 1 / ( SE(K%) "2 ) nal of Counseling & Development, 65, 110-111. 30200 TEST = TEST + ( STD(K%) /( SE(K%) "2 » 30300 NEXT K% THORNDIKE, R. L. (1986b). Bayesian concepts in test interpretation. 40000 LET EST = TEST I PEST Journal of Counseling and Development, 65, 170-172. 50100 RETURN THORNDIKE, R. L. (1986c). The role of Bayesian concepts in test de 50200 50500 CLS velopment and test interpretation. Journal ofCounseling & Develop 50600 PRINT "The REVII:PRINT:PRINT:PRINT ment, 65, 54-65. 50700 PRINT IIEstimate of True test scores for TN$ :PRINT:PRINT 50800 PRINT "Test Name Score Mean SD Coeff icient Error" APPENDIX 51000 PRINT "------------------------------------ -- --- -- --- - ---- - -- ---- --- - -- -- -- -" Program Listing 51200 FOR K% = 1 TO NT% 51300 PRINT N$IK%) 'TAB(15) ,TS%(K%) 'TAB(25) 'TM%IK%)' 1 REM The REV TAB( 35) ,TSD% I K%) ;TAB( 45) ;R( K%); 2 REM Bayesian predictor of true test scores TAB(59) ;SEIK%) 3 REM Ronald D. Franklin, Ph.D. 51400 PRINT 4 REM David B. Allison, Ph.D. 51500 NEXT K% 5 REM The Kennedy Institute and 51700 PRINT "The estimate of the true score is "; EST 6 REM Johns Hopk.ins School of Medicine 51800 PRINT "The standard error of the true score esti 7 REM Send inquiries to R.D. Franklin mate is ";TSE 8 REM 7958 Pines B'l vd ,, Suite 136 51900 PRINT "NOTE: Predictions are based on comparison 9 REM Pembroke Pines, FL 33024 (J05) 431-2158 wi th "; N$ (1) ;". " 10 : 52000 RETURN (Manuscript received January 14, 1991; revision accepted for publication January 9, 1992.).

Load more