The Impact of the Y2K Bug: Perception and Reality

Peter Bock Antonio Sanchez-Aguilar Department of Computer Science Department of Computer Science The George Washington University University of the Americas Washington, DC, USA Cholula, Puebla, Mexico [email protected] [email protected] Alan Dunn Webmaster Montgomery Blair High School Silver Spring, MD, USA [email protected]

Abstract response to the Y2K problem are good predictors. Predictions in all the other categories are either Experts in business and technology readily uncertain and weak or clearly weak. admitted their inability to predict the severity of the impact of the Y2K bug. Nevertheless, some people, regardless of background and training, may have acquired the intuitive knowledge necessary to predict the impact of the Y2K bug as an indirect result of their life experiences. Thus, the occasion of the Y2K bug presented an unusual opportunity to study the relationship between perceptions before the fact and the reality after the fact of the impact of a ubiquitous event that would occur with complete certainty. By means of an internet survey of many individuals from many different vocational and cultural backgrounds throughout the world, data were gathered in two areas: objective personal demography and subjective Y2K impact predictions. The research hypothesis of the study is that, despite an inability to codify their reasoning processes, people with certain backgrounds and life experiences can accurately predict some of the effects of the Y2K event. Although the Type II error probabilities could not be estimated, initial results validate this hypothesis (confidence ≥ 87%), suggesting that successful predictors are age 35 to 55; are married with 2 children; have a masters degree or doctorate; are executives, technicians, or homemakers; have only a little international travel experience, or identify apathy as the cause of the Y2K bug. Poor predictors are under age 25; are single; live in large households; are students; have only a secondary education; or have served as a military officer. In addition, those who think their lives are getting worse are poor predictors, those who are dissatisfied with their government’s response to the Y2K problem are poor predictors, and those who are satisfied with their government’s 1 Problem Statement Predictions of the impact of the Y2K bug by experts in business and technology ranged across the spectrum of severity, from “none” to “severe”. Often the opinions of experts occupied opposite ends of the spectrum, even though their backgrounds and fields of expertise were very similar and apparently highly relevant. Some well known and respected experts in technology were so deeply concerned that they retreated to survival camps in isolated areas, severing all connections with technology support services, such as electric power distribution, communications, and transportation systems. (In all fairness, it was largely their concerns that motivated the worldwide effort to repair this problem before its onset). Others discounted the probable impact completely, continuing with their lives as usual, neither recommending nor taking precautions. [Yourdon, 1999] When experts who announced their predictions publicly were asked to explain the reasoning processes behind these predictions and to codify their prediction models, they often had difficulty doing so. Some built derivative cases based on predictions of others. Some argued by analogy or anecdotally. Others retreated to circular logic or based their conclusions on precarious axioms of human, cultural, or economic behavior. When asked to describe their predictions using probability density functions of the various spaces that represented possible effects (transportation systems, electric power distribution, communication systems, civil disorder, etc.), they were not forthcoming, obviating any statistical computation of Type 1 and Type 2 error probabilities. Formal reasoning models seemed to be elusive. Because the devices and associated software that may have exhibited failures are often deeply embedded within systems that may or may not access or use the vulnerable clock functions of their microprocessors, the direct and indirect effects and consequences of the Y2K bug could not be predicted with any certainty, despite the fact that the Y2K event would happen with complete certainty. [Yardeni, 1999] However, just because a person cannot articulate the The objective part of the questionnaire was divided logic behind his/her conclusions does not necessarily into two sections. Section 1 asked 11 questions about the imply that there is no logic, i.e., that the prediction childhood experiences of the subject. Section 2 asked 18 process is random. There may be very strong and robust questions about the current life of the subject. In both logical processes operating in the unconscious mind that sections most of the questions were multiple choice, but cannot be directly examined, but nevertheless are, to some were short answer. When the data is reduced, an some degree, consistently correct. Moreover, there is no attempt will be made to assign the answers to the short- reason to suppose that such correct unconscious answer questions to a small number of clusters by reasoning processes are limited to experts; perhaps any equating similar answers. mature person who has extracted sufficient relevant The 45 subjective questions prepared for Section 3 of knowledge from the lessons from life can accurately the survey were divided into 6 categories: location, predict some aspects of the impact of a simple but economy, technology, service, transportation, and pervasive technological flaw like the Y2K bug. situation. However, it was decided that asking each subject to answer all 45 questions in all 6 categories (plus 2 Research Objective and Hypothesis the 29 questions in Sections 1 and 2) would place an unreasonable time burden on the subjects. Therefore, The objective of this research study is to identify those Section 3 of the questionnaire presented each subject personal and background characteristics, if any, that are with just one of five randomly-selected subsets of associated significantly with an ability to accurately question categories: location & economy (20 questions), predict the consequences of the Y2K event. Because this technology & service (15 questions), location & study was conducted via a voluntary survey on the technology (18 questions), service & transportation & internet, the validity of the results is limited to people situation (17 questions), economy & transportation & that 1) have access to the internet, 2) willingly and situation (20 questions). Each category is represented in honestly respond to surveys, and 3) are able to two subsets of questions, assuring statistically that each understand and answer the survey questions in either question would be answered the same number of times English or Spanish. over the life of the survey. This study tests the following primary research After the subject had completed all three sections of (alternative) hypothesis HA: Despite an inability to codify the questionnaire, each was asked five final questions, their reasoning processes, people with certain especially designed for this study in consultation with a backgrounds and life experiences can accurately predict clinical psychotherapist. One purpose of these questions effects of the Y2K event in their culture. The null was to identify unusual personality characteristics in a hypothesis H0 that can and must be tested statistically is: subject. Extreme answers to two or more of these There is no significant relationship between the objective questions defined the subject as a probable “outlier”, attributes of the subjects and their subjective predictions whose responses would be disregarded during the of the impact of the Y2K bug. If the null hypothesis is analysis of the survey results. In addition, the answers to rejected, that may suggest the acceptance of the research these questions might yield some additional useful hypothesis. demographic information about the cohort. [Bock, 2001] Using the randomized block design described above 3 Methodology for the subjective questions, the minimum number of In early September 1999, the Y2K Impact Survey questions presented to a subject was 29+15+5 = 49, and questionnaire was posted on the internet at the URL the maximum number of questions presented to a subject http://www.mbhs.edu/~adunn/~questionnaire.html. This was 29+20+5 = 54. It was informally determined that the questionnaire solicited predictions of the severity of the average time for answering the questions in all three impact of the Y2K event in several categories (subjective sections was less than ten minutes. responses). In addition, the questionnaire recorded a Between 1 September and 31 December 1999, 202 variety of demographic attributes about each subject questionnaires were filled out, comprising 10550 (objective responses). The questions were composed by responses across all sections. Only 227 or 2.2% of these the authors based on interviews with a small group of responses were “No answer”, in which the respondent did psychologists, sociologists, and engineers. The complete not provide an answer to the question. No respondents English-language questionnaire including the instructions were eliminated as outliers. and on-line help frames may still be viewed at the survey website. Both English and Spanish versions of the 4 Cohort Demographics questionnaire were available, selected by the user on the Table 1 presents the demographic distribution of the first page of the website. People throughout the world cohort of the 202 respondents from 11 countries. Because were invited to participate. Although subjects were asked of the wide variety of answers that were given for the to answer all the questions presented to them, they were short-answer questions, the results for these questions are explicitly permitted to simply skip any questions they did not included in this paper: Ethnic background, Childhood not wish to answer. It was assumed that no one would city, Childhood country, First language, Childhood attempt to “load” the survey with many copies of religion, Primary expertise, Secondary expertise, identical or near-identical answers in an attempt to skew Citizenship, and Current religion. These factors will be the results. considered in the future.

Section 1: Childhood background Gender Male (62) Female (38) Number of siblings 0 (7) 1 (33) 2 (27) 3 (14) 4 (12) 5 (4) 6 (1) 7 (0) 8+ (2) Order among siblings First born (43) Middle born (28) Last born (25) Only child (4) Environment Rural (14) Suburban (63) Urban (23) Military Base (0) Other (0) Special circumstances None (98) Orphan (0) Foster home (0) Adopted (0) Refugee (2) Childhood experience Tragic (1) Difficult (10) Satisfactory (29) Happy (44) Wonderful (16) ------Section 2: Current Information Age ranges from 15 to 85 with an average of 39 (see Table 2) Marital status Single (46) Married (51) Partnered (4) Others in household 0 (20) 1 (25) 2 (16) 3 (19) 4 (14) 5+ (6) Number of children 0 (58) 1 (7) 2 (20) 3 (10) 4 (3) 5 (2) Physical health Excellent (49) Good (47) Fair (3) Poor (1) Mental health Excellent (62) Good (36) Fair (2) Poor (0) Handicapped Yes (4) No (96) Education level Secondary (24) Bachelors (34) Masters (27) Doctorate (15) Occupation (see Table 2) Income bracket Very low (5) Low (8) Medium (49) High (31) Very high (7) Travel experience None (8) Little (27) Some (40) Much (25) Foreign languages 0 (47) 1 (39) 2 (10) 3+ (4) Military service None (85) Enlisted (11) Officer (4) Criminal record Yes (0) No (100) ------Final Section: Outlier Identification Current life status Getting worse (3) Not changing (44) Getting better (53) Survey responses 1 (100) 2+ (0) Government response Terrible (3) Poor (20) Satisfactory (49) Good (21) Excellent (7) Y2K bug cause Simple oversight (33) Poor planning (52) Apathy (5) Negligence (10) Conspiracy (0) Time aware of Y2K 0-6 months (18) 6-12 months (26) 1-3 years (46) 3-10 years (10) More than 10 years (0) Table 1: Demographic responses of the cohort (% of total number of responses per category)

Inspection of the distributions in Table 1 reveals that each category, three metrics were postulated, a category the majority of the cohort was healthy, middle-income, correlation confidence, a prediction score S, and the well-educated, and internet-savvy. All conclusions drawn prediction score confidence. from this study must be qualified with this observation. The category correlation confidence is the statistical confidence that the correlation of the distributions of the 5 Initial Results and Conclusions number of responses for a demographic category (e.g., As of the submission of this paper (mid-January 2000), occupation) for the noneMinor and sigSevere samples is very few serious Y2K problems have been reported, if non-zero. This confidence was used to test the null any. Thus, the initial analysis of the data proceeded under hypothesis for an entire category. It is reported as a the assumption that the correct overall prediction of the number in the range -100% to +100%, with the sign severity of the Y2K bug was None or Minor, and by reflecting the sign of the correlation, which may range implication any overall estimate of the severity of the from -1 to 1. A correlation confidence of -100% implies Y2K bug as Significant, Major, or Severe was incorrect. that the two distributions are so different that they are The distribution of overall severity estimates by the completely anticorrelated; a confidence of 0 implies that cohort was: None = 6.5%; Minor = 75%; Significant = the distributions are not correlated; and a confidence of 13%; Major = 3.5%; Severe = 2.5%. Clearly, a large +100% implies that the distributions are completely majority of the subjects (82%) estimated the overall correlated. A low or negative correlation confidence severity correctly as either None or Minor. suggests the noneMinor and sigSevere response To determine which answer within each demographic distributions for all answers in the category are very category predicted the overall severity correctly and different. [Mendenhall and Sincich, 1992] incorrectly, the responses of the cohort were split into The prediction score S was postulated as a function two samples: those who responded correctly as None or of the sum and difference of the number of correct and Minor, called the noneMinor sample, and those who incorrect responses for each answer in a category: responded incorrectly as Significant, Major, or Severe, called the sigSevere sample. For each possible answer in S = 100 R - W confidence that the error in the number of sample R + W responses was less than one; and 2) the maximum of the statistical confidence that the number of noneMinor where R = the number of responses for the answer responses was significantly greater than zero, and the in the noneMinor sample (correct) statistical confidence that the number of sigSevere W = the number of responses for the answer responses was significantly greater than zero. These in the sigSevere sample (incorrect) added restrictions help to differentiate between “certain” This prediction score has a range from -100 to +100, and “uncertain” predictions. [Freund, 1960] [Bock, 2001] where -100 means that all answers were incorrect, and Table 2 lists the results of this statistical analysis for +100 means that all answers were correct. This is an the 13 demographic categories that yielded a prediction unforgiving metric, because it penalizes guessing. It was score confidence of at least 87% for at least one answer, used for this analysis because each subject was freely ordered by increasing category correlation confidence allowed to omit any answer, and this score differentiates (bold). This rather low confidence threshold reflects a strongly between omitted answers and wrong answers. natural break in the distribution of the score confidences The prediction score confidence for each answer at about 87%. (The reader may easily apply a higher was postulated to be the product of three statistical confidence threshold, if desired.) Each prediction score confidences of the reliability of the prediction score. confidence that exceeds 87% is followed by a one-word First, to exclude those prediction scores that were not characterization of the quality of the prediction. Scores statistically significant, the statistical confidence of each near zero imply no prediction strength at all, right or score was based on the standard error of the difference wrong, which are defined as “weak” predictions. between two proportions, which is the confidence that the Therefore, the quality of scores whose absolute values are null hypothesis for the specific answer may be rejected. less than 20 but have a confidence of at least 87% are This confidence was then multiplied by two additional marked “X”, meaning that the predictions are “clearly error and data-sufficiency confidences: 1) the statistical weak”. The actual sample response distributions are listed on the righthand side of the table.

category X = clearly weak sample response distributions demographic correlation predictio category conf (%) answer score nconf (%) quality numbe % number % number % r age -36 15 - 20 -56 98 poor 18 9.2 10 6.3 8 22.2 20 - 25 -63 98 bad 26 13.3 13 8.2 13 36.1 25 - 30 34 74 20 10.3 18 11.3 2 5.6 30 - 35 -12 40 27 13.8 21 13.2 6 16.7 35 - 40 45 89 fair 25 12.8 23 14.5 2 5.6 40 - 45 45 89 fair 25 12.8 23 14.5 2 5.6 45 - 50 55 87 fair 16 8.2 15 9.4 1 2.8 50 - 55 100 97 good 12 6.2 12 7.5 0 0.0 55 - 60 39 64 11 5.6 10 6.3 1 2.8 > 60 52 84 15 7.7 14 8.8 1 2.8 number of 1 0 3 14 39 20.0 32 20.3 7 18.9 others in 1 3 16 50 25.6 41 25.9 9 24.3 household 2 54 97 fair 31 15.9 29 18.4 2 5.4 3 32 86 37 19.0 33 20.9 4 10.8 4 -36 93 poor 27 13.8 18 11.4 9 24.3 > 5 -67 98 bad 11 5.6 5 3.2 6 16.2 marital 46 single -27 95 poor 89 45.2 64 39.8 25 69.4 status married 38 96 fair 99 50.3 90 55.9 9 25.0 partnered -12 24 9 4.6 7 4.3 2 5.6 highest 52 secondary -29 92 poor 45 23.0 32 20.0 13 36.1 educational bachelors -17 83 66 33.7 50 31.3 16 44.4 level masters 31 93 fair 57 29.1 51 31.9 6 16.7 doctorate 72 99 good 28 14.3 27 16.9 1 2.8 government 77 terrible -37 58 6 3.1 4 2.5 2 5.6 response to poor -52 97 poor 38 19.7 22 14.0 16 44.4 Y2K problem satisfactory 14 80 95 49.2 81 51.6 14 38.9 good 35 90 fair 40 20.7 36 22.9 4 11.1 excellent 100 98 good 14 7.3 14 8.9 0 0.0 Table 2: Results for demographic categories with at least one prediction score confidence of at least 87% category X = clearly weak sample response distributions demographic correlation predictio both noneMi sigSeve category conf (%) answer score nconf (%) quality numbe % numbernor % numberre % r current 91 getting worse -79 95 bad 6 3.1 2 1.3 4 10.8 life not changing 17 84 84 43.5 72 46.2 12 32.4 status getting better -4 34 103 53.4 82 52.6 21 56.8 international 92 none -37 82 15 7.6 10 6.2 5 13.5 travel little 28 88 fair 52 26.7 46 28.6 6 16.2 experience some -2 14 83 42.6 67 41.6 16 43.2 much -7 33 48 24.6 38 23.6 10 27.0 number of 94 0 18 87 X 89 45.6 77 48.4 12 33.3 foreign 1 -13 73 75 38.5 58 36.5 17 47.2 languages 2 -19 59 24 12.3 18 11.3 6 16.7 > 3 15 24 7 3.6 6 3.8 1 2.8 occupation 98 student -58 97 poor 31 20.1 17 51.5 14 73.7 sales -38 55 6 3.9 4 4.7 2 7.1 teacher -6 14 15 9.7 12 16.9 3 11.5 engineer 9 41 45 29.2 38 30.2 7 25.0 manager 22 46 16 10.4 14 13.2 2 7.1 artist 100 38 1 0.6 1 0.1 0 0.0 lawyer 100 68 3 1.9 3 0.3 0 0.0 bureaucrat 100 68 3 1.9 3 0.3 0 0.0 doctor 100 82 5 3.2 5 0.4 0 0.0 homemaker 100 92 good 8 5.2 8 0.7 0 0.0 technician 100 93 good 9 5.8 9 7.1 0 0.0 executive 100 97 good 12 7.8 12 1.1 0 0.0 military 99 none 10 93 X 158 81.9 133 84.7 25 69.4 service enlisted -4 13 20 10.4 16 10.2 4 11.1 officer -58 98 poor 15 7.8 8 5.1 7 19.4 cause of 100 simple oversight -12 66 65 33.3 50 31.6 15 40.5 Y2K problem poor planning 11 72 101 51.8 85 53.8 16 43.2 apathy 100 95 good 10 5.1 10 6.3 0 0.0 negligence -33 82 19 9.7 13 8.2 6 16.2 conspiracy 0 0 0 0.0 0 0.0 0 0.0 number 100 0 -18 96 X 114 58.2 86 53.8 28 77.8 of 1 49 80 14 7.1 13 8.1 1 2.8 children 2 33 88 fair 39 20.1 35 22.1 4 11.1 3 33 72 20 10.0 18 11.0 2 5.6 > 4 29 47 9 4.6 8 5.0 1 2.8 childhood 100 rural -13 43 26 13.3 20 12.6 6 16.2 environment suburban 15 87 X 115 58.7 98 61.6 17 45.9 urban -19 82 55 28.1 41 25.8 14 37.8 Table 2 (cont.): Results for demographic categories with at least one prediction score confidence of at least 87%

In addition to the demographic categories that asked For each of the 13 categories that are reported in for short answers, the results for several of the multiple- Table 2, the following conclusions may be drawn: choice categories listed in Table 1 are not included in this 1) As age increases from 15 to 55, prediction score paper, because they did not exhibit any statistically increases consistently from bad to good, but declines significant prediction capabilities: gender, number of somewhat thereafter. Respondents between 35 and 55 siblings, order among siblings, childhood experience, are fair to good predictors. Respondents between 15 physical health, mental health, handicapped, income and 25 are poor to bad predictors. The predictions of bracket, and criminal record. other age groups are uncertain and rather weak. 2) As the number of other people living in the reported in Table 2 only indicate the probability of household increases, prediction score decreases avoiding Type I errors. There is no way to compute the almost monotonically from fair to bad. Respondents Type II error probabilities, because the Y2K impact was living with 2 other persons in their households are not mixed; only minor Y2K problems have been fair predictors. Respondents living with 4 or more reported. Thus, there is no way of knowing how well the other persons are poor to bad predictors good predictors would have performed if a category had 3) For the category marital status, single respondents experienced a significant, major, or severe problem. are poor predictors; married respondents are fair However, one may safely, albeit cautiously, conclude predictors; the predictions of partnered respondents that those respondents whose predictions were clearly are very uncertain and weak. weak (marked X in Table 2) should not be selected to make predictions of this kind, i.e., avoid people who 4) As education level increases, prediction score speak no foreign languages, have never been in the increases monotonically from poor to good. The military, are students, have no children, or spent their predictions of respondents with bachelor degrees are childhood in suburbia. weak and somewhat uncertain. The predictions of The initial results and conclusions of this study have respondents with masters degrees are fair, and the been posted on the internet at the following website: predictions of respondents with doctorates are good. http://www.seas.gwu.edu/~pbock (click on Results of the 5) As the perceived quality of the government Y2K Impact Study). All are welcome to visit this site to response to the Y2K problem improves, prediction view new results and conclusions as they emerge from score increases almost monotonically from poor to the ongoing analysis by the survey research teams at the good, although the predictions for two of the five George Washington University and The University of the answers are uncertain and, in one case, weak. Americas. Those who would like copies of the data files 6) For current life status, respondents who think their as a basis for their own research investigations are invited lives are getting worse are bad predictors. The to contact Professor Peter Bock. predictions of respondents who think their lives are The authors would like to express their gratitude to unchanging or getting better are uncertain and weak. the people all over the world who responded to the survey before 1 January 2000. In addition, we are grateful 7) Respondents who have had just a little international to Montgomery Blair High School in Silver Spring, travel experience are fair predictors, but predictions Maryland, for contributing computer time and space for of respondents with no travel experience are the website for this project, and to the professional and uncertain. Predictions of respondents with some or commercial websites that agreed to maintain links to the much travel experience are very uncertain and weak. project website for the duration of the survey. Finally we 8) Respondents who speak no foreign languages are would also like to thank psychotherapist Ms. Donna clearly weak predictors. The predictions of Oberholtzer, MA, MSW, for her help in designing the respondents with knowledge of foreign languages are survey questions in the final section. uncertain and weak. 9) Predictors in occupations of executive, technician, References and homemaker are very good. Students are poor predictors. Predictions of teachers, engineers, and [Bock, 2001] Peter Bock. R&D Methods for Science and managers are weak, but uncertain. Predictions of Engineering. Academic Press, San Diego, 2001 (in press) salespersons, doctors, artists, lawyers, bureaucrats appear good, but are uncertain due to lack of data. [Freund, 1960] John E. Freund. Modern Elementary Statistics. Prentice-Hall, Englewood Cliffs, 1960. 10) Predictions of respondents with no military service are clearly weak. Predictions of respondents who are [Mendenhall and Sincich, 1992] William Mendenhall and or were military officers are poor. Predictions of Terry Sincich. Statistics for Engineering and the respondents who are or were enlisted personnel in the Sciences. Dellen Publishing Co., San Francisco, 1992. military are uncertain and weak. 11) Predictions by respondents who characterized the [Yardeni, 1999] Ed Yardeni. Dr. Ed Yardeni’s cause of the Y2K bug as apathy are fair predictors. Economics Network. http://www.yardeni.com. Deutsche All others are quite weak and uncertain. Bank, 1999. 12) Predictions of respondents with no children are clearly weak. Those with two children are fair [Yourdon, 1999] Ed Yourdon. Y2K Links and Resources. predictors. All other predictions are uncertain. http://www.yourdon.com/y2kresources.html 13) The predictions of respondents whose childhood environment was suburban are clearly weak. The predictions of respondents brought up in urban and rural environments are uncertain and weak.

It is important to note that the statistical confidences