Pierre Philip* Could a Virtual Human Be Used to Universite´ de Bordeaux CNRS USR-3413 Explore Excessive Daytime Bordeaux, France Sleepiness in Patients? and Clinique du Sommeil, CHU Pellegrin Bordeaux, France

Ste´ phanie Bioulac Universite´ de Bordeaux Abstract CNRS USR-3413 Bordeaux, France Excessive daytime somnolence (EDS) is defined as the inability to stay awake in daily and life activities. Several scales have been used to diagnose excessive daytime sleepiness, Poˆle de Pe´dopsychiatrie the most widely used being the Epworth Sleepiness Scale (ESS). Sleep disorders and Universitaire EDS are very common in the general population. It is therefore important to be able Hoˆpital Charles Perrens to screen patients for this symptom in order to obtain an accurate diagnosis of sleep France disorders. Embodied Conversational Agents (ECA) have been used in the field of affective computing and human interactions but up to now no software has been spe- Alain Sauteraud cifically designed to investigate sleep disorders. We created an ECA able to conduct Universite´ de Bordeaux an interview based on the ESS and compared it to an interview conducted by a sleep CNRS USR-3413 specialist. We recruited 32 consecutive patients and a group of 30 healthy volunteers Bordeaux, France free of any sleep complaints. The ESS is a self-administered questionnaire that asks the subject to rate (with a pen and paper paradigm) his or her probability of falling asleep. Cyril Chaufton For the purpose of our study, the ECA or real-doctor questionnaire was modified as Universite´ de Bordeaux follows: Instead of the ‘‘I’’ formulate, questions were asked as ‘‘Do you.’’ Our software CNRS USR-3413 is based on a common 3D game engine and several commercial software libraries. It Bordeaux, France can run on standard and affordable hardware products. The sensitivity and specificity and of the interview conducted by the ECA were measured. The best results (sensibility Clinique du Sommeil, CHU Pellegrin and specificity >98%) were obtained to discriminate the sleepiest patients (ESS 16) Bordeaux, France but very good scores (sensibility and specificity >80%) were also obtained for alert subjects (ESS<8). ESS scores obtained in the interview conducted by the physician Je´ roˆ me Olive were significantly correlated with ESS scores obtained in the interview the ECA con- Universite´ de Bordeaux ducted. Most of the subjects had a positive perception of the virtual physician and con- CNRS USR-3413 sidered the interview with the ECA as a good experience. Sixty-five percent of the Bordeaux, France participants felt that the virtual doctor could significantly help real physicians. Our results show that a virtual physician can conduct a very simple interview to evaluate EDS with very similar results to those obtained by a questionnaire administered by a real physician. The expected massive increase in sleep complaints in the near future likely means that more and more physicians will be looking for computerized systems to help them to diagnose their patients.

1 Introduction

Excessive daytime sleepiness (EDS) is defined as sleepiness that occurs in a situation when an individual would usually be expected to be awake and alert (Littner et al., 2005). This very disabling symptom, which is responsible for Presence, Vol. 23, No. 4, Fall 2014, 369–376 doi:10.1162/PRES_a_00197 ª 2015 by the Massachusetts Institute of Technology *Correspondence to [email protected].

Philip et al. 369

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00197 by guest on 30 September 2021 370 PRESENCE: VOLUME 23, NUMBER 4

many traffic accidents (Philip, 2010), affects up to 19.5% bally and nonverbally. They can attract and maintain the of the general population in Western countries when attention of users in an interaction (Peters & Grandjean, subjective assessment is used (Ohayon, 2008, 2012). 2011), disambiguate written or spoken texts, and make This number is increasing owing to an increase in interactions more expressive and more socially adapted. chronic sleep deprivation in young people (Leger et al., ECAs can also control somewhat the communication of 2011) and an increase in obstructive sleep apnea in older their emotional states. They can display a large range of subjects (Norman & Loredo, 2008). Nevertheless, emotions (Arya, DiPaola, & Parush, 2009), such as dif- although EDS is becoming a real public health issue, ferent types of smiles (Ochs, Niewiadomski, Brunet, & very few physicians are trained to adequately differentiate Pelachaud, 2012). Emotions such as relief, embarrass- fatigue from EDS. Indeed fatigue, for which the treat- ment, anxiety, and regret can also be seen as sequences ment is rest, can be defined as a growing difficulty to per- of multimodal signals (Niewiadomski, Hyniewska, & form with a progressive decrease in performance. Sleepi- Pelachaud, 2011). In addition, several studies have ness, for which the treatment is sleep, is the difficulty in shown that when they use the vehicle of a humanoid remaining awake. face, human–machine interactions are perceived as being Both symptoms can alter performance but their causes similar to human–human interaction (Reeves & Nass, and treatments are radically different. This is a major 1996). The machine is perceived more as a socio-emo- issue because many patients suffering from sleep disor- tional actor than a tool (Reeves & Nass, 1996).This feel- ders are diagnosed as psychiatric patients (mood or anxi- ing is reinforced when the interface takes the form of an ety disorders) and therefore receive central nervous sys- ECA. tem drugs, which increase their level of sleepiness and ECAs have been used in a wide range of clinical appli- driving risk (Blazejewski, Girodet, Orriols, Capelli, & cations and they can act as therapists with whom human Moore, 2012; Orriols et al., 2011). patients can interact. The US Army has funded programs Several scales have been used to diagnose (Akerstedt to test the potential use of these software programs for & Gillberg, 1990; Hoddes, Zarcone, Smythe, Phillips, soldiers coming back from Iraq and suffering from post- & Dement, 1973) excessive or chronic daytime sleepi- traumatic stress disorder (PTSD) (Rizzo et al., 2009). ness, the most widely used being the Epworth Sleepiness They can also act as social companions to help teenagers Scale (ESS; Johns, 1991). This self-administered scale with substance abuse problems (Lisetti & Wagner, developed in the 1990s has been used in thousands of 2007). ECAs can also act as a patient seeking therapeutic studies and is the simplest way to self-evaluate EDS. assistance. Young practitioners interact with them to Sleep disorders are very common and EDS is a major learn how to interact with traumatic patients (Marsella, symptom associated with various sleep disorders such as Johnson, & LaBore, 2003). obstructive sleep apnea syndrome, restless legs syn- ECAs have been used in the field of affective comput- drome, and hypersomnias. It is therefore important to ing and human interactions (Garau, Slater, Pertaub, & be able to screen patients for this symptom in order to Razzaque, 2005) but up to now no software has been obtain an accurate diagnosis. specifically designed to investigate sleep disorders. Since Virtual reality has been used for clinical purposes for the ESS is the most widely validated scale to evaluate Ex- several years (Bioulac et al., 2012; Giroux et al., 2013) cessive Daytime Sleepiness (Johns, 2010), we hypothe- and there is a growing interest in Embodied Conversa- sized that although it was initially designed as a self- tional Agents (ECAs) to be used as a new type of administered questionnaire, it was a suitable model for human–machine interface (DeVault et al., 2013). ECAs assessing the ability of an ECA to investigate excessive aim to unite gesture, facial, and verbal expression to ena- daytime sleepiness. ble face-to-face communication with users, providing a We therefore created software (ECA) able to conduct strong means of human–system interaction. They are an interview based on the Epworth Sleepiness Scale autonomous entities that are able to communicate ver- (ESS) and compared it to an interview conducted by a

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00197 by guest on 30 September 2021 Philip et al. 371

sleep specialist. Our main objective was to assess the sen- asleep. The ESS refers to driving, watching TV, talking sitivity and specificity of ESS scores collected by the to someone, or reading in common daily life situations ECA. where sleepiness is explored. The answers for the eight In order to minimize the self-perceived differences questions are added together to obtain a single score. A between the ECA and the real physician, subjects did score in the 0–8 range is considered to be normal while a not know whether the ECA was operated by a human score above 11 indicates moderate to severe sleepiness, (Wizard of Oz) or by artificial intelligence software. suggesting that expert medical advice should be sought. We have shown that high ESS scores (>15) correlate 2 Methods with the risk of traffic accidents (Philip, 2010), and sev- eral studies have shown that treatment affecting daytime 2.1 Population alertness significantly lowers ESS scores (Hardinge, Pit- We recruited 32 consecutive patients (18 men and son, & Stradling, 1995). 14 women, mean age: 46.7 6 12.8) attending the Bor- The ESS (Johns, 1991) was originally created with the deaux sleep clinic for a consultation and a group of 30 intention of preserving the exact wording of the ques- healthy volunteers (8 men and 22 women, mean age: tionnaire so as to provide a standardized test and pre- 37.3 6 11.0) free of any sleep complaints. Subjects were serve its validity. Its author recommends that the admin- informed that they would have an interview with a vir- istrator of the questionnaire not discuss the results of the tual physician operated by an ECA and then a similar ESS with the subject until it is completed, as this may interview with a real physician. affect the subject’s responses. All subjects were tested for audition and proper vision For the purpose of our study, the ECA or real-doctor in order to be able to see and understand the ECA. Con- questionnaire was modified as follows: Instead of the ‘‘I’’ sequently, no subjects older than 70 were included in formulate, questions were asked as ‘‘Do you.’’ The rest the study. of the questionnaire was strictly identical to the self- All mental disorders interfering with proper interac- administered version. tion with the ECA were considered as exclusion criteria (anxiety disorders, severe mood disorders, psychosis). 2.3 Pretest Only one subject refused to perform the interview with To be sure that this version of the ESS was free of the ECA for ethical reasons (refused the video-record- any learning effect, 49 subjects tested it twice at an inter- ing). val of one hour. The hetero questionnaire was adminis- All participants read an information sheet and gave tered by a clinical research assistant in a similar way to written informed consent to participate in the study. The that of the real physician during the study. Replication of protocol was validated by the Bordeaux University the ESS provided very consistent results with no statisti- Ethics Committee. cal changes between the tests (Pearson: r ¼ 0.8, p < 0.001). 2.2 Epworth Sleepiness Scale 2.4 Virtual Human Software The Epworth Sleepiness Scale (ESS) is a self- administered questionnaire that asks the subject to rate Our ECA system is based on four software mod- (with a pen-and-paper paradigm) his or her probability ules. The first and main module is defined as the inter- of falling asleep. They fill in a scale of increasing proba- view manager. It conducts the whole interview (ques- bility from 0 to 3 for eight different situations that most tions, expected answers, scripted gestures, and scripted people engage in during their daily lives. Zero corre- emotions) and manages the other modules. Instead of sponds to no chances of falling asleep, 1 a small chance, scripted behaviors, this module generates ECA behaviors 2 a moderate chance, and 3 a very high chance of falling based on predefined or random rules. All interviews are

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00197 by guest on 30 September 2021 372 PRESENCE: VOLUME 23, NUMBER 4

3D Rendering Module

ECA Animations picture

Viseme commands

Interview Text to Artificial Voice Synthesizer manager speech voice subject

Recognized answers Subject answers Expected answers dictionary Voice Recognizer

Figure 1. ECA architecture and interactions.

stored in XML files. The second module is a 3D render- head. Figure 1 describes the overall design and interac- ing module. Its role is to display our ECA and play ani- tive mode of the ECA. mations on command. It was created with Unity3D (Unity-Technologies, 2014), a 3D gaming engine, and 2.5 Perception of ECA and Pertinence it uses 81 bones 3D characters from Rocketbox Libraries of the Questions (RocketBox-Libraries, 2014). Characters can be ani- We designed a questionnaire to investigate the mated in terms of gestures, facial expressions, and quality of interaction between the ECA and the subjects. visemes (facial expressions corresponding to enunciation The seven questions asked the subjects the following: of phonemes). The third module operates the voice rec- whether they were face to face with a virtual physician or ognizer. We use the voice recognition module from a real physician animating an avatar; whether they Microsoft Kinect SDK (Microsoft, 2014). The interview believed they were understood correctly by the virtual manager feeds this module with dictionaries containing physician (ECA software); whether the virtual physician’s expected answers from participants. Answers of partici- questions were understandable; whether the interview pants after analysis are transmitted to the interview man- with the virtual physician was a pleasant experience; ager. The fourth and last module is a speech synthesizer. whether the virtual physician was perceived as a compe- It creates ECA speech sent by the interview manager tent physician; whether the ECA relied on a high quality and, for each enunciated phoneme, sends the corre- technological tool; and whether the virtual physician sponding viseme command to the 3D rendering module. could help a real physician to treat a patient. These four modules are thread independent. Modules All questions were scored on the following four- communicate by TCP sockets that can be distributed to choice display: 3) Absolutely yes, 2) Rather yes, 1) several computers. Rather no, and 0) Not at all. The ECA software suite was installed on a standard gaming computer (Windows 8 – i7 [email protected] – 8 2.6 Study Design GB – NVidia 670 GTX) connected to a 40-inch display. As input device, we used exclusively the Microsoft Kinect We submitted the ESS via an interview with an sensor for voice recognition and to monitor the user’s ECA to all subjects and patients. To ensure the quality

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00197 by guest on 30 September 2021 Philip et al. 373

Table 1. Epworth Sleepiness Scale Scores Obtained by Real Physician Versus Virtual Physician

Group by score 0–8 9–11 12–15 16–24 Total

Real physician N 39 10 9 4 62 Mean score ESS 6 SD 5.6 6 2.2 10.1 6 1.0 13.1 6 1.0 19.0 6 2.9 7.7 6 4.9 Virtual physician N 38 11 8 5 62 Mean score ESS 6 SD 5.4 6 2.07 10.1 6 0.7 12.9 6 0.8 17.8 6 2.0 8.2 6 4.4

Table 2. Sensitivity and Specificity of Interview Conducted by 3 Results ECA 3.1 Epworth Sleepiness Scale Scores Cut Off >¼8 >¼11 >¼16 Table 1 presents mean ESS scores in both condi- ECA Sensibility 0.89 0.77 1 tions. To report the full range of answers, we divided the Specificity 0.81 0.97 0.98 results into four categories: 0–8 (fully alert); 9–11 (alert); 12–15 (sleepy); and 16 (very sleepy). In both situations, more than 60% of the subjects did not exhibit of the virtual interview, all subjects were video-recorded. sleepiness, with an ESS score lower than 8. More than Mean levels of anxiety and sadness in the last seven days 20% of the subjects exhibited moderate-to-severe sleepi- were also quantified by a self-administered questionnaire ness in both conditions (>10). rated with a Likert score (0–3). At the end of the ECA interview, subjects filled in a 3.2 Sensitivity and Specificity of the self-administered questionnaire on perception of the ECA Interview ECA and pertinence of the questions. After performing the interview with the ECA, each The sensitivity (ability to identify positive results) subject was then interviewed by a sleep physician using and specificity (ability to identify negative results) of the the same modified ESS. All subjects were free of any pre- ECA interview were measured. The best results were vious testing by the ESS. obtained to discriminate the sleepiest patients (ESS >¼ 16) but very good scores (>80%) were also obtained to discriminate alert subjects (ESS < 8; see Table 2). 2.7 Statistical Analysis

Responses to questions on the Virtual Human ESS 3.3 Correlations Between Epworth and Physician ESS were expressed by descriptive statistics Sleepiness Scale Scores Obtained by Real (numbers of patients and means [6SD]). and Virtual Physicians Correlations were made between the Virtual Human Pearson correlations indicated that raw ESS scores ESS and Physician ESS scores using Pearson’s correla- obtained by the physician were significantly correlated tion test. with those obtained by the ECA (Pearson: r ¼ 0.95, Sensitivity and specificity were calculated for Virtual p < .0001) (see Figure 2). Human ESS and Physician ESS for different ESS scores according to the author’s recommendations (Johns, 3.4 Perception of ECA and Pertinence 1991) with cut-off values equal to or greater than 8, 11, of Questions and 16. Computations were performed by using R for Win- We developed a questionnaire to investigate the dows (R-Project, 2014). quality of interaction between the ECA and the subjects.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00197 by guest on 30 September 2021 374 PRESENCE: VOLUME 23, NUMBER 4

Figure 2. Correlations between Epworth Sleepiness Scale scores obtained by real and virtual physician.

Table 3. Perception of ECA Table 3 presents the responses of the subjects on this questionnaire. Most of the subjects had a positive per- No Yes ception of the virtual physician and considered the inter- Q1: Is virtual physician 2% 98% view with the ECA as a good experience. Sixty-five per- computer-piloted? cent of them considered that the virtual doctor could Q2: Understanding by virtual 5% 95% offer significant assistance to the real physician. physician Q3: Understanding of virtual 3% 97% 4 Discussion physician Q4: Pleasant experience 15% 85% This study is the first to use an ECA to explore ex- Q5: Competent physician 18% 82% cessive daytime sleepiness in humans. To do so, the ESS, Q6: Quality technological tool 8% 92% which is a self-administered questionnaire, was adapted Q7: Can help real physician 35% 65% for an interview between an ECA or a real physician and

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00197 by guest on 30 September 2021 Philip et al. 375

participants. To be sure that no learning effect would and the pertinence of questions are probably very much interfere with our data, a validation study of the test– in favor of the perception of the ECA (82% of the sub- retest value of the ESS was performed as an interview by jects considered the virtual doctor as competent). a research assistant. This showed that the ESS was a ro- Finally, 65% of the subjects considered that the virtual bust tool and that ESS scores were identical between doctor could be of significant assistance for a real physi- two interviews an hour apart. cian. We cannot rule out a selection bias in our small The results show that a virtual physician can conduct a sample of subjects who might have a propensity to like very simple interview to evaluate EDS and obtain results new technologies. However, access to physicians may be very similar to those obtained by a questionnaire very difficult in western countries and many patients pos- (Epworth Sleepiness Scale) administered by a real physi- sibly consider that virtual humans will ultimately become cian. Since we wanted to explore the full range of more accessible than real physicians. Ultimately, because answers in the ESS, we included patients and volunteers the ECA is an autonomous agent, we expect that it could free of sleep complaints. There were minor differences in provide the usability of the self-administered version scores between the ESS administered by a real doctor with the reliability of a physician explaining (and/or and by a virtual physician. Nevertheless, the distribution repeating when necessary) the instructions. between alert, sleepy, and very sleepy subjects remained Acceptability by the medical profession is also a big largely the same. Given the sensitivity and specificity of challenge because some physicians might fear that such the virtual physician software at the three thresholds new software programs will replace their function in so- (>¼8, >¼11, and >¼16), the best results were ciety. The expected massive increase in sleep complaints obtained in the sleepiest patients (ESS >¼ 16). One pos- in the near future likely means that more and more sible explanation for this is that the answer ‘‘a very high physicians will look for computerized systems to help chance’’ is much simpler to select than responses like them in their diagnostic procedures. This study was the ‘‘little chance’’ or ‘‘moderate chance.’’ It is easier to rec- first step in creating an interview performed by an ECA ognize a significant complaint that has an impact on the in the field of sleep disorders. Further studies are subject’s daytime functioning. required on the usability of the ECA scale in larger pop- Another important dimension is how the ECA is per- ulations to confirm the improved comfort and under- ceived and whether the patient is willing to use it alone standing of the ECA scale versus the ESS scale in sleep or in addition to a real doctor. Almost every subject disorder patients. Finally, future research in the field of identified the ECA as being animated by software and therapeutic interventions is also urgently needed to not a Wizard of Oz. Even so, most subjects had the feel- improve sleep treatments in modern societies. ing that the ECA understood their answers satisfactorily and they also considered that they were able to under- References stand the questions formulated by the ECA very well. It is therefore reasonable to believe that even though our Akerstedt, T., & Gillberg, M. (1990). Subjective and objective ECA was not equipped with software able to read non- sleepiness in the active individual. International Journal of verbal cues, it is still convincing as a conversational Neuroscience, 52(1–2), 29–37. agent. In this initial study, we did not analyze the non- Arya, A., DiPaola, S., & Parush, A. (2009). Perceptually valid verbal cues of patients and healthy volunteers because facial expressions for character-based applications. Interna- the ESS per se does not request subjects to express their tional Journal of Computer Game Technology, 1–14. doi: emotions. Another important dimension was also the 10.01155/2009/462315 emotional perception of the virtual doctor (Did you have Bioulac, S., Lallemand, S., Rizzo, A., Philip, P., Fabrigoule, C., a pleasant experience during the test?) by the subjects & Bouvard, M. P. (2012). Impact of time on task on ADHD and patients tested. Here again, 85% of the subjects con- patient’s performances in a virtual classroom. European Jour- sidered that the test was pleasant. The novelty of the test nal of Paediatric Neurology, 16(5), 514–521.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00197 by guest on 30 September 2021 376 PRESENCE: VOLUME 23, NUMBER 4

Blazejewski, S., Girodet, P. O., Orriols, L., Capelli, A., & tional Conference on Artificial Intelligence in Education. Moore, N. (2012). Factors associated with serious traffic Sydney, Australia, 341–348. crashes: A prospective study in southwest France. Archives of Microsoft. (2014). Kinect SDK. Internal Medicine, 172(13), 1039–1041. Niewiadomski, R., Hyniewska, S., & Pelachaud, C. (2011). DeVault, D., Georgila, K., Artstein, R., Morbini, F., Traum, Constraint-based model for synthesis of multimodal sequen- D., Scherer, S., et al. (2013). Verbal indicators of psycholog- tial expressions of emotions. IEEE Transactions of Affective ical distress in interactive dialogue with a virtual human. SIG- Computing, 2(3), 134–146. DIAL 2013 Conference, 193–202. Norman, D., & Loredo, J. S. (2008). Obstructive sleep apnea Garau, M., Slater, M., Pertaub, D. P., & Razzaque, S. (2005). in older adults. Clin. Geriatr. Med., 24(1), 151–165, ix. The responses of people to virtual humans in an immersive Ochs, M., Niewiadomski, R., Brunet, P., & Pelachaud, C. virtual environment. Presence: Teleoperators and Virtual (2012). Smiling virtual agent in social context. Cognitive Environments, 14(1), 104–116. Processing, 13 Suppl 2, 519–532. Giroux, I., Faucher-Gravel, A., St-Hilaire, A., Boudreault, C., Ohayon, M. M. (2008). From wakefulness to excessive sleepi- Jacques, C., & Bouchard, S. (2013). Gambling exposure ness: What we know and still need to know. Sleep Medicine in virtual reality and modification of urge to gamble. Reviews, 12(2), 129–141. Journal of CyberPsychology and Social Networking, 16(3), Ohayon, M. M. (2012). Determining the level of sleepiness in 224–231. the American population and its correlates. Journal of Psychi- Hardinge, F. M., Pitson, D. J., & Stradling, J. R. (1995). Use atric Research, 46(4), 422–427. of the Epworth Sleepiness Scale to demonstrate response to Orriols, L., Philip, P., Moore, N., Castot, A., Gadegbeku, B., treatment with nasal continuous positive airways pressure in Delorme, B., et al. (2011). Benzodiazepine-like hypnotics patients with obstructive sleep apnoea. Respiratory Medicine, and the associated risk of road traffic accidents. Clinical 89(9), 617–620. Pharmacology and Therapeutics, 89(4), 595–601. Hoddes, E., Zarcone, V., Smythe, H., Phillips, R., & Dement, Peters, C., & Grandjean, D. (2011). Novelty processing and W. C. (1973). Quantification of sleepiness: A new approach. emotion: Conceptual developments, empirical findings and Psychophysiology, 10(4), 431–436. virtual environments. In P. Petta, C. Pelachaud, & R. Cowie Johns, M. W. (1991). A new method for measuring daytime (Eds.), Novelty processing and emotion: Conceptual develop- sleepiness: The Epworth Sleepiness Scale. Sleep, 14(6), 540– ments, empirical findings and virtual environments (pp. 545. 441–458). New York: Springer. Johns, M. W. (2010). A new perspective on sleepiness. Sleep Philip, P. (2010). Can the MSLT be a useful tool to assess and Biological Rhythms, 8, 170–179. motor vehicle crash risk in sleepy drivers? Sleep, 33(6), 729– Leger, D., Roscoat, E., Bayon, V., Guignard, R., Paquereau, 730. J., & Beck, F. (2011). Short sleep in young adults: Insomnia R-Project. (2014). The R Project for Statistical Computing. or sleep debt? Prevalence and clinical description of short Retrieved from http://www.r-project.org/ sleep in a representative sample of 1004 young adults from Reeves, B., & Nass, C. (1996). The media equation: How people France. Sleep Med., 12(5), 454–462. treat computers, television, and new media like real people and Lisetti, C. L., & Wagner, E. (2007). Mental health promotion places. Cambridge: Cambridge University Press. with animated characters: Exploring issues and potential. Rizzo, A. A., Difede, J., Rothbaum, B. O., Johnston, S., Paper presented at the Proceedings of the Stanford AAAI McLay, R. N., Reger, G., et al. (2009). VR PTSD Spring Symposium on Emotion, Behavior and Personnel. exposure therapy results with active duty OIF/OEF Littner, M. R., Kushida, C., Wise, M., Davila, D. G., Mor- combatants. Studies in Health Technology and Informatics, genthaler, T., Lee-Chiong, T., et al. (2005). Practice param- 142, 277–282. eters for clinical use of the multiple sleep latency test and the RocketBox-Libraries. (2014). RocketBox-Libraries. Retrieved maintenance of wakefulness test. Sleep, 28(1), 113–121. from http://rocketbox-libraries.com/ Marsella, S., Johnson, W. L., & LaBore, C. (2003). Interactive Unity-Technologies. (2014). Unity 3D. Retrieved from pedagogical drama for health interventions. 11th Interna- http://unity3d.com/

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00197 by guest on 30 September 2021