20170004329.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
111111111111111111111111111111111111111111111111111111111111111111111111 (12) United States Patent (io) Patent No.: US 9,623,324 B2 Pope et al. (45) Date of Patent: *Apr. 18, 2017 (54) PHYSIOLOGICALLY MODULATING A63F 131213 (2014.01) VIDEOGAMES OR SIMULATIONS WHICH A63F 13142 (2014.01) USE MOTION-SENSING INPUT DEVICES A63F 131212 (2014.01) (52) U.S. Cl. (71) Applicant: The United States of America as CPC .......... A63F 131211 (2014.09); A63F 131212 represented by the Administrator of (2014.09); A63F 131213 (2014.09); A63F the National Aeronautics and Space 13142 (2014.09) Administration, Washington, DC (US) (58) Field of Classification Search CPC ..... A63F 2300/1012; A63B 2024/0012; A63B (72) Inventors: Alan T. Pope, Poquoson, VA (US); 2220/806; A63B 69/3608; G09B 19/0038; Chad L. Stephens, Poquoson, VA (US); G09B 19/00 Nina Marie Blanson, Yorktown, VA USPC .......................................... 434/236, 247-261 (US) See application file for complete search history. (73) Assignee: The United States of America as (56) References Cited represented by the Administrator of the National Aeronautics and Space U.S. PATENT DOCUMENTS Administration, Washington, DC (US) 6,450,820 B1 * 9/2002 Palsson .............. G09B 19/22 (*) Notice: Subject to any disclaimer, the term of this 434/236 patent is extended or adjusted under 35 2011/0009193 Al * 1/2011 Bond ................. A63F 13/06 U.S.C. 154(b) by 396 days. 463/36 This patent is subject to a terminal dis- * cited by examiner claimer. Primary Examiner Andrew Iwamaye (21) Appl. No.: 14/212,159 Assistant Examiner Jerry-Daryl Fletcher (74) Attorney, Agent, or Firm Andrea Z. Warmbier; (22) Filed: Mar. 14, 2014 Jennifer L. Riley; Jonathan B. Soike (65) Prior Publication Data (57) ABSTRACT US 2014/0256431 Al Sep. 11, 2014 New types of controllers allow a player to make inputs to a video game or simulation by moving the entire controller Related U.S. Application Data itself or by gesturing or by moving the player's body in whole or in part. This capability is typically accomplished (63) Continuation-in-part of application No. 13/166,166, using a wireless input device having accelerometers, gyro- filed on Jun. 22, 2011, now Pat. No. 8,827,717. scopes, and a camera. The present invention exploits these (60) Provisional application No. 61/361,084, filed on Jul. wireless motion-sensing technologies to modulate the play- 2, 2010, provisional application No. 61/499,733, filed er's movement inputs to the videogame based upon physi- on Jun. 22, 2011, provisional application No. ological signals. Such biofeedback-modulated video games 61/781,355, filed on Mar. 14, 2013. train valuable mental skills beyond eye-hand coordination. These psychophysiological training technologies enhance (51) Int. Cl. personal improvement, not just the diversion, of the user. A63B 69100 (2006.01) A63F 131211 (2014.01) 18 Claims, 13 Drawing Sheets U.S. Patent Apr. 18, 2017 Sheet 1 of 13 US 9,623,324 B2 24 22 IR XMTR RCVR ~— 18 GYROSCOPE 16 /,,— ACCELEROMETER 14 /1-- PROCESSOR 12 Light Source INPUT DEVICE U.S. Patent Apr. 18, 2017 Sheet 2 of 13 US 9,623,324 B2 m 46 PHYSIOLOGICAL PHYSIOLOGICAL SIGNAL ' SIGNAL MEASUREMENT 54 42 GYROSCOPE AID 52 CONVERTER ACCELEROMETER 50 44 POTENTIOMETER PROCESSOR 48 45 POWER SUPPLY MEMORY 1111111111251w ell 40 PHYSIOLOGICAL PHYSIOLOGICAL SIGNAL SIGNAL MEASUREMENT 61 MEMORY 60 63 66 42 A/D PROCESSOR PROCESSOR SENSOR BAR CONVERTER SENSOR BAR PHYSIOLOGICAL SIGNAL DISPLAY 68 62 FIG. 3 U.S. Patent Apr. 18, 2017 Sheet 3 of 13 US 9,623,324 B2 U.S. Patent Apr. 18, 2017 Sheet 4 of 13 US 9,623,324 B2 U.S. Patent Apr. 18, 2017 Sheet 5 of 13 US 9,623,324 B2 CO (DC, co CD C7 C I C~ LL L LL J J D 0 (D (D (D (D LJf] D 0 D VJ V V V J J J J J J J J J J J J J U.S. Patent Apr. 18, 2017 Sheet 6 of 13 US 9,623,324 B2 FIG. 7A FIG. 7 L7 R 7 L7 R7 76 76 L6 ' R6 L6 R6 7$,, Ra L5 i8 R5 G L4 R 4 L4 R 4 L3 R 3 L3 R 3 R2 L2 R2 0 L1 R1 L1 R1 FIG. 7C FIG. 7D L7 R7 L7 R7 76 7:E Z- L6 R6 L~ R5 78 78 R5 L5 RS G L4 R 4 L4 R 4 L3 R3 L3 R3 L2 R 2; L2 R 2 R1 L1 R1 0 U.S. Patent Apr. 18, 2017 Sheet 7 of 13 US 9,623,324 B2 C r U.S. Patent Apr. 18, 2017 Sheet 8 of 13 US 9,623,324 B2 R PHYSIOLOGICAL PHYSIOLOGICAL SIGNAL MEASUREMENT 1- 61 MEMORY 60 63 42 A~ PROCESSOR PROCESSOR CONVERTER 70 PHYSIOLOGICAL 86 SIGNAL DISPLAY MOTION TABLE SENSOR BAR 62 SENSOR BAR 88 U.S. Patent Apr. 18, 2017 Sheet 9 of 13 US 9,623,324 B2 ARM U.S. Patent Apr. 18, 2017 Sheet 10 of 13 US 9,623,324 B2 EU-4 Physiological Signal Physiological Signal Measurement Memory 42 60 63 A/D Converter Processor Processor r— 102 Physiological Signal Servomechanism C Display Galvanometer m 104 Mirror F(. 11 U.S. Patent Apr. 18, 2017 Sheet 11 of 13 US 9,623,324 B2 40 Physiological 'Signal Physiological Signal Measurement 69 Memory EN 60 /-63 A/D Converter Processor Processor EM Physiological Signal Servomechanism Or Display Galvanometer 106 62 Lens U.S. Patent Apr. 18, 2017 Sheet 12 of 13 US 9,623,324 B2 Physiological Signal Physiological Signal Measurement A/D Converter i i• • melt; LED(s) N FIG. 1 U.S. Patent Apr. 18, 2017 Sheet 13 of 13 US 9,623,324 B2 1400 (Measure 1402 Physiological Signal Convert To Digital 1404 Signal Determine Difference Between Physiological Signal And Desired Physiological Signal Level Adjust Mirrors, Lenses, and/or LEDs To Modify Image Received By MW Camera Based At Least In Part: On the Determined Difference FIG. 14 US 9,623,324 B2 2 PHYSIOLOGICALLY MODULATING enhancing physiological self-regulation skills or for thera- VIDEOGAMES OR SIMULATIONS WHICH peutic amplification of healthful physiological characteris- USE MOTION-SENSING INPUT DEVICES tics. Biofeedback, an effective treatment for various physiological problems, can be used to optimize physiologi- CROSS-REFERENCE TO RELATED 5 cal functioning in many ways. The benefits of biofeedback APPLICATION(S) can, however, only be attained through a number of training sessions, and such gains can only be maintained over time This patent application is a continuation-in-part of, and through regular practice. Adherence to regular training, claims the benefit of and priority to, co-pending U.S. appli- especially at home, has been a problem that has plagued the cation Ser. No. 13/166,166 entitled PHYSIOLOGICALLY io field of biofeedback and limited its utility to date. U.S. Pat. MODULATING VIDEOGAMES OR SIMULATIONS No. 6,450,820, which is incorporated herein by reference WHICH USE MOTION-SENSING INPUT DEVICES filed thereto, addressed this to a degree, by blending biofeedback Jun. 22, 2011, which claims the benefit of and priority to into video games, which increases motivation. Biofeedback- each of U.S. Provisional Patent Application No. 61/361,084, modulated video games employing the technology of U.S. filed Jul. 2, 2010, and U.S. Application Ser. No. 61/499,733, 15 Pat. No. 6,450,820 are games that respond to physiological entitled METHOD AND SYSTEM FOR PHYSIOLOGI- signals as well as mouse, joystick or game controller input; CALLY MODULATING VIDEOGAMES WHICH USE these games embody the concept ofimproving physiological HAND AND BODY MOTION-SENSING INPUT functioning by rewarding specific healthy body signals with DEVICES filed Jun. 22, 2011, the entire contents of each of success at playing a video game. Such a biofeedback- the foregoing applications is incorporated herein by refer- 20 modulated game method blends biofeedback into popular ence in its entirety. This patent application additionally off-the-shelf video games in such a way that the games do claims the benefit of and priority to U.S. Application Ser. not lose their entertainment value. This method uses physi- No. 61/781,355, entitled METHOD AND SYSTEM FOR ological signals (e.g., electroencephalogram frequency band PHYSIOLOGICALLY MODULATING VIDEOGAMES ratio) not simply to drive a biofeedback display directly, or WHICH USE HAND AND BODY MOTION-SENSING 25 periodically modify a task as in other systems, but to INPUT DEVICES filed Mar. 14, 2013, the entire contents of continuously modulate parameters (e.g., game character which is incorporated herein by reference in its entirety. This speed and mobility) of a game task in real time while the application is also related to commonly owned, co-pending game task is being performed by other means (e.g., a game U.S. application Ser. No. 13/166,226 entitled TEAM ELEC- controller). Biofeedback-modulated video games represent a TRONIC GAMEPLAY COMBINING DIFFERENT 3o new generation of computer and video game environments MEANS OF CONTROL, the entire contents of which is that train valuable mental skills beyond eye-hand coordina- incorporated herein by reference in its entirety. tion. These psychophysiological training technologies are poised to exploit the revolution in interactive multimedia STATEMENT REGARDING FEDERALLY home entertainment for the personal improvement, not just SPONSORED RESEARCH AND 35 the diversion, of the user. The technology of U.S. Pat. No. DEVELOPMENT 6,450,820 modulates the manual inputs that a player makes to the buttons or joysticks of a video game hand controller. The invention described herein was made by employees This modulation is based on measurements of physiological of the United States Government and may be manufactured signals of the player.