1111111111111111111inuu111111111111111111u ~

(12) United States Patent (lo) Patent No.: US 8,827,717 B2 Pope et al. (45) Date of Patent: Sep. 9, 2014

(54) PHYSIOLOGICALLY MODULATING 4,508,510 A 4/1985 Clifford VIDEOGAMES OR SIMULATIONS WHICH 5,377,100 A 12/1994 Pope et al. 5,626,140 A 5/1997 Feldman et al. USE MOTION-SENSING INPUT DEVICES 5,697,791 A 12/1997 Nashner et al. 5,702,323 A 12/1997 Poulton (75) Inventors: Alan T. Pope, Poquoson, VA (US); 5,743,744 A 4/1998 Cassily et al. Chad L. Stephens, Poquoson, VA (US); 5,907,819 A 5/1999 Johnson Nina Marie Blanson, Yorktown, VA 5,947,868 A 9/1999 Dugan (US) 5,984,684 A 11/1999 Brostedt et al. 6,067,468 A 5/2000 Korenman et al. 6,093,146 A 7/2000 Filangeri (73) Assignee: The United States of America as 6,126,449 A 10/2000 Burns represented by the Administrator of 6,132,337 A 10/2000 Krupka et al. the National Aeronautics and Space 6,212,427 B1 4/2001 Hoover Administration, Washington, DC (US) (Continued)

(*) Notice: Subject to any disclaimer, the term of this OTHER PUBLICATIONS patent is extended or adjusted under 35 U.S.. 154(b) by 395 days. Alan T. Pope, Edward H. Bogart, and Debbie S. Bartolome, `Biocybernetic System Evaluates Indices of Operator Engagement in (21) Appl. No.: 13/166,166 Automated Task", Biological Psychology 40 (1995), pp. 187-195. (22) Filed: Jun. 22, 2011 Primary Examiner Sam Yao (65) Prior Publication Data Assistant Examiner Thomas Hong US 2012/0004034 Al Jan. 5, 2012 (74) Attorney, Agent, or Firm Jennifer L. Riley; Thomas K. McBride, Jr.; Andrea Z. Warmbier Related U.S. Application Data (60) Provisional application No. 61/499,733, filed on Jun. 22, 2011, provisional application No. 61/361,084, (57) ABSTRACT filed on Jul. 2, 2010. New types of controllers allow players to make inputs to a video game or simulation by moving the entire controller (51) Int. Cl. G09B 5106 (2006.01) itself. This capability is typically accomplished using a wire- less input device having accelerometers, gyroscopes, and an (52) U.S. Cl. infrared LED tracking camera. The present invention exploits USPC ...... 434/247; 434/236; 463/39 these wireless motion-sensing technologies to modulate the (58) Field of Classification Search player's movement inputs to the videogame based upon USPC ...... 434/236, 247-261 physiological signals. Such biofeedback-modulated video See application file for complete search history. games train valuable mental skills beyond eye-hand coordi- nation. These psychophysiological training technologies (56) References Cited enhance personal improvement, not just the diversion, of the U.S. PATENT DOCUMENTS user.

4,065,860 A * 1/1978 Linton et al ...... 434/22 4,337,049 A 6/1982 Connelly 20 Claims, 8 Drawing Sheets

40

PHYSIOLOGICAL pHYSIOLOG SIGNAL SIGNAL MEASUREMENT 61

MEMORY 66 63 66 42 AID PROCESSOR PROCESSOR SENSOR BAR CONVERTER', ~

SENSOR BAR PHYSIOLOGICAL SIGNAL DISPLAY 68

62 US 8,827,717 B2 Page 2

(56) References Cited 6,774,885 B1 8/2004 Even-Zohar 6,778,866 B1 8/2004 Bettwy U.S. PATENT DOCUMENTS 6,786,730 B2 9/2004 Bleckley et al. 2003/0013071 Al 1/2003 Thomas 6,259,889 B1 7/2001 LaDue 2003/0013072 Al 1/2003 Thomas 6,261,189 B1 7/2001 Saville et al. 2003/0087220 Al 5/2003 Bessette 6,277,030 B1 8/2001 Baynton et al. 2005/0014113 Al 1/2005 Fleck et al. 6,425,764 B1 7/2002 Lamson 2007/0060384 Al* 3/2007 Dohta ...... 463/43 6,450,820 B1 * 9/2002 Palsson et al . ...... 434/236 2007/0118043 Al* 5/2007 Oliver et al . ...... 600/519 6,463,385 B1 10/2002 Fry 2008/0081692 Al* 4/2008 Pope et al . ...... 463/31 6,491,647 B1 12/2002 Bridger et al. 2008/0291160 Al* 11/2008 Rabin ...... 345/156 6,527,700 B1 3/2003 Manicoetal. 6,682,351 B1 1/2004 Abraham-Fuchs et al. * cited by examiner U.S. Patent Sep. 9, 2014 Sheet 1 of 8 US 8,827,717 B2

FIG. 1 (PRIOR ART)

20 ~— 2z IR RCVR X i8 GYROSCOPE 76 ACCELEROMETER 14 PROCESSOR 12

INPUT DEVICE U.S. Patent Sep. 9, 2014 Sheet 2 of 8 US 8,827,717 B2

46 46 PHYSIOLOGICAL PHYSIOLOGICAL SIGNAL SIGNAL. MEASUREMENT - 54

42 GYROSCOPE -- 52 AID 4 ;CONVERTER ` ACCELEROMETER – 5d 44 PROCESSOR POTENTIOMETER I – 48

45 POWER SUPPLY MEMORY

INPUT DEVICE FIG. 2

40

PHYSIOLOGICAL SIGNAL SIGNAL MEASUREMENT 61

MEMORY !Q ~ S6 42 — AID CONVERTER PROCESSOR ► PROCESSOR SENSOR BAR

r SENSOR BAR PHYSIOLOGICAL SIGNAL DISPLAY 68

62 FIG. 3 U.S. Patent Sep. 9, 2014 Sheet 3 of 8 US 8,827,717 B2 U.S. Patent Sep. 9, 2014 Sheet 4 of 8 US 8,827,717 B2 U.S. Patent Sep. 9, 2014 Sheet 5 of 8 US 8,827,717 B2

00': 4) (J) 'WOO Ox-

2 2

d m U ) n CD co co cD ~ 1 o c0

u-

J J

t

J J J

i

J J J U.S. Patent Sep. 9, 2014 Sheet 6 of 8 US 8,827,717 B2

FIG. 7A FIG. 7B

L7 R7 L7 R7 76 76 L.5 s L6 R6 78 L5 78 R5 L5 R5

L4 R4 L4 Ft4

L3 R3 L'3 R3

L1 R1 L1 f21

FIG. 7C FIG. 7D

L7 R7 R7 0 76 76 L6 R6 a R8 78 78 L5 R5 05 R5

L4 R4 L4 R4

L3 _ F?3 L9 R3

L1 R1 L1 R1 U.S. Patent Sep. 9, 2014 Sheet 7 of 8 US 8,827,717 B2 U.S. Patent Sep. 9, 2014 Sheet 8 of 8 US 8,827,717 B2

40

PHYSIOLOGICAL PHYSIOLOGICAL SIGNAL SIGNAL MEASUREMENT j - 61

MEMORY k 60 / 63 42 A/D - - PROCESSOR PROCESSOR CONVERTER; 70

PHYSIOLOGICAL 86 SIGNAL DISPLAY MOTION TABLE i SENSOR BAR

62 i SENSOR BAR '1 pp FIG. 9 88 US 8,827,717 B2 2 PHYSIOLOGICALLY MODULATING value. This method uses physiological signals (e.g., electro- VIDEOGAMES OR SIMULATIONS WHICH encephalogram frequency band ratio) not simply to drive a USE MOTION-SENSING INPUT DEVICES biofeedback display directly, or periodically modify a task as in other systems, but to continuously modulate parameters This application is related to commonly owned U.S. appli- 5 (e.g., game character speed and mobility) of a game task in cation Ser. No. 13/166,226 entitled TEAM ELECTRONIC real time while the game task is being performed by other GAMEPLAY COMBINING DIFFERENT MEANS OF means (e.g., a game controller). Biofeedback-modulated CONTROL, and to commonly owned U.S. Application Ser. video games represent a new generation of and No. 61/499,733, entitled METHOD AND SYSTEM FOR video game environments that train valuable mental skills PHYSIOLOGICALLY MODULATING VIDEOGAMES 10 beyond eye-hand coordination. These psychophysiological WHICH USE HAND AND BODY MOTION-SENSING training technologies are poised to exploit the revolution in DEVICES, both of which are filed concurrently, the entire interactive multimedia home entertainment for the personal contents of both of which are incorporated herein by refer- improvement, not just the diversion, of the user. The technol- ence in its entirety. This patent application claims the benefit ogy of U.S. Pat. No. 6,450,820 modulates the manual inputs of U.S. Provisional Patent Application No. 61/361,084, filed 15 that a player makes to the buttons or joysticks of a video game Jul. 2, 2010. hand controller. This modulation is based on measurements of physiological signals of the player. STATEMENT REGARDING FEDERALLY However, a new type of video game (illustrated in FIG. 1) SPONSORED RESEARCH AND DEVELOPMENT has a game controller that allows a player to make inputs to a 20 video game by moving the entire controller itself. This capa- This invention was made in part with Government support. bility is accomplished in operator-controlled game input The Government may have certain rights in this invention. devices, including the wireless remote using accelerometers, and an infrared LED tracking camera, and/or FIELD OF THE INVENTION the remote's accessories (e.g., the Nintendo Nunchuk and 25 MotionPlus devices using accelerometers and gyroscopes) The present invention relates generally to the use of bio- (Nintendo, Wii and MotionPlus are registered trademarks of feedback to modify a subject's behavior, mental state, and/or Nintendo of America Inc.) The video game system of FIG. 1 physiological functioning. More specifically, the invention comprises a game console 10, a controller or input device 12, relates to apparatus and methods for modulating an operator's a display 24 (such as a television), and left and right sensor control input to an electronic game or simulator, via a motion 30 bars 26, 28 (note, while the left and right sensor bars are and/or po sition-sensing input device, in response to measured illustrated as separate devices, in practice the left and right physiological activity, such as autonomically-mediated and/ sensor bars are often incorporated into a single device having or EEG physiological activity, wherein the player thereby left and right portions). The game input device 12 comprises learns to control the physiological and/or EEG activity. one or more gyroscopes 18 and one or more accelerometers 35 16 for sensing movement of the input device, an infrared BACKGROUND receiver or camera 20 for detecting the infrared light emitting diodes (LEDs) of the sensor bars, a transmitter 22 (e.g., Blue- Physiologically modulated video games add to the enter- tooth) for communicating with the console 10, user input tainment value of video games by adding the challenge of devices (e.g., buttons, etc.) (not illustrated), and a processor requiring a player to master physiological self-regulation 40 14 for processing the inputs to the game input device and skill as well as hand-eye coordination. Thus, controlling the controlling outputs from the game input device. The game physiological state, or learning to self-induce physiologic system uses the known positional relationship between the changes, is an additional skill requirement or challenge added left and right sensorbars to determine the game input device's to games. Although physiologically modulated videogames approximate distance from the sensor bars and the input have entertainment value by making games more exciting, 45 device's position and/or motion relative to the sensor bars. they also have advantages for encouraging health-enhancing However, the art described in U.S. Pat. No. 6,450,820, physiological self-regulation skills or for therapeutic ampli- requires further advancement to work with these wireless fication of healthful physiological characteristics. Biofeed- controllers. back, an effective treatment for various physiological prob- lems, can be used to optimize physiological functioning in 50 SUMMARY OF THE INVENTION many ways. The benefits of biofeedback can, however, only be attained through a number of training sessions, and such A new type of controller allows a player to make inputs to gains can only be maintained over time through regular prac- a video game or simulation by moving the entire controller tice. Adherence to regular training, especially at home, has itself. This capability is typically accomplished using - been a problem that has plagued the field of biofeedback and 55 tor controlled input devices having accelerometers, gyro- limited its utility to date. U.S. Pat. No. 6,450,820, which is scopes, and infrared LED tracking cameras. The present incorporated herein by reference thereto, addressed this to a invention exploits these wireless motion-sensing technolo- degree, by blending biofeedback into video games, which gies to modulate the player's movement inputs to the increases motivation. Biofeedback-modulated video games videogame based upon physiological signals. employing the technology of U.S. Pat. No. 6,450,820 are 60 In another embodiment of the invention, a method is pro- games that respond to physiological signals as well as mouse, vided for modifying the effect of an operator controlled input joystick or game controller input; these games embody the device on an interactive device to encourage the self-regula- concept of improving physiological functioning by reward- tion of at least one physiological activity by the operator. The ing specific healthy body signals with success at playing a operator controlled input device has motion and/or position- video game. Such a biofeedback-modulated game method 65 sensing capability. The interactive device comprises a display blends biofeedback into popular off-the-shelf video games in area which depicts images, and apparatus for receiving at such a way that the games do not lose their entertainment least one input from the operator controlled input device to US 8,827,717 B2 3 4 thus permit the operator to control and interact with at least respiratory rate. The brainwave electrical activity may com- some of the depicted images. The method for modifying prise at least one of: event-related potentials; and at least one comprises the steps of measuring at least one physiological brainwave frequency band; the at least one brainwave fre- activity of the operator to obtain at least one physiological quency band comprising at least one of: (a) theta, (b) alpha, signal having a value indicative of the level of the at least one 5 (c) sensorimotor response (SMR) (d) beta, and (e) gamma. physiological activity; and modifying the motion and/or po si- The method may further comprise displaying the at least tion sensing capability of the operator controlled input device one physiological signal having a value indicative of the level in response to (a) changes in the measured at least one physi- of the at least one physiological activity. Displaying the at ological signal and/or (b) differences between the measured least one physiological signal may comprise displaying and at least one physiological signal and a corresponding prede- 10 monitoring the physiological signal on a signal display sys- termined desired physiological signal level, thereby modify- tem substantially separate from the interactive device. How- ing the ability of the operator to control and interact with at ever, such a separate display can be connected wirelessly, in least some of the depicted images. some instances, to the interactive device. Modifying the motion and/or position sensing capability of In addition to the method for modifying the effect of an the operator controlled input device may comprise modulat- 15 operator controlled input device on an interactive device to ing a voltage input to one or more motion and/or position- encourage the self-regulation of at least one physiological sensing elements of the input device. The one or more motion activity by the operator, as described above, yet other aspects and/or position-sensing elements may comprise one or more and embodiments of the present invention are directed to an accelerometers and/or one or more gyroscopes. apparatus for modifying the effect of an operator controlled Modifying the motion and/or position sensing capability of 20 input device on an interactive device to encourage the self- the operator controlled input device may comprise modifying regulation of at least one physiological activity by the opera- one or more position reference signal elements that are used tor. by the input device to determine a motion or position of the operator controlled input device relative to the one or more BRIEF DESCRIPTION OF THE SEVERAL position reference signal elements. The one or more position 25 VIEWS OF THE DRAWING(S) reference signal elements may comprise two light emitting diodes (LEDs) or two sets of LEDs having a fixed distance Having thus described the invention in general terms, ref- therebetween. The operator controlled input device may com- erence will now be made to the accompanying drawings, prise a light detector for detecting light emitted from the which are not necessarily drawn to scale, and wherein:

LEDs. Modifying one or more position reference signal ele- 30 FIG. 1 illustrates a block diagram of a video game system ments may comprise mechanically translating the two LEDs that can benefit from embodiments of the present invention; or two sets of LEDs from a first position to a second position, FIG. 2. illustrates a block diagram of portions of a system the difference between the first and second positions being for modifying the effect of an operator controlled input device based on either (a) the difference between the measured at on an interactive device to encourage the self-regulation of at least one physiological signal and the corresponding prede- 35 least one physiological activity by the operator, in accordance termined desired physiological signal level, or (b) the mea- with embodiments of the present invention; sured at least one physiological signal. FIG. 3 illustrates a block diagram of portions of a system The one or more position reference signal elements may for modifying the effect of an operator controlled input device comprise a plurality of LEDs, two of which emit light at any on an interactive device to encourage the self-regulation of at particular time. The operator controlled input device may 40 least one physiological activity by the operator, in accordance comprise a light detector for detecting light emitted from the with alternative embodiments of the present invention; LEDs. Modifying one or more position reference signal ele- FIGS. 4 and 5 illustrate potential positioning options for ments may comprise changing which two of the LEDs are sensor bars of a system for modifying the effect of an operator emitting light at any particular time. Changing which two of controlled input device on an interactive device to encourage the LEDs are emitting light at any particular time may com- 45 the self-regulation of at least one physiological activity by the prise turning pairs of LEDs on and off in a predetermined operator, in accordance with alternative embodiments of the pattern based on either (a) the difference between the mea- present invention; sured at least one physiological signal and the corresponding FIGS. 6A-D illustrate one possible lighting pattern for predetermined desired physiological signal level, or (b) the horizontal sensor bars of a system for modifying the effect of measured at least one physiological signal. 50 an operator controlled input device on an interactive device to The plurality of LEDs may comprise two groups of linearly encourage the self-regulation of at least one physiological arranged LEDs, each of the two linearly arranged groups activity by the operator, in accordance with embodiments of being positioned substantially horizontally. Alternatively, the the present invention; plurality of LEDs may comprise two groups of linearly FIGS. 7A-D illustrate one possible lighting pattern for arranged LEDs, each of the two linearly arranged groups 55 vertical sensor bars of a system for modifying the effect of an being positioned substantially vertically. As another alterna- operator controlled input device on an interactive device to tive, the plurality of LEDs may comprise two groups of LEDs, encourage the self-regulation of at least one physiological each of the groups being arranged in two linearly arranged activity by the operator, in accordance with embodiments of sub-groups, each sub-group of a group crossing the other the present invention; sub-group at a center of each sub-group and at a perpendicular 60 FIG. 8 illustrates an alternative sensor bar of a system for angle. modifying the effect of an operator controlled input device on Measuring at least one physiological activity may com- an interactive device to encourage the self-regulation of at prise measuring at least one of: autonomically-mediated least one physiological activity by the operator, in accordance physiological activity, and brainwave electrical activity. The with embodiments of the present invention; and autonomic physiological activity may comprise at least one 65 FIG. 9 illustrates a block diagram of portions of a system of: skin temperature; skin conductance; electrical activity of for modifying the effect of an operator controlled input device muscles; blood flow; heart rate; heart rate variability; and on an interactive device to encourage the self-regulation of at US 8,827,717 B2 5 6 least one physiological activity by the operator, in accordance method for determining an individual's EEG index of atten- with alternative embodiments of the present invention. tion, which index can be used to assess his or her mental engagement in a task. Industry-standard methods and devices DETAILED DESCRIPTION for measuring these physiological signals may be used as the 5 physiological signal measurement device 40, including but The present invention now will be described more fully not limited to an electroencephalograph, a thermometer, a hereinafter with reference to the accompanying drawings, in galvanic skin response device, an electromyogram, and/or an which preferred embodiments of the invention are shown. cardiotachometer. The physiological signal measurement This invention may, however, be embodied in many different device 40 in conjunction with the analog-to-digital converter forms and should not be construed as limited to the embodi- io 42 and processor 44 may (1) sample amplified and processed ments set forth herein; rather, these embodiments are pro- physiological signals (e.g., electromyogram measuring mus- vided so that this disclosure will be thorough and complete, cular tension), (2) test the signal levels with programmed and will fully convey the scope of the invention to those conditional instructions stored in memory 45 (and possibly skilled in the art. Like numbers refer to like elements through- compare the signal levels with predetermined desired signal out. 15 levels stored in memory 45), and, in accordance with the A new type of video game controller allows a player to physiological signal condition determinations, (3) dynami- make inputs to a video game or simulation by moving the cally adjust digitally controlled potentiometers 50 that are entire controller itself. This capability is typically accom- configured as voltage dividers to modulate the voltage from plished using operator controlled input devices having accel- power supply 48 that powers the accelerometers 52 and/or erometers, gyroscopes, and infrared LED tracking cameras. 20 gyroscopes 54. FIG. 2 illustrates a single potentiometer that The present invention exploits these wireless motion-sensing modulates the voltage to both a single accelerometer and a technologies to modulate the player's movement inputs to the single gyroscope. In alternative embodiments of the inven- videogame based upon physiological signals. tion, separate potentiometers may be used with each acceler- Embodiments of the invention modify the effect of an ometer and each gyroscope, and multiple accelerometers and/ operator controlled input device having motion and/or posi- 25 or multiple gyroscopes may be used in the game input device tion-sensing capability. Embodiments of the invention typi- 46. In a yet further alternative embodiment, one or more cally measure at least one physiological activity of the opera- accelerometers may be modulated with a single potentiom- tor to obtain at least one physiological signal having a value eter and one or more gyroscopes may be modulated with a indicative of the level of the at least one physiological activ- separate, single potentiometer. One, a plurality of, or all of the ity; and modify the motion and/or position sensing capability 3o accelerometers and/or gyroscopes may be so modulated. of the operator controlled input device in response to (a) Referring now to FIG. 3, a block diagram of portions of a changes in the measured at least one physiological signal system for modifying the effect of an operator controlled and/or (b) differences between the measured at least one input device on an interactive device is illustrated in accor- physiological signal and a corresponding predetermined dance with alternate embodiments of the present invention. In desired physiological signal level, thereby modifying the 35 the embodiment of FIG. 3, modification is made to one or ability of the operator to control and interact with at least more position reference signal elements that are used by the some of the depicted images. input device to determine a motion or position of the operator Referring now to FIG. 2, a block diagram of portions of a controlled input device relative to the one or more position system for modifying the effect of an operator controlled reference signal elements. These position reference signal input device on an interactive device is illustrated in accor- 40 elements are commonly referred to as "sensor bars." The one dance with embodiments of the present invention. In the ormore sensor bars each comprise a plurality of LEDs, two of embodiment of FIG. 2, the voltage input to one or more which emit light at any particular time (typically one from motion and/or position-sensing elements (e.g., accelerom- each sensor bar). (Note, the sensor bars are described herein eters and/or gyroscopes) of the input device is modulated as separate physical devices, each having a plurality of LEDs. (increased or decreased). This modulation of the voltage that 45 However, embodiments of the invention may comprise sensor powers the accelerometers and/or gyroscopes in operator bars that are combined in a single physical device, with two controlled input devices changes (increasing or reducing) the sets of LEDs separated by a distance.) The operator con- effect of the player's movement of the controller on the trolled input device comprises a light detector for detecting player-controlled object on the screen. The degree of the light emitted from the LEDs. increase or reduction may be programmed to be proportional 50 In this embodiment, the standard sensor bar of the video to the difference between the player's current momentary game system is replaced with an array of infrared LEDs that physiological signal value and a pre-selected target value. In are individually controllable on and off. This embodiment of this way, the player is rewarded for diminishing this differ- the invention turns LEDs in the array on and off in dynamic ence by an improved ability to control the player-controlled patterns to produce a disturbance in the player-directed con- object (e.g., a cursor) on the screen and/or increased power in 55 trol of the infrared-sensing videogame controller, resulting in the action of the player-controlled object on the screen (e.g., a disruption in the stability of the player-controlled object a golf club). (e.g., a cursor) on the screen. The degree of the disruption is The system of FIG. 2 comprises a physiological signal programmed to be proportional to the difference between the measurement device 40 that receives physiological signal player's current momentary physiological signal value and a information from an operator of the system. One or more of a 60 pre-selected target value. In this way, the player is rewarded wide variety of different measured physiological signals can for diminishing this difference by an improved ability to be used in accordance with the present invention, for accurately position the player-controlled object (e.g., a cur- example, electroencephalogram (EEGs), skin temperature, sor) on the screen. skin conductance, heart rate, and/or event-related potentials In the embodiment of FIG. 3, modifying one or more (ERPs). U.S. Pat. No. 5,377,100, issued on Dec. 27, 1994 to 65 position reference signal elements comprises changing which Pope et al., and which is incorporated herein by reference in two of the LEDs (typically one from each sensor bar) are its entirety, at column 3, line 8 to column 5, line 60 details a emitting light at any particular time. This typically involves US 8,827,717 B2 7 8 turning pairs of LEDs on and off in a predetermined pattern communicating those values to second processor 63 which based on either (a) the difference between the measured at reads a serial value at the initiation of each heart beat, and least one physiological signal and the corresponding prede- controlling the timing and sweep amplitude of a marquee-like termined desired physiological signal level, or (b) the mea- illumination of the LEDs. sured at least one physiological signal. 5 In the above two examples, the individual LEDs in each The plurality of LEDs may comprise two groups of linearly sensor bar are turned on and off in sequence (left to right or arranged LEDs, each of the two linearly arranged groups right to left) to create a marquee-like sweep. This is illustrated beingpositioned substantially horizontally. This arrangement in FIGS. 6A-D. FIGS. 6A-D illustrate two horizontal sensor is illustrated in FIG. 4, which shows the two sensor bars 66, 68 bars 66, 68, each comprising seven linearly arranged LEDs spaced apart and on alternate sides of the television 24. Also io (labeled Ll-L7 for the left sensor bar 66 and R1-R7 for the illustrated is the game console 10, and all of the components right sensor bar 68). First, LEDs Ll and Rl are illuminated are supported on a table 80. As mentioned above, the two (FIG. 6A). Then, Ll and Rl are turned off and L2 and R2 are sensor bars may be combined into a single physical device. illuminated (FIG. 613). Then, L2 and R2 are turned off and L3 Alternatively, theplurality of LEDs may comprisetwo groups and R3 are illuminated (FIG. 6C). Finally, L3 and R3 are of linearly arranged LEDs, each of the two linearly arranged 15 turned off and L4 and R4 are illuminated (FIG. 6D). This groups being positioned substantially vertically. This creates a left-to-right marquee sweep, which causes the input arrangement is illustrated in FIG. 5, which shows the two device to interpret that it is being moved erratically when in sensor bars 76, 78 spaced apart and on alternate sides of the fact it is not. The speed and/or extent of the marquee-like television 24. Also illustrated is the game console 10, and all sweep may be varied, for example in proportion to the differ- of the components are supported on a table 80. As mentioned 20 ence between the player's current momentary physiological above, the two sensor bars may be combined into a single signal value and a pre-selected target value. The specific physical device. As another alternative, the plurality of LEDs pattern used may vary considerably in alternative embodi- may comprise two groups of LEDs, each of the groups being ments of the invention. arranged in two linearly arranged sub-groups, each sub-group In another example embodiment, the infrared LEDs that of a group crossing the other sub-group at a center of each 25 the infrared camera uses to track the movement of the input sub-group and at a perpendicular angle. This embodiment is device are oscillated back and forth horizontally, causing an illustrated in FIG. 8 (discussed in more detail below). oscillation to be superimposed on the player's movements of The system of FIG. 3 comprises a physiological signal the input device and reflected in the movements of the player- measurement device 40 that receives physiological signal controlled object (e.g., a cursor) on the game screen. For information from an operator of the system (as described 30 example, the LEDs may be illuminated in a sequence such as: above). The physiological signal measurement device 40 in L4/R4. L3/R3, L2/R2, L3/R3, L4/R4, L5/R5, L6/R6, L5/R5, conjunction with the analog-to-digital converter 42 and pro- and L4/R4. The amplitude and/or speed of the oscillations cessor 60 may (1) sample amplified and processed physi- may be programmed to be proportional to the difference ological signals (e.g., cardiotachometer measuring heart between the player's current momentary physiological signal rate), (2) test the signal levels with programmed conditional 35 value and a pre-selected target value. This results in an instructions stored in memory 61 (and possibly compare the unsteadiness in the player's ability to control the player- signal levels with predetermined desired signal levels stored controlled object (e.g., a cursor), which diminishes as the in memory 61), and, in accordance with the physiological difference between the player's current momentary physi- signal condition determinations, (3) turning on infrared LEDs ological signal value and a pre-selected target value dimin- in dynamic patterns to produce the described effects on the 40 ishes. player-directed control of an infrared-sensing videogame In another example embodiment, the infrared LEDs that controller. the infrared camera uses to track the movement of the input One example implementation involves positioning one device are swept up and down vertically, causing an oscilla- horizontal row of LEDs, e.g., 7 LEDs, in front and to the right tion to be superimposed on the player's movements of the of the player who is manipulating the input device, and 45 input device and reflected in the movements of the player- another row in front and to the left of the player (as illustrated controlled object (e.g., a cursor) on the game screen. The in FIG. 4). The corresponding LED in each row is simulta- amplitude and/or speed of the sweeps may be programmed to neously illuminated and turned on and off in synch to create be proportional to the difference between the player's current a running marquee effect. This effect results in horizontally momentary physiological signal value and a pre-selected tar- sweeping the infrared camera's two reference points used to 50 get value. This results in an unsteadiness in the player's position the cursor on the screen, producing a disruption in ability to control the player-controlled object (e.g., a cursor), the stability of the player-controlled object (e.g., a cursor) on which diminishes as the difference between the player's cur- the screen. The length of the marquee run, and, consequently, rent momentary physiological signal value and a preselected the degree of the disruption, may be programmed to be pro- target value diminishes. One particular implementation portional to the difference between the player's current 55 involves positioning one vertical column of LEDs, e.g., 7 momentary physiological signal value and a pre-selected tar- LEDs, in front and to the right of the player who is manipu- get value. In this way, the player is rewarded for diminishing lating the input device, and another row in front and to the left this difference by an improved ability to accurately position of the player (as illustrated in FIG. 5). The corresponding the player-controlled object (e.g., a cursor) on the screen. LED in each column is simultaneously illuminated and More specifically, one example implementation involves 60 turned on and off in synch to create a running marquee effect. processor 60 continuously sampling an integrated EMG sig- This effect results in vertically sweeping the infrared cam- nal, serially communicating those values to second processor era's two reference points used to position the player-con- 63 which reads a serial value at the initiation of each sweep of trolled object (e.g., a cursor) on the screen, producing a dis- LED rows and/or columns, and controlling the timing and ruption in the stability of the player-controlled object (e.g., a sweep amplitude of a marquee-like illumination of the LEDs. 65 cursor) on the screen. Another specific example implementation involves processor This is illustrated in FIGS. 7A-D. FIGS. 7A-D illustrate 60 sampling a heart rate signal (cardiotachometer), serially two vertical sensor bars 66, 68, each comprising seven lin- US 8,827,717 B2 9 10 early arranged LEDs (labeled L1-L7 for the left sensor bar 76 pairs of LEDs in sequence: LH7/RH7, then LV1/RV1, then and R1-R7 for the right sensor bar 78). First, LEDs Ll and Rl LH1/RH1, and then LV7/RV7. This illumination sequence of are illuminated (FIG. 7A). Then, Ll and Rl are turned off and the LEDs creates a corresponding movement pattern of the L2 and R2 are illuminated (FIG. 713). Then, L2 and R2 are player-controlled object (e.g., a cursor) on the screen, pro- turned off and L3 and R3 are illuminated (FIG. 7C). Finally, 5 ducing a disruption in the stability of the player-controlled L3 and R3 are turned off and L4 and R4 are illuminated (FIG. object (e.g., a cursor) on the screen. The radial position (i.e., 7D). This creates a bottom-to-top marquee sweep, which how far away from the center LED) of the illuminated LEDs causes the input device to interpret that it is being moved and, consequently, the radius of the circumscribed rotational erratically when in fact it is not. The speed and/or extent of the pattern and, consequently, the degree of the disruption, may marquee-like sweep may be varied, for example in proportion io be programmed to vary in proportion to the difference to the difference between the player's current momentary between the player's current momentary physiological signal physiological signal value and a pre-selected target value. In value and a pre-selected target value. In one embodiment, the this way, the player is rewarded for diminishing this differ- radial position of the illuminated LED, determined from the ence by improved ability to accurately position the player- previously defined physiological signal difference, is the controlled obj ect (e.g., cursor) on the screen. The sweeps may 15 same, i.e., the same distance from the center on the right be continuous up and down oscillations whose amplitude is horizontal, the bottom vertical, the left horizontal and the top proportional to the player's difference signal as above. Or vertical radials, for each rotation about the center, creating a each sweep may be initiated on each heart beat and sweep up, symmetrical diamond rotational pattern. In an alternate or both up and down, to an amplitude proportional to the embodiment, the radial position is determined from the physi- difference between the player's current momentary heart rate 20 ological signal difference for each individual radial in clock- value and a pre-selected target value. If the goal of training is wise rotation, and may vary from radial to radial, creating a to achieve a preselected target heart rate, represented by illu- swirling rotational pattern. In another alternate embodiment, mination of the LEDs in the middle of each column, the LEDs the radial rotation is limited to three of the four radials, in each column may be programmed to sweep up to the resulting in a triangular rotational pattern whose size and, appropriate LED from the LED in the middle of each column, 25 consequently, the degree of the disruption, may be pro- then down to the appropriate LED below the middle of each grammed to vary in proportion to the difference between the column then return to the middle LED, the amplitude of each player's current momentary physiological signal value and a sweep being proportional to the difference between the play- pre-selected target value. The apex of this rotating triangular er's current momentary heart rate value and a pre-selected pattern, the farthest vertex from the horizontal side, may be target value. For example, the LEDs may be illuminated in a 30 above or below the horizontal side, indicating that the sequence: L4/R4, L3/R3, L2/R2, L3/R3. L4/R4, L5/R5, momentary physiological signal value exceeds or falls below, L6/R6, L5/R5, and L4/R4. respectively, a pre-selected target value. The target heart rate may be determined in any of a number In an embodiment using the sensor bar 80 of FIG. 8, these of ways two examples are: (1) the target value may be input functions may be implemented with three programmable prior to game play or (2) the target value may be computed as 35 microcontrollers or processors. One processor may perform a baseline average over N heart beats before the beginning of three functions: (1) controlling analog to digital converter or during a selected period of game play. The target value components to sample amplified and processed physiological could be programmed to change as the player progresses signals (e.g., from a cardiotachometer measuring heart rate); through the game. If the goal of training is simply to lower (2) testing the signal levels with programmed conditional heart rate, the pre-selected target value would be, in essence, 40 instructions, and, in accordance with the physiological signal zero, and the LEDs in the column are programmed to sweep condition determinations; (3) sending a radial position value up from the bottom LED to an amplitude proportional to the (0, 1, 2 or 3) to both of the two other processors. One of the player's momentary heart rate, then back down to the bottom receiving processors may control the two co-linear horizontal LED. If the goal of training is to increase heart rate variability, rows of seven LEDs each in the "H" configuration described the difference between the current momentary heart rate and 45 above; the other receiving processor may control the two the immediately preceding heart rate, or an average heart rate vertical columns of seven LEDs each. The two receiving over N preceding heart beats, is calculated in real time and the processors engage in back and forth handshaking to turn on amplitude of the vertical LED sweeps are made inversely corresponding LEDs (i.e., a pair of LEDs) on the radials of proportional to that difference. The sweeps may correspond each of the two crosses described above in clockwise to the direction of the heart rate change, that is, heart rate 50 sequence (right horizontal, bottom vertical, left horizontal increases produce sweeps up and decreases produce sweeps and top vertical radials for each rotation) so as to create a down, or the sweeps may be opposite in direction to the heart dynamic rotational pattern. rate change, or the sweeps may be in either direction propor- In another alternative embodiment, the sensor bars (which, tional to the absolute value of the heart rate change. again, may be a single device or separate devices) may com- In an additional embodiment, illustrated in FIG. 8, the LED 55 prise two light emitting diodes (LEDs) or two sets of LEDs rows and columns are superimposed in a configuration 90 that having a fixed distance therebetween. The LEDs may be resembles an "H", but with a crossbar comprising two col- mechanically translated from a first position to a second posi- linear 7-LED horizontal rows centered on the verticals of the tion. This may be accomplished by moving the sensor bars "H". This arrangement creates two symmetrical crosses of themselves, or by mechanically moving the LEDs within LEDs with horizontal and vertical radials of 3 LEDs each, 60 each sensor bar. The first alternative is illustrated in FIG. 9, in emanating from center LEDs. Corresponding LEDs on the which the sensor bars 86, 88 are mounted on a motion table 70 radials of each cross (i.e., a pair of LEDs) may be turned on in that may be configured to move the sensor bars, in unison or clockwise sequence (right horizontal, bottom vertical, left separately, horizontally, vertically, or diagonally. The differ- horizontal and top vertical radials for each rotation) so as to ence between the first and second positions may be based on create a dynamic rotational pattern that encircles the center 65 either (a) the difference between the measured at least one (LC and RC) of each cross. Such a clockwise sequence of physiological signal and the corresponding predetermined illumination might, for example, illuminate the following desired physiological signal level, or (b) the measured at least US 8,827,717 B2 11 12 one physiological signal. The measurement and analysis of example, to produce a momentary displacement (bump) in or the physiological signal is similar to that of the embodiment slowing of the movement of the player's character on screen. of FIG. 3. More specifically, one example implementation 6. Physiological signals can be used to modulate the effect involves processor 60 continuously sampling a physiological of the tactile (haptic) and auditory signals delivered to the signal, serially communicating those values to second pro- 5 player through the game input device and/or audio output of cessor 63 which controls the mechanical movement of the the game console. sensor bars via the motion table. In at least one advantageous and exemplary embodiment, The function of the invention may be accomplished by including but not limited to the LED embodiments described using the physiological signals of the player, the signals of above, the current invention interacts wirelessly with a Wii another participant who is not handling a videogame control- io remote through infrared transmission, and has the unique ler, or the physiological signals of one player to modulate the advantage of not requiring direct connection or wiring into controller of an opposing player. the Wii video game or access to the Wii or Embodiments of the invention may employ the integrated firmware. EMG signal, and the heart rate (cardiotachometer) signal. In other exemplary embodiments, the challenge of various Additional embodiments may employ brainwave signals, 15 games can be enhanced by attenuation (joystick, button and/ specifically an "engagement index" derived from brainwave or motion-control dampening or perturbation) and/or by dis- signals, defined in Pope, Bogart and Bartolome (Pope, A. T., ruption (cursor movement modulation). Such outside con- Bogart, E. H., and Bartolome, D. S. (1995). Biocybernetic trols, focus on overt acts for systems like the Wii motion- System Validates Index of Operator Engagement in Auto- control or the MS Kinect motion detector, while working with mated Task. Biological Psychology, 40, 187-195), the con- 20 inside controls and covert acts through body behaviors and tents of which are incorporated herein in its entirety. Other such like NASA's MindShift (demonstrated in 2010 using a physiological signals that may be used include, but are not sensor attached to the player's earlobe, checking the pulse limited to, skin conductance signals, skin temperature signals and wired into the control or sensors attached to the forehead, and respiration signals. seeking the facial muscle strain that is a sign of stress). A noise-reducing enhancement may be desired for obtain- 25 Accordingly, in certain examples, psycho-physiological ing electrical heart signal. The heart rate signal may be modulation of gameplay can be implemented in conjunction derived from a photoplethysmograph sensor or electrocardio- with the Wii system games, including but not limited to, first gram electrodes. One convenient method for obtaining the person shooter games where cross hairs position and point of electrocardiogram heart signal is the use of chest band elec- view modifications are used. Alternative games can include trodes in combination with a wireless transmitter (e.g., Polar 30 without limitation, medical simulations using cursor position Model T31 from Polar Electro USA). Such a technology modification to teach stress management during medical pro- minimizes movement artifact (electrical noise) and enables cedures; sports simulations such as for instance golf swing the invention to be used conveniently with the active and strength, modulation and ball control; and racing games exercise games popular on the video game systems. where vehicle steering attenuation is contemplated. In addition to the methods described above are the follow - 35 All references, including publications, patent applications, ing methods of superimposing movements on the player's and patents, cited herein are hereby incorporated by reference control of in-game objects (e.g., a cursor, character) on the to the same extent as if each reference were individually and screen, producing a disruption in the stability of the player- specifically indicated to be incorporated by reference and controlled object (e.g., a cursor, character) on the screen were set forth in its entirety herein. and/or modulation of game feedback signals that vary in 40 The use of the terms "a" and "an" and "the" and similar proportion to either (a) the difference between the measured referents in the context of describing the invention (especially at least one physiological signal and the corresponding pre- in the context of the following claims) are to be construed to determined desired physiological signal level, or (b) the mea- cover both the singular and the plural, unless otherwise indi- sured at least one physiological signal: cated herein or clearly contradicted by context. The terms 1. Physiological signals can be used to modulate a momen- 45 "comprising," "having," "including," and "containing" are to tary illumination of off-center LEDs in the sensor bars result- be construed as open-ended terms (i.e., meaning "including, ing in a brief change (bump) in the reference position for the but not limited to,") unless otherwise noted. Recitation of position/motion of the input device. ranges of values herein are merely intended to serve as a 2. Physiological signals can be used to modulate the mag- shorthand method of referring individually to each separate nitude of a superimposed momentary displacement (bump) to 50 value falling within the range, unless otherwise indicated control inputs in the input device (buttons, joysticks, acceler- herein, and each separate value is incorporated into the speci- ometers and/or gyroscopes). These displacements may be fication as if it were individually recited herein. All methods superimposed upon ongoing inputs a player provides to the described herein can be performed in any suitable order game (movement in all directions, jump, accelerate). The unless otherwise indicated herein or otherwise clearly con- displacements may also function as additional inputs to the 55 tradicted by context. The use of any and all examples, or game without input from the player (e.g., firing a weapon). exemplary language (e.g., "such as") provided herein, is 3. Physiological signals can be used to simultaneously intended merely to better illuminate the invention and does modulate the magnitude of a superimposed momentary dis- not pose a limitation on the scope of the invention unless placement (bump) of more than one control input. otherwise claimed. No language in the specification should be 4. Multiple physiological signals can be combined to 60 construed as indicating any non-claimed element as essential modulate the magnitude of a superimposed momentary dis- to the practice of the invention. placement (bump) of a single control input. Multiple physi- Preferred embodiments of this invention are described ological signals can be used to simultaneously modulate the herein, including the best mode known to the inventors for magnitude of a superimposed momentary displacement carrying out the invention. Variations of those preferred (bump) of more than one control input. 65 embodiments may become apparent to those of ordinary skill 5. Physiological signals can be combined with the tactile in the art upon reading the foregoing description. The inven- (haptic) and/or auditory signals of the input device, for tors expect skilled artisans to employ such variations as US 8,827,717 B2 13 14 appropriate, and the inventors intend for the invention to be perature, skin conductance, electrical activity of muscles, practiced otherwise than as specifically described herein. blood flow, heart rate, heart rate variability, and respiratory Accordingly, this invention includes all modifications and rate. equivalents of the subject matter recited in the claims 7. The method of claim 1, wherein the at least one physi- appended hereto as permitted by applicable law. Moreover, 5 ological signal comprises at least one or more of brainwave any combination of the above-described elements in all pos- electrical event-related potentials, and at least one brainwave sible variations thereof is encompassed by the invention frequency band; wherein the at least one brainwave frequency unless otherwise indicated herein or otherwise clearly con- band comprises at least one or more of: (a) theta, (b) alpha, (c) tradicted by context. Sensorimotor Response (SMR), (d) beta, and (e) gamma. to 8. The method of claim 1, further comprising: displaying the at least one physiological signal having a The invention claimed is: value indicative of the level of the at least one physi- 1. A method for modifying the effect of an operator con- ological activity. trolled input device on an interactive device to encourage the 9. The method of claim 8, wherein the steps of measuring, self-regulation of at least one physiological activity by the 15 modifying, and displaying are performed wirelessly without operator, the operator controlled input device utilizing one or any direct connection to the operator controlled device and to more position reference signal elements to provide motion the interactive device. and/or position-sensing capability relative to the one or more 10. The method of claim 1, wherein the steps of measuring position reference signal elements, the interactive device and modifying are performed wirelessly without any direct comprising a display area that depicts images and an appara- 20 connection to the operator controlled device and to the inter- tus for receiving at least one input from the operator con- active device. trolled input device to thus permit the operator to control and 11. An apparatus for modifying the effect of an operator interact with at least some of the depicted images, the method controlled input device on an interface device to encourage for modifying comprising the steps of: the self-regulation of at least one physiological activity by the measuring at least one physiological activity of the opera- 25 operator, the operator controlled input device utilizing one or for to obtain at least one physiological signal having a more position reference signal elements to provide motion value indicative of the level of the at least one physi- and/or position-sensing capability relative to the one more ological activity; and position reference signal elements, the interactive device modifying the motion and/or position sensing capability of comprising a display area, a processor for depicting images the operator controlled input device by modifying one or 30 upon the display area, and a receiver device permitting the more position reference signal elements in response to operator to control and interact with at least some of the (a) changes in the measured at least one physiological depicted images, the modifying apparatus comprising: signal and/or (b) differences between the measured at at least one measurement device for measuring at least one least one physiological signal and a corresponding pre- physiological activity of the operator to obtain at least determined desired physiological signal level, thereby 35 one physiological signal having a value indicative of the modifying the ability of the operator to control and inter- level of the at least one physiological activity; and act with at least some of the depicted images, at least one modifying device for modifying the motion wherein the one or more position reference signal elements and/or position sensing capability of the operator con- comprise a plurality of LEDs, two of which emit light at trolled input device by modifying one or more position any particular time; wherein the operator controlled 40 reference signal elements in response to (a) changes in input device comprises a light detector for detecting the measured at least one physiological signal and/or (b) light emitted from the LEDs, and wherein the step of differences between the measured at least one physi- modifying one or more position reference signal ele- ological signal and a corresponding predetermined ments comprises the step of changing which two of the desired physiological signal level, thereby modifying LEDs are emitting light at any particular time. 45 the ability of the operator to control and interact with at 2. The method of claim 1, wherein the step of changing least some of the depicted images, which two of the LEDs are emitting light at any particular wherein the one or more position reference signal elements time comprises turning pairs of LEDs on and off in a prede- comprise two light emitting diodes (LEDs) or two sets of termined pattern based on either (a) the difference between LEDs having a fixed distance therebetween; wherein the the measured at least one physiological signal and the corre- 50 operator controlled input device comprises a light detec- sponding predetermined desired physiological signal level, or tor for detecting light emitted from the LEDs, and (b) the measured at least one physiological signal. wherein the signal modulator comprises a mechanical 3. The method of claim 1, wherein the plurality of LEDs translator for mechanically translating the two LEDs comprises two groups of linearly arranged LEDs, each of the from a first position to a second position, the difference two linearly arranged groups being positioned substantially 55 between the first and second positions being based on horizontally. either (a) the difference between the measured at least 4. The method of claim 1, wherein the plurality of LEDs one physiological signal and the corresponding prede- comprises two groups of linearly arranged LEDs, each of the termined desired physiological signal level, or (b) the two linearly arranged groups being positioned substantially measured at least one physiological signal. vertically. 60 12. The apparatus of claim 11, wherein the at least one 5. The method of claim 1, wherein the plurality of LEDs physiological signal comprises at least one or more of: skin comprise two groups of LEDs, each of the groups being temperature, skin conductance, electrical activity of muscles, arranged in two linearly arranged sub-groups, each sub-group blood flow, heart rate, heart rate variability, and respiratory of a group crossing the other sub-group at a center of each rate. sub-group and at a perpendicular angle. 65 13. The apparatus of claim 12, wherein the measurement 6. The method of claim 1, wherein the at least one physi- device is a chest band electrode in combination with a wire- ological signal comprises at least one or more of skin tem- less transmitter. US 8,827,717 B2 15 16 14. The apparatus of claim 11, wherein the at least one modifying the motion and/or position sensing capability of physiological signal comprises at least one or more of: brain- the operator controlled input device by modifying one or wave electrical event-related potentials, and at least one more position reference signal elements in response to brainwave frequency band; wherein the at least one brain- (a) changes in the measured at least one physiological wave frequency band comprises at least one or more of: (a) 5 signal and/or (b) differences between the measured at theta, (b) alpha, (c) Sensorimotor Response (SMR), (d) beta, least one physiological signal and a corresponding pre- and (e) gamma. determined desired physiological signal level, thereby 15. The apparatus of claim 11, further comprising: modifying the ability of the operator to control and inter- a display sub-area for displaying the at least one physi- act with at least some of the depicted images, ological signal having a value indicative of the level of 10 wherein the one or more position reference signal elements the at least one physiological activity. comprise a plurality of LEDs, two of which emit light at 16. The apparatus of claim 15, whereinthe display sub-area any particular time; wherein the operator controlled is substantially separate from the interactive device. input device comprises a light detector for detecting 17. The apparatus of claim 16, wherein the at least one light emitted from the LEDs, and wherein the step of measurement device, the at least one modifying device, and 15 modifying one or more position reference signal ele- the display sub-area are connected wirelessly without any ments comprises the step of changing which two of the direct connection to the operator controlled device and to the LEDs are emitting light at any particular time, interactive device. digitizing the at least one physiological signal, 18. The apparatus of claim 11, wherein the at least one testing the digitized physiological signal against a corre- measurement device and the at least one modifying device are 20 sponding predetermined desired physiological signal connected wirelessly without any direct connection to the level; and, on the basis of the test, dynamically adjusting operator controlled device and to the interactive device. digitally controlled potentiometers that are configured 19. A method for modifying the effect of an operator con- as voltage dividers to modulate the voltage that powers trolled input device on an interactive device to encourage the the one or more motion and/or position-sensing ele- self-regulation of at least one physiological activity by the 25 ments, thereby modifying the motion and/or position operator, the operator controlled input device having one or sensing capability of the operator controlled input more motion and/or position-sensing elements providing device in response to (a) changes in the measured at least motion and/or position-sensing capability, the interactive one physiological signal and/or (b) differences between device comprising a display area that depicts images and an the measured at least one physiological signal and a apparatus for receiving at least one input from the operator 30 corresponding predetermined desired physiological sig- controlled input device to thus permit the operator to control nal level, thereby modifying the ability of the operator to and interact with at least some of the depicted images, the control and interact with at least some of the depicted method for modifying comprising the steps of: images. measuring at least one physiological activity of the opera- 20. The method of claim 19, wherein the one or more tor to obtain at least one physiological signal having a 35 motion and/or position sensing elements comprise one or value indicative of the level of the at least one physi- more accelerometers and/or one or more gyroscopes. ological activity,