(12) United States Patent (10) Patent No.: US 7,113,848 B2 Hanson (45) Date of Patent: Sep
Total Page:16
File Type:pdf, Size:1020Kb
USOO7113848B2 (12) United States Patent (10) Patent No.: US 7,113,848 B2 Hanson (45) Date of Patent: Sep. 26, 2006 (54) HUMAN EMULATION ROBOT SYSTEM 6,391,233 B1 5/2002 Otani et al. ................... 264/49 2003/O110540 A1 6/2003 Fukui et al. .................. 90.1/50 (76) Inventor: David F. Hanson, 6117 Reiger Ave., 2004/0254771 A1* 12/2004 Riener et al. .................. 703/7 Dallas, TX (US) 75214 OTHER PUBLICATIONS (*) Notice: Subject to any disclaimer, the term of this Hara et al., Real-time facial interaction between human and 3D face patent is extended or adjusted under 35 robot agent, 1996, IEEE., p. 401-409.* U.S.C. 154(b) by 4 days. Bischoff et al., Improving dependability of humanoids, 2001, IEEE, p. 1-8.* (21) Appl. No.: 10/862,809 Gomi et al., elements of artificial emotion, 1995, IEEE, p. 265-268.* 9 Yuji et al. A new multifunctional tactile sensing technique by 1-1. selective data processing, 2000, IEEE, p. 1091-1094).* (22) Filed: Jun. 7, 2004 Fumiya Iida, Harumi Ayai, Fumio Hara, “Behavior learning of a (65) Prior PublicationO O Data faceSymposia, robot using6 pgs., human 1999. natural instruction.” Proc. of 4th Robotics US 2004/O24951O A1 Dec. 9, 2004 k cited. by examiner Related U.S. Application Data Primary Examiner Thomas G. Black (60) Provisional application No. 60/477.310, filed on Jun. Assistant Examiner-McDieunel Marc 9, 2003. (74) Attorney, Agent, or Firm Baker Botts L.L.P. (51) Int. Cl. (57) ABSTRACT G06F 9/00 (2006.01) One aspect is a robot system comprising a flexible artificial (52) U.S. Cl. ...................... 700/245; 700/253; 700/258; skin operable to be mechanically flexed under the control of 521/79 a computational system. The system comprises a first set of (58) Field of Classification Search ................ 700/245, Software instructions operable to receive and process input 700/253, 258; 521/79 images to determine that at least one human likely is present. See application file for complete search history. The system comprises a second set of Software instructions (56) References Cited operable to determine a response to a perceived human presence, whereby the computational system shall output U.S. PATENT DOCUMENTS signals corresponding to the response, such that, in at least 4,076,656 A 2, 1978 White et al. ................. 260f2.5 Some instances, the output signals cause the controlled 4,481,001 A * 1 1/1984 Graham et al. ............. 434,267 flexing of the artificial skin. 6,060,530 A 5/2000 Chaouk et al. ............... 521,64 6,072,496 A * 6/2000 Guenter et al. ............. 345/419 39 Claims, 15 Drawing Sheets U.S. Patent Sep. 26, 2006 Sheet 1 of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 2 of 15 US 7,113,848 B2 F.G.2 U.S. Patent Sep. 26, 2006 Sheet 3 of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 4 of 15 US 7,113,848 B2 FG.4 U.S. Patent Sep. 26, 2006 Sheet S of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 6 of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 7 Of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 8 of 15 US 7,113,848 B2 48 51 37 FG. 8 U.S. Patent Sep. 26, 2006 Sheet 9 Of 15 US 7,113,848 B2 FIG. S. U.S. Patent Sep. 26, 2006 Sheet 10 of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 11 of 15 US 7,113,848 B2 25 U.S. Patent Sep. 26, 2006 Sheet 12 of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 13 of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 14 of 15 US 7,113,848 B2 U.S. Patent Sep. 26, 2006 Sheet 15 Of 15 US 7,113,848 B2 US 7,113,848 B2 1. 2 HUMAN EMULATION ROBOT SYSTEM FIG. 4 illustrates one process by which one may cause the skin to have controlled thickness throughout its form. PRIORITY CLAIM FIG. 5 illustrates one embodiment of expression affector, a system by which expressive movement may be affected in This application claims the benefit (e.g. 35 U.S.C. S 119 a skin. (e)) of U.S. Provisional Application No. 60/477.310, which FIG. 6 illustrates one method by which anchors may be was filed on Jun. 9, 2003. embedded into the skin by affixing them with pins into the mold of the face-form prior to casting the skin. TECHNICAL FIELD OF THE INVENTION FIG. 7 illustrates one embodiment of a mechanical frame, 10 a system that may mechanically Support skin. This invention relates generally to robotics and more FIG. 8 illustrates one embodiment of a system that may be particularly to a robot and system to emulate human behav used to affect naturalistic motion in an artificial lip. ior. FIG. 9 illustrates one embodiment of an eye mechanism, a system that may be used to affect natural-appearing motion BACKGROUND OF THE INVENTION 15 in artificial eyes. FIG. 10 illustrates one embodiment of a system that may The means by which people use a computer may be be used to affect motion in an elastomeric skin of an artificial referred to as a Human Computer Interface (HCI). An HCI human or animal, by a linkage acting within a slot in the generally comprises output device(s) such as a monitor mechanical frame. screen or printer, input device(s) such as a keyboard, touch FIG. 11 illustrates one embodiment of a system that may screen, tablet, Scanner, or mouse. Existing robots generally be used to affect the appearance of the dynamic action of the Suffer from various disadvantages. For example, existing human eyelids. robots may not have realistic looking and acting artificial FIG. 12 illustrates one embodiment combination of skin, skin. Other robots may not be capable of realistically pro anchors, linkage, actuators, mechanical frame, eye mecha ducing appropriate facial expressions or other nonverbal 25 nism, eyelids, and assorted sensors, so as to emulate com communications. municative functions of the human face as a whole. FIG. 13 is a block diagram illustrating one embodiment of SUMMARY OF THE INVENTION a control system configuration that may be used to emulate the cognitive aspects of human communication in an HER. One aspect of the invention is a robot system comprising 30 FIG. 14 is a block diagram illustrating an embodiment of a flexible artificial skin operable to be mechanically flexed a Human Emulation Robot being used to access, navigate, or under the control of a computational system. The system modify the information present on a computer network. comprises a first set of Software instructions operable to FIG. 15 is a block diagram illustrating a method for receive and process input images to determine that at least producing an artificial skin material. one human likely is present. The system comprises a second 35 set of software instructions operable to determine a response DETAILED DESCRIPTION OF THE DRAWINGS to a perceived human presence, whereby the computational system shall output signals corresponding to the response, The preferred embodiment of the present invention and its Such that, in at least Some instances, the output signals cause advantages are best understood by referring to FIGS. 1 the controlled flexing of the artificial skin. 40 through 15 of the drawings, like numerals being used for like The invention has several important technical advantages. and corresponding parts of the various drawings. Embodiments of the invention may have none, Some, or all One aspect of the invention is an HCI system called a of these technical advantages without departing from the Human Emulation Robot (HER), comprising electrome Scope of the invention. The invention provides a more chanical emulation of at least Some naturally-occurring naturalistic mechanism to interact with various electronic 45 structures, patterns, and/or codes associated with human devices. By providing artificial skin with at least some communications, in order to generate human-computer characteristics of human skin, the invention facilitates more interactions that may be more innately intelligible and accurate emulation of human facial expressions and/or ges comforting to humans than previous HCI systems. In one tures. Another important technical advantage is that the embodiment, an HER comprises apparatus and/or software 50 for an HER to sense and perceive at least some natural invention may allow perception of the emotional state of a non-verbal human communication signals, apparatus and/or human with which a robot is interacting and a simulated software for an HER to decide meaningful responses to emotional response by the robot to better facilitate interac sensory perceptions, and apparatus and/or software for an tion between the robot and a human. HER to emulate, at least in part, natural human communi 55 cative output. To sense and perceive natural human com BRIEF DESCRIPTION OF THE DRAWINGS munication signals, the HER may employ (without limita tion) face-tracking machine vision, audio-sensing, facial For a more complete understanding of the present inven biometrics, electronic chemical sensing (smell), and touch tion and the advantages thereof, reference is now made to sensing. Other hardware and/or Software systems may also the following descriptions taken in conjunction with the 60 be used. To decide meaningful responses to sensory percep accompanying drawings in which: tions, the HER may employ any, all, or none of Expert FIG.