Homo Cyberneticus: the Era of Human-AI Integration

Homo Cyberneticus: The Era of Human-AI Integration

Jun Rekimoto The University of Tokyo, Sony CSL, http://lab.rekimoto.org, [email protected]

Author Keywords machines, assuming humans are participating as operators in HCI vision; human-augmentation; human-AI integration automated factories. However, this can now be seen as when human abilities are given to machines, such as a telepresence HUMAN-AUGMENTATION robot [7, 14]. Systems that learn from gathering collective Neo: Can you fly that thing? human intelligence, such as recommendation systems or ma- Trinity: Not yet. chine translation systems, may fall into this category. Thus, Licklider’s counterexamples of symbiosis can be said to show In the movie “The Matrix,” Trinity responds to Neo right two important directions of human augmentation. before having the helicopter’s maneuverability downloaded into her brain [18]. Will such a future come? Although it may take some time to reach the world of The Matrix, technologies HUMAN-AI INTEGRATION that can enhance and augment human abilities are advancing One of my current projects is a silent voice system that gener- rapidly [17, 16,3]. ates a voice from intraoral movements taken from ultrasound echo images using deep-neural networks [10]. The project The idea that technology enhances humanity has a long history. showed the possibility of verbal communication in situations Robert Hooke, the inventor of the optical microscope, con- where vocal speech is difficult, or when someone has vocal sidered the microscope to be an extension of vision, and that cord damage. “There may be found many Mechanical Inventions to improve our other Senses, of hearing, smelling, tasting, touching.” [8]. This human-augmenting system is also a kind of training sys- Vannevar Bush’s MEMEX was intended to EXpand human tem. The system learns to generate voices and helps users MEMory [1]. For Marshall McLuhan, all media technologies improve their silent lip-mouse-moving patterns by listening were extensions of humans, as radio was an ear extension to the sounds that the system generates. Glove Talk II [5] is a and television was an eye extension [13]. Douglas Engelbart voice generation system that uses hand gestures. The user’s explained “The mouse was just a tiny piece of a much larger hand gestures and voice synthesizer parameters are connected project aimed at augmenting human intellect.” [11,4]. His with neural networks. A user can learn to control voices with ultimate goal was to augment human intellect, and mice and hand gestures and can even utter non-verbal expressions such GUIs were tools to realize that goal. However, the results as “Yeah?” with their hands. These examples are similar to of Engelbart and subsequent Alto [9] were so successful that how people learn skills. When we learn utterances, the coordi- our research interests have been more directed towards im- nation of the motor cortex that drives the oral cavity, tongue, proving such tools, interaction technologies and input/output vocal folds, and the auditory cortex forms a tight feedback devices, than exploring and deepening the various possibilities loop to obtain a better speech performance. The examples of enhancing humans. above extend this loop and connects organic neural networks (e.g., the brain) and artificial neural networks in the one loop. In that regard, J.C.R. Licklider’s “Man-Computer Symbio- sis” [12] is worth reviewing. Here, symbiosis means “living I think there are many possibilities in this direction. Rather together in intimate association, or even close union, of two than considering AI an autonomous or separated entity, we dissimilar organisms.” The assumption is that humans and may be able to regard AI as an integrated part of humans.

arXiv:1911.02637v1 [cs.HC] 21 Oct 2019 computers compliment each other with their respective abil- Human motor ability and cognitive ability can be expanded ities. More interestingly, Licklider also gives two examples by AI closely linked with humans. I would like to name this of non-symbiotic human-computer relationships. First, the human-AI integration rather than human-AI interaction (see concept of the mechanically extended man is not symbiotic. the Appendix diagram for the four categories of HCI and AI The concept refers to something like a prosthetic leg or glasses, relationships). As an extension of such research, the combina- but it can now be said that it leads to the development of cy- tion of an invasive brain and AI will also be considered [15]. borgs. The other non-symbiotic example is humanly extended What happens if humans form a network through these con- nections [6]? If such a connection can directly extend our capabilities, we will be able to re-design ourselves, and it will Permission to make digital or hard copies of part or all of this work for personal or be the future of interaction design. classroom use is granted without fee provided that copies are not made or distributed for proﬁt or commercial advantage and that copies bear this notice and the full citation on the ﬁrst page. Copyrights for third-party components of this work must be honored. Arthur Charles Clarke stated, “The old idea that man invented For all other uses, contact the owner/author(s). tools is a misleading half-truth. It would be more accurate to UIST’19, October 20–23, 2019, New Orleans, LA, USA say that tools invented man”[2]. A major theme of future HCI © 2019 Copyright held by the owner/author(s). research will be reinventing humans in an era when humans ACM ISBN 978-1-4503-2138-9. and technology integrate. DOI: 10.1145/1235 Self Human-AI Integration Invisible personal agency Human Augmentation calm Cyborg Prosthetic Implant

XAI IoX (eXplainable AI) EdgeAI Smart City AI + GUI HCI x AI AI as a Tool Smart Environment (Ubicomp+AI)

Human-Robot Interaction (HRI) Intelligent systems explicit Agent autonomous Visible Bot Other

Appendix. Relationships of HCI and AI: The relationship between AI and humans is classiﬁed by two axes. The one axis is visible or invisible; it shows whether an interaction is explicit or a more integrated and invisible (calm) interaction. The other axis is self or other; it classiﬁes by interactions with individuals or by interactions with the outside world. As shown in this diagram, HCI and AI can have four quadrants.

REFERENCES [10] Naoki Kimura, Michinari Kono, and Jun Rekimoto. [1] Vannevar Bush. 1945. As We May Think. The Atlantic 2019. SottoVoce: An Ultrasound Imaging-Based Silent Monthly 176, 1 (1945), 101–108. Speech Interaction Using Deep Neural Networks. In Proceedings of the 2019 CHI Conference on Human [2] Arthur C. Clarke. 1967. Proﬁles of the future: an inquiry Factors in Computing Systems (CHI ’19). ACM, New into the limits of the possible. Bantam Books. York, NY, USA, Article 146, 11 pages. DOI: [3] Mike Daily, Antti Oulasvirta, and Jun Rekimoto. 2017. http://dx.doi.org/10.1145/3290605.3300376 Technology for Human Augmentation. Computer 50, 02 [11] Valerie Landau, Eileen Clegg, and Douglas Engelbart. (feb 2017), 12–15. DOI: 2009. The Engelbart Hypothesis: Dialogs with Douglas http://dx.doi.org/10.1109/MC.2017.39 Engelbart. NextPress. [4] D.C. Engelbart. 1962. Augmeting human intellect: a [12] J.C.R. Licklider. 1960. Man-Computer Symbiosis. IRE conceptual framework. SRI (1962). Trans. on Human Factors in Electronics (1960), 4–11. [5] Sidney Fels and Geoffrey Hinton. 1995. Glove-TalkII: [13] Marshall McLuhan. 1964. Understanding Media: The An Adaptive Gesture-to-formant Interface. In Extensions of Man. McGraw-Hill. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’95). ACM [14] Marvin Minsky. 1980. Telepresence. OMNI July (1980), Press/Addison-Wesley Publishing Co., New York, NY, 45–51. USA, 456–463. DOI: [15] Elon Musk and Neuralink. 2019. An integrated http://dx.doi.org/10.1145/223904.223966 brain-machine interface platform with thousands of [6] Carles Grau, Romuald Ginhoux, Alejandro Riera, channels. bioRxiv (2019). DOI: Thanh Lam Nguyen, Hubert Chauvat, Michel Berg, http://dx.doi.org/10.1101/703801 JuliÃa˘ L. Amengual, Alvaro Pascual-Leone, and Giulio [16] Jun Rekimoto. 2014. A New You: From Augmented Rufﬁni. 2019. Conscious Brain-to-Brain Reality to Augmented Human. In Proceedings of the Communication in Humans Using Non-Invasive Ninth ACM International Conference on Interactive Technologies. PROS ONE (2019). DOI: Tabletops and Surfaces (ITS ’14). ACM, New York, NY, http://dx.doi.org/10.1371/journal.pone.0105225 USA, 1–1. DOI: [7] Robert A. Heinlein. 1942. Waldo. Astounding Science http://dx.doi.org/10.1145/2669485.2670531 Fiction (1942). [17] Jun Rekimoto and Katashi Nagao. 1995. The World [8] Robert Hooke. 1665. Micrographia: or, Some Through the Computer: Computer Augmented physiological descriptions of minute bodies made by Interaction with Real World Environments. In magnifying glasses. J. Martyn and J. Allestry. Proceedings of the 8th Annual ACM Symposium on User Interface and Software Technology (UIST ’95). ACM, [9] A. Kay and A. Goldberg. 1977. Personal Dynamic New York, NY, USA, 29–36. DOI: Media. Computer 10, 3 (March 1977), 31–41. DOI: http://dx.doi.org/10.1145/215585.215639 http://dx.doi.org/10.1109/C-M.1977.217672 [18] The Wachowskis. 1999. The Matrix. (1999).