Rethinking the Relationship Between Humans and Computers

INDISTINGUISHABLE FROM MAGIC

Rethinking the Relationship between Humans and Computers

Xiangshi Ren, Kochi University of Technology

Human-engaged computing aims to recognized the limitations of AI in natural language processing and synergize innate human capacities and computer vision, respectively, and shifted their focus to enhancing hu- technological capabilities to enable us to man capacities.3 Despite such calls, researchers realize our full potential and solve complex haven’t adequately explored the real-world problems. use of information technology to fully realize our innate capabilities. Instead, IT has largely been hen designing for human–computer inter- used to automate tasks to achieve action (HCI), the focus should always be on greater efficiency and productivity. However, this hasn’t the human, not the computer. Otherwise, led to intrinsically better human outcomes. In fact, in computing systems could, at the least, many cases automation ironically has created problems Wlimit innovation and, at worst, put our lives at risk. I be- such as imposing added time pressure on human deci- lieve this is already happening. sion making.4 Some of today’s leading HCI experts share similar Stephen Hawking, Bill Gates, and Elon Musk have all views. Touchscreen innovator Bill Buxton wrote that com- expressed concerns about the negative impact of “su- puters should enable people to realize their full potential.1 per AI” on human progress and survival prospects. This Ben Shneiderman, a pioneer of direct manipulation, said anxiety is echoed by the public. A 2016 Accenture survey that future HCI design should improve both individual of 28,000 consumers in 28 countries found that people and societal outcomes by encouraging “trust, empathy, are bored with today’s technology and worried about its and responsibility.”2 Terry Winograd and Gary Bradski threats to their security and privacy.5

Is digital development serving SYNERGIZING HUMAN capacities by integrating research in- humanity, or vice versa? Are we only CAPACITIES AND sights from multiple disciplines. For technology consumers, conditioned TECHNOLOGICAL example, synthesizing findings from to satisfy corporate bottom lines, or CAPABILITIES neuroscience and positive psychology should technology help us realize our To achieve the high-level wisdom would help HCI researchers under- full potential? Such questions are crit- needed to solve complex real-world stand the costs and benefits of various ical to our future prospects. problems, HCI researchers must technologies to mental development, synergize human capacities and such as being more easily distracted COMPUTING FOR HUMAN technological capabilities. Effective or less able to regulate emotions on POTENTIAL synergism requires thoroughly iden- the one hand and being better able to In 2006, Douglas Engelbart, the vision- tifying human capacities, identifying multitask on the other. In the case of ary who defined the predominant HCI technological hindrances to realiz- a technology such as self-driving cars, paradigm, wrote to me that HCI design ing such capacities, and developing researchers must consider under what should increase “human capabilities novel technologies to enhance those circumstances surrendering control to to develop, integrate and understand capacities. AI is beneficial or detrimental in terms the knowledge required for improving Eastern philosophies recognize of overall human outcomes. society’s survival probability.” This that humans possess powerful inner Once human capacities have been motivated me to rethink the relation- capacities, some of which aren’t mea- thoroughly assessed, HCI researchers ship between humans and computers. surable. Traditional HCI limits human can synergize these with technologi- Pioneers like Engelbart and J.C.R. capacities to cognition as well as per- cal capabilities to maximize symbio- Licklider had no intention of neglect- ception and motor control. However, sis (complementarity) and minimize ing innate human potential or of aban- recent research indicates that raw antibiosis (conflict) between users and doning technological progress. “Men intelligence alone isn’t a sure predic- computers. Research indicates that will set the goals … and perform the tor of excellence—“softer” skills such our brain is malleable, consistently evaluations,” Licklider wrote in 1960. as focus, mindfulness, self-control, adapting to external responses.12 By al- “Computing machines will … prepare self-motivation, empathy, and trust tering the user experience with appro- the way for insights and decisions in can play an even greater role.9,10 More priate technology, HCI designers could technical and scientific thinking.”6 En- fully synergized HCI can only occur preserve, enhance, and even restore gelbart made the same point at the 1995 when the system properly recognizes human capacities. For example, mind- Vannevar Bush Symposium: “You have all individual and collective human fulness apps such as Headspace (www. to deal with both sides of the whole capacities. headspace.com) and Smiling Mind organization, i.e., the people and the Every technological innovation (http://smilingmind.com.au) can reduce machines—we need to find a way where comes at a human cost. For exam- stress and improve focus and self-con- both sides are going to co-evolve.”7 ple, using digital address books can trol, while crowdsourcing apps pro- Existing conceptions that take significantly diminish our memory, vide a means to exploit commonly ac- a more holistic view of HCI inade- while continuous interaction with cessible collective human capacities quately capture this co-evolution. For digital devices can hamper our abil- and help build social trust, empathy, example, human-centered comput- ity to focus. A recent Microsoft study and community. In some cases, human ing (HCC) focuses on “user needs,”8 concluded that our attention span is capacities might be better off without but HCC is often equated with conve- now lower than that of a goldfish.11 If technological intervention. nience or fun. User experience (UX) this trend continues, our natural ca- considers human emotions and val- pacities will decrease to the point at HUMAN-ENGAGED ues but is arguably still technology which the achievment of more deeply COMPUTING oriented, as UX is rooted in industrial synergized HCI will be significantly I refer to the synergism of human ca- and product design. MIT’s Center for limited. pacities and technological capabilities Collective Intelligence (http://cci.mit As a first step to developing better as human-engaged computing (HEC), .edu) explores “how people and com- computing systems, we must com- which is graphically conceptualized in puters can work together more intelli- prehensively define, evaluate, and Figure 1. HEC has three components: gently,” but intelligence is only a small prioritize human capacities as well engaged humans, engaging comput- part of human potential. as technological hindrances to those ers, and synergized interaction.

AUGUST 2016 105 INDISTINGUISHABLE FROM MAGIC

problems that need a sound compos- ite of human capacities and techno- High synergy logical capabilities? The role of HEC is to raise awareness by constantly HEC activates human capacities asking such questions.

Synergized interaction Synergized interaction refers to a state of optimal balance between engaged humans and engaging computers, an Eastern concept detailed in the sidebar “Human-Engaged Com- Human capacities puting: Getting the Balance Right.” In this harmonious state, computing technology isn’t merely harnessed HEC activates to accomplish some predefined goal technological capabilities or purpose; for example, to do some- thing faster or more efficiently. -In stead, the technology is designed Low synergy to complement and enhance human capacities—both existing and Technological capabilities potential—to achieve a higher-level and perhaps as-yet undefined pur- Figure 1. Human-engaged computing (HEC) synergizes innate human capacities and pose. In some cases, synergized inter- technological capabilities by maximizing symbiosis and minimizing antibiosis between action might mean not deploying an users and computers. otherwise useful technology.

holistic consideration of our Engaged humans engaging when they enhance human innate capacities and techno- Humans are engaged when their inner capacities. In other words, the merits Alogical hindrances to those capacities are progressively developed of technologies in HEC aren’t deter- capacities is essential to achieving and thoroughly integrated into a given mined by traditional quantitative or greater synergy between humans and activity. In its purest form, engage- qualitative metrics, such as whether computers. By drawing on the unique ment isn’t dependent on the nature of they increase productivity or are fun perspectives of multiple disciplines— the activity—for example, on whether to use, but by their ability to realize including the social sciences and hu- it’s entertaining or boring, difficult human potential. manities as well as different fields or easy, profitable or not. Rather, it’s a A key challenge in designing such within the HCI community—we can state of consciousness in which one is systems is determining what to do design systems that will enable us to fully immersed in and aligned with the if a technology increases one capac- realize our full potential and create a activity. This feeling of energized fo- ity but decreases other capacities. better world. cus is well known in Eastern practices For example, playing certain types such as mindfulness meditation and of games might improve short-term REFERENCES “no-mind” martial arts, and in Western learning outcomes for children, 1. W.A.S. Buxton, “Human Skills in psychology it’s variously described as but children might also gradually Interface Design,” Interacting with “achieving flow,” “being in the zone,” lose their natural capacity to self- Virtual Environments, L.W. MacDonald and the like. Pure human engagement motivate. Should we stop teaching and J. Vince, eds., Wiley, 1994, pp. 1–11. can’t be produced by external incen- students basic mathematics because 2. S. Lohr, “Humanizing Technology: A tives such as fun mechanisms because computing tools are available? Is it History of Human–Computer Inter- these require a divided mind. wise to facilitate the attenuation of action,” The New York Times, 7 Sept. thinking, spelling, writing, coordi- 2015; http://bits.blogs.nytimes Engaging computers nation, and other skills because a .com/2015/09/07/humanizing Computers (including both de- device can do these for us? What will -technology-a-history-of-human vices and software applications) are happen when our kids face complex -computer-interaction/?_r=0.

106 COMPUTER WWW.COMPUTER.ORG/COMPUTER criterion in making this determination is what best enhances HUMAN-ENGAGED human potential. The Eastern influence in HEC can also be seen in concepts COMPUTING: GETTING including wholeness, well-being, mindfulness, and absorption, THE BALANCE RIGHT which have received a lot of attention lately from companies like Apple, Facebook, and Google. In this respect, HEC rep- he field of human–computer interaction (HCI) has evolved resents a cross-fertilization of computing with Eastern-based Tover the years in four waves. The first wave focused psychological and experiential practices. on human factors in the context of industrial engineer- Other fields have successfully integrated Eastern and West- ing and ergonomics—optimizing the fit between humans ern approaches including yoga psychotherapy,13 herbalism,14 and machines.1,2 The second wave, inspired by cognitive and business management.15,16 Combining the holistic nature psychology, emphasized the similarity between human and of eastern awareness with the analytical nature of western machine information processing.3 The third wave shifted the thought could provide the most effective means to use com- research focus to humans by addressing some of the social puting to meet real human needs1.17 and emotional aspects of HCI.4,5 The fourth and current wave incorporates insights from positive psychology and References cognitive neuroscience to assume an even more human-cen- 1. S. Bødker, “When Second Wave HCI Meets Third Wave tered perspective that considers factors such as physical and Challenges,” Proc. 4th Nordic Conf. Human–Computer Inter- psychological well-being, creativity, emotions, ethical values, action (NordiCHI 06), 2006; doi:10.1145/1182475.1182476. and self-realization.6–9 2. S. Harrison, D. Tatar, and P. Sengers, “The Three Paradigms Human-engaged computing (HEC) shares some of the of HCI,” Proc. Alt.chi. Session of the SIGCHI Conf. Human goals of fourth-wave HCI research but focuses on identify- Factors Computing Systems (CHI 07), 2007 https://people ing ways to engage humans to realize their full potential. It .cs.vt.edu/~srh/Downloads/TheThreeParadigmsofHCI.pdf. combines the concept of human–computer symbiosis first 3. S.K. Card, T.P. Moran, and A. Newell, The Psychology of articulated in the early 1960s by Joseph Licklider10 and Douglas Human–Computer Interaction, Lawrence Erlbaum Associ- Engelbart11 with the Chinese philosophy of the “right balance” ates, 1983. espoused in ancient texts such as the I Ching and Confucian 4. R. Kling and S.L. Star, “Human Centered Systems in the norms such as Chung Yung (Doctrine of the Mean).12 The right Perspective of Organizational and Social Informatics,” (also called optimal or golden) balance is a state of comple- ACM SIGMAS Computers and Society, vol. 28, no. 1, 1998, mentarity between interacting partners such that together they pp. 22–29. achieve more than they could independently. The implication 5. Y. Rogers, HCI Theory: Classical, Modern, and Contempo- of this for HCI is that humans and computers should receive rary, Morgan & Claypool, 2012. proportional, not equal, treatment according to their respec- 6. L. Brannon, “Reimagining HCI: Toward a More Human-Centered tive contributions to overall harmony in any given interaction. Perspective,” Interactions, vol. 18, no. 4, 2011, pp. 50–57. Naively apportioning equal resources to unequally contributing 7. A. Borning and M. Muller, “Next Steps for Value Sensitive partners leads to antibiosis—an inefficient wastage of effort on Design,” Proc. SIGCHI Conf. Human Factors in Computing one partner and neglect of the other. Put simply, there are few Systems (CHI 12), 2012, pp. 1125–1134. situations in which the right balance means expenditures on 8. R. Calvo and D. Peters, Positive Computing: Technology for the constituent elements will be quantitatively equal. Despite Wellbeing and Human Potential, MIT Press, 2014. this, both elements are indispensible to the synthesis regard- less of the respective apportioning of expenditures. The key continued on p. 108

3. J. Markoff, Machines of Loving Grace: .accenture.com/_acnmedia/PDF-3 Brown/MIT Vannevar Bush Symp., The Quest for Common Ground between /Accenture-Igniting-Growth-in 12 Oct. 1995; https://archive.org Humans and Robots, Ecco, 2015. -Consumer-Technology.pdf. /details/XD1941_1_95Vannevar 4. L. Bainbridge, “Ironies of Automa- 6. J.C.R. Licklider, “Man–Computer BushSymTape2_DougEngelbart. tion,” Automatica, vol. 19, no. 6, 1983, Symbiosis,” IRE Trans. Human Factors 8. A. Jaimes et al., “Human-Centered pp. 775–779. in Electronics, vol. HFE-1, 1960, pp. 4–11. Computing: Toward a Human 5. Accenture, “Igniting Growth in Con- 7. D. Engelbart, “The Strategic Pursuit Revolution,” Computer, vol. 40, no. 5, sumer Technology,” 2016; www of Collective IQ,” presentation, 30–34. 2007.

AUGUST 2016 107 INDISTINGUISHABLE FROM MAGIC

continued from p. 107 13. M. Caplan, A. Portillo, and L. Seely, “Yoga Psychotherapy: The Integration of Western Psychological Theory and An- 9. K. Kuutti and L.J. Bannon, The Turn to Practice in HCI: To- cient Yogic Wisdom,” J. Transpersonal Psychology, vol. 45, wards a Research Agenda, Proc. SIGCHI Conf. Human Fac- no. 2, 2013, pp. 139–158. tors in Computing Systems (CHI 14), 2014, pp. 3543–3552. 14. J. Freeman, Integrated East/West Herbalism, Lulu, 2011. 10. J.C.R. Licklider, “Man–Computer Symbiosis,” IRE Trans. Hu- 15. D.D. Ding, “The Emergence of Technical Communication man Factors in Electronics, vol. HFE-1, 1960, pp. 4–11. in China—Yi Jung (I Ching): The Budding of a Tradition,” 11. D.C. Engelbart, Augmenting Human Intellect: A Conceptual J. Business and Technical Comm., vol. 17, no. 3, 2003, pp. Framework, SRI Summary Report AFOSR-3223, Stanford 319–345. Research Inst., Oct. 1962; www.dougengelbart.org/pubs 16. Y. Taguchi, The Managerial Ideas from the East, Y. Noji, /papers/scanned/Doug_Engelbart-AugmentingHuman trans., Babel Corp., 2012. Intellect.pdf. 17. E. Law et al., “Leveraging and Integrating Eastern and West- 12. Y.-L. Fung, A Short History of Chinese Philosophy: A Sys- ern Insights into Human Engagement Studies,” Proc. SIGCHI tematic Account of Chinese Thought from Its Origins to the Conf. Human Factors in Computing Systems (CHI 15), 2015, Present Day, reissue ed., D. Bodde, ed., Free Press, 1997. pp. 2433–2436.

JANUARY/FEBRUARY 2016

IEEE SOFTWARE 9. D. Goleman, Focus: The Hidden Driver WWW.COMPUTER.ORG/SOFTWARE MARCH/APRIL 2016 of Excellence, HarperCollins, 2013. IEEE SOFTWARE WWW.COMPUTER.ORG/SOFTWARE 10. D Kahneman, Thinking, Fast and Slow,

Farrar, Straus and Giroux, 2011. January/February 2016 11. Microsoft Canada (Consumer

March/April 2016 Insights), “Attention Spans,” spring

2015; https://advertising.microsoft CODE CLARITY // 22

SOFTWARE .com/en/WWDocs/User/display/cl

THE FUTURE OF ENGINEERING SOFTWARE ON A COMET // 81 /researchreport/31966/en/microsoft

BIG DATA BIG -attention-spans-research-report.pdf. TINY PROGRAMMING TOOLS // 24

REQUIREMENTS 12. N. Doidge and J. Bond, The Brain’s & SOCIAL RESPONSIBILITY // 109 Way of Healing: Remarkable Discover-

ies and Recoveries from the Frontiers of Neuroplasticity, Penguin Books, 2016. Volume 33 Number 1 Number 33 Volume Volume 33 Number 2 Number 33 Volume

XIANGSHI REN is a professor in the School of Information and director of the Center for Human-Engaged Computing at Kochi University of Technology. Contact him at ren IEEE Software offers pioneering ideas, expert [email protected]. analyses, and thoughtful insights for software professionals who need to keep up with rapid technology change. It’s the authority on translating software theory into practice.

Selected CS articles and www.computer.org/software/subscribe columns are also available for free at http://ComputingNow .computer.org.

108 COMPUTER WWW.COMPUTER.ORG/COMPUTER