1 SHAIL, 14 DECEMBER 2017
TODAY FROM: MALAGA, SPAIN
ROLF PFEIFER, DR. SC. TECHN. ETH LIVING WITH ROBOTS LTD., SHANGHAI JIAO TONG UNIVERSITY, CHINA PROF. EM. UNIVERSITY OF ZURICH 2
Living with Robots Ltd. 3
Department of Automation, Shanghai Jiao Tong University, China 4
SHAIL, 14 DECEMBER 2017 HOW THE BODY SHAPES THE WAY WE THINK - 10 YEARS LATER
ROLF PFEIFER, DR. SC. TECHN. ETH LIVING WITH ROBOTS LTD., SHANGHAI JIAO TONG UNIVERSITY, CHINA PROF. EM. UNIVERSITY OF ZURICH 5
SHAIL, 14 DECEMBER 2017 THE “ROBOT/AI HYPE”, EMBODIED INTELLIGENCE, SOFT ROBOTICS, AND DEEP NEURAL NETWORKS
ROLF PFEIFER, DR. SC. TECHN. ETH LIVING WITH ROBOTS LTD., SHANGHAI JIAO TONG UNIVERSITY, CHINA PROF. EM. UNIVERSITY OF ZURICH EMBODIED INTELLIGENCE 6 WHY DO PLANTS NOT HAVE BRAINS?
(Lewis Wolpert, British biologist)
… EMBODIED INTELLIGENCE 7 WHY DO PLANTS NOT HAVE BRAINS?
“The answer is actually quite simple: they don’t have to move! “
(Lewis Wolpert, British biologist)
… EMBODIED INTELLIGENCE 8 THERE ARE OTHER THINGS BESIDES DEEP NEURAL NETWORKS
… THE “ROBOTIC/AI HYPE” 9 ROBOTICS* 2017 HAS A PROBLEM
* CONSUMER ROBOTICS, SOCIAL ROBOTS - NOT INDUSTRIAL ROBOTICS ▸ Random Interviews with consumers, 04/ 2016 EXPECTATION VS. REALITY 10 THE “ROBOTIC/AI HYPE” 11 EXPECTATION REALITY
- perfect functionality - polished look
- toy look - little functionality THE “ROBOTIC/AI HYPE” 12 REALITY OF „ROBOTICS-REVOLUTION“ 2016/2017
Hefei (China) THE “ROBOTIC/AI HYPE” 13 THERE IS A ROBOTIC HYPE ! THE “ROBOTIC/AI HYPE” 14 THERE IS A ROBOTIC HYPE ! THE “ROBOTIC/AI HYPE” 15 THERE IS A ROBOTIC HYPE ! THE “ROBOTIC/AI HYPE” 16 THE “ROBOTIC HYPE”
CBSNEWS, December 2013 THE “ROBOTIC/AI HYPE” 17 THE “ROBOTIC HYPE”
CBSNEWS, December 2013 THE “ROBOTIC/AI HYPE” 18 THE “ROBOTIC HYPE”
“The dawn of the age of robotics”
Frankfurter Allgemeine Zeitung 9 February 2017, p. 21 THE “ROBOTIC/AI HYPE” 2017 19 THE TABLOID PRESS What is Artificial Intelligence?
Martin Vetterli, President EPFL
18 October 2017 THE “ROBOTIC/AI HYPE” 2017 20 THE TABLOID PRESS
Robots take command in the garden
20min, Tabloid 19 Mai 2017 THE “ROBOTIC/AI HYPE” 21 UBS: AI TO BOOST TRADERS’ PERFORMANCE 2017 Financial Times 7 July 2017
UBS as “Pionier” THE “ROBOTIC/AI HYPE” source: AlphaGo Movie website 22 DEEPMIND - DEVELOPER OF ALPHAGO 2017 Pascal Kaufmann DER ROBOTER/AI HYPE SWISS INITIATIVE:Starmind InternationalMINDFIRE 23 THE “ROBOTIC/AI HYPE” 24 ROBOTS IN FACTORIES
welding robots THE “ROBOTIC/AI HYPE” 25 ROBOTS IN FACTORIES
welding robots THE “ROBOTIC/AI HYPE” 26
THE “CHAMPION”: ROBOTS LEAVING THE FACTORY HALLS HONDA ASIMO THE “ROBOTIC/AI HYPE” 27
THE “CHAMPION”: ROBOTS LEAVING THE FACTORY HALLS HONDA ASIMO THE “ROBOTIC/AI HYPE” 28
THE “CHAMPION”: ROBOTS LEAVING THE FACTORY HALLS - HONDA ASIMO MOVING INTO OUR OWN LIVING SPACE
Robotics is hard !! LIVING WITH ROBOTS Service Robotics 29
29 THE “ROBOTIC HYPE” Service Robotics 30
30 Darwin-OP (Trosson) THE “ROBOTIC HYPE” “Social Robots” 31
Plen2 (Plen2 Proj.) Jibo Robohon/Robi (Robo Garage) Zenbo (Asus)
Balro (Fujisoft) Kiribo (Toyota) Alpha 1s (UBTECH)
Alpha 2 (UBTECH) Nao (Ald./Softbank) Buddy (Blue31 Frog Rob.) Unisrobo (UNIS Corp.) Tapia (MJI Robotics) Jibo THE “ROBOTIC HYPE” Jibo-Imitations 32
Jibo (original)
ElliQ (Intuition Robotics) Jibo imitation
JD.CC HowAreYou? 32 Pebble (Rokid) Olly (Emotech) “Social Robots”
From: WRC Oct 2016 “Social Robots”
From: WRC Oct 2016 “Social Robots”
From: WRC Oct 2016 “Social Robots”
From: Robot Show, Shanghai, July 2017 “Social Robots”
From: Robot Show, Shanghai, July 2017 “Social Robots”
From: WRC, Beijing, 2017, August 2017 “Social Robots”
From: WRC, Beijing, 2017, August 2017 “Social Robots”
functionality of an iPhone THE “ROBOTIC/AI HYPE” 41 SOCIAL ROBOTS 2017 - A SOBERING EXPERIENCE ?
Café Nescafé Harajuku, Tokyo November 2015
Rolf with Softbank’s “Pepper” - the ‘STAR’ THE “ROBOTIC/AI HYPE” 42 SOCIAL ROBOTS 2017 - A SOBERING EXPERIENCE ?
Café Nescafé Harajuku, Tokyo November 2015
Rolf with Softbank’s “Pepper” - the ‘STAR’ THE “ROBOTIC/AI HYPE” 43
SPIEGEL 43, 2016
“Hello, Pepper?” Café Nescafé Harajuku, Tokyo Why a stupid robot gets into the November 2015 headlines? Rolf with Softbank’s Spiegel 43, 2016 “Pepper” THE “ROBOTIC/AI HYPE” 44
SPIEGEL 43, 2016
“Hello, Pepper?” Café Nescafé Harajuku, Tokyo Why a stupid robot gets into the November 2015 headlines? Rolf with Softbank’s Spiegel 43, 2016 “Pepper” THE “ROBOTIC/AI HYPE” 45 CAREFUL: ANTHROPOMORPHIZATION!
ANTRHOPOMORPHIZATION
“… to attribute human form or personality to things not human.” Café Nescafé Merriam Webster Harajuku, Tokyo November 2015
Rolf with Softbank’s “Pepper” THE “ROBOTIC/AI HYPE” 46 ANTHROPOMORPHIZATION
Café Nescafé Harajuku, Tokyo November 2015 THE “ROBOTIC/AI HYPE” 47 ANTHROPOMORPHIZATION LIVING WITH ROBOTS 48 ROBOY: GESICHTSAUSDRUCK
“Hello - my name is Roboy”
Photography:Credit: AdrianTEDx Baer Zurich THE “ROBOTIC/AI HYPE” 49 ROBOY: FACIAL EXPRESSION
Credit: TEDx Zurich THE “ROBOTIC/AI HYPE” 50 SOCIAL ROBOTS - REALITY CHECK ▸ mostly children’s toy look
▸ social interaction but lack of manipulation skills in real world
▸ movement and locomotion limited
▸ novelty wears off quickly
▸ in essence: functionality of iPhone “Social Robots”
▸ “iPhones in tin cans”
▸ “iPhones on wheels” “Social Robots”
▸ “iPhones in tin cans”
▸ “iPhones on wheels”
From: WRC 2017, August 2017 THE “ROBOTIC/AI HYPE” 53 SOCIAL ROBOTS - REALITY CHECK ▸ mostly children’s toy look
▸ social interaction but lack of manipulation skills in real world
▸ movement and locomotion limited
▸ novelty wears off quickly
▸ in essence: functionality of iPhone
FULL-FEATURED HUMANOIDS NOT ANYWHERE IN SIGHT THE “ROBOTIC/AI HYPE” 54 DEEP NEURAL NETWORKS THE “ROBOTIC/AI HYPE” 55 DEEP NEURAL NETWORKS
▸ image processing
▸ AlphaGo
▸ speech and language
▸ self-driving vehicles
▸ medical diagnostics
▸ Big Data
▸ etc. THE ROBOTIC/AI HYPE 56 DEEP NEURAL NETWORKS
“children playing Frisbee” THE ROBOTIC/AI HYPE 57 DEEP NEURAL NETWORKS - HUGE AMOUNTS OF DATA REQUIRED
children playing “Frisbee” DER ROBOTER/AI HYPE 58 DEEP NEURAL NETWORKS
▸ image processing
▸ AlphaGo
▸ speech and language
▸ self-driving vehicles
▸ medical diagnostics
▸ Big Data
▸ etc. THE ROBOTIC/AI HYPE 59 DEEP NEURAL NETWORKS THE ROBOTIC/AI HYPE 60 DEEP NEURAL NETWORKS
Gibbon THE ROBOTIC/AI HYPE 61 DEEP NEURAL NETWORKS
“Statistically impressive, but individually unreliable.” John Launchbary, DARPA Gibbon THE ROBOTIC/AI HYPE THE ULTIMATE HYPE IN DNN: ALPHAGO 62
GO: AlphaGo against Lee Sedol, 2016 DER ROBOTER/AI HYPE ALPHAGO - AN ALGORITHM! 63
GO: AlphaGo against Lee Sedol, 2016 DER ROBOTER/AI HYPE ALPHAGO - AN ALGORITHM! 64
ENTIRELY “DISEMBODIED” - SOFTWARE
GO: AlphaGo against Lee Sedol, 2016 THE “ROBOTIC/AI” HYPE THE “AI” 65
Internet of Things Pepper Baxter Drones Honda Asimo Industrial Robot Self-driving car Amazon Echo
IBM Watson the “AI” ”machine learning” Social robots “deep neural networks” Big Data
IBMs Deep Blue
AlphaGo “Society 5.0” Apple Siri Industry 4.0 “Made in China 2025” THE “ROBOTIC/AI” HYPE THE “AI” 66
- complete separation of hardware and software - intelligence = software Internet of Things Pepper Baxter Drones Honda Asimo - unintelligent mechatronic structure Industrial Robot Self-driving- morphology/materials car completely ignored Amazon Echo - input-processing-output IBM- interaction Watson with environment neglected
the “AI” Social robots
Big Data
IBMs Deep Blue
AlphaGo “Society 5.0” Apple Siri Industry 4.0 “Made in China 2025” THE ROBOTIC/AI HYPE 67 PERFORMANCE VS. COMPETENCE THE ROBOTIC/AI HYPE 68 PERFORMANCE VS. COMPETENCE Performance
AlphaGo AI/Deep Neural Networks human machine learning
(uninteresting) “intellectual”
Competence THE ROBOT/AI HYPE ALPHAGO - COMPETENCE? 69
board 19x19 —> 18x18 ??
GO: AlphaGo against Lee Sedol, 2016 THE “ROBOTIC HYPE” 70 BACK TO “SOCIAL ROBOTICS” - ROBOLOUNGE PROJECT
▸ fulfill expectations of public, deliver on promises
▸ functionality beyond speech and facial expression
▸ comfortable, engagin, exciting interaction, high “coolness factor”
NOT: REPLACING HUMANS, BUT ENHANCING EXPERIENCE THE “ROBOTIC HYPE” 71 THE ROBOLOUNGE PROJECT
Design and make available a space where people can experience the future (not only talk about it)
The “Robolounge”
… Experience a touch of tomorrow !
socialsocial experimentexperiment -- publicpublic windowwindow -- communitycommunity
72 Experience the future !
73 VIDEO: “HOW WILL IT BE …”? THE “ROBOTIC HYPE” 75 ROBOLOUNGE ROBOTS receptionist bartender security (“bouncer”)
“Drone waiters”
… THE “ROBOTIC HYPE” 76 ROBOLOUNGE ROBOTS receptionist bartender security (“bouncer”)
“Drone waiters”
… THE “ROBOTIC HYPE” 77 WHY A “BARTENDER” ROBOT ?
… THE ROBOTIC/AI HYPE 78 THE “BARTENDER” ROBOT
▸ needed for the “Robolounge”
▸ sophisticated sensory-motor and social communication skills required
▸ skills transferable to many other domains
▸ great markets
▸ it’s fun!! THE ROBOTIC/AI HYPE 79 THE ROBOT BARTENDER - CAPABILITIES
▸ customer presence recognition, visual contact
▸ voice recognition, limited speech recognition
▸ understand order, move to beer storage, fetch bottle, pour desired beer
▸ info about specials, drinks, offering humorous comments and jokes
▸ handle payment THE ROBOTIC/AI HYPE 80 THE ROBOT “BARTENDER”: WHY WILL IT WORK ?
▸ embodiment
▸ soft robotics
▸ scaffolding
MUST EXPLOIT THESE FACTORS ! THE ROBOTIC/AI HYPE 81 THE ROBOT “BARTENDER”: WHY WILL IT WORK ?
▸ embodiment
▸ soft robotics
▸ scaffolding
MUST EXPLOIT THESE FACTORS ! EMBODIED INTELLIGENCE 82
“I THINK — THEREFORE I AM”
René Descartes Discours de la Méthode, 1637 83 Lego Mindstorms “creatures”
“Crazy Bird” “Crazy Bird” 84
loosely hanging feet random component rubber/plastic
Design and construction : Mike Rinderknecht Physical embedding (“embodiment”) 85 only brain (control): not sufficient must unterstand: - embedding of brain into organism - morphology (shape) and material chracteristics of robot - environment The “Passive Dynamic Walker” 86
Design and construction : Andy Ruina, Martijn Wisse, Steve Collins - Cornell University, Ithaca, New York The “Passive Dynamic Walker” 87
passive structure exploiting embodiment “cheap design”
Design and construction : Andy Ruina, Martijn Wisse, Steve Collins - Cornell University, Ithaca, New York The “Passive Dynamic Walker” 88
passive structure exploiting embodiment “cheap design”
trade-off: narrow ecological niche
Design and construction : Andy Ruina, Martijn Wisse, Steve Collins - Cornell University, Ithaca, New York The “Passive Dynamic Walker” 89
Memory for walking? EMBODIED INTELLIGENCE 90
THE DANCING ROBOT “STUMPY”
Video : Max Lungarella, Raja Dravid - Dynamic Devices, Zurich EMBODIED INTELLIGENCE 91
THE DANCING ROBOT “STUMPY”
EMERGENCE lifting feet off the ground (“must understand physical dynamics”)
Video : Max Lungarella, Raja Dravid - Dynamic Devices, Zurich EMBODIED INTELLIGENCE 92
ROBOT “FROG”
Task distribution: brain (control) — body (morphology, materials) — environment
i.e.
no clear separation: control — hardware
—> “Soft Robotics” SOFT ROBOTICS - THE POWER OF MATERIALS 93 SOFT ROBOTICS : LEONARDO DA VINCI
Credit: Hiroshi Ishiguro and Minoru Asada, Osaka University, Japan SOFT ROBOTICS - THE POWER OF MATERIALS 94 SOFT ROBOTICS : LEONARDO DA VINCI EMERGENCE soft materials crucial for facial expression (“underactuated system”)
Credit: Hiroshi Ishiguro and Minoru Asada, Osaka University, Japan SOFT ROBOTICS - THE POWER OF MATERIALS 95
factory robots: humans:
hard materials: steel, plastic, 85 % soft materials ! electrical motors HUMANS: LARGELY “SOFT” THE ROBOTIC/AI HYPE 96
SOFT ROBOTICS : THE “POWER OF MATERIALS”
“soft” to touch
built from soft materials
“soft”, natural movements
“soft”, pleasant, safe interaction SOFT ROBOTICS - THE POWER OF MATERIALS 97
GRASPING: TRADITIONAL ROBOT
grasping small object SOFT ROBOTICS - THE POWER OF MATERIALS 98 THE “UNIVERSAL COFFEE-BALLOON GRIPPER”
balloon filled with balloon with ground ground coffee coffee vacuum pump grasping boject
credit: Jaeger/Lipson – Univ. of Chicago and Cornell Univ., New York SOFT ROBOTICS - THE POWER OF MATERIALS 99 THE “UNIVERSAL COFFEE-BALLOON GRIPPER” all objects with same control
adaptivity at periphery (not centrally controlled)
credit: Jaeger/Lipson – Univ. of Chicago and Cornell Univ., New York exploitation of material properties SOFT ROBOTICS - THE POWER OF MATERIALS 100 HOLDING A HARD OBJECT
deformable tissue
adaptivity at the periphery (not centrally controlled)
exploitiation of material properties SOFT ROBOTICS - THE POWER OF MATERIALS 101 GRASPING GLASS WITH THIMBLES - THE “THIMBLES EXPERIMENT” effortless
Try it ! impossible !
experience what materials do for you ! SOFT ROBOTICS - THE POWER OF MATERIALS 102 THE THIMBLES- EXPERIMENT
THE “POWER OF MATERIALS” SOFT ROBOTICS - THE POWER OF MATERIALS 103
THE “SOFT ROBOTICS GRIPPER” SOFT ROBOTICS - THE POWER OF MATERIALS 104
SOFT ROBOTIC ARM - FOR BARTENDER ROBOT SOFT ROBOTICS - THE POWER OF MATERIALS 105
EXPANSION OF DESIGN SPACE morphologies and materials (physical structure, sensory distribution, actuators) design decision: what to put into morphology, materials, control, environment? SOFT ROBOTICS - THEORETICAL CONSIDERATIONS 106
morphology control TRADE-OFFS guided dominent control materials self-organization dominant informational increased reliance on self-organization morphological computation computation
106 pure computer industrial Asimo Roboy Octopus “Frog” Universal Passive vesicles algorithm (running robot (and (compliant (soft (compliant, “coffee- Dyn. W. molecules algorithm) (centralized similar tendon- contin- pneumatic) balloon” (exploiting control) robots) driven) uous) gripper morph.) BEHAVIORAL PRINCIPLES FOR INTELLIGENT SYSTEMS 107 MORE TROUBLE
fundamental behavioral principle - induction of sensory stimulation through action (NOT: “input-processing-output”) - raw material for information processing of brain - learning about environment (cf. John Dewey: “The reflex arc concept in psychology”, 1896 !!) BEHAVIORAL PRINCIPLES FOR INTELLIGENT SYSTEMS 108 JOHN DEWEY, PHILOSOPHER, PSYCHOLOGIST, EDUCATOR, 1896
“We begin not with a sensory stimulus, but with a sensory-motor coordination […] In a certain sense it is the movement which is primary, and the sensation which is secondary, the movement of the body, head, and eye muscles determining the quality of what is experienced. In other words, the real beginning is with the act of seeing; it is looking, and not a sensation of light.”
‘The reflex arc concept in psychology,’ John Dewey, 1896 BEHAVIORAL PRINCIPLES FOR INTELLIGENT SYSTEMS 109 MORE TROUBLE: COMPLEXITY OF DESIGN SPACE
fundamental behavioral principle - induction of sensory stimulation through action (NOT: “input-processing-output”) - raw material for information processing of brain - learning about environment (cf. John Dewey: “The reflex arc concept in psychology”, 1896 !!)
induction of correlations: information structure information overlap between sensory channels BEHAVIORAL PRINCIPLES FOR INTELLIGENT SYSTEMS 110 PHYSICAL DYNAMICS AND INFORMATION STRUCTURE Essence: self-structuring of sensory data through physical interaction with world Note: - physical process — not computational - prerequisite for learning - predictions, expectations
Inspiration: John Dewey, 1896 (!) Merleau-Ponty, 1963 Bajcsy, 1963; Aloimonos, 1990; Ballard, 1991 Sporns, Edelman, and co-workers Thelen and Smith (developmental studies) BEHAVIORAL PRINCIPLES FOR INTELLIGENT SYSTEMS 111 NOT: INPUT-PROCESSING-OUTPUT (FOR HUMAN INTELLIGENCE)
—> sensory-motor contingencies, SMCs
SMCs: law-like relations between movements and associated changes in sensory stimultion; O’Regan and Noë: A sensorimotor account of vision and visual consciousness, 2001)
BEHAVIORAL PRINCIPLES FOR INTELLIGENT SYSTEMS 112 NOT: INPUT-PROCESSING-OUTPUT (FOR HUMAN INTELLIGENCE)
—> sensory-motor contingencies, SMCs
SMCs: law-like relations between movements and associated changes in sensory stimultion; O’Regan and Noë: A sensorimotor account of vision and visual consciousness, 2001)
dependence on: - morphology (sensor type and distribution), materials - action (emergent behavior) - environment (e.g. object) BEHAVIORAL PRINCIPLES FOR INTELLIGENT SYSTEMS 113 HOW IT ALL FITS TOGETHER
loosely swinging arm: arm/joints self-organize into preferred trajectory
exploration: - constrained (“soft”, biomechanical constraints - morphology, materials) - preferred trajectories: easy control, energy-efficient - generation of rich, useful patterns of sensory stimulation (SMCs) - induction of information structure - self-structuring (correlations in different sensory channels) - learning, cross-modal associations - predictions THE “ROBOLOUNGE” PROJECT 114 THE ROBOT “BARTENDER”: WHY WILL IT WORK ?
▸ embodiment
▸ soft robotics
▸ scaffolding
MUST EXPLOIT THESE FACTORS !
… THE “ROBOLOUNGE” PROJECT 115 STRUCTURING ENVIRONMENT - EXPLOITING CONSTRAINTS limited space movement on tracks predefined positions (customers, bottles, glasses) narrow context (conversation) THE “ROBOLOUNGE” PROJECT 116 STRUCTURING ENVIRONMENT - EXPLOITING CONSTRAINTS limited space movement on tracks predefined positions DON’T NEED UNIVERSAL WAITER ROBOT ! (customers, bottles, glasses) narrow context (conversation) Experience the future !
See you in the Robolounge in the near future !
117 THE “ROBOLOUNGE” PROJECT 118 BEWARE: ROBOT TECHNOLOGY IS COMPLEX - SLOW ROI THE ROBOTIC/AI HYPE
Thank you for your attention !