Subproject 10: Neurorobotics
Total Page:16
File Type:pdf, Size:1020Kb
Co-funded by the European Union Subproject 10: Neurorobotics Deliverable D10.7.2 Revised Report, 6 Oct 2017 D10.7.2 (D60.2 D25, SGA1 M12) ACCEPTED 20171108.docx PU = Public 08-Nov-2017 Page 1 of 124 Co-funded by the European Union Project Number: 720270 Project Title: Human Brain Project SGA1 Document Title: D10.7.2 SP10 Neurorobotics Platform - Results for SGA1 Period 1 Document Filename: D10.7.2 (D60.2 D25, SGA1 M12) ACCEPTED 20171108.docx Deliverable Number: SGA1 D10.7.2 Deliverable Type: Report Work Package(s): All WPs in SP10 Dissemination Level: PU = Public Planned Delivery SGA1 M12 / 31 Mar 2017 Date: Submitted: 31 May 2017 (M14), Resubmitted 6 Oct 2017 (M19), Accepted 08 Nov Actual Delivery Date: 2017 Authors: Alois KNOLL, TUM (P56), Marc-Oliver Gewaltig, EPFL (P1) Compiling Editors: Florian RÖHRBEIN, TUM (P56) Contributors: All Task leaders and their representatives SciTechCoord Re- EPFL (P1): Jeff MULLER, Martin TELEFONT, Marie-Elisabeth COLIN view: UHEI (P47): Martina SCHMALHOLZ, Sabine SCHNEIDER Editorial Review: EPFL (P1): Guy WILLIS, Colin McKINNON Abstract: This document describes the progress made in the first half of the first SGA. Keywords: Neurorobotics, virtual robotics, in silico experiments D10.7.2 (D60.2 D25, SGA1 M12) ACCEPTED 20171108.docx PU = Public 08-Nov-2017 Page 2 of 124 Co-funded by the European Union Table of Contents 1. Preamble ........................................................................................................ 8 2. SP Leader’s Overview......................................................................................... 8 2.1 Key Personnel ............................................................................................... 8 2.2 Overview..................................................................................................... 8 2.3 Progress M1-M12 ........................................................................................... 10 2.4 Challenges, consequences, and priorities ............................................................ 12 3. WP10.1. Closed-Loop Experiments (Data-Driven Brain Models) ................................... 13 3.1 Key Personnel .............................................................................................. 13 3.2 WP Leader’s Overview ................................................................................... 13 3.3 Milestones .................................................................................................. 14 3.4 T10.1.1 Locomotion and posture ....................................................................... 15 3.5 T10.1.2 Sensory-motor integration .................................................................... 20 3.6 T10.1.3 Corticospinal Integration ...................................................................... 26 3.7 T10.1.4 Cerebellar Motor Control ...................................................................... 27 3.8 T10.1.5 Sensory-guided neuromotor control ......................................................... 28 3.9 T10.1.6 Simulation of motor rehabilitation experiment in rodents .............................. 30 3.10 T10.1.7 WP Lead .......................................................................................... 35 4. WP 10.2. Closed-Loop Experiments (Functional / Control Models) ............................... 36 4.1 Key Personnel .............................................................................................. 36 4.2 WP Leader’s Overview ................................................................................... 36 4.3 Priorities for the remainder of the phase ............................................................ 36 4.4 Milestones .................................................................................................. 37 4.5 10.2.1 Early sensory processing ........................................................................ 37 4.6 T10.2.2 Behaviour generation .......................................................................... 41 4.7 T10.2.3 Learning body and movement representations for grasping and manipulation ...... 44 4.8 T10.2.4 WP Lead .......................................................................................... 55 5. WP10.3 Components of Closed-Loop Experiments .................................................... 56 5.1 Key Personnel .............................................................................................. 56 5.2 WP Leader’s Overview ................................................................................... 56 5.3 Priorities for the remainder of SGA1 .................................................................. 56 5.4 Milestones .................................................................................................. 57 5.5 T10.3.1 Rodent Body Model ............................................................................. 57 5.6 T10.3.2 Musculoskeletal models of rodents .......................................................... 58 5.7 T10.3.3 Models of robots, sensors and environments .............................................. 61 5.8 T10.3.4 Benchmarking and validation of NR models ................................................ 66 5.9 T10.3.5 WP Lead .......................................................................................... 67 6. WP10.4 Translational Neurorobotics ..................................................................... 68 6.1 Key Personnel .............................................................................................. 68 6.2 WP Leader’s Overview ................................................................................... 68 6.3 Priorities for the remainder of the phase ............................................................ 68 6.4 Milestones .................................................................................................. 70 6.5 T10.4.1 Design and control of muscolo-skeletal robots for the NRP ............................. 70 6.6 T10.4.2 NRP and motor-actuated robots (iCub) ..................................................... 72 6.7 T10.4.3 NRP and Neuromorphic Hardware ........................................................... 73 6.8 T10.4.4 Self-adaption in modular robots ............................................................. 76 6.9 T10.4.5 Real-time control with reservoir networks ................................................. 80 6.10 T10.4.6 WP Lead .......................................................................................... 83 7. WP 10.5 Simulation and Visualisation Tools for Neurorobotics .................................... 84 D10.7.2 (D60.2 D25, SGA1 M12) ACCEPTED 20171108.docx PU = Public 08-Nov-2017 Page 3 of 124 Co-funded by the European Union 7.1 Key Personnel .............................................................................................. 84 7.2 WP Leader’s Overview ................................................................................... 84 7.3 Priorities for the remainder of the phase ............................................................ 85 7.4 T10.5.1 Simulation of physics (mechanics, light, sound, etc.) ................................... 87 7.5 T10.5.2 World Simulation and Closed-loop engine .................................................. 89 7.6 T10.5.3 NRP User Experience (NRP Cockpit) ......................................................... 91 7.7 T10.5.4 Environment and experiment designer ..................................................... 93 7.8 T10.5.5 Robot Designer .................................................................................. 95 7.9 T10.5.6 Brain-Body Integrator .......................................................................... 97 7.10 T10.5.7 Virtual Coach .................................................................................... 98 7.11 T10.5.8 Benchmarking and validation of physics and light simulation ......................... 100 7.12 T10.5.9 Software integration, packaging and release ............................................. 103 7.13 T10.5.10 WP Lead ........................................................................................ 105 8. WP10.6 Neurorobotics Platform ......................................................................... 106 8.1 Key Personnel ............................................................................................. 106 8.2 WP Leader’s Overview .................................................................................. 106 8.3 Priorities for the remainder of the phase ........................................................... 106 8.4 Milestones ................................................................................................. 106 8.5 T10.6.1 Platform integration, deployment and operation ........................................ 106 8.6 T10.6.2 Platform testing, profiling and quality assurance ....................................... 108 8.7 T10.6.3 Documentation, user support and user training .......................................... 113 8.8 T10.6.4 WP Lead ......................................................................................... 115 9. WP10.7 Scientific Coordination and Community Outreach ....................................... 117 9.1 Key Personnel ............................................................................................. 117 9.2 WP Leader’s Overview .................................................................................. 117 9.3 Milestones ................................................................................................