Design Industrial Robotics Applications with MATLAB and Simulink Presenter Name Here Trends in Industrial Robotics Cobots Grow fastest in shipment terms with CAGR of 20% from 2017 - 2023 Growing trend toward compact robots Increasing share of units shipped in 2023 will be payload <10kg 40% of 80% of 82% of Articulated Robots SCARA Robots Cobots Source: Interact Analysis 2 Evolution of Industrial Robotics Technologies Full Autonomy AI-enabled robots High Autonomy Conditional Autonomy Partial Autonomy Cobot UR5 1st Cobot at Linatex, 2008 1961 Human Assistant Traditional Industrial Robots - Automation Unimate at GM, 1961 No Automation 1961 2008 2019- 3 Autonomous Industrial Robotics Systems Workflow Operates independently, without explicit instructions from a human Connect/Deploy Code Generation ROS Autonomous Algorithms Sense (Observe) Perceive (Orient) Plan (Decide) Control (Act) Encoder Camera Environment Localization Feedback Controls Understanding Force/Torque Contact Path Planning Decision Logic Sensor Switches Object Gain Scheduling Proximity Sensors Detection/Tracking Obstacle Avoidance Trajectory Control S1 S3 S2 Platform Prototype HW Physical Model Actuation Model Environ Model Production HW 4 What we’ll discuss today 01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks 5 What we’ll discuss today 01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks 6 Traditional Software Development Cycle Only 6% Of Design/Development Design Concept a Time is Spent on Simulation* * AspenCore - EETimes, “2019 embedded markets study,” EETimes, Tech. Rep., 2019 b Handwritten Code Embedded System c 7 Common Challenges of Industrial Robotics Systems Development Multidomain End-to-End Expertise workflows Technical Depth Complexity of and System Algorithms Stability 8 Key Takeaways In this talk, you will learn Reference workflow for industrial robot development Multi-domain functional areas of Platform, Sensing, Perception, Planning and Control MATLAB and Simulink capabilities to develop new robot algorithms » Kinematic and dynamic models of robots » Perception algorithm design using deep learning » Gazebo co-simulation for sensor models and environment simulation » Path planning with obstacle avoidance » Supervisory logic and control using Stateflow / RL » C/C++ code / ROS nodes generation 9 What we’ll discuss today 01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks Key to developing robust autonomous system Complete Model-Based Design Workflow Modeling & Test & Simulation Verification Need: An end-to-end development solution that includes modeling & simulation, code generation and test & verification. Code Generation Simulate First and Simulate Often! 11 Full Model-Based Design Workflow Connect / Deploy Connect /Deploy Autonomous Algorithms for Manipulators Plan & Perceive Decide Sense Control Platform Platform 12 What we’ll discuss today 01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks 13 Pick-and-Place Manipulators Robot Arm Demo Model-Based Design Deep Learning for detecting an object Path planning with collision avoidance 14 Mechanical Modeling Automatic import from CAD Tools CAD Model Platform Sense Perceive Plan & Connect & DeployConnect Decide Simscape Multibody Control Model 15 Actuators Evaluating motor requirements – actuator sizing Platform Sense Perceive Plan & Connect & DeployConnect Decide Robotics System Toolbox Control Simscape 16 Robotics System Environment Modeling Toolbox Connect to an external robotics simulator ROS Toolbox Robot arm simulation with Gazebo Gazebo: Physics-based simulator with sensors and noise 17 Sensing Computer Vision Point cloud processing for pose estimation Toolbox Platform Sense Colorized point cloud Detect table Intel® RealSense™ Perceive RGB-D camera Plan & Connect & DeployConnect Decide Control Point clouds of objects Remove noise and cluster 18 Computer Vision Sensing Toolbox Common sensors and sensing functionalities for autonomous systems Image Processing Toolbox Platform • Support for Common Sensors Sense • Image analysis • Image enhancement Perceive • Visualizing Point Clouds Plan & • Apps Connect & DeployConnect Decide Control 19 Perception Deep learning for object classification Deep Learning Toolbox Platform Object detector using Deep Learning (YOLO v2) Sense • Perceive • • Plan & Connect & DeployConnect Decide Control 20 Perception Object Classification Platform Sense Perceive Plan & Connect & DeployConnect Decide Control 21 Motion Planning Initial Pose X(t0) Final Pose X(tf) Motion Joint Trajectories Joint Limits Planner Q(t) Obstacles Platform Joint positions Follow & track trajectory Sense Reach waypoints Interact with environment Perceive X(t0) q(t0) Manage gripper actions Plan & Connect & DeployConnect Decide q(t1) q(t3) X(tf) Control q(t2) q(tf) 22 Robotics System Motion Planning Toolbox Model Predictive Path Planning + Trajectory Gen + Trajectory Following Control Toolbox Platform Sense Perceive Plan & Connect & DeployConnect Decide Control 23 Motion Control Decision Logic Stateflow Platform Sense Supervisory Control Perceive Plan & Connect & DeployConnect Decide Control 24 Advanced Control: Reinforcement Learning Grasping an object with image inputs Reinforcement Learning Toolbox AGENT ACTION STATE Policy Policy update Reinforceme nt Learning Algorithm REWARD ENVIRONMENT 25 Hardware Connectivity Code Generation Support ROS Toolbox Platform Application Sense Sense1 Sense2 Simulator Perceive Plan & Connect & DeployConnect Middleware Decide Control Act1 HMI1 HMI2 26 Hardware Connectivity Code Generation Support Stateflow Use publisher/subscriber Platform capabilities in Simulink to connect to ROS topics Sense Jetson Xavier (Ubuntu 18.04) Perceive (Bouncy) Node2 Node1 User Application Ethernet Kinova Plan & System Connect & DeployConnect Node3 Object Decide Kortex API Simulink to ROS2 User Application stand-alone node deployment Allows multi-thread, Control multi-core and pseudo real-time 27 Use the same reference workflow For warehouse pick-and-place (storage shelf) This workflow example highlights the use of Robotics System Toolbox collision-checking algorithms, nonlinear MPC, and Stateflow for MATLAB 28 Use the same reference workflow For warehouse pick-and-place (storage shelf) With obstacle avoidance ON (obstacle shown in blue) With obstacle avoidance OFF (obstacle shown in black for reference) 29 Use the same reference workflow For Delta robot for automated parts sorting Connect Sense Perceive Plan & Control Platform Decide What we’ll discuss today 01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks 31 Full Model-Based Design Workflow Connect / Deploy Connect /Deploy ROS Toolbox Autonomous Algorithms Sensor Fusion and Tracking Tbx Plan & Perceive Navigation Deep Learning Decide Toolbox Toolbox Robotics System Toolbox Stateflow Computer Vision Reinforcement Toolbox Sense Control Learning Toolbox Model Predictive Control Toolbox Platform Platform MATLAB / Simulink Simscape 32 Resources to get started with 33 Concluding Remarks Challenges Develop Software End-to-end workflow in industrial robot with Model-Based for industry robot application Design applications development development Multi-domain Fast Iterations Platform Expertise Complexity of Sense Algorithms Perceive End-to-End Strong Focus workflows on Simulation Plan & Decide Technical Depth and System Stability Control 34 % Thank you! mathworks.com/robotics 35.