Design Industrial Robotics Applications with MATLAB and Simulink
Presenter Name Here Trends in Industrial Robotics
Cobots Grow fastest in shipment terms with CAGR of 20% from 2017 - 2023
Growing trend toward compact robots Increasing share of units shipped in 2023 will be payload <10kg 40% of 80% of 82% of Articulated Robots SCARA Robots Cobots Source: Interact Analysis 2 Evolution of Industrial Robotics Technologies
Full Autonomy AI-enabled robots High Autonomy
Conditional Autonomy
Partial Autonomy Cobot UR5 1st Cobot at Linatex, 2008 1961
Human Assistant Traditional Industrial Robots - Automation
Unimate at GM, 1961
No Automation 1961 2008 2019-
3 Autonomous Industrial Robotics Systems Workflow Operates independently, without explicit instructions from a human
Connect/Deploy Code Generation ROS
Autonomous Algorithms
Sense (Observe) Perceive (Orient) Plan (Decide) Control (Act)
Encoder Camera Environment Localization Feedback Controls Understanding Force/Torque Contact Path Planning Decision Logic Sensor Switches Object Gain Scheduling Proximity Sensors Detection/Tracking Obstacle Avoidance Trajectory Control
S1 S3 S2
Platform Prototype HW Physical Model Actuation Model Environ Model Production HW
4 What we’ll discuss today
01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks
5 What we’ll discuss today
01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks
6 Traditional Software Development Cycle
Only 6% Of Design/Development Design Concept a Time is Spent on Simulation* * AspenCore - EETimes, “2019 embedded markets study,” EETimes, Tech. Rep., 2019
b Handwritten Code
Embedded System c
7 Common Challenges of Industrial Robotics Systems Development
Multidomain End-to-End Expertise workflows
Technical Depth Complexity of and System Algorithms Stability
8 Key Takeaways
In this talk, you will learn
Reference workflow for industrial robot development
Multi-domain functional areas of Platform, Sensing, Perception, Planning and Control
MATLAB and Simulink capabilities to develop new robot algorithms
» Kinematic and dynamic models of robots » Perception algorithm design using deep learning » Gazebo co-simulation for sensor models and environment simulation » Path planning with obstacle avoidance » Supervisory logic and control using Stateflow / RL » C/C++ code / ROS nodes generation
9 What we’ll discuss today
01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks Key to developing robust autonomous system
Complete Model-Based Design Workflow
Modeling & Test & Simulation Verification Need: An end-to-end development solution that includes modeling & simulation, code generation and test & verification.
Code Generation Simulate First and Simulate Often!
11 Full Model-Based Design Workflow Connect / Deploy Connect /Deploy
Autonomous Algorithms for Manipulators Plan & Perceive Decide
Sense Control
Platform Platform
12 What we’ll discuss today
01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks
13 Pick-and-Place Manipulators Robot Arm Demo Model-Based Design Deep Learning for detecting an object Path planning with collision avoidance
14 Mechanical Modeling Automatic import from CAD Tools
CAD Model Platform
Sense
Perceive
Plan & Connect & Deploy & Connect Decide Simscape Multibody Control Model
15 Evaluating motor requirements Actuators Connect & Deploy Perceive Control Plan & Sense Platform Decide – actuator sizing Robotics Simscape Toolbox System 16 Robotics System Environment Modeling Toolbox
Connect to an external robotics simulator ROS Toolbox
Robot arm simulation with Gazebo
Gazebo: Physics-based simulator with sensors and noise
17 Sensing Computer Vision Point cloud processing for pose estimation Toolbox
Platform
Sense
Colorized point cloud Detect table Intel® RealSense™ Perceive RGB-D camera
Plan & Connect & Deploy & Connect Decide
Control Point clouds of objects Remove noise and cluster
18 Computer Vision Sensing Toolbox Common sensors and sensing functionalities for autonomous systems Image Processing Toolbox
Platform • Support for Common Sensors
Sense • Image analysis
• Image enhancement Perceive • Visualizing Point Clouds
Plan & • Apps Connect & Deploy & Connect Decide
Control
19 Perception Deep learning for object classification Deep Learning Toolbox
Platform Object detector using Deep Learning (YOLO v2) Sense
• Perceive • •
Plan & Connect & Deploy & Connect Decide
Control
20 Perception Object Classification
Platform
Sense
Perceive
Plan & Connect & Deploy & Connect Decide
Control
21 Motion Planning Initial Pose X(t0) Final Pose X(tf) Motion Joint Trajectories Joint Limits Planner Q(t) Obstacles
Platform Joint positions
Follow & track trajectory Sense Reach waypoints
Interact with environment Perceive X(t0) q(t0) Manage gripper actions
Plan & Connect & Deploy & Connect Decide q(t1) q(t3) X(tf) Control q(t2) q(tf)
22 Robotics System Motion Planning Toolbox Model Predictive Path Planning + Trajectory Gen + Trajectory Following Control Toolbox
Platform
Sense
Perceive
Plan & Connect & Deploy & Connect Decide
Control
23 Motion Control
Decision Logic Stateflow
Platform
Sense Supervisory Control
Perceive
Plan & Connect & Deploy & Connect Decide
Control
24 Advanced Control: Reinforcement Learning Grasping an object with image inputs Reinforcement Learning Toolbox
AGENT ACTION STATE Policy
Policy update Reinforceme nt Learning Algorithm
REWARD
ENVIRONMENT
25 Hardware Connectivity
Code Generation Support ROS Toolbox
Platform Application Sense
Sense1 Sense2 Simulator Perceive
Plan &
Connect & Deploy & Connect Middleware Decide
Control Act1 HMI1 HMI2
26 Hardware Connectivity Code Generation Support Stateflow
Use publisher/subscriber Platform capabilities in Simulink to connect to ROS topics
Sense Jetson Xavier (Ubuntu 18.04)
Perceive (Bouncy)
Node2 Node1 User Application Ethernet Kinova
Plan & System Connect & Deploy & Connect Node3 Object Decide Kortex API Simulink to ROS2 User Application stand-alone node deployment Allows multi-thread, Control multi-core and pseudo real-time
27 Use the same reference workflow For warehouse pick-and-place (storage shelf)
This workflow example highlights the use of Robotics System Toolbox collision-checking algorithms, nonlinear MPC, and Stateflow for MATLAB
28 Use the same reference workflow For warehouse pick-and-place (storage shelf)
With obstacle avoidance ON (obstacle shown in blue)
With obstacle avoidance OFF (obstacle shown in black for reference)
29 Use the same reference workflow For Delta robot for automated parts sorting Connect
Sense
Perceive
Plan & Control Platform Decide What we’ll discuss today
01 Challenges in Industry Robot System Design 02 Model-Based Design for Autonomous System Case Study: Pick-and- 03 Place Manipulation Application 04 Concluding Remarks
31 Full Model-Based Design Workflow Connect / Deploy Connect /Deploy
ROS Toolbox Autonomous Algorithms
Sensor Fusion and Tracking Tbx Plan & Perceive Navigation Deep Learning Decide Toolbox Toolbox
Robotics System Toolbox
Stateflow
Computer Vision Reinforcement Toolbox Sense Control Learning Toolbox Model Predictive Control Toolbox
Platform Platform MATLAB / Simulink Simscape
32 Resources to get started with
33 Concluding Remarks
Challenges Develop Software End-to-end workflow in industrial robot with Model-Based for industry robot application Design applications development development
Multi-domain Fast Iterations Platform Expertise
Complexity of Sense Algorithms Perceive End-to-End Strong Focus workflows on Simulation Plan & Decide Technical Depth and System Stability Control
34 % Thank you!
mathworks.com/robotics
35