ROS(機器人作業系統) on Windows 及Azure AI的應用案例分享 Tommy Wu 微軟資深物聯網解決方案架構師 Agenda ✓ ROS (Robotics ) ✓ ROS on Windows ✓ ROS with Azure AIoT & Edge ✓ Reference Case Robotics with ROS The Operating System (ROS) is a set of software libraries and tools that help you build robot applications. From drivers to state-of-the-art algorithms, and with powerful developer tools, ROS has what you need for your next robotics project. And it's all open source. ROS Core

Topic /chatter

subscribe subscribe

publish

talker.py listener.cpp snoop.cpp

ROS Node ROS Node ROS Node Sensors

Insights Actions

ROS on Windows Announced an experimental release of ROS1 for Windows at ROSCon, 2018.

Completed the core porting effort, working on upstreaming

Bringing the power of Actively enabling more ROS to ROS packages on Windows Bringing advanced features like hardware-accelerated Windows AI Platform + Azure cloud technologies to industrial

http://aka.ms/ros Windows 10 IoT: now a supported platform for ROS

“We're excited to add Windows IoT as a supported platform for ROS (the Robot Operating System). The ROS developer community can now take advantage of a wide array features in Windows IoT, including hardware-accelerated machine learning, computer vision, and cloud capabilities such as Azure Cognitive Services. I look forward to seeing the next generation of ROS applications that are enabled by Windows IoT support.”

Brian Gerkey, Bringing the power of Windows 10 IoT to ROS

“ has been a valued addition to the ROS-I Consortium. Our membership has expressed significant interest in developments that will bring the advanced capabilities of ROS to industrial applications on the Windows platform and enable richer integration with other Microsoft tools that the Consortium membership have come to leverage.”

Matthew M. Robinson, ROS Community Nodes

Mobility ROS Core Manipulation

Microsoft Nodes Azure Enabling the Visual Studio Code extension for ROS to work with ROS on Windows

Support automatic environment configuration for ROS development

Streamline launching, terminating and monitoring ROS runtime status

Execute rosrun and roslaunch commands interactively

URDF real-time rendering

New Update

Support debugging Python/C++ in ROS environment Windows Machine Learning ROS with Azure AIoT & Edge Development machine

Running ROS on Windows natively Running ROS via Windows Subsystem for Running ROS on Linux for the supported ROS packages for the non-supported ROS packages

Running ROS on Windows • Accelerated graphics and physics • Supported for non-UI & non-media packages • Works with ROS on • Projection of Windows specific • Recommended for proprietary solutions like Windows features Navigation • Azure packages supported • Enterprise security and manageability support Azure IoT Edge accomplishes safe deployment of code through the use of containerized modules.Leveraging Azure IoT Edge and Cognitive Services Containers together, we can build out IoT Solutions which allow for local AI processing in environments where internet connectivity may be intermittent.

General availability of IoT Edge support for Windows 10 IoT Enterprise x64 With Azure Speech Service, control the robot using your voice command

Train and push Azure AI model to publish the detecting object’s bounding box to the according topic. New Update

https://aka.ms/ros_azure_iothub What’s next? Azure ML integration

Azure Machine Learning Services Provide a complete end-to-end workflow for ML training, evaluation, and improvement.

Public Preview Public Preview

A specialized Windows includes Windows includes a 1GB 3.5GB Windows container its own inferencing top-notch media for ML workloads. stack: Windows ML. and audio stack.

Targeting under Windows ML 350MB on-disk size. Tensorflow Get started!

Using ROS on Windows Connect ROS to Azure IoT Hub

Getting started - https://aka.ms/ros_azure_iothub https://aka.ms/ros

Uses Chocolatey for ROS runtime and dependency distribution

ROS nodes are built in a Visual Studio command line or Visual Studio Code ROS Taiwan Community

ROS Taiwan 年會是一個開發者大會。繼過去一度的年會成功舉辦 之後，今年的ROS Taiwan 將在南港舉行ROS台灣社群的年會。與 去年年會類似，為期半天的年會將有ROS相關技術與機器人相關組 件講座分享，這包含了熱情的贊助商與社群成員的分享。 我們的目標是使ROS 的台灣社群繼續的茁壯，讓不論是在機器人 產業工作者，或是學研界的研究人員，或是純粹是喜歡機器人的愛 好者，都有一個互相交流與成長的地方。因此無論你是誰，無論你 做什麼，無論你做什麼，如果你對ROS感興趣，那麼我們希望你來 參與我們的ROS Taiwan 年會。 我們也歡迎提出建議，我們還可以做些什麼來鼓勵更多人參與。如 果您有想要分享的想法，請與我們聯繫。 FB: https://www.facebook.com/groups/ros.taiwan/ https://sites.google.com/view/rostwdevcon2019/首頁 https://ros-taipei.wixsite.com/2018 References Scenarios Scenario 1 – ROS on Windows 10 IoT with Hiwin customized ROS packages

• Vision package - image processing, industrial measurement and image classification from Basler's industrial camera sensors • Azure Speech services - listen and react to voice commands • Azure IoT Central - ability to manage multiple customers and robotics/devices (Hiwin hosted SaaS) Scenario 2 – Voice activated multi-axis articulated robotic arm and Robot speech teaching • Teaching-on mode – voice command recognize and execution • Language understanding – Interactive with operator by natural language Azure Robotics AI Suite with

-ROS -IoT Central -Custom Vision AI Robotics AIoT System Architecture on ROS

ROSCore

/inference_objects /STT Azure Custom

Vision

listen

listen

publish publish

Action azure_cs_luis robotcv Iot-central-app (Speech) Sets

Win10 ROS /cmd_vel Melodic Robotics AI Suite Features

 UI - Robot Dashboard for remote monitoring & control  Speech - Enabled Switch for Speech-to-Text feature  Vision - Object Recognition for customized training object  Action – Inference result to Robot Action conversion  System - Monitor the system software on Robot side.  Model – Quick Switch between AI models.  Analytics – Built-in Robot Data quick analytics UI- ROS Dashboard Application IoT Central Robotics Controller Azure Custom Vision for Model Training

 Object Detection Mode  Visual Labeling  Export for ROS Edge Module  Quick & Easy Training AI model ✓ Start Camera node to identify the toys ✓ Interactive with the robot by speech ✓ Talk to the robot to pick up the toy we want ✓ Start to find the path to the toy with vision inference ✓ Pick up back to me and release the toy. Thank you