Layered Implementation of Robotic Middlewares to the Physical Agent System IROS’06 Workshop WS4 Robotics Standardization Makoto MIZUKAWA, Ph.D. [email protected] Shibaura Institute of Technology, Tokyo Chair ORiN Forum/JARA, Project Leader ISO TC184/SC2 RAPI Contact of OMG Robotics-DTF to ISO TC184/SC2 Oct. 10, 2006, International Conference Beijing, China on Iintelligent Robots and Systems 1 Outline Needs for standard Environment Physical Agent System Layered Implementation of Middlewares ORiN for Robot Collaboration LwRTC for Embedded controller Future work 2 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Needs for standard Environment 3 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Robotics in future Hazardous task Unmanned Autonomy substitute in fields systems Manufacturing High speed Human Industrial robots work /precision Human-Robot-Interaction Embedded/ Service robots Ubiquitous robots symbiosis Entertainment robots HouseholdHousehold Pet/Pet/ healinghealing WorkWork assistassist DailyDaily assistassist New media Communication media RescueRescue MedicalMedical 4 Oct.Daily 10, 2006, lifeBeijing, support China International Conference on Intelligent Robots and Systems I’REX2005 DAEJEON 5 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Related Activities (examples) forfor TelecomTelecom forfor MfgMfg FIPAFIPA:The:TheFoundations Foundationsfor for IntellIntelligentigent ISOISO TC-1TC-18484,, JARAJARA PhysicalPhysical AgentsAgents RobRoboots,ts, DeviceDevice proproffiling.iling. middleware for application http://www.fipa.org/ middleware for application http://http://www.www.oorrin.jp/in.jp/ forfor IT+RTIT+RT servicesservices forfor controllers/OScontrollers/OS MPHPT:Ministry of Public Management, Home Affairs, Posts OROCOSOROCOS and Telecommunications, Japan SoftwSoftwaaresresfforor Robot controller http://Robwww.ot controrocoosller.org/ http://www.soumu.go.jp/english/index.html MIC: Ministry of Information and OMGOMG Communication, KOREA SDOSDO DAIS www.vcl.uh.edu/~rcv03/materials/slides/SangRok.ppt DAIS 6 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Simple Idea Robot is the object that has physical entity which interacts with the real world Collaboration with others through networking From service-applications to devices Tasks: Computers in the logical world vs. Robots in the real world Sensing and processing Interact with the real world Hard real-time issues in controllers Various structure and configuration Various architecture Physical & logical interoperability with other devices and robots 7 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Issues Interoperability Connectivity DomainDomain specificspecific DomainDomain independentindependent Modularity Reusability RT-middleware for robot services Keys:Keys: RT-component-service middleware for controllers RT-component standard interface 8 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems RT middleware-layers PART1 PART2 PART3 controller application application service level common API mediator Model, data-framework RTMW1 Robot RTMW2 Internal task level profiles NW Model, data-framework Vision Ctrl. Sensor Ctrl. Common controller Dev.Profile Dev.Profile motion level I/F Model, data-framework controller Body Ctrl. Arm Ctrl. Robot A Robot B Dev.Profile Dev.Profile motors arm camera devices sensors 9 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Physical Agent System 10 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Motivation From control issues to service issues Daily activity support and collaboration using remotely controlled robot •Operation based on information mainly through video cameras mounted on the remotely controlled robot •Gesture transmission to assist collaboration between remote sites (laser pointer/arm) Media to support Master-slave system equivalent/close relationship 11 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems PAS: Physical Agent System A remotely controlled autonomous robot that acts as an AGENT to support human collaboration over distant places PAS as media for Sharing spatial information between remote sites Transmitting intension/attention of operator via motion direction/orderdirection/order spatial/field info. spatial/field info. 12 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems PPhysicalhysical AAgentgent SSystemystem InputInput devices devices RemotRemotee site Voice control Daily life site PC support Joystick Agent robot Agent with Agent robot Agent with The Internet Info. Sharing Pad physicalphysical camera interactioninteraction Laser pointer •Distributed•Distributed control control PC arm •System•System description description •Image•Image processing processing Activity support Sensory Glove in hazardous environments autonomy WEB Simulator ・・mobilitymobility ・・obstacleobstacle avoi avoidancedance ・・postureposture stabili stabilizationzation camera,camera, LP LP 13 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Applications of PAS Disaster/Rescue Daily life support Elderly/Handicapped Communication within Family in distance Home-electronics・personal robots Distributed/Embedded PAS in Environments Reflects human intension Device profiling using XML to allow inter- operability and flexible configuration 14 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Prototype of Laser Pointer the PAR04R CCD Camera Gimbal Mechanism 5DOF Arm Crawler 15 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems PAS System Configuration < device > < device > < device > < device > < device > Display Keyboard Sensory Glove Joystick Mouse < processor > < on board> < processor > 「TCP/IP」 ･wireless LAN < on board> ･ Agent-Server ･PCI Client-Server input device (windows2000) ･IP7000 (windows2000) < management > < management > ･ Preservation of task and re-execution command ･ Simulator system Agent Robot ･ Communication with Client-Server(TCP/IP) Operation ･ Input control ･ Communication with Data-Server(TCP/IP) ･ Display the interface to an operator ･ Communication control of CAN bus CAN ･ Process of camera image (Controller Area Network)「CAN」 < device > < device > < device > < device > H8S for Power management H8S for Arm control H8S for Crawler control H8S for Camera/LP control Real-time OS Real-time OS Real-time OS Real-time OS < on board> < on board> < on board> < on board> ･ H8S ･ Arm ･ Machine ･ PSD ･ Camera ･ H8S ･ Ultrasonic Sensor ･ LP ･ H8S ･ H8S < management > < management > < management > < management > ･Task control ･Arm control ･Machine control ･Camera/LP control ･Get Current’s value ･Command in another device ･Get Sensor’s value ･Get Angle’s value ･Store Current/Voltage logs ･Command in another device ･Command in another device ･Command in another device 16 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Physical Agent Robot (PAS) PAR04R 17 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Aichi Expo 2005 Robot week, June 9-19 Oct. 10, 2006, International Conference Beijing, China on Iintelligent Robots and Systems 18 Robot Week (Jun 9-19) Aichi Expo Intelligent Wheel Echono-vehicle Chair, “Komawari-kun” Meijo Univ Apri-Alpha & the helper robot Apri-Attenda UEC Toshiba & SUT PAR04R Shibaura Inst. Tech. Collaboration using RT middleware-ORiN Component based on RT Middleware from AIST19 Oct. 10, 2006, Beijing, China InternationalApriAlpha Conference on using Intelligent ORCA Robots and Systems Layered Implementation of Middlewares ORiN for Robot Collaboration LwRTC for Embedded controller 20 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems RT middleware-layers PART1 PART2 PART3 controller application application service level common API mediator Model, data-framework RTMW1 Robot RTMW2 Internal task level profiles NW Model, data-framework Vision Ctrl. Sensor Ctrl. Common controller Dev.Profile Dev.Profile motion level I/F Model, data-framework controller Body Ctrl. Arm Ctrl. Robot A Robot B Dev.Profile Dev.Profile motors arm camera devices sensors 21 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems Toshiba robot Voice operation operator projector visitors Robot profiles XML(CRD) robot control & management system Robot status management robot status storage & display system monitor and refer robot status and location robot control & status acquisition using RAC cameras UEC robots Meijo Univ robots PAR04R 22 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems System integration using ORiN control data robot control & management system GUI Robot control application operation VB.NET command operator table CAO Engine CRD provider RAC provider Robot profiles RAC requests XML(CRD) Voice operation operator 23 Oct. 10, 2006, Beijing, China International Conference on IntelligentRobots Robots and Systems ORiN: Open Resource/Robot Interface for the Network Application ORiN Provider Robot Controller 24 Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems