School of Innovation, design and Engineering Master Thesis in Intelligent Embedded Systems 15Credit advanced level Automated Orchestra for Industrial Automation On Virtualized Multicore Environment Author: Nesredin M.Mahmud ABSTRACT Industrial control systems are applied in many areas e.g., motion control for industrial robotics, process control of large plants such as in the area of oil and gas, and in large national power grids. Since the last decade with advancement and adoption of virtualization and multicore technology (e.g., Virtual Monitoring Machine, cloud computing, server virtualization, application virtualization), IT systems, automation industries have benefited from low investment, effective system management and high service availability. However, virtualization and multicore technologies have posed a serious challenge to real-time systems, which is violating timeliness and predictability of real-time application running on control systems. To address the challenge, we have extended a real-time component-based framework with virtual nodes; and evaluated the framework in the context of virtualized multicore environment. The evaluation is demonstrated by modeling and implementing an orchestra application with QoS for CPU, memory and network bandwidth. The orchestra application is a real-time and distributed application deployed on virtualized multicore PCs connected with speakers. The result shows undistorted orchestra performance played through speakers connected to physical computer nodes. The contribution of the thesis can be considered: 1) extending a real-time component-based framework, Future Automation Software Architecture (FASA) with virtual nodes using Virtual Computation Resource (VCR) and 2) design and installation of reusable test environment for development, debugging and testing of real-time application on a network of virtualized multicore environment. Date: 13 September 2013 Carried out at: ABB Corporate Research, Västerås Advisor at MDH: Moris Behnam Advisor at ABB Corporate Research: Kristian Sandström Examiner: Thomas Nolte ii Acknowledgement I would like to thank my supervisor Kristian Sandström and examiner professor Thomas Nolte for giving me this interesting thesis opportunity. I would like to appreciate Kristian Sandström once more and Aneta Vulgarakis for their invaluable guidance and assistance during my thesis work. Veronika, I admire your willingness to help me solve technical problems. Furthermore, I would like to express my deep gratitude to my supervisor Moris Behnam for his encouragement and loving approach; and also taking the time to read my report. Special thanks to my examiner Thomas Nolte for understanding my enthusiasm to do the work and recommend me to this position. I would like to thank my family for cherishing me moral support. Finally, thank you to ABB and colleagues – it was fun and exciting! iii NOMENCLATURE Abbreviation ICS Industrial Control System FASA Future Automation Software Architecture DCS Distributed Control Systems RTOS Real-Time Operating System VCR Virtual Computational Resource VCPU Virtual CPU VMM Virtual Machine Monitor IEC International Electrotechnical Commission HVM Hardware-assisted Virtual Machine PV ParaVirtualization Dom0 Domain 0 – to show guest machine runs at privilege mode (ring 0) DomU Domain U – to show guest machine runs at unprivileged mode SDN Software-defined Networking SDLC Software Development Life Cycle IOMMU Input/Output Memory Management Unit PVHVM ParaVirtualized HVM PVOPS ParaVirualized Options (paravirt-ops) 4DIAC Framework for Distributed Industrial Automation and Control BVT Borrowed Virtual Time BW Bandwidth pCPU Physical CPU vCPU Virtual CPU SEDF Simple Earlier Deadline First VNC Virtual Network Computing SPICE Simple Protocol for Independent Computing Environment PECOS PErvasive Component Systems VIF Virtual InterFace TCF Target Communication Framework SSH Secure Shell iv CONTENTS Chapter 1 INTRODUCTION 7 1.1 Introduction ........................................................................................................ 7 1.2 Related work ....................................................................................................... 8 1.3 Problem formulation .......................................................................................... 9 1.4 The contribution of the thesis is two folds ........................................................ 10 Extending FASA real-time component-based framework to virtual nodes ......................... 10 Design Reusable Test Environment ....................................................................................... 10 1.5 Analysis of the problem ..................................................................................... 11 Setting up of Virtualized Multicore Environment .................................................................. 11 Development of Automated Orchestra Application .............................................................. 12 Chapter 2 BACKGROUND 14 2.1 Theoretical Background .................................................................................... 14 Component-Based Real-time Software Development (CBSD) ............................................. 14 Future Automation Software Architecture (FASA) ............................................................... 15 Virtualization .......................................................................................................................... 17 2.2 Technological Background ................................................................................ 18 Xen Hypervisor (Xen pronounced /’zɛn/) ............................................................................. 18 Open vSwitch .......................................................................................................................... 19 4DIAC-IDE .............................................................................................................................. 20 Chapter 3 Development of Orchestra components 21 3.1 Component development mechanism .............................................................. 21 3.2 Orchestra organization ..................................................................................... 22 3.3 Define requirements ......................................................................................... 22 3.4 Distributed orchestra application overview ..................................................... 25 3.5 Component modelling ...................................................................................... 26 3.6 Template generated by 4diac-fasa plug-in ....................................................... 27 Application directory .............................................................................................................. 27 Library directory ..................................................................................................................... 27 Module directory..................................................................................................................... 28 3.7 Standard Midi file format and partitioning ...................................................... 28 Standard MIDI file format ..................................................................................................... 28 Software tools and libraries .................................................................................................... 29 Midi file partitioning and rhythm message ........................................................................... 30 3.8 Algorithm Implementation for functional blocks ............................................. 31 Conductor Functional Blocks implementation...................................................................... 31 Principal Functional Block implementation .......................................................................... 31 Musician Functional Block implementation ......................................................................... 32 Chapter 4 Deployment of Orchestra Application 33 4.1 Setting up - Network of Virtualized Multicore environment ............................ 33 Host installation: Ubuntu-12.04 LTS .................................................................................... 33 Xen hypervisor installation .................................................................................................... 34 Open vSwitch installation ....................................................................................................... 34 VM installation: yocto VM, Ubuntu-12.04 LTS headless ...................................................... 34 4.2 Define Virtual computation resource ............................................................... 36 VCR with QoS for CPU, memory and network BW ............................................................... 36 Application Schedule and Execution ..................................................................................... 38 v 4.3 Orchestra Application Management ................................................................. 39 FASA component manager ..................................................................................................... 39 Libvirt and Virt-manager ....................................................................................................... 39 Orchestra distribute management script ............................................................................... 39 Chapter 5 CONCLUSION 40 5.1 Result