School of Innovation, design and Engineering
Master Thesis in Intelligent Embedded Systems 15Credit advanced level
Automated Orchestra for Industrial Automation
On
Virtualized Multicore Environment
Author: Nesredin M.Mahmud
ABSTRACT
Industrial control systems are applied in many areas e.g., motion control for industrial robotics, process control of large plants such as in the area of oil and gas, and in large national power grids. Since the last decade with advancement and adoption of virtualization and multicore technology (e.g., Virtual Monitoring Machine, cloud computing, server virtualization, application virtualization), IT systems, automation industries have benefited from low investment, effective system management and high service availability. However, virtualization and multicore technologies have posed a serious challenge to real-time systems, which is violating timeliness and predictability of real-time application running on control systems. To address the challenge, we have extended a real-time component-based framework with virtual nodes; and evaluated the framework in the context of virtualized multicore environment. The evaluation is demonstrated by modeling and implementing an orchestra application with QoS for CPU, memory and network bandwidth. The orchestra application is a real-time and distributed application deployed on virtualized multicore PCs connected with speakers. The result shows undistorted orchestra performance played through speakers connected to physical computer nodes. The contribution of the thesis can be considered: 1) extending a real-time component-based framework, Future Automation Software Architecture (FASA) with virtual nodes using Virtual Computation Resource (VCR) and 2) design and installation of reusable test environment for development, debugging and testing of real-time application on a network of virtualized multicore environment.
Date: 13 September 2013 Carried out at: ABB Corporate Research, Västerås Advisor at MDH: Moris Behnam Advisor at ABB Corporate Research: Kristian Sandström Examiner: Thomas Nolte
ii
Acknowledgement
I would like to thank my supervisor Kristian Sandström and examiner professor Thomas Nolte for giving me this interesting thesis opportunity. I would like to appreciate Kristian Sandström once more and Aneta Vulgarakis for their invaluable guidance and assistance during my thesis work. Veronika, I admire your willingness to help me solve technical problems. Furthermore, I would like to express my deep gratitude to my supervisor Moris Behnam for his encouragement and loving approach; and also taking the time to read my report. Special thanks to my examiner Thomas Nolte for understanding my enthusiasm to do the work and recommend me to this position. I would like to thank my family for cherishing me moral support. Finally, thank you to ABB and colleagues – it was fun and exciting!
iii
NOMENCLATURE
Abbreviation ICS Industrial Control System FASA Future Automation Software Architecture DCS Distributed Control Systems RTOS Real-Time Operating System VCR Virtual Computational Resource VCPU Virtual CPU VMM Virtual Machine Monitor IEC International Electrotechnical Commission HVM Hardware-assisted Virtual Machine PV ParaVirtualization Dom0 Domain 0 – to show guest machine runs at privilege mode (ring 0) DomU Domain U – to show guest machine runs at unprivileged mode SDN Software-defined Networking SDLC Software Development Life Cycle IOMMU Input/Output Memory Management Unit PVHVM ParaVirtualized HVM PVOPS ParaVirualized Options (paravirt-ops) 4DIAC Framework for Distributed Industrial Automation and Control BVT Borrowed Virtual Time BW Bandwidth pCPU Physical CPU vCPU Virtual CPU SEDF Simple Earlier Deadline First VNC Virtual Network Computing SPICE Simple Protocol for Independent Computing Environment PECOS PErvasive Component Systems VIF Virtual InterFace TCF Target Communication Framework SSH Secure Shell
iv
CONTENTS
Chapter 1 INTRODUCTION 7 1.1 Introduction ...... 7 1.2 Related work ...... 8 1.3 Problem formulation ...... 9 1.4 The contribution of the thesis is two folds ...... 10 Extending FASA real-time component-based framework to virtual nodes ...... 10 Design Reusable Test Environment ...... 10 1.5 Analysis of the problem ...... 11 Setting up of Virtualized Multicore Environment ...... 11 Development of Automated Orchestra Application ...... 12
Chapter 2 BACKGROUND 14 2.1 Theoretical Background ...... 14 Component-Based Real-time Software Development (CBSD) ...... 14 Future Automation Software Architecture (FASA) ...... 15 Virtualization ...... 17 2.2 Technological Background ...... 18 Xen Hypervisor (Xen pronounced /’zɛn/) ...... 18 Open vSwitch ...... 19 4DIAC-IDE ...... 20
Chapter 3 Development of Orchestra components 21 3.1 Component development mechanism ...... 21 3.2 Orchestra organization ...... 22 3.3 Define requirements ...... 22 3.4 Distributed orchestra application overview ...... 25 3.5 Component modelling ...... 26 3.6 Template generated by 4diac-fasa plug-in ...... 27 Application directory ...... 27 Library directory ...... 27 Module directory...... 28 3.7 Standard Midi file format and partitioning ...... 28 Standard MIDI file format ...... 28 Software tools and libraries ...... 29 Midi file partitioning and rhythm message ...... 30 3.8 Algorithm Implementation for functional blocks ...... 31 Conductor Functional Blocks implementation...... 31 Principal Functional Block implementation ...... 31 Musician Functional Block implementation ...... 32
Chapter 4 Deployment of Orchestra Application 33 4.1 Setting up - Network of Virtualized Multicore environment ...... 33 Host installation: Ubuntu-12.04 LTS ...... 33 Xen hypervisor installation ...... 34 Open vSwitch installation ...... 34 VM installation: yocto VM, Ubuntu-12.04 LTS headless ...... 34 4.2 Define Virtual computation resource ...... 36 VCR with QoS for CPU, memory and network BW ...... 36 Application Schedule and Execution ...... 38
v
4.3 Orchestra Application Management ...... 39 FASA component manager ...... 39 Libvirt and Virt-manager ...... 39 Orchestra distribute management script ...... 39
Chapter 5 CONCLUSION 40 5.1 Result ...... 40 5.2 Future work ...... 41 Extending orchestra application and development...... 41 Defining virtual computational resource ...... 41
Chapter 6 REFERENCES 42
Chapter 7 Appendix 1 A. Orchestra modeling using 4DIAC ...... 1 B. Orchestra component resource allocation on a single host and scheduling ...... 4 C. Orchestra component resource allocation on a network level and scheduling .. 5 D. Virtual machine configuration xen hyprevisor ...... 7 E. HP Compaque Elite 8300 Ultra-slim Desktop summary ...... 8 F. Open vSwitch throughput test result using iperf tool...... 1 G. Open vSwitch – jitter and latency test ...... 1
vi
Chapter 1 INTRODUCTION
1.1 Introduction Industrial control systems (ICS) are found in many areas, e.g., in motion control for industrial robotics, process control of large plants such as in the area of oil and gas, and in large national power grids. A control system controls processes using control application by interacting with environment through sensors and actuators. ICS are becoming large and complex now days due to many reasons. For instance, Distributed Control Systems, DCS such as ABB 800xA automation provides extended services beyond controlling simple industrial processes, such as energy efficiency and asset management, energy savings and operator effectiveness [1]. Likewise, embedded control systems are imposed with more regulatory requirements than before for safety reasons (such as IEC 61508, ISO 26262) and other sub segment standards in energy, process automation and control. As a result, it is common and desirable for a single industrial control system to execute more than one application simultaneously to preserve physical space and ease of management. To meet current industry demands for large and complex software systems, integration of software technologies and consolidation of hardware and software is crucial; for example adoption of virtualization and multicore technology. Virtualization powered by hypervisors such VMware ESX and Xen provides the ability to run multiple virtual machines in a single physical node by abstracting underlying hardware resources such as CPU, memory and communication systems; hence, low hardware investment and high energy cost savings. Even service availability and maintenance become easier due to compartment of virtual machines efficiently on a single physical node. Similarly, multicore technology is benefiting general computer industry at large and now a day being applied to industrial and embedded systems such as in mobile systems. Multicore processors have multiple cores in a single chip; therefore, it enables multiple applications to run parallel in each core without the need for separate hardware systems while maintaining application integrity as compared to unicore systems. Moreover consolidation of virtualization and multicore technology can provide spatial and temporal isolation of applications running in a single machine.
Figure 1: Virtualized multicore platform
7
For example in Figure 1, multiple virtual machines are running on top of multicore platform which is virtualized using Xen hypervisor. The hypervisor allocates one-to-one mapping of cores to virtual machines to achieve high degree of temporal and spatial isolation of applications running on each virtual machines. The benefit of virtualization and multicore technology is a trade-off with challenges they incur in real-time systems. Due to complexity of multicore architecture, e.g., shared bus, shared memory and multi-threading, porting of real-time application across different hardware platforms has been challenging. Likewise in virtualization technology, hypervisor schedulers such as Xen credit scheduler incur significant over head for example as compared to traditional real-time schedulers. And this introduces jitter to virtual machine execution which leads to unpredictable real-time application execution. Real-time component-based platforms such as FASA [2], CAmkEs [3], PECOS [4] provide development and runtime environment for real-time component-based application. And the purpose of the thesis is to evaluate real-time component-based framework, FASA in the context of virtualized multicore environment. And this will be demonstrated using an automated orchestra application. The orchestra application is a real-time and distributed application developed using real-time component based software development platform. Finally, the application is expected to produce acoustically pleasant orchestra (unnoticeable delay and latency) through speakers connected to each physical computer node. The rest of the report is organized in the following way. Chapter 1 is Introduction and will continue to analyze the thesis problems in depth. Chapter 2 is Background to theoretical and technological aspect of real-time component based software development, virtualization and software technologies used in the thesis scope of work. Chapter 3 is development of orchestra application; and chapter 4, deployment of orchestra application on network of virtualized multicore PCs. Finally, chapter 5, result, conclusion and future work is discussed.
1.2 Related work In this thesis work, using orchestra to evaluate real time system is motivated by Julien et al. [5]. Among others they used orchestra to show synchronization and integration of several IEC 61499 standard compliant functional blocks on a network of NetBurner MOD5272- 100CR systems. They were able to show how the orchestra was played without distortion using functional blocks showing applicability of IEC 61499 standard for soft real-time application. In this thesis research, however, we will use similar automated orchestra application on virtual and multicore environment. In addition, we will evaluate a real-time component framework as compared to FORTE or similar runtime environment used in the above mentioned research. There have been many attempts to make Xen hypervisor applicable for real-time application, such as in the case of RT-Xen [6] [7]. RT-Xen implements four server based hierarchical scheduling algorithms (periodic, polling, sporadic and Deferrable server) which are based on fixed-priority preemptive scheduling. The algorithms were evaluated and compared to Xen default Credit scheduler for overhead measurements (latency and context- switches) over 10 seconds of execution; and the result showed an average of 0.2% time loss for RT-Xen schedulers and 0.04% for Xen Credit scheduler. To minimize overhead of Xen scheduler, we avoided using RT-Xen; instead, we mapped each virtual machine (aka domains) to a single core so that excepting credit scheduler from load balancing VCPU across physical cores, therefore, increasing timeliness and predictability of component execution on domains.
8
Little work is done, at least to my knowledge, on evaluation of timeliness and predictability of real-time component-based framework on virtual environments. Evaluations of several component technologies for embedded system can be found in Anders Möller et al. [8]. They used technical and development process requirements to prioritize component technologies. Among which PECOS was found to perform better in average; especially for resource constrained application. In this approach; however, only technical evaluation of timeliness and predictability of FASA component model framework will be performed for real-time and distributed application.
1.3 Problem formulation The purpose of the thesis is to extend real-time component-based framework, Future Automation Software Architecture, FASA with virtual nodes using Virtual Computational Resource (VCR); and evaluate timeliness and predictability f real-time application in the context of virtualized multicore network of PCs. And this is demonstrated by modeling and implementing orchestra application. The application is soft real-time1 system and distributed application which implements functionality and synchronization of orchestra performance. The thesis work will try to address the following research sub-problem: Problem #1: Study and implement the concept of Virtual Computational Resource, VCR2 with QoS for CPU, memory and network bandwidth. Underplaying computer hardware resources such as memory, CPU and network bandwidth is divided into a set of virtual resources defined in textual format, identified in this report as Virtual Computational Resource, which enables a virtual system to run in a host system. Virtual Computational Resource creates a layer of abstraction to hardware resources which makes it possible to preserve execution behavior of virtual systems across differ hardware platforms. The benefit of VCR can easily be seen in porting real time application from for instance from unicore to multicore system. Problem#2: Model and implement an orchestra application on a network of virtualized multicore PCs. According to a research made by Nelson Posse et al. [9], human ear is capable of recognizing latency between 20-30 milliseconds. This implies, events occurring within 20-30 milliseconds drift can be identified as separate events by our ear. Furthermore, the research discusses the power of human perception to musical rhythm can reach to 4ms precision. This leaves us how we can exploit our hearing and rhythm perception capability in real-time systems. Therefore, musical orchestra will be automated to demonstrate how real-time component-based frameworks perform under virtual computational resource. Problem #3: Study to what degree the architecture will support software-independence by evaluating how different allocations of application parts impact real-time behavior. Automated Orchestra application will be deployed in one and more physical nodes; and see how this different allocation will result in playing the orchestra properly with maintaining timeliness and predictability of real-time application. Problem #4: Dynamic turn ON/OFF of instruments and prepare test configurations.
1 A real-time system if missed deadline degrades performance but still is usable afterwards; for example, out of sync an instrument produces acoustically unpleasant music but still continues to play the orchestra. 2 VCR is a computational resource obtained by partitioning an actual hardware resource e.g., CPU, memory and network bandwidth for applications as QoS. 9
1.4 The contribution of the thesis is two folds
Extending FASA real-time component-based framework to virtual nodes The real-time component-based framework under analysis, FASA, according to previous records does not support partitioning [2]. In this thesis work, it is extended to support virtual nodes using Virtual Computational Resource (VCR) with QoS for Memory, CPU and network bandwidth. VCR is a computing resource definition created by partitioning actual hardware resource into units that can individually be used for development, allocation, verification and analysis of parts of a system. For instance, FASA application parts, such as automated orchestra components would be allocated to VCRs; and the QoS defined in the VCR e.g., CPU, Memory and network bandwidth could be enforced during runtime by Virtual Machine Monitoring (VMM) such as Xen hypervisor. VCR can be prepared separately without the knowledge of hardware system at hand; consequently, facilitating software engineering process e.g., requirement analysis and design. It also preserves timeliness and predictability of real-time application parts if migrated, for instance to a different hardware architecture than where it originally was running, e.g., from a unicore to multi-core system. Finally, the communication performance of the test environment as show in Figure 2 such as jitter, latency and throughput is compared over Ethernet protocol with and without VCR.
Design and Install Reusable Test Environment The test environment is a network of computer nodes; and consists of two nodes virtualized using Xen hypervisor, two non-virtual nodes for benchmark purpose and a central management system. Three virtual nodes are created per host; and are connected using OpenvSwitch3 – an advanced virtual switch. The virtual nodes and nodes used for benchmark are installed with Real-Time OS developed using Yocto Project Development4 approach – an open source and collaborative project which creates custom Linux distribution for embedded systems. Furthermore, the test environment is setup with Yocto Project Development for System Core developers (e.g., building OS) and Yocto Application Developers (application which run on top Yocto Machines). It provides a platform for further development, debugging, testing and evaluation of real-time OS and network communication performance.
Figure 2: High-level test environment It has central and coordinated management for computer nodes using technologies, e.g., Virtual Network Computing (VNC), Target Communication Framework (TCF), Secure Shell (SSH) and Libvirt - a library for managing platform virtualization technologies such as Xen hypervisor, KVM,VMWare ESX. The test environment would be suitable, e.g., to develop and cross-compile network performance tool for Yocto Machines, test high precision synchronization of virtual nodes and Xen hierarchical scheduler development.
3 OpenvSwitch - an open source virtual switch suitable virtualization. http://openvswitch.org/ 4 Yocto Project is an open source and collaboration project for creating Linux based distribution for embedded systems. https://www.yoctoproject.org/ 10
1.5 Analysis of the problem We may breakdown the thesis work into two parts: Setting up of Virtualized Multicore Environment and Development of Automated Orchestra Application. Using this approach we will discuss how the above mentioned sub problems will be addressed throughout the thesis work
Setting up of Virtualized Multicore Environment Linux OS (aka GNU/Linux)5 has become the choice of academic research in OS, mainframes, supercomputers and now days being used in embedded systems, e.g., mobile phones, network routers, automation control. Android OS is a prominent example of Linux OS used in embedded systems; it is build from Linux Kernel and deployed on mobile phones. Linux OS is free and open source available in several distribution forms; example of commercial distributions includes Ubuntu (Canonical Ltd.), Fedora (Red Hat) and openSUSE (SUSE). Ubuntu is one of the most popular distributions in Linux community; and it uses Debian package manager, dpkg which is powerful and easy to use. We will use Ubuntu-12.04 as a host and management OS in setting up of virtualized multicore environment. The following are a brief description how virtualized multicore environment will be established: 1. Host virtualization - A hypervisor will be used to monitor (or CPU access, memory usage and I/O peripheral access) virtual machines. The most common open source hypervisors at the time of writing the thesis are Xen, KVM and VirtualBox. Xen hypervisor is chosen for I/O performance and maturity reasons. It is supported by a number of companies such as Amazon, Cloud.com and GoGrid; and generally performs better as compared to KVM and VirtualBox according to some study6. 2. Virtual Machine - OS for virtual machine is based on Real-Time OS (RTOS) from Yocto Project Development7 – a lightweight, customized Linux distribution. 3. Communication and synchronization of virtual and non-virtual nodes - Once established the virtual environment; communication between virtual machines is achieved through virtual switch supporting OpenFlow standard. OpenFlow standard is an open standard which defines the control and forwarding layer of software defined networking, SDN8; similar functions found in routing and switching. The most common open source virtual switch supporting openflow are open vSwitch and Indigo. Since open vSwitch hot-plug script9 is supported by default in the latest release of xen-4.3, open vSwitch is used for implementation. Computer nodes are synchronized to central node through Network Time Protocol (NTP). 4. Tuning virtualization (CPU and Memory usage) - The following components of hypervisor if tuned and configured correctly give an optimal virtualization for execution of orchestra application. Scheduler – Xen comes with default credit scheduler; and other schedulers SEDF and BVT. Virtual CPU, VCP is another feature to be considered for optimization. The more VCPU a guest machine has the
5 Linux OS: http://en.wikipedia.org/wiki/Linux 6 Xen Vs. KVM: http://indico.cern.ch/getFile.py/access?contribId=5&resId=1&materialId=slides&confId=45282 7 Yocto project: https://www.yoctoproject.org/ 8 SDN: http://en.wikipedia.org/wiki/Software-defined_networking 9 A script that enables an auto-configuration of open vSwitch for ease of use 11
more CPU access has. In fact, to achieve predictability, Xen can be extended with other schedulers, e.g., addition of hierarchical real-time scheduler I/O – Xen HVM hypervisor enables I/O device emulation and direct device access from virtual machines. High performance is achieved when configured with direct I/O access also called PCI pass-through in Xen. Memory – Virtual machines should not suffer from limited memory access. Instead, virtual machines will be allocated sufficient memory during creation. Once we have a working environment, we define a Virtual Computational Resource (VCR) will QoS for CPU, memory and network bandwidth for each virtual node.
Development of Automated Orchestra Application First a preliminary study of orchestra will be conducted such as organizational structure, communication and synchronization of musicians. This is crucial to imitate relevant features required for our application. Following this, orchestra application will be developed based on approach suggested by de Almeida et al. [9] which emphasizes the need for componentization on design and implementation level through the Software Development Life Cycle, SDLC. Based on detail analysis of orchestra application, first will be to model and implement components required for the application. This will be followed by deployment of the application on virtualized multicore environment which involves resource allocation (CPU, memory, network bandwidth) and development of application schedule. 4DIAC-IDE and FASA are the platform to be used for development and runtime execution of orchestra application. FASA is an ABB in-house research framework for development of distributed control applications. It is used to develop, build and run cyclic application. Since there is close connection to development team of FASA for support, it is favoured over other platforms such as CAmkES and PECOS for this thesis work. The following tools will be used to develop the orchestra application: 1. FreeMind - a mind mapping tool 2. 4DIAC-IDE – an open source component-based software development IDE based on IEC 61499 standards. 3. 4diac-fasa plug-in – a modelling transformation tool from 4IDAC to FASA components model. 4. FASA framework - a development and runtime execution platform for distributed control application. Using the above listed tools, the orchestra components and distributed application will be developed following a software development life cycle process. First, a mind mapping tool, FreeMind is used to describe the basic picture of orchestra organization that is the conductor, principal and musician and the hierarchical relationship. Next a UML (use case and component diagram) tool, Umbrello is used for detail problem specification, component design and relationship. Finally, the design will be realized using 4DIAC-IDE tool to model and implement the prototype orchestra application.
12
Once the orchestra application is developed on the 4DIAC-IDE, a 4diac-fasa plug-in will be used to transform the model which was previously based on IEC 61499 standards to a FASA compatible component modelling platform for runtime execution. Since the 4DIAC- fasa plug-in transformation does not have support yet for resource allocation devices to components, FASA application template, which defines allocation of components, will be manually edited. In addition, to launch the application from a central node, a script will be developed for deploying, starting and stopping the application on the virtualized nodes.
Figure 3: orchestra application deployment logical architecture
13
Chapter 2 BACKGROUND
2.1 Theoretical Background
Component-Based Real-time Software Development (CBSD) A component is a reusable unit of software deployment accessed through an interface10 [10]. Adopting a component based software development, especially for large and complex software architecture simplifies development, encourages inter operability of third party components. Since components are usually tested and maintained frequently, application developed using component based achieves higher quality and shorter development time task based applications. In fact, several industries have benefited from component-based development; for instance, the global industrial automation company, ABB has implemented component based in the new release of industrial automation and collaborative platform product, System 800xA Extended Automation. Industrial Control Systems have strong constraint in resource utilization, especially on hardware resources (CPU, Memory, Network Bandwidth and Hard disk), power consumption, and physical space. On top of this, complexities of such applications are growing from time to time due to additional services; such services for example can be observed on ABB System 800xA Extended Automation. The new automation platform supports real-time system information retrieval and enables decision making right away and has collaborative platform for effective operation [1]. In order to provide a solution to such inevitable industrial demands, a component-based software development has to be extended to address real-time behavior of control application. And this has been possible by implementing a real-time component based development framework. Real-time component-based software development addresses timeliness and predictability demand of real-time control applications. In fact, similar functionality can be achieved with the use of processes, semaphores, and message queues; however, their implementation has the tendency to incur runtime overhead as compared to real-time component due to frequent context-switches. In addition to using real-time component based development, application need to be distributed across multiple devices; e.g., hosts, cores to share execution load. In the following section we will discuss Future Automation Software Architecture, FASA in detail - an example to real-time component-based software development by ABB research group [2].
10 An Interface is with a component is a point of access point based on a well defined and accepted standard, also called a contract. 14
Future Automation Software Architecture (FASA) FASA [2] [11] [12] [13] is a Component-Based Real-Time Software Development platform with its own runtime environment, deployment and management platform. It supports two modes of application development, fast and safe mode. At compile time, the platform enables software engineers to select either one or both mode. Fast mode enables applications to execute in a single process; as a result, components reside in the same address space and communicate through shared memory. Applications developed using fast mode executes faster; however, may introduce memory corruption due to share memory. On the other hand, safe mod allows components of the same application to reside in a separate and dedicated address space; as a result, memory corruption is technically avoided.
FASA KERNEL FASA Framework Manager Schedule Policy OS
HOST
Figure 4: FASA framework FASA framework consists of FASA kernel, FASA framework manager and a schedule policy. Each FASA kernel instance provides a FASA framework manager. The manager is responsible for managing execution of a distributed application at runtime such as schedule policy change and component related operation; e.g., start and stop component execution. 1. FASA Component Framework FASA component framework comprises the following structural elements: Component, Block, Port and Channel. A Component in FASA provides isolation of software application parts by encompassing one or more blocks. A block is a sequence of software instruction that defines FASA unit of execution. Data communication between blocks is achieved through Channels. A channel in FASA is implemented as a one-to-one and unidirectional communication between ports defined in blocks willing to transfer data. A port is a data type interface, which is identified as input/output port, holds data for receiving and transmitting, respectively.
Application
component block port
a
n
c
o
i
i
l t
p B1 channel B2
p A FASA framework
Figure 5: FASA component framework Data communication between blocks is made possible through three different mechanisms depending on components relative deployment. - Components within the same address space – blocks within a single component or components with the same address space use shared memory for data communication. As compared to other communication types discussed below, it has low communication latency due non-context switching, but high possibility of memory corruption11 .
11 Violation of memory location integrity caused by processes interference which ultimately causes program crash or undesirable behaviour, wiki 15
Sender Block Receiver Block
Shared Write Read memory
Figure 6: Shared memory communication mechanism - Components on different address space but within the same Host – FASA implements System V message queue12 for data communication between blocks residing on different address space. This type of communication also applies to components located at separate cores within the same host. A sender block sends message to a shared queue asynchronously while a receiving block retrieves the message synchronously which means the receiving block will be blocked until data is available. This is to avoid faulty data retrieval by the receiving end.
Sender Block Receiver Block
Write Data area Data area Read
Msg_send Msg_receive Message Queue
Figure 7: Message queue communication mechanism - Components on different host (physically or virtually separated hosts) – FASA uses a network proxy to achieve data communication between blocks residing on physically different machines. A network proxy is an application layer protocol based on client-server model which acts on behalf of sender or receiver end. A block ready to send data uses a Network Send Proxy; similarly, a block ready to receive data uses Network Receive Proxy. The network proxies, Netproxy, are special blocks in FASA. They are scheduled like a normal FASA blocks to execute immediately after a sender block or before a receiver block.
Network Send Network Network Proxy Receive Proxy
Sender Block Receiver Block Msg_receive Msg_send
Write Data area Data area Read
Figure 8: Netproxy communicating mechanism
12 Message queue implementation of System V (aka SysV) which is one of the pioneer Unix OS 16
Virtualization Application of virtualization dates back to 1960s when IBM mainframe was started to host multiple applications sharing the same physical resource and running simultaneously. Since the acquisition of x86 based machines and desktop OS such as windows and UNIX, virtualization evolved to refer to a large scale of applications such desktop level, OS level and application level virtualization [14]. The definition of virtualization is buzzword and is defined differently by different authors even today; however, there is a general consensus on the basic concept and application of virtualization. It mainly refers to logical partitioning of physical computing resources regardless of the physical location; and by doing so operational cost is minimized and investment is reduced. The most obvious computing resources to be virtualized are processor, I/O device, memory, communication and storage. And virtualization technology is used in academia and business mainly through desktop virtualization and cloud computing [15]. It’s quite important to differentiate virtualization from emulation. Both of them involve imitation of computing resources as though they exist in reality; however, the methodology and purpose is different. Emulation imitates a hardware or software of interest by translating instructions from one form to another. For example, QEMU is an emulator; we can create or emulate a PCH HDA adapter regardless of the underlying hardware sound adapter is, e.g., it could be Sound blaster. Virtualization uses hypervisors to partition and manage computing resources. An emulator is usually used for testing different processor and peripheral devices for which it is not designed for, avoiding hardware or software expenses; and also is used in virtualization to emulate some devices to the virtual machine. Components of virtualization Virtualization has two main components: a hypervisor and guest machines. A hypervisor is a piece of software (aka called a Virtual Machine Monitor, VMM) that manages resources such as CPU, memory and interrupts used by virtual machines. Depending on CPU mode13 a hypervisor is classified into two types (it is not a clear cut classification though): 1. Type-1 or Bare metal hypervisor (Figure 10: Type 1 hypervisor) – executes right on top of a hardware layer with privileged mode as compared to guest machines. And guest machines run on top hypervisor with restricted mode of operation. Example, Citrix XenServer, VMware ESX/ESXi. 2. Type-2 or operating system level hypervisor (Figure 9: Type 2 hypervisor)– operates on top of conventional operating system such VMworkstation and VirtualBox .
Figure 10: Type 1 Figure 9: Type 2 hypervisor hypervisor
13 The two main CPU modes are privileged mode and restricted mode. Unless a program (a hypervisor or OS) is in privileged mode some instructions are not allowed to execute. http://en.wikipedia.org/wiki/CPU_modes 17
2.2 Technological Background
Xen Hypervisor (Xen pronounced /’zɛn/) XenHypervisor is an open source hypervisor developed by Xen Project Team14. It is a Virtual Machine Monitoring, VMM comprising lightweight software program (<150,000 lines of code) that manages CPU, memory and interrupts for guest machines. Neither does it manage I/O transaction nor does it manage guest machines. Instead, there is a privileged guest machine, Dom0 which is ported with host OS usually from UNIX flavor (Linux, FreeBSD distributions) which manages guest machines and device drivers. Xen is a type 1 hypervisor; and is base to XenCloud platform and XenARM. The Xen architecture has the following components: 1. The Xen hypervisor – manages CPU, memory and interrupts to guest machines 2. Dom0 – a control guest machine which has driver support for guest machines and tool stack to manage DomU. 3. DomU – a guest machine which run under restricted mode with their own OS 4. Xen Tool stack15 – an interface in command line or graphical to manage the guest machines. 5. Modified kernel – Xen requires a modified kernel to act as a Dom0 usually found
from the mainline kernel or build from source.
s
t n
e Dom0 n
o (Management p HVM guest
m guest machine
o PV Guest PVHVM Guest c machine) DomU
n toolstack Machine Machine e
X DomU DomU Xen virtual Qemu-Xen firmware Frontend Backend driver driver Frontend HW drivers driver
Xen Hypervisor Schedule MMU Host HW
Figure 11: Xen components for PV, HVM and PVHVM types Xen hypervisor supports three types of virtualization: 1. Paravirtualization (PV) – is the first type of virtualization supported by Xen. This type of virtualization requires a modified guest machine drivers (aka backend drivers) on the Dom0 side and a frontend drivers on the DomU side. DomU I/O access is through Domo backend drivers. Dom0 has the actual driver to communicate with Host hardware. 2. Hardware Virtual Machine (HVM) aka Full virtualization – this is type of virtualization is supported on host machines with virtualization extension such as Intel VT-x. Hardware resources such CPU, memory and I/O are emulated
14 Xen Project: http://www.xenproject.org/ 15 Xen toolstack: http://wiki.xen.org/wiki/Choice_of_Toolstacks 18
using QEMUL device emulator. Qemu-xen16 is the Xen flavor of QEMU; it provides a virtual firmware to DomU by giving the impression that the guest machine is running directly on physical machine. Emulation is processor intensive and as compared to PV, HVM performs less in I/O throughput. To solve this problem, Xen team introduced PCI pass-through in Xen hypervisor. PCI pass-through enables DomU to access I/O devices directly without the emulation using Input/Output Memory Management Unit, IOMMU (Intel VT-D and AMD IOMMU) to boost communication performance between I/O devices and DomU. 3. Paravirtualization on HVM – this is a recent feature introduced on Xen-4.3 release. It boosts HVM performance using PVHVM drivers (an optimized PV driver for HVM); therefore, without the need for QEMU device emulation. A kernel feature PVOPS in the host machine identifies for virtualization mode and then divers driver request to use PVHVM drivers. Therefore, the HVM network and disk access uses paravirtualization method (with modified driver) instead of emulation.
Open vSwitch Open vSwitch (open virtual switch) is a switch specially designed for bridging virtual machines to physical network [16]. It is based on Software-defined Networking, SDN approach which emphasizes the need for separate control and data plane17. Linux-bridge is the default birding in Linux for virtual machines; however, it’s not designed to support multi- server virtualization18; as a result networking in virtualization tend to go to more advanced
virtual switches such as Open vSwith and Indigo.
s
t n
e xenbr0 Ovs-vswitchd xenbr1 n
o (slow path) p
m Ovsdb-server
o
c
h Packet
c i
t classifier FlowTable
w
S
v
n User mode
e p O Kernel mode NetLink
JSON/RPC Openvswitch_mod.ko (management) Eth 0 (fast path) Hash_lookup Table
Figure 12: open vSwitch components Openvswitch operates at user and kernel space mode. The kernel module, openvswitch_mod.ko processes and forwards packet entering a physical network interface. If bridging traffic to virtual machine is required or advanced switch configuration is needed the packet is forwarded to ovs-vswitchd at user space. Initial switching configuration is stored at ovsdb-server and using JSON/RPC, the configuration is offloaded into flowTable. After creating a bridge, a physical Ethernet configuration is no more used.
16 Qemu-xen: http://wiki.xen.org/wiki/QEMU_Upstream 17 SDN: http://en.wikipedia.org/wiki/Software-defined_networking 18 Why openvSwitch? : http://git.openvswitch.org/cgi- bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=WHY-OVS;hb=HEAD 19
4DIAC-IDE Framework for Distributed Automation and Control, 4DIAC is a free, open, IEC 61499 standard compliant automation and control environment. One of the projects by the 4DIAC initiative is 4DIAC-IDE. It is an integrated development environment of IEC 6199 compliant automation and control, based on Eclipse framework.
Figure 13: 4DIAC-IDE perspectives An IEC 61499 standard provides the architecture, tools and rule of compliance for development of distributed automation and systems [17]. It addresses shortcomings of its predecessor, IEC 61131 standard e.g., programming language independence but moreover it provides distribution and dynamic reconfiguration. The atomic execution unit of IEC 61499 standards, Functional Block, FB is the construct of an application. It has event and data connection to interact with other FB or the environment as indicated by red and blue lines in the 4DIAC-IDE. An application is developed by interconnecting FB instances and allocating them a resource19. And the FBs are distributed to physical devices across a network. Events trigger actions by executing algorithms within the FB according to Execution Control Chart, ECC [18]. And an executed algorithm pushes data to the output I/O for transmission. 4DIAC-IDE has three main perspectives20 for developing an application based on functional block semantics defined by IEC 61499 standards. 1. System management perspective - this is the main perspective where control applications are modeled, devices and resources are configured 2. Type management perspective – is the perspective for managing IEC 61499 compliant types such as using pre-existed FBs, creating blocks and editing functional blocks. 3. Deployment management perspective – finally a system configuration (resources and devices) is prepared and application deployed on controllers launched using runtime programs such 4DIAC - Runtime Environment, FORTE.
19 Resources can be considered processor cores, memory, bandwidth … provided by devices. 20 4DIAC-IDE Quick start: http://sourceforge.net/apps/mediawiki/fordiac/index.php?title=4DIAC-IDE_Quickstart 20
Chapter 3 Development of Orchestra components
The Orchestra application is a composite of components where each component provides a service, e.g., rhythm message and timing specification to another component. The component service is defined using functional blocks modeled using 4DIAC-IDE tool based on IEC 61499 standards and executed on FASA runtime environment (aka FASA kernel). The application constitutes a total of 17x components and 17x functional blocks. Software tools employed during development are FreeMind for mind mapping, Umbrello UML modeling tool (Use Case Model and Sequence diagram), 4DIAC-IDE and FASA for modeling and implementing orchestra application21. In the following subsequent topics, we will discuss the development approach, preliminary study of orchestra in music, and modeling and implementation process: Defining requirements, modeling and implementation using the software tools mentioned above.
3.1 Component development mechanism Initially orchestra application components are developed and implemented using 4DIAC-IDE. Then using 4diac-fasa plug-in, the model is transformed to FASA component model framework where execution of the application is performed. FASA component framework does not have its own IDE; instead, the plug-in is used to transform some of the features of 4DIAC-IDE e.g., basic functional block, sub-application of 4DIAC to blocks and components in FASA. By the time of report writing, basic 4DIAC-IDE features such as device creation, resource allocation and deployment is not supported by the plug-in; as a result, it is done manually. 4DIAC-IDE is build by cloning to the source hosted on sourceforge.net. Then 4diac-fasa plug-in is imported to the set of 4DIAC-IDE plug-ins before building it. The built IDE gives us an eclipse-IDE with 4DIAC features and automated transformation for 4DIAC-to-FASA. The plug-in enables an application engineer to use 4DIAC-IDE to develop application for FASA framework which provides visualization instead of textual work on FASA; however, the transformation is not smooth enough perhaps is under further development. The transformation result is a set of files and directories regarding components and functional blocks; these need to be copied manually to respected structural tree on FASA. Due to this, for every update on the model, it is not reflected automatically on the FASA. I believe this has to do with the deployment feature of 4DIAC not automated for FASA. The perspective is used to deploy an application on a selected runtime environment, similar to FORTE and FBDK22, which can be a prototype for FASA integration in the deployment perspective. We suggest FASA deployment process be integrated in the deployment flow of 4DIAC-IDE in the future. Before diving into the development process, it’s important to understand the context of orchestra in music such as organizational structure, role, communication and synchronization of musicians; and features adopted or imitated to our orchestra application.
21 Does not include resource allocation and device assignment of components, this is done using FASA framework. 22 Functional Block Development Kit, FBDK: http://www.holobloc.com/doc/fbdk/index.htm 21
3.2 Orchestra organization An orchestra23 is a large instrument ensemble usually led by a conductor and principals or sub-conductors. As a rule of thumb, the conductor is responsible for leading the whole orchestra while the principals rehearse a certain group of musicians with similar quality of instruments such as string, percussion, woodwinds and brass. The flow command for playing the orchestra comes from top hierarchy and goes through principal at second level. The basic orchestra activities are imitated to suit the project intended objective that is timeliness and predictability in multi processing environment. If the instrument is played before or after an anticipated time (which is indicated by rhythm note), the orchestra goes out of sync. Out-of- sync implication in this context implies to latency and jitter introduced during execution of components in the virtualized multicore environment and transmission. According to study [9] on human perception to such effects, many researchers share the conclusion that 20-30 ms sound delay is tolerable by our ears. What is more interesting about sound produced by orchestra is human capability to follow rhythm even at slight deviation of 4ms at sub conscious level. This has in an implication to the feedback we get from the orchestra application: the feedback we get from the orchestra application need to be within 20-30ms so that it will not be perceptible by our ears; and second even with acceptable range of latency and jitter, the quality could vary; and this is possible to notice by following the rhythm within high precision or around 4ms.
3.3 Define requirements Automated orchestra application conceived from the concept of orchestra in real world (for example, as shown from the mind map of Royal Stockholm Philharmonic Orchestra) is a two level hierarchal of communication, which is communication between the conductor and principals; and among principals and corresponding group of musicians. Whatever is directed from the conductor is passed through the principals; and rehearsed by musicians asynchronously (in our implementation the communication goal is achieved through cyclic execution of blocks, though).
Figure 14: Mind map of orchestra from Royal Stockholm Philharmonic Orchestra
23 Orchestra wiki: http://en.wikipedia.org/wiki/Orchestra 22
The problem domain identified in the mind map is elaborated further in the Use Case Model.
Figure 15: Use Case Automated Orchestra Application
Use case name Conduct Orchestra Participating Actors Computer Terminal User Precondition The system is setup with roles of musicians and timing information such as start time, tempo in a configuration file. Basic flow 1. The user launches the application by entering ‘application run’ on the terminal. 2. Parses a configuration file and stores orchestra organizational structure and rate (cycles/sec) at which the orchestra to be played in a buffer. 3. Decode midi file and save it into a buffer. 4. Creates rhythm groups24 composed of multiple tracks to be played at constant rate. Alternative flow Post conditions Saves the state of conducting the orchestra such as indices of the last event25 from each rhythm group and absolute time played since the orchestra has started. Store the generated rhythm groups in a shared memory. Special requirements
Use case name Conduct Musicians Participating Actors Precondition Rhythm groups are generated and stored in a shared memory. Basic flow 1. Reads orchestra organizational structure from a configuration file. 2. Split the rhythm groups into a rhythm26. Alternative flow
24 Rhythm group is a set of rhythms. 25 Event is the occurrence of ON /OFF notes indicated in MIDI file protocol standard. 26 Rhythm is a subset of events in a single musical track/instrument in a MIDI file. 23
Post conditions Stores the rhythms in a shared memory. Special requirements
Use case name Play Music Participating Actors Precondition A rhythm is generated and stored. Basic flow 1. Parses the rhythm and identify ON/OFF events. 2. Reads delta time between subsequent events. 3. For each event, the sound system is triggered preceded by a silence of delta time. And this will produce a sound on the speaker. Alternative flow Post conditions Special requirements A speaker should be connected to the system. An audio system which produces a sound should be installed in the system. A synthesizer and sound bank that interpret the notes should be installed in the system.
Automated orchestra application is initiated using a computer terminal. It could deployed eit a standalone system or distributed across a network of computer nodes. The use case, Conduct the Principals process the input files to the system and further decodes the midi file into a group of musical note which is input to another use case, Conduct the Musicians. The later use case distributes the musical note which is finally played in Play Instrument use case. This process is repeated periodically until the midi file timeline is finished playing.
conductor principal musician synchronization
rhythm[] {Forevery event in the Rhythm Index:0 rhythm paly the event} ~microsecond ~microsecond log next cycle duration
Index:1 rhythm[] rhythm 2, 3 ... log next cycle
Figure 16: Orchestra performance UML sequence diagram
24
Component specification From the Use Case Model and mind mapping depicted above we can deduce the basic required components to do the overall functionality of the orchestra application; hence, the following components are indentified: 1. The conductor component 2. The principal component 3. The musician component A summarized number of blocks and components required and functionality. Sub-application/ Functional Block Functionality /Block (FASA) Component (FASA) 1xconductorSA 1xconductorFB - Parse Midi (orchestra) - Divide the midi into group of tracks - Identify set of tracks based on manual conf. - Distribute (rhythm, index, duration message ) to sub-conductors/principals 4xprincipalSA 4xprincipalFB - Divide the rhythm among the musicians based on preconfigured information. 12xmusicianSA 12xmusicianFB - play the rhythm for the allocated duration
Table1: Orchestra application blocks and components and functionality
3.4 Distributed orchestra application overview The following high level design of orchestra application a single path which is conductor- principal-musician. Likewise, the same procedure will be repeated for multiple principals and musicians. This single application which is playing an orchestra will be will be modeled and implemented in the next sub section.
midifile
n
o
i t
a Parse midi file Next cycle Play rhythm
c
i
l
p
p
a
a r Orchestra t Extract rhythm Orchestra s Dispatch rhythm organization XML e groups organization XML h to musicians
c -tracks and events
r O
Dispatch rhythm groups to Port? principals
Extract rhythm Port ? Events per track
Table 2: Orchestra application high level design
25
3.5 Component modelling Finally we are going to model the components based on requirements and high level design described above using 4DIAC-IDE. First we will describe a step-by-step procedure for creating the component types using 4DIAC-IDE type management perspective. Respective snapshots are to be found at Appendix A. Step #1: Create a new project by the name OrchestraProject and fill appropriate fields. File->New->New System and fill the project name ‘OrchestraProject’ Next, import the default Pallets for using default libraries from 4DIAC. Fill information related to the project e.g., type, comment, description … Go to Windows and show ‘System Manager view’, if not opened in the ‘System Manager perspective’; the created project will be available there. Step #2: create the functional blocks Shift perspective to ‘Type Management->Type navigator view’. Select and right-mouse click on OrchestrProject and create a new folder, orchestra; it can be any name for that matter. Right-mouse click on the new folder and go to ‘New Type’ Give a name to the block e.g., conductorFB of ‘Type’ basic block. Create input and output variables as indicated in Appendix Repeat this for principal and musician basic blocks (aka functional block in IEC 61488 standard naming) Step #3: Create orchestra application and use the blocks created in step#2 Go to ‘System Manager’ and right-mouse click and select ‘New application’ Give it a name ‘orchestra’ Drag and drop the blocks created Connect the event and data channels (Note: Since device and resource allocation is not supported by 4diac-fasa transformation plug-in, manual configuration is done. This is only to indicate the work flow of 4DIAC-IDE). The following diagram is a result of a subsequent steps discussed above.
Figure 18: musician functional block
Figure 19: conductor functional block Figure 17: principal functional block
Following orchestra component modelling on 4DIAC-IDE, let’s have a look on the code or template generated by 4diac-fasa transformation plug-in.
26
3.6 Template generated by 4diac-fasa plug-in The transformation plug-in generates a template for the application developer to start with writing algorithms. The template is a collection of directories within are found configuration files and C++ source and header files. These are discussed in detail below:
Application directory This directory holds XML files for each sub application or components modeled in the 4DIAC IDE. These XML files describe a component such as name and related component implementation path. According to modeling, there are 15 XML files that describe a component (1xconductorSA, 2xprincipalSA and 12xmusicianSA). Other XML files includes: and a single XML for a single host the kernel will be executing. - Orchestra.xml –a communication and schedule configuration on a single host. - Orchestra_net_master.XML – a communication and schedule configuration for the conductor when application is distributed across multiple virtual nodes. - Orchestra_net_swartxx.xml – a communication and schedule configuration for principal and musician when application is distributed across virtual nodes. - Swartxx_net_receive/send.xml – a configuration setup for receiving and sending proxies; xx refers to host name.
Library directory This directory holds a template for actual C++ source and header files on a separate folder for blocks and components. - Methods – algorithm which describes the behavior of a block - Variables and State – input/output of blocks and state of a block.
27
Module directory This directory holds a single configuration file which describes location of each component and block in the application. A compiler refers this configuration for reference of blocks and components. #include "conductorSA.h" #include "conductorFB.h" #include
3.7 Standard Midi file format and partitioning
Standard MIDI file format MIDI (Musical Instrument Digital Interface) is a technical standard published by Association of Musical Electronics Industry (AMEI) which defines a protocol, digital interfacing and communication of musical instruments, computer system and other devices. MIDI uses event message to communicate among digital midi devices which basically specifies music notation, pitch, velocity and others. One way of carrying these messages is using MIDI file as compared to midi sequencers which also is used to edit and play music. The format of MIDI file used in the scope of works is produced by International Midi Association, EMI [19]. The file block is made up of chunks and events. Only relevant parts are discussed below: Chunk – is a sequence of 8-bit byte with the following format.
28
single event followed by D key. The time difference between the two notes is called delta-time. The unit of measurement can be ticks or milliseconds.
Event:
Table 3: midi file format
Software tools and libraries Midi file simply holds MIDI notation and controls; therefore, once parsed the midi file. A series of steps is required to produce the actual sound from the speaker. Following are the tools used: Fluidsynth – is an open source software synthesizer27 which converts midi data into audio signal using SoundFont technology28. The orchestra application uses fluidsyth development API to produce sounds based on a midi note data. Please PulseAudio 29– is the default sound server in Ubuntu which accepts sound requests from multiple applications for concurrent access and other advanced audio processing services. For example, the orchestra application and Rhythmbox application (player in Ubuntu) use the sound card concurrently through PulseAudio. ALSA30 – PulseAudio does not contain any drivers for underlying sound cards; instead, it uses Advanced Linux Sound Architecture, ALSA to communicate with sound cards. ALSA is the default audio infrastructure in Ubuntu OS and has
supports for many sound card vendors31 as compared to its predecessor OSS32.
h
t
n y
s Orchestra
d i
u application
PulseAudio l f
Others App
ALSA
Sound Card (Intel HDA)
Figure 21: Orchestra application audio platform on Ubuntu OS 27 Software which imitates and produces sounds of instruments using programming algorithms. 28 Maps recorded and synthesized audio; especially for music composition. 29 PulseAudio: http://www.freedesktop.org/wiki/Software/PulseAudio/ 30 ALSA: http://www.alsa-project.org/main/index.php/Main_Page. Aug 14, 2013 31 http://www.alsa-project.org/main/index.php/Matrix:Main#ALSA_SoundCard_Matrix 32 OSS: http://www.opensound.com/oss.html. Aug14, 2013 29
Midi file partitioning and rhythm message The midi file is partitioned into a set of events per track (also called rhythm in the scope of work). The even types we are interested in are ON_event and OFF_event. ON_event indicates start of playing a key note; and OFF_event indicates end of playing a key note. The orchestra application uses fluid_synth_noteon() and fluid_synth_noteoff() APIs from
playRythm() routine to enforce ON_even and OFF_event on midi driver through synthesizer. g
n Start_abs_time = 0
i
n o
i 0 1 duration t
i rhythm
t
r
a
p
s
t Track#1
n
e
v
e
i Track#n
d
i M ON_event OFF_event
Figure 22: midi file partitioned and ON/OFF events
/* Rhythm information */ Description struct Rhythm { MFEvent *mfevent; -Midi events int track; -Track number int start_index; -Start of event index in midi int end_index; -End of event index in mid int absolute_tick; -Start of tick number in the midi int absolute_tick_played; -Absolute tick plus duration in tick int cycle; -The number of cyclic execution int ticksPerQuarterNote; -Number of ticks needed to sound a }; quarter note, used for conversion from tick to seconds Table 4: Rhythm message format A rhythm message is a message transmitted by the conductorFB block to the musicianFB through principalFB. It is used to direct the musician what to play and for how long to play. This message is sent periodically according to cyclic execution of FASA blocks. Since FASA modeling is periodic, any variable number of events is treated by the musician within that period.
30
3.8 Algorithm Implementation for functional blocks
Conductor Functional Blocks implementation The conductor functional block, conductorFB implements the conductor functionality in the orchestra. It reads midi file and orchestra organization XML and cyclically dispatches rhythm group (rhythm message for multiple tracks) to principal functional blocks. It supports four principal functional blocks, from CH0O to CH3O.
The conductorFB sends m
a midifile
r rhythm group according to
g a
i predefined prinicipal-port
d
w mappings from orchestra
o
l
f
r Orchestra State organization XML file. o
t Extract rhythm -event index
c organization XML
u group -abs time Since FASA blocks are
d n
o stateless, the conductorFB has a C separate state structure, Dispatch rhythm conductorFB_Block_State() to hold group to principals rhythm information (index, absolute tick) of a previous rhythm message.
End of a cycle
Table 5: conductor flow diagram
Principal Functional Block implementation The principal functional block, principalFB implements the functionality of sub conductor in the orchestra. It receives a group of rhythms from the conductor and dispatches the rhythm based on preconfigured track information from orchestra organization XML.
m
a Receive rhythm r
g group from
a
i conductor d
w
o
l
f
l
a Orchestra p i organization XML Extract rhythm
c
n
i r
p
Dispatch rhythm group to musicians
End of a cycle
Figure 23: principal functional block implementation
31
Musician Functional Block implementation The musician Functional block, musicianFB implements the functionality of musician in the orchestra. It receives a rhythm from principal functional block and plays by referring the event index and duration from the rhythm message.
Whenever the rhythm player is m
a start encountered with ON or OFF events, there is
r start g
a always a delta time proceeded to it. And this
i d
delta time is used to delay or hold the note for w
o Port
l that time, event.time. If the musicianFB did
f
l connected
a ? not encounter events for one or more cycles,
p i
c the delta time is accumulated for at least the
n i
r yes ‘duration x the number of cycles not played’. p And by the time an event arrives the Receive rhythm musicianFB, it tries to hold the note for the from principal prolonged amount of time which is not correct. Instead, it is expected to delay since the the latest musicianFB cycle start. For this reason, we need a state to save the number of State cycles the block did not encounter events; and Play rhythm cycle_not_pla yed then refresh it once encountered to zero. Delay provided by the event: not applicable (delta) 1 2 cycle ? duration
End of a cycle Correct_delay = delta – duration x cycle Correct_delay ON event OFF event Figure 24: musician functional block implementation
32
Chapter 4 Deployment of Orchestra Application
In this chapter we will be discussing setting up of network of virtualized multicore environment and deployment of the orchestra application. Specifically the following points will be covered: Setting up - Network of Virtualized Multicore environment Define Virtual Computational Resource Allocate Components to VCR Launch application from central terminal or master node
4.1 Setting up - Network of Virtualized Multicore environment In this section we will describe software packages used to establish a network of virtualized multicore environment. These software packages, which are motivated in problem analysis section 1.5, include Xen, open vSwitch, Linux distribution from yocto project and Libvirt/virt-manager. We will also explain installation and configuration of the software packages on the selected hardware platform. For a complete installation procedure, refer to Xen_ovs_yocto_installation.pdf [20].
HP Compaque Elite 8300 Ultra-slim Desktop summary
Processor : 4x Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz Memory : 946MB (513MB used) Operating System : Ubuntu 12.04.2 LTS Audio Adapter : HDA-Intel - HDA Intel PCH
Table 6: Software technologies
Host installation: Ubuntu-12.04 LTS Xen hypervisor requires a VM, Dom0 which manages the rest of VMs, DomUs. Usually the Dom0 is a variant of Linux, but can also be used from NetBSD or OpenSolaris. The most commonly used Linux variants that have Dom0 support are found from fedora and Ubuntu distribution (Please refer33, for a list of kernels which supports Dom0). If a distribution without Dom0 kernel support is installed, a separate Dom0 installation is expected. Unless, it is necessary to build a separate Dom0, it is always recommended to use one of the distributions which support Dom0 kernel from the list indicated above. In this thesis work, Ubuntu-12.04 LTS is used. It has a long term support from Ubuntu, and also has active community support. In fact, Fedora distribution is also a good alternative.
33 Dom0 kernels for Xen: http://wiki.xen.org/wiki/Dom0_Kernels_for_Xen 33
The Xen hypervisor boots right after a bootloader34; therefore, it is important to understand how the boot process is performed. The two main options35 used to boot are: Basic Input/Output (BIOS) and Unified Extensible Firmware Interface (UEFI). Though UEFI is the most advanced firmware interface and a promised successor to BIOS, we found BIOS to be much easier and error prone to boot Xen hypervisor. Another important point during installation of the host is partitioning the hard disk. The host OS occupies a small portion of space; and the rest can be used for the VMs. Since, it is hard to determine how much size is required for VM ahead, a Logical Volume Partition (LVM) is scheme used. This enables dynamic resizing of partitions in the hard disk after installation which is by far more advantageous than using static partitioning (without LVM). To achieve this, it is recommended to make the hard disk a logical disk during installation of the host OS.
Xen hypervisor installation Xen hypervisor can be obtained either from the Xen project36 or from Ubuntu repository37. Xen hypervisor from Xen project has to be built from source; and the latest version is always available. On the other hand, Ubuntu repository provides a software package which is already build and tested for a specific hardware platform; and usually is more reliable but not usually latest. During the thesis writing, the latest Xen hypervisor releases, Xen-4.3 and Xen-4.1, are available from Xen project and Ubuntu repository, respectively. Due to latest features in Xen-4.3 e.g., open vSwitch hot-plug and latest and long term support tools stack (XL toolstack), Xen hypervisor is configured, build and installed from source. The next step would be to install Open vSwitch and configure networking.
Open vSwitch installation It’s possible to configure networking of virtual machines using a default Linux Bridge38; however, it lucks advanced features of switches such as flow analysis and QoS. Instead, Open vSwitch is used, which is an advanced virtual switch specifically designed to networking of virtual machines without compromising switching performance, especially as compared to Linux Bridge. As mentioned above, the Xen hypervisor already has a hot-plug support for open vSwitch. This implies ease of configuration and integration of open vSwitch product to Xen. But open vSwitch has to be installed like any other Linux software package either from source or Ubuntu package repository. In this installation, we installed open vSwitch from Ubuntu repository, which is much easier and sufficient to our needs (most importantly bridging VM and configuring QoS).
VM installation: yocto VM, Ubuntu-12.04 LTS headless A Dom0 is used for installation of DomUs. Dom0 hosts different tool stacks for managing DomUs, e.g, XL, virsh and virt-manager. It is possible to create and manage all types of Xen VMs (paravirtualized,HVM and PVHVM) using the above mentioned tools. In our scope of work we managed to create and manage HVM for a reason explained on the analysis of problem 1.5. The type of VMs (DomUs) used in this installation are from yocto project, which is an opensource project dedicated for building a Linux distribution mostly for embedded systems; and Ubuntu-12.04 LTS headless distribution (text mode only). Both types of DomUs are built with Real-Time (RT) patch. This enables the DomUs machine to be appropriate for soft real-time applications.
34 A piece of computer program which loads an OS or hypervisor and give services to it, e.g., memory-space mapping http://wiki.osdev.org/Bootloader 35 BIOS vs. UEFI: http://www.logic.nl/products/technology/bios-and-efi.aspx 36 Xen project: http://www.xenproject.org/downloads.html 37 Ubuntu package repository: http://packages.ubuntu.com/ 38 Linux bridge: https://help.ubuntu.com/community/NetworkConnectionBridge 34
To build an RT based Linux distribution from yocto project, a certain procedure has to be followed based on yocto project development process. The development process includes from simple to advanced approaches; e.g., using a predefined configuration template to generate a Linux distribution, modifying kernel configuration, modifying kernel source, are some the activities done on yocto development platform. To start with, it’s recommended to use a graphical tool, Hob tool, which is used to define requirements and generate a Linux distribution. This is the method used to create RT Linux distribution which lightweigh (~500MB) and bootable image. To obtain a headless Ubuntu-12.04 LTS (text based), it is only a matter of configuration change on the bootloader through /etc/default/grub file (CMD_LINE_LINUX = “text”) – which basically tells the kernel to load in text mode, hence, no graphical desktop, operated by command line. Both approaches are implemented in the installation.
Master_node Core-1 Swart1 Swart2 Core-1
OpenvSwitch OpenvSwitch
Cisco switch Core-1 swart11 swart21 Core-1
Core-2 swart112 swart1212 Core-2 Core- swart113 swart1213 Core-3 3
Figure 25: network topology of host and virtual machines using openvswitch The above network topology shows how the VM as connected to each other using open vSwitch and Cisco switch. The host machine swart1 has three VMs swart11, swart12 and swart13; similarly, the host machine swart2 has three VMs swart21, swart22 and swart23. During installation and configuration of the software packages, we encountered several challenges e.g., software incompatibility, BUGS, system crash and so forth. As compared to modeling and implementation of Orchestra application, it took more time than expected due to the challenges mentioned above. From our experience, we find it important to carefully follow installation procedure, document all activities during installation (successful installation or troubleshooting mechanism and challenges) and ccommunicate with respected software technology development teams, such as subscribing to a mailing list for patches, bug reports and advices on using the products. And these type of support can be found either from Ubuntu distribution or the developers e.g., xenProject for Xen hypervisor.
35
4.2 Define Virtual computation resource Virtual Computation Resource, VCR with QoS for CPU, memory and network bandwidth, are computing resource definitions generated by partitioning an underlying and actual hardware resources for allocation, verification and analysis of application parts. VCRs allow portability of real-time components or application parts to multiple hardware architecture while preserving timeliness and predictability of an application; e.g., application porting from 2 core to 4 core machines. Without prior knowledge of hardware resource, VCR can be prepared separately.
VCR with QoS for CPU, memory and network BW Following installation of Xen hypervisor, VM and open vSwitch, we will develop a VCR with QoS for CPU, memory and network BW. 1. QoS for CPU - Available scheduler types in Xen hypervisor by the time of report writing: credit scheduler (default), Simple Earlier Deadline First, SEDF and Borrowed Virtual Time, BVT [21]. Credit scheduler and BVT are based on fair sharing of CPU in mind; and SEDF as a real-time scheduler has some drawbacks that impact predictability of VM execution. It fairly distributes slack time on virtual machines after all other VM finish execution. And this disrupts predictability of virtual machine execution. Therefore, to increase predictability of orchestra application, as a quick solution39, Xen scheduler is exempted from load scheduling VM. Instead, each VM are mapped to a single core using the following XL toolstack commands: #create pool to hold the pcpu xl cpupool-create name=\"Pool-1\" sched=\"credit\" xl cpupool-create name=\"Pool-2\" sched=\"credit\" xl cpupool-create name=\"Pool-3\" sched=\"credit\"
#remove pcpu from Pool-0 xl cpupool-cpu-remove Pool-0 1 xl cpupool-cpu-remove Pool-0 2 xl cpupool-cpu-remove Pool-0 3
#add the pcpu to the created pool xl cpupool-cpu-add Pool-1 1 xl cpupool-cpu-add Pool-2 2 xl cpupool-cpu-add Pool-3 3
Table 7: Create CPU pools for each core The above XL commands create four Pools for holding a single core. And each pool will be mapped to a VM by adding the following line into a VM configuration file. Pool =Pool-x” where x {1,2,3}
cpus = “x” Sets affinity of the VM to x {1,2,3}
vcpus = 1 Limits the number of VCPUS the VM uses to 1.
GRUB_CMD_LINE_XEN = To set affinity of Dom0 to core-0 “domain0_min_vcpus=1 dom0_vcpus_pin”
39 Other solutions include development of real-time scheduler which is out of the thesis work 36
2. QoS for Memory –Memory is allocated to each VM including Dom0 to avoid performance degrade due to DomUs memory starvation caused by Dom0. By default, Dom0 is allowed to use free memory (aka known as ballooning40 ); therefore, it is recommended to limit each VM memory usage to a certain level of memory using the following XL commands:
GRUB_CMD_LINE_XEN= Configure at Xen command line, “dom0_mem=1G,max:1G” limits Dom0 to 1G
Memory = ”512” Limits DomUs to 512M
3. Qos for Network BW – the rate at which the VM communicate is limited using open vSwitch. The following example configures a Virtual Interface, VIF of some VM to 10Mbps: #ovs-vsctl set Interface tap0 ingress_policing_rate=10000 #ovs-vsctl set Interface tap0 ingress_policing_burst=1000
*tap0 – name of VIF connected to VM The following table shows allocation of CPU, memory (RAM) and network BW to each component. Component Machine pCPU vCPU RAM Network BW (hostname) (RxTx) ConductorSA Master node 1xcore 4 1G 10Mbps princiapalSA_00 Swart-1 Dom0 1xcore 4 1G 10Mbps princiapalSA_10 Swart-2 Dom0 1xcore 1 1G 10Mbps musicianSA_10 Swart11 DomU 1xcore 1 512M 10Mbps musicianSA_20 Swart11 DomU 1xcore 1 512M 10Mbps musicianSA_30 Swart11 DomU 1xcore 1 512M 10Mbps musicianSA_40 Swart12 DomU 1xcore 1 512M 10Mbps musicianSA_50 Swart13 DomU 1xcore 1 512M 10Mbps musicianSA_60 Swart21 DomU 1xcore 1 512M 10Mbps musicianSA_70 Swart22 DomU 1xcore 1 512M 10Mbps musicianSA_80 Swart23DomU 1xcore 1 512M 10Mbps musicianSA_90 Swart21 DomU 1xcore 1 512M 10Mbps musicianSA_11 Swart22 DomU 1xcore 1 512M 10Mbps musicianSA_12 Swart23DomU 1xcore 1 512M 10Mbps Table 8: Virtual Computation Resource for components
40 Ballooning is a technique used to dynamically increase memory for Dom0 when needed. 37
Application Schedule and Execution The orchestra components are executed periodically with the following timing parameters. Component Period(ms) Offset(ms) conductorSA 5 10 50 100 0 principalSA_xx 5 10 50 100 1 musicianSA_xx 5 10 50 100 1
Figure 26: Component execution model A 1ms offset is sufficient for network latency between the components, hence a total of 2ms between the conductor and musician. This is found from traffic analaysis of openvirtual switch with an average latency ~0.032ms.
conductorSA
principalSA_00 Op
principalSA_10
musicianSA_00 Om track1 track1
musicianSA_10 track2 track2
musicianSA_20 track3 track3
musicianSA_30 track4 track4
musicianSA_40 track5 track5
musicianSA_50 track6 track6
musicianSA_60 track7 track7
musicianSA_70 track8 track8
musicianSA_80 track9 track9 0 2ms 2+d T T+2ms T+2ms+d 2T
Figure 27: orchestra application components trace
38
4.3 Orchestra Application Management Following allocation of components to virtual computation resource as describe in the previous section, the application is ready for launching. A script is executed from the master node which initiates the FASA kernel instances on each machine to run at an absolute time. All machines in the system are synchronized using Network Time Protocol, NTP protocol. Master node is the NTP server which intern is synchronized with ntp.ubuntu.com. All guest machines including machines where the principal components reside are synchronized to the master node instead of ntp.ubuntu.com to increase precision (because of the geographical vicinity). The following management tools are used to manage components, virtual machines and the orchestra application.
FASA component manager Each kernel instance has a component manager for switching configuration, adding and removing components, add or remove channels, get list of components. This is a standalone and also a distribute management framework of FASA.
Libvirt and Virt-manager This is GUI based on libvirt41 for managing virtual machines. It has a wizard for creating virtual machines and configuring VM easily. It supports VNC42 and SPICE43 client viewers for displaying virtual machine console.
Orchestra distribute management script This is a script developed to manage specifically the orchestra application and the virtual multicore environment in distributed mode.
Orchestra distributed management script
Machines Application -applies to virtual machines -compile and hosts -update machines -create, list, reboot VM -run application -configure network -issue any command -distributed sshkey
Figure 28: orchestra management script features
41 It is hypervisor independent library for managing virtual machines and hypervisor. 42 Virtual Network Computing is a graphical desktop sharing using remote buffering. http://en.wikipedia.org/wiki/Virtual_Network_Computing 43 Simple Protocol for Independent Computing Environment is a remote display system http://en.wikipedia.org/wiki/SPICE_(protocol) 39
Chapter 5 CONCLUSION
In the previous subsequent chapters of the thesis report, we modelled and implemented an orchestra application mainly using 4DIAC-IDE for demonstrating and evaluating FASA component framework on a virtualized multicore environment. And we defined the concept of a virtual computational resource (VCR) for components. Based on the VCR defined, the application was deployed on virtualized multicore PCs. In the following sub sections, we will discuss the result and future work.
5.1 Result The result will be discussed in accordance to sub problems outlined in the problem formulation, chapter 1.3. Problem#1: Model and implement an orchestra application on a network of virtualized multicore In chapter 3, we modeled and implemented Orchestra application mainly using 4DIAC- IDE. First orchestra music and organization structure was studied briefly; followed by, requirement analysis and modeling using various tools but mainly using 4DIAC-IDE based on IEC61499 functional blocks. The model is deployed on FASA runtime environment or kernel where timeliness and predictability of orchestra application was performed. The functionality of the components was implemented first by studying standard MIDI format and partitioning it into manageable rhythm messages. Then the algorithm was defined for each component types (conductor, principal and musician) and tested to work as expected. Finally, we connected all components as a bottom-up fashion to construct the application by defining resources on all devices such as communication channels, CPU schedule and execution sequence. Problem #2: Study and implement the concept of Virtual Computational Resource, VCR with QoS for CPU, memory and network bandwidth. First the concept of VCR was studied in chapter 4.1 and then using Xen hypervisor we were able to virtualize physical resources such as memory, CPU and network bandwidth. Virtual Machines of type Hardware-assisted Virtual Machines (HVM) were created. Following that we defined QoS for CPU and memory using XL toolstack and virt-manager for each virtual machine (aka DomUs) which enabled to achieve special and temporal isolation of virtual machines. We used open vSwitch to define QoS measured in Mbps for network communication between virtual machines for transmitting and receiving rates. And network throughput, latency and jitter of virtual machines was measured using iperf tool on a virtual and physical network connected using open vSwitch and Cisco switch, respectively. And the result shows no datagram lost in 32768 bytes transferred within 10 seconds from a host to virtual machine. And latency was reported by the receiving end was around 0.005ms. For more, refer to Appendix F and G.
40
Problem #3: Study to what degree the architecture will support software independence by evaluating how different allocations of application parts impact real-time behavior. The application is tested in the same hardware architecture by enabling and disabling physical cores; and the orchestra performed as before. However, it is not ported on different hardware platform and tested due to time limitation. But we believe it will perform the same as long as the required Virtual Computational Resource is available in the new hardware platform and using Xen hypervisor as virtualization enabler.
Problem #4: Dynamic turn ON/OFF of instruments and prepare test configurations. A set of configuration is created with different QoS to evaluate the response of the real- time component-based development platform. For more single host component allocation, Appendix A and network level component allocation, Appendix C.
5.2 Future work Future work of the thesis can be viewed from two perspectives. First we can extend the application for evaluation of the FASA framework. Second we can improve the definition of a virtual computational resource. These two are studied in more detail bellow:
Extending orchestra application and development 1. Currently each VM is allocated a single component. In the future we could play multiple instruments (designed as components) in a single virtual machine. This would mean that a virtual machine has to interleave the musician blocks to play the music, similarly on other VMs without any drift from the timeline of the orchestra. This gives more loads to the real-time component execution framework on hierarchical scheduler nature (scheduler by the hypervisor and VM RTOS). 2. Add a midi instrument support e.g., keyboard directly connected to midi interface; and GUI interface for selecting orchestra, dynamic turn ON/OFF of tracks. 3. Integrate FASA runtime environment similar to FORTE in the 4DIAC-IDE. This could ease the development of application for FASA framework. 4. Evaluating other real-time component-based frameworks using the orchestra application.
Defining virtual computational resource Currently Xen lucks an offline real-time scheduler. In the future, Xen can be extended with an offline and static scheduler that can be used for scheduling control applications. In the future the concept of a virtual computational resource could be defined as a separate layer regardless of which real-time component execution framework is used.
41
Chapter 6 REFERENCES
[1] ABB, "System 800xA System Introduction technical document for customers," 2012.
[2] M. Oriol, R. Steiger, S. Stoeter, E. Vardar, M. Wahler, H. Koziolek and A. Kumar, "FASA: a scalable software framework for distributed control systems," in Proceedings of the 3rd international ACM SIGSOFT symposium on Architecting Critical Systems, Bertinoro, Italy, 2012.
[3] K. Ihor, L. Yan, G. Ian and H. Gernot, "G.: CAmkES: a component model for secure microkernel-based embedded systems," 2007.
[4] T. Genßler, A. Christoph, M. Winter, O. Nierstrasz, S. Ducasse, R. Wuyts, G. Arévalo, B. Schönhage, P. Müller and C. Stich, "Components for embedded software: the PECOS approach," Proceedings of the 2002 international conference on Compilers, architecture, and synthesis for embedded systems, Vols. 1-58113- 575-0, no. CASES '02, pp. 19--26, 2002.
[5] J. Chouinard, J. Lavallée, J.-F. Laliberté, N. Landreaud, K. Thramboulidis, P. Bettez-Poirier, F. Desy, F. Darveau, N. Gendron and C.-D. Trang, "An IEC 61499 configuration with 70 controllers; challenges, benefits and a discussion on technical decisions," in 14th IEEE Conference ETFA’07, Patras, 2007.
[6] X. Sisu, W. Justin, L. Chenyang and G. Christopher, "RT-Xen: Real-time virtualization based on hierarchical scheduling," in Embedded Software (EMSOFT), 2011 Proceedings of the International Conference on. IEEE, 2011., 2011.
[7] L. Min, A. S. Krishnakumar, P. Krishnan, S. Navjot and Y. Shalini, "Supporting soft real-time tasks in the xen hypervisor," in Proceedings of the 6th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, Pittsburgh, Pennsylvania, USA, 2010.
[8] A. Moller, M. Akerholm, J. Fredriksson and M. Nolin, "Evaluation of component technologies with respect to industrial requirements," Euromicro Conference, 2004. Proceedings. 30th, pp. pp.56,63, 31 , Aug.-3 Sept. 2004.
[9] E. de Almeida, A. Alvaro, D. Lucredio, A. do Prado and L. Trevelin, "Distributed component-based software development: an incremental approach," in Computer Software and Applications Conference, 2004. COMPSAC 2004. Proceedings of the 28th Annual International, 28-30 Sept. 2004.
[10] I. Crnkovic and M. Larsson, "The Definition and Specification of Components," in Building Reliable Component-Based Software Systems, Norwood, Artech House, 2003, p. 1.
[11] W. Michael, S. Richter, S. Kumar and M. Oriol, "Non-disruptive large-scale component updates for real-time controllers," in ." In Data Engineering In Data 42
Engineering Workshops (ICDEW), 2011 IEEE 27th International Conference on, pp. 174-178, 2011.
[12] S. Richter, M. Wahler and A. Kumar, "A Framework for Component-Based Real- Time Control Applications," in In 13th Real-Time Linux Workshop, Prague, Czech, 2011.
[13] M. Wahler, S. Richter and M. Oriol, "Dynamic software updates for real-time systems," in In Proceedings of the 2nd International Workshop on Hot Topics in Software Upgrades (HotSWUp '09), ACM, New York, NY, USA, , Article 2 , 6 pages, July 31, 2007..
[14] IBM Global Education White Paper, "Virtualization in Education," October 2007.
[15] A. J. Younge, R. Henschel, J. T. Brown, G. v. Laszewski and G. C. F. Judy. Qiu, "A performance Analysis of Xen and KVM hypervisors for hosting Xen Worlds Project," in 2011 IEEE 4th International Conference on Cloud Computing, 2011.
[16] S. B. Pfaff, J. Pettit, T. Koponen, K. Amidon and M. C. Scott, "Extending networking into the virtualization layer," in 8th ACM Workshop on Hot Topics inNetworks (HotNets-VIII), New YorkCity,NY, October 2009.
[17] J. H. Christensen, T. Strasser, A. Valentini, V. Vyatkin and A. Zoitl, "The IEC 61499 Function Block Standard Overview Of The Second Edition," ISA Automation , week 6 2012.
[18] V. Vyatkin, "The IEC 61499 standard and its semantics," in Industrial Electronics Magazine, IEEE , vol.3, no.4, Dec. 2009.
[19] The Internal MIDI Association, "Standard MIDI file format Spec. 1.1," The Internal MIDI Association, Los Angeles.
[20] M. Nesredin, "Xen hypervisor installation and Yocto Machines," AT, ABB CRC, Västerås, 2013.
[21] C. Ludmila, G. Diwaker and V. Amin, "Comparison of the three CPU schedulers in Xen," SIGMETRICS Perform. Eval. Rev., vol. 35, no. 2, pp. 42--51, sep 2007.
[22] T. Strasser, "Framework for Distributed Industrial Automation and Control (4DIAC)," in Industrial Informatics, 2008. INDIN 2008. 6th IEEE International Conference on, Daejeon, 2008.
[23] VMware, "Virtualization in Industrial Automation," 2010.
[24] ABB Company, "800xA solution Handbook," ABB, Sweden, 2013.
[25] Intel Whitepaper, "Applying Multi-core and Virtualization to Industrial and Safety-Related Applications," 2009.
[26] Lago, N. Posse and F. Kon, "The Quest for Low Latency," in PROCEEDINGS OF THE INTERNATIONAL COMPUTER MUSIC CONFERENCE (ICMC2004}, 2004.
[27] W. Shengquan, S. Rho, M. Zhibin, R. Bettati and Z. Wei, "Real-time component- based systems," in Real Time and Embedded Technology and Applications Symposium, 2005.RTAS 2005. 11th IEEE , vol., no., 7-10 March 2005.
43
Chapter 7 Appendix
A. Orchestra modeling using 4DIAC a. Step #1: Creating orchestra project
b. Step #2: create a new folder for the new blocks
1
c. Step#3: create new functional blocks and new application (to the left)
d. Step#3.1:drag and drop functional blocks
2
Step#3.2: convert into sub applications (in FASA term: into components)
e. Step#4: transform the application using 4diac-fasa plug-in (Install the plug-in just like any other eclipse plug-in) f. Write the algorithm of the functional block in Eclipse platform g. Compile, debug and run it on FASA component runtime environment
3
B. Orchestra component resource allocation on a single host and scheduling
4
C. Orchestra component resource allocation on a network level and scheduling
5
6
D. Virtual machine configuration xen hyprevisor
# ======# Example HVM guest configuration
# ======
# This is a fairly minimal example of what is required for an # HVM guest. For a more complete guide see xl.cfg(5)
# This configures an HVM rather than PV guest builder = "hvm" # Guest name name = "swart11-hvm"
# 128-bit UUID for the domain as a hexadecimal number. # Use "uuidgen" to generate one if required.
# The default behavior is to generate a new UUID each time the guest is started.
#uuid = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" #########################
# CPU SCHEDULING
##########################
#nuber of vcpus vcpus = 1
#pin the guest machine to core-1 cpus = "1"
###########################
# MEMORY ALLOCATION (MB) ########################### memory = 1024
#maxmem = 512 ###########################
# EVENT ACTIONS
########################### on_poweroff = "preserve"
#on_reboot = "restart"
#on_watchdog = "destrory"
#on_crash = "destroy"
###########################
# DEVICES ###########################
7
# Network devices vif = [ 'mac=00:16:3E:74:3d:01, model=e1000, bridge=xenbr0,type=ioemu' ]
# Disk
#disk = [ '/dev/swart-1/swart-11,,hda', '/home/swart/ubuntu-12.04.2-desktop-amd64.iso,,hdc,cdrom' ] #pci=[ '00:1b.0' ] disk = [ '/dev/swart-1/swart-11,,hda' ] boot = "c" # Guest VGA console configuration, either SDL or VNC
#sdl = 1 vnc = 1 vnclisten = "0.0.0.0" vncdisplay = 1 soundhw = "hda" device_model_version = 'qemu-xen' device_model_override = '/home/swart/qemu/i386-softmmu/qemu-system-i386'
E. HP Compaque Elite 8300 Ultra-slim Desktop summary
Processor : 4x Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz
Memory : 946MB (513MB used) Operating System : Ubuntu 12.04.2 LTS
Audio Adapter : HDA-Intel - HDA Intel PCH
8
F. Open vSwitch throughput test result using iperf tool
t
u
p
h
g
u
o
r
h
t h c c c c c
t e e e e e
U d s s s s s
-
m i / / / / /
-
o w s s s s s
-
D c c c c c d t t t t t
- - -
h e e e e e ) n i i i i i
- - - -
: t s s s s s t a b b b b b
- - - -
2 d / / / / / l B M M M M M
- - - - -
# i s s s s s u
- - - - -
w t t t t t a 8 3 4 2 8
- - - - -
o d i i i i i f 0 8 6 5 9
- - - - -
i ) n b b b b b e 1 1 1 1 0
- - - - -
r t a M M M M M d 6 6 6 6 6
- - - - -
a l B ( r
- - - - -
n u 3 6 8 1 1 e
- - - - -
e a 6 6 6 7 4 1 e f s s s s s
- - - - -
s f 3 2 1 2 0 0 t s e e e e e
- - - - -
c c c c c e 2 0 1 0 1 0 y n t t t t t
- - - - -
e e e e e d 2 2 2 2 2 5 B a y y y y y
- - - - -
s s s s s ( r M r B B B B B
- - - - -
/ / / / / e t T M M M M M
- - - - -
s s s s s 1 e f s s s s s r 8
- - - - -
r t t t t t r 0 t s e e e e e o 0 5 4 4 2 0
- - - - -
e h i i i i i e 0 y n t t t t t p . 6 4 0 7 4
- - - - -
v t b b b b b v 5 B a y y y y y 0 5 7 7 6 5
- - - - -
i d M M M M M i M r B B B B B P 4 4 4 4 4
- - - - -
e i e t T M M M M M C : 1 1 1 1 1
- - - - -
c w 0 7 1 4 8 c r 8 T e
- - - - -
e d 2 0 7 6 3 e o 0 8 4 6 2 9 r z l
- - - - -
r n 1 3 4 3 7 r p . 3 3 7 4 7 e n i a c c c c c
- - - - -
a 3 2 8 3 2 0 3 3 4 3 1 v o s v e e e e e
- - - - -
B 3 3 2 3 3 P 3 8 0 8 0 i r s s s s s
- - - - -
C : 5 4 5 4 5 e g w e
- - - ) - -
T e c n o t 0 0 0 0 0
- - - s - -
s s s s s z l e i d n . . . . .
- - - e - -
e e e e e n i a c c c c c r n n I 0 0 0 0 0
- - - n - -
t t t t t o s v e e e e e e i 2 2 2 2 2
- - - i - -
r y y y y y r s s s s s t w ] - - - - -
- - - h - -
e B B B B B g w e s D 0 0 0 0 0
- - - c - -
f M M M M M n o t 0 0 0 0 0 i P I . . . . .
- - - a - -
s i d n . . . . . l C 0 0 0 0 0
- - - m - -
n 8 1 0 5 1 n n I 0 0 0 0 0 T [
- - - - -
a 7 3 1 8 6 e i 2 2 2 2 2 r
- - - l - -
r 9 0 9 5 0 t w ] - - - - - e ] ] ] ] ]
) - - - a - -
T 8 7 7 9 8 s D 0 0 0 0 0 v 4 5 4 5 4
0 - - - u - -
7 7 6 7 7 i P I . . . . . r
m - - - t - -
l C 0 0 0 0 0 e
o - - - r - -
T [ S [ [ [ [ [
D - - - i - -
c c c c c r
- - - v - -
e e e e e e ] ] ] ] ]
n - - - - -
s s s s s v 4 5 4 5 4
i - - - 2 - -
r
h - - - 1 - -
l 0 0 0 0 0 e
t - - - t - -
a . . . . . S [ [ [ [ [
i - - - r - -
v 0 0 0 0 0
w - - - a - -
r 2 2 2 2 2
- - - w - -
e - - - - -
d - - - s - -
t 0 0 0 0 0
e - ) - - c c c c c - -
n . . . . .
v - e - - e e e e e o - - c c c c c
I 0 0 0 0 0
i - n - - s s s s s t - - e e e e e
e - i - - / / / / / - 1 - s s s s s
c - h - - s s s s s 1 - 0 - / / / / /
e - c - 1 - t t t t t 1 - 0 - s s s s s
r - a - 0 - i i i i i t - 5 - t t t t t
- c c c c c m - 0 - h b b b b b r - - h i i i i i
d - e e e e e - 5 - t M M M M M a - t - t b b b b b
n - s s s s s l - - d w - r - d M M M M M
a - / / / / / a - t - i 2 3 1 6 6 s - o - i
- s s s s s u - r - w 7 7 7 7 4 - p - w 9 4 7 4 8
d - t t t t t t - o - d 3 2 1 2 0 m - - d 0 8 6 5 9
n - h i i i i i r - p ) - n 2 0 1 0 1 o - P ) - n 1 1 1 1 0
e - t b b b b b i - t - a 2 2 2 2 2 r - C t - a 6 6 6 6 6
s - d G G G G G v - P l - B f - T l - B
- i - C u - - u -
s - w 1 3 7 5 7 1 - T a - s s s s s d - , a - s s s s s
i - d . . . . . 1 - f - e e e e e n - 5 f - e e e e e
- n 3 2 0 8 2 t - , e - t t t t t e - 9 e - t t t t t
c - a 3 3 3 2 3 r - 1 d - y y y y y s - . d - y y y y y
i - B a - . ( - r B B B B B - 5 ( - r B B B B B
f - w - 0 - e M M M M M c - 1 - e M M M M M
f - s s s s s s - . e - f i - . e - f
a - e e e e e - 0 t - s 8 4 6 2 9 f - 0 t - s 5 4 4 2 0
r - t t t t t o - . y - n 3 3 7 4 7 f - 1 y - n 6 4 0 7 4
T r - y y y y y t r - 7 B - a 3 3 4 3 1 a r - . B - a 5 7 7 6 5
( e - r B B B B B e - 2 M - r 3 8 0 8 0 r e - 0 M - r 4 4 4 4 4
d - e M M M M M 1 d - 1 - T 5 4 5 4 5 t d - 1 - T 1 1 1 1 1
t n - f t n - 6 - ( n - 2 -
u e - s 8 1 0 5 1 r e - o 1 - e - o 0 -
p s - n 7 3 1 8 6 a s - t . - c c c c c U s - t . - c c c c c
h - a 9 0 9 5 0 w - 0 - e e e e e m - 0 - e e e e e
g - r 8 7 7 9 8 s - g - s s s s s o - g - s s s s s
u - T 7 7 6 7 7 - n : - D - n : -
o - m - i e - 0 0 0 0 0 - i e - 0 0 0 0 0
r - o - t z - l . . . . . > - t z - l . . . . .
h - r - c i - a 0 0 0 0 0 - - c i - a 0 0 0 0 0
t - c c c c c f - e s - v 2 2 2 2 2 < - e s - v 2 2 2 2 2
- e e e e e - n - r ------n - r - - - - -
0 - s s s s s d - n w - e 0 0 0 0 0 U - n w - e 0 0 0 0 0
m - n - o o - t . . . . . m - o o - t . . . . .
o - l 0 0 0 0 0 e - c d - n 0 0 0 0 0 o - c d - n 0 0 0 0 0
D - a . . . . . s - n - I D - n - I
- v 0 0 0 0 0 - t i - - t i -
: - r 2 2 2 2 2 s - n w - ] ] ] ] ] ] : - n w - ] ] ] ] ] ]
1 - e - - - - - i - e - D 3 3 3 3 3 3 - e - D 3 3 3 3 3
# - t 0 0 0 0 0 - i P - I # - i P - I
- n . . . . . c - l C - - l C -
o - I 0 0 0 0 0 i - C T - [ [ [ [ [ [ o - C T - [ [ [ [ [ [
i f i -
r f r -
a a a -
n r n -
e T e -
S ( s -
1
G. Open vSwitch – jitter and latency test
Server listening on UDP port 5001 Receiving 32768 byte datagrams UDP buffer size: 256 KByte (WARNING: requested 128 KByte) ------[ 3] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 45088 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 3] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.043 ms 0/ 38 (0%) [ 3] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.054 ms 0/ 38 (0%) [ 3] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.044 ms 0/ 38 (0%) [ 3] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.041 ms 0/ 38 (0%) [ 3] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.036 ms 0/ 38 (0%) [ 3] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.087 ms 0/ 38 (0%) [ 3] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.091 ms 0/ 39 (0%) [ 3] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%) [ 3] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.098 ms 0/ 38 (0%) [ 3] 9.0-10.0 sec 1.19 MBytes 9.96 Mbits/sec 0.094 ms 0/ 38 (0%) [ 3] 0.0-10.0 sec 12.0 MBytes 10.0 Mbits/sec 0.103 ms 0/ 383 (0%) [ 4] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 37397 [ 4] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.089 ms 0/ 38 (0%) [ 4] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.103 ms 0/ 38 (0%) [ 4] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%) [ 4] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.108 ms 0/ 38 (0%) [ 4] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%) [ 4] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.106 ms 0/ 38 (0%) [ 4] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.106 ms 0/ 39 (0%) [ 4] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%) [ 4] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.091 ms 0/ 38 (0%) [ 4] 9.0-10.0 sec 1.19 MBytes 9.96 Mbits/sec 0.111 ms 0/ 38 (0%) [ 4] 0.0-10.0 sec 12.0 MBytes 10.0 Mbits/sec 0.117 ms 0/ 383 (0%) 383 (0%) [ 3] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 40925 [ 3] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.032 ms 0/ 38 (0%) [ 3] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.059 ms 0/ 38 (0%) [ 3] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.107 ms 0/ 38 (0%) [ 3] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.097 ms 0/ 38 (0%) [ 3] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.104 ms 0/ 38 (0%) [ 3] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.089 ms 0/ 38 (0%) [ 3] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.089 ms 0/ 39 (0%) [ 3] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.094 ms 0/ 38 (0%) [ 3] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.117 ms 0/ 38 (0%) [ 3] 9.0-10.0 sec 1.19 MBytes 9.96 Mbits/sec 0.116 ms 0/ 38 (0%) [ 3] 0.0-10.0 sec 12.0 MBytes 10.0 Mbits/sec 0.116 ms 0/ 383 (0%) [ 4] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 49472 [ 4] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.080 ms 0/ 38 (0%) [ 4] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.091 ms 0/ 38 (0%) [ 4] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.092 ms 0/ 38 (0%) [ 4] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.100 ms 0/ 38 (0%) [ 4] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.132 ms 0/ 38 (0%) [ 4] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.104 ms 0/ 38 (0%) [ 4] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.128 ms 0/ 39 (0%) [ 4] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.086 ms 0/ 38 (0%) [ 4] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.105 ms 0/ 38 (0%) [ 4] 9.0-10.0 sec 1.19 MBytes 9.96 Mbits/sec 0.202 ms 0/ 38 (0%) [ 4] 0.0-10.0 sec 12.0 MBytes 10.0 Mbits/sec 0.189 ms 0/ 383 (0%) [ 3] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 37970 [ 3] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.099 ms 0/ 38 (0%) [ 3] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%) [ 3] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.107 ms 0/ 38 (0%) [ 3] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.099 ms 0/ 38 (0%) [ 3] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.121 ms 0/ 38 (0%) [ 3] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.130 ms 0/ 38 (0%) [ 3] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.090 ms 0/ 39 (0%) [ 3] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.112 ms 0/ 38 (0%) [ 3] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38
1