Czech Technical University in Prague, Czech Republic Faculty of Electrical Engineering Department of Control Engineering Resource Reservation and Analysis in Heterogeneous and Distributed Real-Time Systems Doctoral Thesis by Michal Sojka Prague, August 2010 Ph.D. Programme: Electrical Engineering and Information Technology Branch of study: Control Engineering and Robotics Advisor: Doc. Dr. Ing. ZdenˇekHanz´alek Department of Control Engineering Czech Technical University in Prague c Copyright by Michal Sojka All rights reserved August 2010 To my wife Lucie. Acknowledgements First of all, I would like to express my thanks to my thesis advisor, ZdenˇekHanz´alek, who supported me both personally and financially during the work on this thesis. Another person who deserves thanks is my colleague Pavel P´ıˇsa.Technical discussion with him was always a great source of inspiration for me and it definitely led to higher quality of this thesis. And last, but not least, I would like to thank my students who worked with me on various real-time related projects. The experience from these mostly practical projects had also positive influence on the work in this thesis. Research leading to the results in this thesis has been supported by the European Commission under grant agreement n.FP6/2005/IST/5-034026, in the context of the FRESCOR Project and by the Ministry of Education of the Czech Republic under project 1M0567 (CAK). Czech Technical University in Prague Michal Sojka August 2010 iv Abstract This thesis describes the design, implementation and evaluation of a software frame- work that facilitates development of real-time, possibly distributed, applications. The basic idea of the framework is to let the application developer specify the temporal (and resource) requirements of his/her application and the framework guarantees keeping of these requirements, provided that there are enough resources in the system. In the case of insufficient resources, the framework does not let the application run. Application requirements are specified in the so called service contract that the application negotiates with the framework. A successfully negotiated contract results in creation of a virtual resource, which represents \a part" of the real resource reserved for the use by the application. To not over-reserve the available resources, the framework employs on-line admission tests that are based on state-of-the-art schedulability analysis. One of the main strengths of presented framework is its modularity with respect to support of additional resources, which is shown by integration of six different resources (CPU, network, etc.) into the framework. The prototype implementation of the framework was developed under Linux operating system and it was extensively evaluated on both synthetic tests and real-world multimedia application. Keywords: real-time, middleware, schedulability analysis v Goals and Objectives The main goals of this work have been set as follows. 1. Design and implement a modular software framework supporting resource reservations on heterogeneous resources for distributed real-time applications. The framework should be easily extensible with support for new resources and should allow task migration between resources. 2. Evaluate the framework on a real multimedia application. 3. Develop and evaluate an admission test for wireless network (Wi-Fi) to be used in the framework. 4. Formulate schedulability analysis for tasks with offsets as an integer linear pro- gramming problem and evaluate the performance on analyzing multiprocessor and distributed systems. vi Contents 1 Introduction1 1.1 Contribution...............................2 1.2 Structure of the Thesis..........................3 2 Basic Concepts and State-of-the-Art5 2.1 Real-Time Computing..........................5 2.1.1 A Model of Real-Time System.................6 2.1.2 Schedulability Analysis Techniques............... 10 2.1.3 Server-Based Scheduling..................... 13 2.2 Distributed Real-Time Systems..................... 16 2.3 Component-Based Development of Real-Time Systems........ 16 2.3.1 Model-Driven Engineering.................... 17 2.3.2 Real-Time Component-Based Middleware Platforms..... 17 3 Contract-Based Resource Reservation Framework 21 3.1 Motivation................................ 22 3.2 Architecture................................ 23 3.2.1 Application Model and API................... 25 3.2.2 Resource Managers........................ 26 3.2.3 Resource Allocators....................... 26 3.2.4 Contract Broker......................... 27 3.2.5 Examples............................. 27 3.3 Advanced Concepts and Internals.................... 28 3.3.1 Representation of Contracts and Virtual Resources...... 28 3.3.2 Contract Negotiation Process.................. 29 3.3.3 Distribution of Spare Capacity................. 32 3.3.4 Negotiation of Multi-Resource Transactions.......... 32 3.3.5 Transaction API......................... 33 3.4 Mathematical Model of the Framework................. 34 4 Resources Supported by the Framework 37 4.1 CPU.................................... 37 4.1.1 The AQuoSA Architecture................... 38 4.1.2 Integration of AQuoSA in FRSH/FORB............ 39 4.2 Disk (BFQ scheduler).......................... 39 4.2.1 Integration of BFQ in FRSH/FORB.............. 40 4.3 Wireless LAN............................... 40 4.3.1 Enhanced Distributed Channel Access (EDCA)........ 41 4.3.2 Testbed setup........................... 43 vii viii Contents 4.3.3 Experiments........................... 44 4.3.4 Simple Admission Test...................... 50 4.3.5 Integration of FWP in FRSH/FORB.............. 53 4.4 Wireless Sensor Networks........................ 53 4.4.1 ITEM Network.......................... 54 4.4.2 Cluster-Tree Network Supporting Variable Data Flows.... 55 4.5 FPGA................................... 56 4.5.1 FPGA reconfiguration capabilities............... 56 4.5.2 FRSH/FORB contracts for FPGA resources.......... 58 5 Framework Evaluation 59 5.1 Negotiation Overhead.......................... 59 5.2 FRSH WLAN Protocol (FWP)..................... 60 5.3 Integrated Case-Study.......................... 62 5.3.1 Parameter Tuning........................ 64 5.3.2 Experience Report........................ 66 5.3.3 Experimental Results...................... 67 6 Integer Programming-Based Approach to Schedulability Analysis for Tasks with Offsets 71 6.1 Computational Model.......................... 73 6.2 Original Exact Response-Time Analysis................ 74 6.2.1 Summary............................. 82 6.2.2 Analysis of Multiprocessor and Distributed Systems..... 83 6.2.3 Applicability to the Resource Reservation Framework.... 84 6.3 ILP Formulation............................. 84 6.3.1 ILP Approaches to Schedulability Analysis.......... 85 6.3.2 Restricted Computational Model................ 87 6.3.3 Linear Schedulability Conditions................ 89 6.3.4 Schedulability of Multiprocessor and Distributed Systems.. 91 6.4 Experimental Results........................... 92 6.5 Conclusion................................ 94 7 Conclusions 95 7.1 Summary................................. 95 7.2 Goals................................... 96 A FRSH API Change Proposal 97 A.1 Introduction................................ 97 A.2 Specific problems in the current API.................. 98 A.3 Conclusion................................ 102 Bibliography 108 Curriculum vitae 109 Author's publications 111 List of Figures 2.1 Parameters of a task; a) a non-periodic task, b) a periodic task...7 2.2 An example of priority inversion; dark color means that the task is in the critical section.............................9 3.1 Block diagram of FRSH/FORB framework............... 24 3.2 A contract and its attributes....................... 28 3.3 Collaboration diagram of FRSH/FORB modules during contract negotiation................................. 30 4.1 Integration of the AQuoSA scheduler within the FRSH/FORB archi- tecture. Source: [Sojka et al., 2010]................... 38 4.2 Principles of Enhanced Distributed Channel Access (EDCA) Medium Access Control (MAC) algorithm (source: [Mangold et al., 2002])... 41 4.3 Our testbed setup............................ 43 4.4 Delay of all access categories under non-saturation condition..... 45 4.5 Delay of all access categories where AC BK is under saturation.... 45 4.6 Influence of AC BE at 20 kbps on AC VO and AC VI......... 46 4.7 Influence of AC BE at 200 kbps on AC VO and AC VI........ 46 4.8 Influence of AC BE at 220 kbps on AC VO and AC VI........ 47 4.9 Influence of fully saturated AC BE to AC VO and AC VI....... 47 4.10 Influence of socket send queue size to delays. Two scenarios with SO SNDBUF set to 0 and 3000...................... 49 4.11 Difference in communication delays between AP and non-AP trans- mitters................................... 50 4.12 Comparison of the utilization based test with measured results for three different experiments........................ 52 4.13 Integration of ITEM protocol with FRSH/FORB............ 54 4.14 Demonstration of ITEM wireless sensor network with FRSH/FORB (FRESCOR project)............................ 55 4.15 Example of data structures describing the transaction involving CPU and Field Programmable Gate Array (FPGA) in the contract framework. There are two variants of possible task execution: A { software only and B { FPGA accelerated................ 58 5.1 Contract negotiation time as a function of the number of negotiated contracts.................................. 60 5.2