Utilizing Hardware Monitoring to Improve the Quality of Service and Performance of Industrial Systems
Total Page：16
File Type：pdf, Size：1020Kb
1 Mälardalen University Press Dissertations No. 270 UTILIZING HARDWARE MONITORING TO IMPROVE THE QUALITY OF SERVICE AND PERFORMANCE OF INDUSTRIAL SYSTEMS Marcus Jägemar 2018 School of Innovation, Design and Engineering 2 Copyright © Marcus Jägemar, 2018 ISBN 978-91-7485-395-7 ISSN 1651-4238 Printed by E-Print AB, Stockholm, Sweden 3 4 5 Antligen!¨ 1 My own translation: Finally! — Gert Fylking, 2000 [107] 1The debater Gert Fylking attended the Nobel literature prize announcement several consecutive years (2000–2002) and exclamated “finally” when the winner was announced. His comment implied that the prize winner was unknown for the people that didn’t belonging to the cultural elite. In this thesis we interpret the quote explicitly, that the thesis is finished at last! 6 7 Abstract HE drastically increased use of information and communications tech- nology has resulted in a growing demand for telecommunication net- T work capacity. The demand for radically increased network capacity coincides with industrial cost-reductions due to an increasingly competitive telecommunication market. In this thesis, we have addressed the capacity and cost-reduction problems in three ways. Our first contribution is a method to support shorter development cycles for new functionality and more powerful hardware. We reduce the development time by replicating the hardware utilization of production systems in our test environment. Having a realistic test environment allows us to run performance tests at early design phases and therefore reducing the overall system develop- ment time. Our second contribution is a method to improve the communication per- formance through selective and automatic message compression. The message compression functionality monitors transmissions continuously and selects the most efficient compression algorithm. The message compression functional- ity evaluates several parameters such as network congestion level, CPU usage, and message content. Our implementation extends the communication capacity of a legacy communication API running on Linux where it emulates a legacy real-time operating system. Our third and final contribution is a framework for process allocation and scheduling to allow higher system performance and quality of service. The framework continuously monitors selected processes and correlate their perfor- mance to hardware usage such as caches, floating point unit and similar. The framework uses the performance-hardware correlation to allocate processes on multi-core CPUs for minimizing shared hardware resource congestion. We have also designed a shared hardware resource aware process scheduler that allows multiple processes to co-exist on a CPU without suffering from performance degradation through hardware resource congestion. The allocation and schedul- ing techniques can be used to consolidate several functions on shared hardware v 8 thus reducing the system cost. We have implemented our process scheduler as a new scheduling class in Linux and evaluated it extensively. We have conducted several case studies in an industrial environment and verified all contributions in the scope of a large telecommunication system manufactured by Ericsson. We have deployed all techniques in a complicated industrial legacy system with minimal impact. We have shown that we can provide a cost-effective solution, which is an essential requirement for industrial systems. 9 10 11 Sammanfattning ELEKOMMUNIKATIONSBRANCHEN star˚ just nu infor¨ en stor ut- maning dar¨ kommunikationsprestanda och snabba leveranstider blir allt T mer viktiga for¨ att positionera sig i den okande¨ konkurrensen. I denna avhandling har vi addresserat detta problem pa˚ tre satt.¨ Det forsta¨ sattet¨ ar¨ att reducera utvecklingstiden genom att replikera hardvarulast˚ fran˚ produktionssys- tem pa˚ testnoder. Den andra genom att forb¨ attra¨ kommunikationsprestandan genom automatisk meddelandekomprimering. Den tredje, och sista, genom att implementera allokerings- och schemalaggningstekniker¨ som mojligg¨ or¨ kon- solidering av mjukvara pa˚ delad hardvara.˚ Vara˚ tekniker reducerar utvecklingstiden genom att flytta en del av pre- standaverifikationen fran˚ slutet av utvecklingscykeln till den mycket tidigare programmeringsfasen. Vi metod borjar¨ med att mata¨ resursanvandande¨ och pre- standan for¨ ett produktionssystem som kor¨ hos en kund. Fran˚ dessa matningar¨ skapar vi en modell som vi sedan anvander¨ for¨ att aterskapa˚ hardvarulasten˚ pa˚ en testnod. Att kora¨ funktionstester pa˚ ett testsystem som har liknande hardvarulast˚ ger ett tillforlitligt¨ resultat. Genom att anvanda¨ var˚ metod kan vi flytta vissa tester fran˚ prestandaverifikationen i slutet av utvecklingscykeln till program- meringsfasen och darmed¨ spara utvecklingstid. Under vara˚ tester markte¨ vi att kommunikationssystemet var overlastat¨ och att processorn inte anvandes¨ fullt ut. For¨ att oka¨ kommunikationsprestandan implementerade vi en metod som automatiskt komprimerar meddelanden nar¨ det finns processorkapacitet att anvanda.¨ Vi implementerade ett reglersystem som valjer¨ den basta¨ ur en mangd¨ av kompressionsalgoritmer. Var˚ mekanism utvarderar¨ automatiskt alla algoritmer och reagerar pa˚ for¨ andringar¨ i processor- last, natverkslast¨ eller meddelandeinnehall.˚ Av ekomiska skal¨ vill foretaget¨ konsolidera flera mjukvarufunktioner till en hardvara.˚ Nar¨ vi testade prestandan fore¨ och efter konsolidering markte¨ vi en synbar prestandafors¨ amring.¨ Orsaken till prestandafors¨ amringen¨ var att programmen som forut¨ korde¨ sjalva¨ nu skall dela pa˚ resurser sasom˚ cache och liknande. Vi har ocksa˚ utvecklat en teknik for¨ att automatiskt allokera processer ix 12 pa˚ ett kluster av karnor¨ for¨ att maximera prestandan. Vi utvecklade ocksa˚ en teknik for¨ att lata˚ flera processer dela pa˚ en karna¨ utan att for¨ den skull paverka˚ quality of service for¨ varandra. I var˚ implementation anvander¨ vi performance monitoring unit (PMU) for¨ att mata¨ resursanvandning.¨ Vi programmerar ocksa˚ PMU sa˚ att den genererar ett avbrott nar¨ en process har uttomt¨ sin tilldelade mangd¨ av resurser. All programvaruutveckling och test har genomforts¨ pa˚ ett industriellt tele- kommunikationssystem tillverkat av Ericsson. Alla tekniker ar¨ implementer- ade for¨ bruk i produktionsmiljo¨ och monitorerings- och modelleringsfunktion- aliteten anvands¨ kontinuerligt i felsokningsysfte¨ av produktionssystemet. De tekniker vi presenterar i denna avhandling ger ocksa˚ en kostnadseffektiv losning,¨ vilket ar¨ en viktigt krav for¨ industriella system. 13 14 15 I dedicate this thesis to my beloved wife Karolinn and my lovely daughters Amelie, Lovisa and Elise. [Art by L. Jagemar]¨ 16 17 Acknowledgements could not imagine the amazing personal journey it is when studying for a Ph.D. I utterly and completely underestimated the level of commitment I and the massive amount of work involved in finishing a Ph.D., as stated by Paulsen [227]: “The Ph.D. education is the highest education available - no wonder that it is demanding and difficult. Accept this.”. I have come to many insights during my studies. The most important personal insight is that this thesis is not a product of a gifted genius that shuffled through the studies and quickly finished it. It is something completely different. Writing a Ph.D. thesis, has for me, been much more of a constant challenge requiring long-term goals that can bridge short-term ups and downs. Having at least a tiny fraction of what is nowadays called grit [72, 73] is something that I discovered in the very final stages of writing this thesis. This determination saw me through seven years of part-time research interrupted with industrial work and several parental leaves. Working on this thesis is one of the most rewarding achievements in my professional life. I wish that all people would get the opportunity to fulfill their goals in the same way as I have. I could not have done this without help from many people. The following people have in various ways been involved in my Ph.D. project. First of all, I would like to thank my supervisors and co-authors, Bjorn¨ Lisper, Sigrid Eldh, Andreas Ermedahl and Moris Behnam for your knowledge, support, patience and constructive discussions during my studies. We have in many ways acted as a team with diverse competencies and achieved many exciting results. I would also like to express gratitude towards my manager at Ericsson, Magnus Schlyter, who supported me from the first day until completing the thesis. The work presented in this Ph.D. thesis has been funded by Ericsson and the Swedish Knowledge Foundation (KK stiftelsen) through the ITS-EASY [279] industrial Ph.D. school at Malardalen¨ University. xv 18 xvi Furthermore, thanks to all students in the ITS-EASY research group, we all share the ups and downs of studying for a PhD; Apala Ray, Daniel Hallmans, Daniel Kade, David Rylander, Eduard Paul Eniou, Fredrik Ekstrand, Gaetana Sapienza, Kristian Wiklund, Markus Wallmyr, Mehrdad Saadatmand, Melika Hozhabri, Sara Dersten, Stephan Baumgart, and Tomas Olsson. I would also like to thank my additional co-authors: Jakob Danielsson, Gordana Dodig-Crnkovic, Rafia Inam, Mikael Sjodin,¨ Daniel Hallmans, Stig Larsson and Thomas Nolte. I enjoyed working with you. I have the greatest gratitude to my parents; my mother and father who always wanted me to study hard to become something they never could. Finally and foremost, I want to express my endless love for my wife Karolinn and our three daughters, Amelie, Lovisa, and Elise. I would not have been able