HelenOSHelenOS –– OperatingOperating SystemSystem BuiltBuilt ofof MicroservicesMicroservices
http://www.helenos.org/http://d3s.mff.cuni.cz Martin Děcký [email protected] Motivation
Andy Tanenbaum at EuroBSDcon 2014, Sofia, Bulgaria Photo by Ollivier Robert “An operating system is said to be reliable when a typical user has never experienced even a single failure in his or her lifetime and does not know anybody who has ever experienced a failure.” [Tanenbaum 2014]
Andy Tanenbaum at EuroBSDcon 2014, Sofia, Bulgaria Photo by Ollivier Robert “An operating system is said to be reliable when a typical user has never experienced even a single failure in his or her lifetime and does not know anybody who has ever experienced a failure.” [Tanenbaum 2014]
“There are no demonstrated examples of highly secure or highly robust unstructured (monolithic) systems in the history of computing.” [Shapiro 2006]
Andy Tanenbaum at EuroBSDcon 2014, Sofia, Bulgaria Photo by Ollivier Robert SoSoftftwareware DependabilityDependability
IEEE definition “Dependability is a measurable and provable degree of system’s availability, reliability and its maintenance support.” Laprie J. C.: Dependable Computing and Fault Tolerance “Dependability is also affected by other measures, such as safety, security, integrity and confidentiality.”
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 7 CauCautitionaryonary Tale:Tale: Therac-25Therac-25
Radiotherapeutic medical device Derived from Therac-6 Two basic modes of operation Safety features in hardware instead of software 6 confirmed accidents between 1985 – 1987 3 confirmed deaths with a root cause of radiation burns Software race condition Poor software design and QA Misleading user interface Root cause: Poor understanding of software reliability issues
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 8 CauCautitionaryonary Tale:Tale: ArianeAriane 55
ESA heavy lift launch vehicle Derived from Ariane 4 A reliable and time-proven vehicle Exploded on its maiden voyage on June 4th 1996 39 seconds after lift-of $370 million in damage 64bit float containing velocity truncated to a 16bit integer in a non-critical software component Caused an uncaught exception that propagated to the control component A safety component triggered mission abort The non-critical component served no actual purpose
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 9 MicroservicesMicroservices
Wikipedia definition “A variant of service-oriented architecture (SOA) that structures an application as a collection of loosely coupled fine-grained services.” Benefits Improved modularity, understandability, verifiability Improved run-time fault isolation, anti-fragility
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 10 MicroservicesMicroservices
Not an entirely new idea ...
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 11 HelenOSHelenOS inin aa NutshellNutshell
application application application
server server server task task task
hardware open source general-purpose multiplatform microkernel multiserver operating system designed and implemented from scratch
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 12 Competition HelenOS HelenOSHelenOS inin aa NutshellNutshell
kernel concurrent synchro- kernel kernel hash console unit log nization remote remote tests table interface console framebuffer
console compositor ELF kernel tracing wait read- loader lifecycle support queues copy- vterm bdsh mgmt update clipboard audio client session input output lists, generic cache work trees, resource coherency spinlocks human interface bitmaps allocator queues
string misc slab address memory space nconfsrv slip routines routines allocator mgmt reservation dnsrsrv dhcp loopip ethip tcp udp
cycle & memory networking link layer transport system memory frame protocols information time backends zones allocator management layer protocols mgmt mgmt
architecture independent interrupt & hardware inetsrv syscall resource dispatch mgmt layer
thread & TMPFS Location FS task IPC mgmt global page hierarchical ISO 9660 UDF MINIX FS hash table page table support support FAT exFAT ext4 thread task scheduler capabilities file system device drivers
abstraction drivers shared shared
dependent platform debugging drivers support hardware shared architecture vfs device manager
platform interrupt platform I/O debugging location library service logger klog routines handling drivers mgmt support naming task service loader monitor init bootstrap CPU context platform atomics dependent architecture routines mgmt switching memory & mgmt barriers kernel
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 14 HelenOSHelenOS inin aa NutshellNutshell
kernel concurrent synchro- kernel kernel hash console unit log nization remote remote tests table interface console framebuffer
console compositor ELF kernel tracing wait read- loader lifecycle support queues copy- vterm bdsh mgmt update clipboard audio client session input output lists, ~generic 337,000cache physicalwork lines of code trees, resource coherency spinlocks human interface bitmaps allocator queues
string misc slab address memory space nconfsrv slip routines routines allocator mgmt reservation dnsrsrv dhcp loopip ethip tcp udp ~ 950cycle & person-monthsmemory worth oflink e layerfort networking transport system memory frame protocols information time backends zones allocator management layer protocols mgmt mgmt
architecture independent (Basic COCOMO Model) interrupt & hardware inetsrv syscall resource dispatch mgmt layer
thread & TMPFS Location FS task IPC mgmt global page hierarchical ISO 9660 UDF MINIX FS 24 master hashtheses, table page table 4 bachelor theses, support support FAT exFAT ext4 thread task scheduler capabilities file system device drivers
abstraction drivers shared shared 3 team projects,dependent platform debugging 11 GSoC and ESA SOCIS drivers support hardware shared architecture vfs device projects, independent contributors manager platform interrupt platform I/O debugging location library service logger klog routines handling drivers mgmt support naming task service loader monitor init bootstrap CPU context platform atomics dependent architecture routines mgmt switching memory & mgmt barriers kernel
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 15
BSD license AMD64, ARM, IA-32, IA-64, MIPS, PowerPC, SPARC Smart algorithms and data structures (RCU) Componentized networking stack (IPv4, IPv6) Audio stack (playing MOD, XM files) Composing desktop GUI TheThe CaseCase forfor “Reinven“Reinventitingng thethe Wheel”Wheel”
Clean-slate design Legacy designs and APIs may be broken, insecure, thread- unsafe, morally obsolete However, not saying that all legacy is broken Thinking out of the box
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 18
FormalFormal VeriVerifificacatitionon inin aa NutshellNutshell
OK
label transition system or
line 10: … line 14: … (open → close) model checker line 22: … temporal logic formula line 47: ... error trace
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 20 FormalFormal VeriVerifificacatitionon ExampleExample
Changeset: mainline,530 Commit log: Fix uninitialized field in error path Found by Clang Analyzer kernel/generic/src/main/kinit.c
if (init.tasks[i].addr % FRAME_SIZE) { printf(“init[%” PRIs “].addr is not frame aligned\n”, i); + programs[i].task = NULL; continue; }
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 21 ConclusionConclusion && FutureFuture WorkWork
State-of-the-art Solid and comprehensive research and development platform Viability demonstrated on several practical use cases Future work Performance tuning Focusing on commercialization
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 22 Q&A JoinJoin us!us!
www.helenos.org
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 24 Backup slides HelenOSHelenOS ContributorsContributors
Sean Bartell Štepán Henek Jan Mareš Jiří Svoboda Tomáš Benhák Vojtěch Horký Julia Medvedeva Agnieszka Tabaka Dmitry Bolkhovityanov Adam Hraška Lukáš Mejdrech Dominik Táborský Sergey Bondari Mohammed Hussain Vojtěch Mencl Jiří Tlach Tobias Börtitz Adrian Jamróz Jiří Michalec Lenka Trochtová Zdeněk Bouška Pavel Jančík Ondřej Palkovský Petr Tůma Tomáš Brambora Martin Jelen Vineeth Pillai Jakub Váňa Jan Buchar Petr Jerman Tim Post Radim Vansa Lubomír Bulej Jakub Jermář Vivek Prakash Laura-Mihaela Vasilescu Tomáš Bureš Fan Jinfei František Princ Ján Veselý Josef Čejka Jiří Kavalík Alexander Prutkov Jan Záloha Aurelio Colosimo Michal Kebrt Marin Ramesa Jiří Zárevúcky Manuele Conti Jakub Klama Pavel Římský Martin Děcký Matěj Klonfar Oleg Romanenko Matúš Dekánek Jan Kolárik Jef Rous Jan Dolejš Michal Konopa Thomas Sanchez Andrey Erokhin Petr Koupý Ondřej Šerý Matteo Facchinet Stanislav Kozina Ľuboš Slovák Beniamino Galvani Sandeep Kumar Antonín Steinhauser Matthieu Gueguen Maurizio Lombardi Petr Štěpán Zbigniew Halas Peter Majer Martin Sucha
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 26
DevelopmentDevelopment ProcessProcess
C Assember Python Shell Other 400000
350000
300000
250000
200000
150000 Physical lines of code of lines Physical
100000
50000
0 Feb 2005 Feb 2007 Feb 2009 Feb 2011 Feb 2013 Feb 2015
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 29 CommitsCommits
8000
7000
6000
5000
4000 Commits 3000
2000
1000
0 Feb 2005 Feb 2007 Feb 2009 Feb 2011 Feb 2013 Feb 2015
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 30 DownloadsDownloads
MINIX HelenOS 19.66 %
10000
Downloads 1000 (source: http://minix3.org/stats/) (source:
100 Feb 2007 Feb 2009 Feb 2011 Feb 2013 Feb 2015
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 31 www.microkernel.info AboutAbout UncertaintyUncertainty
Aleatory (irreducible) uncertainty Uncertainty in the world Unknown unknowns It is impossible to predict exactly how many heads will occur in 100 trials of tossing a coin Frequentist interpretation of probability needed Epistemic (reducible) uncertainty Uncertainty about the world Know unknows Given a coin, I do not know the probability of tossing a head But I can estimate it by doing experiments, learning about the physical properties of the coin, studying historical records about similar coins, etc.
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 33 AboutAbout UncertaintyUncertainty (2)(2)
Definition: Perfect software Software that will never experience a failure in operation, no matter how much operational exposure it has Definition: Possibly perfect software Software that might be perfect (we don’t know it) with some probability Probability of perfection
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 34 AboutAbout UncertaintyUncertainty (3)(3)
How the probability of perfection relates to reliability? Using the theorem of total probability P(SW fails [on a randomly selected demand]) = P(SW fails | SW is perfect) * P(SW is perfect) + P(SW fails | SW is imperfect) * P(SW is imperfect) The first term is zero, because the software does not fail if it is perfect Hence, define
– pnp .. probability the software is imperfect
– pfnp .. probability that software fails if it is imperfect
Then P(SW fails) ≤ pfnp * pnp
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 35 AboutAbout UncertaintyUncertainty (4)(4)
Interpreting P(SW fails) ≤ pfnp * pnp
pfnp and pnp are subjective probabilities (degree of belief) Conservative approximation
Assume software always fails if it is imperfect (pfnp = 1) Then P(SW fails) ≤ P(SW is imperfect) Conclusion Even the most crude validation / verification improves lowers the probability that the software is imperfect This is the upper bound of the probability that the software fails The confidence of the improvement can be even quantified This relieves formal verification tools from the burden of absolute perfection
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 36 ImageImage ReferencesReferences
Therac-25 photo & schematics, Troy Gallagher, included under the fair use doctrine Ariane 5, Ignis, Creative Commons Compartments and watertight subdivision, Andy Dingley, public domain Gears, susannp4, public domain
Martin Děcký, Internet a Technologie 17, June 20th 2017 HelenOS – Operating System Built of Microservices 37