Never forget .... Distributed Universal Constructions a guided tour Michel RAYNAL Institut Universitaire de France Academia Europaea IRISA, Universit´ede Rennes, France Polytechnic University (PolyU), Hong Kong DistributedUniversalConstructions 1 DistributedUniversalConstructions 2 Content Concurrent programming (1) Concurrent Programming: • Short historical perspective and a point of view Algorithms, Principles • From sequential computing to distributed computing and Foundations • Distributed universal constructions by Michel Raynal • Conclusion Springer, 531 pages, 2013 ISBN: 978-3-642-32026-2 DistributedUniversalConstructions 3 DistributedUniversalConstructions 4 Concurrent programming (2) Distributed Message-Passing (1) • Part 1: Lock-based synchronization (3 chap., pp. 1-110) • Part 2: The atomicity concept (1 chap., pp. 111-132) Distributed algorithms for • Part 3: Mutex-free synchronization (5 chap., pp. 133-274) Message-passing systems • Part 4: The transactional memory approach by Michel Raynal (1 chap., pp. 275-302) Springer, 517 pages, 2013 • Part 5: From safe bits to atomic registers (3 chap., pp. 303-368) ISBN 978-3-642-38122-5 • Part 6: The computability power of concurrent objects (4 chap., pp. 369-488) DistributedUniversalConstructions 5 DistributedUniversalConstructions 6 Distributed Message-Passing (2) PART 1 • Part 1: Distributed graph algorithms (5 chap., pp. 1-118) • Part 2: Logical time and global states (4 chap., pp. 119-244) • Part 3: Mutual exclusion and resource allocation Historical perspective (2 chap., pp. 244-300) and ... • Part 4: High level communication abstractions (2 chap., pp. 301-364) a point of view on what is • Part 5: Detection of properties of distributed executions INFORMATICS (2 chap., pp. 365-423) • Part 6: Distributed shared memory (2 chap., pp. 425-470) DistributedUniversalConstructions 7 DistributedUniversalConstructions 8 One upon a time... Plimpton tablet 322 (1800 BC) From the very beginning (?) 15 lines Pythagorean triplets mankind is (a2 + b2 = c2) Sexagesimal base Looking for UNIVERSALITY! Algorithms seem to be born with writing... (only receipes at this time, no formalization, no proofs) DistributedUniversalConstructions 9 DistributedUniversalConstructions 10 a few historical references A little bit later... • Neugebauer O. E., The exact sciences in Antiquity Princeton University Press (1952); 2nd edition: Brown A great step ahead! University Press (1957), Reprint: Dover publications (1969) Axioms: Euclid (≃ 300 BC) • Kramer S. N., History begins at Sumer: thirty-nine firsts in man’s “Ruler + compass” constructions recorded history “Ruler + compass” define the University of Pennsylvania Press, 416 pages (1956) set of allowed operations • Donald Knuth We have: algorithms + proofs Ancient Babylonian Algorithms Communications of the ACM, 15(7):671-677 (1972) DistributedUniversalConstructions 11 DistributedUniversalConstructions 12 Example: Bissecting an angle with compass + ruler BTW: what about trisecting an angle? B r1 • Is it possible to trisect an angle with compass + ruler? A A • One of the hardest pb for Ancient Greeks (squaring the r1 circle) C • Answer : impossibility proved in 1837 by Wantzel P. L.: Recherches sur les moyens de reconnaˆıtre si un probl`eme B B de g´eom´etrie peut se r´esoudre avec la r`egle et le com- r2 r1 r2 pas, Journal de math´ematiques pures et appliqu´ees, A A D 1(2):366-372 (1837) r2 r1 r2 Plus the fact that π is a transcendent number (F. von C C Lindemann 1882) • Hence ruler + compass operations are not universal for geometric constructions! Proof: consists in showing that the triangles ABD and ACD are equal DistributedUniversalConstructions 13 DistributedUniversalConstructions 14 Still a little bit later... A few references M. Ibn Musa Al Khawarizmi - Kitabu al-mukhtasar fi hisabi 780, Khiva - 850, Bagdad al-jabr wa’l-muqabala - Kitabu al-jami‘ wa’t-tafriq bi- hisabi ’l-Hind Contributed to algebra ... (the book of addition and substrac- but gave its name to algorithms! tion from Indian calculus) DistributedUniversalConstructions 15 DistributedUniversalConstructions 16 Closer to us Two great colleagues! 1936 1912-1954 - Turing A. M., On computable numbers with an applica- tion to the Entscheidungsproblem. Proc. of the London 1903-1995 1897-1954 Mathematical Society, 42:230-265 (1936) DistributedUniversalConstructions 17 DistributedUniversalConstructions 18 ALGORITHMICS A unifying view The science of operations Loking for universality! DIGITAL WORLD applications • Founding result: INFORMATICS ⋆ FSA ⊂ Pushdown Automata ⊂ Turing Machines Languages Systems ⋆ Machines to process SYMBOLS Artificial Int. ALGORITHMICS Data bases • Church-Turing Thesis: universal machines Computers • Universality of data representation : Etc. s´equences de bits (books, images, files, etc.) A very nice book by Harel D. and Feldman Y.: Algorithmics: the spirit of computing. 3rd edition Springer, 572 pagers (2012) [First edition: 1992] DistributedUniversalConstructions 19 DistributedUniversalConstructions 20 About informatics (1) About informatics (2) • Main resources: ⋆ up to mid of XX-th century: matter/energy ⋆ from mid of XX-th century: information • Produces a “new” way of thinking (algorithmics-based) • From putting the world into equations ⋆ as matter/energy: information can be collected, con- sumed, transformed, stored, carried, etc. to putting the world into algorithms ⋆ differentlty from matter/energy: it does not burn, it • Informatics is the language of science! can be copied at “zero cost” • Looking for universality (just repeating...) DistributedUniversalConstructions 21 DistributedUniversalConstructions 22 PART 2 The basic unit of sequential computing in f() out = f(in) • The notion of a function From sequential computing • Sequential algorithm to distributed computing • The notion of computability (Turing machine) • The notion of impossibility (e.g., halting problem) • The fundamental hierarchy FSA ⊂ Pushdown Automata ⊂ Turing Machines • Church-Turing’s Thesis DistributedUniversalConstructions 23 DistributedUniversalConstructions 24 The case of parallel computing What is distributed computing? • We look inside the box implementing f() ⋆ mono-processor ⋆ multiprocessor : to be more efficient DC arises when one has to solve a problem in terms of entities (processes, agents, sensors, peers, actors, nodes, processors, ...) such that each entity has only a partial • The problem could ALWAYS be solved by a sequential algorithm, but can be solved more efficiently with sev- knowledge of the many parameters involved in the problem eral computing entities that has to be solved • Parallel computing is an “extension” of sequential com- puting looking for efficiency DC is about Mastering UNCERTAINTY • This has a long story and introduced new techniques and concepts (e.g., task graphs, scheduling, etc.) DistributedUniversalConstructions 25 DistributedUniversalConstructions 26 The basic unit of distributed computing The notion of a (distributed) task T () • A task T is a triple (I, O, ∆) Output O ∈ T (I) Input I [out1, · · · ,outn] ⋆ I: set of input vectors (of size n) pi [ ] in outi in1, · · · , inn i ⋆ O: set of output vectors (of size n) ⋆ ∆: relation from I into O: ∀I ∈ I : ∆(I) ⊆O T () is a relation • I[i]: private input of pi • O[i]: private output of pi • The notion of a task: from an input vector to an output • ∀I ∈ I: • The inputs are DISTRIBUTED ∆(I) defines the set of legal output vectors that can (this is not under the control of the algorithm designer) be decided from the input vector I • Failures belong to the model (in nearly all cases) DistributedUniversalConstructions 27 DistributedUniversalConstructions 28 Examples of tasks Solving a task • Binary consensus A distributed algorithm A is a set of n local automata (Turing machines) that cooperate through specific com- ⋆ I = {all vectors of 0 and 1} munication objects (e.g., message-passing network, shared memory, etc.) ⋆ O = n{0,..., 0}, {1,..., 1}o ⋆ Let X = {0,..., 0} and X = {1,..., 1} 0 0 An algorithm A solves a task T if in any run ∗ ∆(X0) = {0,..., 0} and ∆(X1) = {1,..., 1} ∗ ∆(any vector except XO, X1)= O • ∃ I ∈ I such that each pi starts with (proposes) ini = I[i] • k-set agreement, Renaming, Weak symmetry breaking • ∃ O ∈ ∆(I) such that O[j] = outj for each process pj • k-Simultaneous consensus, etc. that that computes (decides) an output outj DistributedUniversalConstructions 29 DistributedUniversalConstructions 30 Distributed computing: birth certificates A famous quote ... and its formalization • L. Lamport, Time, clocks, and the ordering of events in a distributed system. Communications of the ACM, 21(7):558-565 (1978) • “A distributed system is one in which the failure of a computer you didn’t even know existed can render your ⋆ Partial order on events own computer unusable” (L. Lamport) ⋆ Scalar clocks • Fischer M.J., Lynch N.A., and Paterson M.S., ⋆ State machine replication Impossibility of distributed consensus with one faulty process. • Fischer M.J., Lynch N.A., and Paterson M.S., Impossi- Journal of the ACM, 32(2):374-382 (1985) bility of distributed consensus with one faulty process. Journal of the ACM, 32(2):374-382 (1985) ⋆ Impossibilty result in asynch. crash-prone systems Reminder: DC is about Mastering UNCERTAINTY! ⋆ Notion of valence (captures non-determinism) DistributedUniversalConstructions 31 DistributedUniversalConstructions 32 To summarize PART 3 • Real-time: masters On-time computing • Parallelism: provides Efficiency • Distributed computing: masters Uncertainty Universal constructions