A Branch-And-Merge Approach to Weak Consistency

Home , Sayeret Matkal

TARDIS: A BRANCH-AND- MERGE APPROACH TO WEAK CONSISTENCY Natacha Crooks, Youer Pu, Nancy Estrada, Trinabh Gupta, Lorenzo Alvisi, and Allen Clement. Presented by Tanya Brokhman Agenda

■ What the issue we’re trying to address? ■ TARDiS design ■ Usage example ■ Evaluation Wikipedia: Benjamin Netanyahu Alice Bob

בִּ נְיָמִּ ין Benjamin "Bibi" Netanyahu (Hebrew: About this sound בִּ נְיָמִּ ין Benjamin "Bibi" Netanyahu (Hebrew: About this sound help·info); born 21 October 1949) is the current )"בִּ יבִּ י" נְתַ נְיָהּו help·info); born 21 October 1949) is the current )"בִּ יבִּ י" נְתַ נְיָהּו Prime Minister of Israel. Netanyahu also currently serves as a Prime Minister of Israel. Netanyahu also currently serves as a member of the Knesset and Chairman of the Likud party. member of the Knesset and Chairman of the Likud party.

Born in Tel Aviv to secular Jewish parents,[1][2] Netanyahu is Born in Tel Aviv to secular Jewish parents,[1][2] Netanyahu is the first Israeli prime minister born in Israel after the the first Israeli prime minister born in Israel after the establishment of the state. establishment of the state. Netanyahu joined the Israel Defense Forces shortly after the Six- Netanyahu joined the Israel Defense Forces shortly after the Six- Day War in 1967, and became a team leader in the Sayeret Day War in 1967, and became a team leader in the Sayeret Matkal special forces unit. Netanyahu took part in many … Matkal special forces unit. Netanyahu took part in many …

References: References: 1. David Remnick (23 January 2013). "Bibi's Blues". The New 1. David Remnick (23 January 2013). "Bibi's Blues". The New Yorker. Yorker. 2. "The Enduring Influence of Benjamin Netanyahu's Father". 2. "The Enduring Influence of Benjamin Netanyahu's Father". 3. Amir Buhbut, "Sayeret Matkal is 50 years old“ 3. Amir Buhbut, "Sayeret Matkal is 50 years old“ 4. Saving Sergeant Netanyahu By Mitch Ginsburg, 25 October 4. Saving Sergeant Netanyahu By Mitch Ginsburg, 25 October 2012, Times of Israel 2012, Times of Israel 5. … 5. …

Site A Site B Wikipedia: Benjamin Netanyahu Alice Bob

Site A Site B Wikipedia: Benjamin Netanyahu Kate Bruno

Site A Site B Wikipedia: Benjamin Netanyahu

Born in Tel Aviv to secular Jewish parents,[1][2] Netanyahu is the first Israeli prime minister born in Israel after the establishment of the state. Netanyahu joined the Israel Defense Forces shortly after the Six- Day War in 1967, and became a team leader in the Sayeret Matkal special forces unit. Netanyahu took part in many …

“The data consistency model specifies a contract between programmer and system, wherein the system guarantees that if the programmer follows the rules, memory will be consistent and the results of memory operations will be predictable.” Wikipedia ■ Week consistency: – On READ the (physically) closest version of the data returned. – When to use: getting any data at all is more important than getting the most up-to date data. ■ Strong consistency: – On READ the most recent version of the data returned. – When to use: the most up-to-date and consistent information is essential for applications CAP theorem

“ In the presence of a network partition, one has to choose between consistency and availability” ■ Consistency: Every read receives the most recent write or an error – READ may be blocked ■ Availability: Every request receives a (non-error) response – without guarantee that it contains the most recent write What to do with write-skew anomalies?

בִּ נְיָמִּ ין Benjamin "Bibi" Netanyahu (Hebrew: About this sound help·info); born 21 October 1949) is the current )"בִּ יבִּ י" נְתַ נְיָהּו ■ Preventing them is impossible Prime Minister of Israel. Netanyahu also currently serves as a member of the Knesset and Chairman of the Likud party. ■ So is resolving them transparently Born in Tel Aviv to secular Jewish parents,[1][2] Netanyahu is the first Israeli prime minister born in Israel after the establishment of the state. – Application-specific knowledge is often indispensable Netanyahu joined the Israel Defense Forces shortly after the Six- Day War in 1967, and became a team leader in the Sayeret Matkal special forces unit. Netanyahu took part in many …

References: 1. David Remnick (23 January 2013). "Bibi's Blues". The New Yorker. 2. "The Enduring Influence of Benjamin Netanyahu's Father". Provide applications with the best possible system 3. Amir Buhbut, "Sayeret Matkal is 50 years old“ 4. Saving Sergeant Netanyahu By Mitch Ginsburg, 25 October 2012, Times of Israel support when merging conflicting states 5. … ■ Transactional

■ Asynchronously

■ Replicated TARDiS

■ Divergent

■ Store State DAG

■ In case of write-conflicts TARDiS preserves availability by: I. Branch on conflict II. Inter branch isolation III. Application driven cross-object merge How TARDiS works?

■ Consistency layer ■ Storage layer ■ Garbage collector ■ Replicator Service How TARDiS works? The Storage layer: ■ Content: Key = C ■ Picture: Key = P ■ Reference: Key = R בִּ נְיָמִּ ין Benjamin "Bibi" Netanyahu (Hebrew: About this sound help·info); born 21 October 1949) is the current )"בִּ יבִּ י" נְתַ נְיָהּו Prime Minister of Israel. Netanyahu also currently serves as a Key Value member of the Knesset and Chairman of the Likud party. Born in Tel Aviv to secular Jewish parents,[1][2] Netanyahu is the first Israeli prime minister born in Israel after the C Line1, line2, line3… line_n establishment of the state. Netanyahu joined the Israel Defense Forces shortly after the Six- Day War in 1967, and became a team leader in the Sayeret P Pic.jpg Matkal special forces unit. Netanyahu took part in many … R Ref1, ref2, ref3…ref_n

References: 1. David Remnick (23 January 2013). "Bibi's Blues". The New Yorker. 2. "The Enduring Influence of Benjamin Netanyahu's Father". 3. Amir Buhbut, "Sayeret Matkal is 50 years old“ 4. Saving Sergeant Netanyahu By Mitch Ginsburg, 25 October 2012, Times of Israel 5. … בִּ נְיָמִּ ין Benjamin "Bibi" Netanyahu (Hebrew: About this sound help·info); born 21 October 1949) is the current )"בִּ יבִּ י" נְתַ נְיָהּו Prime Minister of Israel. Netanyahu also currently serves as a How TARDiS works? member of the Knesset and Chairman of the Likud party. Born in Tel Aviv to secular Jewish parents,[1][2] Netanyahu is the first Israeli prime minister born in Israel after the establishment of the state. Netanyahu joined the Israel Defense Forces shortly after the Six- The Storage layer: Day War in 1967, and became a team leader in the Sayeret Matkal special forces unit. Netanyahu took part in many … ■ Content: Key = C ■ Picture: Key = P ■ Reference: Key = R

References: 1. David Remnick (23 January 2013). "Bibi's Blues". The New Key Version Value Yorker. 2. "The Enduring Influence of Benjamin Netanyahu's Father". 3. Amir Buhbut, "Sayeret Matkal is 50 years old“ C 0 Line1, line2, line3… line_n 4. Saving Sergeant Netanyahu By Mitch Ginsburg, 25 October 2012, Times of Israel C 1 Line1, line2, line3… line_n 5. … C 2 Line1, line2, line3… line_n P 0 Pic.jpg Key <-> Version mapping R 0 Ref1, ref2, ref3…ref_n C: 2,1,0 R 1 Ref1, ref2, ref3…ref_n P: 0 R: 1,0 How TARDiS works? The Consistency layer: ■ Vertices in DAG correspond to logical states of

בִּ נְיָמִּ ין Benjamin "Bibi" Netanyahu (Hebrew: About this sound help·info); born 21 October 1949) is the current )"בִּ יבִּ י" נְתַ נְיָהּו the datastore Prime Minister of Israel. Netanyahu also currently serves as a member of the Knesset and Chairman of the Likud party.

Born in Tel Aviv to secular Jewish parents,[1][2] Netanyahu is the first Israeli prime minister born in Israel after the establishment of the state. Netanyahu joined the Israel Defense Forces shortly after the ■ Each transaction that updates a record Six-Day War in 1967, and became a team leader in the Sayeret generates a new state Matkal special forces unit. Netanyahu took part in many …

Key Version Value References: 1. David Remnick (23 January 2013). "Bibi's Blues". The New Yorker. 2. "The Enduring Influence of Benjamin Netanyahu's Father". 3. Amir Buhbut, "Sayeret Matkal is 50 years old“ C 0 Line1, line2, line3… line_n 4. Saving Sergeant Netanyahu By Mitch Ginsburg, 25 October 2012, Times of Israel 5. … C 1 Line1, line2, line3… line_n

בִּ נְיָמִּ ין Benjamin "Bibi" Netanyahu (Hebrew: About this sound help·info); born 21 October 1949) is the current )"בִּ יבִּ י" נְתַ נְיָהּו Prime Minister of Israel. Netanyahu also currently serves as a C 2 Line1, line2, line3… line_n member of the Knesset and Chairman of the Likud party. Born in Tel Aviv to secular Jewish parents,[1][2] Netanyahu is the first Israeli prime minister born in Israel after the establishment of the state. Netanyahu joined the Israel Defense Forces shortly after the Six-Day War in 1967, and became a team leader in the Sayeret R 1 Ref1, ref2, ref3…ref_n Matkal special forces unit. Netanyahu took part in many …

Can read-only transactions be observed in TARDiS? Basic operations with TARDiS

■ Begin transaction – BFS from the leaves up through the DAG to select a “read state” (6a) ■ Commit transaction (write operation): – Go down the DAG to find the most recent state that satisfies end constraint (6b) – Create a new branch (6c) Basic operations with TARDiS

■ Reading records – Locate the latest record version on the selected branch – Read it from the records B-tree Evaluation

Overhead of tracking states (keeping history) Impact of branching on performance How does the system scale for multiple cites?

Setup: ■ BerkleyDB (BDB) v6.2.31: – Supports concurrent database manipulations – Distributed highly available single-master multi-replica database ■ TARDiS-BDB (baseline): – TARDiS configuration that mimics the logical behavior of a traditional sequential storage ■ Each application sees its own writes and the execution is serializable Overhead of tracking states

Despite tracking system history and selecting begin/commit states, TARDiS-BDB performs similar to BDB Impact of branching on performance How the system scales for multiple cites?

■ Tested with Google Cloud services cluster of 3 machines at different geographical locations. Summary

■ TARDiS is a transactional key-value store explicitly designed for weakly-consistent systems ■ It presents new concurrency control mechanism: branch-on-conflict ■ TARDiS keeps history, helping the applications to best resolve any arising write-write conflicts ■ TARDiS coding is simple and improves application performance

Thank you for listening!