DOCSLIB.ORG

Building Large Tarantool Cluster with 100+ Nodes Yaroslav Dynnikov

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Building Large Tarantool Cluster with 100+ Nodes Yaroslav Dynnikov Building large Tarantool cluster with 100+ nodes Yaroslav Dynnikov Tarantool, Mail.Ru Group 10 October 2019 Slides: rosik.github.io/2019-bigdatadays 1 / 40 Tarantool = + Database Application server (Lua) (Transactions, WAL) (Business logics, HTTP) 2 / 40 Core team 20 C developers Product development Solution team 35 Lua developers Commertial projects 3 / 40 Core team 20 C developers Product development Solution team 35 Lua developers Commertial projects Common goals Make development fast and reliable 4 / 40 In-memory no-SQL Not only in-memory: vinyl disk engine Supports SQL (since v.2) 5 / 40 In-memory no-SQL Not only in-memory: vinyl disk engine Supports SQL (since v.2) But We need scaling (horizontal) 6 / 40 Vshard - horizontal scaling in tarantool Vshard assigns data to virtual buckets Buckets are distributed across servers 7 / 40 Vshard - horizontal scaling in tarantool Vshard assigns data to virtual buckets Buckets are distributed across servers 8 / 40 Vshard - horizontal scaling in tarantool Vshard assigns data to virtual buckets Buckets are distributed across servers 9 / 40 Vshard - horizontal scaling in tarantool Vshard assigns data to virtual buckets Buckets are distributed across servers 10 / 40 Vshard - horizontal scaling in tarantool Vshard assigns data to virtual buckets Buckets are distributed across servers 11 / 40 Vshard - horizontal scaling in tarantool Vshard assigns data to virtual buckets Buckets are distributed across servers 12 / 40 Vshard configuration Lua tables sharding_cfg = { ['cbf06940-0790-498b-948d-042b62cf3d29'] = { replicas = { ... }, }, ['ac522f65-aa94-4134-9f64-51ee384f1a54'] = { replicas = { ... }, }, } vshard.router.cfg(...) vshard.storage.cfg(...) 13 / 40 Vshard automation. Options Deployment scripts Docker compose Zookeeper 14 / 40 Vshard automation. Options Deployment scripts Docker compose Zookeeper Our own orchestrator 15 / 40 Orchestrator requirements Doesn't start/stop instances (systemd can do that) Applies configuration only Every cluster node can manage others Built-in monitoring 16 / 40 Clusterwide configuration Must be the same everywhere Applied with two-phase commit topology: servers: # two servers s1: replicaset_uuid: A uri: localhost:3301 s2: replicaset_uuid: A uri: localhost:3302 replicasets: # one replicaset A: roles: ... 17 / 40 Which came first? Database Orchestrator tcp_listen() apply_2pc() 18 / 40 Membership implementation SWIM protocol - one of the gossips protocols family 19 / 40 Membership implementation SWIM protocol - one of the gossips protocols family 20 / 40 Membership implementation SWIM protocol - one of the gossips protocols family 21 / 40 Membership implementation SWIM protocol - one of the gossips protocols family 22 / 40 Membership implementation SWIM protocol - one of the gossips protocols family 23 / 40 Membership implementation SWIM protocol - one of the gossips protocols family Dissemination speed: O(logN) Network load: O(N) 24 / 40 Membership implementation SWIM protocol - one of the gossips protocols family Dissemination speed: O(logN) Network load: O(N) PING ! ! + payload ACK ! ! 25 / 40 Bootsrapping new instance 1. New process starts 2. New process joins membership 3. Cluster checks new process is alive 4. Cluster applies configuration 5. New process polls it from membership 6. New process bootstraps 7. Repeat N times 26 / 40 Bootsrapping new instance 1. New process starts 2. New process joins membership 3. Cluster checks new process is alive 4. Cluster applies configuration 5. New process polls it from membership 6. New process bootstraps 7. Repeat N times N = 100 27 / 40 Benefits so far 1. Orchestration works 28 / 40 Benefits so far 1. Orchestration works 29 / 40 Benefits so far 1. Orchestration works 30 / 40 Benefits so far 1. Orchestration works 2. Monitoring works 31 / 40 Benefits so far 1. Orchestration works 2. Monitoring works 32 / 40 Benefits so far 1. Orchestration works 2. Monitoring works 3. We can assign any role to the instance 33 / 40 Benefits so far 1. Orchestration works 2. Monitoring works 3. We can assign any role to the instance 34 / 40 Role management function init() 35 / 40 Role management function init() function validate_config() function apply_config() 36 / 40 Role management function init() function validate_config() function apply_config() function stop() 37 / 40 Refactoring the bootstrap process Assembling large clusters with 100+ instances is slow N two-phase commits are slow N config pollings is slow 38 / 40 Refactoring the bootstrap process Assembling large clusters with 100+ instances is slow N two-phase commits are slow N config pollings is slow Solution Bootstrap all instances with a single 2pc Re-implement binary protocol and reuse port 39 / 40 Links tarantool.io github.com/tarantool/tarantool Telegram - @tarantool, @tarantool_news Cartridge framework - github.com/tarantool/cartridge Cartridge CLI - github.com/tarantool/cartridge-cli Posts on Habr - habr.com/users/rosik/ This presentation - rosik.github.io/2019-bigdatadays Questions? 40 / 40.
Load more

Recommended publications
  • Tarantool Enterprise Manual Release 1.10.4-1
    Tarantool Enterprise manual Release 1.10.4-1 Mail.Ru, Tarantool team Sep 28, 2021 Contents 1 Setup 1 1.1 System requirements.........................................1 1.2 Package contents...........................................2 1.3 Installation..............................................3 2 Developer’s guide 4 2.1 Implementing LDAP authorization in the web interface.....................5 2.2 Delivering environment-independent applications.........................5 2.3 Running sample applications....................................8 3 Cluster administrator’s guide 11 3.1 Exploring spaces........................................... 11 3.2 Upgrading in production...................................... 13 4 Security hardening guide 15 4.1 Built-in security features...................................... 15 4.2 Recommendations on security hardening............................. 17 5 Security audit 18 5.1 Encryption of external iproto traffic................................ 18 5.2 Closed iproto ports......................................... 18 5.3 HTTPS connection termination.................................. 18 5.4 Closed HTTP ports......................................... 19 5.5 Restricted access to the administrative console.......................... 19 5.6 Limiting the guest user....................................... 19 5.7 Authorization in the web UI.................................... 19 5.8 Running under the tarantool user................................. 20 5.9 Limiting access to the tarantool user...............................
    [Show full text]
  • LIST of NOSQL DATABASES [Currently 150]
    Your Ultimate Guide to the Non - Relational Universe! [the best selected nosql link Archive in the web] ...never miss a conceptual article again... News Feed covering all changes here! NoSQL DEFINITION: Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open-source and horizontally scalable. The original intention has been modern web-scale databases. The movement began early 2009 and is growing rapidly. Often more characteristics apply such as: schema-free, easy replication support, simple API, eventually consistent / BASE (not ACID), a huge amount of data and more. So the misleading term "nosql" (the community now translates it mostly with "not only sql") should be seen as an alias to something like the definition above. [based on 7 sources, 14 constructive feedback emails (thanks!) and 1 disliking comment . Agree / Disagree? Tell me so! By the way: this is a strong definition and it is out there here since 2009!] LIST OF NOSQL DATABASES [currently 150] Core NoSQL Systems: [Mostly originated out of a Web 2.0 need] Wide Column Store / Column Families Hadoop / HBase API: Java / any writer, Protocol: any write call, Query Method: MapReduce Java / any exec, Replication: HDFS Replication, Written in: Java, Concurrency: ?, Misc: Links: 3 Books [1, 2, 3] Cassandra massively scalable, partitioned row store, masterless architecture, linear scale performance, no single points of failure, read/write support across multiple data centers & cloud availability zones. API / Query Method: CQL and Thrift, replication: peer-to-peer, written in: Java, Concurrency: tunable consistency, Misc: built-in data compression, MapReduce support, primary/secondary indexes, security features.
    [Show full text]
  • SCHEDULE SCHEDULE Thanks to Our Sponsors
    CONFERENCE CONFERENCE AND TUTORIAL AND TUTORIAL SCHEDULE SCHEDULE Thanks to our sponsors: This is the interactive guide to Percona Live Europe 2019. Links are clickable, including links back to the timetables at the bottom of the pages. You can register at www.percona.com/live-registration Sections Daily Schedules Talks by Technology Keynotes Monday Tutorials Tuesday Talks Wednesday Talks Speakers Spot talks by technology in the timetable: a color key MySQL MongoDB PostgreSQL Other Databases, Multiple Databases, and Other Topics TUTORIALS DAY MONDAY, SEPTEMBER 30 ROOM ROOM ROOM ROOM A B 7 26 second floor PostgreSQL For Oracle and Accelerating Application MySQL 8.0 InnoDB Cluster: 9:00 MySQL DBAs and MySQL 101 Tutorial Part 1 Development with 9:00 Easiest Tutorial! For Beginners Amazon Aurora 12:00 LUNCH 12:00 Innodb Architecture and Accelerating Application Introduction to 1:30 Performance Optimization MySQL 101 Tutorial Part 2 Development with 1:30 PL/pgSQL Development Tutorial for MySQL 8 Amazon Aurora 4:30 WELCOME RECEPTION – EXPO HALL OPEN UNTIL 6PM 4:30 ROOM ROOM ROOM 8 9 10 Open Source Database MariaDB Server 10.4: Percona XtraDB Cluster 9:00 Performance Optimization 9:00 The Complete Tutorial Tutorial and Monitoring with PMM 12:00 LUNCH 12:00 Test Like a Boss: Deploy and A Journey with MongoDB HA. Getting Started with 1:30 Test Complex Topologies From Standalone to Kubernetes and Percona 1:30 With a Single Command Kubernetes Operator XtraDB Cluster 4:30 WELCOME RECEPTION – EXPO HALL OPEN UNTIL 6PM 4:30 TUESDAY, OCTOBER 1 ROOM Building
    [Show full text]
  • Alexander Turenko
    Alexander Turenko • Date of birth: 1992-04-28. • Education: Lomonosov Moscow State University, Faculty of Computational Mathematics and Cybernetics, 2009{2015. • Jobs (from older to more recent): • Tarantool DBMS, March, 2017 |Now. • Tarantool Server Team. Working on tarantool itself, modules and connectors. Relatively valuable tasks were graphql module and merger builtin module. • Solution Engineering Team. A point-to-point money transfer service for banks. The core idea is to provide an API and a service to perform money transfers using a phone number or an other ID. Developed from a first line of code to production (started as the single member, then working in a team of three members). • Intel, compilers performance analysis, March, 2015 | July, 2016. • Analyzing of code performance / code size / compile time degradations. • Creating and supporting perf. analysis tools. • Got C/C++ compilers background, touched LLVM/Clang. • Notable open-source projects (out of paid work): • BombusMod. J2ME & Android Jabber client. Integrating juick.com microservice, improving hotkeys, lots of small improvements. • Tkabber plugins. Desktop Jabber client written in Tcl/Tk. Juick and notes plugins. • Whatifrussian. Translations of Randall's Monroe popular scientific articles. Proof-reading and translating, Python & JS programming, contributing a bit to projects we use (Pelican, Zepto). • Area of interest: System programming. • Programming languages: • C, Lua, Java (J2SE & J2ME), C++ (mostly C++03 w/o STL), x86 assembler (Intel and AT&T syntax both), sh (and bash), Python, Tcl/Tk. • Fuzzy experience with: Pascal, GLSL, Refal5, SWI Prolog, Lisp, Haskell, PHP, JavaScript, OpenMP, MPI. • Human languages: • Russian | native. • English | intermediate (reading and writing documentation, technical discussions).
    [Show full text]
  • Tarantool Team's Experience with Lua Developer Tools
    03/03/2019 Tarantool team's experience with Lua developer tools Tarantool team's experience with Lua developer tools Yaroslav Dynnikov Tarantool, Mail.Ru Group 3 March 2019 http://localhost:8000/#1 11 //25 25 03/03/2019 Tarantool team's experience with Lua developer tools Tarantool Tarantool is an open-source data integration platform Tarantool = Database + Application server (Lua) http://localhost:8000/#1 22 //25 25 03/03/2019 Tarantool team's experience with Lua developer tools Tarantool Tarantool is an open-source data integration platform Tarantool = Database + Application server (Lua) Core team Focuses on the product development Solution team Implements projects for the Enterprise http://localhost:8000/#1 33 //25 25 03/03/2019 Tarantool team's experience with Lua developer tools Tarantool Solution Engineering 35 Lua developers ~ 50 Git repos ~ 300,000 SLoC Customers IT, Banking, Telecom, Oil & Gas Goals Develop projects fast and well http://localhost:8000/#1 44 //25 25 03/03/2019 Tarantool team's experience with Lua developer tools Writing better code http://localhost:8000/#1 55 //25 25 03/03/2019 Tarantool team's experience with Lua developer tools Development The sooner code fails - the better - Runtime checks local function handle_stat_request(req) local stat = get_stat() return { status = 200, body = json.encode(stet), } end handle_stat_request() -- returns: "null" http://localhost:8000/#1 66 //25 25 03/03/2019 Tarantool team's experience with Lua developer tools Development The sooner code fails - the better - Runtime checks
    [Show full text]
  • 3 Generic Programming with Mutually Recursive Types 27 3.1 the Generics-Mrsop Library
    2 Type-Safe Generic Differencing of Mutually Recursive Families Getypeerde Generieke Differentiatie van Wederzijds Recursieve Datatypes (met een samenvatting in het Nederlands) Proefschrift ter verkrijging van de graad van doctor aan de Universiteit Utrecht op gezag van de rector magnificus, prof. dr. H.R.B.M. Kummeling, ingevolge het besluit van het college voor promoties in het openbaar te verdedigen op maandag 5 oktober 2020 des ochtends te 11:00 uur door Victor Cacciari Miraldo geboren op 16 oktober 1991 te São Paulo, Brazilië Promotor: Prof.dr. G.K. Keller Copromotor: Dr. W.S. Swierstra Dit proefschrift werd (mede) mogelijk gemaakt met financiële steun van de Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO), project Revision Control of Structured Data (612.001.401). To my Mother, Father and Brother Abstract The UNIX diff tool – which computes the differences between two files in terms ofa set of copied lines – is widely used in software version control. The fixed lines-of-code granularity, however, is sometimes too coarse and obscures simple changes, i.e., renam- ing a single parameter triggers the whole line to be seen as changed. This may lead to unnecessary conflicts when unrelated changes occur on the same line. Consequently, it is difficult to merge such changes automatically. In this thesis we discuss two novel approaches to structural differencing, generically – which work over a large class of datatypes. The first approach defines a type-indexed representation of patches and provides a clear merging algorithm, but it is computation- ally expensive to produce patches with this approach. The second approach addresses the efficiency problem by choosing an extensional representation for patches.
    [Show full text]
  • Tarantool Get Started Guide Draft 8
    L A U N C H Y O U R O W N R E S T F U L S E R V I C E I N 9 EASY STEPS Q U I C K S T A R T G U I D E 1 Tarantool has the great advantage of being an entirely self-sufficient backend solution, thus no external language like Node or PHP is necessary. This is because it includes a Lua application server that runs concurrent to its database server. In this tutorial, you will set up a basic Tarantool server on Ubuntu Linux using Digital Ocean. This responds to a route parameter with a user’s home city and returns that user’s name with a hello message. With some adjustments, this can also be accomplished on localhost using OSX, Linux, or Windows with WSL. S T A R T H E R E Sign up for Digital Ocean and create a basic Ubuntu droplet. Once you have its IP address, use SSH to access your instance. If you need help with setting up or 1. accessing your droplet, have a look at the beginning of the tutorial here. Once you have logged in, copy the script from https://tarantool.org/en/download/os-installation/1.7/ubuntu.html and paste it into 2. your terminal all at once. You may need to press enter again for the last line. After that we use apt-get to download an add-on Tarantool http module: 3. sudo apt-get install tarantool-http www.tarantool.io 1 Next we will set up an empty Tarantool database.
    [Show full text]
  • FOSDEM 2020 Schedule
    FOSDEM 2020 - Saturday 2020-02-01 (1/15) Janson K.1.105 (La H.2215 (Ferrer) H.1302 (Depage) H.1308 (Rolin) H.1309 (Van Rijn) H.2213 H.2214 Fontaine)… 09:30 Welcome to FOSDEM 2020 09:45 10:00 The Linux Kernel: We How FOSS could have to finish this thing revolutionize municipal one day ;) government 10:15 10:30 State of OpenJDK Fundamental DNS Devroom Opening Designing and Welcome to the MySQL, Technologies We Need DNS Management in Producing Open Source MariaDB & Friends D… to Work on for Cloud- OpenStack Hardware with MySQL 8 vs MariaDB Native Networking FOSS/OSHW tools 10:45 10.4 LibrePCB Status Update 11:00 LibreOffice turns ten The Selfish Contributor and what's next Explained Skydive HashDNS and MyRocks in the Wild 11:15 FQDNDHCP Wild West! Project Loom: Advanced Open-source design concurrency for fun and ecosystems around 11:30 profit Do you really see what’s FreeCAD happening on your NFV infrastructure? How Safe is 11:45 State of djbdnscurve6 Asynchronous Master- TornadoVM: A Virtual Master Setup? Machine for Exploiting ngspice open source High-Performance 12:00 Over Twenty Years Of The Ethics Behind Your Civil society needs Free circuit simulator Heterogeneous Automation IoT Software hackers Execution of Java Programs Endless Network Testing DoH and DoT The consequences of 12:15 Programming − An servers, compliance and sync_binlog != 1 Update from eBPF Land performance A tool for Community ByteBuffers are dead, Towards CadQuery 2.0 Supported Agriculture long live ByteBuffers! ↴ 12:30 (CSA) management, … Replacing iptables with eBPF
    [Show full text]
  • Watcher Release 0.2.0
    watcher Release 0.2.0 Raciel Hernandez B. May 25, 2021 FIRST STEPS 1 First steps 3 1.1 Prerequisites...............................................3 1.2 Quick Instalation.............................................4 1.3 Watcher features.............................................5 2 Getting started with Watcher 17 3 Step-by-step Guides 19 4 Advanced features of Watcher 21 5 Watcher project and organization 23 i ii watcher, Release 0.2.0 Watcher simplifies the integration of non-connected systems by detecting changes in data and facilitates thedevel- opment of monitoring, security and process automation applications. Think of Watcher as an intercom or a bridge between different servers or between different applications on the same server. Or you can simply take advantage of Watcher’s capabilities to develop your project. “Watch everything” Currently the functionality of detecting changes in the file system is implemented. However, the project has a larger scope and we invite you to collaborate with us to achieve the goal of “Watch Everything”. One step at a time! Come on and join us. Starting with the file system Yes, we have started implementing watcher to observe and detect changes in the file system. You can use watcher to discover changes related to file creation, file deletion and file alteration. Youcan find out more about our all the Watcher features in these pages. Watcher is Free, Open Source and User Focused Our code is free and open source. We like open source but we like socially responsible software even more. Watcher is distributed under MIT license. FIRST STEPS 1 watcher, Release 0.2.0 2 FIRST STEPS CHAPTER ONE FIRST STEPS Your project needs to process inputs that trigger your business logic but those inputs are out of your control? Do you want to integrate your project based on detection of file system changes? Learn about the great options Watcher offers for advanced change detection that you can leverage for your project development.
    [Show full text]
  • Storage Solutions for Big Data Systems: a Qualitative Study and Comparison
    Storage Solutions for Big Data Systems: A Qualitative Study and Comparison Samiya Khana,1, Xiufeng Liub, Syed Arshad Alia, Mansaf Alama,2 aJamia Millia Islamia, New Delhi, India bTechnical University of Denmark, Denmark Highlights Provides a classification of NoSQL solutions on the basis of their supported data model, which may be data- oriented, graph, key-value or wide-column. Performs feature analysis of 80 NoSQL solutions in view of technology selection criteria for big data systems along with a cumulative evaluation of appropriate and inappropriate use cases for each of the data model. Classifies big data file formats into five categories namely text-based, row-based, column-based, in-memory and data storage services. Compares available data file formats, analyzing benefits and shortcomings, and use cases for each data file format. Evaluates the challenges associated with shift of next-generation big data storage towards decentralized storage and blockchain technologies. Abstract Big data systems‘ development is full of challenges in view of the variety of application areas and domains that this technology promises to serve. Typically, fundamental design decisions involved in big data systems‘ design include choosing appropriate storage and computing infrastructures. In this age of heterogeneous systems that integrate different technologies for optimized solution to a specific real-world problem, big data system are not an exception to any such rule. As far as the storage aspect of any big data system is concerned, the primary facet in this regard is a storage infrastructure and NoSQL seems to be the right technology that fulfills its requirements. However, every big data application has variable data characteristics and thus, the corresponding data fits into a different data model.
    [Show full text]
  • Использование Tarantool В .NET-Проектах Анатолий Попов Director of Engineering, Net2phone Тезисы
    Использование Tarantool в .NET-проектах Анатолий Попов Director of Engineering, Net2Phone Тезисы • Что такое NewSql? Куда делся NoSql? • Как использовать Tarantool из .net? • Производительность progaudi.tarantool 2 Обо мне • Работаю с .net с 2006 года • Активно в OSS с 2016 года 3 Тот, кто не помнит прошлого, обречён на его повторение. Джордж Сантаяна, Жизнь разума, 1905 4 RDBMS • General purpose database • Usually SQL • Developed since 1990s or so 5 NoSql • Strozzi NoSQL open-source relational database – 1999 • Open source distributed, non relational databases – 2009 • Types: • Column • Document • KV • Graph • etc 6 Цели создания • Простота: дизайна и администрирования • Высокая пропускная способность • Более эффективное использование памяти • Горизонтальное масштабирование 7 Shiny new code: • RDBMS are 25 year old legacy code lines that should be retired in favor of a collection of “from scratch” specialized engines. The DBMS vendors (and the research community) should start with a clean sheet of paper and design systems for tomorrow’s requirements, not continue to push code lines and architectures designed for yesterday’s needs “The End of an Architectural Era” Michael Stonebraker et al. 8 Результат 9 Недостатки • Eventual consistency • Ad-hoc query, data export/import, reporting • Шардинг всё ещё сложный • MySQL is fast enough for 90% websites 10 NewSQL • Matthew Aslett in a 2011 • Relations and SQL • ACID • Бонусы NoSQL 11 NewSQL: код около данных • VoltDB: Java & sql • Sql Server: .net & sql native • Tarantool: lua 12 Sql Server
    [Show full text]
  • Tarantool a No-SQL DBMS Now with SQL Kirill Yukhin
    Tarantool A no-SQL DBMS now with SQL Kirill Yukhin 1 Agenda What is Tarantool? Performance Storage engines Scaling SQL Plans 2 History Was born @ Mail.ru group Used to store sessions/profiles sessions of Ms of users load web-page 4 instances AJAX request profiles mobile API 8 instances Web servers > 1.000.000 requests per second 3 Must-have and mustn't-have No secondary keys, constraints etc. Schema-less Need a language. *QL is not must-have High-speed in any sense! Simple Extensible Transactions Persistency Once again: it must be fast, no excuses 4 Tarantool: Bird's Eye View No need for cache: It is in-memory But still DBMS: persistency and transactions It regards ACID Single threaded: It is lock-free Easy: imperative language is on board: Lua It JITs It's easy to program business It scales: Replication and sharding 5 DBMS + Application Server C, Lua, SQL, Python, PHP, Go, Java, C# ... Persistent in-memory and disk storage engines Stored procedures in C, Lua, SQL Queries Process handling WAL Network Threads 6 Coöperative multitasking Multithreading That is a stall Losses on caches coherency support Losses on locks Losses on long operations Fibers Event-loop Thread is always busy Lock-free Single core - no coherency issues at all 7 Vinyl In-memory is OK, but not always enough Write-oriented: LSM tree Same API as memtx Transactions, secondary keys 8 Why? Horizontal 9 Horizontal scaling Replication Sharding ABC ABC ABC A C B Scaling computation and fault Scaling computation and tolerance data Replication and sharding A B C A A B B C C Scaling computation, data and fault tolerance 10 Replication Asynchronous Synchronous begin begin commit commit prepare replicate replicate Commit is not waiting for replication to Two phase commit.
    [Show full text]