Voltdb Achieves 3M Ops/Second Scaling Linearly on Telecom Benchmark

BENCHMARKS

VoltDB Achieves 3M ops/second Scaling Linearly on Telecom Benchmark

Executive Overview VoltDB is trusted globally by telecommunications software solution providers as the in-memory database of choice to power their mission-critical applications deployed at over 100 operators around the world. VoltDB was chosen for its performance and features to solve not only the current challenges, but also to support the rapid evolution of systems in the industry. Here is our benchmark that shows how the performance of VoltDB meets or exceeds the requirements of telecom systems. We showcase VoltDB’s high performance, low latency and linear scaling that are necessary to power revolutions in the industry such as 5G. In this benchmark, we tested VoltDB v8.3 in the cloud and observed the performance scale linearly with the number of servers and achieve over 3 million operations/second with latency consistently in single digits.

VoltDB and the 5G Landscape

The advent of 5G brings several key challenges to telecom software solution providers. In addition to the new hardware standards, this network evolution necessitates a transformation of the supporting IT functions of OSS and BSS as well. The opportunity to create additional revenues through new service creation generates pressure on the OSS and BSS functions to be agile and scalable to enable new use cases under ever-increasing load. This pressure on the systems, elevates the role of the supporting database from simply storing data to serving as an active real-time decisioning system.

Simply put, VoltDB is a Telco-grade database that exactly meets the requirements of an active real-time decisioning system such as one that might be needed to support 5G. VoltDB is an in-memory relational database built with scalability, programmability, and consistency as the central and uncompromisable guarantees. Applications built on VoltDB, whether in Telecom, Finance, or Retail, can be characterized as being horizontally scalable, amenable to evolution of complexity, and reliable in consistency and uptime.

In theory, solutions can be cobbled together from multiple open source tools for streaming, cluster management, database storage and in-memory caching, but in reality, these solutions fall short. They often add additional latency to get between systems, have a high operational cost to manage multiple software stacks on a large number of servers, and create a client-side programming burden to detect and manage consistency and correctness.

In addition to being a high performance database, VoltDB provides the core architectural elements necessary to power mission-critical real-time telecom apps:

• Strict ACID compliance

• Materialized views, stored procedures, and User Defined Functions

• In-built High availability, Disaster recovery, and multi-site cross-datacenter replication BENCHMARKS

• Cloud/Container ready

• Import and Export streams

There is simply no room to compromise when it comes to embedding the right database to power 5G apps. Having a proven enterprise-class database with not just the features and functionality, but also the 24x7 support and experi- enced professional services is essential to 5G success.

The Benchmark Online Charging System

The online charging function is an ideal candidate to understand the challenges that 5G poses at the systems level. Solutions must handle not only the load of connections from the billions of new connected devices, but also the complexity of enforcing different policies for the increasing diversity of devices. Online charging in 5G networks, hence, must scale both with load and functional complexity without compromising on consistency or performance.

Why we need ACID

System architectures for these applications take different approaches to tackle the confluence of these challenges. Most employ key-value stores for their purported benefit of unlimited scalability, but are forced to implement transactions using custom mechanisms thereby resulting in complicated systems that do not realize the original promises of simplicity and scaling. Even the fastest key-value store will require additional client-server communications and be- spoke locking and consistency checks, all of which will increase latencies and burden the network. The problems with using traditional databases under scale, on the other hand, are sufficiently well known. Among the databases in the market today, we believe VoltDB occupies the unique position that solves the challenges of scalability and complexity while still providing the guarantees of ACID transactions at the highest consistency level, strict serializable [1].

Implementation Details

We tested VoltDB database instances built from clusters of 3, 9, 18, and 27 nodes provisioned on Google Cloud Platform. The workload simulates a simplified online charging application. After preloading data, the benchmarking application was run for a duration of 10 minutes. Performance statistics of transactions per second (TPS) and 99th percentile latency were gathered for the duration of the run (each VoltDB transaction is a single application “operation” and the terms are interchangeable). The charted data from these statistics demonstrates the linear scalability of VoltDB in handling increasing workload while maintaining predictable and low latency.

Application:

The application consists primarily of a Java client, and a VoltDB database. Stored procedures that implement each operation’s processing logic are coded in a combination of Java and SQL to run on the database. The benchmark application manages the balances of the subscribers while allowing them to purchase new services and add additional quotas such as minutes, data, messages, etc. The client relies on the database to run complex transactions with multiple SQL statements and conditional logic with full ACID guarantees.

Schema Being a relational database, the data in VoltDB is organized into tables as shown below. Each table is either partitioned, where the rows are distributed among sites (sites_per_node * node_count), or is replicated, where each node has a full copy of the table. Tables such as the Product table which are not updated frequently are ideal candidates to be kept as a replicated table. The Product table is, however, part of the main workload since it is looked up to obtain

2 BENCHMARKS UPDATE OCTOBER 2018 BENCHMARKS the price of the product that the user is trying to purchase. The User, Usage and Balance tables are created as partitioned tables, allowing very high concurrency access for both read and write access.

Product User Usage Balance Replicated Partitioned Partitioned Partitioned

productid bigint userid bigint userid bigint userid bigint

unit_cost bigint user_blob productid bigint balance

varchar(8000) Allocated_units bigint bigint

Data: The workload is run on a stable data set that is scaled with the size of the cluster. The size of the starting data set for different clusters follows the following ratios:

Users — 500k records per core Usage — 62.5k records per core Balance — 500k records per core

The number of products is constant at 100.

Workload: The workload consists of running two operations in parallel — Allocate quota, and Add user/balance. The operations are implemented in Java+SQL involve executing multiple SQL statements with conditional logic that provide the benefit of reducing the network trips and achieving more work in each logical transaction.

After the starting data set has been loaded, the two operations in the workload are called in parallel at the same frequency to achieve the target throughput. Both operations are complex, involving decisions that must be made by joining data from multiple tables. By implementing each operation as a VoltDB stored procedure, the entire operation either succeeds or fails as a complete transaction, returning a status to the client application:

• Allocate quota — Accesses 4 tables. Allocates quota to a subscriber only after verifying that there is sufficient balance in their account and deducts the balance after a successful allocation.

• Add user/balance — Accesses 2 tables. Adds a new subscriber to the system or increase the balance of a user.

Metrics The benchmark application is run on different node configurations to demonstrate the scalability of VoltDB for running highly transactional workloads. Data points of latency and throughput were captured at each of the node configurations and graphed to represent the scalability curve.

The data points were captured after a run duration of 10 minutes. This wait period helps ensure that the system reaches a steady state of moderate CPU utilization between 55% and 60% and a rough RAM utilization of 33% per machine.

Environment

The benchmark was run on Google Cloud instances. The node configurations are: n1-standard-4 4 vCPUs 15GB n1-standard-16 16 vCPUs 60GB

3 BENCHMARKS UPDATE OCTOBER 2018 BENCHMARKS

Benchmark Results & Analysis To provide the backdrop for the benchmark results, we’d like to state that the SLAs for the most demanding 5G applications, based on the conversations with our telecom customers, are predictable latencies of under 5 ms, and the ability to process 2 to 6 million rows per second, with the ability to scale linearly. The benchmarking results can be viewed in the context of these stringent SLAs.

Performance on 4 Core Virtual Machines

800000 20

600000 15

400000 10 ops/s Latency

200000 5

0 0 0 5 10 15 20 25 30 Nodes

ops/s latency in ms

Figure 1: Throughput and Latency of VoltDB with 4 partitions running on 4 core machines

This graph demonstrates near-linear scaling of throughput with the node count. The largest cluster tested contained 27 nodes. The highest throughput observed was 740,703 ops/sec.

The above chart also demonstrates that the 99th percentile latency for every cluster size tested meets the desired SLA of 5 milliseconds (except for 27 nodes, where the latency was slightly higher than 5ms). Considering that the 5G grade telecom SLAs are the most stringent across any industry or use case, along with the complexity of the Charging application, VoltDB handily beating both the throughput and latency SLAs is a truly remarkable achievement.

Performance on 16 Core Virtual Machines

3200000 20

2400000 15

1600000 10 ops/s Latency

800000 5

0 0 0 5 10 15 20 25 30 Nodes

ops/s Latency in ms

Figure 2: Throughput and Latency of VoltDB with 16 partitions running on 16 core machines

4 BENCHMARKS UPDATE OCTOBER 2018 BENCHMARKS

The benchmark was run on 16 core machines which would be closer to the configurations found in production systems and hence be a better measure of VoltDB’s performance. Again, VoltDB demonstrates linear scale achieving a throughput of over 3MM ops/sec on the 27 node cluster, and thereby meeting or exceeding the SLA requirements.

VoltDB’s latency is well under the SLA of 5 ms for every cluster size. The smaller cluster of three nodes had a 99th percentile latency of slightly over 3 ms, while the other cluster configurations performed under 3 ms latency!

Comparison of throughput on 16 core and 4 core Virtual Machines

3200000

2400000

1600000 ops/s

800000

0 0 5 10 15 20 25 30 Nodes

16 core machines 4 core machines

Figure 3: Comparison of throughput of VoltDB clusters with 4 partitions running on 4 core machines, and with 16 partitions running on 16 core machines

As expected with increasing partition size and increasing core count the throughput increases significantly. For the 27 node cluster with 4 cores and 4 partitions, the throughput was just under 75K, while for the 27 node cluster with 16 cores and 16 partitions, the throughput reached to over 3MM operations per second!

Comparison of Latency on 16 core and 4 core Virtual Machines

10 ops/s

0 0 5 10 15 20 25 30 Nodes

10 core machines 4 core machines

Figure 4: Comparison of latency of VoltDB clusters with 4 partitions running on 4 core machines, and with 16 partitions running on 16 core machines

5 BENCHMARKS UPDATE OCTOBER 2018 The trend of increasing throughput with server/partition count also applies to latency. The more powerful 16 core servers with 16 partitions have lower 99th percentile latency than the 4 core servers with 4 partitions. While the latency for the 27 node cluster with 4 core machines was just over 5 ms, the latency for the 27 node cluster with 16 core machines is under 3 ms.

A Recipe for High Scalability “As referenced earlier, VoltDB was designed for linear scale. To achieve a throughput of even greater than 3MM ops/sec; one can simply add VoltDB nodes until the desired throughput is achieved.”

TCO In addition to best-in-class performance, VoltDB also offers a much lower TCO than any other solution in the industry. While there may be a perception that “open source solutions are free”, there is no such thing as a free lunch. And many telecom customers who initially went with open source solutions made the move to VoltDB after many production horrors. They also realized that open source tools typically run on a much larger hardware footprint leading to very high CAPEX, and because these solutions require expensive development efforts to build, maintain, and trouble- shoot in production the OPEX costs are much higher than a VoltDB software license that comes with 24x7 production support. One of the largest telecom software solution providers in the world, moved from Redis to VoltDB and saved $1MM in hardware alone.

Conclusion NoSQL databases such as Redis, Cassandra, MongoDB, etc., have emerged in response to the inflexibility of legacy SQL databases such as Oracle, IBM, SQL Server, etc. They widely proclaim to enable scaling by storing and querying unstructured data without enforcing the structured concepts of a relational database. Despite their initial claim of NoSQL databases’ lack of “dependence” on SQL as a benefit, the realization that SQL’s standardization, flexibility, and efficiency in querying large amounts of data cannot be replaced hinders their viability for high performance or mission-critical applications. The attempts by the NoSQL vendors to bolt support for SQL language on top still misses the goal of providing the underlying ACID guarantees similar to actual SQL databases.

Next generation SQL databases like VoltDB offer the best of both worlds: the scalability of NoSQL solutions along with high throughput, low latency, high availability, strong ACID transactions, real-time analytics, and other functionality that are critical for 5G apps.

The results of this benchmarking study demonstrate that VoltDB offers the linear scale, low latency, and high throughput required by demanding 5G apps without sacrificing consistency. VoltDB’s throughput of over 3MM ops/ sec and a 99th percentile latency of under 5 ms meets the desired SLAs for 5G systems, while NoSQL databases fall far short of the performance and latency requirements of telecom apps. If you would like to run the benchmark, we suggest downloading the source code of the application from our public repository and trying it out for yourself: https://github.com/VoltDB/app-telco-charging

About VoltDB VoltDB powers applications that require real-time intelligent decisions on streaming data for a connected world, without compromising on ACID requirements. No other database can fuel applications that require a combination of speed, scale, volume and accuracy. Architected by the 2014 A.M. Turing Award winner, Dr. Mike Stonebraker, VoltDB is a ground-up redesign of the relational database for today’s growing real-time operations and machine learning challenges. Dr. Stonebraker has conducted research on database technologies for more than 40 years, leading to numerous innovations in fast data, streaming data and in-memory databases. With VoltDB, he realized the full potential of tapping streaming data with in-memory transactional database technology that can handle data’s speed and volume while delivering real-time analytics and decision making. VoltDB is a trusted name in the industry already validated by leading organizations like: Nokia, Financial Times, Mitsubishi Electric, HPE, Barclays, Huawei, and more. 01 November 2018