BENCHMARKS VoltDB Achieves 3M ops/second Scaling Linearly on Telecom Benchmark Executive Overview VoltDB is trusted globally by telecommunications software solution providers as the in-memory database of choice to power their mission-critical applications deployed at over 100 operators around the world. VoltDB was chosen for its performance and features to solve not only the current challenges, but also to support the rapid evolution of systems in the industry. Here is our benchmark that shows how the performance of VoltDB meets or exceeds the requirements of telecom systems. We showcase VoltDB’s high performance, low latency and linear scaling that are necessary to power revolutions in the industry such as 5G. In this benchmark, we tested VoltDB v8.3 in the cloud and observed the performance scale linearly with the number of servers and achieve over 3 million operations/sec- ond with latency consistently in single digits. VoltDB and the 5G Landscape The advent of 5G brings several key challenges to telecom software solution providers. In addition to the new hardware standards, this network evolution necessitates a transformation of the supporting IT functions of OSS and BSS as well. The opportunity to create additional revenues through new service creation generates pressure on the OSS and BSS functions to be agile and scalable to enable new use cases under ever-increasing load. This pressure on the systems, elevates the role of the supporting database from simply storing data to serving as an active real-time decisioning system. Simply put, VoltDB is a Telco-grade database that exactly meets the requirements of an active real-time decision- ing system such as one that might be needed to support 5G. VoltDB is an in-memory relational database built with scalability, programmability, and consistency as the central and uncompromisable guarantees. Applications built on VoltDB, whether in Telecom, Finance, or Retail, can be characterized as being horizontally scalable, amenable to evo- lution of complexity, and reliable in consistency and uptime. In theory, solutions can be cobbled together from multiple open source tools for streaming, cluster management, da- tabase storage and in-memory caching, but in reality, these solutions fall short. They often add additional latency to get between systems, have a high operational cost to manage multiple software stacks on a large number of servers, and create a client-side programming burden to detect and manage consistency and correctness. In addition to being a high performance database, VoltDB provides the core architectural elements necessary to pow- er mission-critical real-time telecom apps: • Strict ACID compliance • Materialized views, stored procedures, and User Defined Functions • In-built High availability, Disaster recovery, and multi-site cross-datacenter replication BENCHMARKS • Cloud/Container ready • Import and Export streams There is simply no room to compromise when it comes to embedding the right database to power 5G apps. Having a proven enterprise-class database with not just the features and functionality, but also the 24x7 support and experi- enced professional services is essential to 5G success. The Benchmark Online Charging System The online charging function is an ideal candidate to understand the challenges that 5G poses at the systems level. Solutions must handle not only the load of connections from the billions of new connected devices, but also the complexity of enforcing different policies for the increasing diversity of devices. Online charging in 5G networks, hence, must scale both with load and functional complexity without compromising on consistency or performance. Why we need ACID System architectures for these applications take different approaches to tackle the confluence of these challenges. Most employ key-value stores for their purported benefit of unlimited scalability, but are forced to implement trans- actions using custom mechanisms thereby resulting in complicated systems that do not realize the original promises of simplicity and scaling. Even the fastest key-value store will require additional client-server communications and be- spoke locking and consistency checks, all of which will increase latencies and burden the network. The problems with using traditional databases under scale, on the other hand, are sufficiently well known. Among the databases in the market today, we believe VoltDB occupies the unique position that solves the challenges of scalability and complexity while still providing the guarantees of ACID transactions at the highest consistency level, strict serializable [1]. Implementation Details We tested VoltDB database instances built from clusters of 3, 9, 18, and 27 nodes provisioned on Google Cloud Platform. The workload simulates a simplified online charging application. After preloading data, the benchmarking application was run for a duration of 10 minutes. Performance statistics of transactions per second (TPS) and 99th percentile latency were gathered for the duration of the run (each VoltDB transaction is a single application “opera- tion” and the terms are interchangeable). The charted data from these statistics demonstrates the linear scalability of VoltDB in handling increasing workload while maintaining predictable and low latency. Application: The application consists primarily of a Java client, and a VoltDB database. Stored procedures that implement each operation’s processing logic are coded in a combination of Java and SQL to run on the database. The benchmark application manages the balances of the subscribers while allowing them to purchase new services and add addition- al quotas such as minutes, data, messages, etc. The client relies on the database to run complex transactions with multiple SQL statements and conditional logic with full ACID guarantees. Schema Being a relational database, the data in VoltDB is organized into tables as shown below. Each table is either parti- tioned, where the rows are distributed among sites (sites_per_node * node_count), or is replicated, where each node has a full copy of the table. Tables such as the Product table which are not updated frequently are ideal candidates to be kept as a replicated table. The Product table is, however, part of the main workload since it is looked up to obtain 2 BENCHMARKS UPDATE OCTOBER 2018 BENCHMARKS the price of the product that the user is trying to purchase. The User, Usage and Balance tables are created as parti- tioned tables, allowing very high concurrency access for both read and write access. Product User Usage Balance Replicated Partitioned Partitioned Partitioned productid bigint userid bigint userid bigint userid bigint unit_cost bigint user_blob productid bigint balance varchar(8000) Allocated_units bigint bigint Data: The workload is run on a stable data set that is scaled with the size of the cluster. The size of the starting data set for different clusters follows the following ratios: Users — 500k records per core Usage — 62.5k records per core Balance — 500k records per core The number of products is constant at 100. Workload: The workload consists of running two operations in parallel — Allocate quota, and Add user/balance. The operations are implemented in Java+SQL involve executing multiple SQL statements with conditional logic that provide the ben- efit of reducing the network trips and achieving more work in each logical transaction. After the starting data set has been loaded, the two operations in the workload are called in parallel at the same frequency to achieve the target throughput. Both operations are complex, involving decisions that must be made by joining data from multiple tables. By implementing each operation as a VoltDB stored procedure, the entire opera- tion either succeeds or fails as a complete transaction, returning a status to the client application: • Allocate quota — Accesses 4 tables. Allocates quota to a subscriber only after verifying that there is sufficient balance in their account and deducts the balance after a successful allocation. • Add user/balance — Accesses 2 tables. Adds a new subscriber to the system or increase the balance of a user. Metrics The benchmark application is run on different node configurations to demonstrate the scalability of VoltDB for running highly transactional workloads. Data points of latency and throughput were captured at each of the node configurations and graphed to represent the scalability curve. The data points were captured after a run duration of 10 minutes. This wait period helps ensure that the system reaches a steady state of moderate CPU utilization between 55% and 60% and a rough RAM utilization of 33% per machine. Environment The benchmark was run on Google Cloud instances. The node configurations are: n1-standard-4 4 vCPUs 15GB n1-standard-16 16 vCPUs 60GB 3 BENCHMARKS UPDATE OCTOBER 2018 BENCHMARKS Benchmark Results & Analysis To provide the backdrop for the benchmark results, we’d like to state that the SLAs for the most demanding 5G ap- plications, based on the conversations with our telecom customers, are predictable latencies of under 5 ms, and the ability to process 2 to 6 million rows per second, with the ability to scale linearly. The benchmarking results can be viewed in the context of these stringent SLAs. Performance on 4 Core Virtual Machines 800000 20 600000 15 400000 10 ops/s Latency 200000 5 0 0 0 5 10 15 20 25 30 Nodes ops/s latency in ms Figure 1: Throughput and Latency of VoltDB with 4 partitions running on 4 core machines This graph demonstrates near-linear scaling of throughput with the node count. The largest cluster tested contained 27 nodes. The highest throughput observed was 740,703 ops/sec. The above chart also demonstrates that the 99th percentile latency for every cluster size tested meets the desired SLA of 5 milliseconds (except for 27 nodes, where the latency was slightly higher than 5ms). Considering that the 5G grade telecom SLAs are the most stringent across any industry or use case, along with the complexity of the Charging application, VoltDB handily beating both the throughput and latency SLAs is a truly remarkable achievement.