Big Data Integration

BIG DATA INTEGRATION

DATA SHEET

VoltDB is an in-memory NewSQL operational database architected to act VoltDB is… on Fast Data streaming into applications at the rate of hundreds of thou- An in-memory, NewSQL sands to millions of events per second. Ideally suited as an operational da- Operational Database tabase to process Fast Data, VoltDB also has the ability to export data at for High-Performance, high speed to long-term analytics stores such as HP Vertica and Netezza, Fast Data Applications as well as Hadoop-based data warehouses. • Write and read millions of VoltDB Export continually and transactionally pushes data from VoltDB data events per second into another system, similar to an ETL (extract, transform, load) process. • In-memory performance and Unlike ETL, which pulls data and can stall when data changes at high rates on-disk persistence of speed, VoltDB Export uses a push model that exports data at the same • Relational, ACID-compliant rate at which it is ingested. SQL and JSON

VoltDB Export lets application developers automate the export process by specifying certain tables in the schema as sources for export. At runtime, In-memory Analytics any data written to the specified tables is sent to the Export Connector, • Ability to make automated, which queues the data for export, then sends it to the selected output per-event decisions based target. VoltDB provides connectors for exporting to files, for exporting to on historical data other business processes, for exporting to Big Data’s Hadoop, for export- • Enables current data to be ing to a distributed message queue such as Kafka or RabbitMQ, or for factored into analytics exporting to other relational databases via JDBC. • Query speed needed for The VoltDB export system is a loose coupling managed from within the interactive dashboards VoltDB application. The application has complete control via SQL over • Ability to serve large when and what data moves to the external system. Export Connectors are numbers of concurrent managed by the database servers themselves, helping to distribute the users work and ensure maximum throughput. VoltDB DATA SHEET BIG DATA INTEGRATION page 2

VoltDB supports the following Export Connectors: For more information on VoltDB, or to download a free trial of the a. CSV (flat file): Writes exported data to local files, either as comma- separated or tab-delimited file database (available in cloud or on-premises editions), visit www. b. HDFS (Hadoop): The HTTP connector receives the serialized data voltdb.com. from the export tables and writes it out to Hadoop via HTTP requests to WebHDFS.

c. Kafka: Writes export data to an Apache Kafka distributed message queue, where one or more other processes can read the data

d. RabbitMQ: Writes export data to a RabbitMQ distributed message queue, where one or more other processes can read the data

e. JDBC: Writes data to a variety of destination databases through the JDBC protocol

f. Netezza: Fetches transactional data from VoltDB and writes it, in batches, to the Netezza database

g. Vertica: Fetches transactional data from VoltDB and writes it, in batches, to the Vertica database

h. Build your own: Developers can build their own Export Connectors with simple examples and instructions available in the VoltDB Dev Center: http://voltdb.com/dev-center/cookbook/custom-onserver- export.

For application developers and enterprises looking for an end-to-end solution that combines the long-term, deep analytics capability of the data warehouse or Hadoop with the operational and in-memory analytics power of VoltDB, the Export delivers the operational capabilities of VoltDB (ingest, interactions, transaction, real-time analytics) while enabling data to be moved out of VoltDB once the data has been processed and is no longer of immediate value. Export pushes data at high speed to a historical system that can provide deep historical analytics (including reporting, complex analysis, large storage capacities).

Figure 1: VoltDB enables developers to take advantage of the cyclical nature of the import-export data cycle. For example, inflows of data can be filtered and acted upon based on rules loaded into VoltDB. Based on this filtered and processed export stream, updated rules are generated in Hadoop and frequently reloaded into VoltDB. VoltDB Export enables data to arrive in your analytic store sooner, and allows deep analytics to be leveraged with radically lower latency.