International Journal of Advances in Electronics and Computer Science, ISSN: 2393-2835 Volume-4, Issue-2, Feb.-2017 http://iraj.in ANALYSIS OF MPP-SYSTEMS STRESS TESTING BASED ON BIG DATA 1VASSILIY SERBIN, 2YEVGENIY GORBUNOV 1,2Computer science, 2 Information systems department International Information Technology University E-mail: [email protected],[email protected] Abstract- The comparative analysis of MPP-systems (Massively parallel processing) stress testing based on Big Data on different software and hardware systems is essential for high-performance computing, both in terms of optimizing the load on existing machines, and from the point of view of new platform procurement policy. The objectives are 14 major marketing SAS campaigns of one of the Kazakhstan Banks. They were selected to assess adequately capabilities of computer systems. Additionally, with the help of specialized tools, the authors analyzed the characteristics of the studied systems of massively parallel architectures. Keywords- Greenplum, Nettezza, Exadata, SAS-campaign, Big Data, high-performance computing, stress testing, MPP- systems. I. INTRODUCTION about the expenses and profits of undertaking new Big Data ventures (Vivekananda Gopalkrishnan et al., Big Data are important in many various areas, such as [5]). science, social media, enterprise etc. Banking is the Although storage capabilities have significantly industry where Big Data technologies can be widely grown and data stores are available around the world, applied. Financial institutions have large data arrays, it is still very hard to capture and store big data operate in a highly competitive environment and efficiently and make it easily accessible (Jameela Al- possess enough budget for IT innovation. Jaroodi, et al., [6]). Michael Stonebraker [1] claims that many enterprises Big Data is able to solve almost all of the key tasks of face with integrating a larger and larger number of banks such as customer acquisition, improving data sources with diverse data (spreadsheets, Web service quality, assessment of borrowers, combating sources, XML, traditional DBMSs). Informatica and fraud, etc. By increasing the speed and quality of Cloudera leaders consider as a solution the fact that reporting, increasing analysis depth, participating in the Apache Hadoop data management platform is, in the laundering of illicit funds, Big Data technologies many ways, uniquely equipped to handle the volume, help banks to meet the requirements of financial variety and velocity of unstructured data being regulators. generated within many businesses. Data collected in banks are unstructured by their Wei Fan and Albert Bifet [2] stated that “Large nature. Unstructured data are the fastest growing type quantities of useful data are getting lost since new of data generated today. Experts estimate that 80 to data is largely untagged file based and unstructured 90 % of data in any organization is unstructured [7, data. The 2012 IDC study on Big Data explains that 8]. Big Data paradigm allows to solve the problem of in 2012, 23% (643 exabytes) of the digital universe processing unstructured data. Unstructured data like would be useful for Big Data if tagged and analyzed. text, data or numeric values are not in a defined However, currently only 3% of the potentially useful schema. To make some sense out of unstructured data data is tagged, and even less is analyzed”. some sort of framework needs to be overlaid on the According to AnHai Doan, et. al. [3], “Unstructured raw data to make it more like information. This is the data has now permeated numerous real-world reason that Hadoop and similar tools are needed to applications, in all domains. Consequently, managing provide some structure using key value pairs to create such data is now an increasingly critical task, not just some structure where there is no structure. to our community, but also to many others, such as Methods for working with data are not improved at a the Web, AI, KDD, and SIGIR communities”. rate, with which their volumes grow, and they are not such a product of technology as the results of The problem is not only to store and manage Big scientific work, however, emergence of big data Data, but also to process and retrieve useful problems markedly speeded the trend of research. information from it (Kapil Bakshi [4]). Today the need to work with unstructured data has Many organizations have progressively come to become urgent. understanding of significance of information as a vital asset. As most customers are getting acquainted The goal of the research is to analyze stress testing with Big Data concept, a lot of people are still unsure MPP-systems based on Big Data for Greenplum, Analysis of MPP-Systems Stress Testing Based on Big Data 59 International Journal of Advances in Electronics and Computer Science, ISSN: 2393-2835 Volume-4, Issue-2, Feb.-2017 http://iraj.in Netezza, Exadata based on Big Data SAS Campaign Exadata is a line of software and hardware systems Management. that was being released commercially by Oracle Corporation. From 2008 to mid-2009 it was based on II. METHODS AND MATERIALS Hewlett-Packard server hardware, and later on the hardware from the absorbed Sun Microsystems. The Massively parallel processing (MPP) [9] is a class of complex is a cluster of database management servers, parallel computing system architectures. A distinctive based on Oracle RAC technology, delivered as pre- feature of the architecture is that memory is assembled telecommunication closets of 42 unit physically divided. The system is built from the dimensions filled with servers, node storage, and individual nodes, comprising a processor, a local switches, InfiniBand or Ethernet [12]. memory bank, communication processors and network adapters, in some cases hard drives and other Competitors also noted that being focused on OLTP, input-output devices. Only processors of the same and OLAP-processing simultaneously, makes the node have access to the random access memory bank systems less effective for analytical processing, on of the unit. The nodes are connected by special which similar solutions from Teradata and Netezza communication channels. A user can define a logical are concentrated, in particular, non-optimality of number of the processor to which he or she is usage of the approach with the symmetrical access connected, and organize the exchange of messages from all servers to all storage nodes (symmetrical with other processors. Two modes of operation for parallelism) as opposed to the complete separation of the operating system are used on the machines of data between nodes in competing analytical systems massively parallel architecture: with massively parallel processing is noted [13]. - In one mode, the complete operating system runs only on the management machine (front-end), and III. ANALYSIS ON STRESS TESTING IN TEST each node runs a heavily abridged version of the CAMPAIGNS operating system that supports the operation of the branch of the parallel application allocated to it. In this research, stress testing of Greenplum, Nettezza, Exadata, Oracle systems at tests of - In the second mode, each module operates fully, campaigns is analyzed. Testing was held by start of most often a UNIX-like system which is installed "packs" on 7 campaigns and represented start of sets separately. of test campaigns ("packs"): Large companies’ solutions such as EMC Greenplum, IBM Netezza and Oracle Exadata were selected as the - the first pack ("odd") is the start of 7 test campaigns objective of the study. in a parallel. Campaigns started at the same time in the Execute mode with previously chosen option Greenplum Software is a company that is engaged in "Test". Then all 7 campaigns are expected to be development of a DBMS for data warehouses. The completed. company specializes in Enterprise Data Cloud solutions for large-scale data warehousing and - the second pack ("even") is the start of the remained analytical systems. Greenplum Database DBMS is 7 test campaigns in a parallel. A way of start of based on a modified PostgreSQL database with campaigns is similar to the first round. massively parallel processing (MPP). Greenplum implemented MapReduce functionality and Column- Production schedules on the systems used in the Oriented Organization of tables in their database as analysis are given below. Schedules contain identical part of the so-called Polymorphic Data Storage temporary scales across that at an opportunity to technology [10]. compare schedules with each other. Statistics at tests I gathered on servers with 30 second intervals. Netezza is an American company, the developer of Schedule of Running Task "Task" is the settlement software and hardware data storage - relational task started in system: either SQL inquiry, or the database server clusters, providing massively parallel stored process of SAS (Fig.1, Fig.2, Fig.3). processing. A distinctive feature of all Netezza complexes is the use of programmable gate arrays in the data processing nodes, providing compression and filtering of data, and thus, it allows to reduce the costs of storage and input-output operations during execution of data selection queries. The company was founded in 2000; in 2010 it was absorbed by IBM Corporation, and in 2011 was fully integrated into the corporation, hardware and software systems are being released under the name IBM PureData for Analytics since 2012 [11]. Fig.1. Graphic Running Task. Netezza Analysis of MPP-Systems Stress Testing Based on Big Data 60 International Journal of Advances in Electronics and Computer Science, ISSN: 2393-2835 Volume-4, Issue-2, Feb.-2017 http://iraj.in Fig. 5. Greenplum Segment Node The schedule of Exadata Storage Sell Node is constructed using the means of NMON Analyser, on the basis of the data collected by the utility of nmon on one of Storage of servers and on one of Database of servers (Fig. 6). Fig.2. Graphic Running Task. Greenplum Fig.6. Exadata Storage Sell Node The schedule of Netezza CPU Aggregate is constructed on data of Excel monitoring utility.