Javascript Charting
Total Page:16
File Type:pdf, Size:1020Kb
JAVASCRIPT CHARTING Scaling for the Enterprise with Metric Insights © 2013 Copyright Metric insights, Inc. A REVOLUTION IS HAPPENING ..................................................................................... 3! Challenges .................................................................................................................... 3! Borrowing From The Enterprise BI Stack ...................................................................... 4! Visualization Layer ..................................................................................................... 4! Source Data Layer ..................................................................................................... 5! RDBMS sources ..................................................................................................... 6! Big Data sources .................................................................................................... 6! SaaS and Cloud Application data ........................................................................... 6! Business Intelligence Tool data ............................................................................. 6! Data Connection Layer .............................................................................................. 7! Load Management ................................................................................................. 7! Error Handling ........................................................................................................ 7! Schedules ............................................................................................................... 7! Data Dependencies ................................................................................................ 7! Data Caching Layer ................................................................................................... 8! Data Formatting ..................................................................................................... 8! Data Thinning ......................................................................................................... 8! Lifecycle Management ............................................................................................... 8! User Management and Security Layer ...................................................................... 8! Alerting & Email Distribution Layer ............................................................................ 9! Collaboration Layer .................................................................................................... 9! METRIC INSIGHTS SOLUTION ..................................................................................... 10! BI TOOLS DON’T SUPPORT THIS ................................................................................ 11! © 2013 Copyright Metric insights, Inc. A REVOLUTION IS HAPPENING A revolution is happening. New JavaScript Visualization libraries now make it possible for anybody to create beautiful and powerful web-based visualizations in data. The days requiring a team of developers or expensive business intelligence tools are over. Inspired by the cool, interactive graphics in Nate Silvers’ NYT election blog, web developers and business intelligence professionals are embracing these technologies across the enterprise. A visualization smorgasbord awaits us all, but there are a number of challenges that are holding the feast in check. Challenges The accessibly of the JavaScript Visualization libraries make it easy for anybody with a little programming expertize and sense of design to build engaging visualizations. The challenge arises when you go beyond a handful of visualizations with static data sets to an enterprise-scale solution with many visualizations, with changing data from different data sources, and scores of users with different consumption needs and security requirements. Most of the JavaScript Visualization examples on the web today show a dynamic chart with a static data set. But what has to happen to scale that across the enterprise? How do you load balance across multiple data sources? How do you ensure the wrong users do not see sensitive data? How do go to sleep at night knowing that you have a stable system and will wake up in the morning with happy users? Page 3 Borrowing From The Enterprise BI Stack It turns out that the traditional business intelligence (BI) tools have solved many of these problems. The traditional BI stack is worth looking at to identify components that need to be addressed in a JavaScript Visualization deployment. Most notable in the stack diagram comparison below is the number of areas that are missing from a JavaScript Visualization deployment. Each of these areas must be taken into consideration to create an enterprise-scale JavaScript Visualization solution. Visualization Layer The traditional BI stack has a proprietary visualization component built in as part of the solution. Typically, you can choose different visualizations from a pre-set library, but you don’t have an option to change the libraries. Page 4 A major characteristic of choosing to go with a JavaScript Visualization solution is you can choose the visualization library that makes the most sense for you. JavaScript Visualization libraries abound. Different libraries are optimized for different purposes and audiences. You will want to choose the appropriate library depending on your needs. For example, commercial support might be more important for your organization. Or, perhaps, creating visualizations with a unique look and feel might be your highest concern. Another advantage of JavaScript Visualization libraries is that you can easily combine different visualization libraries to leverage the strengths of each library to most elegantly solve a given visualization challenge. Some of the more popular libraries include: • Highcharts*+*www.highcharts.com* • FusionCharts*+*www.fusioncharts.com* • D3*+*www.d3js.org* • Google*Charts*–*developers.google.com/chart/* There are many other libraries with different purposes as well. For a more comprehensive list, see this article: 50 JavaScript Libraries for Charts and Graphs Source Data Layer All enterprises are characterized by the growing diversity of data sources. The ideal of a single data warehouse that captures all the data in an enterprise has quickly passed to the reality of multiple data silos as new services are implemented at departmental levels and new SaaS applications are introduced. The following are some considerations that must be taken with different types of data sources. Page 5 RDBMS sources Microsoft SQL Servers, Oracle Servers, MySQL databases and many others are commonly deployed in most enterprises. A successful JavaScript Visualization system must be able to connect through an ODBC or JDBC connection and reliably pull data on a regular basis. Big Data sources Hadoop, MongoDB, Cassandra, and other Big Data sources are now becoming common additions to most data-driven companies. Each of these have particular attributes that must be taken into consideration when creating robust JavaScript Visualization based on these sources. For example, the long query times that are a common characteristic of such systems mandate some sort of caching layer to buffer the end users from such long wait times. Further, load control is important as such systems typically can’t handle a simultaneous spike in user requests. SaaS and Cloud Application data Saleforce.com, Google Analytics, Omniture, Twitter, Facebook and many other cloud applications are a treasure-trove of corporate data, but can be difficult to access. Pulling data from each of these systems typically requires custom development through proprietary APIs of each system. Doing regular data dumps via a CSV file is another common alternative that requires a batch file management system to handle. Business Intelligence Tool data Tableau Software, Business Objects, SAS, IBM Cognos, Microstrategy and other business intelligence tools typically have key corporate data that is already defined in a way consistent with corporate goals. Getting access to such data, and combining it with other data, is a key factor for success for JavaScript Visualization deployments. There are two ways to get access to such data: connecting directly to the underlying data sources (RDBMS, Big Data, etc.) or connecting directly to the reports in the Business Intelligence tools. For the latter, each Business Intelligence tool typically has an API that can be used to build a custom script to pull data out of reports. Page 6 Data Connection Layer Getting the data is just the beginning of the process, but this in itself requires effort. Data often resides in multiple places and it requires the developer to understand how the data is stored, how it is related and then write the appropriate scripts to extract what is important. These scripts must be able to incrementally update data as new information becomes available in the underlying source. Load Management As you roll out more and more data visualizations it is necessary to manage query load so as not to overwhelm source systems. At scale, to effectively manage query load you must manage query processing so that no more than a maximum number of data fetches are run concurrently. The logic to support intelligent query load balancing across multiple data sources can be complex to build and time-consuming to maintain. Error Handling Having a scalable environment that can deal gracefully with situations where source