Google BigQuery and MicroStrategy A Functional Overview & Performance Considerations Core message of the presentation Petabytes at your fingertips
• Google BigQuery allows users to interactively query petabytes of data • A Cloud based service that leverages Google’s infrastructure
• MicroStrategy enhances the power of Google BigQuery • Interactive queries are done visually • Combine information from multiple data sources • Build Enterprise reports and dashboards What is Google Bigquery ? Google has to answer quite a few big data questions
• Give me the top 20 apps on the play store • Imagine a DB table with all the apps on all android devices in the world • Identify the slowest machine in my server farm • Imagine a DB table with all the queries done by all the servers
• Solution :
http://research.google.com/pubs/pub36632.html What is Google Bigquery ? Analytics as a Service
• Goal : Disk IO of 1TB/Sec • ~ 10k+ Disks • ~ 5k Processors • Embrace Table Scans
http://research.google.com/pubs/pub36632.html Will I need Google BigQuery ? Do I have terabytes of data that I could analyze
• Web Analytics : All the visits to my webpages • Google Analytics premium allows you to export to Google BigQuery • Social Analytics : Tweets and Facebook posts about my company • Datasift can export your data to Google BigQuery
• Infrastructure Monitoring, Sensor data analysis, Retail transaction data • Machine logs, Mobile App Data, Gaming Telemetry
• For billion-row tables and interactive analysis: GBQ may be the answer
http://research.google.com/pubs/pub36632.html MicroStrategy Connectivity Options to Google BigQuery Native Connectivity through Data Import Products • Desktop Windows • Desktop Mac • Enterprise Windows • Enterprise Linux
Capabilities • Live-Connect • Multi-source In-memory Datasets • Ease of use MicroStrategy Connectivity Options to Google BigQuery ODBC Connectivity through FFSQL Reports Products • Enterprise Windows • Enterprise Linux
Capabilities • Live-Connect • Single-Source In-memory Datasets • Customizability • Value Prompts
Stay Tuned for Modelled Schema support
Demo Google BigQuery and MicroStrategy Best Practices for Performance Denormalize your data
• Google BigQuery is getting better at joins • Added support for Right Outer Join and Full Outer Join in April 2015 • Improved join support by adding Cross Join and multiple join in March 2014
Best Practices for Cost Saving and Performance Pre-Calculate and Setup the tables for your application
• Donot read more columns or rows than you need to • Pre-Execute transformation functions • Create a destination table that is a subset of a larger table • Only provide access to those columns that are useful to the user Best Practices for Cost Savings Shard your data by date and use the TABLE_DATE_RANGE function with Value Prompts • Google charges you by the data it reads to process a query • Shard your data by the day mydata.people20140325, mydata.people20140326, mydata.people20140327 ….. • Create ffsql reports with dynamic date value prompts and you will only read the data you need (For cost savings rather than performance)
https://cloud.google.com/bigquery/query-reference?hl=en Best Practices for Cost Savings Use Table Range Decorators • Use Table Range Decorators to run a more cost-effective query • Table Range decorator will give you all the data between certain time intervals • You can leverage value prompts to reduce data consumption
https://cloud.google.com/bigquery/query-reference?hl=en Best Practices for Cost Savings and Performance Use Google Bigquery to see how you are using Google Bigquery • Understand your Google Bigquery usage • Use Logs Viewer (Boring) • Stream your usage data into Google Bigquery and run queries on it
https://cloud.google.com/bigquery/audit-logs Best Practices for Debugging Use the Google BigQuery Web Console for debugging
https://cloud.google.com/bigquery/audit-logs Conclusion Google BigQuery and MicroStrategy • A powerful combination
• Entire force of Google Cloud Platform in combination with MicroStrategy’s Enterprise Capabilities
• Allow end-users to do adhoc analysis on massive datasets
• Build a mobile applications that use terabytes of data
• In the future, more features and support for Google BigQuery
Questions ?