IT@Intel White Paper Intel IT IT Best Practices BI Data Warehouse Strategy March 2013 Using a Multiple Data Warehouse Strategy to Improve BI Analytics Executive Overview Intel IT’s strategy for multiple Intel IT is implementing a strategy for multiple business intelligence (BI) data business intelligence (BI) warehouses to provide significantly more powerful analytics capabilities to data warehouses enables our business groups across Intel. By providing an array of BI platforms, we are helping business groups to solve more Intel mine a broader range of data faster, deeper, and more cost-effectively. This high-value business problems, expanded architecture enables our business groups to solve more high-value business problems, achieve greater operational efficiencies, and improve their achieve greater operational competitive performance in global markets. efficiencies, and improve their competitive performance in For several years, Intel’s BI needs were • Extreme data warehouse (XDW), for analysis addressed with a “one size fits all” approach of structured and semi-structured data global markets. delivered by a centralized enterprise data • In-memory, for real-time analysis of warehouse (EDW). That solution is no longer streaming volume data sets adequate. With big data, data sets vary widely • Custom, independent data warehouses, for and are predominantly unstructured, complex, analysis of structured, normalized data and in volumes that cannot be managed with traditional relational database methods.1 To Our multiple data warehouse BI strategy has address these limitations, we have introduced enabled us to move from an expensive, one- this BI strategy to enable business groups size-fits-all approach to a more cost-effective, to realize greater value from diverse and multi-tiered data warehouse architecture that optimized data sets. better matches the business requirements and types of data available to our business groups. Intel’s multiple BI data warehouses provide This strategy enabled us to avoid using EDW a dynamic range of BI analytic capabilities, for security BI and Design (HSD) use cases, including: resulting in a cost avoidance of nearly USD • EDW, for analysis of enterprise-wide 250,000 in the first year. Chandhu Yalla structured data BI Engineering Manager/Architecture Owner Intel IT • Apache Hadoop*, for analysis of raw, unstructured data Ajay Chandramouly Big Data Industry, Engagement Manager Intel IT Charles Eden Senior Technical Integrator/Program Manager 1 “ Insight Everywhere: The Growing Importance of Big Intel IT Data and Real-time Analytics.” January 2012 IT@Intel White Paper Using a Multiple Data Warehouse Strategy to Improve BI Analytics Contents BUSINESS CHALLENGE To provide a more comprehensive and accurate view of the company, we implemented a Performing cost-effective business Executive Overview ............................. 1 centralized enterprise data warehouse (EDW) intelligence (BI) analytics in an era of to deliver broader and more sophisticated Business Challenge ............................. 2 big data is an ongoing challenge for BI reporting and data analysis for guiding Intel and other organizations striving to Merging Diverse Data ....................... 2 business decision making. We have allocated improve their business decision making. Big Data’s Impact .............................. 2 the bulk of our BI investments to front-end Intel IT’s overarching BI goal remains tools and data management technology to Solution ................................................... 3 constant: provide the right data to the help improve the integrity of our data. Guiding Principles and Cooperation right people at the right time. Our BI across Business Groups ................... 3 strategy and its implementation are Using a centralized EDW has brought new The Process for Optimizing evolving to accommodate a wide variety levels of data standardization to the company, BI Data Warehouse Selection ......... 4 of business use cases where BI can help strengthening our BI capabilities. For example, Matching BI Data Warehouse solve high-value business problems with having consistent product-name conventions Attributes to the Business actionable insights in near real time. and other identifiers have helped us achieve Use Case ............................................. 5 “a single version of the truth” and enable us Our strategy is shifting as Intel and its business to merge data from diverse sources to gain Conclusion .............................................. 7 groups continue to collect an enormous, business insights from the converged data. diverse, and rapidly expanding volume of Related Topics ....................................... 7 This centralized architecture has proved external and internal data containing potentially effective at Intel, with the EDW a stockpile Acronyms ................................................ 7 valuable insights buried within it. A major for aggregating all enterprise analytics data, portion of this data is large and unstructured, regardless of use case. But the EDW’s key creating up to 90 percent of enterprise data.2 limitation—the inability to deal with raw, The ability to mine and analyze data in various unstructured, and semi-structured data—has forms from many sources gives us deeper become more evident in recent years. and richer insights into business patterns and trends. It helps drive operational efficiencies Big Data’s Impact and competitive advantage in manufacturing, Across the global IT industry, big data product groups, security, marketing, and IT. is prompting a reevaluation of BI data At Intel IT, we strive to provide the optimum warehouse architecture.3 The issue is how to pairing of business-group BI requirements effectively manage data sets whose volume, with the technologies that can perform the variety, and velocity are beyond the ability of tasks most efficiently. traditional database tools to analyze the data. A new generation of big data tools is capable Merging Diverse Data of collecting, processing, and analyzing Over the past decade, Intel’s decentralized unstructured and semi-structured data in a enterprise resource planning (ERP) system timely manner, which means businesses can IT@INteL was aligned to the various lines of business The IT@Intel program connects IT derive meaning from previously unexplored (LOB). While these separate data warehouses professionals around the world with their data sets. With this ability, they can achieve satisfied many business requirements, as a peers inside our organization – sharing deeper and richer insights than were group they were inconsistent in collecting lessons learned, methods and strategies. previously possible in the traditional EDW. Our goal is simple: Share Intel IT best and storing data. For example, the formats for At Intel, our own EDW continues to provide practices that create business value and product and customer names varied across valuable business reporting and ad hoc make IT a competitive advantage. Visit different databases, making it difficult and us today at www.intel.com/IT or contact costly to share data between groups. As a querying with its structured data, but a deluge your local Intel representative if you’d result, the data was shared only rarely. of big data at Intel is escalating both storage like to learn more. 2 “ Mining Big Data in the Enterprise for Better Business 3 “ Data Warehousing Architecture Takes Logical Turn in Big Intelligence,” Intel IT white paper, July 2012. Data Era,” Search Business Intelligence, November 2012. 2 www.intel.com/IT Using a Multiple Data Warehouse Strategy to Improve BI Analytics IT@Intel White Paper and processing costs. Here, as elsewhere, the Our strategy takes into account that knowing offering more rapid development and local introduction of expensive big data platforms what big data can mean to Intel—and managing control. Our strategy has enabled us to avoid has sometimes proved tempting to our it as a corporate asset—is more important than using the EDW for LOB-specific security BI business groups, even though the specific consolidating it in a more traditional, single data and Design (HSD) use cases, resulting in a BI requirements may not justify the cost. warehouse. Expanding our data warehouse cost avoidance of nearly USD 250,000 in the architecture uses the value of the EDW for first year. To offer the optimum solution that best shared enterprise data, yet also extends BI matches BI requirements to platforms, Intel However, BI data warehouses capable of benefits to cases where the unstructured IT’s strategy is to provide a range of BI tackling big data solutions are not the optimal data is evolving, requires special handling, or data warehouses that can accommodate solution in every BI use case. For example, is focused on a limited audience. a variety of needs across Intel’s business depending on the use case, it is often more groups. We have extended our BI strategy to Our EDW remains an important part of our expedient to keep data in a data warehouse include solutions from traditional relational BI strategy. This central data warehouse close to the current transaction system and databases, Not Only SQL (NoSQL) database is optimum when business groups seek data users, minimizing latency problems and systems, data warehouse appliances, data cross-functional, integrated views of the the potential failure points that come with marts, and big data tools—all co-existing with enterprise data. At the same time, it provides each