VIRTUALIZATION CLOUD DEVELOPMENT APPLICATION IT HEALTH NETWORKING ARCHITECTURE STORAGE CENTER MANAGEMENT DATA BI/APPLICATIONS RECOVERY/COMPLIANCE DISASTER SECURITY

1 EDITOR’S NOTE Top-of-Mind Time for 2 IN-MEMORY HELP In-Memory Databases MEET NEED FOR IT SPEED For all the promise they hold, in-memory databases and the process of implementing them require heavy investments in 3 IBM GIVES DB2 MORE GAS WITH company resources and skills. Is it worth it? IN-MEMORY ACCELERATOR 4 ADD-ON SOFTWARE TAKES ORACLE 12C IN NEW DIRECTION EDITOR’S NOTE 1 In-Memory’s Moment in the Sun

In-memory databases used to be terri- 2014 report that in-memory databases could Home tory for niche technology vendors and equally provide “transformational performance im- niche applications. But today vendors of all da- provements” in operational and analytical ap- Editor’s Note tabase stripes—SQL, NoSQL, NewSQL—now plications. But in a video posted on YouTube offer in-memory technology, some as stand- the following month, Rosen said the heavily In-Memory Databases Help alone products and others as add-ons to disk- hyped technology also has “the potential to be Meet Need for IT based database management systems. That the next failed silver bullet from IT.” Challenges Speed includes relational database market leaders Or- abound, he cautioned, including data migration acle, IBM and Microsoft as well as business ap- issues and the proliferation of data silos that IBM Gives DB2 plications bigwig SAP with its HANA system. make it hard to do real-time analytics. More Gas With In an interview with SearchDataManage- This guide explores in-memory database In-Memory Accelerator ment’s Jack Vaughan, data management consul- trends and offers advice to help you get started tant William McKnight said that as the price of on deciding whether the technology is right for Add-On Software RAM declines, “memory in a lot of ways is be- your organization. First, we assess applications Takes Oracle 12c coming the new disk.” There are good reasons that are a good fit for in-memory software. in New Direction for that. Keeping analytical data in memory We follow that with a close look at both IBM’s can sharply reduce query response times and BLU Acceleration technology and Oracle’s in- enable end users to run deeper analyses, Mc- memory add-on for its 12c database. n Knight said. “When you can do that,” he added, “you’re hopefully producing a better business.” Craig Stedman IDC analyst Mike Rosen wrote in a January Executive Editor, SearchDataManagement

2 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES PERFORMANCE 2 In-Memory Databases Help Meet Need for IT Speed

Ten years ago, it would have been un- application performance in two ways. First and Home thinkable to imagine enterprise-class database foremost, maintaining data in main memory management systems running primarily in instead of significantly slower disk-based stor- Editor’s Note main memory. Yet over time, RAM prices have age minimizes or even eliminates the data steadily declined to the point where doing that latency typically associated with database que- In-Memory Databases Help is no longer prohibitively expensive. The cost ries. Second, alternative database architectures Meet Need for IT of memory is orders of magnitude less expen- amenable to in-memory processing enable Speed sive than it used to be, and the plummeting more efficient use of the available memory. prices have opened new opportunities for con- For example, many in-memory technologies IBM Gives DB2 figuring database systems to take advantage of use a columnar layout in tables instead of a More Gas With increased main memory capacities. row-based orientation. Values aligned along In-Memory Accelerator And it’s no longer just startup companies columns are more suitable for compression, developing in-memory databases designed to and the ability to rapidly scan all column values Add-On Software support high-performance processing needs. speeds query execution. Takes Oracle 12c Leading database and software vendors—IBM, in New Direction Oracle, Microsoft, SAP, —are market- ing in-memory database technology, putting A WINNING HAND? money behind their belief that mainstream or- Conceptually, it’s hard to argue against applica- ganizations are ready to consider incorporating tion speed-up and optimized organization of such software into IT systems. data. But in the real world, when should IT and In-memory databases provide accelerated data management practitioners recommend

3 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES PERFORMANCE 2 that transaction processing or business ana- about the stocks of items on trucks or rail cars lytics needs warrant the investments in tech- en route between various sites, updates on traf- nology, resources and new skills required to fic and weather conditions—could help drive transition to an in-memory framework? faster decisions on routing and distribution The practical aspects of that question in- to ensure that goods get to where they need volve weighing the need for increased database to be, when they need to be there. A resulting Home performance versus the associated costs of increase in sales clearly could justify the in- acquiring and deploying an in-memory plat- memory investment. Editor’s Note form. Even though RAM costs have decreased dramatically, systems with large-scale memory In-Memory Databases Help configurations will still carry a healthy price TAKE A LOOK INSIDE Meet Need for IT tag compared with database servers that stay It’s a good idea to take the overall character- Speed with disk storage only. Corporate and busi- istics of your organization into account. In- ness executives might experience sticker shock memory databases are worth considering if one IBM Gives DB2 when they see the in-memory bill. To make an or more of the following terms can be used to More Gas With in-memory technology purchase pay off, you describe the environment you work in. In-Memory Accelerator need to find applications with characteristics that make them a good fit. Open to investing in IT. Corporate execs must Add-On Software The answer lies partly in assessing your or- be willing to spend money on hardware with Takes Oracle 12c ganization’s demand for processing increased enough memory to satisfy the processing needs in New Direction data volumes and the business value of reduced of business applications, even though scaling database response times. Consider this exam- out systems to support in-memory computing ple: Enabling real-time analysis in supply chain carries a higher price than buying disk-heavy management applications that incorporate a database servers does. variety of data streams—inventory data from warehouses and retail locations, information Analytically agile. In-memory systems can

4 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES PERFORMANCE 2 power reporting and analysis applications database. According to a 2013 white paper from that help improve business processes—and data warehouse and database vendor Teradata, results—by enabling end users to make in- 43% of queries against data warehouses it formed decisions on a shorter cycle. For exam- studied accessed just 1% of the available infor- ple, transitioning from weekly to hourly sales mation, while 92% of queries used only 20% of forecasting can lead to the creation of real-time the data at hand. Identifying “hot” data that’s Home product pricing models that increase profit- accessed frequently and keeping it in memory ability—as long as pricing decisions can also should greatly reduce query response times. Editor’s Note be communicated and executed rapidly. In summary, organizations whose business In-Memory Databases Help Supportive of mixed-use development. processes can benefit from real-time data avail- Meet Need for IT Allowing transactional and analytical appli- ability, simultaneous mixed-use applications Speed cations to simultaneously access the same and noticeably faster reporting and analytics database is another way to provide real-time are good candidates for deploying in-memory IBM Gives DB2 analytics capabilities. But resource conflicts databases. There are some scenarios in which More Gas With can cause performance problems with a con- the decision to do so is a no-brainer. But in In-Memory Accelerator ventional relational database, largely due to most cases, the consideration of in-memory the latency associated with finding and access- databases must be aligned with IT spending Add-On Software ing data records stored on disk. With an in- priorities and corporate business objectives— Takes Oracle 12c memory configuration, latency becomes less including a demonstrable awareness of how key in New Direction of an issue. areas of corporate performance could be im- proved by the faster transaction processing and Data-aware. In-memory technology can also access to reports and ad hoc query results that be a valuable tool when a large percentage of in-memory software makes possible. data access calls touch only a small part of a —David Loshin

5 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES TOOLS 3 IBM Gives DB2 More Gas With In-Memory Accelerator

In recent years, technology vendors, in- Some of those traits are the kinds of things Home dustry analysts and corporate data managers that have given analytical databases and have focused much of their attention on ad- NoSQL upstarts much of their allure as alter- Editor’s Note vances in specialized data warehouse engines natives to DB2 and other top relational data- and NoSQL databases. But relational databases bases. For example, columnar table structures, In-Memory Databases Help aren’t standing still. IBM’s addition of a set of often coupled with advanced compression Meet Need for IT in-memory software called BLU Acceleration to techniques, have become associated with the Speed its DB2 database is a case in point. new breed of analytical engines that arose from IBM announced the new technologies in Aster Data, ParAccel and —now part of IBM Gives DB2 2013, expanding its flagship relational database Teradata, Actian and Hewlett-Packard, respec- More Gas With management system (RDBMS) for Linux, Unix tively. But IBM, Microsoft and Oracle have all In-Memory Accelerator and Windows with features such as the ability now released or announced in-memory data- to achieve in-memory processing performance base technologies that use columnar structures Add-On Software even on data sets that are larger than a system’s as add-ons to their RDBMS products. Takes Oracle 12c available memory. Other capabilities added as By reducing the need to pull data off of disks, in New Direction part of the BLU Acceleration rollout include the new in-memory options can reduce the parallel vector processing, support for colum- time it takes to run analytical queries and make nar architectures, enhanced data compres- it easier for data scientists and other end users sion and a function that lets analytical queries to scan data sets in search of relevant informa- skip over unnecessary data to further boost tion—but still within the familiar confines of a performance. SQL-based relational environment.

6 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES TOOLS 3 HIGH-SPEED DATA RAIL increasingly varied forms. The incoming feeds DB2 with BLU Acceleration helped speed up include text and radio messages, videos and “I/O-bound” processing jobs at BNSF Railway other types of non-transactional big data. Co. while still allowing it to capitalize on avail- “I’m up to my elbows in unstructured data,” able in-house RDBMS skills, according to Kent Collins said, before quickly recalibrating that Collins, a database engineer and architect at the statement: “I’m up to my eyeballs.” Having ac- Home Fort Worth, Texas, railroad operator. cess to column-oriented data processing and BNSF, which was an early user of the in- management tools that can be programmed Editor’s Note memory software, has seen speed increases using established SQL methods has been a big of as much as a hundredfold on some queries, step toward taming the overall data deluge, he In-Memory Databases Help Collins said. In addition, the new data com- added. Meet Need for IT pression capabilities had an immediate helpful Speed effect in cutting storage and memory require- ments. “We moved a 400 GB database [to the NEW GAME IN TOWN IBM Gives DB2 in-memory platform], and when we finished While in-memory technologies such as BLU More Gas With it was 80 GB,” he said. And BLU Acceleration Acceleration give relational databases new capa- In-Memory Accelerator doesn’t require the data to be decompressed bilities—and added cachet—they don’t change before being analyzed. the fact that database architectures are being Add-On Software Making room for more data and turbo- opened up to more alternatives, acknowledged Takes Oracle 12c charging queries are both important to BNSF, Bernie Spang, director of strategy and market- in New Direction which is collecting more and more information ing for database software and systems at IBM. about its far-flung railroad operations—and in “We’ve moved from the world where you

New in-memory options can reduce the time it takes to run analytical queries and make it easier for end users to search for relevant information.

7 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES TOOLS 3 defined your data problem and then decided databases are now narrowing the innovation which relational database to use,” Spang said. gap, he added. “Now the question is, ‘What data technology Still, Monash views DB2 with BLU Accel- should I use?’ And even in the RDBMSes, eration as something of a first step down the there’s a difference between the old generation in-memory database road by IBM. The initial and the new generation. It’s a new ball game.” iteration “is a single-server product, and ‘in- Home And many of the recent advances in data memory single server’ is definitely a limita- management technology have come outside tion,” he said. He also noted that the current Editor’s Note of the traditional relational realm, said Curt version is optimized for relatively small 10 TB Monash, president of Monash Research and databases, although it can be ramped up to the In-Memory —Jack Vaughan Databases Help author of the DBMS 2 blog. But relational 20 TB level. Meet Need for IT Speed

IBM Gives DB2 More Gas With In-Memory Accelerator

Add-On Software Takes Oracle 12c in New Direction

8 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES OUTLOOK 4 Add-On Software Takes Oracle 12c in New Direction

The Oracle Database 12c In-Memory thus improving OLTP performance as well. Home Option, announced in September 2013 and During his presentation about the in-mem- slated for release in 2014, brings in-memory ory technology at Oracle OpenWorld 2013, Editor’s Note columnar storage to Oracle databases. The new Oracle CEO Larry Ellison said current applica- technology promises to substantially increase tions won’t need to be modified and database In-Memory Databases Help the performance of analytical queries and, to administrators will be able to implement the Meet Need for IT a lesser degree, accelerate online transaction new option with a simple flick of a software Speed processing (OLTP) applications. Until now, data in Oracle databases has been The new software enables Oracle IBM Gives DB2 stored on disks in a row-oriented format. But 12c users to put data in memory, More Gas With the new add-on enables 12c users to put data in In-Memory using a column-based structure. Accelerator memory, using a column-based structure. The database still maintains the row-based data Add-On Software for OLTP operations, but it copies data to the switch. After data is copied into a system’s Takes Oracle 12c in-memory store for read-only analytical que- memory, analytical queries will automatically in New Direction ries. This dual approach makes it possible to be directed to the in-memory engine, where do real-time, ad hoc analytics on transactional queries won’t incur the typical I/O latency as- data. In addition, the In-Memory Option elim- sociated with data on disks. Because of that inates the need to continue creating analytical and the columnar format, the add-on will be indexes on existing databases that already sup- able to scan billions of rows of data per second, port some (BI) processes, Ellison said. Transactions will continue to be

9 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES OUTLOOK 4 directed to the OLTP engine, and the two data to host it and facilitate communications and stores will be kept transactionally consistent data transfers across your network. with one another. The in-memory software lets you avoid the data warehouse step. Instead of building a sep- arate BI and analytics infrastructure, you scale COST CONSIDERATIONS up an existing OLTP system. Certainly, that Home Whenever you implement or modify a system, also requires an initial outlay, but hardware and you’re likely to incur technology, development, maintenance costs are likely to be less. Editor’s Note quality assurance and deployment costs. Spe- On the other hand, if you’ve already imple- cifics about the 12c in-memory feature are in mented a separate data warehouse, the invest- In-Memory Databases Help short supply now, particularly licensing costs ment required to add the In-Memory Option Meet Need for IT and details about hardware and system resource might not be worth the tradeoff, especially if Speed requirements. But it’s reasonable to assume your infrastructure is operating smoothly. In that users of the Oracle add-on—and similar such cases, you’ll need to evaluate whether the IBM Gives DB2 technologies from other mainstream database real-time benefit of in-memory processing More Gas With vendors—could see lower costs overall com- offsets the associated costs before deciding to In-Memory Accelerator pared with building a separate data warehouse make the switch. to run their BI applications. Add-On Software For example, suppose you plan to implement Takes Oracle 12c a new BI system. You might decide to take a MORE TO THINK ABOUT in New Direction traditional approach: Create a data warehouse There are some other issues to consider as and set up extract, transform and load pro- well. First, you’ll have to decide what do to cesses to pull transactional data into it. That about historical data. A data warehouse serves not only requires the resources needed to plan, as a storehouse for information that is no lon- develop, test, implement and maintain the data ger relevant to day-to-day transaction pro- warehouse, but also new hardware and software cessing but is still needed for analytical uses

1 0 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES OUTLOOK 4 requiring a broad perspective over time. You’ll Sharing system resources also doesn’t become need to account for your organization’s accu- a bone of contention: A processing-intensive mulated data and ensure that you have systems analytical query, for example, won’t affect and processes in place to handle it. Large sets OLTP users. If you do decide to combine trans- of historical data could translate into extremely action and analytical processing on a single large in-memory data sets that bog down system with an in-memory database add-on, Home performance. you must create a contingency plan for dealing with potential performance fallout. Editor’s Note The in-memory approach might be The in-memory approach might be par- ticularly useful for exploratory or investiga- In-Memory particularly­ useful for exploratory tive queries that involve scanning large sets Databases Help or investigative­ queries. Meet Need for IT of data, and for operational BI and analytics Speed applications that look at transactional data in In addition, with separate OLTP and data real or near real time. Again, though, keep in IBM Gives DB2 warehouse platforms, issues that arise in one mind that, despite Oracle’s promise of an easy More Gas With have minimal impact on the other. If one goes implementation, any update of this magnitude In-Memory Accelerator down, the other keeps running. The separa- requires additional technology and resources to tion makes troubleshooting and maintenance deploy and support the new setup. It’s prudent Add-On Software less complicated when systems encounter to ingrain in your memory the old adage that Takes Oracle 12c problems, and fewer end users are affected by there’s no such thing as a free lunch. in New Direction system downtime or reduced performance. —Robert Sheldon

1 1 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES ABOUT THE AUTHORS DAVID LOSHIN is president of Knowledge Integrity Inc., a consulting, training and development services company that works with clients on big data, business intelligence and data management projects. He also is the author of numerous books, including Big Data Analytics. Email him at [email protected]. Top-of-Mind Time for In-Memory Databases is a SearchDataManagement.com e-publication. Home Scot Petersen | Editorial Director JACK VAUGHAN is editor of SearchDataManagement. Editor’s Note He covers topics such as big data management, data Jason Sparapani | Managing Editor, E-Publications warehousing, databases and data integration. Vaughan Joe Hebert | Associate Managing Editor, E-Publications In-Memory previously was an editor for TechTarget’s SearchSOA, Craig Stedman | Executive Editor Databases Help SearchVB, TheServerSide and SearchDomino websites. Melanie Luna | Managing Editor Meet Need for IT Email him at [email protected] and follow him Linda Koury | Director of Online Design Speed on Twitter: @JackVaughanatTT. Neva Maniscalco | Graphic Designer IBM Gives DB2 Doug Olender | Publisher ROBERT SHELDON is a technical consultant and the More Gas With [email protected] author of numerous books, articles and training materi- In-Memory Annie Matthews | Director of Sales Accelerator als related to Microsoft Windows, relational database [email protected] management systems and the design and implementa- TechTarget Add-On Software tion of business intelligence systems. Email him at 275 Grove Street, Newton, MA 02466 Takes Oracle 12c [email protected]. www.techtarget.com in New Direction © 2014 TechTarget Inc. No part of this publication may be transmitted or re- produced in any form or by any means without written permission from the publisher. TechTarget reprints are available through The YGS Group. About TechTarget: TechTarget publishes media for information technology professionals. More than 100 focused websites enable quick access to a deep store of news, advice and analysis about the technologies, products and pro- cesses crucial to your job. Our live and virtual events give you direct access to independent expert commentary and advice. At IT Knowledge Exchange, our social community, you can get advice and share solutions with peers and experts.

1 2 TOP-OF-MIND TIME FOR IN-MEMORY DATABASES