Predictive Analytics in Action: Real-World Examples and Advice Predictive Analytics Projects Are Inherently Complex and Potentially Costly

Home , Predictive analytics

s ation A ppli c ture ture c c e v ery/Complian c o An Essential Guide c e/ I ntellegen ss ation D e v elopment ation ter R e s a c urity ata Center M anagement ata Virtualization Cloud A ppli c N etworking A r c hite Storage D Bu s ine D i s Se

Predictive Analytics in Action: Real-World Examples and Advice Predictive analytics projects are inherently complex and potentially costly. But for organizations that get it right, they can pay off in improved decision making and competitive advantages over business rivals.

1 2 3 4 Editor’s Note Predictive Recipe Surveys Point Analytics for Predictive to Skills, Training Programs Analytics as Predictive Need Open Success Analytics Organizational Includes Hurdles Minds One Part Storyteller To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to editor’s note deselect the #. This will keep the slug text 1 in front of the number All pages: text begins on this baseline

Modeling the Future: OPENER 3 lines is max title length. A Challenging but Rewarding Style title. Then use hard return to push last line of title to sit on Proposition this baseline.

OPENER Who wouldn’t want to predict the future, especially when money is at 1st text baseline begins here. stake? Alas, businesses can’t just rely on crystal balls, tarot cards and palm readers—at least if they want to stay in business. But companies can turn to Home predictive analytics software to help them peer into the business future—for example, to predict which customers are likely to be open to cross-selling of- Editor’s Note fers and which ones might not be worth additional sales attention. But they don’t call it advanced analytics for nothing. If your organization is Predictive looking to deploy and use predictive analytics tools, you’ll need to make sure Analytics Programs Need Open that you have the right level of analytics skills in place. Time for an infusion Organizational of data scientists, perhaps? Building predictive models is a complex, time- Minds consuming process that requires trial-and-error testing in order to get the algorithms to produce the desired analytical results. And convincing busi- Recipe for Predictive Analytics ness and operational managers to trust what the models are telling them and Success Includes adjust their strategies and processes accordingly is another big challenge. One Part Storyteller This three-part guide offers practical advice from experienced analytics professionals and industry consultants on how to successfully manage a predictive analytics program. The lead story details key steps to take in develop- Surveys Point to Skills, Training as ing and implementing a program, starting with ensuring that your company Predictive Analytics Hurdles is open to the possibilities enabled by predictive analytics. Next, we recount the lessons that one analytics exec has learned about building a predictive analytics team. And we report on a pair of surveys pointing to a lack of skills and proper training as predictive analytics inhibitors.

Craig Stedman Executive Editor, SearchBusinessAnalytics.com

2 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Best deselect the #. Practices This will keep the slug text 2 in front of the number All pages: text begins on this baseline

OPENER 3 lines is max title length. Predictive Analytics Programs Style title. Then use hard return to push last line of title to sit on Need Open Organizational Minds this baseline.

OPENER Has the current fervor to pounce on every piece of available data for po- 1st text baseline begins here. tential analytical uses spawned a world in which information often is collected for its own sake? Sometimes it might seem that way. But in the Home ever-expanding universe of “big data,” predictive analytics software is one technology that can take advantage of the great variety of data accumulated Editor’s Note by an organization as it works to model customer behavior and future business scenarios. Predictive And using predictive analytics tools to interpret data is becoming more Analytics Programs Need Open important to businesses: The most successful companies and rising-star en- Organizational terprises sedulously employ them to help point the way forward on business Minds strategies and operations, according to analysts who focus on advanced analytics technologies. But that doesn’t happen magically, they cautioned; orga- Recipe for Predictive Analytics nizations need to take the right steps to develop effective predictive analytics Success Includes programs. One Part Storyteller In many industries, getting a leg up on the competition can be more challenging than ever—especially if companies are set in their ways. The starting point in embracing predictive analytics should be ensuring that an organiza- Surveys Point to Skills, Training as tion has a proper frame of mind about using the technology, the analysts said. Predictive Analytics Hurdles An open, dexterous attitude that’s naturally curious, eager to learn and will- ing to adapt will produce the best results. Douglas Laney, an analyst at Gartner Inc. in Stamford, Conn., thinks a predictive analytics program should begin by questioning historical business methods while searching far and wide for better ones. Companies “should not only focus on how things have been done in the past but be open to big ideas for innovations and transformations,” he said. “This could mean

3 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Best deselect the #. Practices This will keep the slug text 2 in front of the number All pages: applying measures effective in other industries to your industry.” Such a text begins on this baseline mind-set should extend to the point of embracing approaches that “radically OPENER change the way business processes are done” in an organization, Laney added. 3 lines is max title length. Style title. Then use hard return With that in mind, the mentality of the players—particularly the business to push last line of title to sit on this baseline. managers who are being asked to buy into the findings of predictive mod- Once a predictive els—is frequently the key variable that analytics strategy OPENER determines the success or failure of 1st text baseline begins here. is in place, it’s predictive analytics programs. A per- spicacious corporate culture champions time to begin the Home objectivity, welcomes new ideas and is analysis process. naturally flexible. Conversely, a retro- Editor’s Note grade one resists change and draws heavily on existing biases and subjective formulas. “Resisting new ways of doing things is the reason most projects Predictive fail,” said John Lucker, head of Deloitte Consulting LLP’s advanced analytics Analytics Programs Need Open and modeling practice. Organizational Minds Keep Your Eyes on the Business Prize The grand plan of a predictive analytics deployment should also begin with a Recipe for Predictive Analytics clear set of business objectives, said Thomas “Tony” Rathburn, a senior con- Success Includes sultant at The Modeling Agency LLC, a Pittsburgh-based consulting com- One Part Storyteller pany that focuses on data mining and predictive analytics. Then, he added, a team-oriented strategy is needed to advance those objectives. That is best constructed through substantive discussions involving program managers, Surveys Point to Skills, Training as predictive modelers, data analysts and business representatives. Predictive Analytics Hurdles So critical is the strategy development process that Eric King, president and founder of The Modeling Agency, recommends retaining “a seasoned strategic mentor” to help lead the effort and keep it on track. Once a predictive analytics strategy is in place, it’s time to begin the analysis process. Laney said “chewy” questions that probe deeply into data will unearth findings with high operational value. The truly useful ones, he said, are multifaceted—for example, “How can we grow new customers by 20%

4 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Best deselect the #. Practices This will keep the slug text 2 in front of the number All pages: per year for a certain product line without cannibalizing other product lines text begins on this baseline given the range of economic forecasts, competitor trends and changing con- OPENER sumer demands?” Run through predictive models, such questions can con- 3 lines is max title length. Style title. Then use hard return tribute in a big way to driving new business, according to Laney. to push last line of title to sit on this baseline.

Building Models is a Testing Process After choosing and deploying the predictive analytics tools that best fit the OPENER job at hand, developing models is the next step. Mike Gualtieri, an analyst at 1st text baseline begins here. Forrester Research Inc. in Cambridge, Mass., said analytics algorithms should be run on 70% of a data set to create an effective predictive model. “Then Home you test that model on the remaining 30%,” he said. Editor’s Note Analytics strategies Completed models should be regu- and tactics that worked larly tested and enhanced as needed, initially will need to Predictive and a set of performance metrics Analytics Programs be revisited and per- Need Open should be put in place for tracking their Organizational haps revised in order accuracy, Gualtieri added—all part of a Minds process for “continuous monitoring of to continue achieving the predictive analytics model.” optimal results. Recipe for Predictive Analytics Moreover, said other analysts, the Success Includes entire predictive analytics process requires regular monitoring as business One Part Storyteller needs and the nature of the data being collected by an organization change. Analytics strategies and tactics that worked initially will need to be revisited and perhaps revised in order to continue achieving optimal results. Surveys Point to Skills, Training as The mark of a truly successful predictive analytics program, Lucker said, Predictive Analytics Hurdles is when some of the cost savings or business gains realized from an ongoing analysis project can be applied to pay for the next one so no new dollars need to be spent. “Using the value of each project to fund downstream efforts is an evolutionary approach that comes with a [built-in] return on investment,” he said. —Roger du Mars

5 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Team deselect the #. Building This will keep the slug text 3 in front of the number All pages: text begins on this baseline

Recipe for Predictive OPENER 3 lines is max title length. Analytics Success Includes Style title. Then use hard return to push last line of title to sit on One Part Storyteller this baseline.

OPENER The secret to building a successful predictive analytics team is finding 1st text baseline begins here. people with statistical analysis, programming and—perhaps most important—storytelling skills, according to one practitioner. Home It’s important to find multitalented people because, oftentimes, predictive analytics teams are rather small, said Jennifer Golec, vice president of strate- Editor’s Note gic analytics at XL Insurance Inc. Multifaceted individuals offer a higher level of flexibility, she said, and that comes in handy when resources are tight. Predictive Ideally, predictive analytics professionals should be one part programmer, Analytics Programs Need Open Golec said, because they’ll be working with a great deal of information and Organizational conducting exploratory analysis. Commercial software can help in these ar- Minds eas, she explained, but some programming skills will still be helpful for tasks like manipulating or massaging data Recipe for and creating new variables. Predictive Analytics Predictive analyt- Success Includes Predictive analytics professionals One Part ics professionals should also focus on developing sta- Storyteller should focus on tistical analysis skills because those are necessary for building multivariate developing statis- Surveys Point to Skills, Training as models, statistical tools that use multi- tical analysis skills. Predictive Analytics Hurdles ple variables to forecast outcomes. “The third piece is that you have to be part storyteller. You have to be able to interpret those results,” Golec said. “[That means] really being able to interpret the insight that you pull from the data. You have to be able to relay that because if you don’t, you’ll be sitting on this great model and you won’t be able to implement it.”

6 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Team deselect the #. Building This will keep the slug text 3 in front of the number All pages: More Than Just Crunching Numbers text begins on this baseline The popular 2011 film Moneyball—which tells the story of Oakland A’s gen- OPENER eral manager Billy Beane, who used analytics to find undervalued players and 3 lines is max title length. Style title. Then use hard return build a great baseball team—might give the mistaken impression that ana- to push last line of title to sit on this baseline. lytics is all about crunching numbers. But it’s much more than that, according to Golec. Organizations must also strive to understand how the results of predictive models translate to the real business world. OPENER “Sometimes that is the danger with 1st text baseline begins here. products like SAS,” Golec said. “They Organizations must make it so easy to push the data in and Home also strive to under- hit the button and have something stand how the results come out. But if you’re not trained to Editor’s Note of predictive models understand and interpret that output, translate to the real you could end up with junk and you Predictive might not know it.” business world. Analytics Programs Need Open Golec, who has a doctorate in eco- Organizational nomics from the University of Missouri and previously ran a predictive ana- Minds lytics program for insurance provider The Hartford, began working for XL Insurance and its global parent company, XL Group PLC, in October 2011. Recipe for Predictive Analytics Success Includes Analytics Goal: Ratcheting Back on Risk One Part Storyteller One of her first tasks was to find a software vendor that could help the prop- erty and casualty company build out its fledgling predictive analytics program. XL Insurance launched the program to do a better job of avoiding Surveys Point to Skills, Training as unnecessary risk and, ultimately, improving its loss ratio. “The loss ratio is Predictive Analytics Hurdles losses over premiums,” Golec said. “The lower it is, the more profitable you a re.” Golec took a close look at SPSS, which was acquired by IBM in 2011, and Wolfram Research’s Mathematica tools. But she had worked with software from SAS Institute Inc. in the past and decided to do so again. “Half the battle is working with the data, manipulating the data and getting it into a form that allows you to actually do the modeling,” she said.

7 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Team deselect the #. Building This will keep the slug text 3 in front of the number All pages: “SAS allows us to get the data into the shape and form that we want.” text begins on this baseline XL Insurance is using several SAS products, including SAS/STAT, a statis- OPENER tical analysis tool; SAS Graph, a visual tool that allows users to present in- 3 lines is max title length. Style title. Then use hard return formation in charts and graphs; SAS Enterprise Guide (EG), which makes it to push last line of title to sit on this baseline. easier to do exploratory analysis of data stores; and JMP, a data visualization tool.

OPENER Ensuring Adoption Central to Implementation Process 1st text baseline begins here. The team at XL Insurance is in the process of building predictive models for risk assessment. The next step, according to Golec, is to implement those Home models and closely monitor and measure the results. Golec said the toughest aspects of the implementation phase will likely re- Editor’s Note volve around change management and, specifically, getting the right people to adopt predictive analytics findings as part of their usual routines. Making Predictive sure that any workflow or architecture changes are properly documented is Analytics Programs Need Open also a major challenge. Organizational Another is “making sure that we’ve come up with how we’re going to Minds track it and make sure it’s working,” she said. “But I think the big thing in implementation is just achieving that buy-in and making sure that it’s Recipe for Predictive Analytics used.” —Mark Brunelli Success Includes One Part Storyteller

Surveys Point to Skills, Training as Predictive Analytics Hurdles

8 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Challenges deselect the #. This will keep the slug text 4 in front of the number All pages: text begins on this baseline

OPENER 3 lines is max title length. Surveys Point to Skills, Training Style title. Then use hard return to push last line of title to sit on as Predictive Analytics Hurdles this baseline.

OPENER Businesses recognize the potential of predictive analytics, yet there’s 1st text baseline begins here. a large gap between those who see it as important and those who actually use the technology, according to a pair of surveys conducted by Ventana Home Research. The market research and consulting company, based in San Ramon, Calif., Editor’s Note conducted an initial study in early 2011 which found that only 13% of the re- sponding organizations were using predictive analytics. But 80% indicated Predictive it was important or very important, said David Menninger, who was infor- Analytics Programs Need Open mation technology research director at Ventana when he was interviewed for Organizational this story; he has since taken a job with a technology vendor. Minds The reason for that gap? While most businesses consider predictive analytics important, those that struggle with it lack both the skills and the training Recipe for Predictive Analytics required to be successful with the technology, Menninger discovered in a fol- Success Includes low-up study. “Organizations are least mature in the people aspect,” he said. One Part Storyteller That conclusion was drawn from the results of a three-month survey of 198 respondents measured against Ventana’s predictive analytics maturity model, which was used to rate the survey responses across the categories of Surveys Point to Skills, Training as process, information, technology and people. Predictive Analytics Hurdles The survey revealed that self-service predictive analytics, or end users creating and deploying their own analyses, has not been widely deployed, de- spite a wave of easier-to-use predictive analytics tools coming to market.

Analytics Skills Not a Common Trait In fact, almost half of the respondents questioned whether users have the background to produce their own analyses. For the nonbelievers, Menninger

9 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Challenges deselect the #. This will keep the slug text 4 in front of the number All pages: said it came down to two reasons: 83% reported users didn’t have enough text begins on this baseline skills, and 58% reported users didn’t understand the mathematics involved. OPENER “[Predictive analytics] requires the specialist skill set—the data scientist, 3 lines is max title length. Style title. Then use hard return the statistician, the data mining experts—to be successful,” he said. to push last line of title to sit on this baseline. Instead of relying on users, 63% of respondents reported their organization had a specialized team for predictive analytics or that the task fell to the business intelligence (BI) and data warehousing (DW) team. But even OPENER then, Menninger’s research indicated that how satisfied respondents are with 1st text baseline begins here. the way predictive analytics is used in their organizations (two-thirds said they’re satisfied) depends, in part, on Home who does the work. Many organizations The highest levels of satisfaction, Editor’s Note are not doing a great 70%, came from respondents who job providing the ongo- worked for organizations that employed ing support needed to Predictive specialists such as data scientists to Analytics Programs successfully maintain Need Open produce the predictive analytics find- Organizational ings. The lowest levels of satisfaction, a strong predictive Minds 59%, came from respondents whose analytics program. BI and DW teams were in charge of Recipe for Predictive Analytics the work.“I think it’s common for organizations to think this will naturally Success Includes fall out of the BI and DW team,” Menninger said. “But what this tells me is One Part Storyteller that this is not a generalized skill of BI and DW teams.”

Surveys Point to Support Lacking for Predictive Analytics Users Skills, Training as Many organizations are also not doing a great job providing the ongoing sup- Predictive Analytics Hurdles port needed to successfully maintain a strong predictive analytics program, Menninger said. According to the survey results, businesses are most successful at providing concept and technique training (44% of respondents felt this was adequate) and have the most trouble delivering help desk support (24% reported this was adequate). More than a third of respondents, 42%, also found product training to be adequate.

10 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move just outside of blue box to Challenges deselect the #. This will keep the slug text 4 in front of the number All pages: Menninger said concept, technique and product training may drive a stron- text begins on this baseline ger sense of satisfaction because they require “specialized knowledge” over OPENER the broader needs—and the skills—required by something like a help desk. 3 lines is max title length. Style title. Then use hard return “I think it relates back to necessary skills,” he said. “How do you have peo- to push last line of title to sit on this baseline. ple on the help desk supporting a more complicated topic? The help desk re- The most effective sources would need to have specialized type of support was OPENER training and skills to be able to provide 1st text baseline begins here. brought about by meaningful support.” Yet respondents indicated that, in help desk resources. Home addition to concept and technique training, the most effective type of support was brought about by help desk Editor’s Note resources. Organizations that provided either support feature adequately had an 89% satisfaction rating on average, according to the survey results. Predictive “I suspect that organizations probably think first about doing product Analytics Programs Need Open training and less about this generalized set of skills and help desk resources,” Organizational Menninger said. Minds While the level of satisfaction in a predictive analytics program may wax and wane based on training, Menninger said the root of that issue is most Recipe for Predictive Analytics likely derived from what he considers to be an even bigger problem—a lack of Success Includes skills. One Part Storyteller “The skills issue is significant,” he said. “It appears to have been preventing organizations in the past from either choosing to tackle predictive analytics or [being able] to tackle it successfully.” Surveys Point to Skills, Training as Menninger said predictive analytics requires a deeper kind of knowledge. Predictive Analytics Hurdles “It’s unrealistic today to expect the technology to deliver self-service capa- bilities,” he said. “[But] if you have the right skills, the technology is available to be successful with predictive analytics.” —Nicole Laskowski

11 Predictive Analytics in Action: Real-World Examples and Advice To change slug and # txt.

On your page, in this order, deselect slug first, then move about just outside of blue box to the deselect the #. authors This will keep the slug text in front of the number All pages: Roger du Mars is a freelance writer text begins on this baseline based in Redmond, Wash. He has written for Time, USA Today and The Bos- OPENER 3 lines is max title length. ton Globe, and he was the Seoul, South Style title. Then use hard return Korea, bureau chief of Asiaweek and the to push last line of title to sit on this baseline. South China Morning Post. Email him at Predictive Analytics in Action: Real-World [email protected]. Examples and Advice is a SearchBusinessAnalytics.com e-publication. Mark Brunelli is news director for OPENER 1st text baseline begins here. SearchBusinessAnalytics.com and the Barney Beal other websites in TechTarget Inc.’s Senior Executive Editor Business Applications and Architecture Home Jason Sparapani Media Group. He oversees coverage of Managing Editor, E-Publications topics such as business intelligence and Nicole Laskowski analytics, data management, customer Editor’s Note News Editor relationship management and business Craig Stedman applications. Email him at mbrunelli@ Executive Editor Predictive techtarget.com and follow him on Linda Koury Analytics Programs Twitter: @Brunola88. Director of Online Design Need Open Organizational Mike Bolduc Publisher Minds Nicole Laskowski is the news editor [email protected] for SearchBusinessAnalytics.com. She covers business intelligence, analytics Ed Laplante Recipe for Director of Sales and data visualization technologies [email protected] Predictive Analytics and trends. Email her at nlaskowski Success Includes One Part @techtarget.com and follow her on TechTarget Storyteller Twitter: @TT_Nicole. 275 Grove Street, Newton, MA 02466 www.techtarget.com

Surveys Point to © 2012 TechTarget Inc. No part of this publication Skills, Training as may be transmitted or reproduced in any form or Predictive Analytics by any means without written permission from the publisher. TechTarget reprints are available through Hurdles The YGS Group.

About TechTarget: TechTarget publishes media for information technology professionals. More than 100 focused websites enable quick access to a deep store of news, advice and analysis about the technologies, products and processes crucial to your job. Our live and virtual events give you direct access to independent expert commentary and advice. At IT Knowledge Exchange, our social community, you can get advice and share solutions with peers and experts.

12 Predictive Analytics in Action: Real-World Examples and Advice