sign in sign up
<<
Home , Predictive analytics

Predictive

Predictive analytics encompasses a variety of statisti- 2 Types cal techniques from modeling, , and that analyze current and historical facts to Generally, the term predictive analytics is used to mean make about future, or otherwise unknown, predictive modeling, “scoring” data with predictive mod- [1][2] events. els, and . However, people are increasingly In business, predictive models exploit patterns found in using the term to refer to related analytical disciplines, historical and transactional data to identify risks and op- such as descriptive modeling and decision modeling or portunities. Models capture relationships among many optimization. These disciplines also involve rigorous data factors to allow assessment of risk or potential associated analysis, and are widely used in business for segmentation with a particular set of conditions, guiding decision mak- and decision making, but have different purposes and the ing for candidate transactions.[3] statistical techniques underlying them vary. Predictive analytics is used in ,[4] ,[5] financial services,[6] , telecommunications,[7] ,[8] travel,[9] healthcare,[10] 2.1 Predictive models pharmaceuticals[11] and other fields. Predictive models are models of the relation between the One of the most well known applications is credit scor- specific performance of a unit in a sample and one or [1] ing, which is used throughout financial services. Scor- more known attributes or features of the unit. The objec- ing models process a customer’s credit history, appli- tive of the model is to assess the likelihood that a similar cation, customer data, etc., in order to rank-order individ- unit in a different sample will exhibit the specific per- uals by their likelihood of making future credit payments formance. This category encompasses models that are on time. in many areas, such as marketing, where they seek out subtle data patterns to answer questions about customer performance, such as detection models. Predic- tive models often perform calculations during live trans- 1 Definition actions, for example, to evaluate the risk or opportunity of a given customer or transaction, in order to guide a Predictive analytics is an area of data mining that deals decision. With advancements in computing speed, indi- with extracting information from data and using it to pre- vidual agent modeling systems have become capable of dict trends and behavior patterns. Often the unknown simulating human behaviour or reactions to given stimuli event of interest is in the future, but predictive analytics or scenarios. can be applied to any type of unknown whether it be in The available sample units with known attributes and the past, present or future. For example, identifying sus- known performances is referred to as the “training sam- pects after a crime has been committed, or credit card ple.” The units in other sample, with known attributes but fraud as it occurs.[12] The core of predictive analytics re- un-known performances, are referred to as “out of [train- lies on capturing relationships between explanatory vari- ing] sample” units. The out of sample bare no chrono- ables and the predicted variables from past occurrences, logical relation to the training sample units. For exam- and exploiting them to predict the unknown outcome. It ple, the training sample may consists of literary attributes is important to note, however, that the accuracy and us- of writings by Victorian authors, with known attribution, ability of results will depend greatly on the level of data and the out-of sample unit may be newly found writing analysis and the quality of assumptions. with unknown authorship; a predictive model may aid

1 2 3 APPLICATIONS the attribution of the unknown author. Another example 3.1 Analytical customer relationship man- is given by analysis of blood splatter in simulated crime agement (CRM) scenes in which the out-of sample unit is the actual blood splatter pattern from a crime scene. The out of sample Analytical Customer Relationship Management is a fre- unit may be from the same time as the training units, from quent commercial application of Predictive Analysis. a previous time, or from a future time. Methods of predictive analysis are applied to customer data to pursue CRM objectives, which involve construct- ing a holistic view of the customer no matter where their information resides in the company or the department 2.2 Descriptive models involved. CRM uses predictive analysis in applications for marketing campaigns, sales, and customer services to name a few. These tools are required in order for a com- Descriptive models quantify relationships in data in a way pany to posture and focus their efforts effectively across that is often used to classify customers or prospects into the breadth of their customer base. They must analyze groups. Unlike predictive models that focus on predicting and understand the products in demand or have the po- a single customer behavior (such as credit risk), descrip- tential for high demand, predict customers’ buying habits tive models identify many different relationships between in order to promote relevant products at multiple touch customers or products. Descriptive models do not rank- points, and proactively identify and mitigate issues that order customers by their likelihood of taking a particular have the potential to lose customers or reduce their abil- action the way predictive models do. Instead, descriptive ity to gain new ones. Analytical Customer Relationship models can be used, for example, to categorize customers Management can be applied throughout the customers by their product preferences and life stage. Descriptive lifecycle (acquisition, relationship growth, retention, and modeling tools can be utilized to develop further models win-back). Several of the application areas described be- that can simulate large number of individualized agents low (direct marketing, cross-sell, customer retention) are and make predictions. part of Customer Relationship Managements.

3.2 Clinical decision support systems 2.3 Decision models Experts use predictive analysis in health care primarily to determine which patients are at risk of developing certain Decision models describe the relationship between all the conditions, like diabetes, asthma, heart disease, and other elements of a decision — the known data (including re- lifetime illnesses. Additionally, sophisticated clinical de- sults of predictive models), the decision, and the forecast cision support systems incorporate predictive analytics to results of the decision — in order to predict the results of support medical decision making at the point of care. A decisions involving many variables. These models can be working definition has been proposed by Robert Hay- used in optimization, maximizing certain outcomes while ward of the Centre for Health Evidence: “Clinical Deci- minimizing others. Decision models are generally used to sion Support Systems link health observations with health develop decision logic or a set of business rules that will knowledge to influence health choices by clinicians for produce the desired action for every customer or circum- improved health care.” stance.

3.3 Collection analytics

3 Applications Many portfolios have a set of delinquent customers who do not make their payments on time. The financial insti- tution has to undertake collection activities on these cus- Although predictive analytics can be put to use in many tomers to recover the amounts due. A lot of collection applications, we outline a few examples where predictive resources are wasted on customers who are difficult or analytics has shown positive impact in recent years. impossible to recover. Predictive analytics can help opti- 3.7 Fraud detection 3 mize the allocation of collection resources by identifying fective combination of product versions, marketing ma- the most effective collection agencies, contact strategies, terial, communication channels and timing that should be legal actions and other strategies to each customer, thus used to target a given consumer. The goal of predictive significantly increasing recovery at the same time reduc- analytics is typically to lower the cost per order or cost ing collection costs. per action.

3.4 Cross-sell 3.7 Fraud detection

Often corporate organizations collect and maintain abun- Fraud is a big problem for many businesses and can be of dant data (e.g. customer records, sale transactions) as various types: inaccurate credit applications, fraudulent exploiting hidden relationships in the data can provide a transactions (both offline and online), identity thefts and competitive advantage. For an organization that offers false insurance claims. These problems plague firms of multiple products, predictive analytics can help analyze all sizes in many industries. Some examples of likely vic- [13] customers’ spending, usage and other behavior, leading to tims are credit card issuers, insurance companies, re- efficient cross sales, or selling additional products to cur- tail merchants, manufacturers, business-to-business sup- rent customers.[2] This directly leads to higher profitabil- pliers and even services providers. A predictive model ity per customer and stronger customer relationships. can help weed out the “bads” and reduce a business’s ex- posure to fraud. Predictive modeling can also be used to identify high-risk 3.5 Customer retention fraud candidates in business or the public sector. Mark Nigrini developed a risk-scoring method to identify audit With the number of competing services available, busi- targets. He describes the use of this approach to detect nesses need to focus efforts on maintaining continuous fraud in the franchisee sales reports of an international consumer satisfaction, rewarding consumer loyalty and fast-food chain. Each location is scored using 10 predic- minimizing . Businesses tend to re- tors. The 10 scores are then weighted to give one final spond to customer attrition on a reactive basis, acting only overall risk score for each location. The same scoring after the customer has initiated the process to terminate approach was also used to identify high-risk check kiting service. At this stage, the chance of changing the cus- accounts, potentially fraudulent travel agents, and ques- tomer’s decision is almost impossible. Proper applica- tionable vendors. A reasonably complex model was used tion of predictive analytics can lead to a more proactive to identify fraudulent monthly reports submitted by divi- retention strategy. By a frequent examination of a cus- sional controllers.[14] tomer’s past service usage, service performance, spend- ing and other behavior patterns, predictive models can The Internal Revenue Service (IRS) of the United States also uses predictive analytics to mine tax returns and iden- determine the likelihood of a customer terminating ser- [13] vice sometime soon.[7] An intervention with lucrative of- tify tax fraud. fers can increase the chance of retaining the customer. Recent advancements in technology have also introduced Silent attrition, the behavior of a customer to slowly but predictive behavior analysis for web fraud detection. This steadily reduce usage, is another problem that many com- type of solution utilizes heuristics in order to study normal panies face. Predictive analytics can also predict this be- web user behavior and detect anomalies indicating fraud havior, so that the company can take proper actions to attempts. increase customer activity. 3.8 Portfolio, product or economy-level 3.6 Direct marketing

When marketing consumer products and services, there is Often the focus of analysis is not the consumer but the the challenge of keeping up with competing products and product, portfolio, firm, industry or even the economy. consumer behavior. Apart from identifying prospects, For example, a retailer might be interested in predicting predictive analytics can also help to identify the most ef- store-level demand for inventory management purposes. 4 5 ANALYTICAL TECHNIQUES

Or the Federal Reserve Board might be interested in pre- 4 Technology and influ- dicting the unemployment rate for the next year. These ences types of problems can be addressed by predictive ana- lytics using techniques (see below). They can also be addressed via machine learning approaches which Big data is a collection of data sets that are so large and transform the original time series into a feature vector complex that they become awkward to work with us- space, where the learning algorithm finds patterns that ing traditional management tools. The volume, have predictive power.[15][16] variety and velocity of big data have introduced chal- lenges across the board for capture, storage, search, shar- ing, analysis, and visualization. Examples of big data 3.9 Risk management sources include web logs, RFID and sensor data, social networks, Internet search indexing, call detail records, When employing risk management techniques, the re- military surveillance, and complex data in astronomic, sults are always to predict and benefit from a future sce- biogeochemical, genomics, and atmospheric sciences. nario. The Capital asset pricing model (CAP-M) “pre- Thanks to technological advances in computer hardware dicts” the best portfolio to maximize return, Probabilistic — faster CPUs, cheaper memory, and MPP architectures Risk Assessment (PRA)--when combined with mini- — and new technologies such as Hadoop, MapReduce, Delphi Techniques and statistical approaches yields ac- and in-database and text analytics for processing big data, curate forecasts and RiskAoA is a stand-alone predic- it is now feasible to collect, analyze, and mine mas- tive tool.[17] These are three examples of approaches that sive amounts of structured and for new [13] can extend from project to market, and from near to insights. Today, exploring big data and using predic- long term. Underwriting (see below) and other busi- tive analytics is within reach of more organizations than ness approaches identify risk management as a predictive ever before and new methods that are capable for han- [18] [19] method. dling such datasets are proposed

3.10 Underwriting 5 Analytical Techniques

Many businesses have to account for risk exposure due The approaches and techniques used to conduct predic- to their different services and determine the cost needed tive analytics can broadly be grouped into regression tech- to cover the risk. For example, auto insurance providers niques and machine learning techniques. need to accurately determine the amount of premium to charge to cover each automobile and driver. A financial company needs to assess a borrower’s potential and abil- 5.1 Regression techniques ity to pay before granting a loan. For a health insurance provider, predictive analytics can analyze a few years of Regression models are the mainstay of predictive analyt- past medical claims data, as well as lab, pharmacy and ics. The focus lies on establishing a mathematical equa- other records where available, to predict how expensive tion as a model to represent the interactions between the an enrollee is likely to be in the future. Predictive analyt- different variables in consideration. Depending on the ics can help underwrite these quantities by predicting the situation, there is a wide variety of models that can be chances of illness, default, bankruptcy, etc. Predictive applied while performing predictive analytics. Some of analytics can streamline the process of customer acquisi- them are briefly discussed below. tion by predicting the future risk behavior of a customer using application level data.[4] Predictive analytics in the form of credit scores have reduced the amount of time it 5.1.1 model takes for loan approvals, especially in the mortgage mar- ket where lending decisions are now made in a matter of The linear regression model analyzes the relationship be- hours rather than days or even weeks. Proper predictive tween the response or dependent variable and a set of in- analytics can lead to proper pricing decisions, which can dependent or predictor variables. This relationship is ex- help mitigate future risk of default. pressed as an equation that predicts the response variable 5.1 Regression techniques 5 as a linear function of the parameters. These parameters tic model, which is basically a method which transforms are adjusted so that a measure of fit is optimized. Much information about the binary dependent variable into an of the effort in model fitting is focused on minimizing the unbounded continuous variable and estimates a regular size of the residual, as well as ensuring that it is randomly multivariate model (See Allison’s for distributed with respect to the model predictions. more information on the theory of Logistic Regression). The goal of regression is to select the parameters of the The Wald and likelihood-ratio test are used to test the model so as to minimize the sum of the squared residu- statistical significance of each coefficient b in the model als. This is referred to as (OLS) (analogous to the t tests used in OLS regression; see estimation and results in best linear unbiased estimates above). A test assessing the goodness-of-fit of a classi- (BLUE) of the parameters if and only if the Gauss- fication model is the “percentage correctly predicted”. Markov assumptions are satisfied. Once the model has been estimated we would be inter- 5.1.4 Multinomial logistic regression ested to know if the predictor variables belong in the model – i.e. is the estimate of each variable’s contribution An extension of the binary logit model to cases where reliable? To do this we can check the statistical signifi- the dependent variable has more than 2 categories is the cance of the model’s coefficients which can be measured multinomial logit model. In such cases collapsing the data using the t-statistic. This amounts to testing whether the into two categories might not make good sense or may coefficient is significantly different from zero. How well lead to loss in the richness of the data. The multinomial the model predicts the dependent variable based on the logit model is the appropriate technique in these cases, value of the independent variables can be assessed by us- especially when the dependent variable categories are not ing the R² statistic. It measures predictive power of the ordered (for examples colors like red, blue, green). Some model i.e. the proportion of the total variation in the de- authors have extended multinomial regression to include pendent variable that is “explained” (accounted for) by feature selection/importance methods such as Random variation in the independent variables. multinomial logit.

5.1.2 Discrete choice models 5.1.5 regression

Multivariate regression (above) is generally used when Probit models offer an alternative to logistic regres- the response variable is continuous and has an unbounded sion for modeling categorical dependent variables. Even range. Often the response variable may not be continuous though the outcomes tend to be similar, the underlying but rather discrete. While mathematically it is feasible to distributions are different. Probit models are popular in apply multivariate regression to discrete ordered depen- social sciences like economics. dent variables, some of the assumptions behind the theory A good way to understand the key difference between of multivariate linear regression no longer hold, and there probit and logit models is to assume that there is a latent are other techniques such as discrete choice models which variable z. are better suited for this type of analysis. If the depen- dent variable is discrete, some of those superior methods We do not observe z but instead observe y which takes the are logistic regression, multinomial logit and probit mod- value 0 or 1. In the logit model we assume that y follows a els. Logistic regression and probit models are used when . In the we assume that y the dependent variable is binary. follows a standard . Note that in social sciences (e.g. economics), probit is often used to model situations where the observed variable y is continuous but 5.1.3 Logistic regression takes values between 0 and 1.

For more details on this topic, see logistic regression. 5.1.6 Logit versus probit

In a classification setting, assigning outcome The Probit model has been around longer than the logit to observations can be achieved through the use of a logis- model. They behave similarly, except that the logistic dis- 6 5 ANALYTICAL TECHNIQUES tribution tends to be slightly flatter tailed. One of the rea- idation. The identification stage involves identifying if sons the logit model was formulated was that the probit the series is stationary or not and the presence of sea- model was computationally difficult due to the require- sonality by examining plots of the series, autocorrelation ment of numerically calculating integrals. Modern com- and partial autocorrelation functions. In the estimation puting however has made this computation fairly simple. stage, models are estimated using non-linear time series The coefficients obtained from the logit and probit model or maximum likelihood estimation procedures. Finally are fairly close. However, the is easier to in- the validation stage involves diagnostic checking such as terpret in the logit model. plotting the residuals to detect outliers and evidence of Practical reasons for choosing the probit model over the model fit. logistic model would be: In recent years time series models have become more sophisticated and attempt to model condi- • There is a strong belief that the underlying distribu- tional heteroskedasticity with models such as ARCH tion is normal (autoregressive conditional heteroskedasticity) and GARCH (generalized autoregressive conditional het- • The actual event is not a binary outcome (e.g., eroskedasticity) models frequently used for financial bankruptcy status) but a proportion (e.g., proportion time series. In addition time series models are also of population at different debt levels). used to understand inter-relationships among economic variables represented by systems of equations using VAR (vector autoregression) and structural VAR models. 5.1.7 Time series models

Time series models are used for predicting or forecasting 5.1.8 Survival or duration analysis the future behavior of variables. These models account for the fact that data points taken over time may have an internal structure (such as autocorrelation, trend or sea- is another name for time to event anal- sonal variation) that should be accounted for. As a result ysis. These techniques were primarily developed in the standard regression techniques cannot be applied to time medical and biological sciences, but they are also widely series data and methodology has been developed to de- used in the social sciences like economics, as well as in compose the trend, seasonal and cyclical component of engineering (reliability and failure time analysis). the series. Modeling the dynamic path of a variable can Censoring and non-normality, which are characteristic improve forecasts since the predictable component of the of survival data, generate difficulty when trying to ana- series can be projected into the future. lyze the data using conventional statistical models such Time series models estimate difference equations con- as multiple linear regression. The normal distribution, being a symmetric distribution, takes positive as well as taining stochastic components. Two commonly used forms of these models are autoregressive models (AR) negative values, but duration by its very nature cannot be negative and therefore normality cannot be assumed and moving average (MA) models. The Box-Jenkins methodology (1976) developed by George Box and G.M. when dealing with duration/survival data. Hence the nor- mality assumption of regression models is violated. Jenkins combines the AR and MA models to produce the ARMA (autoregressive moving average) model which The assumption is that if the data were not censored it is the cornerstone of stationary time series analysis. would be representative of the population of interest. In ARIMA(autoregressive integrated moving average mod- survival analysis, censored observations arise whenever els) on the other hand are used to describe non-stationary the dependent variable of interest represents the time to time series. Box and Jenkins suggest differencing a non a terminal event, and the duration of the study is limited stationary time series to obtain a stationary series to in time. which an ARMA model can be applied. Non stationary An important concept in survival analysis is the hazard time series have a pronounced trend and do not have a rate, defined as the that the event will occur constant long-run mean or variance. at time t conditional on surviving until time t. Another Box and Jenkins proposed a three stage methodology concept related to the hazard rate is the survival function which includes: model identification, estimation and val- which can be defined as the probability of surviving to 5.2 Machine learning techniques 7 time t. Decision trees are formed by a collection of rules based Most models try to model the hazard rate by choosing on variables in the modeling data set: the underlying distribution depending on the shape of • Rules based on variables’ values are selected to get the hazard function. A distribution whose hazard func- the best split to differentiate observations based on tion slopes upward is said to have positive duration de- the dependent variable pendence, a decreasing hazard shows negative duration dependence whereas constant hazard is a process with • Once a rule is selected and splits a node into two, the no memory usually characterized by the exponential dis- same process is applied to each “child” node (i.e. it tribution. Some of the distributional choices in survival is a recursive procedure) models are: F, gamma, Weibull, log normal, inverse nor- • mal, exponential etc. All these distributions are for a non- Splitting stops when CART detects no further gain negative random variable. can be made, or some pre-set stopping rules are met. (Alternatively, the data are split as much as possible Duration models can be parametric, non-parametric or and then the tree is later pruned.) semi-parametric. Some of the models commonly used are Kaplan-Meier and Cox proportional hazard model Each branch of the tree ends in a terminal node. Each (non parametric). observation falls into one and exactly one terminal node, and each terminal node is uniquely defined by a set of rules. 5.1.9 Classification and regression trees A very popular method for predictive analytics is Leo Breiman’s Random forests or derived versions of this Main article: technique like Random multinomial logit.

Hierarchical Optimal Discriminant Analysis (HODA), 5.1.10 Multivariate adaptive regression splines (also called classification tree analysis) is a generaliza- tion of Optimal discriminant analysis that may be used Multivariate adaptive regression splines (MARS) is a to identify the statistical model that has maximum accu- non-parametric technique that builds flexible models by racy for predicting the value of a categorical dependent fitting piecewise linear regressions. variable for a dataset consisting of categorical and contin- uous variables. The output of HODA is a non-orthogonal An important concept associated with regression splines tree that combines categorical variables and cut points is that of a knot. Knot is where one local regression model for continuous variables that yields maximum predictive gives way to another and thus is the point of intersection accuracy, an assessment of the exact Type I error rate, between two splines. and an evaluation of potential cross-generalizability of In multivariate and adaptive regression splines, basis the statistical model. Hierarchical Optimal Discriminant functions are the tool used for generalizing the search for analysis may be thought of as a generalization of Fisher’s knots. Basis functions are a set of functions used to repre- linear discriminant analysis. Optimal discriminant analy- sent the information contained in one or more variables. sis is an alternative to ANOVA (analysis of variance) and Multivariate and Adaptive Regression Splines model al- , which attempt to express one depen- most always creates the basis functions in pairs. dent variable as a linear combination of other features or measurements. However, ANOVA and regression analy- Multivariate and adaptive regression spline approach de- sis give a dependent variable that is a numerical variable, liberately overfits the model and then prunes to get to the while hierarchical optimal discriminant analysis gives a optimal model. The algorithm is computationally very dependent variable that is a class variable. intensive and in practice we are required to specify an upper limit on the number of basis functions. Classification and regression trees (CART) is a non- parametric decision tree learning technique that produces either classification or regression trees, depending on 5.2 Machine learning techniques whether the dependent variable is categorical or numeric, respectively. Machine learning, a branch of artificial intelligence, was 8 5 ANALYTICAL TECHNIQUES originally employed to develop techniques to enable com- adjust the weights of the network. The backpropogation puters to learn. Today, since it includes a number of ad- employs gradient fall to minimize the squared error be- vanced statistical methods for regression and classifica- tween the network output values and desired values for tion, it finds application in a wide variety of fields includ- those outputs. The weights adjusted by an iterative pro- ing medical diagnostics, credit card fraud detection, face cess of repetitive present of attributes. Small changes in and and analysis of the stock mar- the weight to get the desired values are done by the pro- ket. In certain applications it is sufficient to directly pre- cess called training the net and is done by the training set dict the dependent variable without focusing on the un- (learning rule). derlying relationships between variables. In other cases, the underlying relationships can be very complex and the mathematical form of the dependencies unknown. For such cases, machine learning techniques emulate human 5.2.3 Radial basis functions cognition and learn from training examples to predict fu- ture events. A radial basis function (RBF) is a function which has built A brief discussion of some of these methods used com- into it a distance criterion with respect to a center. Such monly for predictive analytics is provided below. A de- functions can be used very efficiently for interpolation and tailed study of machine learning can be found in Mitchell for smoothing of data. Radial basis functions have been (1997). applied in the area of neural networks where they are used as a replacement for the sigmoidal transfer function. Such networks have 3 layers, the input layer, the hidden layer 5.2.1 Neural networks with the RBF non-linearity and a linear output layer. The most popular choice for the non-linearity is the Gaussian. Neural networks are nonlinear sophisticated model- RBF networks have the advantage of not being locked ing techniques that are able to model complex func- into local minima as do the feed-forward networks such tions. They can be applied to problems of prediction, as the multilayer perceptron. classification or control in a wide spectrum of fields such as finance, cognitive psychology/neuroscience, medicine, engineering, and physics. 5.2.4 Support vector machines Neural networks are used when the exact nature of the re- lationship between inputs and output is not known. A key Support Vector Machines (SVM) are used to detect and feature of neural networks is that they learn the relation- exploit complex patterns in data by clustering, classifying ship between inputs and output through training. There and ranking the data. They are learning machines that are are three types of training in neural networks used by dif- used to perform binary classifications and regression es- ferent networks, supervised and unsupervised training, timations. They commonly use kernel based methods to reinforcement learning, with supervised being the most apply linear classification techniques to non-linear classi- common one. fication problems. There are a number of types of SVM Some examples of neural network training techniques such as linear, polynomial, sigmoid etc. are backpropagation, quick propagation, conjugate gra- dient descent, projection operator, Delta-Bar-Delta etc. Some unsupervised network architectures are multilayer perceptrons, Kohonen networks, Hopfield networks, etc. 5.2.5 Naïve Bayes

Naïve Bayes based on Bayes conditional probability 5.2.2 Multilayer Perceptron (MLP) rule is used for performing classification tasks. Naïve Bayes assumes the predictors are statistically independent The Multilayer Perceptron (MLP) consists of an input which makes it an effective classification tool that is easy and an output layer with one or more hidden layers of to interpret. It is best employed when faced with the prob- nonlinearly-activating nodes or sigmoid nodes. This is lem of ‘curse of dimensionality’ i.e. when the number of determined by the weight vector and it is necessary to predictors is very high. 9

5.2.6 k-nearest neighbours are no longer restricted to IT specialists. As more orga- nizations adopt predictive analytics into decision-making The nearest neighbour algorithm (KNN) belongs to the processes and integrate it into their operations, they are class of statistical methods. The creating a shift in the market toward business users as method does not impose a priori any assumptions about the primary consumers of the information. Business the distribution from which the modeling sample is users want tools they can use on their own. Vendors drawn. It involves a training set with both positive and are responding by creating new software that removes the negative values. A new sample is classified by calculat- mathematical complexity, provides user-friendly graphic ing the distance to the nearest neighbouring training case. interfaces and/or builds in short cuts that can, for ex- The sign of that point will determine the classification of ample, recognize the kind of data available and suggest the sample. In the k-nearest neighbour classifier, the k an appropriate predictive model.[20] Predictive analytics nearest points are considered and the sign of the major- tools have become sophisticated enough to adequately ity is used to classify the sample. The performance of present and dissect data problems, so that any data- the kNN algorithm is influenced by three main factors: savvy information worker can utilize them to analyze (1) the distance measure used to locate the nearest neigh- data and retrieve meaningful, useful results.[2] For exam- bours; (2) the decision rule used to derive a classifica- ple, modern tools present findings using simple charts, tion from the k-nearest neighbours; and (3) the number graphs, and scores that indicate the likelihood of possi- of neighbours used to classify the new sample. It can be ble outcomes.[21] proved that, unlike other methods, this method is uni- There are numerous tools available in the marketplace versally asymptotically convergent, i.e.: as the size of the that help with the execution of predictive analytics. These training set increases, if the observations are independent range from those that need very little user sophistication and identically distributed (i.i.d.), regardless of the dis- to those that are designed for the expert practitioner. The tribution from which the sample is drawn, the predicted difference between these tools is often in the level of cus- class will converge to the class assignment that minimizes tomization and heavy data lifting allowed. misclassification error. See Devroy et al. Notable open source predictive analytic tools include:

5.2.7 Geospatial predictive modeling • scikit-learn

Conceptually, geospatial predictive modeling is rooted in • KNIME the principle that the occurrences of events being mod- eled are limited in distribution. Occurrences of events • OpenNN are neither uniform nor random in distribution – there are spatial environment factors (infrastructure, sociocultural, • Orange topographic, etc.) that constrain and influence where the • R locations of events occur. Geospatial predictive modeling attempts to describe those constraints and influences by • RapidMiner spatially correlating occurrences of historical geospatial locations with environmental factors that represent those • constraints and influences. Geospatial predictive model- • ing is a process for analyzing events through a geographic GNU Octave filter in order to make statements of likelihood for event • Apache Mahout occurrence or emergence.

Notable commercial predictive analytic tools include: 6 Tools • Alpine Data Labs

Historically, using predictive analytics tools—as well as • BIRT Analytics understanding the results they delivered—required ad- vanced skills. However, modern predictive analytics tools • Angoss KnowledgeSTUDIO 10 9 REFERENCES

• IBM SPSS and IBM SPSS Modeler the director of the Institute for Quantitative Social Sci- ence. [22] People are influenced by their environment in • KXEN Modeler innumerable ways. Trying to understand what people will do next assumes that all the influential variables can be • Mathematica known and measured accurately. “People’s environments • MATLAB change even more quickly than they themselves do. Ev- erything from the weather to their relationship with their • Minitab mother can change the way people think and act. All of those variables are unpredictable. How they will impact • (ODM) a person is even less predictable. If put in the exact same • Pervasive situation tomorrow, they may make a completely differ- ent decision. This means that a statistical prediction is • Predixion Software only valid in sterile laboratory conditions, which suddenly isn't as useful as it seemed before.” [23] • Revolution Analytics • SAP 8 See also • SAS and SAS Enterprise Miner • Criminal Reduction Utilising Statistical History • STATA • Data mining • STATISTICA • • TIBCO • Odds algorithm • FICO • Pattern recognition The most popular commercial predictive analytics soft- • ware packages according to the Rexer Analytics Sur- • vey for 2013 are IBM SPSS Modeler, SAS Enterprise Predictive modeling Miner, and Dell Statistica decisions.

6.1 PMML 9 References In an attempt to provide a standard language for express- ing predictive models, the Predictive Model Markup Lan- [1] Nyce, Charles (2007), Predictive Analytics White Paper, guage (PMML) has been proposed. Such an XML-based American Institute for Chartered Property Casualty Un- derwriters/Insurance Institute of America, p. 1 language provides a way for the different tools to de- fine predictive models and to share these between PMML [2] Eckerson, Wayne (May 10, 2007), Extending the Value of compliant applications. PMML 4.0 was released in June, Your Data Warehousing Investment, The 2009. Institute [3] Coker, Frank (2014). Pulse: Understanding the Vital Signs of Your Business (1st ed.). Bellevue, WA: Ambi- 7 Criticism ent Light Publishing. pp. 30,39,42,more. ISBN 978-0- 9893086-0-1.

There are plenty of skeptics when it comes to comput- [4] Conz, Nathan (September 2, 2008), “Insurers Shift to ers and algorithms abilities to predict the future, includ- Customer-focused Predictive Analytics Technologies”, ing Gary King, a professor from Harvard University and Insurance & Technology 11

[5] Fletcher, Heather (March 2, 2011), “The 7 Best Uses for [19] Ben-Gal I., Shavitt Y., Weinsberg E., Weinsberg U. Predictive Analytics in Multichannel Marketing”, Target (2014). Peer-to-peer information retrieval using shared- Marketing content clustering. Knowl Inf Syst DOI 10.1007/s10115- 013-0619-9. [6] Korn, Sue (April 21, 2011), “The Opportunity for Predic- tive Analytics in Finance”, HPC Wire [20] Halper, Fern (November 1, 2011), “The Top 5 Trends in Predictive Analytics”, Information Management [7] Barkin, Eric (May 2011), “CRM + Predictive Analytics: Why It All Adds Up”, Destination CRM [21] MacLennan, Jamie (May 1, 2012), 5 Myths about Predic- tive Analytics, The Data Warehouse Institute [8] Das, Krantik; Vidyashankar, G.S. (July 1, 2006), “Competitive Advantage in Retail Through Analytics: [22] Temple-Raston, Dina (Oct 8, 2012), Predicting The Fu- Developing Insights, Creating Value”, Information Man- ture: Fantasy Or A Good Algorithm?, NPR agement [23] Alverson, Cameron (Sep 2012), Polling and Statistical [9] McDonald, Michèle (September 2, 2010), “New Technol- Models Can't Predict the Future, Cameron Alverson ogy Taps 'Predictive Analytics’ to Target Travel Recom- mendations”, Travel Market Report 10 Further reading [10] Stevenson, Erin (December 16, 2011), “Tech Beat: Can you pronounce health care predictive analytics? extquot- edbl, Times-Standard • Agresti, Alan (2002). Categorical . Hoboken: John Wiley and Sons. ISBN 0-471- [11] McKay, Lauren (August 2009), “The New Prescription 36093-7. for Pharma”, Destination CRM • Coggeshall, Stephen, Davies, John, Jones, Roger., [12] Finlay, Steven (2014). Predictive Analytics, Data Mining and Schutzer, Daniel, “Intelligent Security Sys- and Big Data. Myths, Misconceptions and Methods (1st tems,” in Freedman, Roy S., Flein, Robert A., and ed.). Basingstoke: Palgrave Macmillan. p. 237. ISBN Lederman, Jess, Editors (1995). Artificial Intelli- 1137379278. gence in the Capital Markets. Chicago: Irwin. ISBN [13] Schiff, Mike (March 6, 2012), BI Experts: Why Predictive 1-55738-811-3. Analytics Will Continue to Grow, The Data Warehouse In- stitute • L. Devroye, L. Györfi, G. Lugosi (1996). A Prob- abilistic Theory of Pattern Recognition. New York: [14] Nigrini, Mark (June 2011). “Forensic Analytics: Methods Springer-Verlag. and Techniques for Forensic Accounting Investigations”. Hoboken, NJ: John Wiley & Sons Inc. ISBN 978-0-470- • Enders, Walter (2004). Applied Time Series Econo- 89046-2. metrics. Hoboken: John Wiley and Sons. ISBN 0- 521-83919-X. [15] Dhar, Vasant (April 2011). “Prediction in Financial Mar- kets: The Case for Small Disjuncts”. ACM Transactions • Greene, William (2012). Econometric Analysis, on Intelligent Systems and Technologies 2 (3). 7th Ed. London: Prentice Hall. ISBN 978-0-13- 139538-1. [16] Dhar, Vasant; Chou, Dashin and Provost Foster (October 2000). “Discovering Interesting Patterns in Investment • Guidère, Mathieu; Howard N, Sh. Argamon Decision Making with GLOWER – A Genetic Learning (2009). Rich Language Analysis for Counterterrror- Algorithm Overlaid With Entropy Reduction”. Data Min- ing and Knowledge Discovery 4 (4). ism. Berlin, London, New York: Springer-Verlag. ISBN 978-3-642-01140-5. [17] https://acc.dau.mil/CommunityBrowser.aspx?id= • 126070 Mitchell, Tom (1997). Machine Learning. New York: McGraw-Hill. ISBN 0-07-042807-7. [18] Ben-Gal I. Dana A., Shkolnik N. and Singer (2014). Efficient Construction of Decision Trees by the Dual Infor- • Siegel, Eric (2013). Predictive Analytics: The Power mation Distance Method. Quality Technology & Quanti- to Predict Who Will Click, Buy, Lie, or Die. John tative Management (QTQM), 11( 1), 133-147. Wiley. ISBN 978-1-1183-5685-2. 12 10 FURTHER READING

• Tukey, John (1977). Exploratory Data Analysis. New York: Addison-Wesley. ISBN 0-201-07616- 0.

• Finlay, Steven (2014). Predictive Analytics, Data Mining and Big Data. Myths, Misconceptions and Methods. Basingstoke: Palgrave Macmillan. ISBN 978-1-137-37927-6.

• Coker, Frank (2014). Pulse: Understanding the Vi- tal Signs of Your Business. Bellevue, WA: Ambient Light Publishing. ISBN 978-0-9893086-0-1. 13

11 Text and image sources, contributors, and licenses

11.1 Text

• Predictive analytics Source: http://en.wikipedia.org/wiki/Predictive_analytics?oldid=627141326 Contributors: The Anome, Edward, Michael Hardy, Kku, Karada, Ronz, Avani, Giftlite, SWAdair, Thorwald, Bender235, Rubicon, Calair, Rajah, Andrewpmk, Stephen Turner, Dominic, Oleg Alexandrov, OwenX, HughJorgan, RussBot, Leighblackall, Aeusoes1, Gloumouth1, DeadEyeArrow, EAderhold, Zzuuzz, Talyian, Allens, Zvika, Yvwv, SmackBot, CommodiCast, Mcld, Gilliam, Eudaemonic3, S2rikanth, Onorem, JonHarder, Krexer, Lpgeffen, Doug Bell, Kuru, IronGargoyle, JHunterJ, Ralf Klinkenberg, CmdrObot, Van helsing, Requestion, Pgr94, Dancter, Talgalili, Sci- entio, Marek69, JustAGal, Batra, Mr. Darcy, AntiVandalBot, MER-C, Fbooth, VoABot II, Baccyak4H, Bellemichelle, Sweet2, Gregheth, Apdevries, Sudheervaishnav, Ekotkie, Dontdoit, Jfroelich, Trusilver, Dvdpwiki, Ramkumar.krishnan, Atama, Bonadea, BernardZ, Cyricx, GuyRo, Deanabb, Dherman652, Arpabr, Selain03, Hherbert, Cuttysc, Vikx1, Ralftgehrig, Rpm698, Drakedirect, Chrisguyot, Melcombe, Maralia, Into The Fray, Kai-Hendrik, Ahyeek, Howie Goodell, Sterdeus, SpikeToronto, Stephen Milborrow, Jlamro, Isthisthingworking, SchreiberBike, Bateni, Cookiehead, Qwfp, Angoss, Sunsetsky, Vianello, MystBot, Vaheterdu, BizAnalyst, Addbot, MrOllie, Download, Yobot, SOMart, AnomieBOT, IRP, Nosperantos, Citation bot, BlaineKohl, Phy31277, CorporateM, GESICC, FrescoBot, Boxplot, I dream of horses, Triplestop, Dmitry St, SpaceFlight89, Jackverr, Σ, Peter.borissow, Ethansdad, Cambridgeblue1971, Pamparam, Kmettler, Glenn Maddox, Vrenator, Crysb, DARTH SIDIOUS 2, WikitanvirBot, Sugarfoot1001, Jssgator, Chire, MainFrame, Synecticsgroup, Clue- Bot NG, Thirdharringtonskier, Stefanomazzalai, Ricekido, Widr, Luke145, Mikeono, Helpful Pixie Bot, WhartonCAI, JonasJSchreiber, Wbm1058, BG19bot, Rafaelgmonteiro, Lisasolomonsalford, TLAN38, Lynnlangit, Flaticida, MC Wapiti, HHinman, BattyBot, Raspabill, Jeremy Kolb, TwoTwoHello, Mkhambaty, Cmdima, HeatherMKCampbell, Tentinator, Sgolestanian, Gbtodd29, Brishtikonna, Thisaccoun- tisbs, Mitchki.nj, Jvn mht, JaconaFrere, Pablodim91, Cbuyle, Monkbot, AmandaJohnson2014, JSHorwitz, Thomas Speidel, HappyVDZ, Wikiperson99, Justincahoon, Femiolajiga, Stevenfinlay, Rlm1188, Bildn, Frankecoker and Anonymous: 206

11.2 Images

• File:Question_book-new.svg Source: http://upload.wikimedia.org/wikipedia/en/9/99/Question_book-new.svg License: ? Contributors: Created from scratch in Adobe Illustrator. Based on Image:Question book.png created by User:Equazcion Original artist: Tkgd2007 • File:Text_document_with_red_question_mark.svg Source: http://upload.wikimedia.org/wikipedia/commons/a/a4/Text_document_ with_red_question_mark.svg License: Public domain Contributors: Created by bdesham with Inkscape; based upon Text-x-generic.svg from the Tango project. Original artist: Benjamin D. Esham (bdesham)

11.3 Content license

• Creative Commons Attribution-Share Alike 3.0