A META-ANALYTIC REVIEW OF THE EFFECTIVENESS OF PERSONNEL SELECTION PROCEDURES AND TRAINING INTERVENTIONS IN SALES OCCUPATIONS
by
SEONAID FARRELL
BA. (Honours), Queen's University, 1994
A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF ARTS
in
THE FACULTY OF GRADUATE STUDIES
(Department of Psychology)
We accept this thesis as conforming to the required standard
THE UNIVERSITY OF BRITISH COLUMBIA
August 1999
© Seonaid Farrell, 1999 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission.
Department of
The University of British Columbia Vancouver, Canada
Date
DE-6 (2/88) 11 Abstract
Applied research on the effectiveness of personnel selection procedures and training interventions in sales occupations was examined by meta-analytic techniques. The literature review included 170 predictor- criterion effect sizes of selection procedures, and 12 effect sizes of training interventions. On average, composite-domain selection assessments predicted salesperson performance (multiple R = .59, p < .05), while the effects of single domain selection assessments and training interventions (r = .27, p > .05; r= .16, p > .05, respectively) were not statistically significant. Among the personnel selection procedures, subjective ratings of performance were predicted by personality assessments (r = .20, p < .05), biographical information inventories (r = .20, p < .05), and cognitive ability (r = .32, p < .05). While objective measures of performance were predicted by special purpose sales assessments (r = .49, p < .05).
An analysis of the utility of selection procedures and training interventions based on the average effect sizes found in the present study revealed substantial dollar value gains to the company from the use of personnel selection procedures and training interventions in sales occupations. iii
Table Of Contents
Abstract ii
List of Tables iv
Acknowledgements v
Introduction 1
The Sales Occupation 1 Approaches to the Improvement of Sales Performance 3 Personnel Selection 3 Training 5 Purposes of the Present Study 6 Measurement of Sales Performance 6 Objective measurement of performance 6 Subjective measurement of perfonnance 7 Comparison of objective and subjective measures of performance 8 Effectiveness of Selection Procedures and Training Interventions for Salespeople 8 Effectiveness of predictors of salesperson performance 8 Effectiveness of training interventions for salespeople 11
Method 13
Literature Search 13 Criteria for Inclusion of Results 13 Coding 14 Meta-Analytic Procedures 15
Results and Discussion 20
Sample Description 20 Meta-Analytic Results and Discussion 21 Analysis of the Utility of Selection Procedures and Training Interventions 23
Summary 27
References 30 iv List of Tables
Table 1 Correlation Coefficients by Single Domain Predictor, Composite Domain Predictor, or Training Intervention for the Combined Criterion 41
Table 2 Validity Coefficients by Content of Single Domain Selection Assessment for the Combined Criterion 42
Table 3 Validity Coefficients by Content of Single Domain Selection Assessment for the Subjective Criterion 43
Table 4 Validity Coefficients by Content of Single Domain Selection Assessment for the Objective Criterion 44
Table 5 Relative Utility Estimates of Valid Selection Procedures and Effective Training Interventions 45 V
Acknowledgements
I am grateful to many people who supported the achievement of this research. I am indebted to my committee members for their encouragement and valuable insights. In particular Dr. Ralph Hakstian, who developed my enjoyment of research by challenging and inspiring me. I appreciate my close friends who gave their support and encouragement generously. Finally, I am grateful for a good family who support my endeavors unselfishly, and continually demonstrate the value of determination and personal achievement. Personnel Selection and Training in Sales 1
A Meta-analytic Review of the Effectiveness of Personnel
Selection Procedures and Training Interventions in Sales Occupations
The Sales Occupation
The sales occupation is an important area of study because of the strong influence it exerts on the economy, the wide range in job performance of salespeople, and the potential gains that can be achieved by sales management techniques. There were 14,633,000 salespeople in the United States in 1996, making up
11% off all jobs (Bureau of Labor Statistics, 1998). The number of sales and marketing jobs is expected to rise by 15.5% by 2006; a rate of increase which is greater than the average of all occupations (14%).
The total compensation of salespeople that includes salary, commission, and incentives is substantial. In the United States, the average compensation of a sales representative was $63,200 in 1998
(Pricewaterhouse Coopers, 1998). The predominance of salespeople in the workforce, the expected rate of growth of sales jobs, and their level of compensation suggest that the job performance of salespeople is an important influence on the economy.
The great variability in the job performance among salespeople makes it a compelling area of study. Hunter, Schmidt, and Judiesch (1990) examined the standard deviation of employee output as a percentage of mean output for a variety of occupations and found that the variability was greatest for salespeople. These authors found that the average standard deviation of output, as a percentage of mean output, was 120% for life insurance, and 48% for non-insurance sales, compared with 48% for high complexity non-sales jobs, 32% for medium complexity non-sales jobs, and 19% for low complexity non- sales jobs. The increased variability in output for sales jobs can be expected to result in potentially greater gains in utility by human resource management activities, because the greater the variability (in dollar value to the organization) the greater the average gain in dollars from successful interventions. Hunter et al. (1990) suggested that the large variability in sales output may be due to the multiplicative effect of specific traits and abilities required of salespeople. These authors suggested that the average percentage increase in output from improved selection is approximately 2.5 times greater in sales jobs than in low complexity non-sales jobs. Personnel Selection and Training in Sales 2
Another unique characteristic of the sales occupation is the unusual features that are required to perform the job. Salespeople must work autonomously (Churchill, Ford, & Walker, 1997), relying on their own initiative and determination. They must also endure a high amount of rejection in proportion to successes, requiring persistence and achievement striving. Salespeople must solve the different needs of many clients in a frequently changing marketplace, requiring adaptability and problem solving skills.
A meta-analysis of the predictors of sales performance from a marketing perspective (Churchill,
Ford, Hartley, & Walker, 1985) delineated six variables to account for individual differences in job performance. The six variables include: (1) aptitude, which are abilities and personal traits related to the performance of the job; (2) personal characteristics, which are physical traits, and family, education, and work background experiences; (3) skill level, which refers to the extent of learned proficiencies at performing job activities; (4) role perceptions, which are the employee's perceptions of job demands and expectations from others; (5) motivation, which is the desire to expend effort on specific job activities; and
(6) organizational and environmental factors, which include sales potential of territory, salesperson's autonomy, and the company's competitive strength (Churchill et al., 1997, pp. 327-335). Churchill et al.
(p. 367) suggested that among the variety of human resource management activities, recruitment and selection influence aptitude and personal characteristics, whereas training and supervision influence skill levels and role perceptions.
There are a variety of jobs included in the sales occupation. The various sales jobs can be classified as involving retail or industrial selling (Churchill et al., 1997). Retail selling refers to selling goods and services directly to the consumer of these products. The products are intended for the consumer's personal use. Industrial selling pertains to the sale of goods and services to wholesale customers. Wholesale customers can be: (1) resellers, who sell the product to a retailer, (2) business users, who use the product to produce another product, or (3) institutions, who use the product in their business operation. There are similarities and differences in the selling function of retail and industrial sales jobs.
While retail sales requires many of the same skills and personal characteristics as industrial sales, industrial sales is often viewed as unique because the products are often more expensive and technically complex. Personnel Selection and Training in Sales 3
Approaches to the Improvement of Sales Performance
The importance of sales to the economy, great variability in job performance of sales people, and unique characteristics of the occupation are compelling reasons for the study of salespeople. One aspect of the study of this occupation is the factors that influence improvement in salesperson performance, two of which are personnel selection and training interventions.
Personnel Selection
Personnel selection involves the use of systematic, rational, valid assessment methods to select the best candidates for employment. The objective is to predict future job performance by assessing the applicant's characteristics, capacity to function, and interest in the job. The use of assessment procedures for selection assumes that employees who are selected by valid procedures will be more productive than employees selected by inferior procedures. This is especially true in sales occupations if we can assume that individual differences account for much of the large variability in job performance.
The validity of assessment procedures for selection serves the needs of the salesperson as well. A salesperson who is selected to work at a company should have the requisite abilities to perform the job, or s/he will suffer frustration and failure. Alternatively, the applicant who is not hired for the sales job, but who does indeed have potential, is unfairly treated if this potential is not recognized and the decision is not to hire.
There are many types of selection procedures in use for the selection of salespeople. In the present study their content is categorized into six distinct groups that include:
1. Job skill/knowledge assessments, which measure an applicant's knowledge about sales-related skills and information, such as personal selling, planning, time management, and market potential. In most cases, job skill/knowledge is gained as the result of specific work experience in sales occupations.
2. Personality inventories, which measure enduring personal dispositions that reflect an individual's consistent reactions to situations. The personality dimensions most often studied among salespeople include dominance, empathy, sociability, responsibility, self-esteem, need for achievement, and need for power. Personnel Selection and Training in Sales 4
3. Biographical information, which is demographic, educational, and occupational information about an applicant that is obtained by asking questions related to the applicant's family, work, and lifestyle background experiences. Biographical information most often obtained in the selection of salespeople includes level of education, work experience, activities and preferences, and marital status.
4. Cognitive ability tests, which measure general mental ability and specific mental processes and abilities related to sales ability, such as verbal comprehension, numerical ability, and visual speed and accuracy.
5. Job specific aptitude assessments, which measure personal characteristics and abilities thought to be related to the performance of specific sales tasks, such as ease of interpersonal interactions and business judgment. Job specific aptitude differs from job skill/knowledge in that aptitude is not necessarily gained by specific sales experience and it does not measure concrete knowledge about sales.
6. Special purpose sales assessments are paper and pencil tests that measure aptitudes and personal qualities related specifically to success in sales occupations. They differ from the previously mentioned selection procedures in that special purpose tests are developed only for the prediction of sales performance dimensions, rather than general traits.
The term "selection procedure" (sometimes "assessment procedure" in what follows) will be used to refer to any of the systematic personnel assessment-selection methods identified in this categorization.
More specifically, "single domain predictor" will be used to refer to any one of the selection procedures when used individually to predict sales performance, and "composite domain predictor" will be used to refer to any optimally- or unit-weighted combination of two or more of the selection procedures.
Personnel selection procedures are widely used in organizations. In a review of the use of selection procedures for sales occupations among a sample of 121 firms (Dubinsky & Barry, 1982), 90% of the firms reported use of interviews, 66% reported use of application blanks and reference checks, 33% of larger firms reported use of psychological tests, and 22% of smaller firms reported use of psychological tests.
Historically, the dominant belief has been that personnel selection validity is specific to particular Personnel Selection and Training in Sales 5 situations and individual jobs. A review of the raw validation results across many occupations found high variability in results among studies even when jobs and tests appeared to be similar (Ghiselli, 1966, 1973).
Findings similar to this supported the theory of situational specificity which viewed the generalization of validity results across similar jobs and personnel selection procedures as essentially impossible (Albright,
Glennon, & Smith, 1963, p. 18; Ghiselli, 1966, p. 28). The theory of situational specificity remained dominant until evidence mounted to suggest that the variability in validity study results within job-test combinations was due to statistical and measurement artifacts, rather than real differences between jobs
(Schmidt, Bemer, & Hunter, 1973; Schmidt, & Hunter, 1977; Schmidt, Hunter, Pearlman, & Shane, 1979;
Schmidt, Hunter, & Urry, 1976). The variability among validity results was largely accounted for by error variance such as sampling error caused by small sample sizes (Schmidt, & Hunter, 1998), computational and typographical errors, differences between studies in criterion reliability, differences between studies in amount and kind of criterion contamination and deficiency, and differences between studies in degree of range restriction (Schmidt & Hunter, 1977). Quantitative techniques were developed to combine validity estimates across studies and correct for the statistical and measurement artifacts (Hunter & Schmidt, 1980;
Hunter, Schmidt, & Jackson, 1982).
Training
Training can be defined as a planned program within the organization which endeavors to bring about relatively permanent changes in employee knowledge, skills, attitudes, and behavior (Wexley &
Latham, 1981). The objective of training programs is to improve the performance of salespeople. It is assumed that improved performance of salespeople will lead to increased profits to the company. Training may improve performance by increasing productivity, improving morale, reducing turnover, improving customer relations, and improving management of time and territory (Churchill et al., 1997).
The content of training for salespeople has been categorized into four areas that are described by
Hopkins (1978). These areas include training in (a) product knowledge, (b) company orientation, (c) market/industry orientation, and (d) selling techniques and related topics. It is the fourth area of training, selling techniques and related topics, that is of interest in this thesis. The related topics that are considered Personnel Selection and Training in Sales 6 include: human relations, self-awareness, problem solving and decision making, and motivation/values.
When the term "training intervention" is used in what follows, it should be taken to mean training in selling techniques or a related topic identified in the previous sentence.
In the majority of companies, most salespeople will receive some form of company or product training to enable them to perform their job duties at a minimally acceptable level. This training, however, is specific to the company, and its effectiveness may be irrelevant to training issues in other companies.
Assuming that salespeople have a base of job-, company-, and product-knowledge, training of employees in selling techniques and related topics can be undertaken. Training in this area is interesting because the knowledge and skills that a salesperson learns at this level can be generalized to different sales situations at different companies. The effectiveness of the sales training can be compared across companies because the training is not specific only to the company that sponsors the training program.
Purposes of the Present Study
There were two purposes of the present study. The first was to describe—from a reading of the applied research literature—the extent to which selection procedures and training interventions have improved the performance of salespeople. The second purpose was to compare whether it is selection procedures or training interventions that has contributed more to the improvement of the performance of salespeople.
Measurement of Sales Performance
Salesperson performance is more amenable to measurement than is the performance of other occupations because of the concrete and quantifiable nature of the output. Campbell (1991) defined performance as the behavior that is relevant to the organization's goals. Where behavior is unobservable, it must be inferred from its results, such as sales volume.
Objective measurement of performance
Sales volume is an objective index of salesperson performance. It is the most common outcome measure of salesperson performance (Weitz, 1981) and that most frequently used in academic research
(Comer & Dubinsky, 1985). Objective measures of performance are a function of the individual Personnel Selection and Training in Sales 7 salesperson's skills, effort and external environmental factors. Objective measures are unique in that they require no abstraction or synthesis by the evaluator (Landy & Fair, 1993).
The drawbacks to using an objective measure of salesperson performance such as sales volume include: (1) the extent to which it is influenced by external factors beyond the salesperson's control, such as market fluctuations and type of product sold, and (2) the information about performance that is not measured, such as the strength of the relationship with customers and the commitment to group performance over individual performance. Objective measures may be adjusted for contamination by external factors, by comparing output with a normative reference for the sample, or by reporting sales volume as a proportion of sales quota (Borman, 1994), since sales quota is an index that is established with a consideration of external factors. Borman notes, however, that this practice may be suboptimal because information about particular subgroups of salespeople is lost when large groups of salespeople are used as a normative reference.
In a study examining the stability of salesperson performance measures over time, Levy and
Sharma (1993) found that objective performance measured over six months was significantly related to objective performance over one year; however, objective performance measured over three months was not significantly related. This suggests that reliable measurement of objective performance requires at least a six-month period of assessment.
Subjective measurement of performance
Subjective measures of performance, such as a supervisor's ratings, require an individual to make a judgment about the performance level of another person or of him/herself. The judgment may be based on knowledge of results, such as sales volume, as well as inferences about less quantifiable aspects of the salesperson's performance inferred from his/her behavior, such as communication skills. In this way, subjective measures may include more information about performance and include information that is under the control of the salesperson. Levy and Sharma (1993) found that subjective measures (managerial- and self-ratings) of performance, were upwardly biased and restricted in range. These authors also found that, between the subjective measures examined, managerial ratings were positively related to the length of Personnel Selection and Training in Sales 8 employment of the salespeople, and self-ratings were related to objective performance, whereas managerial ratings were not.
A major drawback of subjective measures is the error that results when the supervisor makes judgments about performance. These judgments, usually recorded on some form of rating instrument, may be influenced by irrelevant factors, such as selective attention to behavior, impression management by the salesperson, inadequate sampling of behaviors on the rating instrument, and biases of the supervisor when rating performance (Bush, Bush, Ortinau, & Hair, 1990).
Comparison of objective and subjective measures ofperformance
Across all occupations there is evidence of only a moderate relationship between subjective and objective measures of performance (Bommer, Johnson, Rich, Podsakoff, & MacKenzie, 1995; Heneman,
1986). In a meta-analysis of salesperson performance, in particular, Bommer et al. (1995) found that the average correlation coefficient between subjective and objective measures of performance was r = .41. It may be that managerial ratings are measuring non-selling aspects of the sales job and should be used in combination with objective ratings to give the most information about job performance. Overall, there is agreement that subjective and objective measures of performance should not be used interchangeably.
Effectiveness of Selection Procedures and Training Interventions for Salespeople
A disparity exists between the amount of information available that describes the effectiveness of selection procedures and that available dealing with training interventions for salespeople. Individual studies and meta-analyses have examined single and composite domain predictors of salesperson performance—predictors that can be applied as selection procedures. On the other hand, few individual studies of the effectiveness of training interventions for salespeople are available.
Effectiveness ofpredictors of salesperson performance
Several meta-analyses have examined the effects of predictors of salesperson performance.
Vinchur, Schippmann, Switzer, and Roth (1998) examined the relationship between a variety of predictors and performance in 129 studies and found biographical information and sales ability (i.e. job skill/knowledge assessments) were relatively strong predictors of subjective ratings (r= .52 and r = .45; Personnel Selection and Training in Sales 9 correlations for this study were corrected for range restriction and criterion unreliability). They also found special purpose sales assessments predicted subjective ratings (r = .45) and objective sales (r = .37).
Among the personality dimensions, they found that Potency (a subdimension of Extraversion) predicted subjective ratings (r = .28) and objective sales (r = .26), and that Achievement (a subdimension of
Conscientiousness) predicted subjective ratings (r = .25) and objective sales (r = .41). Cognitive ability was identified as a unique predictor in that, although it bore a moderate relationship to subjective ratings (r
= .31), it was unrelated to objective sales (r = .04).
A meta-analysis of predictors of sales performance by Ford, Walker, Churchill, and Hartley (1987) from a sales and marketing perspective, examined the relationships between biographical variables
(demographic, background and work experiences, and family status and lifestyle) and psychological variables (aptitudes, mental abilities, personality traits, and learned skills) with sales performance. These authors found that biographical variables bore the strongest relationship to objective performance (r = .46).
Ford et al. (1987) found that among the aptitude dimensions examined, cognitive ability was the best predictor of performance (r = .26) followed by sales aptitude (r = .19). Ford et al. suggested that dominance (r = .16) and self-esteem (r = .14) were the most important personality dimensions. They suggested that the variation among other personality dimensions was due to sampling error.
There is some evidence that a difference in validity appears when comparing higher-level sales representatives with lower-level sales clerks. Ghiselli (1973) found intelligence tests predicted performance of salespeople {r - .30), but not sales clerks (r = -. 10). Personality tests were found to predict performance of salespeople (r = .40) and sales clerks (r = .30).
The individual predictors of salesperson performance differ in the extent of their intercorrelation.
Cognitive ability is moderately related to biographical information (Gray & Rosen, 1956; Johnson, 1940
Vinchur et al., 1998). Cognitive ability is also moderately related to sales ability (Cotham, 1968; Gray &
Rosen, 1956; Vinchur et al.).
Individual predictors of salesperson performance also differ in the extent to which they predict either subjective or objective measures of performance. Ford et al. (1987) found that cognitive ability, Personnel Selection and Training in Sales 10 sales aptitude and self-esteem were better predictors of subjective data; whereas biographical information was a better predictor of objective sales data. These authors suggested that objective sales data may be influenced by external uncontrollable factors, and fail to take into account the strength of the relationship between salesperson and customer.
The single domain predictors of performance vary among themselves with respect to their cost and ease of development and administration, as well as their validity. In a meta-analysis of alternative predictors of performance, Schmidt and Hunter (1998) noted that cognitive ability has the lowest cost of administration and was the best predictor of job-related learning, acquisition, and transfer to actual job performance (Schmidt & Hunter, 1982; Schmidt, Hunter, & Outerbridge, 1986). Alternatively, job knowledge tests, work sample measures, and some structured interviews require more time to develop and assume a base of specific knowledge of the job. Biographical information inventories were found to be difficult and time consuming to construct.
Where there is great variability among effect sizes of the same predictor-criterion relationship, moderator variables may account for some of the variability. Churchill et al. (1985) suggested that customer and product type moderate the size of effect of the determinants of salesperson performance. The performance measure used may also moderate the effect (Ford et al., 1987). Ford et al. found that biographical information was a stronger predictor for salespeople who sell or provide services to individual customers when measured objectively. Alternatively, Ford et al. found that aptitudes were stronger predictors for salespeople who sell to institutions when measured subjectively. The personality dimension of dominance was found to be a stronger predictor for salespeople selling services, rather than products, to individuals. Overall, Ford et al. concluded that the best predictors for salespeople selling industrial goods to institutional customers included: self-esteem, sales aptitude, and cognitive ability. The best predictors for salespeople selling consumer goods to individual customers included: biographical information
(personal history, marital and family status) and interpersonal skills.
Predictors of salesperson performance have also been studied in combination. Schmidt and Hunter
(1998) found the best combinations of predictors of performance in all occupations to be general mental Personnel Selection and Training in Sales 11 ability (GMA) and work sample {multiple R = .63), GMA and integrity test (multiple R = .65), and GMA and structured interview (multiple R = .63). Considering only the sales occupation, Vinchur et al. (1998) found the best combination of predictors of subjective performance to be sales ability and cognitive ability
(multiple R = .36). They found the best predictor of objective data to be the personality dimension, achievement, (r = .23) and noted that other predictors added little to the prediction.
Effectiveness of training interventions for salespeople
Training is well accepted as an effective means of enhancing productivity at work (Burke & Day,
1986; Guzzo, Jette, & Katzell, 1985; Russell, Terborg, & Powers, 1985). However, the evaluation of training interventions in applied settings is deficient (Bunker & Cohen, 1978; Kerr, 1975; Sackett &
Mullen, 1993). Often the only evaluation that is completed is the subjective response of employees to the training program.
There are several reasons that account for limited evaluation of training interventions (Dipboye,
1997). Even when resources are made available for evaluation it can be difficult to establish comparison groups. The perceptions of unfairness by those who remain untrained as a comparison group could threaten the internal validity of the research design. Also, trainers within the organization may oppose the objective, empirical evaluation of training programs. Political support for the training intervention from senior executives may reduce the motivation to objectively evaluate the program.
Despite some attempts to evaluate training interventions, scant empirical evidence exists reporting their effectiveness. Meyer and Raich (1983) found behavior modelling training had a direct effect on the number of sales achieved by the participants. Ivancevich (1974) found that goal setting training had an effect on performance for up to 18 months, after which time the effect disappeared. He suggested that post-training improvements would be observed only after three months. Ralis and O'Brien (1987) found similar positive effects on sales performance from goal-setting training.
Meta-analytic Review of Personnel Selection Procedures and Training Interventions
The goal of the present study is to improve the job performance of salespeople by increasing our understanding of the extent to which personnel selection procedures and training interventions influence Personnel Selection and Training in Sales 12 sales performance. Across the variety of personnel selection procedures, training interventions, and measures of sales performance, meta-analytic techniques were used to reduce the effect of statistical and measurement artifacts on the true effects of these human resource management activities. This meta- analytic review advances our understanding of how to improve sales performance. First, it examines the effects of both personnel selection and training interventions in the same study. Second, it refines previous meta-analyses of the effectiveness of personnel selection procedures by including only independent effect sizes, disaggregating effect sizes by type of performance measure used, using more conservative corrections for statistical artifacts than in a previous meta-analysis of predictors of sales performance
(Vinchur et al., 1998), and examining the homogeneity of the effects of personnel selection and training. Personnel Selection and Training in Sales 13
Method
Literature Search
Several approaches were used to obtain published and unpublished articles, dissertations, and book chapters that examined the effectiveness of selection procedures and training interventions. Computer- based searches of published empirical studies and dissertations were conducted by accessing PsycINFO
(1967-1999), ABI/Inform (1971-1990), Canadian Business and Current Affairs (1982-1998), and ERIC
(1966-1998). Manual searches of the literature were accomplished through reference lists accompanying several previous reviews of the literature (Vinchur et al., 1998; Churchill et al., 1985; Ford et al., 1987), and through individual studies found in the process of the search, along with relevant books and book chapters.
These approaches yielded a paucity of published studies of the effectiveness of training interventions in sales. An extended search was, therefore, directed to procuring more studies of training effectiveness. Three industrial psychologists were contacted personally, and asked for unpublished applied research. A group of 45 salespeople and sales managers in an executive sales management diploma course at the University of British Columbia were approached and asked to provide the names of companies with
"best practices" in salesforce training. Based on the information supplied by 28 respondents, the training departments of five companies were contacted for studies examining the effectiveness of their sales training interventions.
The "file drawer" problem (Rosenthal, 1979) influences meta-analyses, in that unpublished studies not yielding statistically significant results remain in the "file-drawers" of researchers, and are not included in the sample of studies examined by meta-analysis. An attempt was made to address this problem through the approaches previously described.
Criteria for the Inclusion of Results
Studies examining selection procedures. To be included in the present investigation, individual studies of selection procedures had to meet two criteria. First, studies had to present an effect size
(correlation coefficient between predictor, or composite of predictors, and criterion measure of salesperson Personnel Selection and Training in Sales 14 performance) of the relationship between an individual or set of personnel selection procedures and one or more measures of sales performance. Studies with dependent variables of training proficiency, turnover, or absenteeism were not included, to ensure consistency in the criterion. Second, studies had to use either job applicants or incumbents (i.e., not students) as subjects.
To be included in the meta-analysis, the effect sizes of the individual studies had to be independent of each other, where subjects did not contribute to more than one of the same predictor-criterion relationship. Where studies reported more than one correlation coefficient for the same predictor-criterion relationship with the same subjects, the effect sizes were averaged.
Studies examining training interventions. There were four criteria for studies of training interventions. First, the study had to examine a training intervention related to selling effectiveness. Such training programs included topics in human relations, self-awareness, problem solving, motivation, and the sales process. Training programs related to product knowledge, company orientation, or market/industry orientation were excluded. Second, the study had to include a measure of performance indexing a salesperson's behavior or business results such as sales productivity. Thus, measures of salespeople's subjective responses to the training program, or measurements of amount of information learned were excluded. Third, the study had to include a comparison group that did not receive the specific training intervention. The comparison group could receive another intervention, such as a placebo, or no training at all. Fourth, the subjects of the study had to be job incumbents rather than student subjects.
Coding
Coding of specific study variables (customer type, product type, job level, content of training or selection procedure, criterion type, and effect size information) was done independently by the author. The most widely used estimate of reliability of coding was achieved by calculating the agreement rate (Orwin,
1994, p. 147) between codes assigned by the author and a second coder on three studies of selection procedures and two studies of training interventions. The agreement rate, a ratio of number of observations agreed upon to the total number of observations, was found to be 90% on the studies coded. Personnel Selection and Training in Sales 15
Meta-Analytic Procedures
The effect sizes in the individual studies of selection procedures were standard Pearson correlation coefficients between selection assessment and criterion scores—validity coefficients. The results of individual studies of training interventions were reported as either standardized mean differences on the criterion performance variable, or as t-test statistics for comparing the mean difference between the experimental group who received the training program and the control group who did not receive the training program. In order to compare the effect sizes with selection procedures, standardized mean
differences for each rth study, du were converted to Pearson correlation coefficients by the following equation given by Rosenthal (1994, p. 239):
r =
This formula is appropriate for use when the populations of those given the treatment (in this case the training program) and those not given the treatment are regarded as essentially equal in size (Rosenthal,
1994, p. 239). T-test statistics were converted to Pearson correlation coefficients by the following equation given by Rosenthal (1994, p. 239):
where df is the degrees of freedom, (ne + nc - 2), where ne is the number of subjects in the experimental
training condition, and where nc is the number of subjects in the control condition.
Corrections for statistical artifacts. The correlation coefficients for selection procedures were individually corrected for range restriction in the independent variable and criterion unreliability of subjective performance ratings prior to being averaged. The correlation coefficients for training interventions were individually corrected for criterion unreliability of subjective performance ratings only.
In each personnel-selection study, correlation coefficients were corrected for range restriction because low scores on the independent variable were systematically excluded from the analysis; low scores would be Personnel Selection and Training in Sales 16 relevant in the intended population of job applicants. In most selection studies, information necessary to correct for range restriction—ratio of standard deviation of population of scores to standard deviation of sample of scores—is unavailable. Given the lack of information in the studies necessary to correct for range restriction, analyses were completed with an average estimate of range restriction provided by
Schmidt and Hunter (1977) that has been accepted as accurate (Alexander, Carson, Alliger, & Cronshaw,
1989; Vinchur et al., 1998). The correlation coefficient corrected for range restriction was found by the following equation given by Glass and Hopkins (1996, p. 122):
r,
where /v is the correlation coefficient corrected for range restriction, r is the correlation coefficient
uncorrected for range restriction, o~pop is the standard deviation of the scores on the independent variable in
the population, and asampie is the standard deviation of scores on the independent variable in the sample.
Corrections for criterion unreliability were made in the range-restriction-corrected correlation coefficients in selection studies and uncorrected correlation coefficients in training intervention studies, based on reliability estimates of the criterion given in the individual studies. Where criterion reliability information was unavailable in the study, an estimate of the average coefficient alpha—an estimate of intra-rater reliability—for supervisory ratings of overall job performance was used. This average was drawn from the distribution of intra-rater reliability coefficients provided by Viswesvaran, Ones, and
Schmidt (1996). The average coefficient alpha for supervisory ratings of overall job performance provided by this distribution is .86 (SD = .1433). Correlation coefficients for studies in which the criterion was a subjective measure of performance were corrected by standard partial disattenuation, i.e. dividing the correlation coefficient by the square root of the criterion reliability coefficient. Objective measures of performance were not corrected, because it was expected that there would be no systematic measurement error in the chosen criterion. The present study differs from the previous meta-analysis of predictors of salesperson performance (Vinchur et al., 1998), in that the previous meta-analysis reported the inter-rater Personnel Selection and Training in Sales 17 reliability estimate of .52, rather than the intra-rater reliability estimate of .86 in the present study, in correction for criterion unreliability. It was expected that this difference would lead to considerably lower disattenuated validity coefficients in the present study because of much lesser correction for criterion unreliability.
The only effect size corrections made in the present study were for unreliability in subjective measures of the criterion and range restriction of the independent variable in personnel selection studies.
Corrections for predictor unreliability were not made because the influence of this unreliability is relevant to the use of these tools in personnel selection. Hunter and Schmidt (1994) provided means of correcting for the following additional statistical artifacts: artificial dichotomization of the continuous dependent and independent variables split into proportions, imperfect construct validity of the independent variable, range restriction in the dependent variable, bias in the correlation coefficient, and study-caused variation. These additional corrections were not made in the present study because of the lack of evidence that they systematically influenced the results of the individual studies, lack of information on artifacts in the individual studies, and also, because of Rosenthal's (1994, p. 240) caution against over-correcting for the statistical artifacts.
Combining correlation coefficients. In order to assess the relationship between various selection procedures and performance, and to compare selection procedures with training interventions, the weighted mean correlation coefficient for all predictor-criterion relationships, and for all training effects were calculated. Corrected correlation coefficients for each study were first converted to a standard metric by
Fisher's Z-transformation, Zr. The Fisher's Z-transformation was used because it has a normal sampling distribution irrespective of the population correlation coefficient or the sample size of each effect size
(Glass & Hopkins, 1996). The weighted mean transformed correlation coefficient was found by assigning
weights to the studies corresponding to their degrees of freedom and calculating the weighted average Z'r by standard formulas (see, e.g., Shadish & Haddock, 1994), as: Personnel Selection and Training in Sales 18
Z'
where the weight (w,) assigned to each study is («, - 3). The weighted mean Z'r was then converted back
to r by referring to standard tabled values of Zr and r.
2 The sample estimate of the variance component of effect sizes, s (Zr), was estimated by the standard formula for the sample variance, which in the present context is:
k . . r k Ik
where Z is the mean standardized effect size for the zth study, and k is the number of studies contributing to the overall effect size.
2 The expected value of s (Zr) was found by the following equation given by Shadish and Haddock
(1994):
A-2 -
JJ
where v, is the conditional variance of Zr estimated by:
v. = ' («,-3)' and is the sample size of the zth study.
The expected value of the sample estimate of the variance component, a\ , was corrected for sampling error due to small sample sizes, consistent with the literature in integrating results in personnel selection (Hedges, 1983; Hunter, Schmidt, & Jackson, 1982; Schmidt & Hunter, 1977). The corrected variance component was found by the following equation given by Hunter et al. (1982, p. 44): Personnel Selection and Training in Sales 19
2 ^- 2 The value of a2 is estimated by the following equation given by Hunter et al. (1982, p. 70): where k is the total number of studies, N is the total number of subjects, r is the average correlation coefficient across studies, and a is the average ratio of corrected correlation coefficients (corrected for range restriction and criterion unreliability) to uncorrected coefficients across all of the studies, estimated by the following equation: r where rc is the correlation coefficient corrected for all the relevant statistical artifacts, and r is the uncorrected correlation coefficient. The upper and lower limits of the 95% confidence interval for each weighted mean of the correlation coefficient, Z'r, were found by multiplying the standard error of Z corrected for sampling error, o\ , by the critical value of 1.96 (for a = .05), adding and subtracting the resulting product from Z'r, and converting the endpoints of the confidence interval back to rs by referring to standard tabled values of Zr and r. There is a probability of .95 that intervals so constructed will capture the true mean correlation coefficient. The confidence interval also illustrates the degree of precision in the estimate of the true mean correlation coefficient. Statistical test of the assumption of homogeneity of effect. The equation for the weighted mean Personnel Selection and Training in Sales 20 correlation coefficient assumes that the studies share a common population effect size (Shadish & Haddock, 1994, p. 266). This assumption was tested by the following equation for the homogeneity test statistic, Q, given by Shadish & Haddock (1994, p. 266): 2 g = S[(Zr-Z;) /(v!)],. where Zr is the Z-transformed correlation coefficient and v, is the conditional variance of Zr defined above. When Q exceeds the critical value of chi-square with k - 1 degrees of freedom, the conclusion is that the variability in effect sizes is significantly greater than would be expected by chance alone, if all studies shared a common population effect size. In this case, the weighted mean correlation coefficient would not be interpreted, but rather it would be taken as a description of the mean of observed effect sizes. Results and Discussion Sample Description The exhaustive search of the literature produced 170 independent predictor-criterion effect sizes of selection procedures from 86 different samples in 59 studies, and 12 independent effect sizes of training interventions from 11 studies. Independent effect sizes were achieved by ensuring that no subject appeared in more than one of the same predictor-criterion relationship. Among the studies of selection procedures, 58 of the 59 were published articles, with one unpublished doctoral dissertation. Among the studies of training interventions, 7 of the 12 were published articles; the remaining 5 were unpublished doctoral dissertations. The studies were performed during the period 1953 to 1997, with the exclusion of 1971-1976 during which time no study in the sample appeared. Among the selection studies, the research was carried out with 34 samples of industrial salespeople, 26 samples of retail salespeople, and 26 with insufficient information to code for this variable. These salespeople worked in 26 companies that provide consumer goods, 20 companies that provide industrial goods, 37 that provide services, and 3 whose product is unknown. Among the training studies, the research was carried out with 3 samples of salespeople who sell to institutions, 7 who sell to individuals, 1 selling to both institutions and individuals, and 1 with insufficient information on this Personnel Selection and Training in Sales 21 variable. The research was done in 7 companies that provide consumer goods, 1 that provides industrial goods, 3 that provide services, and 1 with insufficient information. The sample size of the studies ranged from 16 subjects to 16,230 in selection procedures, with a median sample size of 88. The studies of training interventions ranged in size from 17 subjects to 1,017 with a median sample size of 105. Meta-Analytic Results and Discussion The overall results of the meta-analysis are presented in Table 1 for the combined criterion (average effect size across subjective and/or objective performance assessments for each independent predictor-criterion relationship). The average corrected correlation coefficients are presented for the single domain predictors of salesperson performance, composite domain predictors of salesperson performance, and training interventions in sales effectiveness. With reference to personnel-selection studies, in what follows criterion-related validity coefficients have been corrected for range restriction in the independent variable and attenuation in the criterion where the criterion was subjective ratings. With reference to training intervention studies, in what follows correlation coefficients for training interventions have been corrected for criterion unreliability where the criterion was subjective ratings. On average, composite- domain selection assessments predicted salesperson performance with a corrected multiple correlation of .59 (p < .05). The average corrected correlation coefficients of single-domain selection assessments (r = .27, p > .05) and training interventions (r = .16, p > .05) were not statistically significant. These results suggest that the job performance of a salesforce can be improved by the use of a composite domain assessment for personnel selection. The non-significant results for single-domain selection assessment and training interventions suggest that we cannot say with 95% confidence that these approaches are effective in salesforce improvement. In all of the analyses in the present study the homogeneity test statistic, Q, exceeded the critical value. This result suggests that the observed variance in effect sizes is greater than what would be expected by chance if all studies shared a common population effect size. When Q is rejected, Shaddish and Haddock (1994, p. 266) suggested disaggregating the study effect sizes by breaking the studies into Personnel Selection and Training in Sales 22 smaller groups by appropriate categories until a homogeneous effect is detected. Due to very small sample sizes found when breaking the studies into smaller groups, a further disaggregation of the study effect sizes was not possible in the present study. The results of the weighted mean correlation coefficients are presented in what follows as a description of the mean of observed effect sizes, with limited interpretation because of the heterogeneity of effects. Insert Table 1 about here The results of the meta-analysis by type of content of the selection procedure for the combined criterion are presented in Table 2. The corrected mean validity coefficients of the single domain selection procedures range between r = .22 and r = .44 (p > .05 for all coefficients). Although the corrected validity coefficients are positive and moderate, the variability of the effect does not allow us to conclude that the results are statistically significant. The results are similar to those obtained by Ford et al. (1987) in a previous meta-analysis of dimensions used in the prediction of salesperson performance. The results of the previous study were disaggregated into more narrowly focussed dimensions than the present study, and uncorrected for statistical artifacts other than sampling error. The range of effect sizes among the set of narrow dimensions in the previous study that are comparable to the broad dimensions in the present study contained the uncorrected effect sizes of the present study. This finding lends support to the estimates of the effect sizes of the various predictors of sales performance, because the studies included in the previous meta-analysis were drawn from journals of marketing and consumer research, whereas the present meta• analysis included studies drawn from the psychology journals. Insert Table 2 about here The results of the meta-analysis by type of content of the selection procedure for the subjective criterion and objective criterion are presented in Tables 3 and 4, respectively. Insert Table 3 about here Personnel Selection and Training in Sales 23 Insert Table 4 about here Subjective ratings of performance were predicted by the single domains of personality dimension (r = .20, p < .05), biographical information (r = .20, p <.05), and cognitive ability (r = .32, p < .05). The results of a previous meta-analysis of sales predictors (Vinchur et al., 1998) found a similar validity coefficient for the prediction of subjective ratings of performance by overall cognitive ability (r = .31), but a larger corrected correlation coefficient for biographical information (r = .52). The discrepancy between the correlation coefficients of biographical information and subjective ratings in the Vinchur et al. study and in the present study may be due to both fewer studies used in the present study (k = 5) than in the previous study (k = 8) and lesser correction for statistical artifacts in the present study. In general, the results suggest that subjective ratings of salesperson performance can be predicted by personality dimensions, biographical information, and cognitive ability. Objective performance assessments were predicted by the single domain of special purpose sales assessments with an average corrected correlation of .49 (p < .05). This result is similar to the average corrected correlation of .45 in the Vinchur et al. (1998) study. It is interesting to note that cognitive ability demonstrated no relationship with objective performance (r = -.09, p > .05), whereas it was moderately related to subjective ratings of performance (r = .32, p < .05). This finding is consistent with previous investigations which similarly found a positive relationship between cognitive ability and subjective ratings of performance, but no relationship with objective criteria (Bommer et al., 1995; Vinchur et al., 1998). That cognitive ability predicts supervisory ratings of performance and not objective performance suggests that some important aspects of a salesperson's job not reflected in their sales productivity may be related to cognitive ability. Analysis of the Utility of Selection Procedures and Training Interventions To evaluate the cost-benefit implications of the meta-analytic results, a utility analysis was conducted. The analysis of utility of human resource management (HRM) activities, such as personnel selection and training, is frequently used in industrial psychology to predict, describe, and/or explain their Personnel Selection and Training in Sales 24 usefulness or desirability (Boudreau, 1983; Cascio, 1980). The purpose of utility analysis is to express the substantial value of particular HRM activities to the organization to allow management to evaluate what is gained from the use of information from industrial and organizational psychology, economics, and information theory. It also allows the company to evaluate the relative contribution of its HRM investments, and establish their value in comparison to investments in other functional areas (Cascio, 1989). Utility analysis models were first developed in response to the deficits of traditional measurement and test theories in expressing the usefulness of tests. Utility analysis has come to reflect the influence of an HRM activity on: (1) the number of employees affected and the duration of the effect, (2) the average effect of the program, and (3) the resources required to implement and maintain the activity. Utility of Selection Procedures. Originally, utility analysis models were developed specifically for selection assessment procedures. One of the first utility analysis models defined the benefit of assessment as the success ratio, which is the proportion of selected employees whose job performance is found to be successful (Taylor & Russell, 1939). This model was found to be insufficient because it lacked information about the number of employees affected and the duration of effect. It was also criticized because it expressed the criterion as a dichotomous variable (success or failure), whereas job performance is usually considered as a continuous variable. A second model of utility defined the benefit of assessment as the expected mean overall job performance of selected applicants (Naylor & Shine, 1965). It expressed the utility of a selection procedure as the difference in average standardized criterion score between those selected by a test and those selected without it. This model was criticized because the difference in average standardized criterion score is difficult to interpret in an applied situation. A third model of utility, and the one employed in the present study, defined the benefit of selection assessment by the dollar-value gained by use of the assessment (Brogden, 1946a, 1946b, 1949; Cronbach & Gleser 1965, pp. 308-309). This model, which will be referred to as the Brogden-Cronbach-Gleser selection utility model, estimates the overall net utility of a selection procedure, AU, based on the number of employees hired by the selection assessment procedure, N, the average tenure of the employees, T, the validity coefficient of the selection Personnel Selection and Training in Sales 25 assessment, r^, the standard deviation of job performance in the applicant pool (preferably in dollars), SDy, the standardized average score on the predictor assessment, zx, and the overall cost of implementing and maintaining the selection assessment procedure, C, by the following equation, AU^NTr^SDy-C The utility of selection procedures in the present study was estimated by the Brogden-Cronbach- Gleser model with the estimated validity coefficients of .27 for the single domain selection procedure and .59 for the composite domain selection procedure. The average tenure of salespeople was estimated to be 2.4 years, based on data from the Bureau of Labor Statistics (1998). The estimation of the standard deviation of job performance (SDy) was obtained from a meta-analysis by Hunter et al. (1990) that examined empirical estimates of the variability of employees' productivity as a percentage of their average salary. Hunter et al. reported separate estimates of SDy for incumbent employees and applicants. Where selection procedures were examined in the present study, the estimate of SDy for applicants was employed because applicants are the population of interest. The average standard score on the predictor, zx, reflects the extent to which an organization can be selective, because the more applicants considered, the greater the difference between those selected and rejected. The estimates of zx used in the present study were obtained from average standard scores on the predictor variable for those selected at various selection ratios (assuming a normal distribution on the predictor), given by Hunter and Hunter (1984). The selection ratio is the ratio of the number of applicants selected for employment to the number of applicants in the applicant pool. The estimate of zx used, based on a selection ratio of .20, was 1.40. The estimate of zx used, based on a selection ratio of .40, was .97. Estimates of the cost, C, of selection assessment procedures were not included because they vary greatly depending on which procedure is used, how it is developed, and how its use is maintained in the organization. The estimates of utility in the present study represent the expected dollar-value gain to the organization from the use of single domain or composite domain selection assessments to hire one sales representative. Therefore, we set TY to 1.0 in the equation for AU. Personnel Selection and Training in Sales 26 Insert Table 5 about here The results in Table 5 illustrate the dollar value gain for each applicant selected by single or composite domain assessment over random selection, for two levels of sales representative at two selection ratios, per year (T = 1) and per average tenure (T = 2.4). Selection by a composite domain assessment procedure when selecting the top 20% of applicants for senior sales representative positions would yield a return to the organization of approximately $24,901 per employee per year of the employee's tenure. Selection by a composite domain assessment procedure when selecting the top 40% of applicants would yield a return to the organization of approximately $17,253 for each sales representative hired per year. Table 5 also includes the total dollar value gain for each applicant selected for the duration of the effect; in the case of personnel selection, the duration of effect is the length of the employee's tenure. With an estimate of the average tenure of a salesperson in the United States of 2.4 years (Bureau of Labor Statistics, 1998), the total return to the company of using a composite domain assessment procedure to select a sales representative from among the top 20% of applicants would be $59,762, and when selecting from among the top 40% of applicants would be $41,407. The cost of development, implementation, and maintenance of the selection assessment has not been taken into account in the present analyses; thus, these are gross utilities. The calculation of the cost- benefit implications of selection can be easily made by subtracting the proposed costs from the corresponding dollar-value benefit given in Table 5 for each sales representative hired. Utility of Training Interventions. Although utility analysis has been applied primarily to personnel selection procedures it has also been developed to express the outcomes of training and development programs (Cascio, 1989; Landy, Farr, & Jacobs, 1982; Schmidt, Hunter, & Pearlman, 1982). Computational formulas exist which estimate the utility of training interventions as a function of their cost, effectiveness, and duration of effect. In the studies examined in the present thesis observed values of cost and duration of effect were unavailable. Likewise, empirical distributions of these values were unavailable. The annual utility of training interventions was estimated by the following equation given by Cascio (1989, Personnel Selection and Training in Sales 27 p. 72): AU = N dSDy, where AU is the expected annual net benefit of training, N is the number of employees trained, d is the standardized mean difference of performance between those trained and those untrained, and SDy is the standard deviation of job performance. The standard deviation of job performance was the same value that was taken from empirical estimates provided by Hunter et al. (1990) in the calculation of utility of selection procedures. The empirical estimates of the utility of training interventions for an average sales representative and senior sales representative are presented in Table 5 along the with utility estimates for personnel selection. Training an average sales representative would yield a return to the organization of approximately $9,773 assuming that the duration of the effect of training was one year. The assumption that the duration of the effect of training endures for at least one year is not established by the empirical research reviewed in the present study. Therefore, the utility estimates of training interventions must be interpreted with caution. The comparison of the relative utilities of single domain selection procedures, composite domain selection procedures, and training interventions are based on the weighted mean effect sizes found by meta- analytic procedures in the present study. The utility estimates refer to the dollar value gain per year for each employee selected by valid selection procedures or trained with an effective training program for the duration of the effect. The duration of the effect of selection procedures is assumed to be the employee's tenure, T; however, the duration of the effect of training is unknown from the evidence in the present study. The cost of developing, implementing and maintaining personnel selection procedures and training interventions has not been taken into account in these utility estimates. On average, the dollar value gain per year from personnel selection procedures is greater than the gain from training interventions. Summary On average, composite domain selection assessments predicted salesperson performance, while the effects of single domain selection assessments and training interventions were not statistically significant. Personnel Selection and Training in Sales 28 The disaggregation of results of personnel selection procedures by type of content and criterion, however, clarified the results. Personality assessments, biographical information inventories, and cognitive ability tests were found to be promising predictors of subjective ratings of performance. Special purpose sales assessments were good predictors of objective ratings of performance. It is worthwhile to consider subjective and objective performance criteria separately in sales occupations, as they appeared to be predicted by different individual difference dimensions. The disaggregation of the results of single-predictor personnel selection procedures by type of predictor content and criterion measurement reduced the variability among the effect sizes of single predictors overall. This reduction in the variability among effect sizes was demonstrated where the effect size of single predictors overall was not significantly different from zero, whereas some of the individual single predictors were significantly different from zero when either a subjective or objective criterion was considered. This finding suggests that the type of content of the predictor and the type of criterion measurement moderate the relationship between single dimension personnel selection methods and sales performance. The applied research of training interventions is limited to a relatively small number of studies that produced small effects with a high degree of variability. This latter finding argues for further research into the effectiveness of training interventions in sales occupations, with a particular focus on the duration of effect. There are two limitations in this study. First, the comparison between selection procedures and training interventions was limited by a disproportionately small number of studies available in training, greater proportion of unpublished studies in training, and greater variability among effect sizes in training. Second, the sales occupation is extremely broad, representing substantial differences in job complexity which range from sales clerks in department stores, to representatives selling industrial products to international customers. The comparison among single- and multiple dimension selection procedures and training interventions suggests that selection by multiple dimensions is the only approach examined in the present Personnel Selection and Training in Sales 29 study that leads to a valid improvement in the job performance of salespeople. Training interventions were marked by a greater amount of variability than selection procedures. It was not possible in this study to examine possible moderators of the effect of training, because of the small number of training studies. Personnel Selection and Training in Sales 30 References References marked with an asterisk indicate studies included in the meta-analysis. Albright, L. E., Glennon, J. R., & Smith, W. J. (1963). The uses of psychological tests in industry. Cleveland, Ohio: Howard Allen. Alexander, R. A., Carson, K. P., Alliger, G. M., & Cronshaw, S. F. (1989). Empirical distributions of range restricted SD in validity studies. Journal of Applied Psychology, 74, 253-258. * Arvey, R. D., Miller, H. E., Gould, R., & Burch, P. (1987). Interview validity for selecting sales clerks. Personnel Psychology, 40, 1-12. * Ash, P. (1960). Validity information exchange No. 13-05. Personnel Psychology, 13, 454. * Baehr, M. E., Williams, G. B. (1968). Prediction of sales success from factorially determined dimensions of personal background data. Journal of Applied Psychology, 52, 98-103. * Bagozzi, R. P. (1978). Salesforce performance and satisfaction as a function of individual difference, interpersonal, and situational factors. Journal of Marketing Research, 15, 517-531. * Bagozzi, R. P. (1980). Performance and satisfaction in an industrial sales force: An examination of their antecedents and simultaneity. Journal of Marketing, 44, 65-77. * Baier, D. E., & Dugan, R. D., (1956). Tests and performance in a sales organization. Personnel Psychology, 9, 17-26. * Barling, J., Kelloway, E. K., & Cheung, D. (1996). Time management and achievement striving interact to predict car sales performance. Journal of Applied Psychology, 81, 821-826. * Barrick, M. R., Mount, M. K., & Strauss, J. P. (1993). Conscientiousness and performance of sales representatives: Test of the mediating effects of goal setting. Journal of Applied Psychology, 78, 715-722. * Bass, B. M. (1962). Further evidence on the dynamic character of criteria. Personnel Psychology, 15, 93-97. * Bass, B. M. (1957). Validity information exchange No. 10-25. Personnel Psychology, 10, 343- 344. Personnel Selection and Training in Sales 31 * Behrens, G. M., & Halverson, R. R. (1991). Predicting successful territory sales performance. Journal of Business and Psychology, 6, 273-277. * Berman, J. A. (1993). Validation of the P.D.I, employment inventory in a retail chain. Journal of Business and Psychology, 7, 413-419. * Bluen, S. D., Barling, J., & Burns, W. (1990). Predicting sales performance, job satisfaction, and depression by using the achievement strivings and impatience-irritability dimensions of Type A behavior. Journal of Applied Psychology, 75, 212-216. Bommer, W. H., Johnson, J. L., Rich, G. A., Podsakoff, P. M., & MacKenzie, S. B. (1995). On the interchangeability of objective and subjective measures of employee performance: A meta-analysis. Personnel Psychology, 48, 587-605. Borman, W. C. (1994). Job Behavior, Performance, and Effectiveness. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of Industrial and Organizational Psychology: Vol. 2 (pp. 271-326). Palo Alto, CA: Consulting Psychologists Press. * Bosshardt, M. J., Carter, G. W., Gialluca, K. A., Dunnette, M. D., & Ashworth, S. D. (1992). Predictive validation of an insurance agent support person selection battery. Journal of Business and Psychology, 7, 213-224. Boudreau, J. W. (1983). Economic considerations in estimating the utility of human resource productivity improvement programs. Personnel Psychology, 36, 551-576. Boudreau, J. W. (1994). Utility Analysis. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of Industrial and Organizational Psychology: Volume 2 (pp. 621-745). Palo Alto, CA: Consulting Psychologists Press. * Bray, D. W., & Campbell, R. J. (1968). Selection of salesmen by means of an assessment center. Journal of Applied Psychology, 52, 36-41. Brogden, H. E. (1946a). On the interpretation of the correlation coefficient as a measure of predictive efficiency. Journal of Educational Psychology, 37, 65-76. Brogden, H. E. (1946b). An approach to the problem of differential prediction. Psychometrika, 14, Personnel Selection and Training in Sales 32 169-182. Brogden, H. E. (1949). When testing pays off. Personnel Psychology, 2, 171-183. * Brown, M. G., Malott, R. W., Dillon, M. J., & Keeps, E. J. (1980). Improving customer service in a large department store through the use of training and feedback. Journal of Organizational Behavior Management, 2, 251-264. * Brown, S. H. (1981). Validity generalization and situational moderation in the life insurance industry. Journal of Applied Psychology, 66, 664-670. * Brown, S. H., Stout, J. D., Dalessio, A. T., & Crosby, M. M. (1988). Stability of validity indices through test score ranges. Journal of Applied Psychology, 73, 736-742. * Bruce, M. M. (1954). Validity information exchange No. 7-22. Personnel Psychology, 7, 157. * Bruce, M. M. (1957). Validity information exchange No. 10-3. Personnel Psychology, 10, 77-78. * Bruce, M. M., & Friesen, E. P. (1956). Validity information exchange No. 9-35. Personnel Psychology, 9, 380. Bunker, K. A., & Cohen, S. L. (1977). The rigors of training evaluation: A discussion and field demonstration. Personnel Psychology, 30, 525-541. Bureau of Labor Statistics. (1998). Employment by major occupational group, 1986, 1996, and projected 2006 (November 1997). Retrieved from World Wide Web: http://stats.bls.gov/emptab6.htm. Bureau of Labor Statistics, U.S. Department of Labor. (1996). Occupational Outlook Handbook, February, 1996 (Bulletin 2470). Latham, Maryland: Bernan Press. Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of managerial training. Journal of Applied Psychology, 71, 232-245. * Burroughs, W. A., & White, L. L. (1996). Predicting sales performance. Journal of Business and Psychology, 11, 73-84. * Bush, R. P., Bush, A. J., Ortinau, D. J., & Hair, J. F. (1990). Developing a behavior-based rating scale to assess retail salesperson performance. Journal of Retailing, 66, 119-136. Bureau of Labor Statistics (1998). Employment by major occupational group, 1986, 1996, and Personnel Selection and Training in Sales 33 projected 2006 (November 1997). Retrieved from World Wide Web: http://stats.bls.gov/emptab6.htm. Campbell, J. P. (1991). Modeling the Performance Prediction Problem in Industrial and Organizational Psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of Industrial and Organizational Psychology: Volume I (pp. 687-732). Palo Alto, CA: Consulting Psychologists Press, 687-732. Cascio, W. F. (1980). Responding to the demand for accountability: A critical analysis of three utility models. Organizational Behavior and Human Performance, 25, 32-45. Cascio, W. F. (1989). Using utility analysis to assess training outcomes. In I. L. Goldstein (Ed.), Training and Development in Organizations (pp. 63-88). CA: Jossey-Bass. Churchill, G. A., Ford, N. M., Hartley, S. W., & Walker, 0. C. (1985). The determinants of salesperson performance: A meta-analysis. Journal of Marketing Research, 22, 103-118. Churchill, G. A., Ford, N. M., & Walker, O. C. (1997). Sales Force Management. Chicago, IL: Irwin. Comer, J. M. and Dubinsky, A. J. (1985). Managing the Successful Sales Force. Lexington, MA: Lexington. * Cotham, J. C, III. (1968). Job attitudes and sales performance of major appliance salesmen. Journal of Marketing Research, 5, 370-375. * Cotham, J. C, III (1969). Using personal history information in retail salesman selection. Journal of Retailing, 45, 31-38. * Crant, J. M. (1995). The proactive personality scale and objective job performance among real estate agents. Journal of Applied Psychology, 80, 532-537. Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personnel decisions (2nd ed.). Urbana, IL: University of Illinois Press. * Dalessio, A. T., Crosby, M. M., & McManus, M. A. (1996). Stability of biodata keys and dimensions across English speaking countries: A test of the cross situational hypothesis. Journal of Business and Psychology, 10, 289-296. Personnel Selection and Training in Sales 34 * Dalessio, A. T., & Silverhart, T. A. (1994). Combining biodata test and interview information: Predicting decisions and performance criteria. Personnel Psychology, 47, 303-315. * Dawson, K. E., Soper, B., & Pettijohn, C. E. (1992). The effects of empathy on salesperson effectiveness. Psychology & Marketing, 9, 297-310. * Diamante, T. (1993). Unitarian validation of a mathematical problem-solving exercise for sales occupations. Journal of Business and Psychology, 7, 383-401. Dipboye, R. L. (1997). Organizational barriers to implementing a rational model of training. In M. A. Quinones & A. Ehrenstein (Eds.), Training for a rapidly changing workplace: Applications of psychological research (pp. 31-60). Washington, DC: American Psychological Association. Dubinsky, A. J., & Barry, T. E. (1982). A survey of sales management practices. Industrial Marketing Management, 11, 135-139. * Dugan, R. D.(1961). Validity information exchange No. 14-01. Personnel Psychology, 14, 213- 216. * Dunnette, M. D., McCartney, J., Carlson, H. C, & Kirchner, W. K. (1962). A study of faking behavior on a forced-choice self-description checklist. Personnel Psychology, 15, 13-24. * Emil Van Leeuwen Teachers College. (1956). Validity information exchange No. 9-36. Personnel Psychology, 9, 381. * Etcof, L. (1983). The effects of coding material availability and descriptive coding inclusiveness on the immediate recall, delayed recall and behavioral reproduction of modeled sales behaviors in an applied learning sales training program. Unpublished doctoral dissertation. University of Toledo. * Farsangi, A. A. (1984). Improving the effectiveness of low-performing salesclerks through a supplementary training program. Unpublished doctoral dissertation. United States International University. * Fitzpatrick, E. D., & McCarty, J. J. (1956). Validity information exchange No. 9-47. Personnel Psychology, 9, 526-527. Ford, N. M., Walker, O.C., Churchill, G. A., & Hartley, S. W. (1987). Selecting successful Personnel Selection and Training in Sales 35 salespeople: A meta-analysis of biographical and psychological selection criteria. In M. J. Houston (Ed.), Review of Marketing (pp. 90-131). Chicago: American Marketing Association. Ghiselli, E. E. (1966). The validity of ocupational aptitude tests. New York: Wiley. Ghiselli, E. E. (1973). The validity of aptitude tests in personnel selection. Personnel Psychology, 26, 461-477. Glass, G. V., & Hopkins, K. D. (1996). Statistical methods in education and psychology. Boston, MA: Allyn & Bacon. * Gray, E. J., & Rosen, J. C. (1956). Validity information exchange No. 9-7. Personnel Psychology, 9, 112. * Gregg, S. (1983). An analysis of the impact of a behavioral style awareness training program on retail sales effectiveness of commission sales personnel in a major department store chain in the southwest. Unpublished doctoral dissertation, North Texas State University. Guzzo, R. A., Jette, R. D., & Katzell, R. A. (1985). The effects of psychologically based intervention programs on worker productivity: A meta-analysis. Personnel Psychology, 38, 275-292. * Hakstian, A. R., Scratchley, L. S., MacLeod, A. A., Tweed, R. G., & Siddarth, S. (1997). Selection of telemarketing employees by standardized assessment procedures. Psychology & Marketing, 14, 703-726. * Harris, W. A., & Vincent, N. L. (1967). Comparison of performance of sales training graduates and nongraduates. Journal of Applied Psychology, 51, 436-441. Hedges, L. V. (1983). A random effects model for effect sizes. Psychological Bulletin, 93, 388- 395. Heneman, R. L. (1986). The relationship between supervisory ratings and results-oriented measures of performance: A meta-analysis. Personnel Psychology, 39, 811-826. * Hogan, J., Hogan, R., & Gregory, S. (1992). Validation of a sales representative selection inventory. Journal of Business and Psychology, 7, 161-171. * Hollenbeck, J. R., Brief, A. P., Whitener, E. M., & Pauli, K. E. (1988). An empirical note on the Personnel Selection and Training in Sales 36 interaction of personality and aptitude in personnel selection. Journal of Management, 14, 441-451. Hopkins, D. S. (1978). Training the Sales Force: A Progress Report (Conference Board Report No. 737). New York: Hopkins, D. S. * Hughes, J. L., & Dodd, W. E. (1961). Validity versus stereotype: Predicting sales performance by ipsative scoring of a personality test. Personnel Psychology, 14, 343-355. Hunter, J. E., & Hunter, R. F. (1984). The validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage. Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis: Cumulating research findings across studies. Newbury Park, CA: Sage. Hunter, J. E., Schmidt, F. L., & Judiesch, M. K. (1990). Individual differences in output variability as a function of job complexity. Journal of Applied Psychology, 75, 28-42. * Ivancevich, J. M. (1976). Effects of goal setting on performance and job satisfaction. Journal of Applied Psychology, 61, 605-612. * Ivancevich, J. M., & Smith, S. V. (1981). Goal setting interview skills training: Simulated and on- the-job analyses. Journal of Applied Psychology, 66, 697-705. Johnson, A. P. (1940). A study of one company's criteria for selecting college graduates. Journal of Applied Psychology, 24, 253-264. Kerr, S. (1975). On the folly of rewarding A, while hoping for B. Academy of Management Journal, 18, 769-783. * Lamont, L. M., & Lundstrom, W. J. (1977). Identifying successful industrial salesmen by personality and personal characteristics. Journal of Marketing Research, 14, 517-529. Landy, F. J. & Farr, J. L. (1983). The Measurement of Performance: Methods, Theory and Applications. New York: Academic Press. Landy, F. J., Farr, J. L., & Jacobs, R. (1982). Utility concepts in performance measurement. Personnel Selection and Training in Sales 37 Organizational Behavior and Human Performance, 30, 15-40. * Lee, C, & Gillen, D. J. (1989). Relationship of Type A behavior pattern, self-efficacy perceptions on sales performance. Journal of Organizational Behavior, 10, 75-81. Levy, M. & Sharma, A. (1993). Relationships among measures of retail salesperson performance. Journal of the Academy of Marketing Science, 21 (3), 231-238. * Maher, H. (1963). Validity information exchange No. 16-02. Personnel Psychology, 16, 14-11. * Merenda, R. F., Musiker, H. R., & Clarke, W. V. (1960). Relation of the self-concept to success in sales management. Engineering and Industrial Psychology, 2, 69-11. * Meyer, H. H., & Raich, M. S. (1983). An objective evaluation of a behavior modeling training program. Personnel Psychology, 36, 755-761. * Mitchell, T. W., & Klimoski, R. J. (1982). Is it rational to be empirical? A test of methods for scoring biographical data. Journal of Applied Psychology, 67, 411-418. * Morrow, C. C, Jarrett, M. Q., & Rupinski, M. T. An investigation of the effect and economic utility of corporate-wide training. Personnel Psychology, 50, 91-119. * Mount, M. K., Barrick, M. R., & Strauss, J. P. (1994). Validity of observer ratings of the big five personality factors. Journal of Applied Psychology, 79, 272-280. Naylor, J. C, & Shine, L. C. (1965). A table for determining the increase in mean criterion score obtained by using a selection device. Journal of Industrial Psychology, 3, 33-42. * Orpen, C. (1985). Patterned behavior description interviews versus unstructured interviews: A comparative validity study. Journal of Applied Psychology, 70,114-116. Orwin, R. G. (1994). Evaluating coding decisions. In H. Cooper and L. V. Hedges (Eds), The Handbook of Research Synthesis (pp. 139-162). New York: Russell Sage Foundation. * Owen, K. J. (1990). An evaluation of the short term outcomes of an experiential training intervention. Unpublished doctoral dissertation, University of Minnesota. * Phillips, J. F. (1992). Predicting sales skills. Journal of Business and Psychology, 7, 151-160. Pricewaterhouse Coopers (1998). Sales Compensation: 1998. Retrieved from World Wide Web: Personnel Selection and Training in Sales 38 http://careers.wsj .com/?content=cwc-salariesindex.htm. * Puffer, S. M. (1987). Prosocial behavior, noncompliant behavior, and work performance among commission salespeople. Journal of Applied Psychology, 72, 615-621. * Rafaeli, A., & Klimoski, R. J. (1983). Predicting sales success through handwriting analysis: An evaluation of the effects of training and handwriting sample content. Journal of Applied Psychology, 68, 212-217. * Rochford, L., & Wotruba, T. R. (1996). The impact of sales management changes on new product success. Journal of the Academy of Marketing Science, 24, 263-270. Rosenthal, R. (1979). The "file-drawer" problem and tolerance for null results. Psychological Bulletin, 86, 638-641. Rosenthal, R. (1994). Parametric measures of effect size. In. H. Cooper and L. V. Hedges (Eds.), The Handbook of Research Synthesis (pp. 231-244). New York: Russell Sage Foundation. * Ross, P. F., & Dunfield, N. M. (1964). Selecting salesmen for an oil company. Personnel Psychology, 17, 75-84. * Rush, C. H. (1953). A factorial study of sales criteria. Personnel Psychology, 6, 9-24. * Russell, J. S., Terborg, J. R., & Powers, M. L. (1985). Organizational performance and organizational level training and support. Personnel Psychology, 38, 849-863. Sackett, P. R., & Mullen, E. J. (1993). Beyond formal experimental design: Towards an expanded view of the training evaluation process. Personnel Psychology, 46, 613-627. Schmidt, F. L., Bemer, J. G., & Hunter, J. E. ( 1973). Racial differences in validity of employment tests: Relaity or illusions? Journal of Applied Psychology, 53, 5-9. Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540. Schmidt, F. L., & Hunter, J. E. (1992). Development of causal models of processes determining job performance. Current Directions in Psychological Science, 1, 89-92. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel Personnel Selection and Training in Sales 39 psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. Schmidt, F. L., Hunter, J. E., & Outerbridge, A. N. (1986). The impact of job experience and ability on job knowledge, work sample performance, and supervisory ratings of job performance. Journal of Applied Psychology, 71, 432-439. Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1982). Assessing the economic impact of personnel programs on productivity. Personnel Psychology, 35, 333-347. Schmidt, F. L., Hunter, J. E., Pearlman, K., & Shane, G. S. (1979). Further tests of the Schmidt- Hunter Bayesian Validity Generalization Model. Personnel Psychology, 32, 257-281. Schmidt, F. L., Hunter, J. E., & Urry, V. W. (1976). Statistical power in criterion-related validity studies. Journal of Applied Psychology, 61, 473-485. Shadish, W. R. & Haddock, C. K. (1994). Combining Estimates of Effect Size. In H. Cooper & L. V. Hedges (Eds.), The Handbook of Research Synthesis (pp. 261-281). New York: Russell Sage Foundation. * Speer, G. S. (1957). Validity information exchange No. 10-16. Personnel Psychology, 10, 206. * Squires, P., Torkel, S. J., Smither, J. W., & Ingate, M. R. (1991). Validity and generalizability of a role-play test to select telemarketing representatives. Journal of Occupational Psychology, 64, 37-47'. * Swenson, M. J., & Herche, J. (1994). Social values and salesperson performance: An empirical examination. Journal of the Academy of Marketing Science, 22, 283-289. * Taylor, E. K., & Hilton, A. C. (1960). Sales Personnel Description Form: Summary of validities. Personnel Psychology, 13, 173-179. Taylor, H. C, & Russell, J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection: Discussion and tables. Journal of Applied Psychology, 23, 565-578. * Taylor, E. K., & Schneider, D. (1954). Validity information exchange No. 7-23. Personnel Psychology, 7, 158. * Vincent, N. L., & Dugan, R. D. (1962). Validity information exchange No 15-03. Personnel Personnel Selection and Training in Sales 40 Psychology, 15, 223-225. Vinchur, A. J., Schippmann, J. S., Switzer, F. S., Ill, & Roth, P. L. (1998). A meta-analytic review of predictors of job performance for salespeople. Journal of Applied Psychology, 83, 586-597. Viswesvaran, C, Ones, D. S., & Schmidt, F. L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-574. * Waters, L. K., & Waters, C. W. (1970). Peer nominations as predictors of short-term sales performance. Journal of Applied Psychology, 54,42-44. * Weekley, J. A., & Gier, J. A. (1987). Reliability and validity of the situational interview for a sales position. Journal of Applied Psychology, 72,484-487. * Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25-49. Weitz, B. A. (1981). Effectiveness in sales interactions: A contingency framework. Journal of Marketing, 45, 85-103. Wexley, K.N., & Latham, G.P. (1981). Developing and training human resources in organizations. Glenview, IL: Scott Foresman. * Wilson, J. E. (1959). Evaluating a four year sales-selecting program. Personnel Psychology, 12, 97-104. * Worbois, G. M., & Kanous, L. E. (1954). The validity of the Worthington Personal History for a sales job. Personnel Psychology, 7, 209-217. Personnel Selection and Training in Sales 41 Table 1 Correlation Coefficients by Single Domain Predictor, Composite Domain Predictor, or Training Intervention for the Combined Criterion (average of subjective and objective criteria) Approach to Improve Job Performance 95% CI for Mean r k N Mean r SE lower upper single predictor 144 58,310 .27 .29 -.30 .69 composite predictor 10 1,246 .59 .20 .27 .79 training intervention 12 2,378 .16 .24 -.30 .57 Note: k = number of studies; N = number of subjects; Mean r = average correlation coefficient corrected for criterion unreliability, and range restriction in the independent variable for single and composite predictors; SE = standard error corrected for sampling error; 95% CI = 95% confidence interval. Personnel Selection and Training in Sales 42 Table 2 Validity Coefficients by Content of Single Domain Selection Assessment for the Combined Criterion (average of subjective and objective criteria) Content of Selection Assessment 95% CI for r( k N Mean rcr Mean rcriT SE Lower Upper job skill/knowledge 15 3,171 .22 .33 .31 -.26 .74 personality dimension 41 4,325 .14 .24 .17 -.09 .51 biographical information 24 44,509 .16 .26 .18 -.09 .54 cognitive ability 22 2,784 .13 .22 .22 -.22 .57 job specific aptitude 16 1,856 .19 .30 .39 -.44 .80 special purpose 24 1,481 .28 .44 .32 -.16 .80 Note: k - number of studies; N = number of subjects; Mean rcr = average validity coefficient corrected for criterion unreliability of subjective ratings; Mean rCKrr = average validity coefficient corrected for criterion unreliability of subjective ratings and range restriction in the independent variable; SE = standard error corrected for sampling error; 95% CI = 95% confidence interval. Personnel Selection and Training in Sales 43 Table 3 Validity Coefficients by Content of Single Domain Selection Assessment for the Subjective Criterion Content of Selection Assessment 95% CI for r( k N Mean rCT Mean rcrrT SE Lower Upper job skill/knowledge 11 1,626 .24 .36 .28 -.17 .74 personality dimension 24 1,919 .12 .20 .03 .13 .27 biographical information 5 670 .12 .20 .08 .03 .36 cognitive ability 17 2,190 .20 .32 .00 .27 .35 job specific aptitude 9 1,350 .17 .28 .29 -.29 .70 special purpose 16 1,069 .27 .42 .38 -.29 .81 Note: k = number of studies; N = number of subjects; Mean rcr = average validity coefficient corrected for criterion unreliability of subjective ratings; Mean rcnrr = average validity coefficient corrected for criterion unreliability of subjective ratings and range restriction in the independent variable; SE = standard error corrected for sampling error; 95% CI = 95% confidence interval. Personnel Selection and Training in Sales 44 Table 4 Validity Coefficients by Content of Single Domain Selection Assessment for the Objective Criterion Content of Selection Assessment 95% CI for r^ k N Mean r Mean r^ SE Lower Upper job skill/knowledge 6 1,666 .20 .30 .40 -.43 .80 personality dimension 18 2,539 .15 .26 .28 -.26 .67 biographical information 17 13,665 .20 .32 .18 -.03 .60 cognitive ability 6 679 -.06 -.09 .38 -.69 .59 job specific aptitude 4 452 .21 .34 .21 -.08 .65 special purpose 10 558 .32 .49 .14 .25 .68 Note: k = number of studies; N = number of subjects; Mean r = average validity coefficient; Mean rrr = average validity coefficient corrected for range restriction in the independent variable; SE = standard error corrected for sampling error; 95% CI = 95% confidence interval. Personnel Selection and Training in Sales 45 Table 5 Relative Utility Estimates of Valid Selection Procedures and Effective Training Interventions Dollar value gain Dollar value gain Human Resource Management Activity per year for the for each employee duration of tenure Selection Assessment Procedure Selection Ratio = . 20 Single dimension (r = .27) Senior sales representative $ 16,119 $ 38,686 Sales representative 11,395 27,349 Multiple dimension (R = .59) Senior sales representative 35,224 84,537 Sales representative 24,901 59,762 Selection Ratio = . 40 Single dimension (r = .27) Senior sales representative 11,168 26,804 Sales representative 7,895 18,949 Multiple dimension (R = .59) Senior sales representative 24,405 58,572 Sales representative 17,253 41,407 Training Intervention Training effectiveness (d - .32) Senior sales representative $ 13,824 Sales representative 9,773