DEVELOPING A SUPPLIER PERFORMANCE MANAGEMENT PROGRAM AT
A SOUTHERN CALIFORNIA UTILITY COMPANY
A Project
Presented to the
Faculty of
California State Polytechnic University, Pomona
In Partial Fulfillment
Of the Requirements for the Degree
Master of Business Administration
By
Jennifer M. Doi
2017 SIGNATURE PAGE
PROJECT: DEVELOPING A SUPPLIER PERFORMANCE MANAGEMENT PROGRAM AT A SOUTHERN CALIFORNIA UTILITY COMPANY
AUTHOR: Jennifer M. Doi
DATE SUBMITTED: Summer 2017
College of Business Administration
Dr. Kevin W. Moncrief ______Project Committee Chair Adjunct Professor, International Business and Marketing
Dr. Cheryl R. Wyrick ______Project Committee Reader Professor, Management and Human Resources
ii ABSTRACT
The purpose of this study was to develop, implement, and evaluate a Supplier
Performance Management program at a Southern California utility company. The Supply
Management organization identified the need to effectively manage the performance of the company’s IT service providers because they represent a significant amount of cost and resources for the organization. Experience within the organization indicated that without a defined strategy for performance management, it is difficult to calculate and preserve the value expected from negotiated contracts. A Supplier Performance
Management (SPM) program provides a method for organizations to track key metrics for cost, quality, and service through the use of scorecards and dashboards. (Rizza, 2006)
The industry research performed on companies of similar size and spend indicates that the most successful companies employ SPM programs to maintain and improve the value of their supplier relationships. The study was focused on establishing a SPM program to identify key internal metrics critical to organizational objectives, measure a supplier’s performance against such metrics, and design a process to review results and share feedback among the participants.
iii TABLE OF CONTENTS
Signature Page ...... ii
Abstract ...... iii
List of Tables ...... vii
List of Figures ...... viii
Glossary ...... ix
List of Abbreviations ...... x
Chapter 1: Introduction ...... 1
1.1 Chapter Overview ...... 1
1.2 Background ...... 1
1.3 Problem Statement ...... 4
1.4 Research Objectives...... 5
1.5 Significance of Research...... 5
1.6 Research Questions ...... 6
1.7 Limitations ...... 7
1.8 Chapter Summary ...... 8
Chapter 2: Literature Review...... 9
2.1 Chapter Overview ...... 9
2.2 Industry Research on SPM Programs ...... 10
2.2.1 Industry Review ...... 10
2.2.2 Benefits of SPM Programs...... 11
2.3 Best Practices for Developing SPM Programs ...... 13
iv 2.3.1 SPM Best Practices ...... 13
2.3.2 Focus on Scorecard Development ...... 15
2.4 Maintaining Effective SPM Programs ...... 20
2.4.1 Challenges of SPM Programs ...... 20
2.4.2 Focus on Continuous Improvement ...... 20
2.5 Research Results ...... 22
Chapter 3: Methodology ...... 23
3.1 Chapter Overview ...... 23
3.2 Research Design...... 23
3.2.1 Qualitative Survey ...... 24
3.2.2 Quantitative Scorecard...... 25
3.3 Survey and Scorecard Design ...... 25
3.4 Sampling ...... 25
3.5 Procedures...... 26
3.6 Data Analysis ...... 27
3.7 Anticipated Results ...... 28
3.8 Chapter Summary ...... 29
Chapter 4: Research Findings ...... 30
4.1 Chapter Overview ...... 30
4.2 Research Question 1 ...... 30
4.3 Research Question 2 ...... 31
4.4 Research Question 3 ...... 31
4.5 Research Question 4 ...... 32
v 4.6 Research Question 5 ...... 32
4.7 Research Question 6 ...... 33
4.8 Research Question 7 ...... 34
4.9 Chapter Summary ...... 34
Chapter 5: Recommendation and Conclusion...... 36
5.1 Chapter Overview ...... 36
5.2 Research Summary ...... 36
5.3 Project Summary...... 36
5.4 Implications for Management ...... 39
5.5 Recommendations for Management ...... 39
5.6 Conclusion ...... 40
References...... 41
Appendix A: Construct and Measurement Table...... 45
Appendix B: Qualitative Survey...... 46
Appendix C: Quantitative Scorecard ...... 47
Appendix D: Supplier A Scorecard ...... 48
Appendix E: Supplier B Scorecard ...... 49
Appendix F: Supplier C Scorecard ...... 50
Appendix G: Supplier D Scorecard ...... 51
Appendix H: Overall Ratings from Qualitative Survey...... 52
Appendix I: Major Issues from Qualitative Survey...... 53
vi LIST OF TABLES
Table 1 Literature Review Table ...... 9
Table 2 Scorecard Development Best Practices ...... 19
vii LIST OF FIGURES
Figure 1 Supplier Segmentation Model...... 3
Figure 2 Benefits of Effective Supplier Management...... 4
Figure 3 Linking the Supply Management Framework to the Balanced Scorecard.. 18
Figure 4 Scorecard and Quarterly Business Review Process and Guidelines...... 27
viii GLOSSARY
Application Development and Maintenance: A term that refers to custom software development, building of Enterprise Application Integration systems and maintenance of systems and subsystems (MindLance, n.d.).
Balanced Scorecard: A set of measures that gives management a fast but comprehensive view of the business (Kaplan and Norton, 1992).
Information Technology: The technology involving the development, maintenance, and use of computer systems, software, and networks for the processing and distribution of data (Information Technology, n.d.).
Key Performance Indicator: A set of quantifiable measures that a company uses to gauge its performance over time (Key Performance Indicator, n.d.).
Return on Investment: A profitability measure that evaluates the performance of a business by dividing net profit by net worth (Return on Investment, n.d.).
Service Level Agreement: A document describing the level of service expected by a customer from a supplier, laying out the metrics by which that service is measured, and the remedies or penalties, if any, should the agreed-upon levels not be achieved (Service
Level Agreement, n.d.).
Strategic Sourcing: A method of managing procurement processes for an organization in which the procedures, methods, and sources are constantly re-evaluated to optimize value to the organization (Strategic Sourcing, n.d.).
Supplier Performance Management: The process of measuring, analyzing, and managing supplier performance for the purposes of reducing costs, mitigating risk, and driving continuous improvement (Aberdeen, 2002).
ix LIST OF ABBREVIATIONS
ADM: Application Development and Maintenance
BSC: Balanced Scorecard
CEB: Corporate Executive Board
CPUC: California Public Utilities Commission
EAI: Enterprise Application Integration
IT: Information Technology
KPI: Key Performance Indicator
PSC: Procurement Strategy Council
QBR: Quarterly Business Review
QL: Qualitative Metric
QN: Quantitative Metric
ROI: Return on Investment
SBR: Supplier Business Review
SLA: Service Level Agreement
SM: Supply Management
SPM: Supplier Performance Management
x Chapter 1
Introduction
1.1 Chapter Overview
This chapter provides an overview of how a Southern California utility company contracts with Information Technology (IT) service providers and manages related supplier performance. The purpose of this project is to explore the ways that Supplier
Performance Management (SPM) can help the utility company to realize the full value of its contracts with IT service providers. Significant loss of contract benefits can occur when the parties governing a relationship fail to develop comprehensive programs to effectively manage supplier performance. By collaborating with suppliers and internal stakeholders to create appropriate performance metrics, Supply Management professionals can measure and improve the operations of IT service providers.
1.2 Background
The Supply Management organization of the Southern California utility company under study is tasked with the procurement of materials and services totaling nearly $4 billion in annual spend. Total spend is divided among categories that represent the largest segments of spend across the entire company, and these categories include professional services, construction services, transportation/delivery services, information technology (IT), energy efficiency programs, office supplies, transmission/distribution materials, and so on. The IT Category of Supply Management oversees the direction of approximately 10-12% of the organization’s total annual spend, or $400-500 million.
The IT Category is responsible for the procurement of goods and services such as hardware (desktops, laptops, servers, computer peripherals), software,
1 telecommunications equipment, and related IT services (applications development and maintenance). Although the actual quantity of products and services that the organization acquires is quite large, this study is focused on developing a program for suppliers providing IT services specific to applications development and maintenance. Application
Development and Maintenance (ADM) refers to custom software development, construction of Enterprise Application Integration (EAI) systems, and maintenance of systems and sub systems. The IT Category purchases many standard, off-the-shelf software products, but ADM services are required to fulfill the unique requirements of the organization. (MindLance, n.d.)
The IT Category manages relationships with many IT service providers delivering a wide range of activities, but this study is focused on developing a SPM program for the four prime suppliers of application development and maintenance services. The IT
Category followed Strategic Sourcing procedures to identify the most qualified industry participants for IT services, and these four suppliers were awarded contracts with the utility based on the evaluations of their qualifications. Figure 1 demonstrates an interpretation of various supplier segments presented by the Procurement Strategy
Council (PSC), which categorizes suppliers by tiers depending on level of supplier spend and business impact. The suppliers selected for the project have been identified by the organization as Tier 1, or Strategic, suppliers, and according to Figure 1 represent
“relationships that are critical to long term success” and “fully integrated activities” within the organization. Additionally, these four suppliers were selected for consideration because they represent approximately 20-25% of the category’s annual spend, and therefore signify a substantial level of effort for the organization.
2 Figure 1. Supplier segmentation model. Adapted from “Supplier Segmentation Toolkit” by Procurement Strategy Council.
The Southern California utility company is regulated by the California Public
Utilities Commission (CPUC) and is accountable to its ratepayers to keep the costs of operations low. The Supply Management organization fulfills this obligation by following Strategic Sourcing practices to select suppliers and overseeing existing contracts to maintain competitive pricing and performance. Ongoing economic fluctuations and technological advances have a sizeable impact on the company and
3 require new approaches to IT performance management to be explored. (Procurement
Strategy Council, n.d.a)
Figure 2. Benefits of effective supplier management. Adapted from “Supplier relationship management” by Procurement Strategy Council.
1.3 Problem Statement
The Supply Management organization of the Southern California utility company applies vast resources to discover, qualify, and develop contracts with qualified IT service providers. Figure 2 demonstrates that Supply Management organizations spend approximately 60% of supplier-focused time on Strategic Sourcing and without proper contract management, most of the projected savings will dissolve over the term of the agreement. The company can potentially experience significant performance losses when
IT service providers are not managed effectively, as previously there has not been an established formal and uniform Supplier Performance Management (SPM) program.
Other negative consequences of overlooking SPM practices are the deterioration of
4 supplier relationships, frustration of internal clients, and devaluation of established agreements.
1.4 Research Objectives
There are several objectives of the research on Supplier Performance
Management programs to be addressed by this project study. One objective is to demonstrate that such a program will help the Southern California utility company to more effectively manage its IT service providers. Another is to identify best practices to develop and implement a SPM program that offers an approach for the IT Category to measure and improve the performance of its Tier 1 IT service providers. Finally, results of the research should associate leading SPM practices and concepts with a program that provides a comprehensive view of selected Tier 1 relationships to the managing parties, allowing them to maximize the value delivered to company.
The program development involves participants at various levels throughout the organization to ensure that the needs of all parties are represented. Continuous improvement is an important objective of the project, and steps to achieve the desired progress should be regularly documented and tracked. The IT Category managers are expected to provide consistent and reliable feedback to both internal client and supplier participants, and to higher levels of management in both organizations. An open and collaborative Supplier Performance Management program provides an opportunity for the company to achieve enhanced results from its IT service providers.
1.5 Significance of Research
The project objectives are significant for the organization because they present a method to protect valuable company resources such as time and money, and help the
5 company identify industry best practices regarding SPM programs. Results of the project should help company resources managing the relationships determine how well IT service providers are performing compared to client expectations and their competitors, and against industry standards. Additionally, the outcomes help detect how satisfied the end users of the IT services are with supplier performance, as well as areas that require immediate attention and remediation.
1.6 Research Questions
The research questions for this project are aimed at measuring the satisfaction with the performance and capabilities of IT service providers. To assist in accomplishing this objective, it is critical to develop metrics to measure the satisfaction level of the company’s internal clients and end users of the services. Direct comments and feedback from clients are essential to developing performance metrics and recognizing potential areas for growth and improvement. Some possible areas for metrics and performance development include supplier resource quality and capabilities, account management experience, and quality and cycle time of the IT services.
The following questions shall represent the ideas and intention of supplier performance management for this project and should be answered by the research. The research questions address the issues that were most critical to the program managers at the time of implementation. The concepts measured to evaluate the variables of each research question make up the related construct, and the constructs specified for each research question are included in the list below. (Zikmund, Babin, Carr, & Griffin, 2010)
Additional detail regarding the measurement and operationalization of each construct is
6 included in a Construct and Measurement Table in Appendix A and described further in
Chapter 3.
R1: How is supplier’s overall performance measured?
Construct: Overall satisfaction with supplier performance
R2: How is the quality of the supplier relationship measured?
Construct: Attitude toward relationship quality
R3: How is the quality of supplier’s resources evaluated?
Construct: Attitude toward quality of supplier’s resources
R4: How are supplier’s project management capabilities assessed?
Construct: Attitude toward supplier’s project management capabilities
R5: What is the overall satisfaction level with the supplier’s account management team?
Construct: Overall satisfaction with supplier’s account management team
R6: How is the quality of the services measured?
Construct: Attitude toward service quality
R7: How often are services delivered according to schedule?
Construct: Overall satisfaction that services are delivered according to schedule
1.7 Limitations
There are some limitations in conducting the research on Supplier Performance
Management at the Southern California utility company. The results of the survey and scorecard are only as comprehensive and current as provided by the participants in the program. The project is limited to four suppliers and results may not be representative of the performance and attitudes toward other IT service providers. Additionally, depending on the time period covering the development and implementation of the project, there
7 ma y not be enough time to reflect actual performance improvements from the selected suppliers. This project focuses on developing a program in accordance with industry best practices and the intended effect of implementing a Supplier Performance Management program.
1.8 Chapter Summary
The IT Category of the Southern California utility company is responsible for managing the performance of the IT service providers that are selected following the process of Strategic Sourcing. The company does not have a formal Supplier
Performance Management program, but there is significant potential to increase the value of supplier relationships by measuring performance and detecting areas for growth and improvement. Managing supplier performance will support the effort that related parties commit to the relationship by sharing feedback on past performance and communicating expectations and plans for future progress.
8 Chapter 2
Literature Review
2.1 Chapter Overview
An initial assessment of the IT Category’s practices revealed that there was no centralized method for managing awarded contracts and that many supplier relationships were left unattended by Supply Management after the Strategic Sourcing events had concluded. Industry research provides strong indications that enhanced operations can be achieved through the effective management of supplier performance. Therefore, it is practical and essential to scan the business environment to identify practices of the most successful companies.
Table 1
Literature Review Table
Subcategory of Literature Review Contribution to Study Section 1: Industry Research on SPM Programs 2.2.1 Industry Review Provides information about leading sources of information for the industry. 2.2.2 Benefits of SPM Programs Presents information about the advantages of implementing SPM programs. Section 2: Best Practices for Developing SPM Programs 2.3.1 SPM Best Practices Provides industry leading practices for developing SPM programs. 2.3.2 Focus on Scorecard Development Presents benefits and best practices for developing a balanced scorecard. Section 3: Maintaining Effective SPM Programs 2.4.1 Challenges of SPM Programs Provides information about challenges of managing SPM programs. 2.4.2 Focus on Continuous Improvement Presents benefits of continuous improvement for maintaining effective SPM programs.
9 2.2 Industry Research on SPM Programs
2.2.1 Industry Review
The Supply Management organization of the Southern California utility company being studied maintains memberships with both the Corporate Executive Board’s
Procurement Strategy Council and the Ariba Network. The collective knowledge and tools made available from these organizations provided valuable, relevant, and timely material for developing a SPM program. The widespread membership of these two organizations suggests that affiliates have access to the shared knowledge of potentially millions of participating companies which should be expected to represent the best practices across multiple industries.
One of the primary sources of industry research used in the development of the
SPM program is the Corporate Executive Board’s Procurement Strategy Council. The
Corporate Executive Board (CEB) was founded in 1979 as the Research Council of
Washington. Today, the CEB serves large, mid-sized and government organizations in areas including Innovation and Strategy, Procurement and Operations, and Financial
Services and its clients include over 75% of the Fortune 1000. (Procurement Strategy
Council, n.d.b)
The Procurement Strategy Council (PSC) draws on the expertise of procurement professionals from the world's leading companies and provides insights, tools, analysis, and advisory support to help procurement organizations improve performance and drive innovation. The best practices of thousands of PSC member companies are combined with advanced research methodologies to equip organizations with actionable solutions to transform their operations. (Procurement Strategy Council, n.d.b)
10 Another source for industry research is Ariba, Inc., an SAP company that has developed the Ariba Network, which is a business commerce network designed to allow companies to collaborate in a global community for buying, selling, and managing cash.
The Ariba Network helps organizations of all sizes connect with trading partners using a cloud platform, which provides internet users from over 190 countries globally access to the network’s shared application software and databases. Network members can connect and collaborate with nearly 2 million potential partners to lower their costs and risks, enhance revenues, and manage cash and working capital. Its users benefit from information contributed by experienced partners in the areas of compliance, risk, and performance and enhanced market visibility and intelligence allows users to evaluate changing market conditions for greater business decisions and competitive advantage.
(Ariba, n.d.)
2.2.2 Benefits of SPM Programs
In a progressively global economy, the success of most companies is becoming increasingly dependent on a well-managed Supply Management organization. Ariba, Inc. emphasizes that many companies are challenged to measure and enhance the performance of their suppliers, and therefore miss opportunities to increase profits and proactively mitigate risks. Company experts advocate that an appropriately designed
SPM solution can help organizations define, measure, and enhance supplier performance to meet business goals and potentially free up resources to perform other critical business functions. (Ariba, 2008)
Formal SPM programs are associated with supplier performance improvement results estimated around 20 percent according to The Supplier Performance Measurement
11 Benchmark Report, published by the Aberdeen Group in 2002. Organizations that solely implement SPM programs by focusing on strategic or critical suppliers can identify and target performance issues that are most impactful and imperative to the success of its operations. The report cautions that organizations that fail to measure and manage their largest suppliers likely expose themselves to quality and service deficiencies, as well as cost overruns that reduce profits and any value obtained from strategic sourcing practices.
(Aberdeen, 2002)
Supply Management organizations have historically focused SPM practices on suppliers of manufactured goods, even as the services sector has become an increasingly important factor in the global economy. The general perception that service performance cannot be measured as precisely as manufacturing specifications has reinforced the notion that traditional SPM practices cannot be applied effectively to service providers.
Due to the growing significance of and spend with service providers, Supply
Management experts have concluded that SPM practices are necessary for “improvement, cost control and minimization of value leakage” in organizations to maintain a competitive advantage. (Ellram, Tate, & Billington, 2004)
The process of supplier evaluation is defined as “quantifying the efficiency and effectiveness of supplier action” and is designed to influence decision-making within a buying organization while encouraging positive behaviors by supplying companies.
Results of studies conducted on supplier evaluation practices indicate that if attempts to influence positive behaviors are successful, they can result in improved supplier performance and capabilities and behavior in better alignment with the objectives of the buying organization. Supplier evaluation is a critical element of an effective SPM
12 program and can serve as an effective instrument to influence supplier performance.
(Hald and Ellegaard, 2011)
Supplier performance management is a natural extension of the purchasing function and when implemented carefully, can provide clear value to an organization.
However, calculating actual return on investment (ROI) of SPM programs is often difficult. One approach to making a business case for supplier performance management is to estimate the costs associated with poor quality and late delivery that might be avoided. This estimate can provide managers with useful information of the potential value of implementing SPM programs, which includes greater supply quality, lower costs and reduced supply risk. (Hebrand, 2013)
2.3 Best Practices for Developing SPM Programs
2.3.1 SPM Best Practices
Industry best practices show that a formalized system to evaluate supplier performance is critical to maintaining effective operations. One of the common recommendations for developing a successful and collaborative SPM program is open communication and data sharing among relevant parties. Performance targets and metrics developed jointly with suppliers and internal clients have greater probability for successful application than those imposed by Supply Management organizations.
(eSourcing Forum, 2006)
A common practice is strategic supplier selection, or concentrating significant purchase activities with a few suppliers to enable effective management of contracts and quality improvement initiatives. Upon selection of strategic suppliers, it is important to circulate scorecards to internal clients regularly to ensure that the appropriate metrics are
13 being tracked and evaluated. The scorecards should be reviewed often to ensure that the metrics represent the requirements of the relationship and supplier performance issues are resolved quickly. (eSourcing Forum, 2006)
SPM is a collection of processes that allows organizations to achieve business goals by defining, measuring, and collaboratively increasing the performance of key suppliers.
The formation of an effective SPM program will typically vary by organization and should generally include the following steps:
1. Establish a SPM strategy in alignment with corporate objectives
2. Develop uniform supplier performance criteria and expectations
3. Select evaluation tools and process steps to collect, monitor and measure
performance and communicate plan to both the stakeholder and supplier
communities
4. Collect supplier performance data (qualitative and quantitative) and establish time
period associated with data collection
5. Review data against criteria and share results with suppliers and stakeholders
6. Set improvement goals and plans to influence supplier behavior
7. Review and recalibrate goals, strategy and metrics as business grows and changes
(Gordon, 2008)
Further, Supply Management professionals are encouraged to establish supplier performance measures because:
x Companies can’t manage what they don’t measure
x If companies measure suppliers, they will improve
14 x Companies can uncover and remove hidden waste and cost drivers in the supply
chain
x Companies can facilitate supplier performance improvement
x Companies can increase competitiveness by shrinking order cycle times and
inventory levels
x Companies can make informed business decisions that impact the enterprise
(Gordon, 2005)
2.3.2 Focus on Scorecard Development
The Balanced Scorecard (BSC) was first introduced to the business community by
Robert Kaplan and David Norton in 1992 to provide companies with a comprehensive structure to align their strategic objectives with a clear set of performance measures. The
BSC was designed to be customized to fit an organization’s mission, strategy, technology, and culture, and not intended to be viewed as a general template to accommodate all businesses and industries. (Kaplan and Norton, 1993)
The Balanced Scorecard is well suited to organizations seeking to create specific measures to express performance, quality, time, and service goals. Additionally, many key organizational initiatives such as customer-supplier partnerships, cross-functional integration, global scale, continuous improvement, and team accountability can be addressed with the BSC approach to performance management. The BSC is intended to facilitate strategy development, implementation, and communication, and should deliver dependable results for management control and performance evaluation when constructed properly. (Kaplan and Norton, 1992)
15 Organizations benefit from implementation of the Balanced Scorecard if they can successfully integrate all of their seemingly contrasting and competing objectives with the tool. The BSC forces managers to review and consider the combined results of all operational measures, and protects against the promotion of certain objectives at the detriment of less favored goals. However, the scorecard criteria should be reviewed and updated regularly to avoid representing obsolete objectives. A potential disadvantage of the BSC is that it does not validate the relevance or applicability of an organization’s strategy, it only translates the strategy into measurable objectives. (Kaplan and Norton,
1992)
The effectiveness of the Balanced Scorecard is examined in one study of a
Fortune 500 manufacturing firm’s evolution of a BSC over a 15-year period. The results of the study strongly suggest that the BSC has prevailed due to its use as a learning and communication tool and its ability to influence organizational behavior. The following list of considerations is recommended for developing and maintaining a successful scorecard:
x Make the development process collaborative among the key stakeholders and
establish clear processes for making changes to the scorecard. This creates buy-
in, makes expectations explicit, and eliminates surprises.
x Communication is critical. Whether top-down, bottom-up, or among evaluatees,
communication builds trust in the scorecard process and fosters legitimacy and
information sharing.
x Create a balance of objectivity and subjectivity in scorecard design and use.
Objective performance measurement is the unachievable “holy grail” and
16 subjectivity in measurement and evaluation of performance helps overcome its
inherent deficiencies.
x Embrace the evergreen nature of the scorecard. Stability in its purpose and use,
coupled with flexibility in its design may be critical to success.
x Delay using the scorecard extensively for compensation. As experience with the
scorecard is gained, trust is established and scorecard integration into the
compensation system may be seamless.
x Periodically review the scorecard for its effectiveness in promoting desired
behaviors and financial results. When necessary, make changes to its design or
use.
(Malina, 2013, pg. 911)
The Balanced Scorecard can be applied to support many of the primary goals of a
Supply Management organization and can be modified to establish a framework for performance measurement that is consistent with specific organizational objectives.
Figure 3 demonstrates an association between a Supply Management framework and the
Balanced Scorecard. This view was originally intended to explain how a BSC can be used to measure the performance of a Supply Management organization, but also provides for the study an example of the applicability to various Supply Management performance management goals. The integration of diverse perspectives provides visibility to performance results to managers across an organization’s functional borders.
(Brewer and Speh, 2000)
17 Supply Management Balanced Scorecard
SM Goals • Waste reduction Business Process • Time compression • Flexible response Perspective • Unit cost reduction
Customer Benefits • Improved product/service quality Customer • Improved timeliness Perspective • Improved value • Unit cost reduction
Financial Benefits • Higher profit margins • Improved cash flows Financial • Revenue growth Perspective • Higher return on assets
SM Improvement • Product/process innovation Innovation and • Partnership management • Information flows Learning Perspective • Threats/substitutes
Figure 3. Linking the supply management framework to the balanced scorecard.
Adapted from “Using the balanced scorecard to measure supply chain performance,” by
P.C. Brewer and T.W. Speh, 2000, Journal of Business Logistics, 21, pg. 85.
A study designed to develop a BSC for a Supply Management organization further investigated the significance of the diverse perspectives represented by an effective scorecard. Following is the description and significance of each of the four perspectives presented in Figure 3:
Business Process Perspective: establish core competencies and processes at
which companies must excel and that have greatest impact on customer
satisfaction
18 Customer Perspective: translate customer service strategy into specific measures
that matter and reflect most value to customers
Financial Perspective: indicate whether the strategy, implementation and
execution are contributing to a company’s bottom line improvement
Innovation and Learning Perspective: determine ability to innovate, improve
and learn to create continuous value for company
Recommendations of the study indicate that each of the perspectives should reflect strategic objectives through corresponding metrics, and should be regularly reviewed and updated as needed. (Bhagwat and Sharma, 2007)
Table 2 presents an overview and recommendations for SPM scorecard development based on best practices reviewed and documented by the PSC.
Table 2
Scorecard Development Best Practices
Principles Benefits Create category-based scorecards Enables analysis of similar suppliers in an apples-to-apples fashion Involve internal business partners before Demonstrates how scorecard focuses on launching the scorecard elements business partners care about (i.e., metrics beyond cost) Focus on the “critical few” KPIs Maintains focus on the most important (approximately three to five per metric) drivers of supplier performance Select KPIs that are within the Measures outcomes that can be influenced organization’s control by the organization and/or the supplier Use a mix of both quantitative and Contributes to a fact-based scorecard qualitative KPIs while allowing for day-to-day user perception Incorporate existing KPIs and/or SLAs Encourages adherence to existing contract where appropriate terms and conditions Automate data collection, aggregation, Creates a scalable performance and reporting management process
Source: Procurement Strategy Council, 2012.
19 2.4 Maintaining Effective SPM Programs
2.4.1 C hallenges of SPM Programs
There are many potential hazards for organizations when implementing SPM programs. One of the common problems that program administrators encounter is a shortage of managers to oversee supplier performance. Other drawbacks include a lack of involvement and/or commitment from suppliers and other groups within an organization. (Hebrand, 2013)
The failure to implement a joint set of processes across different business units within an organization is another potential difficulty of SPM programs. Program managers can align SPM processes to manage suppliers that work with multiple business units and suppliers within the same segment. Benefits of aligning SPM processes include the ability to measure the performance of one supplier against another, shared training and communication, and process efficiency. (Hebrand, 2013)
2.4.2 Focus on Continuous Improvement
The drive toward continuous improvement is a strategic initiative of most modern companies, but practical application can vary widely depending on variables such as size, position, or industry of an organization. As continuous improvement measures are motivated by the desire to reduce expenses and enhance profitability, common considerations are improved quality, reduced cycle time, and regular process evaluation.
The failure to assess the results of operational improvements can cause poor financial outcomes if excess capacity created by quality or cycle time developments are not used to eliminate expenses or generate revenue. (Kaplan and Norton, 1992)
20 The primary objective of SPM is to improve the performance of both the supplier and the purchasing organization. The development of supplier performance metrics can help organizations ensure that suppliers adhere to contract agreements and market standards. The five key areas that should be considered when establishing SPM programs are supplier segmentation, management, scorecard, development, and evaluation. (Hebrand, 2013)
Supplier segmentation refers to the level of effort that SPM programs should expend in engaging suppliers of various segments. Supplier management suggests that teams should be established to structure the supplier relationships with meetings and definition of roles and responsibilities. A supplier scorecard should identify critical success factors, define key performance indicators (KPIs), and develop processes for gathering data. Supplier development refers to the process created to foster the growth and continuous improvement of supplier partners. Supplier evaluation describes the method for using supplier performance data in assessing suppliers for potential contracts.
(Hebrand, 2013)
Continuous improvement practices are in alignment with all primary initiatives of a Supply Management organization, which is constantly seeking solutions to reduce waste, compress time, respond flexibly, and reduce unit costs. Some of the regular practices that successful Supply Management organizations employ to achieve their objectives are:
• redesign of products or processes to enhance value
• partner with other organizations to leverage the human knowledge base
• improve the flow of accurate, timely, and supportive information
21 • monitor the external marketplace for potential threats and/or substitutes
An organization must consistently evaluate its methods and pace for improvement to ensure that its practices remain competitive within the industry. (Brewer and Speh, 2000)
2.5 Research Results
After several years of lacking a formal program to manage supplier performance, the IT Category of the Southern California utility company has gathered information to support the efforts to develop, implement and maintain a SPM program. The research on
SPM programs suggests that there is significant value to be retained and further achieved through their strategic development and implementation. The Supply Management group is optimistic that SPM practices will help the company retain the value gained by its
Strategic Sourcing efforts and deliver increased performance to the company’s IT service clients.
22 C hapter 3
Methodology
3.1 Chapter Overview
After completing the research to determine the Southern California utility company’s requirements for a Supplier Performance Management program, the IT
Category identified some industry best practices to develop and implement a program to manage the performance of its four largest IT service providers. This chapter includes details regarding the research design, sample size, and procedures used to administer a program survey, as well as an overview of the methods used to analyze the resulting data.
This project outlines the required processes and participants of a SPM program designed to deliver an effective tool for suppliers, internal clients and Supply Management to derive added value from the relationships.
3.2 Research Design
Research conducted on the design and implementation of successful SPM programs indicated that a thoughtfully developed scorecard allows organizations to measure and report appropriate performance management metrics. Based on the industry best practices identified for scorecard development, the program included both quantitative and qualitative metrics for survey. The required responses were limited to five (5) quantitative and five (5) qualitative metrics to simplify the process for program participants. Two separate groups were selected to participate in the program; one group included the internal client end users of the IT services that received the qualitative survey and the other group included a small group of IT services contract managers responsible for completing the quantitative scorecard. The following queries represent
23 the requirements of the qualitative survey and the quantitative scorecard (“QL” indicates qualitative metrics and “QN” indicates quantitative metrics).
3.2.1 Qualitative Survey
Scoring Scale:
3 = Supplier Exceeds Expectations
2 = Supplier Meets Expectations
1 = Supplier Does Not Meet (or Partially Meets) Expectations
N = Not Applicable/Don’t Know
QL1: Supplier consistently demonstrates a strong commitment to the relationship by being responsive, flexible, and easy to work with.
QL2: Supplier provides high caliber resources who consistently demonstrate a solid understanding of the business.
QL3: Supplier exhibits strong project management competency and highly effective practices around communications, cost, schedule and scope management, accuracy and timeliness in invoicing/reporting while being fair and transparent in pricing, contracts and agreed upon Service Level Agreements (SLAs).
QL4: Supplier’s account management team is always responsive, accessible, accountable, and collaborative.
QL5: Please rate your overall satisfaction with supplier (1 = Not Satisfied, 2 = Satisfied,
3 = Very Satisfied). Please provide a brief assessment of supplier’s strengths and weaknesses, along with any recommendation(s) you may have for the supplier to improve.
24 3.2.2 Quantitative Scorecard
Related to Application Maintenance:
QN1: Percent of implementations completed on time
QN2: Percent of defect-free implementations
Related to Application Development:
QN3: (Number of code defects)/($10K in application development work)
QN4: Number of projects 10% behind schedule
QN5: Number of projects 10% over budget
3.3 Survey and Scorecard Design
The qualitative survey was distributed to internal client end users of the IT services by email and included five questions containing a satisfaction rating scale. A 3- point Likert-type scale was used for each qualitative response to determine the level of satisfaction with a supplier and whether or not client expectations were met. The quantitative scorecard distributed to IT service contract managers included five criteria requiring calculation of actual quarterly performance metrics. A ratio scale was used to compare actual scorecard data with established service level requirements to determine supplier’s performance against contractual commitments. Appendix B includes a sample of the qualitative survey format used for distribution and response and Appendix C displays a sample subset of the data requested to calculate the quantitative scorecard.
3.4 Sampling
For the initial qualitative survey, 178 participants within the IT organization were surveyed and 104 responses were received (58%), but only 84 participants completed the survey for a total response rate of 47%. The quantitative scorecard was distributed to the
25 company’s IT group responsible for managing the SLAs of IT services contracts, and only one scorecard was required and returned from the group for a total response rate of
100%. The IT group had been tracking the IT service providers’ performance based on pre-established agreements in the contracts, but was not in the practice of reporting this information back to the suppliers in previous periods.
3.5 Procedures
Following each quarter, the Supply Management group compiled completed survey and scorecard response data and calculated results for each IT service provider participating in the SPM program. All scores were populated and totaled in the supplier scorecards, including client comments provided on the qualitative survey, as represented in the examples in Appendices D through G. The completed supplier scorecard and summary information was provided to each client manager for final review and approval prior to quarterly presentation to IT stakeholders and supplier leadership teams.
The IT Category designed the SPM program to clearly communicate performance expectations and client feedback to suppliers to facilitate shared understanding of the program’s progress. Participants in the program regularly include Category managers, business unit stakeholders, and supplier representatives. Those responsible for conducting the program are required to familiarize suppliers with the performance metrics and reporting system, provide suppliers with visibility into assessment guidelines to foster awareness about the metrics, and communicate results and rankings relative to other suppliers. The goal of the reviews is to provide actionable supplier information and to improve supplier performance. (Procurement Strategy Council, n.d.a)
26 Figure 4. Scorecard and quarterly business review process and guidelines. Adapted from
Southern California utility company internal process documents.
Figure 4 outlines the quarterly business review (QBR) process and guidelines followed by those implementing the SPM program. The IT Category conducted QBRs with its internal stakeholders to review the results of the surveys and scorecards.
Conducting the internal QBRs provided opportunities for the IT Category managers to deliver performance and progress updates and present any issues requiring visibility at the executive level. Following the QBRs, the IT Category conducted Supplier Business
Reviews (SBR) with each supplier to review quarterly performance, provide feedback, and discuss areas for improvement.
3.6 Data Analysis
The results of each quarterly survey and scorecard were entered into Microsoft
Excel for analysis and validation. The associated construct and measurement table is included in Appendix A to demonstrate how the quantitative and qualitative responses
27 operationalize the construct of each research question. The table specifies the construct definition of each research question and the corresponding survey or scorecard responses referenced to operationalize the constructs.
The qualitative survey responses were evaluated using descriptive statistics to determine the percentage of respondents who were either satisfied or very satisfied versus not satisfied with each category of performance. Similarly, the quantitative scorecard responses were also evaluated using descriptive statistics to calculate the rate at which the supplier’s overall quarterly performance had met the established contract criteria and
SLAs. A preliminary review of the analysis results suggest that the program criteria can be reasonably measured and logically presented in association with the contract requirements and expectations.
3.7 Anticipated Results
The IT Category of the Southern California utility company has been able to determine from the responses of its internal clients and contract managers which of the IT service providers are meeting client expectations and/or complying with SLAs. The
QBRs and SBRs have provided sufficient opportunity for the IT Category managers to meet with internal stakeholders and supplier representatives to ensure that actual performance is aligned with organizational goals. The SPM program has facilitated an environment of open communication and feedback between internal end users and service providers that is often undermanaged in large organizations.
3.8 Chapter Summary
The methods that have been used to implement the SPM program at the Southern
California utility company have been developed by combining key internal business
28 requirements with industry best practices research. Responses to the qualitative survey and quantitative scorecard have provided the IT Category with relevant data necessary to determine the satisfaction level with IT service providers and the efficiency of supplier relationships. The quarterly results indicate enhanced value for the organization resulting from proactively managing service contracts and encouraging favorable competition among the IT service suppliers.
29 Chapter 4
Research Findings
4.1 Chapter Overview
This chapter examines the data gathered from the surveys and scorecards distributed to end users of IT service providers at the Southern California utility company. The tracking, evaluation, and analysis of the submitted data was conducted by the IT Category primarily through the use of Microsoft Excel software. The scorecards for the four IT service providers presented in Appendices D through G and the scorecard summary in Appendix H provide the data required to address the objectives of the
Supplier Performance Management program. The findings from the initial quarterly review in the first quarter of 2012 relative to each of the research questions are presented in the following sections.
4.2 Research Question 1
The first research question, “How is supplier’s overall performance measured?” is related to the associated construct of “overall satisfaction with supplier performance.”
The construct is addressed in the responses to the fifth qualitative survey question, “QL5:
Please rate your overall satisfaction with supplier (1 = Not Satisfied, 2 = Satisfied, 3 =
Very Satisfied). Please provide a brief assessment of supplier’s strengths and weaknesses, along with any recommendation(s) you may have for the supplier to improve.”
The results from the survey for this first research question show that on average, respondents were generally satisfied or very satisfied with the four suppliers included in in the first round of SPM. Suppliers A and D scored 91%, Supplier C scored 88% and
30 Supplier B scored 83% for overall satisfaction level. The information that may be more revealing is the percentage of respondents that were not satisfied with certain suppliers.
For example, Supplier B received an overall score of 16% not satisfied, and it will be useful for the IT Category and executive sponsors to understand the reasons for the dissatisfaction. Appendix I provides a summary of the major issues identified from the survey, and while it does not include specific remarks or comments, the table includes the top 3 strengths and weaknesses for each supplier.
4.3 Research Question 2
The second research question, “How is the quality of the supplier relationship measured?” is related to the associated construct of “attitude toward relationship quality.”
The construct is addressed in the responses to the first qualitative survey question, “QL1:
Supplier consistently demonstrates a strong commitment to the relationship by being responsive, flexible, and easy to work with.”
The results from the survey show that the respondents mostly felt that the suppliers met or exceeded their expectations in relation to the quality of the relationship and commitment of the suppliers. Suppliers A and D again scored the highest at 94%,
Supplier C had a score of 85% and Supplier B again had the lowest of the four with 84%.
4.4 Research Question 3
The third research question, “How is the quality of supplier’s resources evaluated?” is related to the associated construct of “attitude toward quality of supplier’s resources.” The construct is addressed in the responses to the second qualitative survey question, “QL2: Supplier provides high caliber resources who consistently demonstrate a solid understanding of my business.”
31 The results from the survey show that the respondents generally felt that the suppliers met or exceeded their expectations in regards to the quality of resources provided by the supplier. Supplier C scored the highest at 93%, Suppliers A and D each had a score of 90%, and Supplier D scored the lowest at 85%.
4.5 Research Question 4
The fourth research question, “How are supplier’s project management capabilities assessed?” is related to the associated construct of “attitude toward supplier’s project management capabilities.” The construct is addressed in the responses to the third qualitative survey question, “QL3: Supplier exhibits strong project management competency and highly effective practices around communications, cost, schedule and scope management, accuracy and timeliness in invoicing/reporting while being fair and transparent in pricing, contracts and agreed upon Service Level Agreements (SLAs).”
The results from the survey show that the respondents largely felt that the suppliers met or exceeded their expectations in relation to their project management capabilities. Suppliers A and D each received scores of 92%, while Suppliers B and D both scored 88%. The average score among the four suppliers was relatively high, which indicates that the sampled suppliers tend to perform well in the project management activities.
4.6 Research Question 5
The fifth research question, “What is the overall satisfaction level with the supplier’s account management team?” is related to the associated construct of “overall satisfaction with supplier’s account management team.” The construct is addressed in the
32 responses to the fourth qualitative survey question, “QL4: Supplier’s account
management team is always responsive, accessible, accountable, and collaborative.”
The results from the survey for this question show that the respondents were
generally satisfied with the supplier’s account management team, although the overall
scores for this category were lower on average. Suppliers A and D both received scores
of 88% and Suppliers B and C each scored 84%. The lower average scores among the
four suppliers indicate that account management is an area where the suppliers tend not to
perform well, but implementation of the SPM program should provide some necessary
improvement to resolve this issue.
4.7 Research Question 6
The sixth research question, “How is the quality of the services measured?” is
related to the associated construct of “attitude toward service quality.” The construct is
addressed in the responses to the second and third quantitative scorecard questions,
“QN2: Percent of defect-free implementations” and “QN3: (Number of code defects)/($10K in application development work).”
The data presented in the supplier scorecards in Appendices D through G serves to determine whether each of the suppliers is meeting predetermined, contractual goals regarding service quality. All four suppliers scored above the goal of 95% for defect-free implementations, which suggests that this is not currently an area of great concern for the resources managing the IT Service suppliers. Although 3 of the suppliers met the goal for number of code defects per $10K in application development work, Supplier D fell very short of the target and this suggests a focus for the performance review as well as an opportunity for supplier improvement.
33 4.8 Research Question 7
The seventh research question “How often are the services delivered according to schedule?” is related to the associated construct of “overall satisfaction that services are delivered according to schedule.” The construct is addressed in the responses to the first, fourth, and fifth quantitative survey questions, “QN1: Percent of implementations completed on time”, “QN4: Number of projects 10% behind schedule”, and “QN5:
Number of projects 10% over budget.”
The data presented in the supplier scorecards regarding the overall satisfaction that suppliers are delivering services according to schedule indicates that performance in this area is very high. All four suppliers exceeded the goal of 95% of implementations completed on time, and all four suppliers met the goals of 0 projects 10% behind schedule or 10% over budget. These results imply that the suppliers are strong in this area of performance and that each works hard to meet the established objectives of the contract.
4.9 Chapter Summary
The data offered in the survey and scorecard responses has provided the IT
Category with useful preliminary information regarding specific performance measures of the company’s prime IT service suppliers. The analysis presented in this chapter suggests that the research questions are directionally aligned with the survey and scorecard responses solicited as part of the SPM program. Additionally, these questions address service provider performance results that are aligned with and most critical to the
IT organization’s objectives. There appears to be significant material from the initial
34 SPM implementation to present management with meaningful results and to effectively plan for future quarters.
35 Chapter 5
Recommendation and Conclusion
5.1 Chapter Overview
This chapter presents a summary of the research conducted and data reviewed and analyzed in support of the research objectives set forth in Chapter 1. The purpose of this project was to explore the ways that a SPM program can help the Southern California utility company to realize the full value of its contracts with IT service providers and to develop and implement a program in alignment with the best practices identified from the research. The methods applied to design, implement and evaluate the SPM program were determined by research focused on industry analysis and best practices for developing and maintaining effective supplier performance.
5.2 Research Summary
The industry research on SPM programs promote the value that companies can realize with thoughtful consideration and implementation. The Southern California utility company studied in this project has invested extensive resources to establish contracts with strategic suppliers, and the benefits of a SPM program promise to help retain some of the value typically lost when supplier relationships are not managed effectively. The implementation of the SPM program is a time-consuming and resource- intensive process, so the aim of this project was to pilot the study with four prime IT service suppliers for a period before extending the program to additional suppliers.
5.3 Project Summary
Chapter 1 identified the problem statement as the company’s potential to experience a significant decline in service provider performance when suppliers are not
36 managed effectively. The lack of a formal SPM program was associated with poor supplier relationships, frustrated business clients, and diminishing value from established agreements. The research objectives to be addressed by the study were to demonstrate that a SPM program would help the company to more effectively manage its IT suppliers, to identify best practices to develop and implement such a program, and to associate leading SPM practices best aligned with the organization’s performance management objectives.
The literature review in Chapter 2 provided strong evidence that enhanced operations could be achieved through effective management of supplier performance.
The first section of the literature review offered leading industry sources of information for Supply Management and presented supporting evidence of the benefits of SPM programs. The second section discussed industry leading methods for developing SPM programs as well as best practices for building a balanced scorecard to support such a program. The third section examined the challenges of maintaining effective SPM programs and presented the benefits of promoting continuous improvement techniques to preserve value within the program. The research conducted suggests that there is significant value to be retained and further achieved through the strategic development and implementation of SPM programs.
Chapter 3 presents details regarding the research design, sample size, and procedures used to administer a program survey, as well as an overview of the methods used to analyze the resulting data. Research results indicated that successful SPM programs leveraged thoughtfully developed scorecards to allow organizations to measure and report appropriate performance management metrics, and it was determined that the
37 project s hould include both quantitativequalitative and metrics for survey. Two groups were selected to participate in the program; one group of internal client end users was selected to respond to the qualitative survey and the other group included IT services contract managers responsible for completing the quantitative scorecard.
The collection of data was achieved with 84 responses to the qualitative survey and one response to the quantitative scorecard. The qualitative data was analyzed using
Microsoft Excel to perform descriptive statistics analysis to validate the satisfaction of respondents with the service providers’ performance. The quantitative scorecard responses were also evaluated using descriptive statistics to calculate the rate at which the supplier’s overall quarterly performance had met the established contract criteria and
SLAs.
Chapter 4 examined the data gathered from the surveys and scorecards to determine whether the results adequately addressed the objectives of establishing a SPM program. Results of the analysis were generally as expected, and service providers scored high in most capacities including resource caliber, project management, projects on schedule and within budget, and on-time and defect-free implementations. Service providers scored lower overall in areas of relationship quality and account management, but still within acceptable ranges of performance, and implementation of the SPM program should provide some necessary improvement to resolve the deficiencies. The analysis suggests that the research questions are directionally aligned with the survey and scorecard responses and that these questions address service provider performance results that are aligned with and most critical to the IT organization’s objectives.
38 5.4 Implications for Management
The information presented by this project is useful for management because work performed by the suppliers chosen for this study encompasses approximately 20-25% of the annual spend of the IT Category. A combination of a large volume of work and a high percentage of total spend makes this group ideal for introducing the practice of SPM across the organization. The initial quarterly results of SPM indicate areas for improvement and areas of solid performance for each of the four suppliers. The computed scores allow the company’s managers to quickly identify suppliers that perform best in each category and provide for future work to be awarded to the most qualified candidates. An important implication is that management can more easily identify areas of concern with a supplier’s performance to present in quarterly business reviews.
5.5 Recommendations for Management
Although the results seem to indicate that SPM is a valuable practice, it is critical to remember that it must be properly managed to be effective. Based on the lower scores observed in the relationship quality and account management areas, it is recommended that the program participants should focus attention on these areas during initial quarterly review meetings. Another recommendation that should be considered is consolidating suppliers by commodity for more efficient analysis. This allows company resources to evaluate common suppliers using similar efforts and provides a simple and convenient way to compare like suppliers outside of a formal competitive process. The organization should centralize the operation of the SPM program to a concentrated group of resources.
SPM programs are costly and time-consuming to establish and implement, and those
39 resources best prepared to execute the analysis should assume the work. A final recommendation is to continually evaluate the effectiveness of the SPM programs to identify areas of improvement. If multiple quarters of data demonstrate little to no variation or progress, there is probably a need to re-evaluate the program practices.
5.6 Conclusion
The SPM program was eagerly anticipated by the IT Category at its inception following the first quarter of 2012. Several IT executives and managers were anxious to establish a program to evaluate the performance of suppliers that were receiving such a significant percentage of the annual budget. The industry research showed that cost savings and value retention were potential benefits that would eventually offset the expense of implementing a SPM program.
After four quarters of evaluation, the Chief Information Officer and the Vice
President of Supply Management jointly determined that there were diminishing returns provided by the program. While the effort has been put on hold for IT service suppliers, the work was extended to include other categories of suppliers, such as telecommunications providers and IT hardware and infrastructure resellers. The final conclusion as to the effectiveness of the SPM programs is that a company can gain value through the implementation of a carefully planned and executed platform, and continuous improvement is a key component of long-term success.
40 REFERENCES
Aberdeen Group. (2002). The supplier performance measurement benchmarking report.
Boston, MA: Minahan, T.A. and Vigoroso, M.W.
Ariba, Inc. (n.d.). The Ariba network. Retrieved from http://www.ariba.com/about/the-
ariba-network
Ariba, Inc. (2008). Ariba offers tips for effectively managing suppliers. Retrieved from
http://www.businesswire.com/news/home/20080313005001/en/Ariba-Offers-
Tips-Effectively-Managing-Suppliers
Bhagwat, R. and Sharma, M.K. (2007). Performance measurement of supply chain
management: A balanced scorecard approach. Computers & Industrial
Engineering, 53, 43-62.
Brewer, P.C. and Speh, T.W. (2000). Using the balanced scorecard to measure supply
chain performance. Journal of Business Logistics, 21 (1), 75-93.
Brown, C.M. (2010, December 30). 7 tips for rating and evaluating your suppliers and
vendors. Retrieved from http://www.inc.com/guides/2010/12/7-tips-to-rate-and-
evaluate-your-suppliers-and-vendors.html
Ellram, L.M., Tate, W.L. & Billington, C. (2004). Understanding and managing the
services supply chain. The Journal of Supply Chain Management, Fall 2004, 17-
32. eSourcing Forum. (2006, July 30). Supplier performance management III: Best practices.
Retrieved from h ttpp://www.esourcingforum.com/archives/2006/07/30/supplier-
performance-management-iii-best-practices/
Gordon, S.R. (2005). Seven steps to measure supplier performance. Quality Progress, 38
41 (8), 20-25.
Gordon, S.R. (2008). Supplier evaluation and performance excellence: A guide to
meaningful metrics and successful results. Fort Lauderdale, FL: J. Ross
Publishing.
Hald, K.S. and Ellegaard, C. (2011). Supplier evaluation processes: the shaping and
reshaping of supplier performance. International Journal of Operations &
Production Management, 31 (8), 888-910.10.
Hebrand, V. (2013, September 6). Utilizing the true potential of supplier performance
management. Retrieved from http://www.capgemini.com/blog/procurement-
transformation-blog-capgemini/2012/08/utilizing-the-true-potential-of-supplier-
performance-management
Information Technology. (n.d.). Merriam-Webster.com. Retrieved from
https://www.merriam-webster.com/dictionary/information%20technology
Kaplan, R.S. and Norton, D.P. (1992). The balanced scorecard – Measures that drive
performance. Harvard Business Review, January-February, 71-79.
Kaplan, R.S. and Norton, D.P. (1993). Putting the balanced scorecard to work. Harvard
Business Review, September-October, 134-147.
Key Performance Indicator. (n.d.). Investopedia.com. Retrieved from
http://www.investopedia.com/terms/k/kpi.asp#ixzz4lvLZKfDe
Malina, M.A (2013). The evolution of a balanced scorecard. The Journal of Applied
Business Research, 29 (3), 901-912.
MindLance, Inc. (n.d.). Outsourced application development & maintenance. Retrieved
from http://www.mindlance.com/documents/application_management_wp.pdf
42 Procurement Strategy Council. (n.d.a). Benefits of effective supplier management.
Retrieved from
https://psc.executiveboard.com/Members/Topics/Abstract.aspx?cid=100246742&
s=dd&utm_source=psc.executiveboard.com&utm_medium=home&utm_campaig
n=TopicsList&utm_content=title_Supplier_Relationship_Management&utm_ter
m=cid_100246742
Procurement Strategy Council. (n.d.b). Who we are. Retrieved from
https://psc.executiveboard.com/Members/About/Who-We-Are.aspx
Procurement Strategy Council. (2012). Scorecard development best practices
(PSC5896216SYN). Retrieved from
https://www.cebglobal.com/member/finance/research/report/12/scorecard-
development-best-practices.html
Procurement Strategy Council. (2016). Compendium of supplier segmentation models
(PSC2499912SYN). Retrieved from
https://www.cebglobal.com/member/finance/research/report/15/compendium-of-
supplier-segmentation-models.html
Return on Investment. (n.d.). Entrepreneur.com. Retrieved from
https://www.entrepreneur.com/encyclopedia/return-on-investment-roi
Rizza, M.N. (2006). Supplier performance management can make a difference. Supply
Chain Management Review, 10 (7), 13.
Service Level Agreement. (n.d.). CIO.com. Retrieved from
http://www.cio.com/article/2438284/outsourcing/outsourcing-sla-definitions-and-
solutions.html
43 Southern California Edison Company. (2012). Supplier performance management
scorecard and QBR process. Unpublished internal document.
Strategic Sourcing. (n.d.). BusinessDictionary.com. Retrieved from
http://www.businessdictionary.com/definition/strategic-sourcing.html
Zikmund, W.G., Babin, B.J., Carr, J.C., & Griffin, M. (2010). Business research methods
(8th ed.). Mason, OH: South-Western Cengage Learninng.
44 APPENDIX A: CONSTRUCT AND MEASUREMENT TABLE
Construct Definition Operationalization Source of Types of Scale Research Type of of the Construct the Questions Variables/Statistics Measure Overall satisfaction QL5 Made Interval scale; 3 point Likert-type scale R1 Descriptive Statistics with supplier (1 = not satisfied & 3 = very satisfied) Attitude toward QL1 Made Interval scale; 3 point Likert-type scale R2 Descriptive Statistics quality of relationship with not applicable/don’t know option (1 = supplier does not meet expectations & 3 = supplier exceeds expectations)
Attitude toward QL2 Made Interval scale; 3 point Likert-type scale R3 Descriptive Statistics quality of supplier’s with not applicable/don’t know option resources (1 = supplier does not meet expectations & 3 = supplier exceeds expectations) 45 Attitude toward QL3 Made Interval scale; 3 point Likert-type scale R4 Descriptive Statistics supplier’s project with not applicable/don’t know option management (1 = supplier does not meet expectations capabilities & 3 = supplier exceeds expectations) Attitude toward QL4 Made Interval scale; 3 point Likert-type scale R5 Descriptive Statistics supplier’s account with not applicable/don’t know option management team (1 = supplier does not meet expectations & 3 = supplier exceeds expectations) Quality of application QN2 Made Ratio scale; actual scorecard data used R6 Descriptive Statistics maintenance and QN3 development
Cycle time of QN1 Made Ratio scale; actual scorecard data used R7 Descriptive Statistics application QN4 maintenance and QN5 development
SURVEY
QUALITATIVE QUALITATIVE
APPENDIX B: APPENDIX Source: Southern California Edison Company, California 2012. Edison Southern Source:
46
SCORECARD
QUANTITATIVE QUANTITATIVE
APPENDIX C: C: APPENDIX Source: Southern California Edison Company, California 2012. Edison Southern Source:
47 APPENDIX D: SUPPLIER A SCORECARD
App Development App Maintenance Metric Actual Goal MAL Score Metric Actual Goal MAL Score Quantitative Quantitative 1.1 Defects / $10k Cost N/A 0.50 1.00 20.00 2.1 % On-time Implementations 97.62% 95.00% 85.00% 20.00 1.2 # of Projects with Zero Sev1 or Sev2 - 0 0 0.00 2.2 % Defect Free Implementations 100.00% 95.00% 85.00% 20.00 1.3 # of Projects Behind Schedule 0 0 0 20.00 2.3 Metric 3 0.00% 0.00% 0.00% 0.00 1.4 # of Projects Over Budget 0 0 0 20.00 2.4 Metric 4 0.00% 0.00% 0.00% 0.00 1.5 Metric 5 0.00% 0.00% 0.00% 0.00 2.5 Metric 5 0.00% 0.00% 0.00% 0.00 SCE Internal Metric SCE Internal Metric 1.1 Defects / $10k Cost 0.77 1.00 0.00 2.6 % On-time Implementations 90.00% N/A 0.00 1.2 # of Projects with Zero Sev1 or Sev2 0.00% N/A N/A 0.00 2.7 % Defect Free Implementations 90.00% N/A 0.00 1.3 # of Projects Behind Schedule 90% on time del. N/A 0.00 2.8 Metric 8 0.00% 0.00 1.4 # of Projects Over Budget 90% within plan N/A 0.00 2.9 Metric 9 0.00% 0.00% 0.00% 0.00 1.10 Metric 10 0.00% 0.00% 0.00 2.10 Metric 10 0.00% 0.00% 0.00% 0.00 48 Possible Score 60.00 Actual Score 60.00 Possible Score 40.00 Actual Score 40.00
Qualitative Survey Qualitative Survey Metric % Meets Does not Meet Score Metric Actual Goal MAL Score Qualitative Qualitative 4.1 Relationship Quality/Commitment 94% 4% 0.00 5.1 Metric 1 0.00% 0.00% 0.00% 0.00 4.2 Resource Caliber 90% 9% 0.00 5.2 Metric 2 0.00% 0.00% 0.00% 0.00 4.3 Project/Engagement Management 92% 7% 0.00 5.3 Metric 3 0.00% 0.00% 0.00% 0.00 4.4 Account Management 88% 11% 0.00 5.4 Metric 4 0.00% 0.00% 0.00% 0.00 4.5 Overall Satisfaction 91% 8% 0.00 5.5 Metric 5 0.00% 0.00% 0.00% 0.00 Qualitative - Top 3 Strengths Qualitative - Areas for Improvement 4.6 Testing/Quality 5.6 Knowledge of Company 4.7 Responsiveness 5.7 CW On/Offboard Process 4.8 Estimation/Contracting 5.8 Resource Management (Offshore)
Source: Southern California Edison Company, 2012. APPENDIX E: SUPPLIER B SCORECARD
App Development App Maintenance Metric Actual Goal MAL Score Metric Actual Goal MAL Score Quantitative Quantitative 1.1 Defects / $10k Cost 0.42 0.50 1.00 20.00 2.1 % On-time Implementations 100.00% 95.00% 85.00% 20.00 1.2 # of Projects with Zero Sev1 or Sev2 D - 0 0 0.00 2.2 % Defect Free Implementations 100.00% 95.00% 85.00% 20.00 1.3 # of Projects Behind Schedule 0 0 0 20.00 2.3 Metric 3 0.00% 0.00% 0.00% 0.00 1.4 # of Projects Over Budget 0 0 0 20.00 2.4 Metric 4 0.00% 0.00% 0.00% 0.00 1.5 Metric 5 0.00% 0.00% 0.00% 0.00 2.5 Metric 5 0.00% 0.00% 0.00% 0.00 Internal Metric Internal Metric 1.1 Defects / $10k Cost 0.77 1.00 0.00 2.1 % On-time Implementations 90% on time del. N/A 0.00 1.2 # of Projects with Zero Sev1 or Sev2 Defects N/A N/A 0.00 2.2 % Defect Free Implementations 90.00% N/A 0.00 1.3 # of Projects Behind Schedule 90% on time del. N/A 0.00 2.8 Metric 8 0.00% 0.00% 0.00% 0.00 1.4 # of Projects Over Budget 90% within plan N/A 0.00 2.9 Metric 9 0.00% 0.00% 0.00% 0.00 1.5 Metric 10 0.00% 0.00% 0.00 2.10 Metric 10 0.00% 0.00% 0.00% 0.00
49 Possible Score 60.00 Actual Score 60.00 Possible Score 40.00 Actual Score 40.00
Qualitative Survey Qualitative Survey Metric % Meets % Does not Meet Score Metric Actual Goal MAL Score Qualitative Qualitative 4.1 Relationship Quality/Commitment 84% 15% 0.00 5.1 Metric 1 0.00% 0.00% 0.00% 0.00 4.2 Resource Caliber 85% 14% 0.00 5.2 Metric 2 0.00% 0.00% 0.00% 0.00 4.3 Project/Engagement Management 88% 11% 0.00 5.3 Metric 3 0.00% 0.00% 0.00% 0.00 4.4 Account Management 84% 15% 0.00 5.4 Metric 4 0.00% 0.00% 0.00% 0.00 4.5 Overall Satisfaction 83% 16% 0.00 5.5 Metric 5 0.00% 0.00% 0.00% 0.00 Qualitative - Top 3 Strengths Qualitative - Areas for Improvement 4.6 Resource Caliber (onshore/offshore) 5.6 Domain Knowledge (i.e. Company & Utility business) 4.7 Flexibility / Responsiveness 5.7 Work Estimation 4.8 Communication Effectiveness 5.8 Resource Management (i.e. Selection of CWs to support "niche" projects)
Source: Southern California Edison Company, 2012. APPENDIX F: SUPPLIER C SCORECARD
App Development App Maintenance Metric Actual Goal MAL Score Metric Actual Goal MAL Score Quantitative Quantitative 1.1 Defects / $10k Cost 0.38 0.50 1.00 20.00 2.1 % On-time Implementations 98.20% 95.00% 85.00% 20.00 1.2 # of Projects with Zero Sev1 or Sev2 Def - 0 0 0.00 2.2 % Defect Free Implementations 99.10% 95.00% 85.00% 20.00 1.3 # of Projects Behind Schedule 0 0 0 20.00 2.3 Metric 3 0.00% 0.00% 0.00% 0.00 1.4 # of Projects Over Budget 0 0 0 20.00 2.4 Metric 4 0.00% 0.00% 0.00% 0.00 1.5 Metric 5 0.00% 0.00% 0.00% 0.00 2.5 Metric 5 0.00% 0.00% 0.00% 0.00 Qualitative Qualitative 1.6 Metric 6 0.00% 0.00% 0.00% 0.00 2.6 Metric 6 0.00% 0.00% 0.00% 0.00 1.7 Metric 7 0.00% 0.00% 0.00% 0.00 2.7 Metric 7 0.00% 0.00% 0.00% 0.00 1.8 Metric 8 0.00% 0.00% 0.00% 0.00 2.8 Metric 8 0.00% 0.00% 0.00% 0.00 1.9 Metric 9 0.00% 0.00% 0.00% 0.00 2.9 Metric 9 0.00% 0.00% 0.00% 0.00 1.10 Metric 10 0.00% 0.00% 0.00% 0.00 2.10 Metric 10 0.00% 0.00% 0.00% 0.00
50 Possible Score 60.00 Actual Score 60.00 Possible Score 40.00 Actual Score 40.00
Qualitative Survey Qualitative Survey Metric % Meets % Does not Meet Score Metric Actual Goal MAL Score Qualitative Qualitative 4.1 Relationship Quality/Commitment 85.00% 14.00% 0.00 5.1 Metric 1 0.00% 0.00% 0.00% 0.00 4.2 Resource Caliber 93.00% 6.00% 0.00 5.2 Metric 2 0.00% 0.00% 0.00% 0.00 4.3 Project/Engagement Management 88.00% 11.00% 0.00 5.3 Metric 3 0.00% 0.00% 0.00% 0.00 4.4 Account Management 84.00% 15.00% 0.00 5.4 Metric 4 0.00% 0.00% 0.00% 0.00 4.5 Overall Satisfaction 88.00% 11.00% 0.00 5.5 Metric 5 0.00% 0.00% 0.00% 0.00 Qualitative - Top 3 Strengths Qualitative - Areas for Improvement 4.6 Relationship Commitment 5.6 Resource Caliber 4.7 Domain Expertise (i.e. Billing, Post Billing, & Testing) 5.7 Domain Knowledge (i.e. CSS) 4.8 Responsiveness/Customer Service 5.8 Tier 2 DBE Spend
Source: Southern California Edison Company, 2012. APPENDIX G: SUPPLIER D SCORECARD
App Development App Maintenance Metric Actual Goal MAL Score Metric Actual Goal MAL Score Quantitative Quantitative 1.1 Defects / $10k Cost 1.20 0.50 1.00 0.00 2.1 % On-time Implementations 96.90% 95.00% 85.00% 20.00 1.2 # of Projects with Zero Sev1 or Sev2 D - 0 0 0.00 2.2 % Defect Free Implementations 100.00% 95.00% 85.00% 20.00 1.3 # of Projects Behind Schedule 0 0 0 20.00 2.3 Metric 3 0.00% 0.00% 0.00% 0.00 1.4 # of Projects Over Budget 0 0 0 20.00 2.4 Metric 4 0.00% 0.00% 0.00% 0.00 1.5 Metric 5 0.00% 0.00% 0.00% 0.00 2.5 Metric 5 0.00% 0.00% 0.00% 0.00 SCE Internal Metrics SCE Internal Metrics 1.1 Defects / $10k Cost 0.77 1.00 0.00 2.1 % On-time Implementations 90.00% N/A 0.00 1.2 # of Projects with Zero Sev1 or Sev2 D 0.00% N/A N/A 0.00 2.2 % Defect Free Implementations 90.00% N/A 0.00 1.3 # of Projects Behind Schedule 90% on time del. N/A 0.00 2.8 Metric 8 90.00% N/A 0.00% 0.00 1.4 # of Projects Over Budget 90% within plan N/A 0.00 2.9 Metric 9 0.00% 0.00% 0.00% 0.00 1.10 Metric 10 0.00% 0.00% 0.00 2.10 Metric 10 0.00% 0.00% 0.00% 0.00
51 Possible Score 60.00 Actual Score 40.00 Possible Score 40.00 Actual Score 40.00
Qualitative Survey Qualitative Survey Metric % Meets % Does not Meet Score Metric Actual Goal MAL Score Qualitative Qualitative 4.1 Relationship Quality/Commitment 94% 4% 0.00 5.1 Metric 1 0.00% 0.00% 0.00% 0.00 4.2 Resource Caliber 90% 9% 0.00 5.2 Metric 2 0.00% 0.00% 0.00% 0.00 4.3 Project/Engagement Management 92% 7% 0.00 5.3 Metric 3 0.00% 0.00% 0.00% 0.00 4.4 Account Management 88% 11% 0.00 5.4 Metric 4 0.00% 0.00% 0.00% 0.00 4.5 Overall Satisfaction 91% 7% 0.00 5.5 Metric 5 0.00% 0.00% 0.00% 0.00 Qualitative - Top 3 Strengths Qualitative - Areas for Improvement 4.6 Resource Caliber 5.6 Adherence to Company Standards (i.e. following Company protocols for projects) 4.7 Understanding company (i.e. environment and process) 5.7 Quality of Deliverables (i.e. project documentation quality & completeness) 4.8 Big Picture Thinking & Consulting 5.8 Complacent
Source: Southern California Edison Company, 2012. APPENDIX H: OVERALL RATINGS FROM QUALITATIVE SURVEY
Supplier A Supplier B Supplier C Supplier D % % % % % % % % (Does Not (Does Not (Does Not (Does Not # Metric Definition (Meets / Meet / (Meets / Meet / (Meets / Meet / (Meets / Meet / Exceeds) Partially Exceeds) Partially Exceeds) Partially Exceeds) Partially Meets ) Meets ) Meets ) Meets ) Relationship 1 Quality / 94% 4% 84% 15% 85% 14% 94% 4% Commitment 2 Resource Caliber 90% 9% 85% 14% 93% 6% 90% 9%
52 Project/Engagement 3 92% 7% 88% 11% 88% 11% 92% 7% Management Account 4 88% 11% 84% 15% 84% 15% 88% 11% Management 91%* 8%* 83%* 16%* 88%* 11%* 91%* 8%* (Satisfied/ (Satisfied/ (Satisfied/ (Satisfied/ 5 Overall Satisfaction (Not (Not (Not (Not Very Very Very Very Satisfied) Satisfied) Satisfied) Satisfied) Satisfied) Satisfied) Satisfied) Satisfied) Averaged Overall 91% 7% 85% 14% 88% 11% 91% 7% Ratings
Source: Southern California Edison Company, 2012. APPENDIX I: MAJOR ISSUES FROM QUALITATIVE SURVEY
SPM Scorecard - Qualitative Survey - Major Issues
Supplier A Supplier B Supplier C Supplier D
Top 3 (Strengths) 1. Testing / Quality 1. Resource Caliber 1. Relationship 1. Resource Caliber (onshore/offshore) Commitment / Collaboration 2. Responsiveness 2. Flexibility / 2. Domain Expertise (i.e. 2. Understanding SCE (i.e. Responsiveness Billing, Post-billing & environment & process) Testing) 3. Estimation / 3. Communication 3. Responsiveness / 3. Big Picture Thinking & 53 Contracting Effectiveness Customer Service Consulting Bottom 3 (Opportunities) 1. Knowledge of SCE 1. Domain Knowledge (i.e. 1. Resource Caliber 1. Adherence to SCE SCE & Utility business) Standards (i.e. following SCE protocols for projects) 2. CW onboard/offboard 2. Work Estimation 2. Domain Knowledge (i.e. 2. Quality of Deliverables Process CSS) (i.e. project documentation quality & completeness) 3. Resource Mgmt. 3. Resource Management 3. Tier 2 DBE Spend 3. Complacent (offshore) (i.e. Selection of CWs to support "niche" projects)
Source: Southern California Edison Company, 2012.