How to Use Existing Technology Roadmaps to Evaluate Early-Stage Breakthrough Innovations? Development and Testing of a Workshop-Based Toolkit

Home , Technology roadmap

How to use existing technology roadmaps to evaluate early-stage breakthrough innovations? Development and testing of a workshop-based toolkit

Carl Brinkmann Darwin College University of Cambridge

Supervisor: Dr. Rob Phaal This dissertation is submitted for the degree of Master of Philosophy in Industrial Systems, Manufacture and Management September 2017

Abstract

Purpose – This research aims to understand how to evaluate early-stage breakthrough innovations using existing strategy plans. Technology roadmapping is already widely used to identify and evaluate new opportunities. However, the potential of using existing roadmaps to evaluate and scope new ideas on a continuous basis remains largely unexploited. This mainly relates to the challenges of keeping roadmaps up to date. This research focuses on the first step of a continuous process to overcome the identified challenges. Approach – A case-based action research is performed to design and test a workshop-based toolkit. The design incorporates existing methods & tools, company considerations and consultant feedback. The workshop-based toolkit is then tested in company settings to respond to the research question and further develop the process. Design – The workshop is performed during the pre-selection step and uses a market-pull approach. The toolkit is based on a linking grid to help visualise synergies between ideas, relevant roadmap information and the selection factors. Scorings related to priority and information quality are added to the linking grid in order to give a dimension of importance to each driver/product/technology. Findings – The workshop-based toolkit was effective at evaluating early-stage ideas using existing roadmaps. Participants used “synergies” and “gaps” within each roadmap layer in order to evaluate each selection factor. For some selection factors, “unknown” information was missing from the roadmaps, thereby hindering the evaluation process. Roadmaps therefore need to include this missing information in order to become useful at evaluating new ideas. Research limitations – The workshop-based toolkit should be further validated in different situations and its wider applicability should be investigated. In order to validate the complete continuous process, future research should extend the approach by considering the development of new project plans and their integration into the innovation strategy. Theoretical implications – This research provided theoretical contribution by developing and testing methods & tools for evaluating new ideas using existing roadmaps. Practical implications – Practicing innovation managers are provided with a workshop- based toolkit to ensure that selected innovations are consistent with strategic goals and future company developments.

iii

Acknowledgements

My sincerest gratitude goes to my supervisor Dr. Rob Phaal for guiding me through this exciting dissertation project. I would also like to thank the interview participants that kindly took their time to provide insightful inputs to my research. The workshop would not have been possible without the help from the host company, workshop participants and the facilitator. Lastly, I would like to express my gratitude to researchers and staff, directly or indirectly involved in providing support to enable the success of the project.

Statement of Original Authorship

This dissertation is submitted for the award of Master of Philosophy in Industrial Systems, Manufacture and Management. I hereby certify that this dissertation is entirely my own work, written by myself and any work from others used is referenced to the best of my knowledge. The length of this dissertation does not exceed 15,000 words.

Total number of words: 14,994

Carl Brinkmann

Table of Contents

1. Introduction & Motivation ...... 1

2. Literature Review ...... 2 2.1. Innovation management ...... 3 2.2. Evaluating early-stage innovations ...... 7 2.3. Innovation strategy planning ...... 12 2.4. Research gap ...... 22

3. Research Methodology ...... 23 3.1. Research Focus ...... 23 3.2. Research Objectives ...... 24 3.3. Philosophical considerations ...... 25 3.4. Research method ...... 25 3.5. Conclusion ...... 34

4. Interview Results ...... 35 4.1. Innovation types and strategy models ...... 35 4.2. Innovation processes ...... 35 4.3. Requirements related to people, time and tools ...... 37 4.4. Workshop limitations and opportunities ...... 41 4.5. Interview conclusions ...... 42

5. Toolkit and Workshop Design ...... 44 5.1. Design considerations ...... 44 5.2. Toolkit design ...... 45 5.3. Workshop design ...... 48

6. Workshop results ...... 59 6.1. Workshop feedback ...... 59 6.2. Potential of the toolkit to evaluate new ideas ...... 61

7. Conclusion and Future Research ...... 63 7.1. Conclusion ...... 63 7.2. Research Limitations ...... 65 7.3. Opportunities for Future Research ...... 66 7.4. Theoretical implications ...... 66 7.5. Practical implications ...... 67

8. References ...... 69

9. Appendices ...... 74 9.1. Feedback questionnaire ...... 74 9.2. Tool templates ...... 77 9.3. Workshop schedule ...... 80

vii

1. Introduction & Motivation

The globalisation and increasing pace of technological change have forced multinational organisations to bring innovation management at the heart of corporate decision-making. Companies rely on innovation strategy plans to ensure the development of technologies and products that match high-level strategic goals. Technology roadmaps have been developed for that purpose, by representing relations among technologies and resources as they evolve toward practical product applications and markets (Kostoff et al., 2001). One of the objectives of developing a technology roadmap is to identify and evaluate new opportunities within all layers of the organisation (Phaal et al., 2001). Similarly, the developed roadmaps could help evaluate and scope new ideas on an on-going basis, thereby ensuring consistency with future company goals and developments. This is especially useful for early-stage and breakthrough innovations, in order to reduce the high assumption uncertainties (Dissel et al., 2006). However, this potential remains largely unexploited due to the challenges of keeping roadmaps up-to-date with new developments (Vatananan et al., 2012). In order to respond to the identified challenges, a process framework is proposed to evaluate new ideas and update roadmaps on a continuous basis. This research aims at validating the first step of the proposed process by answering to the following research question: How to use existing technology roadmaps to evaluate early-stage breakthrough innovations? The research question is addressed through the development and testing of a workshop-based toolkit. The approach follows a case-based action research consisting of five successive steps: 1. The literature review identifies existing methods and tools. 2. Company interviews ensure compatibility with industry practices and challenges. 3. Consultant interviews incorporate consulting methods into the design. 4. Toolkit & workshop designs synthesize the strategy formulation process. 5. Testing the design in company settings helps validate the research question and refine the process further. The dissertation is organised as follows. Chapter 2 reviews the literature and identifies the research gap. Chapter 3 introduces the process framework and outlines the research methodology. This is followed by the analysis of company interviews in Chapter 4. Chapter 5 then describes the toolkit and workshop design, followed by the workshop results in Chapter 6. Finally, the dissertation is concluded in Chapter 7.

2. Literature Review

The following Chapter outlines the literature that was examined in order to investigate the research question. Figure 1 shows the relationship between the three main research domains and the corresponding concepts within each, which forms the structural basis of this chapter. The first section on innovation management aims at identifying relevant innovation types, strategy models and processes for early-stage breakthrough innovations. The second section aims at identifying relevant tools and methods for evaluating early-stage innovations. After understanding basic principles, the section discusses design and process considerations associated to scoring tools. The final section aims at identifying relevant tools and methods within innovation strategy planning. After considering different tools, the section focuses on technology roadmapping. Following an overview of the tool, the implementation process is studied in greater detail by considering its three main stages: initiation, development and integration.

Innova&on Innova-on Management strategy The process of Types of models innova-on innova-ons Evalua&ng Technology early-stage Roadmapping innova&ons Purposes Formats Tools for Principles & strategy Challenges Roadmapping T-Plan Ini&a&on & planning evalua-on tools process Development Assessment Innova&on methods Roadmap Integra-on Strategy Quality Challenges Planning Scoring So@ware Tool design Integra&on Scoring integra-on Tool process

Focus Area

Figure 1: Research domains and focus area

2.1. Innovation management

2.1.1. Types of innovations

An invention is defined as a new idea or combination of ideas (Kirchhoff, 1994). Inventions alone cannot be commercialised, but become saleable products or services through the process of innovation (Kirchhoff, 1994). Organisations rely on different types of projects to drive their innovation strategy. This section outlines the different classifications of innovation projects.

2.1.1.1. Degrees of innovation

There is no universally accepted terminology for the degrees of innovation (Goffin et al., 2016). However, one of the most common classifications uses the terminology of derivative, platform and breakthrough innovation projects (Clark et al., 1993): • Derivative projects aim to achieve an incremental product modification, resulting in cost reduction or the addition of a new feature. • Platform projects aim to achieve significant changes in cost and quality, functional improvements and the foundation for follow-on products. • Breakthrough projects aim to achieve major changes to existing processes and products, often through the discovery of a major invention. This research focuses on breakthrough innovation projects.

2.1.1.2. Project maturity

Innovation projects are also associated to varying levels of maturity. For technology- intensive innovation, the Technology Readiness Levels (TRLs) (Mankins, 1995) provide a nine-point measure of technology maturity (Table 1). Although mostly used in aerospace and defence applications, its guidelines are applicable to a broader context. For the purpose of this research, an early-stage technology includes technology readiness levels 1 and 2, before research and development is initiated.

Table 1: The nine technology readiness levels for space applications (Mankins, 1995)

2.1.2. Innovation strategy models

Due to the constant change in the innovation landscape, including increasingly faster technology developments, managers have modified their company’s innovation strategy models over the years. Yet, most of the innovation strategy models consist of a variation between the two extremes: technology push and market pull (Hobday, 2005): • Technology push focuses on R&D innovation without concern about market attractiveness or applications of the developed technologies into products. The model assumes that scientific discovery pushes the development of technological innovations through the successive stages of applied research, engineering, manufacturing and marketing. • Market or demand-pull models use market needs to allocate investments in technology and product developments. The marketplace is seen as the main source of ideas and the role of R&D is to meet these market demands (Hobday, 2005). Although the use of one innovation strategy usually predominates in an organisation (Pearson, 1990), Brem et al. (2009) argue that both strategies are equally important to all kinds of organisations and that adopting a single strategy may affect long term competitiveness. Furthermore, Zhou et al. (2005) found that both market-pull and technology-push strategies have a positive impact on the development of breakthrough technology-based innovations.

2.1.3. The process of innovation

Introduced by Hayes et al. (1988), the innovation funnel represents the innovation process from idea creation (“Fuzzy Front End”) to new product development, all the way to commercialization. Figure 2 shows an adaptation of the innovation funnel by Mitchell et al. (2014), emphasizing the different filters at each stage of the innovation process. At the “Fuzzy Front End”, early-stage innovations generally pass through three main filters before the development process. First, the “project type filter” separates projects into different innovation types. If many projects come out of the first filter, a second “triage filter” reduces the number of projects to a more manageable amount. Between the triage and selection filters, coarse proposal preparations and pre-selection filters further refine the selected projects. At the selection filter, the remaining projects are analysed with greater scrutiny (usually during a strategic review meeting) and any project passing through can progress to the development process. The latter is usually governed by stage gates, such as the ones described by Cooper et al. (2001).

Figure 2: The innovation funnel (Mitchell et al., 2014 - adapted from Hayes et al. (1988))

Although the stage-gate process is usually applied to the subsequent development process (Cooper et al., 2000), Cooper (2006) also designed a technology development stage-gate process for the “Fuzzy Front End” that takes into account the high risks and technical uncertainties related to early-stage innovations (Figure 3). Gates 1 and 2 reflect the “pre- selection filter” and the “selection filter” from Figure 2. At Gate 2, the decision is taken to begin limited experimental work in Stage 2.

Figure 3: The technology development Stage-Gate process (Cooper, 2006)

The innovation funnel and the technology development stage-gate process identify five main process steps for early-stage innovations (Figure 4). The project type filter is not considered since this research only focuses on breakthrough innovations.

Coarse Pre- Project Selec0on Triage proposal selec0on scoping ﬁlter

Figure 4: Process steps for early-stage breakthrough innovations (Adapted from Cooper (2006); Mitchell et al. (2014))

2.2. Evaluating early-stage innovations

2.2.1. Principles and challenges

Portfolio management is fundamental to new product development (Cooper et al., 2000). It is vital to understand which development projects should be selected, and how a business should allocate resources (capital and people) between these different projects (Cooper et al., 2000). Additionally, the long-term success of a firm is closely related to the presence of effective decision-making processes along the new product development lifecycle (Cooper et al., 1999; Chao et al., 2008). More specifically related to project selection, portfolio management aims at maximising the value of the new projects, achieving a balanced portfolio and ensuring alignment with business strategy (Robert G Cooper et al., 2001). When selecting new projects, the challenges of portfolio management are closely related to the nature of early-stage innovations (Dissel et al., 2006): • High uncertainty about project assumptions • Difficulty to decide make or buy decisions • Difficulty to assess the future value of the project • Difficulty to forecast future project applications Therefore, an organisation must have processes and tools in place to identify, assess, prioritise and implement new innovation projects (Mitchell et al., 2014).

2.2.2. Assessment methods

Financial assessment methods such as NPV, ECV, discounted cash flow, decision trees and real options are widely used to assess the value of potential projects (Cooper et al., 2000), (McGrath, 1997), (Faulkner, 1996). However, due to the high uncertainties related to early- stage innovation projects, financial methods are often regarded as inadequate (Dissel et al., 2006). On the other hand, qualitative methods usually rely on the judgment of experts on different project selection factors. Related selection tools include variations such as checklists, paired comparisons and scoring models, which also have the potential of incorporating financial measures (Cooper et al., 2000). Robert G Cooper et al. (2001) argue that scoring models are the most appropriate for qualitative project selection.

2.2.3. Scoring models

Scoring models are widely used for selecting innovation projects, with many famous examples being implemented within companies such as Celanese AG, DuPont and EXFO (Robert G Cooper et al., 2001). Robert G Cooper et al., (2001) and Mitchell et al., (2014) developed two generic scoring models based on multiple industry case studies. The process from Mitchell et al. (2014) was specifically designed for the evaluation and selection of early-stage innovation projects, by only including qualitative factors. It built upon the methods from Robert G Cooper et al. (2001) by providing better guidance on how to structure the scoring process and how to choose selection factors. The following sections contrast the design of the two scoring tools and give an overview of the selection process management.

2.2.3.1. Scoring tool design

Both design processes from Robert G Cooper et al. (2001) and Mitchell et al. (2014) take into account six major design considerations: 1. Factor classification: Selection factors can be classified in numerous ways. The generic scoring model developed by Robert G Cooper et al. (2001) separates them into strategic alignment & importance, product & competitive advantage, market attractiveness, leverage of core competencies, technical feasibility and financial reward. Mitchell et al., (2014) further argued that selection factors can be divided into two orthogonal considerations; opportunity and feasibility. Opportunity factors measure how much value the project could bring to the organisation, while feasibility factors assess the necessary investments required to grasp the opportunity. Hence the product and visualisation of Opportunity x Feasibility can provide a rough indication of potential Value over Investment (Figure 5). 2. Choosing the selection factors: The choice of factors varies greatly depending on the organisation and project maturity (Robert G Cooper et al., 2001). Mitchell et al., (2014) provide examples of opportunity and feasibility factors taken from the literature and practical experience (Tables 2 and 3), focusing on product-related, early-stage innovation projects. 3. Scaling statements: The scaling statements are used to guide experts in scoring each factor. Similarly to the selection factors, scaling statements depend greatly on project maturity (Robert G Cooper et al., 2001). Mitchell et al. (2014) suggest to first choose the

scaling statements for the “base factor”, for which clear and objective scaling statements can be designed. The midpoint should then be identified, which indicates the baseline for a successful project. Finally, all other scaling statements are chosen by reference to the base factor. The outer most scaling statements should represent conditions that might be met occasionally, not impossibilities. The authors recommend using a 0 to 12 scale, including around 5 scaling statements. 4. Weightings: Cooper et al. (2001) found that many companies use equal weights for all factors while others argue that factors require weightings due to the relative importance of each. Mitchell et al. (2014) argue that careful choice of scaling statements should make the weighting of each factor obsolete. However, varying weightings can be given to the opportunity or feasibility groups after the scoring stage to reflect the maturity of innovation projects. If early-stage innovations are considered, the potential opportunity should be more important than feasibility, and should therefore be given a higher weighting (Mitchell et al., 2014). 5. Risk, uncertainty and confidence: Mitchell et al. (2014) put a greater emphasis on uncertainty and confidence than Robert G Cooper et al. (2001), by suggesting to select the “most reasonable upper and lower extremes” from the scoring of each factor (Figure 6). This helps understand divergences in opinions among decision makers, thereby improving the quality of information.

Figure 5: Visualisation of opportunity and feasibility scores for each early-stage innovation project, including upper and lower extremes (Mitchell et al., 2014)

Table 2: Examples of opportunity factors (Mitchell et al., 2014)

Table 3: Examples of feasibility factors (Mitchell et al., 2014)

2.2.3.2. Scoring tool process for early-stage innovations

The previous section identified shortcomings in the scoring model designed by Robert G Cooper et al., (2001), especially concerning early-stage innovation projects. This section therefore focuses on the scoring model from Mitchell et al. (2014) to understand the management of the scoring process. Figure 6 shows the separation of the workshop-based scoring process into preparation, scoring and decision phases.

Figure 6: Preparation, scoring and decision phases for selecting early-stage innovation projects (Mitchell et al., 2014)

2.3. Innovation strategy planning

Innovation strategy planning is an important part of the overall business strategy (Goffin et al., 2016). It determines and shows the broad lines of when, where and how innovation is required to meet the aims of an organisation. Goffin et al. (2016) and Kaplan et al. (2000) identify six main tools for innovation strategy planning to be used in a workshop environment: scenario planning, strategic landscape maps, technology roadmaps, business model canvas, balanced scorecards and strategy maps. Among all the innovation strategy planning tools, technology roadmaps display the greatest granularity at the different levels within an organisation (Goffin et al., 2016). Technology roadmaps should therefore be appropriate to evaluate new innovation projects, and are therefore chosen as the focus of this research.

2.3.1. Overview of technology roadmapping

Technology roadmaps represent the structural and temporal relations among technologies and resources as they evolve toward practical product applications and markets (Kostoff et al., 2001). As a mean to initiate and update roadmaps, the technology roadmapping process provides a way to identify, evaluate and select strategic alternatives to achieve a desired company objective (Kostoff et al., 2001).

2.3.1.1. Purposes of technology roadmaps

An examination of approximately forty technology roadmaps by Phaal et al. (2004) identified eight types of roadmapping purposes (Figure 7). Since this research focuses on updating the innovation strategy with early-stage innovations, the list can be narrowed down to two relevant purposes (Phaal et al., 2004): 1. Product planning is by far the most common type of technology roadmap and relates to the insertion of technology into manufactured products of different generations. 2. Strategic planning takes a higher-level approach by displaying more layers within the organisation. It is more suitable for evaluating different opportunities and threats at the business level. The focus is on the planning of a future vision of the business, including markets, business, products, technologies, skills, culture, etc. The roadmap helps identify gaps by comparing the future vision with the current position. Strategic options are then explored at all levels in order to bridge the gaps.

Figure 7: Types of technology roadmap purposes: (a) product planning, (b) service/capability planning, (c) strategic planning, (d) long range planning, (e) knowledge asset panning, (f) program planning, (g) process planning, (h) integration planning (Phaal et al., 2004)

2.3.1.2. Formats of technology roadmaps

Similarly to roadmap purposes, Phaal et al. (2004) identified six main types of roadmapping formats (Figure 8). Vatananan et al. (2012) showed that a roadmap’s level of granularity depends heavily on its function and on its target audience. This research focuses on multilayer and tabular formats due to their higher level of granularity (Phaal et al., 2004):

1. The multi-layer format is the most common roadmapping format. It includes layers and sub-layers such as market, product and technology & resource versus a time axis. The roadmap shows the evolutions within each layer and interlayer dependencies in order to attain the future objectives. 2. The tabular format is well suited when performance is easy to quantify, or if activities are clustered in specific time periods. The corresponding axes often show performance or requirements over time.

Figure 8: Types of technology roadmap formats: (a) multiple layers, (b) bars, (c) tabular, (d) graphical, (e) pictorial and (f) flowchart (Phaal et al., 2004)

2.3.1.3. Technology Roadmapping Implementation

Through a review of the roadmapping literature, Vatananan et al. (2012) divide the technology roadmapping process into three main stages, with different aims (Table 4). If an organisation’s goal is to guide its strategic vision as a one-time effort, the roadmap implementation can be terminated after the development stage (Strauss et al., 2004). However, continuing with the integration stage allows to assimilate the technology roadmapping process into on-going business operations (Vatananan et al., 2012). Table 4: Technology roadmapping implementation stages and aims (Vatananan et al., 2012)

Stage Aims 1. Initiation Get an organisation ready before implementing the roadmapping process. 2. Development Perform a step-by-step approach to develop a roadmap of the desired purpose and format, by engaging the right people and gathering relevant information. 3. Integration Constantly review and update roadmaps to keep roadmaps alive after their implementation.

Section 2.3.2 discusses the literature related to initiation and development stages, while section 2.3.3 considers the integration stage.

2.3.2. Roadmap Initiation and Development

Roadmapping initiation and development are generally performed during a workshop that involves cross-functional teams with individuals from different levels of an organisation (Phaal et al., 2004).

2.3.2.1. The T-Plan process: a fast workshop based approach

Vatananan et al. (2012) recall many developments of roadmapping processes for specific industry uses. The T-Plan process (Figure 9) was later developed by Phaal et al. (2001) to support product planning using a standard customizable process. Predominantly based on a market-pull approach, the first three phases focus on the three main layers of the roadmap (market, product and technology). At each layer, the different options are identified, grouped and prioritised. The key knowledge gaps are also recognized.

The final workshop then helps to bring the three layers together on a time-basis to construct the final roadmap. Participants identify project milestones, chart the product evolution and consider appropriate technology responses. T-Plan also considers three workshop management phases by giving guidance on setting up, managing and following on from the process.

Figure 9: T-Plan standard process steps (as described by Phaal et al. (2003))

2.3.2.2. Roadmapping evaluation tools

The literature shows the use of various management tools to support the initiation and development of technology roadmaps. Vatananan et al. (2012) classifies these tools according to their functionality into market analysis, technology analysis and supporting tools. While some are used for data collection or dissemination, others are used to evaluate and prioritise arising opportunities (Table 5).

Table 5: Roadmapping evaluation and prioritization tools (adapted from Vatananan et al. (2012) and Carvalho et al. (2013)).

Tools Aim/ Use References Analytic hierarchy Helps pick the best technology for (Fenwick et al., 2009); process a particular solution (Gerdsri et al., 2007)

Competitive features Compare product offerings and (Fenwick et al., 2009) matrix identify gaps Strategy map Display high level company (Phaal et al. 2012) strategy Perceptual map rank Rank and compare the value of (Fenwick et al., 2009) valuation product features Portfolio Evaluate strategic options using a (Oliveira et al., 2010); management combination of weighted scoring (Phaal et al., 2006); (Kerr and 2x2 matrices et al., 2015) Linked analysis grids Linkage grids: Map the (An et al., 2008); (Lee et (including QFD) relationship between roadmap al., 2009); (Groenveld, layers, including current and future 1997); (Kameoka et al., views. 2004); (Phaal et al., 2001); QFD: Use a matrix structure to (Kerr et al., 2015); translate customer requirements (Lindsay, 2000) into technology solutions Technology Rank the impact of technologies (Fenwick et al., 2009); Development over a time interval (Gerdsri, 2007) Envelope Idea Proposal Describes an opportunity by (Farrukh et al., 2014) Template answering what, why, how and when questions.

Value proposition Evaluates an idea capability to (Fenwick et al., 2009) meet customer drivers

While some articles show the combination of tools with roadmapping during a single use case, Phaal et al. (2012) used a modular toolkit during numerous roadmapping workshops to enable a flexible iterative workshop process. The toolkit integrates technology roadmaps together with linking grids, portfolio matrices and business case proposals in order to evaluate new opportunities (Figure 10). A “concept foundation for a scalable toolkit platform” was later developed as a more generic approach (Kerr et al., 2015).

Figure 10: Modular toolkit used during roadmapping workshops (Phaal et al., 2012)

2.3.3. Roadmap Integration

The integration of a technology roadmapping process into on-going business operations is essential for ensuring the long-term benefits of the process (Phaal et al., 2004), (Groenveld, 1997), (Farrukh et al., 2001), (Gerdsri and Vatananan, 2007), (Strauss et al., 2017). In fact, Phaal et al. (2010) argue that “roadmaps should not be considered as a separate independent process, but as a structured visual framework that can support strategy and innovation processes”. The main integration activities include roadmap updating and reviewing as part of a new product development process (Phaal et al., 2010). After understanding integration challenges, this section identifies roadmap quality assessments and software integration as key tools for successful roadmap integration (Phaal et al., 2010).

2.3.3.1. Integration challenges

A 1999 survey of 3,000 medium-large UK manufacturing firms revealed that keeping the roadmapping process “alive” remains the greatest implementation challenge (Figure 11) (Phaal et al., 2000). Roadmap integration is highly complex and involves major changes in the structure and culture of an organisation (McMillan, 2003). One of the main challenges relate to the participation of empowered stakeholders as part of a cyclical process (Cosner et al., 2007). Another challenge relates to change management. The nature of roadmapping initiatives involves change management in the implementation of roadmapping decisions, but also in the integration of the roadmapping process (Phaal et al., 2010). It is therefore important to “provide appropriate change management techniques and proper training to prepare all key players for roadmap implementation” (Vatananan et al., 2012). Furthermore, Vatananan et al. (2012) recognise the lack of guidance and tools related to roadmap integration in order to: • Ensure roadmap data quality • Integrate existing management tools into technology roadmapping processes and the on-going operations of an organisation

Figure 11: Roadmapping challenges (Phaal et al., 2000)

2.3.3.2. Roadmap quality assessment

The literature shows the use of many different factors to assess the quality of the roadmapping process (Kostoff et al., 2001, 2004; Wall et al., 2005; Ma et al., 2006; Lee et al., 2007). A different type of quality assessment is related to the roadmapping tool itself, which is very important for useful roadmap integration (Kostoff et al., 2001) but has been much less covered in the literature. Albright (2003) argues that the best measure of roadmap quality is the value created for the organisation. However these measures are usually only available after a long period of time. Therefore, in order to provide immediate feedback, Albright (2003) provides a score card for assessing the quality of technology roadmaps for each layer (market, product and technology) (Table 6). The scorecard also enables to track the performance of the roadmapping team over the creation of many roadmaps.

Table 6: Roadmap quality assessment scorecard

Apart from strategic planning, data quality is also critical for information management systems, where comprehensive methodologies have been developed to assess and improve information quality. Lee et al. (2002) developed the PSP/IQ model (Table 7) by separating information quality dimensions into four quadrants: sound, dependable, useful and usable.

Table 7: The PSP/IQ model (Lee et al., 2002)

2.3.3.3. Software tool for roadmap integration

Once roadmapping integration progresses, the use of software to support roadmap operationalization becomes more and more relevant, especially in large organisations (Phaal et al., 2010). Although simple presentation, word processing and spreadsheet software are suitable during early stages, sophisticated software-enabled roadmaps can be used in more dynamic ways. As one example, Hassan et al. (2006) developed a web-based software to store, retrieve, edit and document different types of technology roadmaps.

2.4. Research gap

The evaluation of early-stage breakthrough innovations remains challenging due to high assumption uncertainties (Dissel et al., 2006). In addition, roadmap integration remains challenging due to the absence of methods and tools to update and review technology roadmaps (Vatananan et al., 2012). Methods and tools are widely available to evaluate new opportunities that arise during roadmapping initiation & development (Phaal et al., 2003; Vatananan et al., 2012; Carvalho et al., 2013; Farrukh et al., 2014; Kerr et al., 2015). However little research has been performed on using the developed roadmaps to evaluate new innovations during the integration phase.

3. Research Methodology

3.1. Research Focus

One of the main activities falling under roadmap updating relates to the incorporation of new innovation projects in the innovation strategy plan of the company. Figure 12 proposes a process framework to perform these activities in a continuous process. The first step towards this goal is to compare new ideas to the company’s existing strategy plans. New ideas are evaluated and prioritised by considering future company developments and objectives. This ensures the preparation of project plans (for the selected ideas) that are consistent with the planned strategy. At a later stage, consistent project plans can easily be integrated back into the strategy plan for a faster updating process. In order to start investigating the proposed updating process, this research focuses on the first step.

New project 2. Prepare new scoping 3. Easily project plans that integrate are consistent consistent with future project plans company back into the developments innova-on and objec-ves strategy

New ideas Innova-on Strategy Plan

1. Evaluate and priori-se new ideas using exis-ng strategy plans

Figure 12: Proposed process framework to update the innovation strategy with new projects.

The first step can be performed by visualising the innovation strategy plan using existing technology roadmaps. The potential of roadmaps to evaluate new ideas relates to the tool granularity and the roadmapping process: • Roadmaps show the innovation strategy on at least three levels (market, product and resources) (Phaal et al., 2003) • Roadmaps usually display (Phaal et al., 2003): o Priorities for the drivers o Timeframes for products & resources • The roadmap initiation phase ensures a consensus between the different layers in the organisation (Phaal et al., 2003) • The roadmaps in question are already existing within the company, thereby avoiding new tool creation and reducing management tool proliferation (Love et al., 2011) • Roadmaps are platforms to easily integrate other management tools (Carvalho et al., 2013; Kerr et al., 2015) The following chapter elaborates on the research design that was utilized to validate the first step of the process framework.

3.2. Research Objectives

The research aims at validating the first step of the proposed process framework by responding to the following research question:

How to use existing technology roadmaps to evaluate early-stage breakthrough innovations?

The research question is addressed through the development of a workshop-based toolkit. The research then becomes a design process aiming to identify the relevant innovation models, processes and methods & tools by responding to the following sub-questions: 1. Innovation models: What are the strategy models related to breakthrough innovations? 2. Processes: What are the innovation processes for early-stage evaluation? 3. Methods & Tools: What are the methods and tools to evaluate early-stage ideas and implement innovation strategies?

3.3. Philosophical considerations

In order to investigate the research objective in an appropriate manner, the nature of the research first needs to be considered in terms of ontology (what is the form and nature of reality?), epistemology (what is the nature of knowledge?) and methodology (how can the inquirer find what he/she believes can be known?) (Guba et al., 1994). With regards to the concept of ontology, this research follows a subjective rather than an objective approach. In fact, due to the nature of management workshops, individuals are responsible for creating and forming facts, rather than solely relying on fundamental laws (Bryman, 2012). Literature about epistemology state four main approaches: “positivism”, “interpretivism”, “relativism” and “action research” (Bryman, 2012). Within the context of researching organisation strategy, Susman et al. (1978) argue for the adoption of action research to improve relevance to the world of practicing managers. This research therefore uses a case- based action research to design and validate the workshop process.

3.4. Research method

In order to respond to the main research question, the case-based action research needs to incorporate the development, testing and refinement of a management process, which can be used in industrial settings. For this purpose, Platts (1993) proposes “a research approach to researching manufacturing strategy”. Although the corresponding case based action research was originally applied to manufacturing strategy, the described methodology has since been applied to the design of innovation management processes (Phaal et al., 2012). It is therefore compatible with the objectives of this research and forms the basis of the chosen research method. Platts (1993) outlines the main requirements for strategy formulation: • The process must link to existing frameworks • There must be adequate empirical testing and verification of any proposed process • The results of the research must be relevant to the world of the practicing manager In order to answer these requirements, Platts (1993) divided the development process into six main steps. This research follows the proposed method from steps 1 to 5 (Figure 13). Step 6 was not considered due to time constraints but should be performed in future work (Section

7.3). Since step 1 has already been discussed in Chapter 2, the following section describes the research design for steps 2 to 5 in greater detail.

1. Literature Review: Iden%fy exis%ng strategy frameworks

2. Company Interviews: Ensure compa%bility with industry prac%ces and challenges

3. Consultant Interviews: Incorporate consul%ng methods

4. Toolkit and Workshop design Synthesize the strategy formula%on process

5. Test the workshop-based toolkit in company seEngs Test and reﬁne the process

6. Perform company surveys Inves%gate the wider applicability Figure 13: Research method (based on Platts, (1993))

3.4.1. Company interviews

Following the guidance from Platts (1993), this research performs company interviews in order to investigate the practical application of frameworks from the literature, thereby ensuring compatibility with industry practices and challenges. The interview objectives respond to the three research sub-questions, followed by the main research question: 1. Recognise the strategy models used for each innovation type. 2. Understand innovation process steps for early-stage evaluation. 3. Identify the methods and tools used to evaluate early-stage ideas and implement innovation strategies, including people and time requirements. 4. Gather feedback on the potential to evaluate early-stage breakthrough ideas using existing roadmaps.

3.4.1.1. Case study design

A case study approach was used for the interviews. Yin (2013) defines four different types of case study designs as illustrated in Figure 14.

Figure 14: Distinction between single and multiple case study designs with a holistic or embedded analysis (adapted from Yin (2013))

In order to ensure that the designed workshop is flexible and can be applied to the innovation systems of various industry types (Phaal et al., 2006), this research chose an embedded multiple-case design. The embedded approach reflects the need to investigate different units of analysis within each case. In this research, the units of analysis correspond to the four interview objectives. The disadvantage of the multiple-case design is that more resources are needed (in terms of people and time), therefore providing less depth per case (Voss et al., 2002). However, this limitation is partially overcome by testing the workshop process at a later stage.

3.4.1.2. Case selection

As mentioned previously, case selection was performed to ensure workshop applicability to a wide range of companies. The companies were therefore chosen to obtain variations in: • Industry sector: the case study research considered companies within the secondary industry sector (manufacturing) and the tertiary industry sector (consultancy and consumer services). • Size and location: The size of the companies varied in revenues from less than GBP 25 mn to more than GBP 10 bn. Regardless of their size, all interviewed companies

operated internationally, which ensures the applicability of the workshop process to many working cultures. • Roadmapping and innovation experience: The first group of companies included members of an innovation consortium. One of the key consortium activities is to run roadmapping workshops. Therefore, these companies were more likely to have previous roadmapping experience. The second group was composed of companies outside of the innovation consortium and from one of the most innovative areas in the world: the San Francisco Bay. Although they were less likely to have previous roadmapping experience, the maturity of their innovation processes and feedback on workshop applicability was seen as very useful for the design of the workshop. Twelve companies were contacted in each group, resulting in a total of seven interviews. Table 8 shows the list of the interviewed companies and interviewees, in chronological order. All interviewees were familiar with the company’s innovation processes and participated in the evaluation and prioritization of innovation projects. Table 8: Summary of case studies from the selected companies.

Group Company Company Information Interviewee Information

Research and technology C1 Engineering Manager consulting

Manufacturer of chemical C2 Research Manager products Innovation Manufacturer of specialty C3 Innovation Manager Consortium materials Manufacturer of products for Principal Research C4 the energy and transport sectors Engineer (Europe)

Manufacturer of industrial Vice President of C5 automation and residential/ Engineering commercial solutions San Research and technology Francisco C6 Research Group Leader consulting Bay Innovation C7 Telecom operator Innovation Manager Consortium

3.4.1.3. Data collection

The interviews were performed in person or by videoconference and lasted from 40 to 90 minutes. The discussions followed a semi-structured qualitative interview process. After performing the first interview using a set of open-ended questions, the corresponding answers were coded into different categories using a methodology suggested by Bryman et al. (2003). A generic framework was then created to reduce the interview duration, improve data relevance and facilitate data analysis (Figure 15). The main concern was to create a framework that is relevant to any innovation process, in order to avoid influencing the interviewee’s answers. The framework was sent to the remaining interviewees prior to the interview to guide the discussions. The framework is divided into four successive sections, each one responding to one of the four interview objectives: 1. After understanding the company and interviewee background, the first section aims at identifying the main types of innovation projects and innovation strategy models present within the company. The framework suggests the two extremes of innovation projects and strategy models, thereby allowing the interviewee to elaborate on potential nuances. 2. The second section investigates the different processes for evaluating early-stage innovation projects prior to development. The framework remains vague by considering the different phases of idea preparation, evaluation and post-process. 3. For each process step, the third section aims at understanding various process specifications (related to people and time), company challenges, as well as evaluation, selection and planning tools. 4. The fourth section does not appear on the framework sent to interviewees in order to avoid influencing their preceding answers. After a short introduction to the research objectives, interviewees are asked the following open-ended question: “What do you think of evaluating early-stage breakthrough innovations using existing company roadmaps and is it applicable to your company?” One of the main limitations of company interviews within large organisations is that the interviewee often does not possess all information regarding the innovation processes within the whole company. Therefore the research only considers the processes that the interviewee is involved with. This ensures greater data validity at the expense of ignoring other important innovation processes.

Background Overview Incremental or Market pull or Company Interviewee 1 Breakthrough Technology push

Tools to evaluate new Tools to select the porLolio of Tools to plan the strategy of innova/on projects new innova/on projects new innova/on projects

Selec/on methods Type Aims Format Type Aims Format Structure Criteria Scaling Granularity Ownership Granularity Ownership Weigh/ngs Uncertainty & Criteria evalua/on Criteria evalua/on risk 3

Challenges Challenges Challenges

Dura/on Frequency People involved

Prepara/on Evalua/on Post process 2

Feedback on: Evalua/ng new projects using exis/ng company roadmaps 4

Figure 15: Interview framework

3.4.1.4. Data analysis

The various sections of the interview framework relates directly to the interview objectives. The data analysis therefore responds to each interview objective successively, by using results from the corresponding framework section. The detailed data analysis is presented in Chapter 4.

3.4.2. Consultant Interviews

Platts (1993) argues that interviewing consultants is important to “provide inputs from consultants’ methods” and “formalize some of the consultants craft skills”. This research interviewed three innovation consultants at several occasions during the design process to gather feedback on the research progress. One of the consultants also helped with workshop facilitation. Table 9 outlines the experience of each consultant and their contribution to this research. There is no separate chapter outlining their contribution since consultant feedback was performed within all development stages.

Table 9: Consultant experience and contribution to the research

Consultant Relevant Experience Research Contribution 1 Portfolio management and strategic Feedback on company interview results technology Feedback on workshop design 2 Academic research and workshop Feedback on workshop design facilitation for technology roadmapping, portfolio and project selection 3 Workshop facilitation for technology Feedback on workshop design roadmapping and project selection Workshop facilitation Post-workshop feedback

3.4.3. Toolkit and workshop design

The toolkit and workshop design include the findings from the literature review, company interviews and consultants in order to synthesize the strategy formulation in a form that can be used by practicing managers (Platts, 1993). In addition, the following sections outline recommendations to design management toolkits and workshops for industrial applications.

3.4.3.1. Toolkit design

Platts (1993) mentions the importance of formulating processes which link to existing frameworks. The research respects this requirement for the toolkit design, by integrating proven tools from the literature. To ensure appropriate tool integration, the toolkit follows the recommendations from Kerr et al. (2013): 1. The toolkit should be human-centric in order to allow their users to participate, engage and collaborate with each other, leading to co-created solutions. 2. The toolkit should be flexible in order to adapt to different company goals. 3. The toolkit should be modular, by integrating the constituent tools and tool outputs with one another. 4. The toolkit should be scalable in order to be employed at different levels within an organisation. 5. The toolkit and corresponding results should have a visual form that allows easy reporting and communication. Section 5.2 outlines the toolkit design in greater detail.

3.4.3.2. Workshop design

The workshop design is performed at two levels. The first level requires the design of a toolkit process. Since the toolkit design is based on existing management tools, the toolkit process incorporates the existing processes of these management tools to construct a step-by-step approach. The second level relates to the management of the workshop before, during and after the toolkit process. The resulting management phases are based on the principles of toolkit-based workshops: 1. Setting up the process: The planning phase aims at defining the objectives and scope of the workshop, identifying the relevant participants, preparing the workshop schedule and performing preliminary work (Phaal et al., 2001). 2. Managing the process: A neutral facilitator should be present to enable group interactions among participants during a combination of plenary and small group activities (Kerr et al., 2013). 3. Following on from the process: At the end of the workshop, participants should agree on future activities, develop a plan to implement the workshop results and integrate the toolkit within the company (Phaal et al., 2001). Section 5.3 outlines each step of the toolkit process and workshop management in detail.

3.4.4. Test the workshop-based toolkit in company settings

The strategy formulation process needs to be tested in companies in order to assess its feasibility and refine the process (Platts, 1993). Due to time constraints, the workshop was only tested within one of the interviewed companies. Company 2 was chosen since they already have technology roadmaps in multilayer and tabular format. Table 10 outlines the test characteristics.

Table 10: Test characteristics

Planning Test characteristics (Company 2) considerations Scope New ideas: Three early-stage breakthrough technology-push ideas from the research department of the company Technology roadmaps: The relevant roadmaps were selected by considering four business units for which the ideas could be beneficial. The twelve selected roadmaps included six multilayer roadmaps and six tabular roadmaps. Company selection factors: The company already had a set of ten selection factors, which were reduced to nine during the preparation phase. People Company Participants: three scientists and two innovation managers Facilitators: Two facilitators, including the researcher and an innovation consultant Schedule The schedule was similar to the one presented in Section 5.3. The definition & preparation only included the three scientists, whereas the workshop involved all participants. Both facilitators were present during all activities.

The first part of the assessment phase aims at understanding the potential of the workshop. Platts (1993) suggests assessing the workshop by considering feasibility, usability and utility: 1. Feasibility: The feasibility is tested by repeating the process in several companies using different facilitators. 2. Usability: The workshop usability is assessed by identifying problems occurring at each stage of the process and understanding how each stage of the process is organized. 3. Utility: The workshop utility is demonstrated at the practical level by assessing the completion of objectives, identifying problems and generating an action plan. The subjective level is assessed through participant feedback. Feasibility did not apply to the research, since only one test was performed. The usability was assessed through a feedback discussion between the two facilitators. The utility was assessed at the end of the workshop using the feedback form shown in Appendix 9.1. Section 6.1 describes the detailed feedback analysis. The second part of the assessment phase aimed at responding more specifically to the research question by investigating the potential of the toolkit to evaluate new ideas. This was performed by analysing the results of the final tool: the idea evaluation template. The detailed analysis is described in Section 6.2.

3.5. Conclusion

Figure 16 shows the five steps of the development process and the corresponding expected outcomes. Each step aims at answering the main research question by considering innovation models, innovation processes and methods & tools. Figure 30 and 31 in the conclusion chapter completes Figure 16 by including the main results.

INNOVATION PROCESSES METHODS AND TOOLS MODELS InnovaDon management EvaluaDng Technology 1. early-stage roadmapping InnovaDon Literature InnovaDon innovaDons Types & Review Processes Models Tools Methods Methods Tools

2. Company COMPANY PRACTICES AND CHALLENGES Interviews

Workshop 3. 4. Relevant Toolkit & Relevant Consultant Process Toolkit Interviews Workshop Models Design stages

5. Workshop REQUIRED DESIGN MODIFICATIONS Results

Reﬁned Workshop Other Other Future Process Reﬁned Toolkit work Models stages

Figure 16: Development process and expected outcomes

4. Interview Results

4.1. Innovation types and strategy models

Figure 17 shows the percentage of market pull versus technology push innovation projects, and incremental versus breakthrough innovations. There is a clear correlation between the type of strategy model and the type of innovation, since the use of a market pull approach showed higher reliance on incremental innovation. All companies relied on both market pull and technology push strategy models to drive their innovation processes.

Figure 17: Percentage of market pull versus technology push models, and incremental versus breakthrough innovations across companies.

4.2. Innovation processes

Table 11 gives an overview of the different processes for evaluating and selecting new innovations within each company. Most companies had several innovation processes, split by the type of innovation strategy model and by their objectives. Each company used different terminologies for each process step and tools. In order to obtain common terminologies, the interviewee’s descriptions were aligned with the descriptions from the literature. Table 12 shows the five process steps and identifies which steps were mentioned within each company process. The selection filter (or strategic review meeting) is a more structured process

35 consisting of five sub-steps. It is important to note that the absence of a step from a particular process may be due to two factors: the process does not include the step or the interviewee did not mention it. Section 4.3 describes how each step is performed using different management tools.

Table 11: Innovation process characteristics

Innovation strategy Company Process Process objectives model

Ensure project funding is relevant for P1.1 Market Pull partners C1 Develop new capabilities to push to P1.2 Technology Push partners Market Pull & Review ideas suggested internally by C2 P2 Technology Push marketing or engineering

P3.1 Market Pull Respond to customer requests C3 Select internal ideas to develop P3.2 Technology Push differentiating projects

C4 P4 Technology Push Select ideas proposed internally

Evaluate an innovation proposed by an P5.1 Technology Push external company C5 P5.2 Market Pull Selection of proposals from marketing

Ensure the development of solutions that P6.1 Market Pull are relevant for clients Market Pull & Develop solutions that respond to internal P6.2 Technology Push challenges Support the development of organic C6 P6.3 Technology Push projects without initial support Develop innovations at the request of top P6.4 Technology Push management

P6.5 Technology Push Select ideas proposed internally

Market Pull & C7 P7 Select ideas proposed internally Technology Push

Table 12: Steps mentioned within each innovation process.

Selection filter (strategic review meeting) Prepare Pre- Project Evaluate Project Allocate Company Process Triage coarse Review Decide GO, selection scoping project voting funding proposals ranked NO-GO or proposals and and projects ON-HOLD (Discussion) ranking resources

P1.1 X X X X X X C1 P1.2 Unstructured Process

C2 P2 X X X X X X X

P3.1 X X X C3 P3.2 X X X X X

C4 P4 X X X X X X X X

P5.1 X X X X X C5 P5.2 X X X X X X

P6.1 X X X X X X

P6.2 X X

C6 P6.3 X X

P6.4

P6.5 X X X X X X

C7 P7 X X X X X

4.3. Requirements related to people, time and tools

The interviewed companies used various management tools in order to evaluate new innovation projects. Table 13 indicates the tools that are used within each process. Although tool formats vary greatly among companies, some similarities were identified (Table 14). Similarly to the process steps, the absence of a management tool from a particular process may be due to two factors: the process does not use the tool or the interviewee did not mention it.

Table 13: Management tools used within each process

C2 P2 PP1, TR ST1, TR TR, PP1, PM ST2

P3.1 PP1 PP2 PP2 ST2 C3 P3.2 PP1 ST1 PP2 PP2 ST2

C4 P4 PP1 ST1, TR PP2, TR PP2, TR ST2

P5.1 ST1 PP2 PP2 ST2 C5 PM, SM, P5.2 PP1 PP2 PP2 ST2 CA

P6.1 PP2 PP2 ST2

P6.2 PP2

C6 P6.3 PP1 ST1

P6.4 PP1

P6.5 PP1 ST1

C7 P7 PP1 ST1, SM PP2 P22 ST2

Table 14: Tool descriptions

Code Tool Description PP1 Coarse proposal Rely mostly on qualitative aspects (P1.1, P1.2, P2, P3.2, P7). PP2 Project scoping Includes estimates of financial valuation such as return on investment (P3.1, P3.2, P4, P6.1, P7). ST1 Pre-selection All companies use different but mostly qualitative selection factors (P2, scoring tool P3.1). ST2 Selection filter Consists mostly of scoring tools without scaling statements (P1.1, P2, P3.2, scoring tool P4, P5.2, P6.1, P7). Selection factors include financial assessments (P3.1, P4, P5.2, P6.1). SM Strategic map High-level and low-level strategy goals of the company (C3, C5, C6, C7). TR Technology Multilayer (C2), tabular (C1) and other formats (C4). roadmap PM Portfolio matrix Visualise all projects together from a specific angle (P1.1, P2, P5.2) CA Competitive Compare competitive product features (P5.2). assessment

Table 13 shows that most of the companies don’t use technology roadmaps (TR) or strategy maps (SM) to prepare or evaluate new project proposals. For some processes however (P1.1, P2, P4, P5.2 and P7), technology roadmaps and strategy maps play a major role in evaluating new innovation projects. Sections 4.3.1 and 4.3.2 explore the different possible ways to prepare and evaluate new project proposals with and without the use of roadmaps. The triage filter is not considered since it does not rely on management tools. Triage relies mostly on quick judgment due to the high number of projects (P2).

4.3.1. Proposal preparation and project scoping:

1. Generic process: P5.2 and P6.1 first rely on the relevant marketing department to prepare coarse proposals on a continuous basis. The later then transfers the proposal to engineers to evaluate the technical aspects and related risks. Project scoping takes up to 8 weeks and is completed on a quarterly or annual basis using expert judgment from various departments (P2, P6.1, P7). The interviewee from Company 7 explains the corresponding process: “Each project proposal is completed by collecting relevant information from across the company. I am well networked and therefore I know all the people within the relevant areas of expertise. I contact them to check whether they already perform the new innovation project, whether the project would add value to their business unit and whether they have concerns or objections regarding the new project.” 2. Use of technology roadmaps / strategic maps: P1.1 and P1.2 use technology roadmaps in each technology section to help identify: • Technology gaps to be met by internally or externally funded research • Competencies and strengths of the company • Industry needs P2 uses technology roadmaps to identify the desirable product attributes required and the kind of capabilities that need to be developed.

4.3.2. Proposal evaluation during pre-selection

1. Generic process: In addition to the pre-selection scoring tool, portfolio matrices are sometimes used to “weed out crazy ideas and know how to allocate funding” (P1.1). The process usually involves the research team and innovation manager for technology push projects (P6.3, P6.4, P4, P5.1, P7). Market pull projects involve sales managers (P3.1), marketing experts (P2, P5.2) and internal sponsors (P6.1, P6.2). The pre-selection takes about 10 minutes per project (P4) and is usually performed on a quarterly basis (P1.2, P6.1, P7). 2. Use of technology roadmaps/ strategic maps: P1.1 compares coarse proposals against technology roadmaps in order to pre-select the projects for the strategic review meeting. Although P2 has no formal process of using roadmaps to evaluate proposals, the decision makers are “well aware of the technology roadmaps” and use this knowledge in their judgments. P7 ensures strategic alignment of the new projects by comparing the coarse proposals to the strategic map. This helps identify the level of impact of the innovation, the market potential and how big of a development is needed. During P4, programme managers filter the projects by judging all selection factors using technology roadmaps. The displayed priority of each technology area reflects its importance to the company in terms portfolio balance, market value and competitor analysis. Therefore, the explicit priorities hide implicit information that are used for proposal selection.

4.3.3. Proposal evaluation during the selection filter

1. Generic process: Similarly to pre-filtering, other evaluation tools are sometimes used with the selection filter scoring tool to facilitate the process (P2, P5.2). The strategic review meeting takes half a day to one day on a quarterly or annual basis and relies on 12 to 25 participants, including senior management (P1.1, P4, P5.1, P6.4, P6.5, P5.2, P2, P6.1, P7, P3.1). 3. Use of technology roadmaps/ strategic maps: Within P1.1, participants have access to roadmaps and project proposals before the meeting, to help participants prepare their voting in advance. P1.1 and P2 gives access to roadmaps during the meeting, however there is no structured process on how to use them. Within P5.2, participants compare the list of prioritised projects to the strategic map of the company to ensure strategic alignment. Similarly to pre-selection, P4 uses the priorities of each technology area to select the best projects during the meeting.

4.4. Workshop limitations and opportunities

Companies saw the potential of the toolkit and workshop to: • “Have a more structured process” (Company 1) • “Help making roadmaps more interactive to show the relationships between the roadmap, the portfolio and the decisions that are made.” (Company 2) • “Integrate the different selection factors to the roadmap.” (Company 2) • “Improve the consistency of our tools and selection factors” (Company 5) In addition, Table 15 outlines the interviewees’ feedback on using existing roadmaps for evaluating early-stage innovations. Table 15: Interviewee feedback

Company Feedback C1 Using company roadmaps to evaluate new projects during the meeting would demand too much from the committee members. C2 Roadmaps are very detailed and very complex, therefore we are struggling to make them truly interactive C3 The roadmap needs to incorporate the diversification strategy to be helpful with project evaluation C4 New ideas should be evaluated from explicit and implicit roadmap information C6 The next step would be to digitize roadmaps: • The selection factors and the text should be searchable • For each new idea, the software would show the alignment with the roadmap, supporting arguments and counter-arguments C7 The workshop should be performed at an early stage to check whether another part of the company is performing the same work already. The roadmap needs to be updated regularly to be up to date at all levels, which may be very challenging.

4.5. Interview conclusions

The interviews helped narrow down the innovation model and process stage foci (Table 16), and gave design considerations for the workshop-based toolkit (Table 17).

Table 16: Interview conclusions for innovation model and processes

Interview conclusions

The market pull approach is more reliant on incremental Innovation model innovations than the technology push approach.

There are too many projects to use management tools Triage (C2).

Coarse proposal The workshop should be performed at an early stage to preparation check whether a similar project is already ongoing (C7).

C1, C2, C4, C7 already use information from existing technology roadmaps to accelerate proposal evaluation. Pre-selection Processes The pre-selection takes 10 minutes per project (P4)

C1 and C2 already use existing technology roadmaps to accelerate the proposal preparation. Project scoping The generic process for preparing project proposals takes up to 8 weeks (P2, P6.1) and requires the collection of information from across the company (C7).

Performing the workshop during the selection filter would Selection filter demand too much from participants (C1).

Table 17: Interview conclusions for methods & tools

Interview conclusions All companies use scoring tools, but without Scoring tool scaling statements. C4 uses implicit roadmap information (priorities and technology readiness levels) to evaluate all selection factors. Companies use both strategic planning and product planning purposes. Roadmapping Companies use multilayer (C1) and tabular formats (C2), as well as other formats (C4). Methods & Tools Roadmaps are too complex and not interactive (C2).

All companies use varying and inconsistent Idea proposal proposal formats.

Strategy map, portfolio Companies use strategic maps, portfolio matrices matrices and competitive and competitive assessments to create and assessments evaluate new project proposals (C1, C2, C5).

Software tool The next step would be to digitise roadmaps.

5. Toolkit and Workshop Design

5.1. Design considerations

5.1.1. Innovation model

The literature review considered early-stage breakthrough innovations and showed that most companies use a combination of market pull and technology push strategy models to drive their innovation processes and stay competitive. Company interviews then showed that incremental innovations are mostly developed using a market-pull innovation process whereas breakthrough innovations are usually developed using a separate technology-push process. The workshop aims at bridging the gap between the two strategy models by evaluating technology-push early-stage breakthrough innovations using a market-pull approach.

5.1.2. Processes

The literature review identified five main innovation process steps for early-stage innovations. The interview conclusions then showed that roadmaps were already used or had potential to evaluate ideas during the coarse proposal preparation, pre-selection and project scoping steps. On the other hand, the workshop was seen as unsuitable for the triage due to the number of projects to analyse, and impractical during the selection filter meeting. Therefore, the workshop focuses on the pre-selection step, and extending to coarse proposal preparation and project scoping (Figure 18). The pre-selection step evaluates projects in a short timeframe whereas project scoping is usually performed over several weeks. It is therefore assumed that a one-day workshop would meet industry requirements.

Coarse Pre- Project Selec0on Triage proposal selec0on scoping ﬁlter

Figure 18: Process focus

5.1.3. Methods & Tools

The literature review identified the scoring model as the most appropriate tool to evaluate early-stage innovations. It also showed that technology roadmaps are used in combination with other evaluation tools during the initiation & development phases, and that the integration toolkit consists of quality assessments and roadmapping software. In addition, the research method outlined generic design considerations for both the toolkit and the workshop. Interviews then showed that scoring tools, roadmaps and idea proposals are commonly used among companies, however with varying purposes and formats. Since some companies already use their existing roadmaps to evaluate new ideas, their feedback and practices were incorporated in the toolkit design. A linking grid was chosen as the central tool of the toolkit and a quality assessment template was incorporated. Scorings related to quality, priority and timeframes were added to the linking grid in order to give a dimension of importance to each driver, product and technology/ resource. Finally, the amount of roadmap information was reduced by narrowing down information from one layer to the next (Figure 19).

Drivers Relevant drivers Products Relevant products Tech/ Res Relevant Tech/Res

Figure 19: Narrowing down of information from one layer to the next.

5.2. Toolkit design

5.2.1. Tools

The toolkit incorporates six management tools to evaluate new innovation projects from existing technology roadmaps. Table 18 shows descriptions of each tool in terms of its aims, structure, inputs, process, outputs and the inspiration source from the literature. Figures 32 to 37 in the Appendix show the corresponding representations of each tool.

Table 18: Tool descriptions 2014) 2014) 2001) 2004) 2003) 2002) et al., al., et et al., et et al., et et al., al., et et al., al., et et al., et (Mitchell (Mitchell the purpose of this purpose of this the (Phaal (Phaal (Lee (Lee et al., 2015); (Lindsay, (Lindsay, 2015); al., et Literature basis Literature (Farrukh 2003) (Albright, (Cooper (Phaal (Kerr 2001); al., et (Phaal for was created tool The 2000) workshop 3. Technologies & resources resources & Technologies 3. roadmap 2. Product features with with features 2. Product priorities statements for each selection selection for each statements on a score from 1 to 3 1 to from score on a 2. Opportunity & feasbility feasbility & 2. Opportunity factors for opportunity and and for opportunity factors each roadmap layer, normalised normalised layer, roadmap each scores for each new idea scores for each across layers feasibility factor Outputs of the description Multilayer for quality information Total selection of generic 1. List corresponding 1. Drivers with idea each between 1. Synergies that factors of selection 1. List 2. List of selection factors that that factors of selection 2. List 4. Interdependencies within and and within 4. Interdependencies new idea in a way that reflects reflects way that a in new idea and roadmap layer roadmap and can be evaluated from the the from evaluated be can with corresponding timeframes corresponding with cannot be evaluated from the the from evaluated be cannot 2. List of five scaling scaling of five 2. List roadmap corresponding timeframes corresponding the roadmap structure roadmap the to roadmapping initiation for more for more initiation roadmapping to of the idea by answering the heading heading the by answering idea of the of each roadmap layer using a score score using a layer roadmap of each 2. Explain what was missing from wasfrom what missing 2. Explain quality) details on process details completeness and understandability understandability and completeness from 0 to 12 0 to from for each company factor company for each the roadmap to properly evaluate the the evaluate properly to roadmap the factor Process knowledge current the Incorporate technology the at By looking factor selection generic 1. Choose a not are roadmaps technology The in process is explained detailed The new idea: For each from 1 (low quality) to 3 (high 3 (high to 1 (low quality) from roadmaps, assess the validity, assessvalidity, roadmaps, the evaluate each selection factor selection each evaluate statement corresponds to a score score a corresponds to statement of each row in a language that can can that language rowa in of each to statements Assign scaling five 2. participants each generic factor. Each scaling scaling Each factor. generic each they belong to the company. Refer Refer company. the to belong they be understood by all workshop by all understood be designed within the workshop the since within designed Workshop design Workshop 1. Explain how the roadmap helped helped roadmap how the 1. Explain product features and and features product new idea 4. Relevant drivers, drivers, 4. Relevant and understandability of understandability and belong to the company. company. the to belong on inputs 2. Opportunity & & 2. Opportunity 2. Total quality score for score quality Total 2. the workshop since they workshopthey the since selection factors for factors selection coarse proposals coarse initiation for more details details for more initiation technologies/ resources, resources, technologies/ with corresponding corresponding with each roadmap layer roadmap each Inputs of the knowledge Current roadmaps Technology 1. of relevant list Company roadmaps technology The titles 1. Idea each between 1. Synergies priorities 3. Generic selection selection 3. Generic idea and roadmap layer roadmap and idea Refer to roadmapping roadmapping to Refer factors the roadmap information roadmap the idea, including previous previous including idea, 2. Knowledge of the of the 2. Knowledge are not designed within within designed not are opportunity and feasibility and opportunity feasbility scores for each scores for each feasbility validity, completeness, completeness, validity, validity, completeness and and completeness validity, scaling statements. scaling and two columns evaluating evaluating two columns and feasibility factors feasibility for each roadmap layer roadmap for each of each layer's information information layer's of each What, Why, When, ...) When, Why, What, quality scores and priorities priorities scores and quality equivalent) displayed on a on a displayed equivalent) the potential and and potential the suggestions of generic suggestions of generic 3. Scoring of each new idea new idea of each 3. Scoring Structure of six rows composed Table main on the based Table giving tables Two format or tabular Multilayer Grid incorporating: of the composed Table 2. Synergies between each each between 2. Synergies new idea and roadmap layer roadmap and new idea is judged in terms of terms in is judged 1. Roadmap numbers, numbers, 1. Roadmap based on opportunity and and on opportunity based Product, Technology (or Technology Product, time axis. time shortcomings of roadmaps of roadmaps shortcomings to evaluate the new idea the evaluate to with three layers: Driver, Driver, layers: three with different selection factors factors selection different that reflect the technology technology the reflect that roadmap layers. The quality quality The layers. roadmap opportunity and feasibility feasibility and opportunity roadmap structure (How, (How, structure roadmap factors; each chosen factor factor chosen each factors; corresponds to a row of five row a of five corresponds to understandability evaluate new ideas evaluate company in order to to order in company factors, and corresponding corresponding and factors, strategy plan of the of the plan strategy the selection factors factors selection the understandability layers in terms of validity, of validity, terms in layers roadmaps helped evaluate evaluate helped roadmaps synergies roadmap information to to information roadmap retrieve relevant relevant retrieve Aims of description a Provide Assess information the & Choose opportunity Show innovation the proposals, idea Integrate how Understand the the new ideas based on based new ideas the quality of all roadmap roadmap of all quality identify roadmap roadmap identify completeness and and completeness feasibility selection selection feasibility information selection factors and and factors selection scaling statements statements scaling roadmap assessments, roadmap each new idea to easily easily to new idea each assessment assessment roadmap template template template quality quality Tool proposal Idea Roadmap Selection Technology grid Linking evaluation Idea factors template factors 1 2 3 4 5 6 #

5.2.2. Toolkit

All six management tools are used as part of a management toolkit, where the outputs of one tool correspond to the inputs of another tool (Figure 20). The idea proposal, roadmap assessment and selection factors templates are completed during the preparation & definition to provide direct inputs to the linking grid. Workshop parts 1, 2 and 3 incorporate roadmap information in the linking grid to evaluate synergies between each idea and each roadmap layer. Finally, workshop part 4 evaluates each idea based on the chosen opportunity and feasibility factors, thereby providing the inputs required for the idea evaluation templates.

ROADMAP QUALITY IDEA PROPOSAL ASSESSMENT TEMPLATE Prepara)on & Deﬁni)on TEMPLATE R1 R2 … VALIDITY IDEA TITLE DRIVERS COMPLETENESS SELECTION FACTORS TEMPLATES WHAT UNDERSTANDABILITY TOTAL PRODUCTS & VALIDITY SERVICES PRODUCTS COMPLETENESS WHY UNDERSTANDABILITY HOW TOTAL VALIDITY

WHEN TECH/ RES COMPLETENESS UNDERSTANDABILITY TOTAL

Workshop Workshop LINKING GRID Part 1 SELECTION FACTORS Part 4 DRIVERS PRODUCTS TECH/ RES Part 2 O F Part 3

ROADMAP QUALITY R1 R2

PRIORITY … TOTAL TOTAL TOTAL OPPORTUNITY TOTAL FEASIBILITY TOTAL I1 I2

IDEAS …

TECHNOLOGY ROADMAP IDEA EVALUATION TEMPLATE IDEA R# 0-2Y 2-6Y 6-10Y 10+Y SELECTION ROADMAP MISSING FACTOR INFORMATION INFORMATION DRIVERS

PRODUCTS

TECH/ RES

Figure 20: Toolkit overview

5.3. Workshop design

5.3.1. Setting up the workshop

Table 19 shows how each planning aspect from section 3.4.3.2 is applied in the context of the workshop. Table 19: Workshop aspects and considerations

Aspect Workshop considerations Objectives Evaluate new ideas using existing technology roadmaps Outcomes Evaluation of new ideas and initial project scoping Opportunities for improving the company’s technology roadmaps Scope Identify the ideas, technology roadmaps and company selection factors to consider People Company participants should have the right mix of management and technical backgrounds relevant to the ideas. The number of participants should reflect the number of ideas to consider. Facilitators should be external to the company and have previous experience with innovation consulting in the relevant industry. Schedule The workshop is performed during an entire working day, although the schedule may vary depending on the number of participants and ideas to consider. Appendix 9.3 shows the corresponding workshop timeline. Preparatory The templates, management tools and relevant roadmaps should be printed work beforehand in large formats. A second copy of the linking grid template should be available in a spreadsheet in order to perform the calculations. The participants should be contacted beforehand and given a briefing of the workshop process.

5.3.2. Workshop

The workshop consists of five main steps (Figure 21). The main workshop inputs are the company roadmaps, new ideas and company selection factors. The main workshop outputs are the opportunity-feasibility diagram and the completed idea evaluation template. Each step is explained in the following sections.

Company roadmaps O-F diagram Part 2: Part 3: Part 4: Deﬁni&on & Part 1: New ideas Product Technologies Idea Prepara&on Drivers Features & Resources Evalua&on Idea Company Evalua&on selec&on Template factors Figure 21: Workshop process

Each step uses the linking grid to insert or retrieve information. Figure 22 shows the different linking grid sections as referred in the step descriptions.

SELECTION FACTORS DRIVERS PRODUCTS TECH/ RES O F

D1 P1 T1

ROADMAP D2 P2 T2 O1 F1 QUALITY D3 P3 T3 R1 R2 D4 P4 T4

PRIORITY … TOTAL TOTAL TOTAL OPPORTUNITY TOTAL FEASIBILITY TOTAL I1 I2 D5 D6 P5 P6 T5 T6 O2 F2 O3 F3

IDEAS … Figure 22: Linking grid sections

5.3.2.1. Definition & Preparation

Aims: The main aims of the definition & preparation step is to process the information provided by the company (new ideas, technology roadmaps and selection factors) into useful workshop inputs.

Process: Figure 23 shows the process used during the Definition & Preparation step. Before starting the preparation, the workshop boundaries need to be defined by selecting the relevant new ideas, existing roadmaps and selection factors within the company. These three inputs are then used to perform three consecutive activities: 1. Complete an idea proposal template for each new idea and include the title of each idea in the linking grid (I1, I2, …). The contents may be based on previous coarse proposals from the company. 2. Assess the quality of each roadmap layer using assessment templates. 3. Choose generic selection factors and corresponding scaling statements using the selection factors template. The generic selection factors may be based on a list of factors given by the company. Include the obtained generic factors for opportunity and feasibility in the linking grid (O1 and F1).

People: The participants may be divided into teams to accelerate the process. Each team would then focus on fewer ideas, roadmaps and selection factors, but the results must be crosschecked at the end of each activity.

Deﬁne workshop Prepara3on boundaries

Idea Proposal Template Include ideas New ideas (Why, What, in linking grid How)

Exis@ng Roadmap Assess Assessment roadmap roadmaps Template quality

Choose Choose Company’s generic scaling selec@on selec@on statements factors factors

Figure 23: Definition & Preparation

5.3.2.2. Workshop part 1: Drivers

Aims: The “Drivers” part aims at evaluating the synergies between new ideas and the driver layer of the technology roadmaps.

Process: Figure 24 shows the process used during the “Drivers” part: 1. Select the highest priority drivers from all the relevant technology roadmaps. To ensure consistency between the different roadmaps, assign a scale from 1 (lowest priority) to 3 (highest priority) to all selected drivers. Also remove any duplicates among drivers from different roadmaps to ensure that each selected driver is unique. 2. Insert these drivers, their corresponding priorities and the number of their roadmap in the linking grid (D1, D4 and D2 respectively). 3. Evaluate synergies between each new idea and all selected drivers in the linking grid (D5). The synergies are evaluated using integers from -3 (the new idea has a highly adverse effect on the driver) to +3 (the new idea has a highly positive effect on the driver). A score of 0 indicates that there are no synergies between the new idea and the driver. If the synergy is hard to assess or if the definition of one driver is unclear, participants can look back at the roadmaps. Participants should base their decisions on roadmap information as much as possible. 4. The previous activity helps identify the drivers with highest synergies (score of +3 for at least one new idea).

People: The participants may be divided into teams, each one responsible for evaluating the synergies of one new idea. Participants from management and marketing backgrounds should be present in all teams due to their in-depth knowledge of business drivers. The obtained synergies must be crosschecked between teams to ensure consistency and remove bias as much as possible.

Deﬁne Workshop Part 1: workshop Prepara3on Synergies between ideas and drivers boundaries

Idea Include Proposal New ideas in Template linking ideas (Why, grid What, How)

Evaluate Drivers synergies with Insert between highest Highest drivers, ideas and synergy priority priori5es and drivers (+3) drivers roadmap # in (-3 to +3) linking grids

Exis5ng roadmaps

Roadmap Assess Assessment roadmap Template quality

Choose Choose Company’s generic scaling selec5on selec5on statements factors factors

Figure 24: Workshop Part 1

5.3.2.3. Workshop part 2: Product Features

Aims: The “Product Features” part aims at evaluating the synergies between new ideas and the product layer of the technology roadmaps.

Process: Figure 25 shows the process used during the “Product Features” part: 1. The “Drivers” part identified the drivers with the highest synergies. Gather the roadmaps where these drivers are displayed. In each product layer, identify product features that are relevant for at least one new idea. 2. For each relevant product feature, identify the displayed timeframe at which the feature will become mature enough for the new idea. Remove any duplicates among product features. Assign a priority score that reflects the future timeframe: • Less than 2 years = priority 3 • Between 2 and 6 years = priority 2 • Between 6 and 10 years = priority 1 • More than 10 years = priority 0.5 • Not shown = priority 0 3. Insert the relevant product features, their corresponding priorities and the number of their roadmap in the linking grid (P1, P4 and P2 respectively). 4. Evaluate synergies between each new idea and all relevant product features (P5) using the same synergy scores as for the drivers (-3 to +3). Participants may also use the roadmaps in their decision process. 5. The previous activity helped identify the product features with highest synergies (score of +3 for at least one new idea).

People: The participants may be divided into teams, each one responsible for evaluating the synergies of one new idea. The knowledge of the company’s products should be split equally among teams. Again, the obtained synergies must be crosschecked between teams.

Deﬁne Workshop Part 2: workshop Prepara

Idea Include Proposal New ideas in Template linking ideas (Why, grid What, How) Evaluate synergies Product Drivers Insert product between features with Iden%fy Iden%fy features, ideas with highest relevant relevant priori%es and highest synergy %meframe product and product synergy (+3) and assign features roadmap features (+3) priority # in linking (-3 to +3) grids Exis%ng roadmaps

Roadmap Assess Assessment roadmap Template quality

Choose Choose Company’s generic scaling selec%on selec%on statements factors factors

Figure 25: Workshop Part 2

5.3.2.4. Workshop part 3: Technologies & Resources

Aims: The “Technologies and Resources” part aims at evaluating the synergies between new ideas and the technology & resource layer of the technology roadmaps.

Process: Figure 26 shows the process used during the “Technologies & Resources” part: 1. The “Product Features” part identified the product features with the highest synergies. Gather the roadmaps where these product features are displayed. In each technologies & resources layer, identify technologies & resources that are relevant for at least one new idea. 2. For each relevant technology or resource, identify the timeframe at which it will become mature enough for the new idea. Remove any duplicates and assign a priority score (0 to +3) that reflects the future timeframe.

3. Insert the relevant technologies & resources, their corresponding priorities and the number of their roadmap in the linking grid (T1, T4 and T2 respectively). 4. Evaluate synergies between each new idea and all relevant technologies & resources (T5) using the same synergy scores as for the drivers (-3 to +3). Participants may also use the roadmaps in their decision process.

People: The participants may be divided into teams, each one responsible for evaluating the synergies of one new idea. Participants from technical background should be split equally among teams and the obtained synergies must be crosschecked.

Deﬁne Workshop Part 3: workshop Prepara=on Synergies between ideas and technologies/ resources boundaries

Idea Include Proposal New ideas in Template linking ideas (Why, grid What, How) Product Evaluate features synergies Insert between with Iden%fy Iden%fy technologies/ highest relevant relevant ideas and resources, synergy technologies %meframe technologies priori%es and / resources (+3) and and assign roadmap # in (-3 to +3) resources priority linking grids

Exis%ng roadmaps

Roadmap Assess Assessment roadmap Template quality

Choose Choose Company’s generic scaling selec%on selec%on factors statements factors Figure 26: Workshop Part 3

5.3.2.5. Workshop part 4: Idea Evaluation

Aims: The “Idea Evaluation” part aims at evaluating each new idea based on the chosen selection factors.

Process: Figure 27 shows the process used during the “Idea Evaluation” part: 1. The roadmap assessment template helped identify total quality scores (1 (low) to 3 (high)) for each layer of each roadmap. Insert these total quality scores in the linking grid (D3, P3 and T3). 2. The linking grid now includes: • The “synergies” between the new ideas and each roadmap layer (-3 to +3) • The “priority” of each driver, product feature and technology/ resource (0 to +3) • The “quality” of each driver, product feature and technology/ resource (0 to +3) 3. Multiply these inputs to calculate the total synergy score between each idea and each layer (-3 to +3) using the excel spreadsheet (D6, P6, T6). 4. Evaluate each idea on all selection factors, by using the score (0 to 12) corresponding to each scaling statement. The participants should look at the total synergy scores and other numbers from the linking grid to help with their decisions. The final scores can be displayed on an opportunity-feasibility diagram. 5. Complete an idea evaluation template for each new idea by explaining how each selection factor could or could not be evaluated from the roadmaps.

People: The participants should perform the scoring of each idea individually. Results must then be crosschecked to ensure consistency. The idea evaluation templates may be completed by splitting participants into teams, each one responsible for one idea.

Deﬁne Workshop Part 4: workshop Prepara2on Evalua2on of each idea using selec2on factors boundaries

Synergies between Idea Include Ideas Proposal New ideas in AND Template linking Drivers ideas (Why, grid Product features What, How) Technology/ Resource (-3 to +3) TOTAL Synergy Complete score Evaluate Priority of each an idea between each idea Driver evaluaKon Each Idea on all Product feature template AND factors Technology/ for each Each Layer (0 to 12) Resource idea (0 to +3) (-3 to +3) ExisKng roadmaps Quality of each roadmap at: Insert Driver layer Roadmap Quality Product layer Assessment in Grids Template Tech/ Res layer (0 to +3)

Choose Choose Idea Company’s generic scaling EvaluaKon selecKon selecKon factors statements Template factors Figure 27: Workshop Part 4

5.3.4. Following on from the workshop

As shown in Figure, the main outputs of the workshop are the idea evaluation templates. However, these templates are not an end goal. Instead, the idea evaluation templates provide inputs for initial project scoping and improving the technology roadmaps (Figure 28).

WORKSHOP

Idea Understand more about each Project proposals idea’s poten3al Idea scoping evalua3on Company Understand what important template Modify roadmaps informa3on is missing roadmaps

Figure 28: Workshop process steps to create project scopes and modify roadmaps.

5.3.4.1. Initial project scoping

Once the ideas are evaluated, the best ideas can be selected to initiate their corresponding project scoping and subsequent thorough proposal. In order to facilitate the process, the project scoping should include categories that relate directly to the selection factors from the workshop. During the workshop process, participants unintentionally scope the ideas by understanding how they relate to each selection factor and to the innovation strategy of the company. The idea evaluation template shows how the roadmap information helped evaluate each selection factor, thereby helping participants to create more detailed project scopes. Selection factors that could not be judged from the roadmaps should be investigated using other methods until the technology roadmaps are improved.

5.3.4.2. Opportunities for improving the technology roadmaps

The selection factors that could not be evaluated present opportunities for improving the technology roadmaps, by incorporating the missing but vital information during the next roadmap update.

6. Workshop results

6.1. Workshop feedback

Feedback results showed that participants saw the workshop as highly valuable for the team and the company. They felt that the overall process met the objectives of the workshop, but that modifications are required. In general, participants were motivated during the workshop and worked well with each other and with the various tools (Figure 29). Table 20 shows the rankings from participants and required modifications for each process step. The comments and required modifications were extracted from participant and consultant (2nd facilitator) feedback. The easiest and most valued part of the process was choosing selection factors and scaling statements. This step was facilitated by an experienced consultant, who performed this task before within other companies. The most difficult part of the process was to assess the roadmap quality due to the large amount of roadmaps and the limited knowledge of the participants in terms of roadmap validity and completeness. Table 21 shows the ranking from participants and the required modifications related to people and time. The main limitation was the absence of portfolio decision makers among the participants. Their presence during future workshop testing may lead to more conservative outcomes.

Figure 29: Participants engaging during the workshop

Table 20: Participant feedback for each process step

Ranking Process Activity from Required modifications step participants Understand how existing Develop roadmaps by keeping in mind Overall roadmaps can be used to this evaluation process 3.7 Process evaluate new transformative Standardise the roadmaps and keep ideas them up to date Better explanation of what each section requires Complete idea proposal 3 Write and distribute idea proposals to Definition participants beforehand and Reduce the number of roadmaps or Preparation Assess roadmap quality 2.3 use integrated roadmaps instead Choose selection factors and Ensure the selection factors are 4 scaling statements agreed by decision makers Workshop Identify synergies between 4 Part 1 ideas and drivers Better definition of synergy and the Workshop Identify synergies between 3 related scoring Part 2 ideas and product features Allocate more time for workshop part Identify synergies between Workshop 1 than for the other parts ideas and technologies/ 3.7 Part 3 resources More facilitation is required to help Evaluate each idea using 4.3 participants identify relevant Workshop selection factors information Part 4 Complete idea evaluation 3.3 _ templates Table 21: Participant feedback regarding people and time

Ranking Dimension Factor from Required modifications participants Perform the workshop with a bigger group to include more knowledge Good composition of the team 4 Include decision makers with more conservative views People Everybody could participate 4.7 _ The company could do the process without outside 4 _ involvement The process was a fast way to 3.7 _ evaluate ideas Time There was enough time to Allocate more time to discuss 4 discuss important issues important issues

6.2. Potential of the toolkit to evaluate new ideas

Table 22 summarises the outcomes of the idea evaluation templates based on nine selection factors. Results from the three ideas were grouped together to show how participants used existing technology roadmaps to evaluate each selection factor. The roadmap information used by participants came from all three layers of the roadmap (drivers, products and technologies/resources). Three types of information were identified: 1. “Synergy” refers to a driver, product or technology/resource that displayed a synergy with the new idea. 2. “Gap” refers to a driver, product or technology/resource that is not considered in the innovation strategy of the company, and is therefore not on the roadmap. 3. “Unknown” refers to missing information that is necessary in order to properly evaluate the selection factor. Table 22 shows how each synergy and gap was used to evaluate each selection factor. The presence (synergy) or absence (gap) of a driver, product or resource/technology in the roadmap helped determine whether the idea positively (green) or negatively (red) responded to each selection factor. The corresponding statements show that a synergy does not always lead to a positive judgment. Similarly, a gap does not always lead to a negative judgment. In fact, the positive or negative impact of a synergy or gap entirely depends on the selection factor. For example, strategic fit is positively related to driver synergies, whereas diversification is positively related to driver gaps. The main limitation is that judgments are solely based on assumptions related to the presence or absence of information. For some selection factors such as “differentiate market position vs. competition”, participants even recognised the impossibility to evaluate the idea based on the roadmap. This is due to missing competitor information within all layers of the roadmap. Therefore the granularity of the roadmap does not allow to properly evaluate ideas on all selection factors. Roadmaps require modifications to include the “unknown” information in order to become useful at evaluating new ideas. The impact of roadmap quality and driver/product/technology priority on idea evaluation could not be assessed since participants did not take into account quality or priority dimensions to perform their evaluation process. This may be due to the lack of guidance from the tool or from the facilitator.

Table 22: Results from idea evaluation templates

Information Roadmap Layer Adapted generic factor type Drivers Products Resources/ Technologies Idea enhances customer Enhance position Synergy segment Idea extends product line in customer Idea is not related to Idea presents an Idea presents an segment Gap existing customer segments opportunity to innovate opportunity to innovate Unknown Customer needs Details on products Idea responds to customer Idea provides a solution to Synergy need existing customers Solution to Idea does not respond to Idea enters an unexploited customer need Gap customer need customer niche Details on products in the Details on technologies in Unknown long term the long term Idea has potential financial Synergy Idea has a potential value gain Increase revenue Gap Opportunity Amount of potential Unknown customers Idea uses existing Idea uses existing Learning potential Synergy competencies competencies and attract Idea relies on absent Idea relies on absent industry talent Gap competencies competencies Unknown Company reputation Synergy Idea encourages Idea encourages Idea encourages Diversification Gap diversification diversification diversification Unknown Idea strengthens Idea strengthens Differentiate Synergy competitive advantage competitive advantage market position Idea encourages Idea encourages VS. competition Gap competitive disadvantage competitive disadvantage Unknown Competitors Products from competition Patents from competitors Idea uses current and Leverage technical Synergy future technical capabilities capability Gap Idea relies on missing skills Unknown Idea should be financially

Feasibility Synergy supported Availability of Uncertainty about financial finance Gap availability Company financial Unknown commitments

Align with Synergy Idea is strategically aligned strategy Gap Unknown

7. Conclusion and Future Research

7.1. Conclusion

The conclusion responds to the research question “How to use existing technology roadmaps to evaluate early-stage breakthrough innovations?” A workshop-based toolkit was developed through five successive stages: literature review, company interviews, consultant interviews, toolkit & workshop design and testing in company settings. The first four stages identified and incorporated relevant innovation models, processes and methods & tools. The research question was then validated through the testing phase. The main design-related results are shown in Figures 30 and 31 (see end of chapter).

Innovation models: The final design of the workshop-based toolkit evaluates technology-push early-stage breakthrough innovations using a market-pull approach in order to bridge the gap between the two strategy models.

Processes: Based on company feedback, the workshop is performed during the pre-selection phase, and extends to coarse proposal preparation and project scoping.

Methods & Tools: The literature helped identify relevant scoring models, roadmapping evaluation tools and integration tools. Company interviews showed that these tools are widely used in industry and that some companies already use their existing roadmaps to evaluate new ideas on an ongoing basis. Their feedback helped choosing and further refining the tools to become practical for industry use. The toolkit is based on six management tools. The linking grid is the central tool and helps visualise synergies between new ideas, roadmap information and selection factors. Scorings related to quality, priority and timeframes are added to the linking grid in order to give a dimension of importance to each driver, product or technology/ resource. The linking grid focuses on relevant roadmap data by narrowing down information from one layer to the next.

The workshop process is performed as follows: • Setting up the workshop involves defining the objectives, outcomes and scope, identifying the relevant people and performing preparatory work. • The workshop itself includes five phases: o The idea proposal, roadmap assessment and selection factors templates are completed during the definition & preparation phase to provide direct inputs to the linking grid. o Workshop parts one, two and three incorporate roadmap information from all three layers into the linking grid in order to evaluate synergies with each idea. o Finally, workshop part four evaluates each idea based on the chosen opportunity and feasibility factors, thereby providing the inputs required to complete the idea evaluation templates. • Following on from the workshop, the idea evaluation templates help prepare project scopes for the selected ideas and provide directions for roadmap modifications.

Potential of using existing roadmaps to evaluate new ideas: Testing the design in company settings helped to answer the research question, by assessing the workshop feasibility and by analysing the results from the idea evaluation templates. The workshop feedback from participants and the second facilitator showed that the workshop was seen as highly valuable and engaging, but that some modifications were required and should be considered in future work. The results from the idea evaluation templates responded to the main research question by identifying how the existing roadmaps helped evaluate each idea. Three types of roadmap information were identified: “synergy”, “gap” and “unknown”. The statements from evaluation templates identified how “synergies” or “gaps” within each roadmap layer can help evaluate each selection factor. Results showed that some selection factors were easier to evaluate than others, depending on whether the roadmap displayed all the necessary information or whether participants had to make assumptions. For some selection factors, participants even recognised the impossibility to evaluate the idea based on the roadmaps. This was due to “unknown” information that was missing from all roadmap layers. Roadmaps therefore need to include “unknown” information responding to each selection factor in order to become useful at evaluating new ideas. The influence of quality and priority dimensions on idea evaluation could not be assessed due to a lack of guidance, and should be considered for future testing.

7.2. Research Limitations

This thesis has limitations related to the research method as well as to each step of the development process.

First, the goal of the research is limited to testing the feasibility of the strategy formulation process, not understanding whether the formulation process improves the performance of the company (Platts, 1993). Second, the chosen research method does not have the same rigour as scientific testing. Since the tester is not independent of the test, the later is not repeatable. However, Platts (1993) argues that this limitation does not make it any less valid for the type of research undertaken. Managers require practical results, therefore they are more interested in development and improvement rather than the “more static testing required by the principles of physical science” (Platts, 1993).

The literature review relied mostly on the methods and tools developed within the University of Cambridge. This is due to the affiliation of the researcher with the university and the related practical implications. The research attempted to overcome this limitation by performing comparisons with other methods and tools such as the ones developed by Cooper et al. (2001). The company interviews have limitations related to the sample size and the terminology used. Only seven companies were interviewed in total. Each one had different innovation processes and only three companies had technology roadmaps. In addition, the terminologies used to describe innovation types, tools, methods, processes and people varied greatly between companies. The presence of so many variables within a small sample size may question the validity of the comparisons. The toolkit and workshop design did not incorporate all of the considerations from the company interviews (including other methods and tools) in order to keep the workshop simple and short. The omitted considerations should therefore be incorporated and tested during future design iterations. Finally, the testing of the workshop-based toolkit had limitations related to the involvement of the researcher and the generalizability of results. Since the researcher was also a facilitator, it is unclear whether the results were influenced by his involvement in the workshop. In addition, the workshop was only performed once. Further testing is required to assess the generalizability of results.

7.3. Opportunities for Future Research

Future research should consider the continuation of the development process and the validation of the complete process framework.

The next design and test iteration should incorporate the main modifications suggested by every stage of the development process, as shown in Figures 30 and 31 (see end of the chapter). This includes extending the process focus to create coarse proposals and project scoping. The development process should then follow the approach from Platts (1993) by testing the workshop in various company settings. The researcher should “modify the process in light of experience” and “test its feasibility in different situations” in order to improve robustness and usefulness (Platts, 1993). This will help to identify the workshop activities that are crucial to evaluate new ideas and the ones that are optional, in order to apply the toolkit in a lightweight manner and allow customization (Kerr et al., 2013). In addition, further testing should identify the information required to evaluate each selection factor, thereby helping to design an optimal roadmap format. As the final development stage, Platts (1993) suggests investigating the wider applicability of the workshop, by performing a survey among users and non-users. Results from the survey will help to perform more quantitative analysis, thereby evaluating the effect of company characteristics on process use.

This research aimed at validating the first step of the suggested process framework by investigating the evaluation of new ideas using existing strategy plans (section 3.1). In order to validate the complete framework, future research will need to consider the development of new project plans and their integration into the innovation strategy. This can be performed through in-depth action research in companies over several years, thereby ensuring the successful integration of the continuous process.

7.4. Theoretical implications

Methods and tools are widely available to evaluate and prioritise new opportunities that arise during technology roadmapping. However, little guidance is given on using the developed roadmaps to evaluate new innovations on a continuous basis. This research contributes to the body of knowledge by providing methods & tools to evaluate and scope new ideas using existing technology roadmaps.

7.5. Practical implications

Practicing innovation managers are provided with a workshop-based toolkit to ensure that selected innovations are consistent with strategic goals and future company developments. The workshop enables the systematic evaluation and prioritisation of breakthrough early- stage ideas. In addition, workshop outcomes help scope the selected ideas and update roadmaps with important missing information.

INNOVATION MODELS PROCESSES Innova&on management Innova&on Innova&on Processes Types & Models 1. Literature Early-stage & Coarse Pre- Project Selec&on Review Breakthrough Triage proposal selec&on scoping ﬁlter Market pull & Technology push are important

Too many Roadmaps Separate processes projects already used Imprac&cal for 2. T-push & M-Pull Company Poten&al Roadmaps Interviews T-Push linked to already used breakthrough innova&ons Long current process 3. Consultant 4. Market pull Interviews Toolkit & evalua&on of Coarse Pre- Project Selec&on Triage Workshop technology-push proposal selec&on scoping ﬁlter Design ideas

5. Create one Complete Toolkit & proposal project Workshop only scoping Results

Future Applicability to Coarse Pre- Project Selec&on Triage work M-Pull & proposal selec&on scoping ﬁlter Incremental ideas Figure 30: Main results for each step of the development process (models and processes)

METHODS AND TOOLS

EvaluaDng early- Technology roadmapping stage innovaDons IniDaDon & Development IntegraDon 1. Purpose Formats Literature Scoring model Workshop EvaluaDon tools Toolkit Review

Strategic MulDlayer Toolkit process Idea Linking Other Quality SoKware Tool Workshop Proposal grids tools assessment integraDon Product Tabular Workshop mgmt

Other Use Used without Use Roadmaps Roadmaps Next step is formats Strategy scaling stat. Use both diﬀerent are not need to be to digiDse 2. too map, Company formats interacDve PorQolio up to date roadmaps Interviews Use roadmap matrix, prioriDes and comp. TRLs assessment Workshop Toolkit SelecDon Roadmap Idea Quality 3. Linking factors proposal assessment Consultant 4. Strategic MulDlayer grids template template template Interviews Toolkit & Workshop Product Tabular Design

Time People

5. Not enough OpDmisDc Impact Not Toolkit & Dme to Too many bias and Unclear Unclear could enough Workshop discuss roadmaps incomplete not be facilitaDon Results issues knowledge assessed

Inﬂuence BeVer BeVer Assess Allocate Use Bigger group Roadmap Future of Increase explanaDon deﬁniDon Integrate impact of more integrated & decision soKware work selecDon facilitaDon of each of other tools quality Dme roadmaps makers tool factors secDon synergy (& priority) Figure 31: Main results for each step of the development process (methods & tools)

8. References

Albright, R. (2003) ‘A Unifying Architecture for Roadmaps Frames a Value Scorecard’, IEMC ’03 Proceedings, pp. 383–386. doi: 10.1109/IEMC.2003.1252298. An, Y., Lee, S. and Park, Y. (2008) ‘Development of an integrated product‐service roadmap with QFD’, International Journal of Service Industry Management, 19(5), pp. 621–638. doi: 10.1108/09564230810903497. Brem, A. and Voigt, K.-I. (2009) ‘Integration of market pull and technology push in the corporate front end and innovation management—Insights from the German software industry’, Technovation, 29, pp. 351–367. doi: 10.1016/j.technovation.2008.06.003. Bryman, A. (2012) Social research methods. Oxford University Press. Bryman, A. and Bell, E. (2003) Business Research Methods. 3rd edn. Oxford University Press. Carvalho, M. M., Fleury, A. and Lopes, A. P. (2013) ‘An overview of the literature on technology roadmapping (TRM): Contributions and trends’, Technological Forecasting and Social Change., 80(7), pp. 1418–1437. doi: 10.1016/j.techfore.2012.11.008. Chao, R. O. and Kavadias, S. (2008) ‘A Theoretical Framework for Managing the New Product Development Portfolio: When and How to Use Strategic Buckets’, Management Science, 54(5), pp. 907–921. doi: 10.1287/mnsc.1070.0828. Clark, K. and Wheelwright, S. (1993) ‘Managing new product development—text and cases’, Harvard Business School. Cooper, R. G. (2006) ‘Managing Technology Development Projects’, Research Technology Management, 49(5), pp. 23–31. doi: 10.1109/EMR.2007.329141. Cooper, R. G., Edgett, S. J. and Kleinschmidt, E. J. (1999) ‘New product portfolio management: practices and performance’, Journal of Product Innovation Management, 16(4), pp. 333–351. doi: 10.1016/S0737-6782(99)00005-3. Cooper, R. G., Edgett, S. J. and Kleinschmidt, E. J. (2000) ‘New problems, new solutions: making portfolio management more effective’, Research-Technology Management, 43(2), pp. 18–33. Cooper, R. G., Edgett, S. J. and Kleinschmidt, E. J. (2001) ‘New Product Portfolio Management for New Product Development : Results of an Industry Practices Study’, R&D Management, 31(4). Cooper, R. G., Edgett, S. J. and Kleinschmidt, E. J. (2001) Portfolio management for new

69 products Cosner, R. R. et al. (2007) ‘Integrating Roadmapping into Technical Planning’, Research Technology Management, (Nov-Dec), pp. 31–48. Dissel, M. C. et al. (2006) ‘Value Roadmapping’, Research-Technology Management, 52(6), pp. 45–53. Farrukh, C. et al. (2014) ‘Light-weighting innovation strategy: A roadmap-portfolio toolkit’, Proceedings of PICMET ’14 Conference: Portland International Center for Management of Engineering and Technology, pp. 97–101. Farrukh, C., Phaal, R. and Probert, D. R. (2001) ‘Industrial practice in technology planning - implications for a useful tool catalogue for technology management’, PICMET ’01. Portland International Conference on Management of Engineering and Technology, p. 1. doi: 10.1109/PICMET.2001.951925. Faulkner, T. W. (1996) ‘Applying “options thinking” to R&D valuation.’, Research Technology Management, 39(3), pp. 50–56. Fenwick, D., Daim, T. U. and Gerdsri, N. (2009) ‘Value Driven Technology Road Mapping (VTRM) process integrating decision making and marketing tools: Case of Internet security technologies’, Technological Forecasting and Social Change, 76(8), pp. 1055–1077. doi: 10.1016/j.techfore.2009.04.005. Gerdsri, N. (2007) ‘An analytical approach to building a technology development envelope (TDE) for roadmapping of emerging technologies’, International Journal of Innovation and Technology, 4(2), pp. 121–135. doi: 10.1109/PICMET.2005.1509682. Gerdsri, N. and Kocaoglu, D. F. (2007) ‘Applying the Analytic Hierarchy Process (AHP) to build a strategic framework for technology roadmapping’, Mathematical and Computer Modelling, 46, pp. 1071–1080. doi: 10.1016/j.mcm.2007.03.015. Gerdsri, N. and Vatananan, R. S. (2007) ‘Dynamics of Technology Roadmapping (TRM) implementation’, in Portland International Conference on Management of Engineering and Technology, pp. 1577–1583. doi: 10.1109/PICMET.2007.4349482. Goffin, K. and Mitchell, R. (2016) Innovation management : effective strategy and implementation. 3rd edn. palgrave. Groenveld, P. (1997) ‘Roadmapping integrates business and technology’, Research Technology Management, 50(6), pp. 49–58. doi: 10.1080/08956308.2007.11657472. Guba, E. and Lincoln, Y. (1994) ‘Competing paradigms in qualitative research’, Major Paradigms and Perspectives, pp. 105–117. Hassan, S., Chishti, A. and Elamvazuthi, C. (2006) ‘Web-Based Documentation System for

Dynamic Roadmaps’, AACE Journal, 14(3), pp. 257–268. Hayes, R. H., Wheelwright, S. C. and Clark, K. B. (1988) Dynamic manufacturing : creating the learning organization. Free Press. Hobday, M. (2005) ‘Firm-level Innovation Models: Perspectives on Research in Developed and Developing Countries’, Technology Analysis & Strategic Management, 17(2), pp. 121– 146. doi: 10.1080/09537320500088666. Kameoka, A. and Meng Li (2004) ‘New direction and strategy of MOT practice in Japan, IEEE International Engineering Management Conference IEEE, pp. 342–347. doi: 10.1109/IEMC.2004.1407132. Kaplan, R. S. and Norton, D. P. (2000) ‘Focusing your Organization on Strategy - with the Balanced Scorecard, 2nd Edition’, Harvard Business Review. Kerr, C. et al. (2013) ‘Key principles for developing industrially relevant strategic technology management toolkits’, Technological Forecasting and Social Change, 80(6), pp. 1050–1070. doi: 10.1016/j.techfore.2012.09.006. Kerr, C. and Phaal, R. (2015) ‘A scalable toolkit platform: Configurations for deployment in technology and innovation strategy workshops’, R&D Management Conference, pp. 23–26. Kirchhoff, B. A. (Bruce A. (1994) Entrepreneurship and dynamic capitalism : the economics of business firm formation and growth. Praeger. Kostoff, R. N., Boylan, R. and Simons, G. R. (2004) ‘Disruptive technology roadmaps’, Technological Forecasting and Social Change, 71(1–2), pp. 141–159. doi: 10.1016/S0040- 1625(03)00048-9. Kostoff, R. N. and Schaller, R. R. (2001) ‘Science and technology roadmaps’, IEEE Transactions on Engineering Management, 48(2), pp. 132–143. doi: 10.1109/17.922473. Lee, S. et al. (2007) ‘Technology roadmapping for R&D planning: The case of the Korean parts and materials industry’, Technovation, 27(8), pp. 433–445. doi: 10.1016/j.technovation.2007.02.011. Lee, S. et al. (2009) ‘Business planning based on technological capabilities: Patent analysis for technology-driven roadmapping’, Technological Forecasting and Social Change., 76(6), pp. 769–786. doi: 10.1016/j.techfore.2009.01.003. Lee, Y. et al. (2002) ‘AIMQ: a methodology for information quality assessment’, Information & Management, 40, pp. 133–146. Lindsay, J. (2000) ‘The Technology Management Audit — the Tools to Measure How Effectively You Exploit the Technological Strengths and Know-how in Your Company’, Financial Times/Prentice Hall, London.

7 1

Love, C. and Tillmanns, P. (2011) ‘Your Strategy Needs a Strategy', pp. 1–24. doi: 10.1002/smj.555. Ma, T., Liu, S. and Nakamori, Y. (2006) ‘Roadmapping as a way of knowledge management for supporting scientific research in academia’, Systems Research and Behavioral Science, 23(6), pp. 743–755. doi: 10.1002/sres.708. Mankins, J. C. (1995) ‘TECHNOLOGY READINESS LEVELS’. McGrath, R. G. (1997) ‘A real options logic for initiating technology positioning investments’, Academy of Management Review, 22(4), pp. 974–996. doi: 10.5465/AMR.1997.9711022113. McMillan, A. (2003) ‘Roadmapping - Agent of change’, Research-Technology Management, 46(2), pp. 40–47. Mitchell, R., Phaal, R. and Athanassopoulou, N. (2014) ‘Scoring methods for prioritizing and selecting innovation projects’, PICMET 2014 - Portland International Center for Management of Engineering and Technology, Proceedings: Infrastructure and Service Integration, (2001), pp. 907–920. Oliveira, M. and Rozenfeld, H. (2010) ‘Integrating technology roadmapping and portfolio management at the front-end of new product development’, Technological Forecasting & Social Change, 77, pp. 1339–1354. Pearson, A. W. (1990) ‘Innovation Strategy’, Technovation, 10(3), pp. 185–192. Phaal, R. et al. (2003) ‘Starting-up Roadmapping Fast’, Research Technology Management, p. 52. doi: 10.1109/EMR.2003.24906. Phaal, R. et al. (2010) Roadmapping for strategy and innovation : aligning technology and markets in a dynamic world. University of Cambridge, Institute for Manufacturing. Phaal, R., Routley, M., et al. (2012) ‘Charting Exploitation Strategies for Emerging Technology’, Research-Technology Management, 55(2), pp. 34–42. doi: 10.5437/08956308X5502021. Phaal, R., Kerr, C., et al. (2012) ‘Towards a modular toolkit for strategic technology management’, International Journal of Technology Intelligence and Planning, 8(2), p. 161. doi: 10.1504/IJTIP.2012.048475. Phaal, R. and Farrukh, C. J. P. (2000) ‘Technology Planning Survey - Results’, (October 1999), p. 29. Phaal, R., Farrukh, C. J. P. and Probert, D. R. (2004) ‘Technology roadmapping - A planning framework for evolution and revolution’, Technological Forecasting and Social Change, 71(1–2), pp. 5–26. doi: 10.1016/S0040-1625(03)00072-6.

Phaal, R., Farrukh, C. J. P. and Probert, D. R. (2006) ‘Technology management tools: Concept, development and application’, Technovation, 26(3), pp. 336–344. doi: 10.1016/j.technovation.2005.02.001. Phaal, R., Farrukh, C. and Probert, D. (2001) T-Plan : the fast-start to technology roadmapping : planning your route to success. University of Cambridge, Institute for Manufacturing. Platts, K. W. (1993) ‘A process approach to researching manufacturing strategy’, International Journal of Operations & Production Management, 13(8), pp. 4–17. Strauss, J. D. and Radnor, M. (2017) ‘Research-Technology Management Roadmapping for Dynamic and Uncertain Environments’. doi: 10.1080/08956308.2004.11671620. Strauss, J. and Radnor, M. (2004) ‘Roadmapping for dynamic and uncertain environments’, Research Technology Management, 47(April), pp. 51–58. Susman, G. I., Evered, R. D. and Susman, G. 1 (1978) ‘An Assessment of the Scientific Merits of Action Research’, Source: Administrative Science Quarterly, 23(4), pp. 582–603. Vatananan, R. S. and GERDSRI, N. (2012) ‘the Current State of Technology Roadmapping (Trm) Research and Practice’, International Journal of Innovation and Technology Management, 9(4), p. 1250032. doi: 10.1142/S0219877012500320. Voss, C., Tsikriktsis, N. and Frohlich, M. (2002) ‘Case research in operations management’, International Journal of Operations & Production Management, 22(2), pp. 195–219. doi: 10.1108/01443570210414329. Wall, B., Jagdev, H. and Browne, J. (2005) ‘An approach to developing an eBusiness roadmap’, PRODUCTION PLANNING & CONTROL, 16(7), pp. 701–715. doi: 10.1080/09537280500290136. Yin, R. K. (2013) Case study research : design and methods. 5th edn. Zhou, K. Z., Yim, C. K. (Bennett) and Tse, D. K. (2005) ‘The Effects of Strategic Orientations on Technology- and Market-Based Breakthrough Innovations’, Journal of Marketing, 69(2), pp. 42–60. doi: 10.1509/jmkg.69.2.42.60756.

9. Appendices

9.1. Feedback questionnaire Post-Process Questionnaire

Completed by: …………………………………. Date: ……………………………………………

Usefulness

The overall objective of the workshop was: O. Understand how the company’s existing roadmaps can be used to evaluate new breakthrough ideas. The corresponding sub-objectives were: 1. Assess Roadmap Quality 2. Select criteria and scaling statements 3. Identify synergies between ideas and drivers 4. Identify synergies between ideas and products 5. Identify synergies between ideas and technologies/ resources 6. Evaluate each idea on all criteria 7. Complete an idea evaluation template Rate how well the Evaluation process supported the desired objective and sub-objectives of the workshop:

Objective 5. 4. 3. 2. 1. Comments Excellent Very Good Satisfactory Poor & Good Suggestions for improvement O

Other Feedback

Objective 5. 4. 3. 2. 1. Comments & Strongly Strongly Suggestions for agree disagree improvement Good composition of the team Everybody could participate

The process was a fast way to evaluate ideas The process gave a complete and thorough evaluation

There was enough time to discuss important issues

The company could do the process without outside involvement

We discovered many new information about our company

General comments

9.2. Tool templates

TITLE Summary descrip.on of idea/opportunity WHAT Unique and / or valuable features of the idea

Product and Closest product- service lines Services Areas WHY Why should we invest in this idea? i.e. how does it ﬁt with business aims, external drivers or chosen future

HOW How can we realise it? i.e. how do we use technology, competences, resources

WHEN When can we realise this idea? Short term, medium term, long Your ini:als: A.B term

Figure 32: Idea proposal template

Roadmap 1 Roadmap 2 …

Validity Is the information true?

Completeness Drivers Does it include all relevant information?

Understandability Is the information easy to understand? TOTAL

Validity Is the information true?

Completeness Products Does it include all relevant information?

Understandability Is the information easy to understand? TOTAL

Validity Is the information true?

Completeness Technologies/ Does it include all relevant information? Resources Understandability Is the information easy to understand? TOTAL

Figure 33: Roadmap quality assessment template

Figure 34: Selection factors template (Feasibility)

Figure 35: Selection factors template (Opportunity)

HOW?: Selection Factors WHY/ WHY NOT?: WHAT?: Technologies/ Drivers Product Features Resources OPPORTUNITY FEASIBILITY Driver1 Driver2 … ProductFeature 1 ProductFeature 2 … Resource 1 Technology/ Resource 2 Technology/ … Roadmap

Quality

Roadmap 1

Roadmap 2

Priority … TOTAL TOTAL TOTAL Opportunity1 Factor Opportunity2 Factor … Feasibility1 Factor Feasibility2 Factor … OPPORTUNITY TOTAL FEASIBILITY TOTAL Project 1 Factor Project 2

New scoring Projects Innovation Innovation …

Figure 36: Linking grid

Idea Selec)on How did the roadmap informa)on What was missing from the roadmap Factor help evaluate the factor? to properly evaluate the factor?

Figure 37: Idea evaluation template

9.3. Workshop schedule

Time Ac(vity People 9:00-9:30 Room prepara(on Facilitators 9:30-10:00 Introduc(on All 10:00-12:00 Deﬁni(on & Prepara(on All 12:00-13:00 Lunch All 13:00-13:30 Introduc(on + Morning summary All 13:30-13:45 Workshop Part 1 – Drivers All 13:45-14:00 Workshop Part 2 – Products All 14:00-14:30 Workshop Part 3 – Technology and Resources All 14:30-15:00 Workshop Part 4 – Idea Evalua(on All

15:00-15:30 Workshop Feedback & Closure All