Master's Thesis

MASTER'S THESIS

A Lightweight Approach for Sharing Lessons Learned Development of a supporting tool for sharing Lesson Learned in early stages of

product development

Jens Jönsson

Master of Science Mechanical Engineering

Luleå University of Technology Department of Business Administration, Technology and Social Sciences Acknowledgment

First and foremost I owe my gratitude to the Lord for his presence and strength he has given me during this work.

I would like to thank my family especially my father Dominique, and my mother Pauline, for their support throughout the years and to my fiancée Greta for her unbelievable patience and support throughout the thesis work.

I give many thanks to my supervisors, Marco Bertoni and Christian Johansson, for all their friendly guidance and patience during this thesis. Also thanks to my examiner, Åsa Ericson, who has made this work possible.

Additionally I want to thank my collaborative research colleague Koteshwar Chirumalla who has given valuable contribution to product development process/performance.

Last but not least, I would like to thank my opponent Alexander Abgar Afram for his careful reading of this report and for giving me valuable comments and linguistic feedback. Finally, I owe a debt to test participants, friends and acquaintances who have supported me, and the work, either by contributing knowledge, or inspiration.

Thanks. Jens Jönsson

,Luleå, 2011-10-09

Abstract

This thesis reports on a development project of a Lessons Learned knowledge management system. lessons learned (LL) is a knowledge sharing method that takes advantage of useful knowledge from previous or on-going project events in order to be reused in on-going or following project activities. Product Development (PD) is an area particularly of particular interest when it comes to sharing lessons learned. In a early design phase, in fact, various experts have to share what they know about the hardware and its services to create a successful product concept. This knowledge takes often the form of lessons learned, that is the form of recommendations that are specific and measurable. These lessons, however, are very seldom formalized in a way that can be readily used across the organizational boundaries by the product development. Most of this knowledge is usually tacit and in general difficult to formalize and share. Accordingly, there is a need to establish a structured Lesson Learned System (LLS) that takes into consideration all obstacles in conveying knowledge amongst different people. This involves giving guidelines on how to structure the LL in different cross-functional development activities. The development of the LLS kicked off with the analysis of the user needs and the identification of relevant requirements. The definition of the requirements list was aided by a generic knowledge lifecycle derived from the literature and composed by five different steps: collecting, verifying, storing, disseminating and reusing LL. In the concept development phase a strong emphasis has been given to the concept of “lightweight” knowledge sharing technologies. The guiding principles and mechanism of Web 2.0 have been considered particularly interesting to enhance the level of interaction between the system users and to leverage the way the system is populated, maintained, updated and validated across the company. Eventually, the thesis presents the results of the prototyping phase. A LLS mock-up has been developed and tested both with researchers and practitioners working in the laser welding domain, as well as with in design sessions with students. The lessons learned system detailed specifications as well as the feedback from the validation activities have to be considered the main results of this master work.

iii Table of content

Acknowledgment ...... ii Abstract ...... iii Abbreviations ...... vi 1. Introduction ...... 1 1.1 Objective ...... 2 1.2 Delimitations ...... 2 2. Research approach ...... 3 3. Theoretical framework ...... 6 3.1 Product development ...... 6 3.1.1 Activities in early stages of product development ...... 7 3.2 Knowledge Management ...... 11 3.3 Lessons Learned ...... 12 3.3.1 Capturing and managing lessons ...... 13 3.4 Lightweight Technology ...... 16 4. Towards a lightweight Lessons Learned capturing system ...... 19 4.1 Trends using Lessons Learned ...... 19 4.2 Lessons Learned Systems – from customer expectations to benchmarking metrics ...... 20 4.2.1 Formulating features needed ...... 21 4.3 Requirements definition for the LL system ...... 22 4.4 Defining a metrics for the LL system ...... 24 5. Generating Lessons Learned System concepts ...... 27 5.1 Review of complete and potential Lessons Learned System ...... 27 5.2 Approach of Lessons Learned System ...... 27 5.3 Advantages and disadvantages in studied Lessons Learned System ...... 28 5.4 Choice of Lessons Learned System approach ...... 28 5.5 Decomposition of Lessons Learned System steps ...... 29 5.6 Concept generation ...... 32 6. LL system concept selection and detailed description...... 36

iv 6.1 Detailed design ...... 37 7. LL system testing and validation ...... 40 8. Concluding remarks ...... 48 References ...... 50

v Abbreviations

AAR After Action Review

ALDS Active Lesson Delivery System

DSS Decision Support System

FFE Fuzzy Front End

KM Knowledge Management

LL Lessons Learned

LLS Lessons Learned System

MD Monitored Distribution

MOKA Methodology and tools Oriented to Knowledge-Based Engineering Applications

OMAS Open Multi Agency System

PA Personal Assistant

PD Product Development

RCA Root Cause Analyses

R&D Research and Development

SA Service Agent

SotA State Of the Art

vi 1. Introduction The early stages of PD can be seen as a set of activities devoted to the gathering of knowledge with the goal to develop a product that fits the end users needs. Given the criticality of the task and the stringent deadlines that characterize the product development process, it is important to utilize the most effective tools and methods for collecting such information. In addition to saving time on selecting the right tools and methods, it is important to create a working environment that supports cross- functional teamwork, and supports knowledge sharing between them.

This can be done by taking advantage of knowledge from the different people involved in the process through reusing Lessons Learned (LL) in the form of recommendations that are specific and measurable. This can be improved by facilitating a Lesson Learned System (LLS) that support engineers and designers in collecting relevant product knowledge and in recognizing success factors from previous, as well as ongoing, PD projects.

Two famous quotes - “Skill to do comes of doing" described by Emerson in The Atlantic Monthly (1862) and “any man can make mistakes, but only an idiot persists in his error” dating back to Cicero and quoted by Cohen (2002) – picture well the problem of taking advantage of reusing knowledge from experience. They stress the importance of being aware of drawbacks and of the necessity to learn from mistakes. These quotes well describe what commonly happens in product development too. In order to be innovative in the design of new products, there is a need to increase awareness on what has been learnt by experience, and to do it in a way that people from different functions can contribute to the definition of the product. Obstacles can be experienced when supporting cross-functional collaborations with text-based LLS such as, communication difficulties based on the difference in technical terms used by participants in the cross-functional collaboration, and tacit knowledge that can be lost when documenting lessons learned. LL, in fact, are closely connected to the way people carry out tasks and solve problems, hence they are difficult to articulate and record (Polanyi, 1966). This suggests to investigate how lightweight technology, and Web 2.0 technologies, can enhance the “collecting, verifying, storing, disseminating and reusing” of LL trough a system used to support the early stages of PD by complementing existing text-based solutions. This implies the development of guidelines on how to structure the shared LL from different functional areas in a way that can be interpreted by the product development team in present and future projects.

1.1 Objective The purpose of the thesis is to propose a LLS that is adapted to the early stages of PD and that exploit some of the mechanism typical of “lightweight” Web 2.0 applications. More in detail, the objective is to:

1. Identify the requirements for a LLS that allows integrating knowledge from various process steps and people, which includes providing suggestions on how LL could be identified, verified, captured, documented, retrieved, and obtained, in order to be re-used. 2. Elaborate the specifications for the lightweight LLS, with particular focus on which media and mechanisms might be able to leverage a collaborative approach in LL capturing and sharing. 3. Develop and test a LLS mock-up and validate its capability to support engineers in routine design and day-by-day decision making activities.

1.2 Delimitations The thesis proposes a LLS that is based from insights from literature, empirical testing and the evaluation of a mock up.

It will not take into consideration its lifecycle cost or implementation cost of the system. This thesis will not take into consideration software programming of the system concept mock-up, thus will solely be presented as functions in pre-existing lightweight technology solutions.

2 2. Research approach The research approach used during the project was based on an adapted version of the Methodology and tools Oriented to Knowledge-Based Engineering Applications (MOKA) methodology. MOKA is an approach for the development of Knowledge Based Engineering (KBE) applications and makes use of software solutions to aid in capturing and re-using knowledge. The development process of MOKA is shown in figure 1 described by the founders Callot (2003).

Figure 1: Project plan for developing MOKA

The thesis methodology is inspired by two of the main objectives in the MOKA methodology: integrate knowledge from various process steps, and support engineers in routine design and their decision making activities.

The project has kicked-off with a literature review – i.e.. with the State-of-the-Art and State-of-Practice - in the area LL, Knowledge Management (KM) and PD in order to apprehend and identify the current problem that had to be solved or aided in text- based LLS´s.

Brainstorming sessions have been also conducted to complement the notions collected during the literature review and to support the generation and evaluation of LLS concepts - as shown in Figure 2. The brainstorming sessions were performed in order to define key features and functions that had potential to solve the current, and future, problem of sharing LL. Features from the brainstorming session and previous notion of successful features in PD were then identified by searching for corresponding functions in pre-existing lightweight technology in order to generate LLS concepts. The objective of the following step was to rank the generated concepts functions, based on tests, to select a single concept with the greatest potential to be successful in practical use. Specific functions with high rank in the nonelected concepts were also implemented in the detailed design. Testing and verification activities were conducted in three different environments, with different objectives, setups and equipment:

1. Internal validation in a laboratory environment (i.e., the LLS has been tested and used by a limited number of people with experience on knowledge sharing system to verify the feasibility and relevancy of the approach)

2. Validation in a research environment, involving manufacturing experts (i.e., with researches in the laser welding domain, as a means to verify the LLS capabilities related to the capture of LL, as well as to test the necessary hardware and software equipment).

3. Validation in design sessions with students (i.e., with participants in the M7014T course in the Product Development Master Programme at the Luleå University of Technology, to validate the performances of the system for what concerns the reuse of LL in PD tasks).

Figure.2: PD process for LLS supported by lightweight technology based on adapted MOKA approach.

A “learning-by-doing” approach has been essentially chosen as preferred means to verify the relevancy and effectiveness of Web 2.0 mechanism for what concerns the development of the LLS. The continuous feedback received from all the users involved in the recording of lessons learned have given precious information to refine both the methodological as well as technological guidelines proposed in the thesis.

3. Theoretical framework The aim of the thesis was to aid the PD teams in the early stages of PD with LL that can be seen as knowledge or understanding gained by experiences. Functions in lightweight technology in the form of Web 2.0 were selected in order to aid the sharing of LL. In order to understand the relevance of the topics PD, KM and Web 2.0 are described in this chapter which illustrates the necessity of where, why, when, how and for what LL can be used.

3.1 Product development As described by Ulrich & Eppinger (2008) pp12 a “PD process is the sequence of steps or activities which an enterprise employs to conceive, design, and commercialize a product”. This involves activities, which use different methods and tools to develop a product in a satisfactory manner, based on the demands of marketing, design, and manufacturing aspects etc. Development of a product can involve technical solutions in physical products, services or a combined solution of both. Depending on the product developed, different methods and techniques are selected, combined and used by the PD team to generate the final product.

By selecting the most effective method and tools it can improve and optimize the work process and add value to the final product, i.e. re-using previous successful methods and tools. By locating and addressing these issues early in the PD process, the design team can avoid what often becomes costly errors as the project moves to more complicated computational models and eventually into the physical realm (Kusiak, 1992). In addition to knowledge shared within the PD team, it is also an advantage in considering experiences from end users interacting with the product or similar products. This can bring valuable knowledge on how to improve the product and where additional resources should be assigned in the early stages of PD.

PD can follow different processes and use different tools that are usually customized and combined in order to develop a specific product (Maffin, Thwaites, Alderman, Braiden, & Hills, 1997). The selection and customized process is based on the final product and how it should satisfy the needs of the user. The procedures can vary depending on if it is a radical or incremental innovation and can involve an alternative approach for New Product Development (NPD) described by Cooper & Kleinschmidt (2007).

The requirements for a system do not always consist of current requirements but can also include potential future requirements that have been interpreted by trends in the market. Through a market pull or the need of efficient manufacturing in industrial companies, e.g. to reduce lead time and cost for producing industrial products (Callot, 2003). In other words, how can the final product adapt to the changing needs over time?

6 3.1.1 Activities in early stages of product development This section will briefly describe the different activities, which can be involved in the early stages of PD in which the developed LLS should support. Since a PD process can differ in activities and process, these descriptions should only be seen as guidelines on how LL can be used in the different activities and to understand how downstream knowledge of activities affects the early stages of PD. The scope of the early phase in PD includes phase zero planning, also known as the Fuzzy Front End and phase one involving the concept development illustrated in figure 3 according to Ulrich & Eppinger (2008). These two phases has been the focus of this thesis.

Figure.3 A schematic product development process, after Ulrich & Eppinger (2008) pp16.

Phase zero and one can be described according to their activities as followed in next page;

Phase 0

The planning phase, also recognized as FFE, can be described as the front-end activities where organizations formulate concepts, justify further procedure and start the product development process. That is why supporting LL knowledge that aid in phase 0 can be seen as important. The layout of the activities in phase 0 can differ depending on if it is a radical or incremental innovation. Nevertheless they involve influences from elements in the five steps of product planning. These are progressing as sequential steps but are iterative and take influences from each other to create the final outcome. According to Koen (2001) the five following steps of product planning can be described as:

Opportunity identification: In this step opportunities are identified based on input from different sources. Among others they can originate from both internal opportunities in research and technology development in companies as well as from external sources based on customer needs, market research etc. Aggregated knowledge in LLS that has been used in previous projects, creates opportunities in acknowledging e.g. drawbacks in methods in order not to repeat them.

7 Opportunity analysis: In this step the opportunity identifications have to be translated into business and technology opportunities. This can be done through observing and documenting information on focus groups. With an easy to use LLS that also involves the end user it can add feedback on LL that can support decision making in this step. For instance, if the end user has experienced a drawback on a similar product or service it can give suggestions on how it can be modified in order to satisfy their needs. The knowledge on LL can be used in order to formulate the business and technology opportunities.

Idea genesis: In this step the opportunities are combined and reshaped, modified and upgraded to create the final idea. This is an iterative process that can involve feedback from the end user/customer, relation with cross-functional teams, and collaborations with other companies to create the suggested idea of a concept. The idea can also be generated based on unusual requirements from the end users of the product or through suppliers offering new material. This can be supported by a LLS with updated information from the end users, to get new and emerging requirements. Thus it can be supported in the development of products and services.

Idea Selection: The main goal in this step is to select an idea that can bring the highest business value. Strategic methods and processes can be used in order to select the idea with the greatest potential. In order to achieve this it is important to consider different contributing aspects that can impact on the concept development in order to create a successful outcome, such as competitive products, innovative advantage, financial aspects, organization capabilities, technology risk, and investment level. The condition is still quite loose in this step in order to let the idea grow whilst proceeding in to the PD process. LLS´s can, in this step, give influential information from similar projects in order to predict if additional aspects should be considered in the final idea selection.

Concept and technology development: The final step involves the development of a business case. According to Koen (2001) the business case is formulized and based on estimates of market potential, customer needs, investment requirements, competitor assessment, technology unknowns and overall project risk. These estimations can be supported by knowledge on trends and market outcome from similar and previous projects, which can be supported by previous experience in the form of LL.

Phase 1

After the product development project is planned in phase 0, it is followed by the initiation of the concept development phase 1. This phase involves different activities with the purpose to generate concepts from the “mission statement” that was formulized in the previous phase and bring these concepts to the development plan. The influences from different activities can be described in Ulrich & Eppingers (2008) front- end process as illustrated in figure. 4.

Figure.4 Ulrich & Eppingers (2008) pp54 Front-End Process of concept development.

The activities involve amongst other things cross- functional team collaborations, data gathering from end user and benchmarking. The activities are described generally as followed in order to illustrate the potential use of LLS´s in the front-end process of phase 1.

Identify customer needs

This step involves the understanding of the end user/customer needs. Data collected from users/customers are collected interpreted, organized, reflected on, and ranked according to importance of requirement. The work process can differ, but still include a vast amount of data collected from the end users. This can be done more efficiently by involving the end user in LLS. It can support the development team with decreased workload and genuine LL knowledge since it’s from user experience. The knowledge can be gathered by encouraging and guiding users of a product, in sharing LL from interactions with similar product or services.

Establish target specification

In this step the user/customer needs are translated into what the product actually will deliver. Depending on the product, these can be quantifiable in the form of specific numeric or subjective measurements (requirements). The information provides guidance in describing what the required functions of the developed product should achieve in order for it to become successful. In order to achieve this it is important to establish a complete set of specifications that takes into consideration all significant aspects. Project teams can compare LL from similar projects to ensure that they are not overlooking any significant specifications. If LL describes drawbacks on products it can be interpreted as a specification that was overlooked or did not meet the required value thus not satisfying its function.

Generate product concept

In this step different concepts are generated based on combinations of solutions generated in the previous activities. Insights from expertise are restricted by the knowledge from team members. Thus it is important to invite influences from other sources to shape the final outcome (concepts). This can be done through an open knowledge-sharing source aided by a LLS.

Select product concept

The different generated concepts from the previous stage are then compared and related to the initial needs and requirements. The concepts the with highest potential to be successful are advanced to the next step in the process (test product concept). Since the process is iterative it allows additional concepts to be generated based on a combination of high ranked functions in specific parts of the different concepts, and can create new concepts with greater potential. A larger scope of opinions from the users can give valuable inputs on suggested selection of the final concept. This can be aided through knowledge gained from external sources and could be supported by a LLS.

Test product concept

In this step the concepts are tested and evaluated to add important feedback to ensure that the physical product meets the requirements. Experience and knowledge from users interacting with the product gives necessary information, which can make or break a successful final product. Through incorporating an interactive knowledge-

10 sharing tool in the form of a LLS, it can aid this process in giving the end users of the product the ability to verify the product concept.

Set final specifications

In this step the initial target specifications are compared with the result from the concept testing in order to see if they satisfy the targets. If not, trade-offs can be made by taking into consideration additional factors such as financial aspects etc. that may lead to a redesign of the product. To avoid critical errors in the trade-off there is a need to compare which parameters to change. A supporting tool in the form of an LLS can aid in the justification of the choices by comparing similar projects, e.g., which trade-off has been done in similar projects and what was the result.

Plan downstream development

The final activity of the front-end concept development phase is the planning of downstream development. In this activity different project management issues are managed, planned, and documented in order to bring the final developed product to the end user. The activity can involve evaluating the performance of the project, which can add important knowledge and, if shared, bring supportive LL to forthcoming projects.

3.2 Knowledge Management Successful knowledge sharing method does not only involve disseminating knowledge but also the ability to be comprehensibly collected from the sources in order to be shared. The loop should be closed and a successful KM system that shares knowledge should facilitate exchange of knowledge in organizations and projects. Once competence is captured in a system, it needs to be available to other individuals in such a way that it improves their capacity to act, otherwise the investment is a waste as described by Karl- Erik Sveiby (2001).

Knowledge can be seen as a resource in a company or organization (Leonard- Barton, 1992). In order not to restrict the resources amongst specific employees or technology it is important to be able to share the knowledge amongst each other. By making the knowledge accessible it can be managed and shared between different sources. For instance, support cross-functional collaborations in the early stages of PD and aid them with easy accessible knowledge on how to use methods and tools can in this sense support ongoing activities in strategic decision-making.

Knowledge sharing is important throughout a company or organization in the sense that activities in production, development, marketing etc, seldom involve collaboration within one expertise area (Chirumalla, Larsson, Bertoni, & Larsson, 2011). To facilitate these collaborations it is important to share knowledge that affects the entire PD process. In activities with intense cross-functional collaborations it is particularly

11 important to take advantage of different suggestions on solutions from different areas of expertise in order to produce, develop and deliver a successful product.

Knowledge is in many cases interpreted as learned through experience. Since experiences are perceived in day-to-day activities it is important to collect and share knowledge in on-going events (Bertoni, Eres, & Isaksson, 2011). Knowledge should not be seen as static but rather seen as a dynamic phenomenon. This can be described in the sense that knowledge is perceived to be successful in one moment and can easily be unsuccessful in the next, e.g., can change depending on market change (trends, prices, demands etc.). Through active knowledge sharing as a day-to-day activity it acknowledges current knowledge through frequent updates.

3.3 Lessons Learned Baird, Handerson, & Watts (1997) reviews the background of LL. They report that a systematic way to use LL was first adopted by the US Army Centre for Army Lessons Learned (CALL) in the middle of the 1980's. It started with the creation of a training and doctrine command of the US Army in 1973. It was an agency with the overall responsibility for soldier and leadership development. Four subdivisions were created in 1984 under the title, Combat Training Centres. One approach of their methods was working to execute realistic war exercises with the aid of laser technology and simulations. To utilize the exercises effectively, they created the CALL in 1985. The department mission was to record LL during exercises to benefit from them in succeeding activities. Better simulation technology and virtual reality was developed during the following years, which lead to supportive tools in managing LL knowledge, e.g. giving a comprehensive illustrative visualization of documented LL. This was followed by alternative scope of use, which was implemented in the development of technical equipment and leadership methods. The method for observing and sharing of LL progressed in 1993 as the documentation and sharing of information shifted from not only taking place after the exercises but was also performed during the process of the exercises. LL was recognized by observing participants during operations allowing knowledge to be captured, analysed, synthesized, and centralized to provide fast and available suggestions for participants in on-going exercises. This was used to provide information in training and development as well as for the commanders to facilitate in their day-to-day decision-making (Baird et al., 1997). This methodology has since been used in alternative scopes and is highly effective in areas where current suggestions on improving knowledge are needed, such as in the early stages of PD. Current LLS are used in this area but are still commonly used throughout methods describing LL in text documentations. These do not fully support the complexity of sharing LL in cross functional collaborations. Since a PD team can contain experts from different areas, it can raise challenges that include different technical terms used and tacit knowledge that

12 demand improvements in the closed loop of recording, storing, verifying, disseminating and reusing LL. In order to find a solution on how to develop a LLS that’s adapted to the new requirements, it was chosen to look back in history and investigate how previous systems were developed. The author thought it was evident that improvements and development of previous systems were done in parallel with the development of supporting hardware and software technology solutions. It was thus chosen to investigate the area of lightweight technology and the interacting functions in Web 2.0 that could support the challenges in LLS. In this there was a need to examine which the most efficient technology was, since Web 2.0 is often used as open sources e.g. Blogs, Twitter, Facebook etc., which may lead to distribution of unusable information.

3.3.1 Capturing and managing lessons LLS are intended to guide the user and manage LL information. Since lessons are not the only KM artefact designed for reuse, it is necessary to structure knowledge on LL as to locate and share it in a successful manor. Organizations use similar collection, verification, storage, dissemination, and reuse processes for objects such as incident reports or alerts (Weber & Aha, 2003). All LL databases specify a structure for the information that includes specific fields or categories of information (Braun & Macal, 1993). In order to create a systematic knowledge structure it is necessary to reformulate the LL to specific, measurable and applicable knowledge, which is stored in a categorized repository to subsequently aid in the reuse. This allows the knowledge of LL to be aggregated and comprehensive, which results in conveyed predicted outcome. Two examples of LLS are explained below.

Intelligent delivery of military lessons learned

One of the processes investigated was Intelligent delivery of military lessons learned as described by Weber & Aha (2003). The process is based on connecting together the information gap between the military LL distribution of documentation and upcoming users. Distribution is described as divided into two categories: push and pull. The information delivery method can be described as passive information search (pull) where the entire workload is on the user. This way of searching LL can be done through library and web search. Push method is based on relieving user work effort by initiatives to disseminate information on knowledge, e.g., automated mailing lists that supplies potential stakeholders with potentially useful LL knowledge. Push method is advocated in the system since it implies that the user can dedicate resources to activities other than information search. The system is presented as a software tool, used in the mission planning stage. Monitored distribution (MD) is presented as an approach of knowledge extraction based on the LL and its applications. This is done to improve accurate selection of the desired LL knowledge. Implementation of MD is made in an existing system Active Lesson Delivery System (ALDS), which can be implemented in Decision

13 Support System (DSS) to distribute the LL whenever and wherever its needed. An interconnected lifecycle of the LL knowledge is formulated in the Weber & Aha (2003) LL process that describes the LL transfer from a person experience to another person reusing the LL in figure 5.

Figure.5 Weber & Aha (2003) process, transferring LL from a members experience to member’s reusing the LL.

A multi-agent system for acquiring and sharing Lessons Learned

An Open Multi Agency System (OMAS) described by Tascal & Barthès (2003) advocated the use of different agents that are responsible for knowledge management in Research and Development (R&D) projects. This is done through using staff assistant agents that can relieve duties of project team members. A large amount of information-rich LL can arise during R&D projects activities, which requires additional events and time to be utilize and handled. OMAS utilized provides a readily adaptable structure by adding or removing agents to manage knowledge and offer guidance on how to convert these in to LL. OMAS uses two types of agents:

1. Service Agents (SAs) that provide particular types of services corresponding to specific skills (Tascal & Barthès, 2003). One of the services is the interface to foreign platforms.

2. Personal Assistants (PAs) that are in charge of interfacing humans to the system. Their particular skills are devoted to understanding their masters and presenting the information in a timely fashion (Tascal & Barthès, 2003).

The various components of an agent begin with the knowledge that is extracted and further processed and transmitted to the user. The information process starts through a structured way of gathering information on a predetermined approach and ends up with an interface that allows dialogue with the user figure 6.

Figure. 6 Information process of LL by Tascal & Barthès, (2003)

The availability of LL may differ significantly depending on how and where it is extracted and used. Different sources were examined in addition to the pre developed LLS in order to generate a LLS which satisfies the requirements formulated in order to develop a system that aids the early stages of PD. In addition to study on how to manage LL in a LLS, other scopes of KM methods were reviewed. These had potential to manage LL even though they were not focused on the specific topic of LL. These KM methods are explained as followed.

After Action Review (AAR) is described by Allen (1994) as a process that consists of short reviews of various activities performed. This is usually done in the form of meetings after performed activities. Those who were involved in the activities attend the meeting and contribute information on the course of events. Unlike a LLS the AAR takes into account the entire project and is not just focus on a specific LL. Information that creates AAR reports does not necessarily include LL. However, if the information is analysed it may lead to useful LL.

Archival information is a form of document management systems that is aimed at older form of repositories with text documents, hand-drawn design drawings, etc. and managed by librarians. The limitation of the system is that information is not electronically stored. In order to extract LL from the repository it is necessary to have a professional archivist who assesses, collects, organizes, preserves, provides, and maintains the information. The information must be interpreted and exercised in order to formulate and locate a LL.

Best practice, are usually industry-oriented because they describe successful complete processes as benchmarks (Weber & Aha, 2003). The methodology or technique consists of recommendations that lead to more efficient work/activities. This could include an iterative development process, requirements management, quality control, or change of

15 management that provides improved results. The knowledge to be conveyed is based on taking advantage of all available information to create the best recommendations. Best practice can be seen as conveyed LL.

Accident/incident report is a report that describes the events that caused an accident or injury (Williams, 2004). Typically, design of a report can be in the form of templates in which information can be filled in to provide detailed information on describing the cause of an accident. The report is designed to record information associated to the incidents. Those involved in the accident contribute to the report. This report can further be interpreted and formulated in to LL. The information is solely based on what went wrong and what action should be taken into consideration in order to avoid recurrence.

3.4 Lightweight Technology Web 2.0 is a lightweight technology approach that can be described as a web-based application that supports users in sharing information through interacting and collaborating in order to create a common web based repository. According to O´Rreilly (2008) there are four features that categorize a Web 2.0 service.

1. The users should be able to contribute to the sites’ content and have access to it.

2. The users should be able to have control over their contributed information.

3. The interface should be user friendly and interactive.

4. And it should use AJAX technology, which is an acronym for JavaScript and XML which are a collection name for technologies used to create interactive web applications (a standardized software programming language).

The combined feature gives an application that enables information sharing through publishing and extracting information and enables collaborations amongst individuals on the World Wide Web.

In comparison to Web 1.0 the users of Web 2.0 have the ability to give feedback and the capability to be involved in posted information. This feedback function enables the users to evaluate the posted information in order to verify it to others by posting ranks or comments based on their own experience of the information.

16 Examples of Web 2.0 applications that use interacting capabilities are social media networks (Murphy, 2010). E.g. Twitter, Wiki, Blogging, Forums and Podcasting explained in the following page.

Microblogs Microblogs (e.g., Twitter) can be described as an information source that enables updates of information concerning progress and current activities. The information is shared through text messages with a limitation of 140 characters. When a profile is created, the user can share or subscribe to information from selected members that are active in on- going projects and activities.

Wikis Wikis enables users to create, add and edit information on a common website. The content is created and moderated collectively by the users

and is usually focused on a specific topic or knowledge domain.

Weblogs Weblogs, abbreviated as Blogs, enables people to be reporters and publisher of information, through filing in a form with information and publishing it on a website. These are then archived for easy search and retrieval. The system features a two-way communication with the readers. Posts of information can be commented on and the author can hold a discussion with the readers through the common website.

Podcasts Podcast (video blogging) is a series of recorded digital media files that can either be video or audio files. Video and audio recordings are uploaded to a web site so as to share information using alternative media and thus making otherwise latent information available. In comparison to sharing information through text and pictures it adds another dimension to the information shared.

Forums Forums are an open source that enables discussion among a group of people. A discussion subject is presented and allows users to ask questions, get answers from other users and retrieve information from others discussions.

Other useful functions in Web 2.0 application is different approaches of searching information through tagging and review aggregators that aid in the information retrieval, also explained in the following page.

Review The system collects reviews from different products and sources on the aggregator World Wide Web (e.g. methods, products and tools etc.). E.g. scores can be collected from web pages that enable visitors to rank products, films,

17 services etc. The aggregator can then assign numerical values on which reviews that brought greatest positive outcome and grade them in a sequential list. It can then be used to gather the scores in order to aid in selecting the most desirable from the users perspective.

Tagging Tagging is used to categorize content and assign a label on the information in order to make it searchable. It is a type of meta data that can be assigned to describe or define an aspect in an information resource, e.g., text documents, web pages, digital images, etc. This aids in finding information in a larger information repository.

18 4. Towards a lightweight Lessons Learned capturing system The practice of using LL has spread from being used in the initial expertise area of military applications to alternative scope of use. One of the reasons is that organizations are becoming more conscious of the importance of sharing LL. The new environments have brought new challenges that the LLS have to take in to consideration. The change in environments creates changes in demand and thus it demands the outcome of adaptation of requirements of the LLS. Combined with an increasing need for industrial companies to reduce lead times and costs for industrial products (Callot, 2003), it has led to the challenge of optimizing the process and thus including the use of sharing LL not only from a specific area of expertise, but from a holistic stance in organizations. It has been particularly sought after in areas where KM is the outcome of operations and is dependent on cross-functional collaborations.

4.1 Trends using Lessons Learned PD activities can involve different parts developed by different experts in order to be combined and assembled to create the end product. This has yielded obstacles in correlating knowledge within the PD team. They face the challenge of working in closer collaboration in order to create a successful product with components produced from multidisciplinary relationships, even though they belong to different expertise areas. The combination of these factors leads to a generic market pull of intelligent LLS that takes into consideration the cross functional collaborations and the necessity to create supportive knowledge management systems that share knowledge on LL between members involved in a project. In addition to this, the LLS should not add extra workload in the sense of mandatory knowledge sharing, such as, writing reports, but should be seen as supporting knowledge in suggestions of supporting success factors. For example, it can include interpreting technical reports from different expertise areas, which involve interpreting unfamiliar technical terms or writing additional LL reports that can be partially difficult since it can include describing LL experience that often is

Figure.7 Generic market pulls of a LLS that decreased workload.19 unconscious, tacit, and only perceived in day-to-day activities.

This leads to a generic market- pull towards a LLSs that supports the increasing demands of larger cross-functional collaborations in the early stages of product development. In order to decrease the workload of working in cross-functional collaborations due to the increasing complexity of products and development of production over time, there is a need to give a facilitated system that decreases the workload which is accumulating by increasing the blue volume illustrated in figure 7.

4.2 Lessons Learned Systems – from customer expectations to benchmarking metrics Given the theoretical framework and trends of using LL, a main issue was selected that had the possibility to support the activities in the early stages of PD through sharing LL. The goal of the suggested issue was to feature investigated LLS concepts that could take advantage of the right knowledge at the right time whilst taking into consideration both tacit- and downstream knowledge used in cross- functional.

To justify the choice of the issue, five strategy questions were defined and considered. This was done through an adapted question-asking method called the 5-Why method (Ohno, 1988) that considers LL sharing through a closed loop system in the early stages of PD and managed LL in a systematic manner including how to collect, verify, document, disseminate, and reuse LL. The original 5-Why question method consists of five following questions with answers from a single question in order to identify one root cause of a problem in the final answer. It was chosen to adapt this method to the LLS for it to not only focus on a specific problem but also create a holistic view of the LLS and its main root causes. The method was divided in two parts. The first part concerned five following questions on why to share LL in the early stages of PD. This gave the outcome of a single answer: because it is important to find the right information at the right time.

The second part took into consideration all five of the following additional questions and answers that were described in the context of using LLS, followed by five main answers and required features needed to solve them. These were then used as references for the following work. The formulated questions, answers, and features needed were interpreted and based on literature reviews, state of the art reviews, and brainstorming sessions with experts on the topic of LL and KM. The interpreted requirements of a satisfactory LLS was then followed by a correlated requirement/ metric in order to find features for improvement that could be found in lightweight technology.

4.2.1 Formulating features needed Explained below is the sequential question of “why is it important to find the best (right) information at the right time?” followed by answers. Included the answers is also additional features needed in order to improve a closed loop LLS in the early stages of PD.

*Why is it important to find the best (right) information at the right time?

1. *To be able to use knowledge that has shown to be successful in previous projects.

Through the reuse of success factors from previous or on-going projects it allows companies to save time and money by avoiding “trial and error” approaches. This involves LL from similar projects or activities.

Feature needed: The LLS has to store knowledge elements from previous projects in the form of LL on improving success factors on method of working and success factors based on the final product. And it should give guidelines on collecting, documenting, verifying, extracting, and reusing LL.

2. *To avoid drawbacks in on-going projects.

This can be done through acknowledging mistakes and drawbacks from previous projects. This is considered since the main goal is to create a LLS that involves knowledge gained from both undesirable and desired experiences.

Feature needed: The LLS has to store success indicators and rank knowledge on LL based on the importance of drawback in the final product. Regarding the documentation of LL and aiding the reuse of LL.

3. *To save time (and often money) in unnecessary work efforts.

By reusing LL that provide positive or negative outcome, positive knowledge on LL can be adapted and improved. Also knowledge on drawbacks can be avoided to facilitate an efficient work method in the PD activities.

Feature needed: LLS has to give the possibility to record user input on applicability in the form of LL knowledge from interactions with the product, regarding the documentation of LL and the verification of LL in the system.

4. *So that effort and resources can be applied to other activities.

By saving effort and resources on reusing LL from previous projects it is not always necessary to commence projects from scratch. This leads to available resources that can be assigned to other activities.

21 Feature needed: LLS has to label knowledge on its context, where it is useful e.g. with different kinds of tools or methods. And it should also support the possibility to build on reused LL.

5. *To enhance performance of the final products outcome.

The collective procedure of saving effort and resources leads to the possibility of additional time spent on development activities.

Feature needed: LLS has to categorize the information of LL based on its improving knowledge e.g. saved time, avoided accident, etc. regarding the verification and reuse of LL.

All features needed could be connected to the initial root cause of the problem. That was to develop an LLS that was facilitated with guidelines and could in a systematic manner collect, verify, document, disseminate, and reuse LL.

4.3 Requirements definition for the LL system The high-level expectations and features formulated in previous chapter and needed in LLS to support the early stages of PD were further cascaded down and detailed in terms of requirements (Table 1). In order to develop the requirements’ list, the author has used as reference the supporting success factors in product development as stated in Leonard- Barton (1992) knowledge sets:

1. Knowledge and skills embodied in employees.

2. Employee knowledge and skills embedded in technical systems.

3. The managerial system is the creation and control of knowledge.

4. Values and norms, transversal to the other three dimensions.

In addition to Barton´s knowledge sets, the requirements definition activity was also aided by the list of critical product development success factors presented by Cooper & Kleinschmidt (2007). These factors include:

5. Proficiency of technological activities.

6. Proficiency of marketing activities.

7. Proficiency of up-front (homework) activities.

8. Speed to market.

9. Proficiency of financial/business analysis.

22 The list of “Knowledge sets” and “Success factors” served the purpose of defining relevant and self-explanatory requirements from the list of features highlighted in the previous section. They have been used to guide the selection and decomposition of the features. Taking feature 1 as an example, knowledge sets and success factors have helped in synthesizing the crucial aspects of the requirements for the LLS. Feature 1, for instance, expresses 2 requirements, requirement number 1 and 2 as seen in the table (Table 1). This decomposition was added by the knowledge sets and success factors. The rationale is that, to define a requirement, knowledge sets and success factors needs to be met in order to create a knowledge repository that is measurable, attainable, relevant, time bound and supports successful reuse.

Table.1 Requirements of the LLS

23 4.4 Defining a metrics for the LL system The requirements’ list has been cascaded down to quantitative and qualitative metrics, mainly through brainstorming exercises that have involved researchers with experience in product development projects. Ideas generated in the brainstorming exercises were aimed to give elements that could be supported by lightweight technology. These elements could be further mapped against the functional requirements defined in the previous step.

Two brainstorming sessions were performed using different methods, in order to facilitate idea generation on the topic of knowledge sharing. Specific rules were set in order to aid the sessions. These rules included:

1. Time-limit of 20 minutes sessions.

2. All ideas should be accepted.

3. Build on other’s ideas.

4. Generate as many ideas as possible.

The brainstorming methods used were Alter Ego and Working with Gnomes. Participants in the Alter Ego session were asked to generate ideas on how to share knowledge with an alternative mind-set in order to generate as many ideas as possible and avoid limitations based personal preferences. The second brainstorming session Working with Gnomes was conducted to aid the participants in sharing knowledge with infinite assistants in order not to limit the idea generation by personal physical labour.

The generated ideas were clustered in four groups on a chart. The ideas that did not use lightweight technology were translated or associated to software solutions that either used or could be used in lightweight technology. The Y-axes and X-axes on the chart consisted of pull or push information, corresponding to searching or, by automated means, getting the information needed. The X-axes consisted of getting information on people and Y-axes getting information on LL according to figure 8.

Figure. 8 Ideas generated during brainstorming exercise clustered on pull or push information on people or LL.

24 Conclusions from the brainstorming exercise gave ideas on which elements could effect and develop optimal LLS with lightweight technology. These were then interpreted in terms of metrics indicators for the evaluation of different concepts (and could be through amongst other things; user perspective that could be measured quantitative or qualitative):

Interface interaction Interface interaction, which can be qualitatively measured in terms of how much users consider the LLS interface user friendly to interact with.

Media richness Media richness e.g., video, text, audio and pictures, which gives the possibility to use different media that could be measured qualitatively.

Context Giving information on current LL and its context, which can be measured quantitatively as yes/no if the system has ability to structure the content.

Web--based Web based tools, which can be measured quantitatively as yes/no.

Time (to capture LL) Time to capture L, which describes the time to record the LL, and can be measured quantitatively in minutes and seconds.

Time (to identify LL) Time to identify LL, which describes the time to post process and verify the LL by the user, and which can be described and measured quantitatively observing the time spent by the user on the system.

Interaction Interaction and building on existing knowledge through feedback, which can be measured quantitatively as yes/no if the system has the opportunity to incorporate web 2.0 feedback functions.

Comments Commenting capabilities in the sense of the possibility to give and take specific knowledge from comments, which can be measured quantitative as how many feedback options the system incorporates.

Applicability Label knowledge on applicable area of use and to what role, discipline, etc, which can be qualitatively measured in terms of users perception on how well the system facilitates the categorization of LL information.

Success factors Describing visualize drawback and success factors, which can be measured quantitatively as yes/no if the system could support it.

25 Usability Usability, which can be qualitatively measured in terms of how much users consider the LLS system interface comprehensible without required explanation.

A correlation matrix was developed to ensure all the requirements were covered by the metrics indicators requirements and metrics were correlated as shown in Table 2 (were 1=correlation and “blank”=no correlation). The matrix has been iteratively developed, modified and validated with the support of researchers’ with deep knowledge on LL supporting systems through there inputs and comments.

Table. 2 Requirements vs. Metrics mapping (1= correlation and “blank”= no correlation)

26 5. Generating Lessons Learned System concepts In order to generate solution concepts a generic knowledge lifecycle has been used to decompose the problem into more manageable sub-problems.

This thesis will from this point on (amongst other things) describe:

1. How the complete and potential LLS approaches have been reviewed define which LL process to focus on.

2. How the generic knowledge lifecycle has been used to decompose the LLS approach to aid the identification of suitable solution concepts.

3. How the LLS concepts have been generated and how do they embed Web 2.0 mechanisms.

5.1 Review of complete and potential Lessons Learned System LLS can be divided in two groups: complete and potential.

A complete LLS is a knowledge system based on experience from previous or ongoing projects, which is then implemented and contributes value to following or ongoing project. This means that the knowledge life cycle is closed and has moved from cradle to grave and contributed value to the project supported by improved critical success factors. The structure of the knowledge can be further developed through contributions from users through adding and/or commenting information on the knowledge. It is therefore most suitable to say that the knowledge life cycle should have a closed loop also known as from cradle to cradle as described by Murphy (2010). In this sense, knowledge has been converted to LL and continues to develop based on the contributions from re- users by allowing them to build on the knowledge in the LLS repository.

Potential LLS is based on experience from previous or ongoing projects which for some reason does not reach the intended re- user, or if the contributed LL knowledge does not add any improvement to the project. Before the hypothesis becomes evidence, it must be proven by tests and results. This means that if the original criteria are not met the knowledge can only be seen as a potential LL. The knowledge has potential but needs to be reinterpreted, adapted, and further re-used to see if it contributes with improvement that can be justified as an LL in a complete LLS.

5.2 Approach of Lessons Learned System Different systematic pre-existing methods of LLS were studied in order to explore and develop a system that fit the early stages of PD. Two main LLS general approaches were

27 identified. The two approaches either used personal assistants in order to support the sharing, finding and using of LL (e.g. multi-agent system for acquiring and sharing lessons learned) or based on involving the users of the LLS (Intelligent delivery of military lessons learned). Practical examples of the different approach were described with pros and cons in the following chapter in order justify the final choice of system to develop further.

5.3 Advantages and disadvantages in studied Lessons Learned System Advantages and disadvantages of the investigated LLS approaches were studied and compared with respect to the defined requirements.

1. A multi-agent system for acquiring and sharing lessons learned

Advantages: Information sharing of LL can take place on a daily basis in order to take advantage of current knowledge in LL and does not add workload to the PD team since it is done by devoted agents.

Cons: Knowledge must go through an intermediary, thus adding an extra step of sharing LL which adds extra time before it can be reused. Also the LL could be incorrectly interpreted thus giving the re- users insufficient knowledge about the LL.

2. Intelligent delivery of military lessons learned

Advantages: By using monitored distribution of the LL it can be linked to the context and the target process of the organizations. It reduces workload through automated information sharing of LL. The users can predict potentials by examining the lessons rationale. The translation of these lessons is guided through descriptions of all necessary attributes.

Cons: Since the LL information is added in the system by people involved in the project and is not interpreted through an intermediary it can contain information in the form of unnecessary noise.

5.4 Choice of Lessons Learned System approach Based on the initial requirements, it was chosen to investigate and further develop the Weber & Aha (2003) LL process (Intelligent delivery of military lessons learned) in order to develop a system that could be supported by lightweight technology with functions of Web 2.0. The approach combined with Web 2.0 functions enabled the users of the system to interact with the system. This allowed them to determine what LL knowledge it should contain, justified by the opinion of reused LL.

28 5.5 Decomposition of Lessons Learned System steps In order to further develop the Weber & Aha (2003) LL process with functions in lightweight technology that supported the early stages of PD, it was needed to decompose the complex problem into smaller and manageable sub problems. This aided in generating adapted suggestions on improved concepts and their components. An information transfer cycle of the LL process follows five steps according to Weber & Aha (2003). This initial process was divided in these steps as to be able to give suggestions on alternative improvements in collecting, verifying, storing, disseminating and reusing LL. The solutions were based on interpreted ideas generated during initial brainstorming sessions combined with literature studies and web based field studies on pre-existing Web 2.0 functions. The decomposed complex problem and its divided smaller and manageable sub problems of the LL process were described as followed:

Collect LL

How can the collection of LL be performed and in which form should it be documented?

In this sense all alternative media were investigated, such as, collecting LL through text reports, video recordings, audio recordings or/and pictures. It also includes the question of when to capture the LL. Questions investigated decided whether or not documentation should be performed during or after the action was performed (e.g., LL as AAR). Based on the sub problem of collecting LL it was chosen to examine text reporting in comparison to video recordings which included vocal and picture illustrations. Advantages and disadvantages in the two approaches were then considered and compared described in table 3.

Table. 3 Advantages and disadvantages of reporting and capturing LL through text or videos.

The pro´s from both text and video recorded LL was used in order to generate concepts.

29 Verify LL

How is it possible to verify if a LL is useful and should be stored or not, in other words how to avoid “garbage in garbage out” in the LLS repository? E.g. how can LLS verify if the LL is useful in order to avoid a filled repository with unnecessary information? Should it be dependent on the users own common sense or should it follow a specific SMART (Specific, Measurable, Attainable, Relevant and Time-bound) criterion? The solution was to create the verification through supported functions Web 2.0 and its interaction abilities. Scoring and commenting from re- users of the LL knowledge could verify if the LL was useful in practice through their experience and opinion.

Store LL

In what forum should the LL be stored? Should the LL be stored in the form of video logs, blogs or mini blogs? Two main criteria were based on the requirements, i.e., limited amount of information describing LL document and that it should be stored in a web based repository in order to be shared electronically. The first criterion was set in order to limit the information so as to make it specific and exclude unnecessary information (noise). The functions of the limited amount of storage space were inspired by the Twitter methodology, which restricts the users in reporting with a maximum amount of characters in text posts (140 characters). The following outcome of the methodology guides the users in storing curtail and specific LL knowledge since there is no space for noise. The effect of limited amount of posted information guides the users to prepare what LL information to record and store. Through rethinking before documenting (storing) it creates a pre-processed LL that is filtered by the initial user. The second criterion was justified based on avoiding the old form of storing in archival information in which to avoid intermediary assistants (professional archivist) in the LLS.

Disseminate LL

How can LL be disseminated? How can the users of the LL be able to access it? Should it be through searching for tags? And in that case, how should LL be tagged in order to make it searchable? Should it be possible to find direct contact information to the knowledge source (founder of the LL) with the possibility to reconnect through video conferencing, email, MSN (Microsoft Network Messenger) etc.

Different lightweight technology functionalities were investigated that aids the user in disseminating the LL. Some of the functions found were:

1. Subscribing to the disseminators LL knowledge posts. The functions of subscribing to LL knowledge was inspired by the method of lightweight technology in RSS (Really Simple Syndicator) feeds in which the disseminators can syndicate the content automatically so that the re-users of the LL knowledge can get summarized feeds (subscriptions).

30 2. Proposal of LL based on previous searches and visited sites (web crawler). The functions of lightweight technology that supported the push method can be in form of tracking the electronic footprints, through the user's pattern of events that is recorded and provide reference data on the area of interest. This can be the basis for recommendations on current information (using web crawlers and spiders). 3. Proposed LL knowledge based on web profile. With a preset profile on what the user currently is working on as to get the right feeds on LL at the right time. Similar functions in lightweight technology are used in web based communities where suggestions on events and commercials are sent to the user based on their interest registered in their profile.

4. Searchable web based repository, based on tags on information. E.g. in order to subscribe to the interested LL knowledge sources they have to initially be found. That is why there is a need to have a searchable repository similar to the functions of lightweight technology as Google Search and other web based search engines.

Reuse and build on existing LL

How to re-use and build on existing LL? What approach should be taken in order to make it possible to get feedback from the re-users? As previously mentioned, LL can only be seen as potential LL in form of knowledge until it is reused and adds effect in the form of improvement in the reapplied activity or event. In this sense feedback through ranking in Web 2.0 functions was investigated. E.g. thumbs up and thumbs down in YouTube with additional comment opportunities. This gave the user an opportunity to add comments on their experience from reusing the posted LL.

31 5.6 Concept generation The clarification and decomposition of the problem combined with brainstorming sessions, literature studies, web based field studies on how to share LL and knowledge gave the support to generate three main concepts. LLS concepts with functions supported by lightweight technology and could support engineering teams in the early stages of PD.

Concept 1: Video sharing

This concept is based on video recording of the LL in order to capture interactions in the activity or event. Collecting LL starts with locating the LL in day-to-day activities followed by video recording the activity or event. Time tags are added according to topics discussed in the video recording. The topics explained in the LL video describe the context, measurable (in order to compare it with previous actions e.g. time saved etc.), suggestion on alternative contexts and describing the tools and methods used. The video recording is then uploaded to be shared with others through a web bases repository. When re-users view the recordings, they are supported with time stamps that aid in locating specific information in the video example 2 min 12 seconds in to the video the results of the LL are discussed described in figure 9. When the spectators have reused the LL in the video, they have the opportunity to rank and comment the reused LL. This gives feedback to other users and verifies if the video was useful or not. The additional function of adding comments creates the opportunity to add information to the re-used LL with the outcome of building on re-used LL knowledge.

Figure. 9 Concept 1 Guided agenda of video recorded LL with time stamps on topics discussed in the video.

32 Concept 2: Forums

This concept is based on lightweight technologies with forum functions to generate interactions through posting an initial question on a specific topic in a common forum. The interface enables the users to post comment, answer, and get feedback from others questions associated to the initial question e.g. figure. 10. The LLS concept has a top down structure since the LL knowledge is based on an initial question in order to be verified and defined through the aggregated discussions and answers from the forum community. Example of this interaction can be described as a following scenario:

Person 1

Event: I have acknowledged that painting my bicycle with two clear coats of paint can prevent corrosion since the parts on my bicycle that have only one clear coat is corroded and the part with two clear coats are not. LL:

Paint bicycle with two clear coats of paint to avoid corrosion on a bike that has been exposed to two years of outdoor use.

Posted initial question in forum repository

Topic: Prevent corrosion

Question: How can I prevent and avoid corrosion on exposed metallic surfaces? Answer to own question: Paint bicycle with two clear coats of paint in order to avoid corrosion. Person 2

Event: I have acknowledged that it takes longer for the clear coat to dry outdoors in comparison to let it dry indoors.

LL:

It is possible to save 1 hour by drying a painted bike in an indoor environment in comparison to outdoors. Subsequent topic posted on forum repository

Topic: Save time when painting.

Question: How can I save time when painting?

Answer to own question: Dry painted objects in warmer environment in order to save time.

The posted LL is always formulated as a question that engages the forum community to build on the LL knowledge through adding comments and follow-up questions. This leads to creating new LL through other peoples experiences in additional contexts that then starts new topics discussed in the forum.

Figure.10 Open interfaces, forum using questions that lead to new topics discussed in the forum.

34 Concept 3: Social networks

This concept is based on reconnecting to the knowledge sources (personally experienced LL) through a review aggregator in the sense that the search on a topics of LL gives contact information (i.e., telephone number, e-mail, blog, or twitter address) to people that are recommended and could give information on the LL that is needed. The review aggregator is connected to information of a created profile in a web-based community. The profile consists of posts on current summarized LL and information on what individuals have previously worked on, current work and total of experience (e.g. how many years of experience), which can justify the LL. The profile resembled functions off blogs, e.g., readers can subscribe to blog posts to keep up to date with newly added LL. If additional information is needed it can be given from reconnecting to the LL source through text messages, video meetings, etc described in figure 11.

Figure. 11 Concept 3 Connect with knowledge source

35 6. LL system concept selection and detailed description In this phase of the thesis the generated concepts were compared and evaluated with respect to the formulated requirements by the intended users. The importance of elements that could be improved by lightweight technology was ranked in accordance to how suitable they were utilized in the different concepts. The ranking was divided into three scales one, five and nine accordingly to following;

1. 1 - Low = elements was not supported by lightweight technology

2. 5 - Medium = elements was to some extent supported by lightweight technology

3. 9 - High = elements was supported by lightweight technology

The ranking numbers are then added to obtain a total referred sum as to have a comparing number, which was used as a selecting parameter in the concept selection table 4.

Table. 4 Concept selection matrix

The concept with the greatest potential based on the ranking was selected as the main concept for the lightweight LLS. With the score of 79, the video sharing approach was selected in order to proceed in the PD process. The development was an iterative

36 process and high ranked functions from the other concepts were also considered in the detailed design.

6.1 Detailed design The conclusions of weighing the text- and video-recorded of LL combined with the ranking of the concepts in Table 4 lead to a formalisation of a prototype concept with an initial process map figure 12. In comparison to Weber & Aha (2003) process, the adapted approach engaged the users to verify the LL when they re-use it. This was done in order to not exclude any important LL knowledge.

Figure. 12 Process map

Sequential steps were formulated in order to aid the process

1. Locate LL in on-going day-to-day activities and record it with easy to use and accessible video equipment.

1. Locate LL from previous projects stored in the LLS´s web based repository.

2. Prepare a recording agenda in order to document structured LL videos.

2. Document timeline on discussed topics in the agenda bullet points in order to locate interesting topics to be discussed in later stages.

2. Upload the video recording to web based repositories.

37 2. Tag videos based on projects, roles and topics.

3. Subjects that are wanted can be searched in the open interfaces.

4. Select specific LL according to tags in global search and local tags in video time line. This is done in order to mimic interactions in the video recordings in order to give the desired outcome of the LL.

5. Verify and build on reused LL through ranking and commenting on experiences and opinions of reusing the LL in the videos.

Agenda description

In order to support the final process map, sequential step were supported by suggestions of a structured LL agenda, Web 2.0 functions and easy access recording equipment.

Table. 5 Video recording agenda with time line, comments and ranking options.

The initial benchmarking of different textural LL agendas with additional LL elements in different forms was revisited to ensure that necessary knowledge was captured in the agenda bullet points used – as also described in Table 5:

1. Working Context: Presentation of the participants and their context e.g. work role, what task e.g. if it is a user of the developed product or if it is in the PD stage.

2. What went well: Task disruption in the sense of tools and methods used.

3. What went wrong: Are there any problems notified and where? Problem statement?

4. LL: How can the problem be avoided? What was the action for new improving solutions? Explaining the experience whilst dealing with the action?

38 5. LL measurable: What was the improvement while giving measurable comparisons with previous approaches?

6. Applicability: Examples of alternate areas that can benefit from the LL? Suggestions of what to avoid and what to benefit from? Summary statement.

Lightweight technology

The lightweight technology could support web based video repository, tagging timeline in videos, comment and ranking options.

Since the project did not include software development it was chosen to use existing software solutions to provide desired functions. In regards to web-based repository, YouTube’s1 figure 13 functions could be used in the regards to uploading videos, ranking and commenting features.

Figure. 13 Example of how the agenda bullets can be used to support the users in navigating through the video.

The functional element of local tagging of the video timeline was illustrated in the functions of figure 13 that enabled to add tags in order to find specific topics discussed according to the in the agenda followed in the video. Thus enabling the re-users to find specific LL knowledge.

1 http://www.youtube.com/

39 7. LL system testing and validation This chapter describes the testing phase of the LLS in the PD process, including three environments and setups that were conducted in order to test and verify the chosen concept.

Preliminary validation in a laboratory environment

The initial test setup was conducted with a selected group of people (including the author) using to evaluate and verify the feasibility of the video approach for LL capturing and sharing. The initial tests were conducted on scenarios and activities not immediately related with product development, as a way to lower the threshold for knowledge capturing and reuse. These generic examples had the purpose of testing the proposed guidelines as well as to test the different recording equipment. This “learning- by-doing” approach has been found useful refine the approach definition by collecting a deeper insight on the user unexpressed needs.

Screenshots from the LL videos recorded during this first phase are shown in Figure 14.

Figure.14 People interacting and trying out the practical use of LLS concept using a mock up with over- imposed agenda bullet points in the video using the annotation tool embedded on YouTube in order to time tag the discussed topics in the video.

The initial test contributed to better understanding and modifications of:

1. The mock- up concept and how it can be used in practice. It also gave knowledge on needed recording equipment, video editing software and how to use these in practice based on pre-existing lightweight technology.

40 2. Time tags in order to manage these in an effective manner and how to manage the post-processing of the recordings in order to upload the data to the LLS repository.

3. What technical equipment is the most suitable for the purpose of recording LL. The test scenario was initially conducted using built in laptop web-camera followed by using mobile phone cameras. Webcams and mobile phone cameras, for instance, although quite similar in terms of resolution, have been found very different in terms of handling and usability. These preliminary observations have contributed to develop a framework to benchmark recording equipment vs. the purpose of recording LL in a PD setting (Table 6).

Validation in a research environment

In order to further develop the LLS based on the initial tests, additional validation activities were conducted with participants working in PD on industrial environment applications. The LLS mock-up was tested in a welding workshop and in an office environment concerning PD of welding techniques and maintenance systems services at Luleå University of Technology. Recording equipment ware further tested (Table 6). The elements tested and compared on the equipment were with regards to setup of the necessary components in order to record. The second aspect was how fast and easy recordings could be exported and imported from the different equipment used. The third was the quality of the video, taking into consideration that the LL should be easily understood and not limited by the resolution of the recording. The fourth and final aspect was convenience, since the aim of the LLS was documented in day-to-day activities and the importance of having easily accessible equipment.

Table. 6 Benchmarking of video recording equipment using cell phone camera (Sony Ericsson Xperia mini), in-built laptop webcam and a studio camera (Sony HDR FX1E).

The research environment test gave a comparison of different equipment with focus on recordings with cell phone and studio equipment. It contributed to a better understanding of using mobile recording equipment. It showed that the recordings could sufficiently be conducted with the use of a cell phone even though the quality in

41 light, resolution and audio was better in the studio equipment. In correlation to the extra work effort in recording with studio equipment it did not add additional noticeable value to the understanding of the recorded LL. In addition to benchmarking the equipment, the test also gave an understanding of the need to post- process the topics described in the recordings in order to describe the required knowledge. This can create understandable and successful LL recordings.

Validation in design sessions with students

In order to validate the LL video sharing approach both from the perspective of capturing the lessons learned from a realistic design task, and from the perspective of applying such lessons learned for new designs, an 2-step exercise (on Sept. 27th 2011 and October 4th 2011) was conducted with the student in the M7014T - Produktutvecklingsprocesser (Product Development Processes) course, to verify in a laboratory environment the underlying mechanisms of capturing and reviewing the lessons learned.

Exercise 1 – The Bridge exercise

The purpose was to build a structure resembling a bridge, according to the given requirements and within the allocated time slot of 45 min.

The bridge-like structure should only have been built using A4 paper sheets, with no tape or glue. It was supposed to be positioned between 2 tables that were 50 cm far away, and it should have been at least 40 cm high. In the end, the structure should have been able to stand a weight of approx. 0.5 Kilograms.

The bridge was composed of 5 main parts: the base, the pylon, the beam, the platform and the counterweight. The high-level schema of the bridge-like structure is represented

in Figure 15.

Figure.15 structure resembling the bridge in exercise 1

42 After having designed, realized and tested the structure, the two groups have been asked to collect the lessons learned from the exercise using video recordings.

The first team – which eventually did not succeed in meeting the requirements - used the structured video format, i.e. has recorded the LL using the guidelines provided and described in the previous section. A professional camera (Sony HDR FX1E) was mounted on a tripod and used for recording.

The second team – which eventually succeeded in meeting the requirements - had been given more freedom, i.e. they have not been provided of specific guidelines, but only of a generic introduction on the meaning of lessons learned and on how to use the technical equipment’s. For this recording, the students have used the in-built IPhone 4 camera.

In the second stage, the purpose was to verify if the students could make use of the lessons learned recorded in the previous step to cope with a more complex assignment in less time. For this reason, the students were randomly selected and divided in two groups. Some of the students had taken part to the previous exercise, while some did not. Some students belonged to the team that succeeded at stage 1, while others to the team that failed.

Exercise 2 – The Crane exercise

The objective of the second step of the exercise was to build a more challenging crane- like structure (i.e., similar to the bridge-like structure, but using only one pylon) and to verify how the LL videos recorded during the first session could help the teams in improving their design performances.

The requirements for the crane were similar to the ones used for the bridge exercise, i.e. a distance between the Pylon and the Platform of 250mm and 400 mm height. The exercise featured also a new material - LEGO® bricks - together with the paper, to make the exercise more complex and to introduce more ambiguity in the design Figure.16.

In this case the two teams have been given less time to complete the tasks (i.e., only 30 minutes) together with more challenging requirements: with only 1 pylon more counterweight is needed to balance the weight, which ask for the beam and the pylon to be reinforced, and the base to be more solid and stable.

In preparation of the exercise, the students have reviewed the lessons learned videos created for the bridge exercise.. Both the team essentially succeeded in the task, although one of the teams had problems with the structure

approaching 90% of the weight limit.

Figure.16 structure resembling the crane in exercise 2

Table 7 collects the reflections from the students about the exercise, and gives indications of Pros and cons of the approach.

Pros Cons Suggested improvements Relevance of  The exercise  LL videos  The outcome the exercise replicated quite were not of the nicely a product relevant exercise was development because some somewhat problem. people knew biased by what to do the unequal  It is a good from the distribution simulation of a previous of the Lego real PD project, exercise. bricks. since it features challenging  The deadline requirements, was so strict strict deadlines that we did and team not really member with apply a different skills. product development  The “process”, introduction of rather we the Lego bricks worked in a well mimicked trial-and- the introduction error mode, of a new making a lot technology in a of prototypes, real PD process. Relevance of  We could  Our “tacit”  the LL for the successfully experience development conclude the from the process. exercise even previous with a more problem complex helped a lot. problem 25% less time.  The team succeeded  The LL videos because it limited the understood numbers of how to use technical the Lego solutions to be bricks explored to find correctly, and a good one. that was independent  The LL videos from the LL have been used reviewed.

44 as the main benchmark for the development of the crane. Relevance of  The video was  Using text  After trying the video great to explain instead of a couple of approach for a complex videos would times, capturing LLs feature or have recording structure in just probably the videos is 2 or 3 seconds. added a lot of likely to details, (but it become a  I found helpful would have routine, so to have the 6 required the message stages to guide much more is delivered me in making time). more the video. effectively.

 I did not find difficult to stand in front of the camera.

 I would have been much harder to share the LL from the exercise with a textual report. Relevance of  You can take  Not all the  It is import- the video advantage of relevant ant to tell approach for what the other aspects of the the people reviewing teams have construction right from LLs learnt in the were the beginni- step before. revealed in ng what they the video. need to do.  The video gave  LLs have complementary limited the information innovation opportunity  Good to see the to some videos in the extent, i.e. the beginning teams has because we only applied could learn “old” and from our “known” mistakes. solutions to the new  I think it was problem and useful to see the did not make actual any effort to construction of explore the bridge. different (and

45 potentially  For beginners even better) the videos were solutions. We interesting already had a because I could way that see which worked, components were working  By watching and which were the LL videos not. we completely  The LL has skip the trial- contributed in error phase, this, especially which was because they good source have prevented for us to make the innovation in same mistakes phase 1. all over again.  The LL had  The video gave the effect to me detailed standardize knowledge the two about how to solutions, build it. which in fact are mostly  After knowing similar. the basis on how to build the bridge I thought we had more possibilities to have a successful product.

Relevance of  Structured  Unstructured  The details structured videos were videos were of the guidelines very descriptive more helpful construction and helped in to grasp the solution understanding technical should be the task better. details, highlighted because the more in the  In the camera was existing unstructured freer to guidelines video the task glance description was through the missing. construction details.

Table.7 Collection of reflections from the students exercise, that gives indications of Pros and cons of the approach.

The answers from the participant gave feedback that supported the initial requirements of the developed LLS. Thus was sufficient to verify that the system had potential to support the early stages of PD. But it should also be considered that the exercise teams consisted of participants that were selected and grouped randomly at a small class during a course at Luleå university of technology. Both exercises were conducted with a mixture of same participants from the course and create potential errors in the test result, as same participants in both the group that recorded the LL experience and future reuses their own experience. That is why it is suggested to complement the tests with additional test setups that involved participants that are independent of each other in order to get more accurate and non-partial test results.

The follow-up tests could indeed follow the same structure as the ones in this thesis since it gives a clearer understanding of PD improvements based on LL recordings.

The structure is based on two exercises that consisted of one easier exercise followed by a similar exercise with higher severity in order to see if improvements could be made.

47 8. Concluding remarks

The objectives of the thesis were to propose a lessons learned system that could support engineering activities in the early stages of product development by facilitating the cross-functional sharing of “downstream” lessons learned from the later lifecycle phases (e.g., from manufacturing, maintenance, end user, etc.). The LL approach has proven to useful in reuse of;

1. Supporting cross functional collaboration.

2. Establishing a general system that took advantage of and facilitated the collaboration among all people involved.

3. Speed up the process of finding the best solution to the PD.

This was achieved by developing a lightweight lessons learned system able to exploit Web 2.0 capabilities to lower the threshold for capturing the lessons learned and to enhance the interaction between the system users. Web 2.0 concepts introduce a more participatory approach to knowledge sharing, turning individuals from passive users to active contributors, publishing, commenting, tagging and ranking information for others.

Videos are intended to be uploaded on a web-based repository, to enable the LL videos to be subscribed and searched through the time tags and tags on its described context.

During the empirical study we have observed that video tweets can be beneficial as knowledge carriers for several reasons. Firstly, videos lower the threshold for capturing the personal experience and observations compared with written reports. Then, they are less time consuming to review, especially during a team meeting. Thirdly, videos can capture the context of where a lesson learned is generated. Furthermore, they allow navigating through the product, to better explain complex concepts such as the interaction between parts. Eventually, videos allow picturing dynamic situations that are not easy to capture using still photography or text.

The development of methodological enablers for supporting lessons learned sharing using video tweets has to be considered the main result of the thesis.

The work has proposed and detailed a video sharing lessons learned system concept, which has been prototyped to support – through continuous testing activities - iterative development and validation.

Moving from existing examples of video sharing solutions, the thesis has detailed how videos should be structured, recorded, categorized and shared for the benefit of product development teams. This has included the exploration of how ranking mechanisms as

48 well as likes/dislike feedback can support users in identifying the LL videos that have the best quality or that are most relevant for them, for instance exploiting.

Recording lessons learned is not just about switching on a camera. The users need be guided through the video-making process to produce something other users can benefit from.

Hence we have developed a framework and guidelines to support the users in structuring the videos around crucial information – such as task description, measurements, applicability - that makes lessons learned worthwhile to be shared (display the framework and mock-up idea here).

Time stamps have been also introduced to browse the video list and to help the users in jumping from a topic of interest to another.

The most important verification was from the re-users that experienced the reused LL.

In terms of technical equipment, the testing activities has shown that smartphone cameras ensure enough audio and video quality for the recording, of day- to- day work activities.

One area that could be future studied is the aid of streaming technology, which enables the recording to be automatically uploaded from live recordings. E.g. create smartphone applications that can be downloaded and used as supporting software solution. Since the PD process did not concern any software programmed solutions it should be highly considered for future work, in order to retest the final system solution thus verifying if it works or not.

49 References Allen, G., Major. (1994). After action review in military training simulations.

Baird, L., Handerson, J. C., & Watts, S. (1997). Learning from action: An analysis of centre for army lessons learned (CALL).Vol.36 No.4, pp385–395.

Bertoni, M., Eres, H., & Isaksson, O. (2011). Functional thinking for value creation : Proceedings of the 3rd CIRP international conference on industrial product service systems CIRP Conference on Life Cycle Engineering,Germany. 141-146.

Braun, M. D., & Macal, C. M. (1993). Development of a system for accessing lessons learned., Page 94.

Callot, M. (2003). MOKA: Methodology and tools oriented to knowledge-based engineering applications. Retrieved October 08, 2011, from http://web1.eng.coventry.ac.uk/moka/project.htm

Chirumalla, K., Larsson, A., Bertoni, M., & Larsson, T. (2011). Knowledge sharing across boundaries: Web 2.0 and product-service system development. Indian Istetute of Science,Bangalore. (International conference on research into design)

Cohen, H. (Ed.). (2002). Secrets of the obvious Infinity publishing.com.

Cooper, G. R., & Kleinschmidt, J. E. (2007). Winning businesses in product development critical success factors.

Koen, P. (2001). Providing clarity and a common language to the"fuzzy front end"., pp46-55.

Leonard- Barton, D. (1992). Core capabilities and core rigidities: A paradox in managing new product development.13(Special Issue: Strategy Process: Managing Corporate self- Renewal), 111-125.

Maffin, D., Thwaites, A., Alderman, N., Braiden, P., & Hills, B. (1997). Managing the product development process: Combining best practice with company and context.9(Technology Analysis & Strategic Management)

Murphy, G. (2010). Using web 2.0 tools to facilitate knowledge transfer in complex organisational environments. CRC for Integrated Engineering Asset Management (CIEAM)

Ohno, T. (Ed.). (1988). Toyota production system :Beyond large -scale production

ORreilly, T. (2008). Key differences between web 1.0 and web 2.0.13

Polanyi, M. (Ed.). (1966). The tacit dimension (First edition ed.). United states of America: Library of congress.

50 Tascal, C. A., & Barthès, J. P. (2003). A multi- agent system for acquiring and sharing lessons learned.

The Atlantic Monthly. (1862), Old age.9, 134-136.

Ulrich, K. T., & Eppinger, S. D. (Eds.). (2008). Product design and development (Fourth Edition ed.) McGraw Hill.

Weber, R., & Aha, D. W. (2003). Intelligent delivery of military lessons learned.

Williams, K. (2004). A summary of unmanned aircraft Accident/Incident data: Human factors implications No. DOT/FAA/AM-04/24). The Defence Technical Information Centre Ft. Belvior, VA. 22060, The National Technical Information Service Springfield, Virginia 22161: US Department of transportation, Federal Aviation Administration.