DEGREE PROJECT IN INDUSTRIAL ENGINEERING AND MANAGEMENT, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2020
Transforming Corporate Learning using Automation and Artificial Intelligence
An exploratory case study for adopting automation and AI within Corporate Learning at financial services companies
PETTER KLINGA
KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF INDUSTRIAL ENGINEERING AND MANAGEMENT Abstract
Transforming Corporate Learning using Automation and Artificial Intelligence
An exploratory case study for adopting automation and AI within Corporate Learning at financial services companies
by
Petter Klinga
Master of Science Thesis TRITA-ITM-EX 2020:233 KTH Industrial Engineering and Management Industrial Management SE-100 44 STOCKHOLM
2 Abstract
En ny era av utbildning genom automatisering och Artificiell Intelligens
En explorativ fallstudie kring möjligheten att implementera automatisering och AI inom utbildningsorganisationen på finansbolag
av
Petter Klinga
Examensarbete TRITA-ITM-EX 2020:233 KTH Industriell teknik och management Industriell ekonomi och organisation SE-100 44 STOCKHOLM
3 Abstract
Master of Science Thesis TRITA-ITM-EX 2020:233
Transforming Corporate Learning using Automation and Artificial Intelligence
An exploratory case study for adopting automation and AI within Corporate Learning at financial
services companies
Petter Klinga Approved Examiner Supervisor 2020-06-X Matti Kaulio Jannis Angelis Commissioner Contact person Sana Labs Joel Hellemark
Abstract
As the emergence of new technologies are continuously disrupting the way in which organizations function and develop, the majority of initiatives within Learning and Development (L&D) are far from fully effective. The purpose of this study was to conduct an exploratory case study to investigate how automation and AI technologies could improve corporate learning within financial services companies. The study was delimited to study three case companies, all primarily operating in the Nordic financial services industry. The exploratory research was carried out through a literature review, several in- depth interviews as well as a survey for a selected number of research participants. The research revealed that the current state of training within financial services is characterized by a significant amount of manual and administrative work, lack of intelligence within decision-making as well as a non-existing consideration of employee knowledge. Moreover, the empirical evidence similarly reveled a wide array of opportunities for adopting automation and AI technologies into the respective learning workflows of the L&D organization within the case companies.
Key words: Artificial Intelligence; Automation, Learning & Development; Corporate Learning; Corporate Training; Learning Workflows; Technology Adoption; Financial Services
i Sammanfattning
Examensarbete TRITA-ITM-EX 2020:233
En ny era av utbildning genom automatisering och Artificiell Intelligens
En explorativ fallstudie kring möjligheterna att implementera automatisering och AI inom
utbildningsorganisationen på finansbolag
Petter Klinga Godkänt Examinator Handledare 2020-06-X Matti Kaulio Jannis Angelis Uppdragsgivare Kontaktperson Sana Labs Joel Hellermark
Sammanfattning
I takt med att företag kontinuerligt anammar nya teknologier för att förbättra sin verksamhet, befinner sig utbildningsorganisationer i ett märkbart ineffektivt stadie. Syftet med denna studie var att genomföra en explorativ fallstudie gällande hur finansbolag skulle kunna införa AI samt automatisering för att förbättra sin utbildningsorganisation. Studien var begränsat till att undersöka tre företag, alla med verksamhet i den nordiska finansbranschen. Den explorativa delen av studien genomfördes med hjälp av en litteraturstudie, flertal djupgående intervjuer samt en enkät för ett begränsat antal deltagare i forskningsprocessen. Forskning påvisade att den existerade utbildningsorganisationen inom finansbolag är starkt präglat av ett överflöd av manuellt och administrativt arbete, bristande intelligens inom beslutsprocesser samt en bristande hänsyn för existerande kunskapsnivåer bland anställda. Studien påvisade därtill en mängd möjligheter att införa automatisering samt AI för att förbättra utbildningsflödena inom samtliga deltagande bolag i fallstudien.
Nyckelord: Artificiell Intelligens; Automatisering; Lärande och Utveckling; Företagsutbildning; Utbildningsflöden; Teknikinförande; Bank och Finans
ii Acronyms
Acronyms
AI Artificial Intelligence
RPA Robotic Process Automation
IPA Intelligent Process Automation
L&D Learning and Development
ML Machine Learning
SaaS Software-as-a-Service
AIaaS AI-as-a-Service
SFSA Swedish Financial Supervisory Authority
iii Table of Contents
Table of Contents
1. INTRODUCTION ...... 1
1.1 BACKGROUND ...... 1 1.1.1 Challenges and Opportunities in Learning and Development ...... 1 1.1.3 Technology-Driven Personalization ...... 2 1.1.4 A New Era of Learning ...... 3 1.1.2 The Financial Services Industry ...... 4 1.2 THE COMMISSIONING COMPANY ...... 5 1.3 PROBLEM STATEMENT ...... 5 1.4 PURPOSE ...... 6 1.5 RESEARCH QUESTION ...... 6 1.6 DELIMITATIONS ...... 6 1.7 EXPECTED CONTRIBUTION ...... 7 2. LITERATURE REVIEW ...... 8
2.1 CORPORATE LEARNING ...... 8 2.1.2 Learning Management Systems ...... 9 2.1.3 Integrating Training With Company Processes ...... 9 2.1.4 Organizational Learning ...... 10 2.2 ARTIFICIAL INTELLIGENCE ...... 11 2.2.2 Key components of Artificial Intelligence ...... 11 2.2.3 Adaptive and Intelligent Web-based Educational Systems ...... 13 2.3 ROBOTIC PROCESS AUTOMATION ...... 15 2.3.1 Capabilities and Areas of Application ...... 15 2.3.2 RPA in L&D ...... 16 2.3.2 Limitations of RPA ...... 17 2.4 UNIFYING RPA AND AI ...... 17 2.4.1 Tackling the challenges posed by RPA ...... 17 2.3 STRATEGIC IMPLICATIONS OF IMPLEMENTING AUTOMATION AND AI ...... 19 2.4 SUMMARY OF LITERATURE REVIEW ...... 24 3. DEVELOPING A CONCEPTUAL FRAMEWORK ...... 25
3.1 THE RATIONALE BEHIND DEVELOPING AND UTILIZING A CONCEPTUAL FRAMEWORK ...... 25 3.2 AI INDICATIONS OF OUTPUT ...... 27 3.3 AUTOMATION INDICATIONS OF OUTPUT ...... 28 3.3 AUTOMATION AND AI COMPATIBLE L&D PROCESSES ...... 29 4. METHOD ...... 31
4.1 RESEARCH DESIGN ...... 31 4.2 RESEARCH PROCESS ...... 32 4.2.1 Selecting the case ...... 33 4.2.2 Conducting a preliminary investigation ...... 34
iv Table of Contents
4.2.3 Collecting data ...... 36 4.2.4 Analyzing data ...... 40 4.3 QUALITY OF SCIENTIFIC RESEARCH ...... 42 4.5.1 Reliability ...... 42 4.5.2 Validity ...... 43 4.5.3 Source Criticism ...... 44 4.6 RESEARCH ETHICS ...... 45 4.5.3 Academic Perspective ...... 45 4.5.3 Professional Perspective ...... 46 5. RESULTS AND ANALYSIS ...... 48
5.1 STATUS QUO OF L&D WITHIN FINANCIAL SERVICES ...... 48 5.1.1 The need for training emerges ...... 49 5.1.2 Creating Content ...... 52 5.1.3 Distributing Content ...... 53 5.1.4 Documentation and Reporting ...... 54 5.1.5 Holistic As-is Perspective of Corporate Learning ...... 55 5.3 BARRIERS OF ADOPTING AUTOMATION AND AI WITHIN L&D ...... 56 5.3.1 Technical Barriers ...... 56 5.3.2 Organizational and Regulatory Barriers ...... 59 5.4 FUTURE STATE OF AI AND AUTOMATION FOR L&D WITHIN FINANCIAL SERVICES ...... 62 5.4.1 Empowering Decision-makers with Learning Analytics ...... 62 5.4.2 Elevating Tedious Tasks from Administrators and Human Resources ...... 63 5.4.3 Optimizing the Learning Path for Each Employee ...... 64 6. DISCUSSION AND CONCLUSION ...... 65
6.1 CRITIQUE ...... 65 6.2 MAIN FINDINGS ...... 65 6.3 CONTRIBUTION ...... 67 6.4 SUSTAINABILITY ...... 67 6.5 AI, AUTOMATION AND ETHICS ...... 69 6.6 FUTURE RESEARCH ...... 70 APPENDIX A: INTERVIEW TRANSCRIPT ...... 71 APPENDIX B: SURVEY TRANSCRIPT ...... 73 REFERENCES ...... 74
v List of Figures
List of Figures
Figure 1 The biggest obstacles as well as activities perceived as most helpful in terms of job-related learning (Linkedin, 2019)
Figure 2 Relationship between adaptive and intelligent educational systems Brusilovsky and Peylo (2003)
Figure 3 AIWBES technologies and their origin (Brusilovsky and Peylo, 2003)
Figure 4 An adaptive learning structure (Cleave, 2020)
Figure 5 A segmentation of four respective areas of data analytics
Figure 6 Conceptual framework
Figure 7 The forgetting curve as popularized by Ebbinghaus (Cloke, 2018)
Figure 8 Structure of the research process in accordance with Collis and Hussey (2009)
Figure 9 An exponential non-discriminative snowball sampling method (Dudovskiy, 2017
Figure 10 Overview of coding process
Figure 11 A high-level perspective of the L&D process within the Financial Services industry
Figure 12 Holistic and generalized perspective of how corporate learning is conducted at the case companies
Figure 13 Share of companies using in-house-built software within L&D
Figure 14 Time for deployment of most recent external software solution
vi List of Tables
List of Tables
Table 1 Some prominent fields of AI
Table 2 Overview of case companies
Table 3 Summary of interviews
Table 4 Survey respondents
vii Foreword
Foreword
This report was written during the spring semester of 2020 as a master thesis project for a master’s degree in Industrial Management at the Royal Institute of Technology (KTH) in Stockholm, Sweden.
Acknowledgement
Firstly, I would like to express my most sincere gratitude to the commissioning company, Sana Labs, for providing me with the opportunity to conduct my master thesis. More specifically, I would like to acknowledge the contributions made by Joel Hellermark and Samuel Björklund which provided exceptional supervision throughout the research process. As time was a significantly limiting factor, you never failed to engage in thought- provoking discussions and ensure that my thesis was headed in the right direction, of which I am immensely grateful for.
Secondly, I would like to thank my supervisor at the Royal Institute of Technology, Associate Professor Jannis Angelis, Docent in Operations Strategy. I always managed to leave our meetings with new perspectives, theories, and methods which proved to have a highly valuable contribution in terms to the outcome of my thesis. I would also like to express my gratitude to my seminar advisor and examiner Matti Kaulio for his academically insightful seminars.
Thirdly, I would like to thank the interview participants for taking time out of their immensely busy schedules to engage in intense discussions about AI, automation, and corporate learning. The research could never have been conducted without your valuable contribution and knowledge.
Finally, I would like to express my gratitude to my friends and family for supporting my five years spent pursuing a degree at KTH.
Petter Klinga
Stockholm, June 2020
viii 1. Introduction
1. Introduction
In the following chapter, the background of the study is presented in order to familiarize the reader with the area of study and its importance. A problematization is then formulated which subsequently describes the purpose, research question as well as delimiations of the study.
1.1 Background As the emergence of new technologies are continuously disrupting the way in which organizations function and develop, the majority of initiatives within Learning and Development (L&D) are far from fully effective according to a survey conducted by McKinsey (2019). Furthermore, as we are moving towards a lifetime of learning, solutions within Artificial Intelligence and Automation unlock significant opportunities for companies to improve the learning process of their staff.
1.1.1 Challenges and Opportunities in Learning and Development As a noticeable percentage of market capitalization is based on intangible assets (McKinsey, 2016), having exceptional leadership, skilled employees and transferable knowledge all have a causal and measurable effect on business outcomes. As a basic definition, Corporate Learning is defined as a system of development activities designed to educate employees, in which the majority of these activities are performed within the organization in a non-extracurricular manner. Whereas the intent of Corporate Learning is to serve as a function that enables people to develop and maintain their skills in order to subsequently allow the organization to perform exceedingly better, it has never before faced more scrutiny. As executives are consistently demanding a strong Return on Investment, Corporate Learning is constantly being pressured to prove its value and contribution to the development of an organization. However, justifying the value of Corporate Learning is an immensely challenging task.
According to a study conducted by McKinsey (2019), 70% of employees report that they do not have the mastery of the skills needed to do their jobs. However, simply giving these employees additional training would not necessarily solve the explicit problem considering that only 12% of employees actually apply their new skills learned during an L&D program and that 25% believed that training measurably improves performance.
1 1. Introduction
Thus, the overall lack of skill that is present amongst many organizations is not necessarily correlated with a lack of actual training, but rather how these L&D initiatives are conducted. Additionally, this frustration is equally present amongst managers, in which the same study concluded that 75% of managers surveyed across 50 organizations were dissatisfied with their L&D function. However, despite the consensus in terms of the inefficiency of contemporary L&D functions, it is a tremendously growing market. A study conducted by LinkedIn (2019) revealed that 87% of companies are growing their expenditures within L&D, resulting in a market valued over USD $240 billion in 2019, which constitutes an increase of 7% in relation to the previous year. Moreover, the market is expected to grow additionally in the 7-9% Computed Annual Growth Rate (CAGR) range until 2022. Nevertheless, a significant contributing factor towards this growth is the shift towards digital, whereas 59% of companies have increased their expenditure on digitized learning as opposed to a 24% increase on instructor-led education, which subsequently opens up for additional markets for companies to address with their product offerings and opportunities to apply new technologies.
Conclusively, we are living in the most exciting age of learning but also the most confusing. On one hand, we are seeing workplace learning becoming an essential strategic component of an organization whilst we are rapidly moving towards a lifetime of learning and a workforce that is continuously required to update and maintain their skills (Dineen, 2019). However, embarking on this journey towards success by choosing and utilizing the appropriate technology is immensely confusing as L&D, along with many functions in an organization, is in the midst of digital disruption.
1.1.3 Technology-Driven Personalization In line with the immense increase of Software-as-a-Service (SaaS) offerings that have emerged in accordance with the global digitalization as well as the shift towards a knowledge-based economy, personalization is creating significant opportunities for additional technological intervention in learning experiences across several organizations and industries (Accenture, 2018). As seen in companies such as Netflix, who are able to create new popular series based on viewing patterns (Hunt, 2015), or Amazon, who can acclaim 35% of their sales from suggesting additional products for consumers based on their purchasing behavior (Mangalindan, 2012), the possibilities of personalization to create a strong business value are endless.
2 1. Introduction
The common denominator amongst such applications within these companies is the usage of recommendation systems, constructed by algorithms that are able to identify specific attributes of a user in order to make a subsequent suggestion, which, in line with a previously mentioned example, might be an additional product or service to purchase. The more people that interact with the recommendation algorithm, the more data is collected, which subsequently allows it to create more granular and customized recommendations. This process of algorithms improving from receiving increasing quantities of data, or being ‘’trained’’, is referred to as machine learning (ML) (Marr, 2016).
1.1.4 A New Era of Learning In more recent years, a new branch, which applies highly-advanced technologies within the education space has emerged rapidly, namely educational technology (EdTech). Moreover, the most prominent area in regard to the intersection of AI and learning is personalization, which, in the context of this study, refers to the phenomena of individuals receiving a unique learning path based on their interaction data in order to maximize their learning outcomes. This form of personalized learning experiences through AI is referred to as adaptive learning, which challenges conventional classroom learning by capturing detailed information about each student’s learning path in order to personalize content based on their progress and previous knowledge (Bughin, et al., 2017). As personalization enables companies to create differentiable and superior product offerings to their consumers, utilizing AI and Automation to improve the internal learning workflows of employees creates additional opportunities for an enhanced L&D function. Across organizations, people are constantly reading, searching, listening, and watching in the flow of work. However, the disconnect between these independent learning activities and enterprise L&D efforts means limited visibility into which skills employees have and which ones they are working to build. Without this detailed data on learning behaviors and objectives, leaders can only provide standardized — and largely ineffective — corporate L&D strategies, as opposed to an adaptive learning experience. Moreover, when looking at the biggest obstacles that employees face in terms of job-related learning and professional development as well as the perceived most helpful activities for making their learning more relevant to their job or career goals (Harvard Business Review, 2019), a significant amount can be derived from a lack of personalization in the learning experiences, as summarized below.
3 1. Introduction
Figure 1, the biggest obstacles as well as activities perceived as most helpful in terms of job-related learning (Linkedin, 2019).
As shown above, the challenges and nonetheless opportunities for improvement expressed by the contemporary workforce in terms of job-related learning functions are forcing organizations to look beyond their traditional approach of educating their staff in order to maintain the level of competitiveness derived from having a skilled workforce. As mentioned, a significant amount of the expressed challenges and opportunities can be tackled by enhancing the main objective behind applying AI in a learning context, namely enhancing personalization, all of which have been highlighted as a blue bar in Figure 1 above. In terms of learning, AI has the capability to not only provide learners with a highly-customized learning path based on both the content itself which the learner interacts with but also based on existing skill-gaps of the learner. Thus, a learner could be provided with a learning experience that is continuously adapting itself to the learner in order to maximize engagement, save a significant amount of time and subsequently achieve the most optimal learning outcome. Furthermore, regardless of what metric an organization uses to measure successful learning outcomes, having a highly-skilled workforce has an evidently significant business impact.
1.1.2 The Financial Services Industry As established, having a well-skilled workforce helps companies achieve a competitive advantage in that it, amongst many other things, facilitates the reduction of general errors made by an employee which subsequently reduces the operating costs. However, for some industries, continuously educating their workforce does not only serve as an additional source of competitiveness but also allows them to be kept compliant with specific regulations imposed on the industry. Whereas in many industries, regulations are immensely volatile and require employees to undergo extensive training in order to both stay qualified for their job and compliant. One industry that is unmistakably characterized by such volatility is the financial services industry.
4 1. Introduction
Within financial services, employees are exposed to a significantly complex environment that requires them to acquire hard skills. In addition, employees are continuously exposed to a noticeable amount of change, such as the introduction of a new framework for software development, or are required to periodically go through training which they already know. Decades of scandals and financial collapse have resulted in the emergence of a new, more stringent, regulation within financial services (Farrell, 2019). As a result of the significant increase in regulation, financial services companies are more or less forced to restructure their learning initiatives in order to be compliant.
1.2 The Commissioning Company The commissioning company of this thesis project is Sana Labs AB, henceforth referred to as Sana, which assigned the author to investigate how Automation and AI can be utilized to improve corporate learning workflows of companies that operate in highly-regulated industries. Sana is an artificial intelligence company that applies machine learning to personalize educational content for students. The company was founded in 2016 by Joel Hellermark, who developed a series of algorithms that could be used to optimize the learning path of each student, regardless of the actual learning content. As of today, Sana predominantly engages in partnerships with publishers of educational content, such as Pearson and Nationalencyklopedin, to provide personalized learning experiences for students. However, with the emerging importance of corporate learning, the technologies developed by Sana could similarly be applied in an organizational context which subsequently opens up for additional business opportunities of which this thesis aims to evaluate.
1.3 Problem Statement
As Corporate Learning serves as an essential aspect in terms of sustainable success for organizations, many fail to conduct such activities in an efficient manner. As a result, the L&D function of many organizations has received extensive backlash whilst being pressured to withhold a strong return on investment. Nevertheless, industries which require considerable training are naturally suffering the most, in which the financial services industry serves as a textbook example considering their obligation to comply with a wide array of external regulations. Furthermore, as the emergence of intelligent technologies allows for internal training activities to be optimized, companies are now faced with the opportunity to apply these in order to gain and sustain a competitive advantage by having a highly-skilled workforce.
5 1. Introduction
However, the adoption of these technologies to allow for the L&D organization of financial services companies to be re-designed face significant barriers, both specific to the organization as well as industry-wide.
1.4 Purpose
The purpose of this research was to conduct an exploratory case study of the financial services industry and evaluate how automation and AI could be utilized to improve their Learning and Development organization. Additionally, the study emphasized and investigate the barriers financial services companies are required to overcome in order to successfully adopt automation and AI technologies within a Learning and Development context.
1.5 Research Question The following has been established as the main research question for the study:
How can automation and AI improve Corporate Learning for financial services companies?
In order to answer the main research questions, the following sub-research questions were formulated:
• How is Corporate Learning currently conducted within the financial services industry? • What are the biggest barriers that financial services companies are facing in terms of adopting automation and AI technologies within their L&D organization?
1.6 Delimitations
Firstly, this study will be delimited to study the Swedish financial services industry. More specifically, the L&D organization of the financial services industry will be studied. However, given the fact that many companies use different terms to dename their L&D organization or it might very well simply be integrated into their HR-department, the study will simply emphasize the functions of financial services companies that develop and deliver training to their employees.
6 1. Introduction
One immensely important factor that needs to be taken into consideration in order to fully understand the research and its subsequent goals is that no explicit technology implementation took place. This entails that, upon discussing and analyzing the implementation opportunities and benefits of automation and AI within L&D, solely hypothetical approaches will be incorporated.
An additional factor worth mentioning is that the thesis was conducted in the midst of the global COVID-19 pandemic. The immense impact of the virus resulted in a significant amount of companies being forced to lay off large portions of their workforce, with the financial services industry being no exception. Naturally this resulted in the HR- department being immensely pressured and potentially diverging from prioritizing learning innovation in their strategic agenda. Consequently, this might have affected the outcome of the research in the sense that the virus might had discouraged the research participants to explore the opportunities of technological advancements within L&D.
1.7 Expected Contribution This study expects to contribute to the field of Operations Management by investigating how AI could be used to optimize learning workflows and subsequently have measurable and nevertheless valuable outcomes for an organization. Operations Management as a field partially emphasizes organizing the administration of different business practices to create the highest level of efficiency possible within an organization. Moreover, this view can be aligned with this specific research study due to its scope, which emphasized the evaluation of existing L&D organizations of financial services companies and subsequently how they could utilize automation and AI to optimize the creation and execution of training programs.
7 2. Literature Review
2. Literature Review
As very few studies exist specifically for the usage of AI and automation to optimize corporate learning within financial services companies, a literature review was conducted in order to gain a broader understanding of research conducted within corporate learning as well as the application of AI and Robotic Process Automation across different industries and functions of an organization.
2.1 Corporate Learning
As the result of an organization’s capacity to treat knowledge having evolved to one of its most significant factors towards sustainable success, the need for a system to manage knowledge and learning within an organization has emerged rapidly. However, the increasing need to manage knowledge within an organization in order for people to effortlessly access and share information has enabled the possibility to implement a wide array of solutions for effective knowledge management. What has previously been tacit knowledge held by a very specific individual can be transformed into available information for others. However, some scholars agree that this might not be enough for the creation of knowledge. Organizations may encounter knowledge or skill gaps between what they need in order to successfully engage with unfamiliar challenges and the current competencies of the employees can be made available. Furthermore, as the learning processes of an organization serves as a tool to create corporate knowledge and, consequently, a learning organization, corporate learning has evolved into an essential component for any organizations seeking sustainable success (Crocetti, 2012). Additionally, in accordance with Crocetti, 2012 acquiring new human resources can be an immensely costly procedure and, at times, be stumped by the lack of desired competencies on the labor market. Therefore, training has become a vital organizational function in order to support the demand of organizations to constantly upgrade the skills of their workers.
Furthermore, as concluded by a wide array of researchers, the success of any organization can be associated with the quality of the workforce (Christensen & Overdorf, 2000; Fiol & Lyes, 2005). However, in order to maintain this level of quality, many organizations are faced with a significant number of obstacles. As providing the workforce of an organization with training has been proven to be a strong contributor in terms of enhancing its ability to achieve the organizational objectives.
8 2. Literature Review
Moreover, such obstacles include both attracting and recruiting intelligent and enthusiastic people to the organizations but nevertheless being able to sufficiently motivate According to the European Foundation for The Improvement of Living Conditions (2000), quality of work-life serves as a multidimensional construct made up of multiple interrelated factors. These factors include job security, job satisfaction, productivity, health, safety, competence development as well as professional skill development, which subsequently highlights the contemporary state of corporate learning being an essential component of modern organizations.
2.1.2 Learning Management Systems LMSs enable the creation, management and subsequent delivery of training in an easy-to- administer narrative across organizations and has therefore established itself as a vital software platform within the L&D function of an organization. As described by Panjaka (2015), LMSs have a well-defined method for planning, implementing as well as assess training, all of which are delivered to the employee online in order for them to gain the knowledge and skills needed to perform their tasks. In order to implement and adopt an LMS within an organization, a significant amount of planning as well as preparation, as indicated by Mahoney et al. (2014). Moreover, the adoption of an online training platform such as an LMS within an organization has an evidently positive impact on both the employees that are using it as well as the organization in its entirety (Mahoney et al, 2014).
2.1.3 Integrating Training With Company Processes Training does not simply belong in the HR-department of an organization and must be a quintessential aspect of all company-related activities. As suggested by Crocetti (2012), there are several company processes in which training is a key element, whereas some of these are:
Management: On a daily basis, managers typically deal with investors, partnerships, financial information, strategic priorities, leadership as well as budget definition. In a continuously changing environment, managers are required to keep up with the intensity of the market. One of the most prevalent issues in terms of coping with a fluctuating market is the reluctance of management to progress from the way they are used to conducting their business, to newer and more effective strategies. Thus, managers are in need of life-long training, both in terms of their specific area of expertise, such as finance, accounting or leadership, as well as on external organizational routines and procedures.
9 2. Literature Review
Research and Development (R&D): Regardless of the industry, every organization needs to have a well-skilled and up-to-date R&D department in order to successfully transform a product from an idea to its production. In many cases, an R&D department finds themselves searching for the right resources for a specific project, i.e., identifying the knowledge from within an internal pool of knowledge.
Sales: The sales department of an organization is in constant need of receiving training for both new products as well as new techniques for selling. Additionally, sales departments often experience quite heavy turnovers in which people resign and new salespeople are recruited regularly, thus requiring significant training.
Marketing and Communications: As the contemporary marketplace is constantly changing, thus creating both significant opportunities and nevertheless challenges which can be addressed through training. When one company is acquired, or two merge, brand names need to be modified accordingly, and a shared corporate identity needs to be spread throughout the recently formed entity. Thus, training is a part of this communication process in which online courses can be deployed in order to spread the new corporate identity in order to allow employees to align themselves with company goals.
2.1.4 Organizational Learning As organizations are constantly being pressured to produce enhanced value by combining innovation, quality, and customization in a scalable manner, the new sources of value cannot be achieved by doing more of the same (Christensen & Overdorf, 2000). In fact, depleted operating models and patterns of thinking are in drastic need of replacement with novel ones. In order to complete this transition, organizations must stimulate new ways of thinking in accordance with Christensen & Overdorf (2000), whereas an organization’s capacity to learn has been established as a fundamental strategic capability (Fiol & Lyes, 2005) as well as a significant source of competitive advantage (Stata, 1989). While some academics argue that the concept of organizational learning is simply defined as the sum of what individuals learn within an organization (Kim, 1993; Simon 1991), other research suggest that serves as a reflection of the collective ideas, systems, activities, and structures of an organization (Levitt & March, 1988; March, 1991). Moreover, considering that all learning occurs amongst and through people, much attention within research has emphasized the exploration of how learning is facilitated amongst individuals and within groups, as well as how such learning is subsequently transformed and embedded into specific organizational goals and systematic processes.
10 2. Literature Review
2.2 Artificial Intelligence During the 21st century, Artificial Intelligence (AI) has evolved into an essential area of research in virtually all fields, with exceptional progress being made within engineering, finance, medicine and, nevertheless, education (Halal, 2003). As described by Lacity and Wilcocks (2018), AI serves as an umbrella term for making computers act and think similarly to humans, whereas one of the most differentiable qualities that humans possess is the ability to make rational decisions based on previous experience. Historically, computers have been significantly quicker in terms of making complex calculations as opposed to humans. Moreover, one significant obstacle for computers is the lack of intelligence, preventing them from approaching problems without a set of predetermined rules to define their subsequent behavior. However, the most recent years have demonstrated great advancements within the area of AI, making computers increasingly suitable for executing tasks that require human interference. A significantly contributing factor towards these advancements is the development of Neural Networks (NN), which attempts to resemble a computational replica of the human brain in which large quantities of information is processed through
The development of AI is still very much ongoing, especially in regard to its application across societal and organizational contexts. As described by Burgees (2018):
‘’AI is in the arena of the conscious unknown’’
Which refers to the fact that there we are aware that we do not know enough about AI. However, as the application of AI is still in its infancy, its implications have had a significant impact across multiple industries, such as manufacturing as well as retail and e-commerce (Burgees, 2018).
2.2.2 Key components of Artificial Intelligence As mentioned, AI serves as an umbrella term for a wide array of different concepts and techniques that are applicable. The following table encapsulates and summarizes some of the most prominent fields of AI and their areas of application as described by a wide array of researchers and industry experts.
11 2. Literature Review
Table 1, some prominent fields of AI
Concept Description Example of Application
Machine Learning A subset of AI which utilizes statistical Algorithms powering search (ML) models to enable computers to learn from engines, such as Google, to historical data rank web pages after a search
Supervised An area of ML where the machine is Algorithms classifying Learning provided with an input and output with whether an email is a spam or the intent of identifying patterns to not connect these. As the classification of data is known, the objective is to identify a function that corresponds to the correct classification (Hudson & Cohen, 2012)
Unsupervised As opposed to supervised learning, Clustering algorithms that (Deep) Learning unsupervised, or deep, learning refers to group data points into specific when a machine is solely provided with categories input data during its training. The classification of data is unknown as well as how many classes that should be considered by the machine (Hudson & Cohen, 2012)
Natural Language An umbrella term describing the Google Translate Processing (NLP) interactive process between human language and computers.
Neural Networks A subset of ML, which draws inspiration Grammar spell-checking (NN) from the neural networks of the human brain and consists several layers of nodes that form a uniform model that can be trained
Natural Language Not to be confused with its umbrella term Apple’s Siri function Understanding NLP, NLU refers to the technique for a (NLU) machine to transform unstructured data (human language) into structured data.
12 2. Literature Review
2.2.3 Adaptive and Intelligent Web-based Educational Systems According to Brusilovsky and Peylo (2003), adaptive and intelligent Web-based educational systems (AIWBES) provide an alternative solution as opposed to the traditional ‘’just-put-it-on-the-Web’’ approach in the development of educational software. Put briefly, AIRBUS attempts to be more adaptive in the sense that it builds a model of the goals, knowledge and preferences of each individual learner by utilizing this model across the interaction with the learner to adapt to their unique needs. They also strive to be more intelligent by incorporating aspects traditionally conducted by a human teacher - such as providing coaching or diagnosing and remediating their misconceptions.
Brusilovsky and Peylo (2003) continue by distinguishing between adaptive Web-based educational systems and intelligent Web-based educational systems and how they are not in fact synonymous with each other. Speaking about adaptive systems Brusilovsky and Peylo (2003) stress that these systems strive to be different for different learners by taking into account information that has been gathered. As for intelligent systems, these systems apply techniques from the field of AI to provide more extensive and better support for the learners within a Web-based educational system. While the two categories share many similarities and several existing Web-based educational systems attempt to incorporate both, they often fail to do so according to Brusilovsky and Peylo (2003). For instance, many intelligent diagnosis systems are non-adaptive in the sense that they will provide the same diagnosis as a response to the identical solution to a problem without taking into account the learner’s past experience using the system.
Figure 2, relationship between adaptive and intelligent educational systems Brusilovsky and Peylo (2003)
13 2. Literature Review
Moreover, taking into account the two fields of origin established by Brusilovsky and Peylo (2003), the two authors further describe five resulting groups of technologies.
Figure 3, AIWBES technologies and their origin (Brusilovsky and Peylo, 2003)
The figure above describes Adaptive Hypermedia, which in accordance with the previous research by Brusilovsky (1999), differentiates linear media in the sense that the user receives a tailored experience based on what their goal is. As described by Brusilovsky and Peylo (2003), Adaptive Presentation entails adapting the content presented to a learner in each hypermedia node, or simply put; learner page, to their learning goals, existing knowledge as well as other relevant learner information. In a system utilizing Adaptive Presentation, the pages surfaced to a learner are not simply static but adaptively generated for each user. Moreover, the goal Adaptive Navigation Support is to assist the learner in terms orientation and navigation by changing the appearance of visual features of the learning platform. This might, for instance, entail adaptively sorting or partly hiding certain theory items or questions available to the learner.
On the other hand, the figure similarly explains the phenomena of Intelligent System and its enabling of Intelligent Tutoring. Furthermore, Curriculum Sequencing entails providing the student with the most appropriate individually-planned sequence of topics to learn by finding their optimal path. In a learning platform, Curriculum Sequencing can simply be implemented in the form of a recommended link or an adaptive ‘’next button’’. Intelligent Solution Analysis, on the other hand, deals with a learner’s solutions of educational problems. Such intelligent analyzers can identify what is wrong or incomplete and which incorrect or missing pieces of knowledge might be the responsible factor for an error, in order to provide a learner with extensive error feedback during their learning process.
14 2. Literature Review
Lastly, Interactive Problem Solving Support enables learners to be provided with intelligent help during each step of problem solving, from simply giving learners a small hint to executing the explicit next step for the learner.
2.3 Robotic Process Automation
As a basic definition, Robotic Process Automation is an application of technology, governed by a structured set of inputs, which aims at automating a wide array of processes usually conducted by a human. Using RPA tools, companies can configure software, often referred to as a bot, in order to both capture and interpret applications for communicating with other digital systems, processing a transaction or even manipulating data. Moreover, what distinguishes RPA technologies, as opposed to any other process automation, is the fact that the information systems that underlie the technology remain unchanged (Aalst et al., 2018.). Thus, reinstating a human to execute the automated task would not be a troublesome process considering that both the user interface as well as central information systems have remained intact.
2.3.1 Capabilities and Areas of Application As suggested by Schatsky et al. (2016), RPA enables a wide array of companies to reap the many benefits of RPA, including reducing staffing costs, lowering rates of error, scaling operations more efficiently, as well as improving compliance. Put briefly, bots enabled by RPA can be seen as a virtual workforce that interacts with applications precisely as humans would. As described by Fung (2014), there are certain criteria in terms of which tasks that are suitable for a bot to conduct, whereas some of these are:
● High volume of transactions: Voluminous transactions are generally routine and repetitive, thus making automation is an ideal choice in order to minimize manual work ● High value of transactions: Normally, low transaction volume is challenging to justify for an investment from a business case perspective. However, ● Frequent access to multiple systems: Processes that require employees to frequently access several systems in order to complete their job ● Manual IT-processes prone to error or re-work: Tasks in which humans tend to make a significant amount of error or, potentially as a result of such errors, requires re-work to be conducted ● Low Cognitive Requirements: Processes that require little subjective judgment or interpretation skills
15 2. Literature Review
Moreover, the usage of RPA spans both across several industries as well as areas within the organization. Similarly, Fung (2014) describes multiple use cases for RPA, whereas some of these are database and middleware automation which incorporates bots that automate some administrative activities pertaining to database and middleware provisioning, compliance verification, patching, as well as release administration.
2.3.2 RPA in L&D As described by Blankenship (2018), maintaining the L&D initiatives of an organization in an effective manner requires overseeing several course schemas, program requirements, and general activities for the employee. However, in L&D, similar to many other functions within an organization, RPA-enabled bots have been found to be capable of handling a significant amount of day-to-day tasks that consequently slows administrators down. As a result, RPA allows administrators and their L&D teams to deliver more value to the organization with significantly less manual intervention and effort.
Moreover, considering that the size and nonetheless scope of an organization’s L&D initiatives determines the job duties of training administrators respectively, the role looks different across different industries and companies. However, one common denominator is that many training administrators are struggling to harness their daily efforts in order to achieve the greatest organizational impact. Several scholars agree that training administrators should, ideally, spend the bulk of their time helping L&D teams in terms of optimizing different strategies, as opposed to continuously managing busywork (Blankenship, 2018; Lhuer, 2016).
Similar to its application in any other organizational function, RPA can mimic repeatable human processes and thus allow companies to automate rule-based activities in the learning organization. According to Blankenship (2018), even with fairly basic Learning Management Systems (LMSs) and processes, bots can do a significant amount of step-by- step facilitation that training administrators do on a daily basis, such as request processing or course scheduling. Conclusively, utilizing RPA to shift the administrative efforts and talent to more impact-driving projects as opposed to repetitive tasks, organizations will gain an improved understanding in terms of which learning activities best support the business. As concluded by several scholars, this will trigger the emergence of a new era within L&D, characterized by agility and intelligence, in which each training activity is personalized and beneficial for each employee (Lhuer, 2016; Berruti et al., 2017).
16 2. Literature Review
2.3.2 Limitations of RPA As described by Torres (2020), automating processes within an organization must be aligned with the overall business goals. However, organizations should similarly beware of optimizing processes that are inefficient. According to Hill (2019), deploying RPA successfully starts by identifying the bottlenecks, then infusing the technology into the processes of the organization. Hill follows by confirming that, in order to extract the most value from RPA, organizations must prioritize spending the time and resources to identify which processes should and will be automated, whilst stating:
‘’Automating something that does not work well just makes it not work well faster’’
Thus, as opposed to the AI which can be applied to process that does something incorrect in order to potentially fix it, RPA might simply mimic a process which from the beginning was an inefficient process.
2.4 Unifying RPA and AI In accordance with Schatsky et al. (2016) the rapidly growing market for RPA is already pointing in the direction of an important emerging trend in which companies are starting to embrace RPA integrated with additional technologies such as speech recognition, natural language processing, and machine learning in order to automate perceptual and judgment-based tasks priorly reserved for humans. Moreover, the integration of cognitive technologies and RPA is enabling automation technologies to be extended into new areas and assist companies in becoming more efficient and agile as they progress in their journey towards becoming fully digital.
2.4.1 Tackling the challenges posed by RPA As described by Mohanty and Vyas (2018), RPA-enabled bots will do exactly what you tell them, which serves as both their greatest strength as well as their greatest weakness. Whereas one can be assured that the bot will execute a well-defined piece of code, several problems arise if the bot encounters situations or patterns of data which are unfamiliar or undefined for the bot.
17 2. Literature Review
Therefore, the rule-based algorithms that underlie RPA serve as a noteworthy limitation in terms of which tasks that can be automated. As a result, an additional area of research has emerged which investigates how RPA can be integrated with other, primarily cognitive technologies, also known as Intelligent Process Automation (IPA).
According to Berruti et al. (2017), IPA combines fundamental process redesign, RPA as well as machine learning, in which the traditional levers of value that are characterized by rule-based process automation are enhanced with additional decision-making capabilities. Moreover, Berruti et al. (2017) describe that IPA, in its full extent, encompasses five core technologies:
1. Robotic Process Automation: As already described in chapter 2.3, RPA entails software automation tools that automate routine tasks by deploying bots to perform tasks such as performing specific calculations, creating documents and reports as well as accessing and managing various IT-systems 2. Smart workflow: A process-management software tool that emphasizes the integration of tasks performed by groups of humans and machines Smart workflows allows users to track and initiate end-to-end processes in real-time whereas the software manages the handoffs between different groups, including bots and humans. 3. Machine learning & Advanced analytics: As already described in chapter 2.2.2, ML and advanced analytics comprise of algorithms that identify patterns in structured data in order to produce predictions and insight 4. Natural-language generation (NLG): Software engines that enable frictionless interaction between humans and a respective technology by following a set of rules that translate observations from data into prose 5. Cognitive agents: Technologies that combine NLG and ML in order to construct a completely virtual workforce, also referred to as an agent, that is capable of communicating, executing tasks, as well as decision-making based an emotion detection from the users
Conclusively, IPA-enabled bots allow for human clicks to be replaced (RPA), text-heavy communication to be interpreted (NLG), rule-based decisions to be made without having to be explicitly programmed (ML), suggestions to be made to the user (cognitive agents) and provide real-time tracking of handoffs between people as well as different systems (smart workflows).
18 2. Literature Review
Moreover, as a full range of benefits can be derived from implementing the complete IPA suite, organizations can in fact unlock noteworthy value quite rapidly through individual components of IPA. As established previously, RPA alone can drive significant productivity improvement across multiple functions in an organization.
2.3 Strategic Implications of Implementing Automation and AI As earlier, extracting value from technologies such as automation an AI, organizations are required to not only prioritize allocating resources into first identifying which process to automate and subsequently deploying the solutions, but also establish and maintain a long-term strategic plan for their future AI and automation initiatives (Burgess, 2018). According to Lacity and Wilcocks (2018), it is not only very easy for organizations to fail in their journey towards automation but also explicitly perceive the journey in its entirety as a failure due to the organization not understanding what success entails within automation and AI.
Within Learning and Development, automation and AI AI has during the most recent years gained a significant amount of momentum and exposure. As described earlier, from a holistic perspective, automation and AI allows the L&D function of an organization to be transformed from two perspectives; providing a personalized learning experience and using advanced analytics to derive insights from vast amounts of organizational data, both which are discussed as follows:
Personalization: As continually emphasized throughout this report, one of the most significant benefits that can be derived from applying AI within L&D is the ability to personalize learning experiences (Cleave, 2020). Given the rapid change of both technology and society in its entirety, L&D functions must position themselves in the forefront in terms of the newest approaches and methodologies within learning. The frequently utilized one-size-fits-all model has proven to be obsolete by a wide array of researchers, in which AI allows insights to be gathered from the vast amount of data generated across an organization in order to facilitate customization to an individualized degree. The generated insights can, amongst many things, develop a granular understanding of learning behaviors, thus leading to enhanced predictive capacities. Using these insights, organizations can create training programs characterized by a higher degree of intelligence, in which the content that is surfaced is adaptive and responsive to the learner’s unique path.
19 2. Literature Review
As described by Cleave (2020), each individual has a different understanding of the worlds, different potentials to achieve goals, and therefore different needs for learning. As mentioned, many organizations often deliver the same training to everyone in accordance with a one-size-fits-all approach, as if they are identical. Moreover, as suggested further by Cleave (2020), this suggests that although employees may learn from the training, it is highly inefficient. Consequently, an emerging field within corporate learning is adaptive learning, in which the training that is delivered to the employees provides them exactly what they need, and nothing else. Contrasting this to contemporary approaches in which each employee receives more or less the same training, adaptive learning systems dynamically adjust to the individual learner’s understanding, skills, or interests.
Digital adaptive learning experiences can be constructed in multiple ways, usually consisting of a series of lessons and a final exam to ensure that the learner has gained the required level of proficiency. Illustrated below is a sequence of activities for adaptive learning as suggested by Cleave (2020):
Figure 4, an adaptive learning structure (Cleave, 2020).
20 2. Literature Review
Put briefly, learners who enter the training program are provided with the opportunity to demonstrate their proficiency and take a ‘’test-out’’ exam for that specific content. By passing, they can instantly take the final exam and thus have to navigate through significantly less material. If the learner is not confident in terms of their proficiency, they can progress through the content before attempting the final exam, and if at any point think they are proficient, can take an exam in order to fast-forward through the remaining material. Ultimately, different people with unique learning paths will eventually all achieve the desired learning outcomes but in a significantly more efficient manner.
From an organizational perspective, adaptive learning can bring a wide array of benefits. Firstly, learners are not forced to sit through training that is not beneficial for them, thus saving both time and money. Through adaptive learning, the learner receives a unique learning path allowing them to reach mastery in a significantly quicker manner, resulting in more time spending on actual work tasks. Secondly, by not having to progress through training that is irrelevant, the learner is more likely to pay attention and be more engaged to the actual content provided, thus fostering greater retention of knowledge. Learners enjoy being in control of their training and empowering them with the opportunity to shape it themselves will likely translate into a better, more joyful training experience. Lastly, adaptive learning does not only make training more effortless for the learner but also surrounding stakeholders within the organization. If a course is conducted in a one- size-fits-all approach, the one responsible for designing the course, also referred to as instructional designer, has to manually ensure that every bit of training is relevant to each individual within the target audience. While there might be disagreement internally within the organization in terms of what the employees should learn, the training program might include redundant content only to satisfy the requirements of a certain stakeholder. However, an adaptive training program would allow for all content to be directed solely to those employees who need it without wasting valuable resources, and simultaneously address every concern held by a stakeholder in terms of content quantity. Conclusively, as a result of AI-enabled personalization, employees will be more to immerse themselves in more engaging learning experiences that foster knowledge retention, which subsequently yield improved business value.
Advanced Analytics: Upon collecting and analyzing the immense amount of data that circulates within an organization, L&D leaders and stakeholders can unravel a wide array of insights to reveal information about, for instance, hidden costs, valuable content as well as learner progress and retention in order serve as a foundation for redesigning training initiatives.
21 2. Literature Review
Generally speaking, learning analytics refers to the collection and analysis of data in regard to a specific learner as well as their surrounding environment which serves the purpose of improving their learning outcomes (Fung, 2014). From a practical perspective, deploying learning analytics solutions brings an array of use cases, in which some of these are:
● Measuring the key indicators of learner performance ● Helping support learners in their professional development ● Provide insights in terms of how specific pieces of content is perceived by learners ● Inform strategic decision-makers in terms of course and curriculum development
From a holistic perspective, learning analytics can be segmented into four respective dominant areas (Fung, 2014), all with the potential to provide insights into different aspects of the L&D organization.
Figure 5, a segmentation of four respective areas of data analytics
Descriptive Analytics: An area of analytics that collects data to give insights about past performance. For instance, in a L&D context, this could entail data insights pointing toward increased dropout rates which subsequently might signify an unengaging learner experience or content with low pedagogical quality.
22 2. Literature Review
Diagnostic Analytics: An area of analytics that emphasizes asking questions related to why something happened. Data from a diagnostic analysis might show that a training program received low rates of completion amongst the management team whilst people undergoing onboarding found it effective. Further diagnostic analysis would potentially suggest that the program was simply too easy for executives thus resulting in a low completion amongst executives, pointing towards the need to create a more challenging course.
Predictive Analytics: As the name suggests, predictive analytics investigates what is likely to happen. Put briefly, by building on the findings of existing data, predictive analytics can make forecasts about the future. Moreover, an example of usage within L&D entails analyzing data from post-training-surveys taken by previous learners which subsequently revealed that learners did not prefer accessing an online training program via their desktop.
Prescriptive Analytics: The purpose of prescriptive analytics is to identify solutions to understand what should be done. Furthermore, in addition to finding solutions to what will happen, prescriptive analytics should additionally be utilized to in order to understand why it will happen in order to, in an L&D context,
As described above, learning analytics offer stakeholders within an organization the opportunity to act upon the immense amounts of accumulated data from conducting training amongst, at times, thousands of employees. Furthermore, it creates a narrative for individuals without extensive technical knowledge to make data-driven decisions which could have a significant impact on how their L&D organization is operated and subsequently how training programs are carried out. Conclusively, while the application of AI within L&D is still at its infancy and many concepts have yet to breach theory, rapid advancements are made yearly. As a result, organizations are faced with the challenge of implementing these technologies in order to design drastically improved training programs which subsequently create more skilled employees and a potential competitive advantage.
23
2.4 Summary of Literature Review Put briefly, researchers agree upon the fact that Automation and AI have respectively revolutionized many aspects of how our society functions and demonstrated significant business value across multiple industries. However, little research exists within the field of how these technologies can be utilized in an organizational L&D context and more importantly which benefits such organizations could expect. Moreover, while information technology has grown rapidly, scholars across different research fields have emphasized the importance of IT with automation and AI technologies, with limited attention being explicitly paid to the barriers inhibiting successful adoption.
24 3. Developing a conceptual framework
3. Developing a conceptual framework
This chapter describes how the wide array of theories were utilized and synthesized in order to draw interferences with the experiences of the commissioning company considering their presence in the EdTech marketplace. The framework was primarily developed due to the the significant lack of previous research incorporating aspects of automation, AI and corporate learning, which thus allowing the conceptual framework to reflect the thinking of the researcher throughout the entire research process. The structure of the conceptual frameworks will firstly emphasize the explicit indications of output upon adopting automation and AI technologies and secondly in which aspect of L&D one can expect such output. Additionally, a framework will be introduced which takes into account which processes are compatible with automation and AI technologies as well as the evaluation of potential barriers for adoption.
3.1 The Rationale Behind Developing and Utilizing a Conceptual Framework In accordance with Camp (2001), a conceptual framework is a structure which the researcher believes can be explain the natural progression of the explicit phenomenon to be studied. Peshkin (1993) follows by describing its linkage with the concepts, empirical research and important theories used in promoting as well as systemizing the knowledge espoused by the research. Put briefly, it is the researcher’s explanation of how the research problem would be explored. Naturally, given that a very limited amount of existing research was available which incorporated all aspects within the scope of this specific research study, the conceptual framework allowed the researcher to construct their worldview of the identified problem. Additionally, this is a view supported by Akintoye (2015), as he describes that a conceptual framework is best utilized by a researcher when existing theories are either not applicable or simply sufficient in creating a firm structure for the given research study. Of course, theoretical frameworks from the respective bodies of research that is automation, AI and corporate learning, could have been respectively utilized in order to guide the research process. However, this would have potentially resulted in a research study characterized by a lack of cohesion and thus fail to contribute to a research field incorporating all three aspects.
25 3. Developing a conceptual framework
As discussed throughout chapters 2.2.3 AI in L&D and 2.3.2 RPA in L&D, the capabilities displayed by the two technologies have the potential of disrupting the contemporary L&D of organizations across an array of industries. However, existing research does not explicitly provide significant aspects to which elements of such technologies could be utilized. Therefore, the following sections will attempt to establish how the capabilities of automation and AI can be aligned within a contemporary L&D context. Moreover, the particular elements of the conceptual framework were not only based on findings in literature but was also influenced by the commissioning company, who are well- experienced in the field of automation, AI and education as described in chapter 1.2 The Commissioning Company. The empirical framework is summarized in the figure below.
Figure 6, conceptual framework
As visualized in the figure above, the conceptual framework emphasizes two respective aspects; processes which are compatible for adopting automation and AI technologies, as well as the respective indications of output of the two technologies.
26 3. Developing a conceptual framework
Moreover, the goal of the framework was to firstly gain an understanding in terms where, within the context of the L&D organization for financial services companies, automation and AI technologies might provide value and, secondly, what financial services companies could expect from such an adoption. Furthermore, the respective elements of the conceptual framework are dissected in the following subchapters.
3.2 AI Indications of Output Personalized Content: In terms of personalizing content to each learner, an array of opportunities exist which utilizes several fields of AI. One of the most prominent uses of AI is Adaptive Assessment, which can be utilized to gain an understanding of a learner’s current knowledge and skill gaps. For instance, when a learner first enters a learning platform, he or she can be met with adaptive assessment in the shape of a placement test which takes into account their answers, response times and an array of contextual information (Sana Labs, 2020). As a result, the platform has been provided with information that enables future personalized training. By understanding each learner’s skill gaps and areas which they have mastered, the platform is able to tailor the training to the needs of each learner and thus create an immensely more efficient training process considering that the learner can, for instance, skip content which they already understand.
Moreover, advanced algorithms powered by AI are able to accurately model how a learner forgets by capturing the memory decay of specific items (Sana Labs, 2020). As illustrated in the forgetting curve below, memory retention decreases over time after it is initially learned.
Figure 7, the forgetting curve as popularized by Ebbinghaus (Cloke, 2018)
27 3. Developing a conceptual framework
The forgetting curve was discovered by the German psychologist Hermann Ebbinghaus and describes the process of how memory decays over time (Cloke, 2018). Moreover, if a learner were to never review first-learned content again, they would forget it rather quickly. However, by repeatedly providing the learner with opportunities to review the content, known as Spaced Repetition, long-term memory will successfully be created. Furthermore, by analyzing generated learner data, AI can produce an array of predictions and essentially model the forgetting curve, which consequently allows the forgetting curve to be tackled by providing a learner with appropriately placed review sessions.
Learning Analytics: In an L&D context, getting a comprehensive view of both learners and the content they interact with as an immensely challenging task (Sana Labs, 2020). Moreover, taking into account that stakeholders within an L&D organization have limited capabilities in terms of extracting, understanding and acting upon generated data as it is usually far beyond their technical capabilities.
3.3 Automation Indications of Output As described in the literature review, RPA software automated repetitive, rule-based processes usually performed by people working at a computer. Moreover, by interaction with an array of applications, similar to what a human would do, software robots can, for instance, open an email attachment, record data, complete forms and essentially conduct most human action. More, specifically, the capabilities of automation within an L&D context is dissected in more detail below.
Automated communication: As sending emails manually may seem as a rather trivial task, it takes up a considerable amount of time for any individual administering a training process. If the administrator or stakeholder is sending emails to an array of employees at different stages in their learning journey, it can be immensely challenging to track and subsequently ensure that they are receiving the right communication when they need it. In order to counteract this dilemma, is to automate the communication channels between a stakeholder and an employee undergoing training. Moreover, in a L&D setting, this could entail general course information, survey requests and even encouragement for course completion, rather than having to manually email an employee to provide such information. Furthermore, this will entail significantly less manual work for any administrative interaction with a training program, subsequently resulting in, in accordance with Blankenship (2018), more time available for taking critical decisions within L&D whilst simultaneously minimizing human errors.
28 3. Developing a conceptual framework
Automated distribution: Understanding which content, a specific employee requires in order to conduct an efficient training program is an immensely challenging task for organizations given the often-large quantities of employees undergoing training. However, as described in 3.2 Personalized Content, AI can drastically help organizations understand which competence a specific employee requires by effectively pinpointing their skill levels and subsequently tailor their training in order to address those. Moreover, in terms of automation, an array of opportunities exist in terms of how the optimized content reaches the intended employee. By understanding the knowledge of each employee and thus their required training, automation can eliminate the tedious process of an administrator having to email this content to them manually.
Automated reporting and documentation: As described in the introductory part of this thesis, the SFSA acts as a significant influencer in terms of how training is conducted within the financial services industry. Creating and distributing the same report is moreover an essential task but similarly contributes to a noticeable amount of redundant manual work and thus waste time for any stakeholder involved in the training. By simply knowing preemptively which data is relevant for the reporting initiative, the creation and distribution of such reports can be extensively automated.
3.3 Automation and AI Compatible L&D Processes
As described throughout this thesis, automation and AI technologies can yield immense benefits when applied to the appropriate processes. However, taking into consideration that none of these technologies can be delivered and implemented in a plug-and-play manner, a significant amount of considerations needs to be taken in terms of which processes are most suitable for automation adoption within L&D. Moreover, the following subchapter will elaborate a framework for understanding which processes within L&D are compatible for adopting automation and AI technologies.
Stakeholder-to-employee interaction: As identified throughout this chapter in terms of task automation, a significant amount of processes which comprise any sort of interaction between the employee and the stakeholders serves as an opportunity for introducing automation within L&D.
29 3. Developing a conceptual framework
Content creation and distribution: As described in the previous sub-chapter, distributing content can be an immensely tedious task with regards to manual labor. In addition, given the often-large size of financial services companies today, all process which include any aspect of distributing content, whether via email or any learning management system, are immensely compatible for automation adoption. Moreover, after having finalized the creation of the content, it needs to successfully reach the employees involved in the training program, which can be an immensely tedious tasks with regards to how many different training programs that might exist and subsequently how many should receive it.
Decision-making processes: Making decisions in relation to corporate learning is an immensely challenging task, especially given the often-large size of financial services companies as well as the difficulty in measuring what is perceived as a good training program. Thus, any process related to decision-making with regards to, for instance, who should undergo a training program or which content it should incorporate, are compatible for the intelligence provided by AI.
Static learning experiences: As described earlier, each individual has different understanding of different topics and are thus required different needs for learning. Therefore, any learning experience which takes no consideration to the previous knowledge of an employee and simply provides the learning in a linear fashion, also referred to as static learning (Sana Labs, 2020), is compatible for AI-powered personalization of content.
30 4. Method
4. Method
This chapter will cover the process of the research and how it was conducted. Firstly, the chosen method will be presented and argued for why the particular research design was developed for this specific research study. Furthermore, the quality of the research will be critically discussed in terms of reliability, validity as well as ethical considerations.
4.1 Research Design In order to cope with the established problematization and subsequently fulfill the purpose of the research study, an inductive approach was applied by conducting a case study at three financial services providers primarily operating in the Nordic region. As advocated by Eisenhardt and Graebner (2007), the inductive theory building approach corresponds rather fittingly upon attempting to understand how and why, but is less sufficient in terms of how often or how many. Similarly, Eisenhardt and Graebner (2007) also describe case studies as an appropriate fit for understanding complex research questions which require extensive qualitative data in order to subsequently enable new understandings of organizational and social processes that quantitative data alone cannot sufficiently reveal. Therefore, considering that the main research object, namely learning workflows is rarely standardized and formulated in a quantitative manner, using a primarily qualitative case study was deemed as most appropriate. Additionally, as advocated by Ridder et al (2014) and Blomkvist and Hallin (2018), case studies as a research design has the possibility to generate understandings from both intense and profound research into the the study of a phenomenon, which evidently leads to an abundance of empirical descriptions as well as theory processing. Naturally, simply conducting three case studies of companies operating in the Nordic financial services industry does not necessarily provide sufficient accuracy to statistically provide a generalization. Instead, the empirical findings and their applicability will rather serve to provide a foundational understanding in terms of future exploration of the topic of adopting automation and AI technologies within corporate learning for financial services companies. Therefore, the study will utilize different approaches and their respective advantages to complement each other and subsequently successfully answer the research question.
31 4. Method
Moreover, the research is exploratory in the sense that it investigates an area with an immensely limited amount of previous research and, in accordance with Collis & Hussey (2012), seeks to study patterns, concepts or a nonetheless and hypothesis in which the findings can provide a foundation for future research initiatives. The research was initiated by a literature review as well as interviews with industry experts at the commissioning company in order to the three main subject areas; AI, Automation and Learning and Development with emphasis on corporate learning.
The research was primarily conducted from the offices of the commissioning company, Sana Labs, in close collaboration with the CEO as well as a Business Development Manager leading the entire corporate learning initiative. The primary reason for this was to both be able to immerse in their industry expertise within corporate learning but also to ensure that the thesis was moving towards the right direction, given its complexity as well as lack of previous research within similar areas. However, the interviews at the respective case companies were naturally conducted at their offices in order to ensure an environment where each participant could feel completely comfortable.
4.2 Research Process As described previously, the empirical aspect of this thesis was heavily reliant on qualitative research conducted at three selected case companies, all operating within the financial services industry. However, considering that the commissioning company did not provide any clear directions which companies would be suitable for conducting a case study, a significant amount of effort was allocated to establishing a rigorous empirical process in order to fully capture the research topics, as described in the following sub- chapters. The final scope and objective of the thesis was the result of an iterative process between the student researcher as well as the commissioning company in order to sufficiently satisfy both the academic requirements whilst simultaneously ensuring that the subsequent outcome of the research is aligned with the expectations of the commissioning company.
As described by Collis and Hussey (2012), a case study consists of five steps, namely: 1) selecting the case, 2) conducting preliminary investigation, 3) collecting data, 4) analyzing data and, lastly, 5) writing the report. As follows, each step will be discussed in more detail in terms of how they were executed throughout the research process. This structure can similarly be visualized as a sequential process in the figure below and explained in the following sections:
32 4. Method
Figure 8, structure of the research process in accordance with Collis and Hussey (2012)
4.2.1 Selecting the case This initial step concerns choosing which case is deemed as most appropriate for the specific research, providing the opportunity to identify and capture an issue that the researcher finds interesting (Collis and Hussey, 2012).
Considering that the topic of the research predominantly originated from the commissioning company, the selection of case companies was not predetermined apart from being a financial services company. However, simply being a financial services company did not automatically serve as a qualifier in terms of being suitable for conducting qualitative research. Considering the rather specific scope of the research, all case companies had to undergo an initial qualifying test prior to being approved as a suitable subject of research, which was established during the initial contact with a company representative in order to ensure a transparent and efficient research process. If, for instance, one would have proceeded to conduct a qualitative study at a company which evidently turned out to have strictly confidential training information, the company would be unable to provide any adequate value towards the explicit research objective. Furthermore, the companies were evaluated based on the following criteria and deemed suitable if, and only if, all were met:
● Financial Services Company: As described previously, the purpose of the research was to investigate the financial services industry, and in order to firstly determine if a potential case company operated in that industry, the definition established by the Finance and Development department of the International Monetary Fund (IMF) (2019) was utilized. In accordance with the IMF, a financial services company offers professional services involving the investment, lending, and management of money and assets.
33 4. Method
● Learning and Development: All case companies must have a department within the organization that manages its entire L&D as well as their respective activities and initiatives across the organization.
● Information Accessibility: Considering that the purpose of the research study was to analyze the learning workflows within the financial services industry, information restrictions in terms of what these workflows explicitly consisted of would be considered a significantly limiting aspect and consequently be unable to provide any value towards the research. Therefore, the case companies’ ability to share information was deemed as valuable for the purpose of the research was essential.
Consequently, three case companies that operate within the financial services industry were deemed as appropriate for the specific purpose of this research, which will be described in chapter 4.2.3 Collecting Data.
4.2.2 Conducting a preliminary investigation In accordance with Collis and Hussey (2012), a preliminary investigation concerns the process of when the researchers familiarize themselves of the context in which the research study will be conducted. Prior to taking any explicit decision in terms of the direction of the thesis, a preliminary investigation was conducted in order to firstly gain an understanding of the commissioning company, their reasoning behind the presented research topic as well as the expected outcomes. Thus, unstructured interviews were conducted with the main stakeholders of the commissioning company, namely the CEO as well as a Business Development Manager, both heavily involved at the corporate learning initiative at Sana Labs.
Moreover, a literature review was conducted in order to gather and synthesize various concepts, methods and identified and described by a wide array of research both within L&D as well as AI and automation. However, considering the limited amount of research that emphasized the explicit intersection of AI, Automation and L&D, the literature review predominantly consisted of analyzing the field separately in order to identify similarities and draw inferences accordingly.
34 4. Method
When searching for literature, the Royal Institute of Technology KTH’s library service Primo as well as Google scholar both served as the main platform. Initially, the search was limited towards well-cited and infamous journals, in which the goal was to identify studies with noticeable academic influence in order to ensure the high level of research quality for which the study aimed to obtain. Relevant keywords included in the search were, for instance Artificial Intelligence; Adaptive Learning; Personalized Learning; Robotic Process Automation; Learning Automation; Training Automation; Automated Training Cycles. However, upon quickly realizing that the available literature was rather limited, the scope was significantly broader by additionally taking into account industry reports. The available literature in academia proved to be helpful in terms discussing automation and AI as well as L&D separately, and thus left the researcher to identify appropriate inferences between the two bodies of knowledge. However, in regard to investigating the intersection of the two, industry reports as well as internal documents from the commissioning company served as most useful. Conclusively, the literature study significantly benefited the overall research process by, firstly, enabling the creation of an insightful and contributory research question based on the existing body of knowledge, which was rather limited. Secondly, it allowed for a deeper foundation of knowledge in regard to automation, AI and, in particular, L&D considering the fact that it is a rather rare subject to encounter in detail within a master’s degree in engineering. Lastly, and perhaps most importantly, the literature review enabled the research to construct an approach to explicitly answering the research question by primarily revealing the capabilities of automation and AI as well as the process that goes into creating an efficient L&D process within an organization.
Moreover, a significant portion of the initial interviews which were conducted at the selected case companies were similarly executed with the intention of understanding their current L&D organization and evaluating whether the research study would be feasible to conduct considering the previously mentioned criterion, such as information availability. As an outcome of this stage, the scope of the research study is clarified and its subsequent feasibility verified, to some extent, at the respective case companies which consequently allowed the purpose and research questions of the thesis to be finalized.
35 4. Method
4.2.3 Collecting data Moreover, an information collection entails determining how, where and when the data was collected (Collis & Hussey, 2009). The information collection process for this research study consisted of both qualitative and quantitative methods, with significant emphasis on the former. The reason for this, as described earlier, is because the learning workflows being studied at the case companies are very rarely standardized and formalized in a manner that allows for quantitative methods to be suitable. For each case company, a significant amount of the empirical process was spent translating the findings from the interviews into expected workflows. Thus, in accordance with Collis & Hussey (2012), considering that the core purpose of the research was to investigate how Automation and AI can be applied in order to optimize learning workflows, without explicitly knowing what to expect, a qualitative research approach was deemed as most appropriate. Furthermore, the qualitative research predominantly consisted of semi-and unstructured in-depth interviews with employees at the respective case companies. The main goal of conducting these interviews was to be able to map out learning workflows for specific roles at these companies as well as their interaction with their surrounding organization, all of which are described more detailed in the forthcoming subchapter. However, upon having conducted the majority of the interviews, a secondary round of empirical research was conducted which targeted the research participants with a more technical experience of software implementation in order to evaluate the barriers of adopting automation and AI technologies. This stage of the empirical research was conducted in accordance with structured interview questions provided to the participants by a survey. The reason behind adopting survey as a tool for gathering data was both a result of the COVID-19 pandemic resulting in significant restrictions for personal encounters but also to attempt to strengthen the qualitative findings with quantitative data.
Case companies: In total, three case companies were included in this research study. As described previously in Chapter 4.2.1 Structure, an array of criteria was respectively utilized in order to successfully identify appropriate case companies. In total, 10 financial services companies were contacted, all with offices located in the Stockholm area. However, by taking into account the criteria, 3 companies were subsequently deemed as appropriate to serve as case companies.
36 4. Method
Table 2, overview of case companies
Number of Case Company Description Research Participants
Case Company A is a Swedish bank providing universal banking services, including traditional corporate Case Company A 4 transactions, investment banking and trading as well as consumer banking including life insurance
Case Company B is a Swedish financial group for corporate customers, institutions and private individuals with Case Company B 4 activities comprising of mainly banking services but also significant life insurance operations
Case Company C is a Nordic-Baltic banking group offering retail banking, Case Company C 4 asset management, financial and other services.
Research Participants: Moreover, a snowball sampling methodology was utilized in order to capture each research participant, which will be explained more in-depth in the following chapters. For each case company, a Learning and Development Expert served as the first subject in accordance with Table 2 above, primarily due to them most likely possessing the most knowledge of the research subject. Subsequently, the Learning and Development Experts recruited additional participants to interview. Summarized below in Table 3 is an overview of the research participants of each case company and their respective position within the sampling method.
37 4. Method
Table 3 summary of interviews
Respondent Role Organization Duration Date Type
Case Company March 3rd Face-to- RA1 L&D Specialist 60 min A 2020 face
Case Company March 3rd Face-to- RA2 L&D Director 60 min A 2020 face
IT Project Case Company March 18th RA3 60 min Skype Manager A 2020
Product Case Company March 18th RA4 60 min Skype Manager A 2020
Case Company March 3rd Face-to- RB1 L&D Expert 60 min B 2020 face
Product Case Company March RB2 60 min Skype Owner B 27th 2020
Internal Case Company March RB3 Auditing 45 min Skype B 27th 2020 Manager
IT Operations Case Company March RB4 45 min Skype Manager B 30th 2020
Case Company March 13th RC1 L&D Director 60 min Skype C 2020
Engineering Case Company April 3rd RC2 90 min Skype Manager C 2020
Compliance Case Company April 3rd RC3 45 min Skype Expert C 2020
Product Case Company April 9th RC4 60 min Skype Manager C 2020
38 4. Method
In total, 12 interviews were conducted across all three case companies respectively, covering roughly 12 hours of interviewing.
Interviews: In terms of the interviewees, all participated on a voluntary basis and were informed in advance regarding the topics that would be covered as well as their role within the empirical research process. Moreover, all interviews were dedicated approximately 45 to 60 minutes each considering the complexity of the L&D functions within each case company and in order to avoid making any preempted mistakes due to a lack of understanding of the case companies and their respective processes. Additionally, the interviews were conducted in both Swedish and English. Naturally, the interviews conducted in English might have posed an additional factor of uncertainty in terms of interpretation. However, this factor was considered outweighed taking into account the fact that it provided the participants with the ability to fully express their opinions and thoughts. During the interviews which were conducted in a semi-structured manner, the participants were given a topic and the subsequent opportunity to discuss their opinions surrounding it, as described in Appendix 1, whilst the interviewer had a passive role and occasionally provided some additional questions to further drive the discussion. These additional questions were both opened and closed, and primarily consisted of asking the interviewee to clarify a statement or provide more detailed information, without necessarily creating a strict frame for the interview.
Considering that there was a significantly limited amount of previous research within the area, understanding which stakeholders at the respective case companies to interview in order to extract the correct information was an additionally challenging aspect in regard to the empirical process. In order to address this, a snowball sampling or chain-referral sampling was utilized. Put briefly, snowball sampling is where research participants recruit other participants in order to further advance in the research process (Dudovskiy, 2017). According to Dudovskiy (2017), snowball sampling is best utilized where potential participants are difficult to identify and consist of two rather simplistic steps:
1. Identify potential subjects within the population. Generally, only one or two potential subjects can be identified initially. 2. Those subjects are then asked to recruit other people and, if required, those people are asked to make additional recruitments, and so forth.
39 4. Method
These steps are repeated accordingly until the appropriate sample size is reached. More specifically, as described by Dudovskiy (2017), an exponential non-discriminative snowball sampling method was utilized, in which the first subject recruited to the sample group provides several referrals which are explored until sufficient amount of primary data is collected.
Figure 9, an exponential non-discriminative snowball sampling method (Dudovskiy, 2017)
Within the frame of this research study, a snowball sampling was deemed an appropriate method considering that, firstly, the research emphasized specific companies and the collection of primary data from employees of those companies. Secondly, as described previously, the limited amount of research failed to provide any insights in regard to which stakeholders of the case companies that were optimal subjects to interview in order to successfully attain the appropriate information.
4.2.4 Analyzing data In accordance with Collis and Hussey (2012), the analysis of data generated from a case study can be conducted as a cross-case analysis or within-case analysis. Moreover, given that this specific research involved several case studies as well parts of the research purpose entailed establishing a generalized understanding of L&D within financial services companies from investigating multiple sources, cross-case analysis was deemed as most suitable. The data analysis was conducted in accordance with a rigorous mapping of the generated data, which will be described in depth as follows.
40 4. Method
Coding of Interviews: All interviews conducted at the respective case companies were recorded. The interviews which were conducted virtually were recorded in Zoom which were automatically transcribed by utilizing the software MAXQDA. Naturally, the automatically generated transcripts were subsequently reviewed and revised based on the original recording in order to avoid any misalignment with the actual interview. However, taking into consideration that the empirical research and all analysis work was conducted by the same research, the usage of automation would allow the research process to be as transparent as possible. MAXQDA enabled similarities to be identified between all 12 interviews and thus allow for conclusions to be drawn from the large quantities of empirical data gathered. In accordance with Blomkvist & Hallin (2014), this allowed for a thematic analysis to be conducted in which patterns and similarities are identified amongst all research participants. Taking into consideration that a significant amount of information was generated, a significant benefit of this process was to eliminate information deemed as redundant or irrelevant and, as described previously, draw conclusions from a rather complex collection of information. The coding process is visualized in the figure below, categorized based on their respective context and theme.
Figure 10, overview of coding process
41 4. Method
4.3 Quality of Scientific Research When conducting qualitative research, the responsibility of the researchers to maintain a high level of quality is essential and nevertheless inevitable. Theory developed from case study research is likely to have important strengths like novelty, testability, and empirical validity, which arise from the intimate linkage with empirical evidence (Blomkvist and Hallin, 2014). In order to analyze the quality of a scientific report, the concepts of reliability and validity can be utilized as suggested by Blomkvist and Hallin (2018). Therefore, these two concepts are introduced as two seperate subchapters and analyzed based on the research process utilized throughout this study in order to subsequently provide potential actions for improving its credibility.
4.5.1 Reliability Put briefly, reliability is a term used in order to evaluate the issue of measurement accuracy and precision, as well as the similarity of the generated results if the research study was to be repeated (Collis & Hussey, 2012). The research has consistently to maintain the highest level of reliability, primarily by having an objective view and treating collected data impartially in order to eliminate potentially misleading biases (Blomkvist & Hallin, 2018). Moreover, considering that a commissioning company is present and rather involved throughout the research process, an array of potentially inherent biases need to be taken into account. For instance, the purpose of this research, from the perspective of the commissioning company, was to investigate how their product offering could be utilized within corporate learning after having seen immense success within the education industry. Thus, considering the fact that these success stories had been clearly communicated to the researcher prior to initiating the study, a bias might be present in which solely positive outcomes are expected regarding the applicability of these technologies within corporate learning, by drawing hurried inferences with the education industry. In order to avert this bias of occurring throughout the research process, a very detailed process for conducting the research was established which both enhanced being consistently transparent as well as the inability to discuss and possibly tweak results before the empirical research had been finalized.
42 4. Method
4.5.2 Validity The second pillar of a report’s credibility is validity, which indicates whether the measurements have been done accordingly, how they reflect the research subject investigated and is highly dependent on establishing and utilizing an appropriate research design (Collis & Hussey, 2012). In social science research, construct validity as well as internal validity are two prominent ways of describing validity as a concept and can similarly be utilized to evaluate the methodological rigour of a case study (Greener, 2008). In terms of construct validity, which refers to the method utilized for measuring what the researchers intend to explicitly measure, several challenges occur during the explicit data collection process. Moreover, construct validity takes into account whether a clear chain of evidence exists in order to enable a potential reader to subsequently reconstruct the process of a researcher, from stating a research question to eventually drawing conclusions (Greener, 2008). Considering that the two main components of this study, namely automation and AI as well as corporate learning, have historically been treated as two rather distant topics, both in an organizational setting as well as academia, several challenges arise during the interview process. Furthermore, a significant challenge is therefore to ensure that each research participant understands the explicit relation between automation and AI technologies and corporate learning in order to avoid potentially misleading or confused answers. Considering that technologies such as AI tend to be frequently associated with buzzwords, such as self-driving cars, the researcher spent a noticeable amount of time during the initial interview stages to understand their knowledge within the topic and, if necessary, provide them with relevant facts and whilst emphasizing the purpose of the study. Additionally, each research participant was given the opportunity to verify that any provided information which was potentially utilized in the study was correctly interpreted, prior to the thesis publication.
Moreover, as described throughout this report, the research is based primarily on qualitative data, primarily due to the exploratory nature of the study but also because of lack of relevant non-qualitative data at the respective case companies that could be beneficial. In order to strengthen the research triangulation by using multiple data sources and thus enhance the construct validity, additional quantitative sources could be valuable. Ideally, a research study investigating the opportunities concerning automation and AI within L&D would incorporate an actual implementation process. This would provide additional insights regarding the explicit benefits of adopting such technologies by, for instance, incorporating aspects of time saved for employees in training.
43 4. Method
However, this would naturally go beyond the current research question, which is of an exploratory nature, and more likely investigate the benefits of adopting automation and AI technologies within L&D. Furthermore, an additional significant effort to verify and increase the construct validity of the research was to utilize the peer reviews conducted by both professors, supervisors and fellow students of the Royal Institute of Technology, KTH, as these provided immensely valuable advice in terms of the research method.
Internal validity refers to causality and can moreover be explained by examining whether a factor (the independent variable), causes an effect (the dependent variable) to happen and should, however, not necessarily be confused with simple association between the two factors (Greener, 2008). As described, the scope of this thesis did not entail conducting an actual implementation of automation and AI technologies within L&D. Primarily considering the lack of time but also the fact that it would most likely go beyond the subject of which this thesis is written under, namely Industrial Management. Additionally, another challenging factor in terms of strengthening the internal validity concerns making comparisons to previous research, of which there were immensely few that captured automation, AI and L&D. However, research and literature that separately emphasized automation and AI implementation as well as L&D were triangulated in order to enhance the internal validity.
4.5.3 Source Criticism The primary as well as secondary sources of literature utilized in the literature review of this thesis report as well as constructed methodology can be derived primarily from academic journals, published books as well as industry reports, all of which have been discussed critically throughout the reports with peers from the Master’s program in Industrial Management and KTH Royal Institute of Technology as well as the thesis supervisor. In order to improve the quality of sources derived from primary research, a significant amount of effort was put into ensuring that the selected case companies were appropriate, as described in 4.3.2 Structure. Specifically, the companies were required to operate within the financial services industry, have an established L&D organization and be able to share relevant information in regard to their L&D processes. Additionally, the stakeholders targeted to participate in the interviews within the case companies were also selected after significant consideration and research, partially with input from the commissioning company based on their industry expertise within L&D and which stakeholder would most likely hold the required information.
44 4. Method
4.6 Research Ethics When conducting research within social science, there are multiple dilemmas that one can expect to encounter in terms of research ethics that need to be considered and, to some extent, integrated into the research process. As it is the responsibility of the researcher that the design and subsequent execution of the research is aimed towards the generation of new knowledge. However, taking into consideration that the research itself contains uncertainties, potential harms are not necessarily recognized before conducting the research. Therefore, it is the duty of a researcher to prevent or, in a worst-case scenario, minimize the potential harm caused to research participants as a result of having a research process that incorporates potentially unethical aspects. In the following subchapters, this. According to the Swedish Research Council (Blomkvist & Hallin, 2018), there are four basic ethical requirements within social science that have to be met upon conducting research; (i) the information requirement, (ii) the consent requirement, (iii) the confidentiality requirement as well as (iv) the good use requirement. In the following subchapters, a critical discussion in regard to research ethics from both an academic as well as professional perspective will be held, departing from the four ethical requirements mentioned previously.
4.5.3 Academic Perspective Naturally, researchers face significant challenges and dilemmas in terms of research ethics during the empirical process in which research participants are most likely to be encountered. In a case study scenario, research participants could potentially possess highly-sensitive company-specific information which, if mistreated by the researcher, might jeopardize their position and job safety. To begin with, for this research in particular, each participant was priorly informed in terms of the purpose of the study and ensured that the collected information would be used for the intended and stated purpose. This is an aspect that is particularly important considering the involvement of a commissioning company. The research participants might be hesitant to share information if they believe it would solely serve the interest of the commissioning company, which is the reason why it is crucial to clearly state which information is relevant and what purpose it would serve for the outcome of the study. However, although a researcher might continuously emphasize transparency throughout the research process, the participant might still display signs of discomfort and unwillingness to fully participate if a certain level of trust has failed to establish. In fact, complete disclosure of a research process does not guarantee that an individual interviewee
45 4. Method
Moreover, in terms of research ethics and the snowball sampling method utilized during the empirical process for particular research study, the study participants were never asked to identify other potential participants. Rather, they were asked to encourage other potential participants to come forward.
Prior to each interview, the participants were informed in regard to the explicit purpose of the study, their role in the research, how the information would be utilized as well as the expected outcome, all of which were repeated and emphasized furtherly during the interviews. Moreover, a representative from each respective case company was allowed to fully inspect the report prior to publication in order to ensure that there was no misleading, inaccurate or inappropriate information related to the case company and their L&D organization. Also, considering that the thesis was to be published and shared on a digital university platform, namely Diva, it would remain accessible for each case company in case of future inquiries. In terms of confidentiality of the collected data, each case company and their respective research participants were fully anonymized throughout the reporting process and furthermore never revealed to the commissioning company in order to avoid conflicting interests. Additionally, although the commissioning company had provided the researcher with a computer to conduct the thesis project, company-specific data was never stored on that computer. Furthermore, the collaboration with the participants was continuously treated as a central aspect throughout the research process in order to guarantee that the interests of the participants were aligned with the expected outcome of the empirical process.
4.5.3 Professional Perspective Research ethics surrounding a professional perspective is much rather characterized by ensuring that the results and recommendations of the study adhere to norms of scientific work when presenting the outcomes in both written and oral form. The findings are both required to be presented in a way so that is can be evaluated by others and simi
An equally important factor from a professional perspective, in accordance with the Code of Honor established by the group Swedish Engineers (2019), entails consistently striving for factual representations whilst avoiding any misleading or false statements whilst simultaneously respecting entrusted and potentially confidential data. Moreover, the Code of Honor highlights one aspect which is of significant importance regarding the outcome of this research in particular and similarly reflects the purpose of the study in its entirety.
46 4. Method
This aspect entails offering the accumulated knowledge in the specific research context to ensure the best possible basis for decisions and illuminate risks associated with technology. Concurrently, the findings derived from this research incorporate rather advancades and, for many, unfamiliar technologies. Therefore, one absolutely essential aspect would be to ensure that the results and recommendations provide a valuable foundation for further decision-making and that the targeted organizational function, namely the L&D department, can acquire appropriate knowledge in regard to the limitations and capabilities of such technologies. After all, regardless of much time and effort is spent on developing and perfecting a proposed solution, which in this case consists of a learning workflow characterized by automation and AI, it becomes practically invaluable if the stakeholders do not possess the appropriate understanding or feel uncomfortable immersing in such technologies. Therefore, it should be considered the personal responsibility of the researcher to support those most affected by the outcome of the research, which in the context of this thesis concerns the stakeholders of the L&D departments. From a practical perspective, this was done by equally involving the case companies as well as the commissioning company within the research design process. As a result, the research was able to both capture the interests of the commissioning company whilst taking into the consideration the potential limitations of the case companies in terms of utilizing the outcome of the research.
47 5. Results and Analysis
5. Results and Analysis
This section presents the findings derived from the empirical process of the research presented and analyzed, in which data was collected through conducting case studies in the form of interviews at three financial services companies, primarily operating in the Nordic region. Firstly, the current state of how training is conducted within financial services is presented in order to acquaint the reader with the status quo of L&D. Secondly, the identified barriers of adopting automation and AI technologies are presented in order to, lastly, propose an learning workflow which takes into account both the adoption barriers as well as the capabilities of automation and AI.
5.1 Status Quo of L&D within Financial Services In order to be able to evaluate how the L&D function within the financial services industry could utilize automation and AI technologies, the first step would naturally entail understanding how it currently is conducted. The following chapter will discuss a generalizable status quo of L&D amongst all case companies included in the scope of this research. As found in the empirical research, the process of creating and conducting a training can be categorized into four high-level steps as summarized in the figure below.
Figure 11, a high-level perspective of the L&D process within the Financial Services industry
First, prior to conducting a training, the content needs to be created or sourced internally or externally. Secondly, the content is distributed to the user, in which the empirical findings revealed that this could be conducted in many ways. Most commonly, a LMS was utilized. However, several research participants reported using more outdated methods such as sending emails to each participant in order to distribute the content. Thirdly, the employees conducted the training, which could either be carried out in an LMS, or simply consist of them reading through endless pages of pdfs. Lastly, the training was reported and documented. The most commonly used metric was found to be participation and pass rates of the employees having undergone the training, with very little focus put on knowledge retention or content mastery.
48 5. Results and Analysis
Additionally, there was little to no structure behind the reporting initiatives of the case companies, in which many simply had a list in excel of each participant whose name they checked of upon completing the training. Moreover, each step of the described L&D process will be analyzed in more detail in the following subchapter.
5.1.1 The need for training emerges The decision-making process of whether a specific training should be initiated and carried out to the employees was found to be rather two-dimensional from the empirical research. Interestingly, the way in which the training was conducted did not differ between different roles at the specific case companies apart from which content that the training contained. However, the need for employees to undergo a specific training would either arise from regulatory requirements from Swedish Financial Supervisory Authority (Finansinspektionen), or due to internal requirements where, for instance, a regional manager has noticed that several employees have displayed skill gaps for certain financial products. As follows, the two triggers are discussed in more detail.
Regulatory requirements: As described, the Swedish Financial Supervisory Authority, hereby abbreviated as the SFSA, tend to interact with financial services companies in two ways in terms of conducting training amongst employees. In the first instance, the SFSA could either pose a rather hypothetical question to a financial services company in terms of how they conduct their business and whether they are compliant with Swedish financial regulations and laws. A L&D manager at one of the case companies described one example how these questions can be formulated by the SFSA:
‘’How can you as a financial services company ensure that your customer-facing employees are knowledgeable within a certain credit product?’’
In this scenario, the financial services company would be required to conduct a training without knowing any specific requirements of what that training should entail. As for the second case of regulatory requirements, the financial services company would simply be provided with specific content by the SFSA related to, for instance, a new financial product, data protection or anti-money laundering. Moreover, in this scenario, the financial services companies are also obliged to report to the SFSA that the specific people that are affected by the new training actually have conducted it, usually in the shape of a completed quiz, or face rather severe legal consequences. As described, the directives provided by the SFSA can be rather vague and, in order to bring some additional clarity from a training perspective, are subsequently interpreted by an internal auditing function at the respective company.
49 5. Results and Analysis
As a result, the internal auditing function is able to produce a requirement specification based on the information provided by the SFSA in terms of which aspects the training should cover and who should be involved
As for the second case of regulatory requirements, the financial services company would simply be provided with specific content by the SFSA related to, for instance, a new financial product, data protection or anti-money laundering. Moreover, in this scenario, the financial services companies are also obliged to report to the SFSA that the specific people that are affected by the new training actually have conducted it, usually in the shape of a completed quiz, or face rather severe legal consequences. As described, the directives provided by the SFSA can be rather vague and, in order to bring some additional clarity from a training perspective, are subsequently interpreted by an internal auditing function at the respective company. As a result, the internal auditing function is able to produce a requirement specification based on the information provided by the SFSA in terms of which aspects the training should cover and who should be involved. As found in the empirical research, the internal auditing function, in terms of translating external requirements into specifications for internal training.
Internal requirements: As for the other trigger of whether a specific training should be initiated in which a lack of knowledge served as a significant driver, the structure was rather different as opposed to regulatory requirements. In this case, someone in the organization would recognize a lacking competence within a specific area. This individual would, as found in the empirical research, most commonly either be someone in a managerial process or one of their subordinates. What was considered as ‘’lacking competence’’ was rather undefined at the respective case companies as well as for specific roles. One financial services company utilized their Customer Relationship Management (CRM) system quite heavily, in which feedback generated from existing and prior customers was utilized to evaluate their performance. Whereas some relied on receiving occasional feedback from regional managers, pointing out that a specific employee made, for instance, several errors. Nevertheless, the definition was rather subjective.
However, one common theme amongst all case companies was to conduct an annual or bi- annual employee competency mapping. Essentially, this was utilized for the case companies to grasp which competencies that are internally insufficient amongst their workforce. Naturally, the competencies required for a specific employee was shown to vary depending on their role and job tasks. However, there were certain competencies that were required for each employee, regardless of their role specifics.
50 5. Results and Analysis
Examples of such unconditional competencies entailed ethics, company-specific policies, anti-money laundering and GDPR compliance. Apart from these generic competencies, there was also an array of role-specific competencies that were unique based on the function which the employee had at the company. Furthermore, departing from the competencies that are both role-specific and generic for the organizations in its entirety, the employee and their closest manager evaluate how well the employee is aligned with these competencies. As a result, the two agree which skills that the employee is currently lacking and thus which should be addressed until the next competency mapping.
However, as found in the empirical research, a significant challenge for financial services companies lies within understanding whether a specific competence mastered in, for instance, an LMS accurately portrays their explicit performance. As described earlier, one of the most information- and training-intensive roles within financial services companies is within retail banking, in which employees have to obtain and maintain an array of licenses in order to be allowed to legally discuss certain financial products with a customer. Therefore, taking into account that customer service and relationship management are significantly difficult to measure considering their qualitative nature, it can be rather challenging for an organization to understand the knowledge that a certain employee possesses and which skills they lack. For instance, simply because an employee has mastered an online course within sales, does not naturally make them excellent salespeople. Ideally, the LMS utilized by employees within retail banking would be integrated with their Customer Relationship Management (CRM) system. Being customer- centric was not only empirically proven to be an essential aspect for retail banking, but for the financial services companies in their entirety as described previously. An IT-project manager at one of the case companies summarized their specific mentality towards being customer-centric and the importance of emotional intelligence.
‘’In today's financial climate, we have to be customer-centric in everything we do, which is quite a challenging metric to measure amongst our employees’’
Nevertheless, a rather poor and at times misleading measurability was a frequently occuring theme amongst all case companies. The data used by decision-makers and other stakeholders proved to
51 5. Results and Analysis
5.1.2 Creating Content Naturally, after having identified the need for training to be conducted within the organization, the next step entailed creating the actual content intended to be utilized within the training. As described previously, in some instances the SFSA would simply provide the content to the organizations which would subsequently be distributed to the relevant individuals. However, during instances in which internal requirements or regulatory requirements will force the financial services company to develop the content themselves, both of which are discussed in this subchapter.
Regulatory requirements: As described in the following subchapter, the empirical research suggested that financial services organizations have an internal auditing function that holds the responsibility of creating a requirement specification. This requirement specification describes which elements the training has to encompass in order to be compliant with the SFSA. The specification is subsequently sent to a functional expert, which is someone within the organization who is deemed competent within the specific area. Consequently, this person is now responsible for developing the explicit content to enable employees to receive training in order to be compliant with the regulatory requirements. This functional expert can be anyone working as an IT-project manager, auditor or software engineer. Furthermore, the common theme amongst all case companies was that this functional expert possessed very specific competencies. However, in order to ensure the pedagogical quality of the content being developed by the functional experts, the HR department would similarly be involved. The HR department, as found in all three case companies, would have an L&D expert work collectively with the functional expert, in which their main objectives, generally, were to:
1. Ensure that the content holds an appropriate pedagogical quality Some of the aspects that an L&D expert would look at in terms of the pedagogical quality of newly-developed content. Although no case company had a specific framework in terms of whether content was perceived to hold an appropriate level of pedagogical quality, some commonly-occuring actions entailed:
a. The content does not contain any unfamiliar terms b. The content is phrased appropriately and grammatically correct
2. Ensure that the content is at the appropriate level of difficulty The L&D expert would evaluate the content in order to ensure that it’s difficulty is suitable for its intended audience by:
52 5. Results and Analysis
a. Looking at the competency mapping of each employee involved in the training in order to ensure that their skill gaps and existing knowledge is aligned with the content difficulty. b. If necessary, make adjustments together with the functional expert to ensure appropriate level of difficulty
Afterward, the content is yet again sent to the internal auditing function, where they are able to ensure that the training is compliant with the initial regulatory requirement by the SFSA.
Internal requirements: As for internal requirements, the process of developing content is rather similar. One distinguishing factor was that, if the training required to develop the competencies in the field which was found to be insufficient already exists, the involved employees can simply access their LMS and conduct their training. Additionally, if the content does in fact not exist, the creation process is similar to when facing regulatory requirements, whereas the only exception is the lack of involvement of the internal auditing function. In essence, the manager who identified the lacking competencies contacted a functional expert, without explicitly knowing if that person is the most appropriate to use as a source of knowledge. Subsequently, the functional expert developed the content in collaboration jointly with the HR-department in order to ensure pedagogical quality.
5.1.3 Distributing Content Furthermore, upon having either created or sourced to content to be utilized as a training, the content is subsequently distributed to the respective employees. Optimally, this would entail ingesting the content into the LMS of the organization and thereafter assigning the content in, for instance, the form of a new course, to each employee. However, taking into consideration that migrating content to a learning platform can be a rather time- consuming process, each case company revealed that the employee was simply sent the content in the form of a document. Naturally, this would result in an array of functionalities within the existing LMS being unutilized and thus similarly affect the learning experience and outcomes. Furthermore, as revealed by the empirical findings, all case companies reported frequently simply sending the content to the employee as a pdf. The rationale behind deploying content manually amongst the case companies was quite scattered. However, a common denominator seemed to be related to the explicit functionalities of the LMS. Moreover, in order to publish the content within an LMS, it
53 5. Results and Analysis either has to be converted into an appropriate format compliant with the LMS or be authored manually within the authoring tool of the LMS. Normally, content created from the previously described content creation process seemed to vary quite extensively in terms of, for instance, which media format was utilized as well as how well they were objectively written. Additionally, participants from each case company expressed time being a significant constraint in terms of ingesting the content to their respective LMS and subsequently distributing it to the relevant employees. Moreover, when time was not an issue, several L&D representatives expressed a significant dissatisfaction towards the explicit authoring functionalities of their LMS, implying that the overall user experience was rather poor and simply not worth the time as opposed to distributing the content manually.
5.1.4 Documentation and Reporting Being able to report that an employee has undergone a specific training is an essential element of an efficient L&D process, especially for highly regulated industries such as the financial services industry. The existing LMSs utilized within the case companies provide features to support stakeholders in the documentation and reporting process by, for instance, showing completion rates. However, as described earlier, by taking into account the rather poor adoption of LMSs, several of these features go unutilized similar to when distributing the content. As a result, a significant portion of work related to documentation and reporting conducted by the case companies were done manually. This would entail creating a spreadsheet with each employee and manually checking of each individual’s name when they reported having finished the training. Therefore, this process was completely reliant on both trusting that the employee actually had engaged with the content and afterwards remembered to send a follow up email to the administrator in order for them to document their completion. This was despite the fact that the empirical research pointed towards financial services companies being required to provide documentation to the SFSA that the employees involved in the training actually have conducted it. In addition, as for the training generated by internal requirements, there is rarely any follow up to ensure the retention of knowledge for the employee. Once the employee has reported their completion, the training program is more or less finalized, with no explicit activities to ensure that the employee explicitly understood the content after the training and, more importantly, in the future when applying the knowledge.
54 5. Results and Analysis
5.1.5 Holistic As-is Perspective of Corporate Learning The complete process of conducing a training program which has been described throughout this chapter can be visualized in the figure below:
Figure 12, holistic and generalized perspective of how corporate learning is conducted at the case companies
55 5. Results and Analysis
5.3 Barriers of Adopting Automation and AI within L&D As found in the empirical research, none of the case companies had adopted any element AI within their current internal L&D organization. As described earlier, the usage of AI within an L&D context is usually related to creating a tailored path for each learner in order to address their pressing skill gaps in the most time-efficient manner. However, the empirical evidence pointed towards a practically non-existing adoption of AI technologies within the L&D function of the participating case companies. Moreover, the following subchapter will utilize the previously described conceptual framework, which incorporates aspects relevant for organizations to create a readiness towards adopting automation and AI, in order to describe the empirically-identified barriers.
Interestingly, by looking at the surrounding organization of the respective case companies and nevertheless the financial services industry in its entirety, AI serves as a powerful tool deployed across multiple functions. One noteworthy use case is the usage of chatbots and virtual assistants, in which elements of customer-facing services can be automated and augmented using cognitive technologies. Moreover, as for automation, the scenario was similar to AI but slightly more mature. In terms of adopting external solutions within L&D, a noticeable theme amongst all case companies was the usage of LinkedIn Learning. Put briefly, LinkedIn Learning is an online educational platform that helps employees discover and develop their skills by immersing themselves in an array of expert-led course videos. However, from an AI adoption perspective, LinkedIn Learning does not in fact use any elements of personalization to power their courses, interestingly.
5.3.1 Technical Barriers Naturally, adopting automation and AI technologies will require quite the complex integration process in order to yield a successful adoption. A common theme amongst all case companies was that they all build IT-tools and systems which they are heavily reliant on. One IT-project manager at Case Company A described their mentality towards utilizing IT-tools and systems internally:
‘’I think financial services companies in general are picky in terms of which systems and applications they use. They like things their own way. Finding and implementing IT-systems of the appropriate degree of customizability can be very difficult and time-consuming which is why when we need something, we build it ourselves.’’
56 5. Results and Analysis
Consequently, this results in an array of challenges regarding the adoption of additional technologies. As described earlier, AI-powered solutions for L&D can be delivered in an SaaS-manner, meaning that an organization can simply license the software to power their platform or application. However, regardless of its near plug-and-play deliverability, the empirical findings pointed towards several underlying elements which affect whether the adoption of automation and AI technologies is technically feasible.
Figure 13, share of case companies using in-house-built software within L&D
Not to mention, two out of the three case companies were using in-house software development to create internal tools to be used within L&D as illustrated in Figure 13 above, which naturally results in several barriers that were evaluated furtherly. Out of the two case companies who utilized in-house software for L&D, they were also asked to clarify which challenges this subsequently causes in terms of adopting automation and AI technologies, which can be summarized in Table 4 below.
57 5. Results and Analysis
Table 4 Survey respondents
From 1 = do not agree at Statement all to 5 = fully agree
In-house-built software creates a technical rigidity in which it makes the organization inflexible towards adopting and 4.4 potentially integrating complementary technologies
In-house-built software creates flexibility in the sense that it 3.2 enables quick customization
In-house-built software requires significant investments. 4.0
Maintaining a desired level of quality of the in-house-built software can be challenging as a result of lacking technical 4.3 expertise
Maintaining a desired level of quality of the in-house-built 4.1 software can be challenging as a staff dismissal
As derived from the empirical research and presented in Table 4 above, it is rather clear that all case companies perceive technical rigidity as a significant factor for adopting complementary technologies, such as automation and AI. As of now, given that two thirds of the case companies were utilizing in-house-built software within L&D, thus making them susceptible to technical rigidity, a significant challenge would nevertheless be to integrate automation and AI technologies. Furthermore, regardless of the fact that many of the solutions powered by automation and AI can be delivered in a SaaS context, as described earlier, the currently limiting functionalities of many in-house-built software systems might serve as a significantly limiting factor. For instance, one essential aspect in order for AI to function at its maximum capacity and if the existing software systems are unable to provide sufficient amounts of data, the technology itself might not provide any additional value due to a lack of training data. Similarly, for automation technologies access to multiple system is an essential component for optimal performance and could thus be hindered due a rigid software solution with poor cross-functional access.
58 5. Results and Analysis
5.3.2 Organizational and Regulatory Barriers Naturally, adoption automation and AI technologies entail barriers simply beyond technical aspects. Taking into consideration the mere complexity of the financial services industry, the organizations themselves account for an array of barriers which might hinder the adoption process. As derived from the empirical findings, all three case companies expressed that all of their L&D initiatives are characterized by a significant high degree of self-leadership. This entailed that each individual employee was, to some extent, responsible for developing and managing their individual learning journeys. However, looking at the general L&D organizations of the respective case companies, adopting new technologies was a rather insignificant feature of their current strategic agenda. Naturally, strategies for driving digital transformation exist on higher-level agendas of the case companies, but the empirical evidence points towards a quite poor diffusion to how training is conducted.
Moreover, as derived from the empirical findings, integrating a complementary technology to the existing L&D organization was an immensely time-consuming process, which can be summarized in the figure below:
Figure 14, time for deployment of most recent external software solution
59 5. Results and Analysis
One interesting factor for this specific empirical finding was that all case companies had LinkedIn Learning as their most recent external software adoption and thus enabled the time for adoption to be explicitly compared amongst the case companies. Moreover, from an AI adoption perspective, LinkedIn Learning does not in fact use any elements of personalization to power their courses. As described earlier LinkedIn Learning is simply a plug-and-play online educational platform and does hence not require any extensive technical integration. Moreover, given the additional layer of technical complexity of automation and AI technologies as opposed to a quite primitive platform solution such as LinkedIn learning, one can expect that the adoption process of automation and AI technologies would take longer. As described earlier, automation and AI solutions thrive on data in which their performance is more or less correlated with both the quantity and quality of data. Therefore, given the need for data within automation and AI technologies, the adoption process would most likely be more extensive as opposed to a platform solution and thus, as described, require more time.
Moreover, upon diving deeper into the issue regarding barriers for adopting automation and AI technologies, several research participants holding a technical role expressed quite a significant dissatisfaction in terms of all case companies having quite an extensive software procurement process. Naturally, the procurement processes for L&D solutions varied between the respective case companies but there were however some common denominators as revealed by the empirical research, all which will be discussed as follows.
Allowing substandard quality for lower cost: Interestingly, two out of three L&D representatives from the respective case company expressed that the L&D tends to be neglected in terms of company-wide strategic agendas. Many argued that L&D in general is treated as a rather low prioritization in regard to company-wide digital transformation that many financial services companies are currently in the midst of undergoing, Therefore, it was rather clear that L&D innovation was not a clear target for allocating significant amounts of capital. As a result, the L&D organizations were reluctantly forced to adopt solutions of poor quality in order tackle their insufficient budgets.
Unclear specifications and requirements: Referencing the previously described lack of strategic prioritization of L&D innovation across financial services companies, several research participants similarly expressed that during the few instances in which it actually is prioritized, it is usually characterized by buzzwords and a non-existing plan for execution. Therefore, given that the requirements of the L&D solution become rather vague, the L&D organizations tend to find themselves confused in terms of which specific technology should be prioritized to adopt.
60 5. Results and Analysis
As described earlier, status quo within corporate learning for financial services companies is to measure completion and pass rates, which is quite different in comparison to the capabilities of AI in learning, which instead tend to emphasize knowledge mastery and retention. In addition, none of the investigated case companies had any previous history of adopting AI within their L&D organization, which subsequently results in the concept being rather unfamiliar for them. Therefore, given both the poor specifications and requirements for L&D solutions as well as the general unfamiliarity for automation and AI solutions within L&D, the adoption of such technologies might be challenging.
61 5. Results and Analysis
5.4 Future state of AI and Automation adoption within Corporate Learning for Financial Services Companies
Given the current state established in the previous sub-chapter of how corporate learning is conducted across the respective case companies, the following sections will aim to emphasize and evaluate how the current L&D organization can utilize automation and AI technologies.
5.4.1 Empowering Decision-makers with Learning Analytics As derived from the empirical research, the entire process of developing and conducting training across the case companies was characterized by a significantly low measurability. Moreover, an imperceptible amount of intelligence was utilized when making critical decisions regarding, for instance, who should conduct a training, which content should be deployed and how difficult should it be. However, analytics in the context of L&D does not in fact need to incorporate the most advanced mathematical models in order to be insightful for stakeholders and decision-makers. In fact, collecting and tracking the simplest data could enable an immense amount of findings and insight to surface and subsequently empower critical decisions within training.
Content Insights: To begin with, the empirical evidence revealed that the case companies receive no intelligence in terms of which program is popular amongst employees. However, simply tracking the number of users accessing a course, their test results as well as time spent on a specific course or assignment, would yield an immense amount of relevant insights to shape how future training programs should be conducted. Moreover, in the more advanced dimensions of AI, certain systems are able to effectively assign a difficulty level for specific elements of content, at an immensely granular level. Thus, understanding which content are perceived as difficult by the learner would allow administrators to not have to manually estimate the difficulty level of a specific content element, as described in 5.1.2 Creating Content.
Learner Insights: On the other hand, one of the most significant issues with the current state was the lack of transparency in terms of the learner having completed a specific training. Currently, for those training programs conducted outside of the existing LMS, the case companies simply relied on the employees being honest in their reporting and trusted that the employees would in fact conduct an assigned training. Thus, learning analytics would yield an immense array of insights in terms of understanding how learners are progressing through their training.
62 5. Results and Analysis
In addition to explicitly understanding if an employee has conducted a training, pass rates could reveal whether a training program was too difficult or too easy, which was one of the main tasks of Human Resources during the content development process.
5.4.2 Elevating Tedious Tasks from Administrators and Human Resources As described throughout the thesis, an extensive array of administrative tasks was conducted in order to produce and execute a corporate learning program end-to-end. Not to mention, the administrators throughout the training development journey play a crucial role in delivering a high-quality program. However, a noticeable amount of their work is susceptible to several opportunities for automation and AI implementation. To begin with, the most primitive rule-based software robots would enable several tasks of an administrator to be eliminated. For instance, one common administrative task conducted by many stakeholders involved in creating a training program was to manually communicate with the employees. This would occur constantly throughout the process of creating and conducting a training program and naturally resulted in administrators having to conduct redundant manual work.
Moreover, as the empirical evidence reveled that compliance is an immensely important component of financial services companies, an array of opportunities exists in terms of automating compliance functions. For instance, if an employee failed to comply with the latest Swedsec regulations by a certain threshold, both the administrator and employee could receive a triggered notification enabled by a software robot that the employee should review the material again. Moreover, an array of opportunities for triggered notifications exist within the L&D organization of financial services companies. For instance, simply notifying learners that they have been enrolled to a new course eliminates a significant portion of the administrative work conducted by Human Resources at financial services companies today. Moreover, taking into account triggered notification powered by software robots are customizable in the sense that the administrator can simply modify the message received by the learner, they can be tailored to fit an array of purposes within L&D. As revealed by the empirical evidence, the SFSA play the most central role for financial services companies in terms of how which content is deployed the learners and who are required to undergo the training. Thus, the SFSA would similarly benefit from triggered notifications. As opposed to the administrator, as described in Figure 12, having to manually report to the SFSA that specific employee, or rather group of employees, has completed the training, the SFSA could simply receive an automatically generated notifications triggered by the completion of a course.
63 5. Results and Analysis
The reporting function was also a rather unorganized process in terms of training conducted from internal requirements. As opposed to having to rely on gut-feeling during the bi-annual employee competency mapping, the manager can simply receive a notification that the employee has demonstrated a lack of skill within a specific subject and subsequently assign them with a remedial training program.
5.4.3 Tailoring the Learning Path for Each Employee The empirical evidence revealed that very little attention is paid towards taking into account the existing skills and knowledge gaps of employees undergoing training. As described in chapter 5.1.2 Creating Content, administrators have no explicit understanding of whether an employee has undergone a specific training and similarly what their current skill levels are. This is essentially the result of there not existing a centralized function which tracks if a learner has completed a training or not, as revealed by the empirical evidence. Each employee of any organization, with financial services companies being no exception, have widely different understanding of the worlds as well as different potentials to achieve goals and thus different needs for learning. As described, the current state of L&D within financial services is characterized by high degree of one-size-fits-all. Furthermore, as employees might learn from the training, it is a highly ineffective process and might in some cases even discourage the employee from participating in future training simply due to the sheer redundancy. Therefore, the current learning experiences within the respective case companies could utilize adaptive learning to personalize the content for each learner, tailored to their specific needs.
64 6. Discussion and Conclusion
6. Discussion and Conclusion
In this section, the generated results will be discussed and dissected in terms of how well they are able to answer the previously stated research questions. The main findings of the research will additionally be described as well as its contribution to the field of both Operations and Technology Management. Secondly, aspects of sustainability and ethics are discussed in relation to the research subject. Lastly, the chapter is concluded by discussing suggested areas of future research.
6.1 Critique The most significant issue with regards to this specific research was that it did not in fact include an explicit implementation of any automation or AI technologies. However, given the exploratory nature of the study, the scope was rather to align the current workflows within the L&D of financial services companies with the capabilities of automation and AI. Hence, the aim of the study was to establish a foundational understanding of how and where automation and AI technologies could improve the L&D organization of financial services companies, as the existing body of knowledge is still at infant stage.
6.2 Main Findings The purpose of this study was to conduct an exploratory case study of the financial services industry and evaluate how automation and AI technologies could be utilized to improve their Learning and Development organization. Additionally, the study emphasizes the barriers identified that financial services companies are required to overcome in order to successfully adopt automation and AI technologies within a Learning and Development context. In order to successfully fulfill the purpose of the study, the following research questions sought to be answered throughout the research process. The following has been established as the main research question for the study:
How can Automation and AI improve Corporate Learning for financial services companies?
In order to answer the main research questions, the following sub-research questions were formulated:
• How is Corporate Learning currently conducted within the financial services industry? • What are the biggest barriers that financial services companies are facing in terms of adopting automation and AI technologies within their L&D organization?
65 6. Discussion and Conclusion
Put briefly, the research points towards several opportunities for adopting automation and AI technologies within the L&D organizations across financial services companies. Moreover, the empirical evidence reveals that, firstly, the current process of developing and conducting training is characterized by a lack of measurability throughout the learning journey of an employee as well as imperceptible intelligence during the decision- making process within learning and development. Similarly, the research revealed that very little consideration is taken in terms of the existing knowledge levels of employees, thus resulting in many employees having to conduct redundant training. However, the reasoning behind this, from the perspective of the case companies, was in fact, to some extent, logical. The empirical evidence revealed that the Swedish Supervisory Authority (SFSA) are essentially the most critical stakeholders in terms of which training should be conducted and who should be involved. Thus, the power in which the SFSA possess is not something that can be taken for granted by financial services companies but, within the scope of this research study, could similarly be transformed using automation and AI.
Therefore, an array of opportunities exists for the adoption of automation and AI within the L&D organization of the financial services companies themselves. For instance, the large amount of tedious administrative work by the administrators, such as manually emailing the content to the employees, could be replaced with automation robots and thus enable them to spend time on more high-leverage and value-adding aspects of their L&D organizations. Moreover, the significant lack of overall measurability as well as intelligence within the decision-making processes of the financial services companies all point towards an array of opportunities to implement learning analytics. This would, firstly, allow stakeholders abandon their gut-feeling when designing and deploying training programs by being empowered by insights for the behavior of learners as well as the content they interact with. In addition, given the fact that one of the most significant challenges at the case companies was the general redundancy of training and the non- existing consideration of existing skill levels, AI-powered systems would allow each learner to be provided with personalized learning experiences tailored to their specific needs. However, given the current state of the financial services companies, an array of challenges exist which might potentially hinder the adoption of automation and AI technologies within L&D. For instance, two thirds of the case companies reported using in- house-built software systems within their L&D organization which naturally might result in technical rigidity in the sense that it could prevent automation and AI technologies to reach their maximum capacity.
66 6. Discussion and Conclusion
6.3 Contribution Within the body of knowledge that is Industrial Management, this research has contributed to the area of Operations Management. Operations Management as a field partially emphasizes organizing the administration of different business practices to create the highest level of efficiency possible within an organization. Moreover, this view can be aligned with this specific research study due to its scope, which emphasized the evaluation of existing L&D organizations of financial services companies and subsequently how they could utilize automation and AI to optimize the creation and execution of training programs. Moreover, given the fact that the study captured several aspects of managing the use of technology for human advantage, one might also say that the findings of the research could contribute to the field of Technology Management. In the context of this study, the human advantage entailed, for instance, creating an optimal learning path for each learner or empowering decision-makers with data-driven insights, all of which were a result of using a specific array of technologies.
6.4 Sustainability In order for the opportunities of adopting automation and AI technologies within corporate learning for financial services companies, as described in the previous chapter, to be valid, one must consider its implications on aspects related to sustainability. The general concept of sustainability was defined by the United Nations (UN) in 1987 and comprises of three respective dimensions; economic, social and environmental sustainability (United Nations, 1987).
Firstly, in terms of economic implications, the role of automation and AI and their respective impact on businesses and nevertheless the global economy has been an increasingly popular topic during the last years. One interesting topic related to economic sustainability on an organizational level is how automation and AI could potentially lead to a significant performance gap between companies on the edge of adoption as well as slow- or non-adopters. Moreover, within the context of Learning and Development, this is directly related to the fact that a noticeable percentage of market capitalization is based on intangible assets (McKinsey, 2016), thus making exceptional leadership, skilled employees and transferable knowledge essential components for any organizations aiming for success. A second aspect to consider within the scope of this study is naturally the employees. Furthermore, a widening gap might similarly unfold between individual employees in addition to organizations.
67 6. Discussion and Conclusion
Put briefly, demands for jobs could shift towards those who are cognitively and socially driven given the fact repetitive tasks could be completely replaced by automation and AI technologies. Therefore, job profiles characterized by a high degree of repetition and administrative work might experience a noticeable decline as a share of total employment across multiple industries. Concurrently, the issue related to economic sustainability and the adoption of automation and AI technologies is in many ways directly related to the balance between job creation and elimination, a phenomenon which will be discussed more thoroughly in chapter 6.2 AI, Automation and Ethics.
As for social sustainability, which potentially serve as the most diffuse dimension amongst the three, this specific study mostly related to the explicit people affected by the adopted technologies. To begin with, one specific sustainability-related risk is the emergence of automated bias. Such bias can occur when the model learns to identify patterns in data and subsequently makes recommendations or predictions on, for instance, gender, age or ethnicity. This type of bias can occur in multiple stages of how automation and AI technologies are utilized within organization. Moreover, the bias can occur already in the training data, in which the data that is collected is unrepresentative of reality or simply reflects an existing prejudice. Within an L&D context, this could, for instance, occur when an AI system recommends women to take a specific program simply because it was trained on historical decisions which favored men over women and hence learned to make the same decision. Interestingly, Google AI have developed a set of practices in order to build fairness into AI systems. In direct relation to the previously mentioned problem of bias built in the raw data, Google AI suggests that organizations should simply examine their raw data carefully. Some aspects worth mentioning in terms analyzing raw data in order to avoiding future biases by the AI system are (Google, 2020):
• Does the data contain any mistakes (e.g. incorrect or missing values)? • Is the data sampled in a way that represents users appropriately (e.g. will be used for both women and men, but the AI system only has access to training data from male users) and the real-world setting (e.g. will be user year-round, but the AI system only has access to training data from the winter)? • Is the relationship between the data labels problematic for the specific item you are trying to predict (e.g. if you are using data label X as a proxy to predict a label Y, in which cases might a gap between X and Y be problematic)?
68 6. Discussion and Conclusion
Moreover, all of these are examples mentioned above are immensely important aspects for organizations to consider before the adoption of automation and AI technologies in order to minimize a potentially negative impact in terms of social sustainability.
Lastly, in regard to environmental sustainability, which may naturally seem as a distant topic in relation to this specific research study, there are several aspects which one need to consider. Goodland (1995) defines environmental sustainability as: “seeks to improve human welfare by protecting the sources of raw materials used for human need and ensuring that the sinks for human wastes are not exceeded, in order to prevent harm from humans”. Furthermore, this definition allows for several dimensions to be discussed in terms of how automation, AI, Corporate Learning and environmental sustainability interact. To being with, automation and AI technologies require resources in terms of raw materials in order to build computers systems to deploy these technologies, which naturally becomes an environmental aspect in need of consideration. Another important aspect to consider in regard to environmental impacts of adopting automation and AI are the storage of data and, more specifically, data which is stored on cloud platforms. In order to continuously keep these platforms running and maintain their availability for the automation and AI systems, vast amount of energy is required. In his report The Cloud Beings with Coal, Mills (2013) explains that, as opposed to hard drive storage, energy consumption increases with the amount of data. Naturally, this is a critical aspect to consider for any organization utilizing cloud services to power automation and AI technologies, given the fact that performance is directly correlated with the amount of accessible data.
6.5 AI, Automation and Ethics Automation and AI is seen by many as a highly transformative technology, with the financial services industry being no exception. However, it is similarly immensely important to think beyond the purely functional capabilities of such technologies and also emphasize the ethical aspects behind creating and deploying technologies of such transformative and life-consequential power.
A natural reaction for many upon even discussing automation and AI technologies is that they will replace workers across an array of industries. As a result, automation and AI usually result in mixed emotions and opinions for employees in the context of job creation and most commonly, job elimination. However, many scholars are beginning to point towards the fact that AI and automation is not necessarily a killer of job, but rather, a killer of certain job categories.
69 6. Discussion and Conclusion
This means that jobs are not in fact destroyed, instead employment shifts from one place to another, thus resulting in the creation of entirely new categories of employment. Moreover, a significant portion of this study aims to capture this transitional phenomenon which will take place on a wide array of organizations in the near future. As the empirical evidence revealed that a noticeable amount of administrative and manual work is included within the L&D organization of financial services companies. Furthermore, taking into consideration the vast capabilities of automation and AI described throughout this research, such technologies possesses the opportunity to allow employees to focus on high- value activities, as opposed to spending their days manually entering data. Thus, adopting automation and AI would instead allow them to focus on more strategic elements of learning and development, which is an aspect consistently highlighted throughout this research study.
6.6 Future Research Based on the conclusion that the SFSA plays an essential role in how the L&D organizations of the financial services industry function, investigating how instead they could utilize automation and AI to empower their decision-making process when communicating training initiatives to financial services companies. From a holistic perspective, the SFSA is simply a component of the existing L&D ecosystem in which financial services companies exist. Moreover, the SFSA were in fact contacted to participate in this research study but could however not contribute due to limitations in regard to information sharing. Additionally, as described earlier, the research was heavily reliant on qualitative data sources and could thus use additional quantitative measures which subsequently could strengthen the research triangulation. In an ideal scenario, an actual implementation process of automation and AI technologies within L&D would take place. This would furthermore provide additional insights in terms of the subsequent benefits of adoption of such technologies. This could, for instance, include aspects of time saved for employees in training, changes in completion rates or simply how learners navigate through the content. However, this would also extend the current research scope which emphasizes an exploratory approach and instead most likely investigate the potential benefits of adopting such technologies to optimize corporate learning. Furthermore, considering that the scope of the research emphasized a more generalized approach to learning, another potentially interesting area of future research would be to investigate which specific learning types that would be suitable to augment with a specific technology. Given that, for instance, an employee might learn compliance and function- specific content very differently, an interesting area to explore would be to investigate how well certain technologies are aligned with the specific type of learning.
70 Appendix A: Interview Transcript
Appendix A: Interview Transcript
Question 1. Could you describe your role within Case Company X?
Question 2. What is Case Company X’s current approach and vision of L&D (e.g. what makes an L&D initiative successful)?
Question 3. What does the decision-making process look like for initiating a training program (e.g. who decides when a new training should be developed and what it should contain)?
Question 4. Which stakeholders within the organizations interact with an employee’s training?
Question 5. How do you ensure that each employee receives the right training?
Question 6. How do you create and manage content?
Question 7. How do you deploy the content?
Question 8. What would you consider being the most training- and/or information intensive role within Case Company X?
Question 8. Is the process of corporate learning different for different roles? If so, how is training conducted for the following roles?
I. Compliance II. Operations III. Retail Banking
Question 9. How is a new employee integrated with the current training process and how do you manage if an employee changes their role?
Question 10. Are managers able to follow the training ?
Question 11. Do you integrate RPA and AI technologies within your L&D today? If so, how are such technologies integrated?
71 Appendix A: Interview Transcript
Question 12. If not, is integrating RPA and AI technologies part of the strategic agenda within your L&D organization?
Question 13. If so, what was the biggest challenge in terms of adopting a specific automation and AI technology?
72 Appendix B: Survey Transcript
Appendix B: Survey Transcript
Question 1. Are you using in-house-built software within your L&D organization?
Question 2. If so, to what extent do you agree with the following statements concerning that in-house-build software? From 1 (do not agree at all ) to 5 (fully agree).
I. In-house-built software creates a technical rigidity in which it makes the organization inflexible towards adopting and potentially integrating complementary technologies. II. In-house-built software creates flexibility in the sense that it enables quick customization III. In-house-built software requires significant investments. IV. Maintaining a desired level of quality of the in-house-built software can be challenging as a result of lacking technical expertise. V. Maintaining a desired level of quality of the in-house-built software can be challenging as a staff dismissal.
Question 3. What was your most recent external software adoption within your L&D organization?
Question 4. Looking at your most recent external software adoption within L&D, how long did the implementation process take (from initial contact with the software provider to a deployed solution).
73 References
References
Aalst, P van der, M Bichler, and A Heinzl (2018). Robotic Process Automation. Business Information Systems Engineering.
Accenture, (2018). Making it personal. Accenture Interactive.
Akintoye, A. (2015). Developing Theoretical and Conceptual Frameworks. The Academy of Management Journal.
Blankenship, L. (2018). How L&D Initiatives Encourage Creativity in the Workplace. HR Technologist. Available at: https://www.hrtechnologist.com/articles/learning-development/how-ld-initiatives- encourage-creativity-in-the-workplace/ [Accessed 2nd Feb 2020]
Berruti, F., Nixon, G., Taglioni, G. and Whiteman, R. (2017). Intelligent process automation: The engine at the core of the next-generation operating model. McKinsey Digital. Available at: https://www.mckinsey.com/business-functions/mckinsey-digital/our- insights/intelligent-process-automation-the-engine-at-the-core-of-the-next-generation- operating-model [Accessed 4th March 2020]
Blomkvist, P. & Hallin, A., (2018). Method for Technology Students: Degree Project Using the 4- Phase Model. First Edition. Lund Studentlitteratur
Brusilovsky, P. and Peylo, C. (2003). Adaptive and Intelligent Web-based Educational Systems. International Journal of Artificial Intelligence in Education.
Brusilovsky, P. (1999). Adaptive and Intelligent Technologies for Web-based Education. Künstliche Intelligenz.
Bughin, J., Hazan, E., Ramaswamy, S., Chui, M., Allas, T., Dahlström, P., Henke, N. & Trench, M., (2017). Artificial intelligence the next digital frontier?. McKinsey Global Institute.
Burgess, A. (2018). The executive guide to artificial intelligence. The Academy of Management Journal.
74 References
Camp, W. G. (2001). Formulating and Evaluating Theoretical Frameworks for Career and Technical Education Research. Journal of Vocational Educational Research
Cleave, J. (2020). Tailoring Corporate Learning for Each Learner. The Academy of Management Review.
Crocetti, C. (2012). Corporate learning: A knowledge management perspective. The Internet and Higher Education.
Dudovskiy, J. (2016). Writing a dissertation. Research Methodology
Eisenhardt, K and M Graebner (2007). Theory Building from Cases: Opportunities and Challenges. The Academy of Management Journal.
Farrell, L. (2019). Learning and Development in Financial Services: Creating Impactful Corporate Training. Synapse
Fiol, C. and Lyles, M., (2005). Organizational Learning. The Academy of Management Review.
Fung, H (2014). Criteria, Use Cases and Effects of Information Technology Process Automation (ITPA). Advances in Robotics & Automation.
Google. 2020. Responsible AI Practices. Available at: https://ai.google/responsibilities/responsible-ai-practices/ [Accessed May 2nd 2020]
Greener, S., 2008. Business research methods. 1st ed. London: BookBoon.
Halal, W.E. (2003). Artificial intelligence is almost here. On the Horizon.
Hudson, D and M Cohen (2012). Neural Networks and Artificial Intelligence for Biomedical Engineering. Wiley.
Hunt, N. (2015). The Netflix Recommender System: Algorithms, Business Value, and Innovation. ACM Transactions on Management Information Systems.
Inkpen, Andrew & Crossan, Mary. (2007). Believing Is Seeing: Joint Ventures and Organization Learning. Journal of Management Studies.
75 References
Jacqueline, B. Christensen, L. van Dam, N. (2019). The Essential Components of an L&D Strategy. McKinsey Digital. Available at: https://www.mckinsey.com/business-functions/organization/our-insights/the- essential-components-of-a-successful-l-and-d-strategy [Accessed 11th Feb 2020]
J. Collis and R. Hussey. (2012). Business Research: A Practical Guide for Undergraduate and Postgraduate Students. Palgrave Macmillan,
Kim, D.H. (1993) The Link between Individual and Organizational Learning. Sloan Management Review
Lacity, M and L Willcocks (2016). A new approach to automating services. Sloan Management Review Fall.
Levitt, B. and March, J., (1988). Organizational learning. Annual Review of Sociology
Linkedin. (2019). Annual Workplace Learning Report. Available at: https://learning.linkedin.com/content/dam/me/business/en-us/amp/learning- solutions/images/workplace-learning-report-2019/pdf/workplace-learning-report- 2019.pdf [Accessed 20th Jan 2020]
Lhuer, X. (2016). The next acronym you need to know about: RPA. McKinsey Digital. Available at: https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/the- next-acronym-you-need-to-know-about-rpa. [Accessed 1st Feb 2020]
Mangalindan, J. (2012). Amazon's Recommendation Secret. Fortune Magazine. Available at: https://fortune.com/2012/07/30/amazons-recommendation-secret/ [Accessed 15th Feb 2020]
Marr, B., (2016). What Is The Difference Between Artificial Intelligence And Machine Learning? Forbes.
Michael Gibbert, Winfried Ruigrok, and Barbara Wicki. (2015). What passes as a rigorous case study? Strategic Management Journal.
76 References
Mills, M. (2013). The Cloud Begins with Coal: Big Data, Big Networks, Big Infrastructure, and Big Power. Digital Power Group.
Mohanty, S and S Vyas. (2018). How to compete in the age of artificial intelligence. Apress.
N. Pankaja. (2015). A comparative study of popular online platforms for E learning solutions. Educational Media Research Center, Univ. of Mysore, Mysore
Peshkin, A. (1993). The Goodness of Qualitative Research. Educational Researcher.
Ridder, H-G., Hoon, C. and McCandless Baluch, A. (2014) Entering a dialogue: Positioning case study findings towards theory. British Journal of Management
Sana Labs AB (2020). Discussions with CEO Joel Hellermark and Business Development Manager Samuel Björklund. Stockholm.
Schatsky, D., Muraskin, C., and Iyengar, K. (2016). Robotic Process Automation. Deloitte University Press
Simon, Herbert A. (1991). Organizations and Markets. Journal of Economic Perspectives
Stata, R. (1989). Organizational Learning — The Key to Management Innovation. Sloan Management Review.
The International Monetary Fund. (2020). Financial Services: Getting the Goods. Available at: https://www.imf.org/external/pubs/ft/fandd/basics/finserv.htm [Accessed 26th Feb 2020]
Torres, R. (2020). To succeed at RPA adoption, don’t automate inefficient processes. CIO Dive. Available at: https://www.ciodive.com/news/to-succeed-at-rpa-adoption-dont-automate-inefficient- processes/571584/?linkId=100000010614548 [Accessed 14th Feb 2020]
United Nations, 1987. General Assembly Resolution on Implementation of Agenda 21, the Programme for the Further Implementation of Agenda 21 and the outcomes of the World Summit on Sustainable Development. (A/RES/64/236)
77 TRITA ITM-EX 2020:233
www.kth.se