Open Research Online The Open University’s repository of research publications and other research outputs

Beyond prototypes: Enabling innovation in technology-enhanced learning

Other

How to cite:

Scanlon, Eileen; Sharples, Mike; Fenton-O’Creevy, Mark; Fleck, James; Cooban, Caroline; Ferguson, Rebecca; Cross, Simon and Waterhouse, Peter (2013). Beyond prototypes: Enabling innovation in technology-enhanced learning. Open University, Milton Keynes.

For guidance on citations see FAQs.

c 2013 The Authors

Version: Version of Record

Link(s) to article on publisher’s website: http://beyondprototypes.com

Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyright owners. For more information on Open Research Online’s data policy on reuse of materials please consult the policies page. oro.open.ac.uk TEL Technology Enhanced Learning Research Programme tel.ac.uk

BEYOND PROTOTYPES Enabling innovation in technology-enhanced learning TEL Technology Enhanced Learning Research Programme tel.ac.uk beyondprototypes.com CONTENTS

Executive summary 05 1 Introduction 09 2 Methods 10 3 What is TEL? 12 Case study: Microelectronics Education Programme 13 3.1 Understanding education 14 3.2 Education as a super-stable system 15 4 Achievements underpinned by TEL 17 UK centres of excellence in TEL 18 4.1 Success story: mobile learning 19 4.2 Success story: Scratch 19 4.3 Success story: xDelia 20 5 TEL innovation: challenges and misconceptions 22 5.1 Shift the focus from technology to pedagogy and practice 23 5.2 Look beyond the formal education sector 23 5.3 Widen the TEL community 24 5.4 Connect TEL research and practice 25 5.5 Find new ways to assess the contribution of TEL 25 5.6 TEL success is not necessarily commercial success 26 Case study: Cognitive Tutor software 27 6 The process of TEL innovation 28 6.1 The TEL Complex 28 Case study: Yoza Cellphone Stories 30 6.2 Configurational innovation and bricolage 30 6.2.1 Configurational innovation 30 6.2.2 Social practices as part of the TEL complex 31 6.2.3 Bricolage 31 6.2.4 Bricolage and ‘persistent intent’ within the TEL space 32 Case study: iZone Driver Performance 32 6.3 The TEL Innovation Process 33 6.3.1 Vision of educational change 33 6.3.2 Pedagogical research and expertise 33 6.3.3 Developing practices in parallel to formal education 34 6.3.4 Co-constructing new practices 34 6.3.5 Methodology of Design-Based Research 35 7 Implications for research 36 7.1 Recommendations for researchers 37 8 Implications for policy and for funding 38 8.1 Recommendations for funders and policy makers 39 9 Challenges and opportunities 40 Bibliography 42 Acknowledgments 46 Research team

tel.ac.uk 3 THE TEL RESEARCH PROGRAMME (TEL)

The TEL research programme, which ran from 2007 to 2013, has generated some substantial gains in our understanding of how to design and deploy technologies for learning.

These findings, together with the growing field of technology-enhanced learning internationally, are witnessing the growth of TEL research into a vibrant academic field, extending throughout the UK and beyond. Yet there is a surprising failure to translate the findings, prototypes and outputs of projects into commercial products and services that individually and collectively achieve radical change in the quality of teaching and learning.

This difficulty seems part of a general problem of translating innovation in the laboratory (or classroom or school) into commercial gain:

A key recurring issue that has been raised in the Science and Technology Committee’s previous inquiries is the difficulty of translating research into commercial application, particularly the lack of funding—the so-called “valley of death”. (Commons Select Cttee, 20111).

The field of Technology Enhanced Learning, despite some notable exceptions, is rife with results that never made it across the valley of death. In the TEL research programme, there were some exciting and innovative examples of working prototypes that solved significant research problems. Yet few of these projects have successfully taken their prototypes to market. Three of the eight funded TEL projects achieved success in gaining follow-on funding from the ESRC specifically earmarked for the achievement of “impact”, although it is too soon to know if and how such impact will be achieved, and more generally, the relationship between impact and the commercial exploitation of projects’ outputs. In general, despite the fact that all projects successfully designed and built effective prototypes of systems: the question is how to move from prototype to product.

This report addresses this issue head-on from an interdisciplinary perspective that brings together experts in diverse relevant fields including educational technology, organizational behavior, innovation dynamics. I am grateful to the authors and all those who gave their time to help clarify these difficult and important issues.

Prof Richard Noss Director, Technology Enhanced Learning Research Programme. London Knowledge Lab | Institute of Education | University of London

1http://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and- technology-committee/news/111215-new-inquiry---bridging/

4 Beyond Prototypes: Enabling innovation in technology-enhanced learning EXECUTIVE SUMMARY

The Beyond Prototypes report provides an in-depth examination of the processes of innovation in technology-enhanced learning (TEL).

It sets out what can be done to in the report, links each with improve the process of moving from recommendations to enable academic research and innovative successful TEL innovation and, finally, prototypes to effective and outlines the structure of the report. sustainable products and practices. In doing so, it shows that technological development is only a small part of Key insight 1: The TEL Complex the picture. Significant and lasting Technology-enhanced learning TEL innovation requires long-term consists of much more than a set shifts in practice. These shifts are of research-informed products. It not confined to the classroom or is a complex system, which includes training environment; they require communities, technologies and alterations to many different practices that are informed by elements of the education system. In pedagogy (the theory and practice order to make these shifts, different of teaching, learning and assessment). communities and groups need to The many elements of the ‘TEL work creatively together over time, Complex’ must all be taken into so policymakers and funders should account as an innovation is designed, plan for engagement with teams developed and embedded. At the able to initiate, implement, scale and heart of the TEL Complex is a vision of sustain long-term innovation. how learning may be enhanced by the use of technology. This vision requires An expert multidisciplinary team the development or adaptation of carried out the research underpinning technology over time in order to this report. Initial analysis of the support a pedagogical approach. field of TEL research, development and policy was used to select key Concurrent implementation of examples of TEL innovation for pedagogy and technology requires detailed study. Innovation was taken consideration of current practices, to be the practical implementation including the activities and of new ideas and technologies with expectations of learners and teachers. the intention of having an observable Less obviously, implementation must impact on teaching and/or learning. also take into account practices related The initial phase included systematic to areas as diverse as local and national analysis of data collected from in- assessment criteria, health and safety, depth interviews with key figures from staff training, administration and research and industry. Each member of provision of technical support. In order the research team brought substantial to understand these, researchers need personal expertise to the research to consider the ecology and technical process, enabling them to set the context into which an innovation is findings within a broader context. This to be introduced. To do this, they was a strength of the study, allowing must be able to work effectively team members to link their analysis with diverse other communities, not only to the field of educational particularly learners, teachers and technology but also to understandings technology developers. developed in the fields of organisation behaviour and innovation dynamics. This innovation work takes place in a wider setting that includes the This executive summary introduces local, national and international the four key insights described environment. Research and education

tel.ac.uk 5

into products and practices over an extended period of time. This Look beyond product development and pay close attention to requires both long-term commitment and focused action. Teams of the entire process of implementation. researchers need persistent intent in order to develop their work over time with a shared educational goal are carried out within a context that in mind. Many different academic and is shaped by regional and national business-based research projects may policy, by funding constraints, and be aligned in order to work towards by the need to generate sufficient the same educational goal. revenue to ensure that an innovation is sustainable even when short-term Persistent intent motivates funding has run out. A successful researchers to work closely with the process of innovation pays attention communities that will be involved to these high-level issues as well as in implementation, developing a to issues that can be shaped and shared vision that is owned not only controlled locally. by the project team, but also by those who will take it forward once Summary: TEL involves a complex the research programme is complete system of technologies and practices. and the development team has left. In order to embed significant TEL To carry out this work successfully, innovation successfully, it is necessary researchers need opportunities to to look beyond product development develop the skills that will enable and pay close attention to the entire them to bridge the gaps between process of implementation. those different groups.

Recommendations: Policy and Summary: Significant innovations funding should support changes are developed and embedded over in pedagogy and practice, as well as periods of years rather than months. the technological developments that Sustainable change is not a simple will support these. Project teams matter of product development, should be encouraged to identify testing and roll-out. the elements that must be taken into account in order to enable sustainable Recommendations: Policy and implementation of an idea or funding should take into account prototype in the context of a vision the need for extended development. of the enhancement of learning. In There should be capacity to support order to do this, researchers need individuals and teams to engage to engage with the individuals and in long-term projects capable of communities that will play a role in turning inspirational ideas into fully the implementation process. embedded products and practices. Researchers and developers should be encouraged to plan for Key insight 2: Persistent intent sustainability. The implementation The diverse nature of the TEL and success of plans for sustainability Complex means that a linear model of should be evaluated. innovation in which research findings are applied and then adopted is rarely appropriate within the education Key insight 3: Bricolage system. Success in TEL is associated The work involved in successful with ‘persistent intent’ – efforts by a TEL innovation can be characterised group of vision-enactors to develop as ‘bricolage’. This is a productive inspirational ideas and turn them and creative innovation process

6 Beyond Prototypes: Enabling innovation in technology-enhanced learning

that involves bringing together Summary: TEL innovation is a courses through online learning to and adapting technologies and process of bricolage. This process people who are unable to study at pedagogies, experimentation to includes informed and directed a conventional university). generate further insights and a exploration of the technologies willingness to engage with local and practices required to achieve We need new ways of judging communities and practices. an educational goal. It involves whether an innovation is successful Bricoleurs do not start a project experimentation to generate fresh in enhancing learning. Traditional and then consider which tools and insights, and a creative use of scientific methods developed for materials will be required to achieve available resources. It also requires the laboratory, such as randomised their goals. Instead, they review engagement with a range of controlled trials, have an important their available tools and resources communities and practices. role to play in revealing the changes and work out how to use them to produced by altering individual achieve their goal or something Recommendations: Policy variables, but are not adequate to deal close to their goal. and funding should encourage with situations in which everything the development of skilled, has changed because a process of Bricolage is a practical process of multidisciplinary teams that are able bricolage has engaged the entire TEL innovation. It may be informed by to complete the TEL innovation Complex. Quantitative measures can a deep understanding of theory and process. Policy and funding should signal whether learners know more underpinning research, but does not also support the experimentation things, but are less useful in assessing rely solely on a theoretical model that is necessary to generate fresh whether their understanding has of what should work. It engages insights and achievable visions of been deepened or whether they have with relevant communities to ensure educational developments. acquired the skills to learn more or that innovation works in practice and in context. Bricoleurs pay attention to the restrictions and constraints of a situation, and take Methods of evaluation are required that can be applied to steps to overcome or compensate for them. Through creative processes of innovation and to institutional change. reinterpretation and arrangement of local practices and resources, they can enable new possibilities. Key insight 4: Evidence to work independently in the future. Development and implementation Without valid methods of identifying Successful TEL innovators, both of new approaches to teaching and success, evidence-based innovation in academia and in business, are learning must be trialed and tested cannot take place. bricoleurs who achieve educational so that widespread adoption of TEL goals by bringing together pedagogic innovation is based on evidence and Summary: Successful implementation approaches, diverse technological not on theory alone. This evidence of TEL innovation requires evidence elements, frameworks and social could come from comparative that the projected educational goal practices. If TEL innovation is to be trials in classrooms or in training has been achieved. Reliable evaluations embedded successfully, bricoleurs environments, judging success by must be carried out; their findings need to be able to understand and improvement in test scores. It could must be disseminated and acted on. take into account the perspectives come from newly emerging sources, Methods of evaluation are required of different stakeholders, and to such as the learning analytics and that can be applied to processes of build links between the experiences comparative (A/B) testing that are innovation and to institutional change, and knowledge of different enabled by the increasing amounts as well as those that can be applied to communities. Inter-disciplinary of data generated by virtual learning shifts in technology usage. collaboration will involve education environments, MOOCs and other theorists, policy makers, software educational technologies. Or Recommendations: Policy and technology developers, teachers, it could arise from educational and funding should require the learners and other practitioners, transformations, opportunities evaluation of TEL innovations all with a shared intent to move to teach or learn in ways that in terms of their educational ideas across boundaries and to are simply not possible without impact. The findings of these explore new approaches to learning technology (such as distance evaluations should be available to and teaching. education providers offering degree other researchers and developers,

tel.ac.uk 7

including those without access to which successful innovation requires Section 7 examines the implications university libraries. New methods attention to a range of different of this report for research. Persistent of evaluation, such as the use of pressures and practices. intent, engagement over time and the learning analytics or comparative use of an appropriate methodology A/B testing where appropriate, Section 4 provides an overview of are identified as priorities. Successful should be developed and put TEL successes, pointing to the UK’s research also requires engaging into practice. role as a world leader in this area with the practices and stakeholders and identifying TEL’s role in that must be taken into account The key insights of this report developments as diverse as the if research-informed innovation is indicate that changes are necessary World Wide Web and the iPad. Three to be embedded in practice. The to the ways in which TEL innovation is areas of success – the field of mobile section ends by considering the planned for, funded and implemented. learning, the development of the implications for researchers of the The continuing need is not for Scratch educational programming recommendations identified in this abstract grand challenges or short- language, and the xDelia project that Executive Summary. term initiatives, but for a sustained developed learning applications for building of capacity in technology- financial traders – are considered Section 8 examines the implications enhanced learning, through graduate in detail. The section also identifies of this report for policy and programmes and investment in reasons why TEL successes may go for funding, identifying current national hubs of expertise that share unnoticed, including the significant problems, particularly in relation talent and facilities. timescales involved. to sustainability, and proposing

Organisation of the report The key insights and recommendations The continuing need is not for abstract grand challenges or set out in this Executive Summary are developed in more detail in the main short-term initiatives, but for a sustained building of capacity body of the report. This is structured in technology-enhanced learning. as follows.

Following the Executive Summary and Introduction: Section 5 deals with challenges to solutions. The section ends by TEL research and innovation. Six considering the implications for Section 2 describes the study on misconceptions about these are funders and for policy makers of the which this report is based. In the examined, and recommendations recommendations identified in this first phase, close to 100 projects, that will increase the potential of Executive Summary. products and programmes were TEL to achieve widespread impact considered. Desk research, case study are identified. These show that Section 9 focuses on the way forward and interviews were used to examine funders, researchers and policy for TEL research. It identifies a a selection of these in depth. In the makers all have a role to play in continuing need for sustained building second research phase, project team achieving that impact. of capacity in TEL, through graduate members, with a range of expertise programmes and investment in in different fields, worked together to Section 6 sets out key contributions national hubs of expertise that share analyse the data, develop insights and of this report. TEL should be talent and facilities. write the report. considered as a technology complex, made up of a series of interconnected Boxed case studies provide examples Section 3 defines technology- elements that cannot be changed of TEL innovation in practice. ● enhanced learning, tracing its in isolation. A model of the TEL origins, identifying its main areas of Complex is set out, centred on a focus, linking it to related fields and vision of educational change The TEL identifying key past policy initiatives. Innovation Process is also modelled, The section goes on to explore and bricolage is set at its heart. different conceptions of education, Design-based research is identified introducing the idea that education as a core methodology to support is a super-stable system within evidence-driven innovation.

8 Beyond Prototypes: Enabling innovation in technology-enhanced learning 1 INTRODUCTION

In 2007, the Economic and Social Research Council and the Engineering and Physical Sciences Research Council jointly funded a national Technology Enhanced Learning (TEL) Research Programme.

That programme supported eight within UK and international contexts, In order to address this problem, large interdisciplinary projects to (2) Perspectives from different this report examines effective ways demonstrate how learning can be sectors, (3) Audit of UK examples of establishing connections between transformed through innovative use commercialised internationally, (4) research and successful adoption of digital technology, ranging from Assistance to market, (5) Examples of of TEL. It proposes a model of TEL developing the social skills of autistic success and (6) Relationship between that extends beyond an initial period children to training dentists through impact and commercial exploitation. of research and development and it haptic (sense-of-touch) simulators. connects this model to a process of Three of these projects have received One puzzle to be explored is that TEL innovation (see Section 6). ● further funding to embed the findings successive decades have seen and technology into educational major innovations in TEL that have practice. promised to transform education. These innovations have included As a final initiative of the TEL educational television in the 1960s, Research Programme, this report language labs in the 1970s, computer- was commissioned to explore based instruction in the 1980s, the broader issues of translating integrated learning systems in the innovation from early prototypes to 1990s, virtual worlds for learning in sustained impact. Such impact may the 2000s, and Massive Open Online come through widespread adoption Courses (MOOCs) in the 2010s. Some of new technology-enabled methods of these have received extensive press within the education sector, through coverage. Yet for none of these has successful educational products, the initial roll out been underpinned or through new activities in homes, by extensive research, nor have any museums and outdoor environments been subject to systematic trials of that involve learning with technology. the kind that might be carried out The remit of the report is to indicate before the introduction of a new drug the barriers to impact of innovations into medical practice. in technology-enhanced learning and to propose new routes to achieving Most of these innovations have faded large-scale sustained transformations from national attention. Some, such in teaching, learning and assessment as language labs and virtual worlds, that benefit society. have been adopted on a small scale within formal educational settings. The report addresses the question: MOOCs are still in the headlines; few are underpinned by research, and What should researchers, funders providers are currently looking for and policy makers do to improve ways of assessing the educational the translation from innovative impact of various formats. Meanwhile, prototypes to effective and schools, universities, businesses and sustainable products and practices? individual learners have gradually adopted digital technologies for a In order to provide answers, the wide range of educational purposes, study on which this report is based ranging from note taking to online considered six facets of the general assessment. TEL researchers are theme of moving beyond prototypes. often left playing catch-up, assessing These can be summarised as: (1) the effectiveness of technologies that Challenges of commercialisation have already been rolled out at scale.

tel.ac.uk 9 2 METHODS

Section 2 describes the study on which this report is based. Close to 100 projects, products and programmes were considered. Desk research, case studies and interviews were used to examine a selection of these in depth. Project team members, with a range of expertise in different fields, worked together to analyse the data, develop insights and write the report.

The findings reported here are each innovation was considered based on a systematic analysis disruptive or in accord with current of data collected from in-depth educational practice; and possible interviews with key opinion leaders, actions to improve impact of TEL plus selected exemplar cases, research and development. published reports and commentary. To this process was brought the During the course of the study, substantial personal expertise of close to 100 projects, products individual members of the project and programmes in the area of TEL team. The team included experts were identified. The project experts with extensive experience in assessed each of these potential pedagogy and technology gained case studies in terms of how well both inside and outside the university it aligned with the six facets of the sector. These included professors project. They were also assessed of organisational behaviour and in terms of the required level, innovation dynamics as well as nature and ease of access to professors of open education relevant data. A full list of cases and educational technology. This considered in the selection and combination meant that the analysis research process is available on could draw on and combine insights the project website (see Project from these different fields. The report website box). Approximately half is therefore not narrowly focused the identified cases were selected on educational technology, but for further consideration by initial places this in the wider context of desk research and, in the light of understandings of the different types this work, eight were selected for of innovation that are employed in focused case study. This approach market-oriented organisations. created a tapestry of data, at varying levels of granularity, relating to An additional strength of this study a wide selection of UK and has been the sustained involvement international TEL initiatives. of project team experts at all stages, from research design to data analysis. A series of interviews complemented This has made it possible to review the desk research. These helped to the data critically through multiple identify and explore issues in a more interpretative lenses and to adopt nuanced way than would otherwise a constructivist grounded theory have been possible, and provided approach [1] to identifying and insight from a range of senior understanding emergent themes. perspectives. Interviewees included international applied researchers The first phase of the research as well as directors or senior determined an initial list of managers of research institutes, interviewees and cases, and defined public organisations and private research criteria for the interview and companies. In all, 14 semi-structured case exemplars. Of particular interest interviews were carried out, each were the impacts, success factors and based on a protocol developed jointly issues encountered in implementing by the project team. Interviewees’ TEL innovation; the degree to which consent was obtained to record and

10 Beyond Prototypes: Enabling innovation in technology-enhanced learning 2 METHODS

use their data in the research and, Project website where possible, interviewees were provided with a sample list of Full details of the Beyond Protoypes study are available on the questions prior to interview. The project website http://beyondprototypes.com/, including the project website gives details of the methodologies for data collection and analysis, an overview interviewees and of the protocol of case studies considered, and some in-depth case studies used. Interviews were recorded and transcribed, and the interviewer checked these transcriptions. often lacked sufficient information In addition, previously published about non-academic project presentations, commentaries and implementation and about the media interviews were used to impact and exploitation of the work provide contextual information. during the lifetime of the project and after its formal conclusion. In The second phase of the study some cases, it was difficult to find involved an iterative weekly cycle links to final or impact reports of data collection and analysis that produced by funded projects. Desk spanned a two-month period. This research therefore highlighted the allowed theoretical sampling to be need for more information about used to augment the list of cases the actual, and often challenging, and interviewees as themes began practice and experience of to emerge. Each week, every member implementing research and achieving of the project team was required to impact. It also drew attention to code and identify key themes within the lack of clarity demonstrated a selection of in-depth interview by many projects about plans for transcripts and data from case exploitation and, specifically, the research [2, 3]. After the weekly reasoning and evidence for claims coding had taken place, the team such as expecting ‘large-scale use met to compare the coded data, of project results’. This initial finding agree on emerging themes, and has been taken into account in this compare their analysis with that report’s recommendations related of previous weeks. This process to evidence. provided scope for the expert team members to analyse the data from The theory development phase the perspective of their field of of the analysis initially focused on expertise. Emerging themes were creating a broad reporting narrative recorded as memos on the raw through which to present the interview transcripts and case study themes identified. It then went on to documentation, as well as in the develop a model for the innovation minutes of weekly meetings, and in process. As with earlier phases, post-analysis interviews conducted existing theory and concepts with each team member. In addition, were considered together with the narrative thematic analysis [4] was themes emerging from the research. undertaken on a selection of TEL The breadth of the study’s scope research linked to the UK Teaching meant that theoretical saturation and Learning Research Programme was not expected across categories. that gave rise to this report. Where Nevertheless, a high degree of possible, project impact reports were confluence was observed across examined, in order to explore the interviews, indicating that measures of impact reported by appropriate emergent themes each project. had been identified. These themes, which are explored in Section 6, During analysis, it was found that were the TEL Complex, persistent publicly available documentation intent, bricolage and evidence. ●

tel.ac.uk 11 3 WHAT IS TEL?

Section 3 defines technology-enhanced learning, tracing its origins, identifying its main areas of focus, linking it to related fields and identifying key past policy initiatives. The section goes on to explore different conceptions of education, introducing the idea that education is a super- stable system in which successful innovation requires attention to a range of different pressures and practices.

Technology-enhanced learning reference for the research community (TEL) research focuses on how working in this area. The term is technologies can add value to more generous and encompassing learning and teaching processes. of new practices than the wide Today’s learners have access to range of related labels, including increasingly powerful and affordable ‘educational technology’, ‘computer- handheld computing devices, aided learning’, ‘Information and including smartphones, games Communication Technologies’ (or consoles and tablet computers. ‘ICT’, as they are often referred to in They can share, interact and immerse the schools sector), and ‘e-learning’, themselves online with others to name but a few. ‘Technology- through the use of social networks enhanced learning’ stresses that the and virtual worlds. They can also technology is employed in service create identities and user-generated of the learning, and that it is not resources that potentially have a just adopted, but is expected to virtual worldwide audience enabled deliver improvement. by the Internet. Learners’ activities can be captured in real time and References to TEL, in relation to feedback processes automated support and training, began to with increasing precision through emerge in the 1990s and the first learning analytics. Technologies that TEL conference appears to have allow users to post material and taken place at the end of that messages online have the potential decade. TEL-isphere 1999, held in to support learner inquiry, to offer Barbados, focused ‘on the use of new modes of representation and communications and information expression requiring new forms technologies, and their potential of literacy, to support innovative to enhance learning experiences thinking and problem solving through helping students become through collaboration, and to allow active participants in the educational publication of work to an authentic process’ [7]. external audience [5]. Even before the emergence of TEL is able to make use of different TEL as a named field of research, forms and formats of technology in UK government policy was driving the pursuit of more engaging and innovation in this area. The Beyond beneficial forms of teaching, learning, Prototypes website includes an pedagogy and assessment. As this overview of relevant policies in the report highlights, good pedagogic UK over the past decades, and a intentions lie behind some of this selection of those from Scotland, development but, ‘many important which has developed distinctive TEL developments have often come policies of its own. The website also from innovating with technologies contains a detailed version of the developed for other purposes’ [6]. boxed case study included here, which deals with the implementation Technology-enhanced learning has of the Microelectronics Education emerged as a preferred term of Project, providing an early example

12 Beyond Prototypes: Enabling innovation in technology-enhanced learning 3 WHAT IS TEL?

of government policy driving a well- Case study: Microelectronics Education Programme structured process of TEL innovation. The £23 million Microelectronics Education Programme (MEP) In 2001, the final report of the for England, Wales and Northern Ireland was established by the European Commission’s Open Government in November 1980 and ran for six years. The aim was Consultation Process on ‘New to support schools in preparing children ‘for life in a society in which Research Challenges for Technology devices and systems based on microelectronics are commonplace Supported Learning’ clearly set out a and pervasive’. To complement this work, the Department of Industry series of recommendations designed made £16 million available to help local education authorities purchase to further the research agenda for computers for schools. technology-enhanced learning. These were to be carried out as part of the MEP took into account areas as diverse as curriculum development, Sixth Framework Programme (FP6) teacher training, resource organisation and support. It promoted [10]. The report offered a response to change at national, regional and local levels, encouraging the fragmented nature of intellectual, collaboration and cross-fertilisation of ideas. Plans for evaluation and disciplinary and research community field studies were incorporated from the start. Although there was agendas related to technology and relatively little emphasis on pedagogy, the programme did note the education at the time [11]. A diversity potential to ‘add new and rewarding dimensions to the relationship of TEL research effort had resulted between teacher and class or teacher and pupil’ [8]. from the different cultures, traditions and trajectories associated with the A report by Her Majesty’s Inspectorate in 1987 found that, while various national educational systems, the MEP was only one of the agents of change and innovation in and their highly differentiated the field, ‘work with IT in schools and associated staff development experience of and ambitions for using owed much directly, and even more indirectly, to the programme.’ technology in support of learning. It noted the challenges associated with attributing direct impact to the programme because there was much activity in schools Successive programmes funded around microelectronics at the time and MEP resources were often by the European Union (EU) have channeled indirectly to schools. However, the report found that the served to aid the integration of strategy adopted by MEP considerably strengthened the number research and the emergence of new of well-informed teachers and trainers, that those involved often research and doctoral communities showed a resilience that allowed them to overcome difficulties and in relation to technology-enhanced uncertainties which was necessary for successful implementation, learning. These have included the that an impressive range of materials was developed, that closer PROLEARN Network of Excellence links were formed between training and curriculum development that dealt with technology-enhanced in IT, and that the cascade principle of teacher training worked well professional learning (2004-2008) where opportunities were offered [9]. and the Kaleidoscope Network of Excellence focused on technology- enhanced learning and access to as well as authoring systems and Despite these moves towards unity, cultural heritage (2004-2008). immersive environments. there still exist many closely related These were followed by STELLAR, and overlapping research areas that the European Network of Excellence STELLAR played an important role would not describe themselves as in TEL (2009-2012). The STELLAR in unifying research agendas and primarily TEL communities. These website specified that the network setting mid-term challenges for include, for example, those focused represented ‘the effort of the the research community, while on educational data mining, artificial leading institutions and projects in recognising the complexity within intelligence in education, networked European Technology-Enhanced and between the educational learning and learning analytics, Learning (TEL) to unify our diverse systems of partner countries [12]. as well as those identified in the community’. Each of these networks TEL research communities now paragraph above. brought together broad teams of seek to adopt more collaborative researchers working in diverse fields, and inherently interdisciplinary Within the UK, the work of the ESRC including computer-supported approaches that bring together Technology Enhanced Learning collaborative learning (CSCL), educational, learning and computer Research Programme echoed that blended and informal learning, sciences, as well as related disciplines. of the European Commission by

tel.ac.uk 13 3

and professional learning are very much part of the TEL research Arenas for informal learning, non-formal learning, lifelong agenda. This engagement with different forms of learning adds and professional learning are very much part of the TEL layers of complexity to an already research agenda. busy research arena. While members of the TEL research community have sought to define promoting unity within the research and reach agreement on the key field. The initiative represented a questions and the ‘Grand Challenges’ substantial programme of work that can drive innovation in that was funded equally by two of educational and learning systems, it is the UK’s Research Councils, the ESRC worth considering the nature of the and the EPSRC, from 2007 to 2012. challenge faced by TEL. Everyone has The core aims of the TEL Research experienced education and will have Programme were to design and a view about the role of technology evaluate systems that would advance in supporting that experience. understanding of learning and While there may be openness to teaching in a technological context, the integration of technology into and that would also improve teaching other aspects of everyday life, there and learning practice. is a variable but understandable resistance to innovations that tamper In some ways, the TEL label can with the dominant educational or place a bind on researchers. From training practices, unless a particular the perspective of practitioners innovation is occupying a new niche. and policy makers, ‘technology- enhanced learning’ captures the need to realise more from the potential 3.1 Understanding education of technology to assist learning In reflecting upon the general theme processes. However, the term can of moving beyond prototypes, it is also obscure the need for teacher important to understand different support and development that is ways in which the role of education often required to ensure positive may be characterised. impact in the classroom or other educational context. Education as a service: In some instances, education has been TEL is not limited to technology- considered as a service, in the sense enhanced education; the world of that learning resources are provided learning goes on outside the formal for learners and teachers. The BBC learning settings of the classroom Bitesize and OpenLearn websites and the lecture hall. Lifelong learners are among the current examples also engage in non-formal learning, of good quality educational including vocational and workplace resources available online, many training, which is not accredited by of which are available free of charge. a traditional academic institution. Due to the wealth of resources At other times, they will be involved available, learners and teachers in informal learning, in settings need to be strategic in determining where they choose their own which resources to access and how methods, define their own goals, to use them to build on current or work towards shifting goals [13]. understandings. Understanding As the case studies in this report education as a service prompts demonstrate, arenas for informal a focus on innovation relating to learning, non-formal learning, lifelong brand (the distinctive sets of values

14 Beyond Prototypes: Enabling innovation in technology-enhanced learning 3

associated with an organisation), channel (the means for delivering You look at the [US] educational establishment, and if resources to learners and teachers), product system (the overall bundle there’s any change it’s very slow. I don’t think the educational of services) and service (interactions establishment has really embraced these ideas. And, to the providing value for users) [14]. extent they embrace them, I think a lot of times it’s surface These product-focused types rhetoric and the reality underneath hasn’t changed. Business of innovation can influence how leaders will say, ‘We need a different type of workforce in the efficiently teaching is delivered, and the quality of the resources available future. We need different types of learners. We need people to learners, but they are concerned who are creative and collaborative.’ But then you see what with only limited areas of teaching they oftentimes recommend for the schools, as just a small and learning and do not require an incremental change from the way school has always been. improved pedagogic approach. So the rhetoric of what people say is needed doesn’t get Education as media production and matched with what they really call for in schools. presentation: Some Massive Open Online Courses (MOOCs) present Mitch Resnick education as a process of media Head of the Lifelong Kindergarten group at the MIT Media Lab production and presentation. These courses, sometimes referred to as ‘xMOOCs’ or ‘behaviourist MOOCs’, conversations about learning to 3.2 Education as a super- are distinctive in that they are focused take place both asynchronously stable system on interaction with content. They and synchronously, but concerns A system typically combines a set of have a traditional linear structure, about focus, privacy and e-safety, interdependent components. In the with targeted activities punctuating as well as the logistics involved in educational system, innovators who the acquisition of new knowledge providing the necessary technological are aiming for educational impact [15]. High enrolment rates on these infrastructure and support, all work not only have to consider aspects of courses show that this approach against adoption of the use of teaching and learning, but also how can be very successful in attracting these tools in the classroom. This the change will interact with other potential learners but, at the same understanding of education opens aspects of the system’s operation. time, high drop-out rates point to the the way to significant TEL innovation, disadvantages of treating education focused on changes in practice and Unless an innovation is well aligned simply as a process of production pedagogy that, enabled by the use of with the system into which it is and presentation. Such MOOCs may technology, can produce measurable introduced, the chances are that it will adopt an innovative profit model, an educational impact. be resisted or ignored by the system. innovative structure, an innovative Unless the necessary development process or an innovative channel The three paragraphs above set out and support systems are available, for course delivery [14]. Once again, different ways in which education teachers will rarely have the time or although innovation in these settings can be understood. The first two the inclination to test and adopt new may be significant, it does not require focus on resources and delivery practices. This means that education an improved pedagogic approach and are associated with processes is a super-stable system, and the or imply an increase in educational of innovation and development classroom is a challenging space impact on individual learners. that are already well understood within which to innovate. In order to [14]. The third, education as a understand these challenges, which Education as a conversation: conversation, is the one that is are inherent in the TEL innovation Theorists from Dewey to Laurillard most reliant on the process of TEL process, it is necessary to understand see education as a conversation innovation rather than on innovation why education, as a super-stable during which we adjust to each in areas familiar from business, system, is resistant to change. other’s perspectives [16, 17]. such as brand and profit model. In Learning is a process of coming order to engage successfully in TEL An important theme in organisation to know through meaning making innovation, it is important to look theory research concerns and conversation. Online tools at education not only as a process explanations of stability and allow extensive and extended but also as a system. homogeneity in organisational

tel.ac.uk 15 3

configurations and practices. A strong underpinned and reinforced by what is needed [...] and we’re taking and overlapping set of explanations multiple overlapping social forces the school in that direction. Policy come from open systems theory [18] and intertwined with other mutually comes at us and we’ll sort of harness and from institutional theory [19]. reinforcing practices. Coercive it to continue going in that direction’ Both emphasise the role of multiple pressures include not only central [23 , p.51]. overlapping and mutually reinforcing government legislation and regulation systemic pressures in maintaining but also national examination systems Accounts of successful change stability in organisational practices. and the bodies that control these. within such institutionalised systems These pressures also play a role in Normative pressures include public emphasise the importance of bridging bringing about homogeneity between and political expectations that are and brokering across organisational practices in organisations facing mediated and amplified by the boundaries and understanding that similar environment pressures. media, the role of professional practices have to be recreated in bodies and employer organisations; new contexts to function within Institutional theory emphasises cultural cognitive mindsets are the ecology of practices into which the role of three forms of pressure reinforced and transmitted by the they are being introduced [21, 24, in determining and legitimating strong role of apprenticeship in 25]. Section 4 provides an overview particular practices. These pressures teacher development. of some of the significant successes are coercive (laws and regulation), achieved by TEL, demonstrating normative (social values and Within education systems, different some of the ways in which boundary expectations), and cultural-cognitive practices play a mutually reinforcing crossing takes place. ● mindset (culturally determined and reinforced understandings of the world). Where organisational practices are strongly institutionalised, Changing a single system within an organisation often fails to in that they are underpinned by such multiple reinforcing sets of produce intended changes without changes to associated systems. determinants, they can be very hard to disturb. For example, research shows that management innovations role. Pedagogic practices interact in a parent company may be hard with and are constrained by to transmit to subsidiaries in a timetabling practices, budgeting different country where practices are practices, safeguarding practices, underpinned by different institutional inspection practices, data capture pressures [20, 21]. practices, governance practices, assessment practices and many Other research suggests that others. Initiatives that seek to change changing a single system within an just one practice component are organisation often fails to produce unlikely to achieve traction unless intended changes without changes attention is paid to other elements to associated systems. For example, of the system. a large study of middle managers in private sector firms showed training Ball and colleagues [23] in new management approaches documented the difficulties of to be ineffective in bringing about policy implementation in schools. desired change without attention to Their findings mirror some of the role of pay systems, promotion the conclusions presented in this systems, allocation of resources and report. They found that policies are symbolic communication from senior translated, interpreted and absorbed managers in reinforcing existing into existing ways of doing things, behaviours [22]. often being markedly changed or de- natured in the process. As one of the Education systems are, in the sense deputy heads they interviewed stated, above, strongly institutionalised. ‘I think we know what we want to Educational practices are do with our school, we know exactly

16 Beyond Prototypes: Enabling innovation in technology-enhanced learning 4 ACHIEVEMENTS UNDERPINNED BY TEL

Section 4 provides an overview of TEL successes, pointing to the UK’s role as a world leader in this area and identifying TEL’s role in developments as diverse as World Wide Web and the iPad. Three areas of success – the field of mobile learning, the development of the Scratch educational programming language, and the xDelia project that developed learning applications for financial traders – are considered in detail. The section also identifies reasons why TEL successes may go unnoticed, including the significant timescales involved.

The UK is currently a world leader that have impact on a similar scale to in the area of TEL. BETT – formerly that produced by other educational known as the British Educational interventions [see, for example, 28, Training and Technology Show – 29, 30]. Interventions that have has been running since 1985 and is claimed a larger impact, such as the largest educational technology Cognitive Tutors (see boxed case event in the world, bringing together study), have taken decades to over 35,000 educators and learning develop for a limited range of professionals. A recent list of the top curriculum topics. As Borgman 20 e-learning companies in Europe and her colleagues comment, in terms of their innovation, scale, market impact and revenue growth New technologies follow complex included 10 UK firms [26]. trajectories often supported or thwarted by other technologies, Successful TEL interventions have infrastructural issues, competing been identified and catalogued in standards, social systems, political different reports [6, 27]. In System decisions, and customer demands Upgrade: Realising the Vision for [31, p17] UK Education, Noss highlighted one recent set of examples: The extended period of development that precedes successful The Technology Enhanced implementation means that the scale Learning research programme of TEL success may go unnoticed has spent more than four years by observers who are expecting fast developing systems and software results or by those who are looking that, for example, use artificial for a new product or procedure and intelligence to teach teenagers thus do not notice the emergence algebra and help autistic children of an entire field, such as mobile with their social skills. We have learning, from TEL research. It may created virtual islands where go unnoticed by observers who young people acquire the expect the results of TEL research confidence to tackle some of to be confined to the field of life’s bigger challenges. We have education and so do not make the exploited the potential of giant connection between TEL innovation touch-screen tables to encourage and life-changing developments young children to work together. such as the World Wide Web. It We have taken sense-of-touch may also go unnoticed by those technology – the sort that makes who are expecting a linear model that gaming controllers vibrate that proceeds from idea to pilot to – and used it to train dentists full-scale roll out. The ecological cheaply and effectively [6] model of TEL points to the ways that different components combine and International reviews suggest that intermingle; one vision diffuses and TEL interventions lead to outcomes inspires others.

tel.ac.uk 17 4

UK centres of excellence in TEL sometimes prove easier to market outside formal education. This UK centres of excellence in TEL research have been prominent in has been true of Alan Kay, whose national, European and international networks. The largest assembly work on the handheld Dynabook of TEL researchers in the UK is at The Open University, with some learning device in the 1970s led 30 academics in the Institute of Educational Technology and a similar to the development of the iPad number of people with a core research interest in TEL spread across [34]; Nicholas Negroponte, whose the Knowledge Media Institute and university faculties. One Laptop Per Child project also developed technology for wider The London Knowledge Lab, a partnership between the Institute consumer use [35]; Seymour Papert, of Education and Birkbeck, has around 50 academics engaged with whose work on constructionist research into digital technologies, new media and knowledge. At the learning led to collaboration with University of Nottingham, the Learning Sciences Research Institute LEGO in creating the Mindstorms is a centre of excellence for research in the learning sciences and robot-building kits and to millions technological innovation, engaging 12 core academics. These three of children worldwide learning to institutes have formed CTEL – a collaboration aimed at sharing program computers [36]; and Tim research and innovation. This includes running a series of ‘What the Berners Lee, whose work on managing research says…’ events for industry, practitioners and policy makers. knowledge led directly to the creation of the World Wide Web [37]. Other centres of research excellence include the Centre for Innovation in Technologies and Education at Southampton; the The Web began as a project to Centre for Learning, Knowing and Interactive Technologies at provide physicists at CERN with Bristol; the Serious Games Institute at Coventry; the Learning and facilities that could support learning New Technologies Research Group at Oxford; the Educational by information sharing and data Technology Research Group at Warwick; the Centre for Studies exchange. These physicists formed in Advanced Learning Technology at Lancaster; the Children and a widely dispersed and computer- Technology Lab at Sussex; the Centre for Applied Research in literate group, using different Educational Technologies at Cambridge; the Institute of Learning computers. Tim Berners Lee wrote Innovation at Leicester; the Caledonian Academy research centre a simple hypertext program called at Glasgow Caledonian, and the Technology Enhanced Learning Enquire to keep track of people and Group at Durham. programs; enabling mail and file interchange between different types A recent national initiative has been the formation of the Future of computer system and different Learning Academic Network. The network brings together networks. Developing the Enquire academics from Birmingham, Edinburgh, Exeter, Glasgow, Leeds, code led him to something much Loughborough, Nottingham, The Open University, Reading, Sheffield, larger, ‘a vision encompassing the Southampton, Strathclyde and the University of East Anglia, all decentralized, organic growth of of whom are engaged with research related to FutureLearn and ideas, technology and society’ [37:1]. innovations in learning with technology. This was not a smooth progression.

The Web arose as the answer to Educational technology has been the a conceptual framework for an open challenge, through the inspiration or catalyst for many other augmenting the human intellect [33]. swirling together of influences, activities. TEL is an endeavour that ideas, and realizations from many attracts people from many different Researchers working to develop sides, until, by the wondrous fields. The theme of ‘enhancing technologies for educational offices of the human mind, a new education through technology’ enhancement have generally found concept jelled. It was a process of has captured the imagination and that creating successful educational accretion, not the linear solving efforts of innovators in computing, technology is harder than they had of one well-defined problem after technology and psychology. This can anticipated, that it takes longer than another. [37:3] be seen as far back at the 1950s, when they had planned, and that the route Skinner introduced the concept from vision to implementation is more TEL research has led to the of ‘teaching machines’ [32], and the circuitous than they had expected. development of major innovations 1960s, when Engelbart developed The products of their innovation that have taken root across the world.

18 Beyond Prototypes: Enabling innovation in technology-enhanced learning 4

It has also had important successes In the early 21st century, with cheap Based on his experience of mobile both within and beyond the field and robust mobile technologies learning projects, Steve Vosloo, senior of education, ranging from broad widely available, the European Union project officer in mobile learning at areas of development to more began to fund major multinational UNESCO, highlights the challenge of focused achievements. The following partnership projects, including assessing the new skills that young sections consider three very MOBIlearn and M-learning. This people are learning through TEL: different examples of TEL success. supported the development of In each case, innovation is an mobile learning from a small-scale There aren’t really recognized extended process, in which an research interest to an international measures for these. What you find understanding of the ecologies and phenomenon [41]. More than 26 doesn’t fit neatly into the standard communities within which innovation million people in Bangladesh have assessment system. So actually will be embedded is developed over now accessed the BBC Janala conducting the assessment, when time. Although these innovations language learning service on the standard formal assessment require the use of technologies, mobile phones [42]. Alongside does not necessarily recognise all they are not dependent upon a developments in Europe and Asia, of the learning that’s taking place, single technology, the equipment the USA has also awoken to the was definitely a challenge. That’s that is employed changes over time. possibilities of mobile learning, probably something that comes up This is particularly clear with the and the spread of mobile devices. in TEL quite a lot. [Steve Vosloo] first example, mobile learning.

4.1 Success story: mobile There are schools that buy the latest tablet computer or learning The development of mobile learning handheld gadget and only then consider how to use it. can be traced back to its origins in TEL research. Following mobile technology Providing evidence-based research In the earlier half of the last decade, research in education at the that can support learners and sophisticated mobile technology beginning of the century, such as teachers to make informed decisions was scarce, fragile, expensive and the Palm Pioneers project [43], there about engagement with mobile difficult, and was the prerogative has been a stream of innovation learning is a continuing challenge. of institutions. In the second half in the US around 1:1 learning with of the decade, mobile technology handheld devices in classrooms became universal, robust, cheap, and lecture theatres. 4.2 Success story: Scratch diverse and easy. [38] A different form of TEL success is Sheer weight of numbers provides represented by Scratch. This is a Researchers began to study some indication of the success of visual programming environment the potential of mobile devices these innovations, but it is difficult for children to create and share to enhance fieldwork, allowing to use scientific methods developed interactive stories, games and students to learn when and where for laboratory use to assess a shift animations and to think creatively, they needed to do so [39]. The in learning behaviour on this scale. reason systematically, and work findings of small-scale studies led It is clear that people are benefiting collaboratively [44]. Scratch is a to a vision of mobile devices able from opportunities to learn in project of the Lifelong Kindergarten to support lifelong learning, devices contexts that were never possible Group at the MIT Media Lab, headed that would be portable, available in the past, but there is still by Mitch Resnick, and has received anywhere, adaptable and intuitive much work to be done to enable funding from the US government to use [40]. Since then, mobile consistent educational impact. National Science Foundation, private phones have become smart phones, While some schools have adopted companies and foundations. First and generations of personal digital innovative approaches to developing launched in 2007, and provided free assistants (PDAs) and other mobile mobile learning between home of charge, approximately 4 million technologies have come and gone and school: there are schools that Scratch projects have already been but researchers have continued buy the latest tablet computer or uploaded and shared by users. A spin- to work towards that vision of handheld gadget and only then off online community for educators educational innovation. consider how to use it. called ScratchEd was launched in July

tel.ac.uk 19 4

2009, highlighting the value of the to the Museum of Science, Boston. products produced by children. Its programming environment for formal Sponsors have since included Adobe, innovative features come from the as well as informal learning. Autodesk, Hewlett-Packard and LEGO configuration of software and social systems and today there are over 200 networking elements. Like much The project had its origins in more ClubHouses around the world. TEL innovation, it has involved the than 40 years of development at assembly of existing skills, ideas and MIT of research on children learning This long history of development resources in a new and productive through programming. This was and of community building over way. It has also involved extensive strongly influenced by Seymour a period of more than 40 years, work in building and engaging with Papert’s work from the late 1960s provided the inspiration and basis the communities involved. onwards, including teaching children for the development of Scratch. The to control a programmable ‘turtle’ influence of this work is still increasing using the Logo programming and is becoming increasingly evident 4.3 Success story: xDelia language [45, 46]. Papert’s research, in Europe. In the UK, government Extended work with stakeholder in turn, drew on the work published policy now places increased communities was key to the success by developmental psychologist Jean emphasis on programming within of xDelia (www.xDelia.org). This was Piaget from the 1930s to the 1970s. the curriculum. This is linked with a a three-year project that combined growth in after-school programming research into the decision-making, In working with Seymour Papert clubs and in teacher training. The learning and trading practices of and being involved with the organisation Code Club, for example, professional and private financial Logo community, I saw a lot of has over 1200 clubs nationwide. Both traders with the development of what drove it and what limited in school and out of school, Scratch learning applications. The project it. I saw both the strengths and is used as a gateway to programming. explored the role of emotions and the weaknesses of the Logo Elsewhere in Europe, there is emotion regulation in financial Community. We went on to similar interest in programming decision-making and how learning start our own network of after- and computing environments. For interventions that focused on school centres, the Computer example, the Portuguese government improved emotion regulation might ClubHouses. The initial inspiration has developed a partnership improve financial decision-making. In for Scratch came from our work between the government, a higher its later stages, the project evaluated at ClubHouses, where we saw what education institution, and internet a series of learning interventions kids were looking for and realised provider SAPO that seeks to provide that exploited a combination that there were no good tools continuing professional development of physiological sensors and available. [Mitch Resnick] and that has a focus on Scratch. serious games approaches. These interventions were combined in an overall pedagogic approach founded on an understanding of the role of The initial inspiration for Scratch came from our work at emotions in trading, practice-based approaches to learning and a close ClubHouses, where we saw what kids were looking for and understanding of traders’ trading realised that there were no good tools available. and learning practices. The project was funded by the EU as part of the Seventh Scratch emerged not only as In 2013, the first European Scratch Framework Programme (FP7) and a consequence of researchers’ conference was held in Spain and was carried out by a consortium educational vision and experience ‘Scratch Day’ was marked by 184 of academic institutions and a but also as a consequence of the events in 47 countries. commercial partner. The initial sustained engagement the team had stage of the project took the form with young learners through the Scratch is an example of an innovation of in-depth studies of the behaviour ClubHouses. The first, and flagship, that has succeeded by running of traders and the ways in which Intel Computer ClubHouse in the alongside the education system in their emotional state influences Network was opened in collaboration its use in informal settings outside the their decision making. The project with the MIT Media Lab in 1993 at the classroom and in the development focused on a particular emotion- Computer Museum and later moved of a network for sharing the creative driven trading bias as a proof of

20 Beyond Prototypes: Enabling innovation in technology-enhanced learning 4

concept: the tendency to hold loss- that were carried out at the start. in TEL, sensors, the psychology making assets longer than assets at These provided the consortium of decision-making and emotion a gain (the disposition effect). with a detailed understanding of the regulation, behavioural economics, environments in which their work neuroeconomics, and financial Novel use was made of physiological would be implemented. trading and markets. sensors to provide feedback on emotion regulation capabilities, and games were developed that mimic important aspects of trading xDelia was able to move from initial concept to a fully embedded and financial decision making. The project successfully demonstrated concept with demonstrable educational impact. links between engagement in the learning interventions and real- world behaviour; it also built on The work also extends forward, • There was a strong commitment work demonstrating links between beyond the period of project to dissemination by engagement traders’ effective emotion regulation funding. Early work has now begun, rather than by broadcast. and performance. Outputs from this in collaboration with commercial work are being exploited in several organisations, to explore the potential These features are shared with the different contexts and Saxo Bank’s of the approaches developed by xDelia other TEL successes considered in online investor education platform in other fast-paced environments this section. A close attention to the for its clients has developed an such as training racing car drivers and innovation process by consortium approach to developing clients’ air traffic control. A vision of using members, and an understanding of the understanding of the role of learning interventions to improve importance of previous developments emotions in their trading based emotion regulation in high-pressure and cross-community engagement, on outputs from the project. situations links this work over time. As enabled the xDelia team to work with the other examples in this section, through the innovation process In comparison with the other examples the core of the work is not a single consciously and relatively quickly. In discussed in this section, xDelia is technology, but a willingness to make Section 6, these features are brought a good example of a time-bounded use of technology to achieve a vision together in a model of the TEL project that, in three years, was able of enhancing learning. Innovation Process. This process also to move from initial concept to a fully takes into account the misconceptions embedded concept with demonstrable xDelia can therefore be understood and challenges that may be educational impact. This process, as one element of a large body of encountered by researchers. These though, was based on extended work that extends over time. However, are considered in Section 5, together engagement with the communities it was also successful in delivering a with ways of addressing them. ● involved. Jeffrey Lins, head of significant innovation in a relatively advanced research and innovation at short time frame. Key features of the Saxo Bank, was a project partner. From project enabled this success. his perspective, the establishment • Research team members had a and success of the consortium were close understanding of the ecology rooted in earlier work of practices in the areas where the innovation would be embedded The academic lead on xDelia has • There was close collaboration with a a remarkable understanding about commercial partner from the outset, how it works inside investment which played a significant role in the banks, not only because he’s research and development process. studied it academically but, having • Exploratory studies investigated interacted with these kind of the nature of existing informal organisations, he understands and formal learning practices in them [Jeffrey Lins] these environments. • The target audience was involved He also attributed the success of in early studies and trials. the project to exploratory studies • It engaged a highly cross- and to reviews of previous work disciplinary team, with expertise

tel.ac.uk 21 5 TEL INNOVATION: CHALLENGES AND MISCONCEPTIONS

Section 5 deals with challenges to TEL research and innovation. Six common misconceptions are examined and recommendations are identified that will increase the potential of TEL to achieve widespread impact. These show that funders, researchers and policy makers all have a role to play in achieving impact.

The TEL research community has it would not claim to have aligned had undoubted success in extending the efforts of thousands of learning the vision and reach of innovations scientists. that have reached millions of people worldwide, notably mobile learning Grand challenges of TEL such as [47, 48] and MOOCs [49]. TEL has ‘Make use and sense of data to provided theoretical underpinning improve teaching and learning’ for technologies that are used to [12] have the potential for more support learning, and policy briefings immediate social impact than the advising on how these technologies hunt for the Higgs Boson, but can be used to achieve educational have never captured the public impact. This work has included critical imagination. Why is this? One reason appraisals of tools that are widely is the complexity of TEL, which used in schools and universities, will be examined in Section 6. The including integrated learning systems most straightforward approach to [50], interactive whiteboards [51] innovation is to focus on marketing and virtual learning environments a technology, rather than on the [52]. The enduring success of The complexities involved in using that Open University is closely allied with technology to achieve significant research into innovations in TEL educational impact. However, the [53, 54]; this research drives the technology by itself is not the continued adoption of new methods innovation, the importance of the of distance learning and assessment. technology lies in the ways in which it can enhance learning by supporting Despite its achievements, the TEL or transforming a particular research community has neither pedagogy or practice. Without these the coherence nor the scale of underpinning elements, it is likely to other scientific communities such be reduced to an expensive way of as particle physicists or climate doing something that was done more scientists. For example, the World cheaply in the past. Climate Research Programme ‘organizes large-scale observational To give one example, the original and modelling projects and provides conception of interactive whiteboards the international forum to align as a TEL innovation brought together a efforts of thousands of climate complex of technology and practices. scientists to provide the best possible These included practices related to climate information’ [55]. By contrast, pedagogy, classroom organisation, the STELLAR European Network of teacher-pupil relationships, staff Excellence in TEL [11], funded by the development and technical support. European Commission, integrated Changes to all these practices were 15 leading research organisations in required in order to realise the boards’ TEL between 2009 and 2012. It was potential for fostering interaction, successful in coordinating research, creativity and collaboration. However, informing governments of TEL a study of their use in schools innovations, and supporting initiatives noted that ‘the tools of educational such as the European Conference in technology have no magical power Technology Enhanced Learning, but in themselves; only by being embedded

22 Beyond Prototypes: Enabling innovation in technology-enhanced learning 5 TEL INNOVATION: CHALLENGES AND MISCONCEPTIONS

in the practices of teachers and technological innovation. A visible assessment could have a sustained learners do their mediational means technology such as One Laptop Per effect on the practice of education. come into play’ [51]. Child (OLPC) has an obvious appeal Learning through social networking, for governments and media, but its the use of mobile devices to support Without training and technical development has been criticised for lifelong learning, the use of analytics support, sometimes without reliable divorcing the provision of technology to improve learning design – all network connection, careful set-up, from its content, training and use [56]. these visions of the enhancement of or budget for replacement parts and learning by technology involve the repairs, it is difficult for teachers to Technology should not be the creation and implementation of new use interactive whiteboards in ways primary driver of educational systems rather than specific pieces that support learner engagement and activity; it should support it. of software. interaction. Nevertheless, the boards Technology such as the XO-1 [the have often been purchased without computer model developed for RECOMMENDATION: Policy attention to these practices. It has the OLPC project] should only and funding should support and proved easier to present the boards ever be considered as supportive encourage changes in pedagogy and purely as an innovative technology that of educational practice, never as practice, as well as the technological can be bought, installed and used. This core to it. [56:244] developments that will support these. approach has reduced opportunities for significant pedagogical impact because it treats the new technology as a direct substitution for the Technology should not be the primary driver of educational previous blackboard or whiteboard, and pays little or no attention to the activity; it should support it. support and resources required in order for teachers and learners to gain extra benefits. Starting with an educational challenge 5.2 Look beyond the formal is more likely to produce successful education sector In such cases, the failure to achieve educational transformation than educational impact is associated starting with a technology. A review MISCONCEPTION: TEL innovation with a failure to recognise that TEL of the use of interactive whiteboards should be focused on formal innovation consists of a process in schools in 2011 concluded that education. of implementation rather than a technology. The sub-sections below there is a need to reassess the Historically, government policy set out a series of misconceptions use of computer technology from related to TEL has focused on that can limit the success of TEL an educational, rather than a formal education. In the 1980s, innovation, and identify ways of technological, perspective; and the Microelectronics Education addressing these and thus improving develop a more sophisticated Programme (see the boxed case the potential of the TEL research conceptual model of how ICT can study in Section 3) aimed to ensure community to achieve widespread facilitate teaching and learning in that school leavers would be familiar public or commercial impact. the classroom. [51:362] with computers and their potential applications. In the 1990s, the While the new ‘app economy’ that Teaching and Learning Technology 5.1 Shift the focus from markets software applications could Programme provided impetus for technology to pedagogy theoretically produce educational adoption of TEL across the university and practice software that can be transported into sector. In 2005, the Department classrooms and shown to improve for Education and Skills report on MISCONCEPTION: The technology learning outcomes, this is unlikely ‘Harnessing technology: transforming is the innovation. Innovation therefore given the past failure of individual learning and children’s services’ follows a linear path from idea to technological resources or tools to included references to lifelong prototype, deployment and evaluation. have a major influence on education. learning electronic portfolios and We need to look beyond the linear occupational training, but focused As set out above, the temptation, model of TEL innovation, in order most of its attention on the formal particularly in time-bounded, grant- to see how new technology-enabled sector [57]. In the same decade, funded projects, is to focus on a methods of teaching, learning, and the ‘Strategy for e-learning’ and

tel.ac.uk 23 5

What’s quite unique about Coventry is that they have this informal learners into formal learners very large Business Development Support Office […] What’s by providing pathways to enrollment and to qualifications. The success good about that is that they know how to behave around of Scratch is closely associated with industry and they know how to deal with industry and they the way in which socially supported know how to connect things together. And that’s a huge informal learning acts as a bridge advantage for Coventry because we have this very long into formal learning. tradition of working with industry and I think that’s really RECOMMENDATIONS: In order to made a big impact. address the growing importance of the informal learning sector, policy Sara de Freitas and funding should support the Founder of the Serious Games Institute at Coventry University experimentation that is necessary to generate fresh insights and achievable visions of educational its successor ‘Enhancing learning the end of the report. Even some developments. Research findings in and teaching through the use of team members who had access to all areas of TEL should be available technology’ published by the Higher university library facilities found to researchers both inside and Education Council for England it difficult to locate some end-of- outside the university sector. (HEFCE) focused, as would be project reports. expected from HEFCE, on the higher education sector [58, 59]. During the past decade, the informal 5.3 Widen the TEL community learning sector has emerged Attention to informal learning has not as an important area for large- MISCONCEPTION: Specialized areas been entirely absent. The Computer scale commercial TEL innovation. of expertise drive TEL innovation. Literacy Project of the 1980s Individuals are adopting open promoted public understanding educational resources and mobile Research and development in learning of microelectronics technologies learning software on a massive scale. with technology are fragmented, through the medium of television. To take the example of just one with separate communities for Earlier this year, the Department provider, from 2008 to 2013 there TEL, e-learning and computer- for Business Innovation and Skills were over 64 million downloads by based training. Specialist research published a research paper on around 9 million unique visitors of conferences have been established ‘The maturing of the MOOC’ that the open educational resources in many areas, including Computer- made recommendations regarding shared on iTunes by The Open assisted Language Learning, Mobile future research [60]. Nevertheless, University. Worldwide, there is Learning, Computer Supported most policy in this area has been an increasing demand for lifelong Collaborative Learning, Networked concerned with the teaching and learning, for staff development and Learning, Serious Games, Open learning that takes place in schools, for on-the-job training opportunities. Learning, Artificial Intelligence and colleges and universities. Education, and Educational Media. Entrepreneurial universities, offering Despite the work of the European A related issue is that access to TEL new methods of informal learning on Networks of Excellence in engaging research publications remains limited a worldwide scale, have fuelled the a broad range of researchers, the TEL for those outside the university recent explosive growth of MOOCs. community is not able to speak with sector. Researchers working in In 2010, 200 students enrolled on one strong voice. business and industry have restricted Stanford’s Introduction to Artificial access to research findings, and are Intelligence course. The following A focus on establishing specialised therefore hampered in their ability year, when it was made freely communities of researchers means to engage in evidence-informed available online, 160,000 students that less attention is paid to building innovation. While working on the from 190 countries enrolled [61]. links with learners, teachers, policy Beyond Prototypes research study, makers and industry. This means it is a team member who works in the Developments in the informal sector difficult to complete the innovation commercial sector was unable to support developments in the formal process because experience, access all the literature cited in sector. Universities are currently expertise and visions of educational the bibliography that appears at investigating how they can convert change are not widely shared, and

24 Beyond Prototypes: Enabling innovation in technology-enhanced learning 5

there are limited opportunities TEL research largely operates at be capacity to support individuals and for understanding the ecology of an elevated level and focuses on teams to engage in long-term projects practices within which innovations medium- to long-term innovation. capable of turning inspirational will be deployed. It is focused on universities, though ideas into fully embedded products this is hardly surprising, given that and practices. Researchers and Various initiatives suggest ways of the funding for UK research comes developers should be encouraged broadening the TEL community mainly through the Research Councils to plan for sustainability and to and making links between research, and the European Commission. identify the elements that must be teaching and industry. The BETT The focus in the forthcoming EC taken into account in order to enable education show is currently Horizon 2020 programme on small sustainable implementation of an idea developing a research strand. In companies may help to shift that or prototype, in the context of a vision addition, it encourages the sharing of emphasis, but TEL is not a central of the enhancement of learning. The best practice and teaching innovation theme in that programme. implementation and success of plans by connecting with practitioner-run for sustainability should be evaluated. TeachMeet gatherings and hosting The Mobile Learning Network a large-scale TeachMeet each (MoLeNET) programme of capital year. The Association for Learning funding for further education (FE) 5.5 Find new ways to assess Technology (ALT) works to improve institutions to embed learning with the contribution of TEL practice, to promote research and to mobile technologies into FE was a influence policy, providing a forum good example of TEL research being MISCONCEPTION: Scientific for researchers and practitioners embedded directly into practice. methods developed for laboratory in further and higher education. Training practitioners to become research are the best way of evaluating The series of ‘What Research TEL researchers is not the solution the impact of TEL innovations. Has to Say’ events organised by because practitioners are already the London Knowledge Lab, the fully occupied – rather there is a ‘No significant difference’ is an University of Nottingham, and The need to form enduring partnerships issue that has dogged TEL from Open University has communicated between academic TEL researchers, its inception. It has been difficult new developments in TEL research practitioners in schools, colleges to demonstrate a significant directly to companies and policy and workplaces, and innovative positive impact associated with the makers. Other opportunities need to e-learning companies. introduction of TEL into a classroom. be found to integrate the disparate research and practitioner groups and the emphasis needs to be on dissemination by engagement with MISCONCEPTION: Scientific methods developed for laboratory learning taking place by all parties, rather on dissemination by broadcast. research are the best way of evaluating the impact of TEL innovations. RECOMMENDATIONS: In order to widen the TEL community, researchers need to engage with the individuals and communities that will In order for TEL innovations to have A major study in the form of a play a role in the implementation of long-term educational impact, they second-order meta-analysis of the innovations. Policy and funding should must be embedded successfully. impact of technology on classroom encourage the development of skilled, In some cases, they are embedded learning (a synthesis of the findings of multidisciplinary teams that are able to within universities but, in most cases, meta-analyses, encompassing 1,055 complete the TEL innovation process. they are deployed elsewhere. It is research studies) found an effect size important to pay attention to and of 0.35 [62]. To put this in context, it plan for this element of the process. is below an effect size of 0.4, which 5.4 Connect TEL research is the level at which the effects of and practice RECOMMENDATIONS: Policy and innovation enhance achievement in funding should take into account such a way that real-world differences MISCONCEPTION: Most of the the importance of this stage of the can be observed [62] and lower TEL innovation process takes place innovation process and the need for than the effect size of some other within universities. extended development. There should educational innovations, including

tel.ac.uk 25 5

reciprocal teaching (0.74) and Kirkwood and Price investigated 5.6 TEL success is not mastery learning methods (0.58). what enhancement of learning necessarily commercial success means in the context of TEL The authors of the study indicate [64]. They note that it is difficult MISCONCEPTION: Effective and that simply measuring the effect to attribute causality when sustainable TEL products and of introducing technology misses independent variables are not held practices have commercial value the important point that it is how constant; the comparative study and commercialisation should the technology is used that makes method is only appropriate where therefore always form part of a difference. For example, effect other elements of teaching are the innovation process. sizes from computer technology replicated. A learning enhancement used as ‘support for cognition’ that is associated with the provision The assumption that innovation were significantly greater than of additional resources or tools for should be associated with those related to computer use one group of learners may simply be commercialisation was built into for ‘presentation of content’ [62]. attributable to extra time spent on the Beyond Prototypes study The implication is that it is more the task or extra teacher attention. from the start. Each example of helpful to examine the influence If a study does succeed in replicating TEL innovation was considered of a combination of innovative all elements of the learning in the context of challenges to pedagogy and technology rather experience other than the one being commercialisation, assistance to than technology alone. The way in assessed, it is not necessarily clear market and relationship between which the innovation is introduced, what has been enhanced. Different impact and commercial exploitation. including management support and evaluations of the same innovation This analysis showed that, although teacher development, is also likely to can give different results, and the very important in some contexts and influence the outcomes substantially. same results can be interpreted for some projects, commercialisation or presented in different ways as is not an essential part of the TEL Evidence-based practice is crucial the Cognitive Tutor Software case innovation process. for TEL because it ensures that study (see box) shows. learning is enhanced by technology Innovation is not synonymous with in practice, as well as in theory. The comparative approach is invention and it is rare that TEL However, the methods for assessing associated with behaviourist research produces an individual educational impact have to be views of learning, and makes the item of exploitable technology carefully chosen and appropriate assumption that enhancement will be in the short term. University-led to the innovation. Goldacre associated with quantitative change research is more likely to be focused suggests that ‘randomised trials are – improvement in test scores – rather on long-term educational impact generally the most reliable tool we than qualitative changes that are than on commercial success. have for finding out which of two more difficult to measure, such as a Academic researchers are recruited interventions works best’ [63]. richer or deeper understanding. An and trained to research the theory There are certainly cases in which innovation that is tailored to meet and science of learning. They are this method can be used reliably. the requirements of a randomised well placed to test and evaluate TEL The large numbers of participants controlled trial may have to limit its innovations, because education is engaged in MOOCs allow scope, thus reducing its impact, in the core business of a university. comparative testing, in which order to do this. However, they are not encouraged randomly selected groups assigned to create educational enterprises, to one of two conditions and the RECOMMENDATIONS: There which might compete with their outcomes assessed quickly. This is an urgent need to ensure that university for students. Nor are differs from randomized control TEL innovation is evidence based they encouraged or resourced to trials, in being part of a process and has demonstrable impact. In replicate the role of an R&D unit of rapid testing and development. order to do this, policy and funding within a large company. In other situations, though, it can should require the evaluation of prove impossible to alter one or TEL innovations in terms of their The belief by many academics in two variables while other factors educational impact. New methods open research and free access to remain constant. This is particularly of evaluation, such as the use of learning does not necessarily fit true when TEL innovation learning analytics or A/B testing well with a commercial imperative involves changes to a series of where appropriate, should be for profitability and return on interconnected practices. developed and put into practice. investment. Publicly funded

26 Beyond Prototypes: Enabling innovation in technology-enhanced learning 5

universities have a strong motivation Case study: Cognitive Tutor software to share their research and development with the public that Carnegie Learning has been a provider of innovative, research-based paid for it. Funders may reinforce mathematics curricula for middle and high school students in the this impetus towards openness USA for over a decade and in 2012 was announced as the winner of by requiring that the results of the Software & Information Industry Association (SIIA) CODiE for research are made freely and openly Best Mathematics Instructional Solution. The company was founded available. However, trying to make in the 1990s by staff at Carnegie Mellon University working with everything open and free makes it practising teachers. In 2011 the Apollo Group acquired the company hard for industry partners to build for $75 million, with an additional $21.5 million payable to the on and extend the work in order university for related technology. to make money. Some companies, such as Google and Facebook, have Throughout this period there has been sustained debate around achieved commercial success based the issue of how adaptive learning software may enhance student on ‘freemium’ models that provide learning. One focus has been on student test scores with some free services or content and make research noting little or no statistically significant change, other money in other areas. However, studies identifying significant impact, and some reporting mixed, many industry partners do not wish anecdotal or selective data [65-69]. What is clear, however, is that to take on the difficulties and risks a favourable perception can be bolstered by research and that this inherent in the implementation can influence purchasing decisions. However, it was found that the of such models. There is a need effectiveness of software was dependent on teachers’ ability to use for sustainable funding that can it [70]. This required consideration of the complex of TEL elements, support both open research and resulting in the development of materials such as textbooks, a commercial exploitation. recommended programme of teacher training, and pedagogy with greater emphasis on individual learning. Academic TEL research has an effective role to play in contributing A recent comprehensive report by RAND on Cognitive Tutor has to the fundamental research and found that the cumulative effect can be that ‘treatment group evaluation of design-based research teachers reported less implementation of traditional practices such partnerships that are put in place as lecturing with students taking notes and greater implementation to develop new learning-technology of more progressive practices such as facilitating student work or systems. This is a larger-scale assigning students to work in groups and give presentations’ [68]. enterprise than developing individual pieces of software or carrying out small-scale evaluations. It requires FutureLearn, currently partnered made earlier. When project teams coordination across institutions, with 26 universities worldwide identify the elements that must be involving academic, practitioner and as well as other institutions) or a taken into account in order to enable commercial partners. This should long-term relationship between a sustainable implementation of an not be an afterthought, but should university and companies (as with idea or prototype, in the context be planned from the start of the the Serious Games Institute) is of a vision of the enhancement project. Diana Laurillard, Professor likely to have more successful and of learning, they should take the of Learning with Digital Technologies, sustainable outcomes than working possibility of commercialisation into notes that on the assumption that universities account. This requires engagement are suppliers of TEL inventions. with the individuals and communities early association between academics Firms need access to university that will take responsibility for and a company wanting to make expertise, but are unlikely to expect commercialisation. Policy and funding a certain product is critical. The the key contribution of university should encourage the development company needs to feel this is researchers to be the production of skilled, multidisciplinary teams important for its future portfolio of new technology. that are able to complete the TEL of products. innovation process. If a commercial RECOMMENDATIONS: Where outcome is required, this should A partnership role, with the commercialisation is an issue, it be specified from the start and university as an investor or should be taken into account when the project team should include innovation partner (as with implementing the recommendations commercial expertise. ●

tel.ac.uk 27 6 THE PROCESS OF TEL INNOVATION

Section 6 shows that TEL should be considered as a technology complex, made up of a series of interconnected elements that cannot be changed in isolation. A model of the TEL Complex is set out, centred on a vision of educational change. The TEL Innovation Process is also modelled and bricolage is set at its heart. TEL innovation should be evidence driven, and a core methodology is identified as design-based research.

6.1 The TEL Complex rather than TEL, together they Although, as Section 5 showed, TEL capture the sense that innovation is is rarely simply a product, a common generally complex and its successful tendency is to focus solely upon the achievement may involve changes to technology element and its transfer many different elements in a manner into practice. This implicit assumption that is multiple rather than linear. of a linear model of innovation often underlies the expectations of policy The Beyond Prototypes case makers. It is typically assumed that studies show that TEL should be processes of research, development understood as a ‘complex’ comprising and diffusion follow sequentially. a series of components that need Sometimes a phase during which to be addressed together. A generic opportunities or gaps in the market ‘technology complex’ includes a wide are identified may precede the range of elements, including purpose, research. Rothwell [71] provides a materials, procedures, knowledge, useful summary of different models organisational structure, industry of innovation, identifying two linear structure, location, social relations models: technology push and market and culture [73]. pull. The merits and deficiencies of the linear model formed the focus of In the case of the TEL complex, session held at a Nobel symposium several key elements must be taken in 2002 [72]. into account in order to develop and realise a vision of innovation. These are A study of 2000 cases of innovation set out in the model in Figure 1, and by Keeley and his colleagues [14] considered in the paragraphs below. provided a counterpoint to the simple ‘kit’ view of innovation. The study Pedagogy is a crucially important identified ten areas of innovation, component of successful TEL including the processes involved innovation and goes well beyond the in providing services, the services technical elements used to support that provide value for customers, it. In terms of the areas of innovation the profit model, the organisational identified by Keeley and his colleagues structure, the product performance, [14], pedagogy comprises an the channel by which the services extremely complex and distinctive are delivered and the process of process which involves both student customer engagement. While and teacher engagement, delivering a innovation may take place in any set of educational services by means of these areas, it often includes of specific channels. several working together. For example, an innovative form of Technical components are the most customer engagement may require a visible components when considering new form of organisational structure. innovation within the TEL Complex. Although these areas of innovation They are the technological elements are framed in relation to market- that are used to support the oriented business organisations pedagogy with the aim of achieving

28 Beyond Prototypes: Enabling innovation in technology-enhanced learning 6 THE PROCESS OF TEL INNOVATION

a vision that is concerned with Figure 1: The Beyond Prototypes model of the TEL Complex enhancing learning in a specified way.

The ecology of practices and LICY CONTEX technical context must be taken PO T into account, because any TEL NT COMM DE UNIT innovation will be implemented STU Y in a specific ecology of practices. OF PR OGY ACT For example, the development of OL IC P C ES E Yoza Cellphone Stories (see box) E D A Y AGO D GY G E took into account the limited access T PE I O N

N G to books experienced by some V U I G C I

R M communities in South Africa as well

N R

O

I M

E

as local practices associated with N

O S

D

C VISION C

M

the use of mobile phones. N

O R

U

E E

M

F

H N

M

C

T

U

Current practices are not easily A

N E

T

T I

altered; they are at the core of EC AL T HNIC Y super-stability in the overall educational system. In the TEL T ECH TEXT Complex, practices include explicit CON TE S aspects of teachers’ practice as CH ITIE NICA MUN well as the tacit knowledge acquired L COM through extensive apprenticeship, RE N VEN IO training programmes and long UE GENERAT experience. Students’ practices are also crucial, and systematically relate to those of teachers. How students learn, both formally Communities involved in the objectives and above all, expertise. through structured teaching and TEL Complex include students, This strong community presence learning programmes, and informally, teachers, researchers and those within the TEL Complex constitutes through social and peer interaction, engaged in technical development. a major challenge for TEL innovation, is important for the effective In literature dealing generically and in many cases exhibits super- operation of TEL innovation. with innovation, these would be stability, meaning that change is characterised as suppliers and extremely difficult to achieve. In In stable systems, innovation that customers. These four communities particular, current expectations involves changes throughout the have others associated with them, of teachers and students affect entire ecology is characterised including the parents of young the adoption of TEL innovations. as ‘system innovation’. Many different sub-components have to work together, with each subcomponent subject to the Any TEL innovation will be implemented in a specific ecology constraints of the overall system. These complex interdependencies of practices. make it difficult to get any one element to work or make a difference by itself without learners, the families of mature The wider context (including consideration of the whole [74]. students, the managers of policy, funding and revenue Moreover, different components educational institutions and the generation) Although TEL is can combine and recombine in many people responsible for teacher not typically a conventional different configurations. Ultimate training and technical support. The market-oriented business example, success depends on the totality of communities associated with these there is always a need for sources the configuration or bundle, rather different sets of stakeholders often of funding to initiate, sustain and than on any single component. have different values, perspectives, support the processes of innovation

tel.ac.uk 29 6

Case study: Yoza Cellphone Stories 6.2.1 Configurational innovation Technological innovations, like TEL The aim of the Shuttleworth Foundation funded Yoza Cellphone innovations, often do not rest on Stories project (Yoza), formally entitled m4Lit, was to promote new technological components leisure reading by the distribution of m-novels to mobile phones but rather on the ways in which in South Africa – a country where less than 10% of public schools pre-existing and well-understood have functional libraries but 70% of urban youth have internet- technologies are configured to meet enabled mobile phones. The project began in 2009, taking new challenges. Fleck introduced the inspiration from work done in Japan, using an existing mobile idea of ‘configurational technologies’ chat platform to release content and advertise, and publishing in to describe and analyse the ways in local languages, including Afrikaans and isiXhosa, as well as English. which technical systems are created Yoza considers the key innovation in this process of bricolage and configured to conform to the not to be the use of phones, but the provision of really engaging contingencies of specific applications. stories (some published in episodes), available easily and affordably, Local contingencies crystallise to form with readers able to comment and see others’ comments in near technological configurations [78]. real time. Peine built on this work and used In early 2013, Yoza won the Netexplo Award in Paris and had the case of Smart Home systems a catalogue of over 50 openly licensed m-novels, poems and to show how learning and innovation plays, some of which deal with difficult subjects such as living develop in the application of with HIV. Use of the service has been strong, with over half a configurational technologies [79]. million completed reads and 50,000 user comments recorded in The configurational nature of Smart the 17 months to December 2012. Securing further funding has Home technologies is inherent in proved challenging. However, content has been reused elsewhere, the wide range of technological including by Young Africa Live, and the model has helped pave systems and expertise that must be the way for other initiatives in South Africa such as the FunDza brought together to create smart Literacy Trust. homes. It is also inherent in the need for these homes to work in the context of the local social and development. This need for The model of the TEL Complex practices and everyday routines sustainable funding has recently illustrated in Figure 1 shows that, of homeowners. been identified in the innovation in order for innovators to develop literature as an associated ‘business and achieve a vision of TEL, it Both Fleck and Peine emphasise model’ [75] which incorporates is necessary to engage with all that configurational innovation an ‘earning logic’ [76] or ‘revenue these elements: pedagogy and arises from ‘learning by trying’, by mechanism’ [77]. It involves, at technology, current practices and which they mean active engagement the very least: (a) the provision of communities, the local ecology and in design and local experimentation value, (b) the effective utilisation the wider complex. This is a process in response to local practices. This of assets or resources in providing of configurational innovation that places the stress on innovation that value, as well as (c) the securing requires research teams to engage as implementation. of sustainable support through in ‘bricolage’. some form of revenue generation. There are important parallels here In the case of TEL, if policy dictates for TEL innovation, but this report that funding for TEL is subsumed 6.2 Configurational goes further in stressing the role of within general educational budgets innovation and bricolage practices as part of TEL innovations, or within special project funding, A central theme in the Beyond not just as part of their context. then competition with regular Prototypes case studies and TEL innovations are most readily demands or the time-limited interviews is that successful understood as configurations, not nature of project funding can work innovations in TEL are often not just of technological components against long-term sustainability new inventions. They more often but also of social practices. As shown and adequacy of support. This involve assembly of technological by the example of the interactive is important, because complex elements and practices, most of which whiteboard in Section 5, relevant innovations typically require already exist, into novel configurations, practices include, but are not decades for effective diffusion. applied in new settings. limited to, pedagogical practices.

30 Beyond Prototypes: Enabling innovation in technology-enhanced learning 6

and then consider which tools and materials will be required to achieve Although TEL research produces novel technologies and their goals. Rather, they review their available materials and tools and work pedagogies, such work is only a small part of TEL innovation. out how to use them to achieve their goal or something close to their goal [80]. Above all, bricolage is rooted Although TEL research produces happens on the simulator. You in engagement with the concrete novel technologies and pedagogies, know, when we first start, the properties of a situation and the such work is only a small part of hardest thing is to get people available materials, rather than with TEL innovation and should be seen to accept that to be successful an abstract model of how they will as just one component of broader they’ve got to change their behave. For Lévi-Strauss, bricolage configurational work. lifestyle. [Alex Hawkridge, chairman does not only apply to the material of iZone Driver Performance] but also to the realm of ideas and 6.2.2 Social practices as part of the social practices. TEL complex The technologies afford new The Beyond Prototypes case studies developments in coaching practices The ‘bricoleur’ is adept at and interviews foreground the role and these afford new ways of performing a large number of that is played by social practices, not configuring the technologies. diverse tasks; but, unlike the just as a context for TEL innovation engineer, he does not subordinate but as important elements in the The TEL innovation process thus each of them to the availability of configuration of TEL innovations. For involves many different stakeholders, raw materials and tools conceived example, in Mitch Resnick’s account of all of whom are embedded in and procured or the purpose the development of Scratch quoted distinct communities with different of the project. His universe of in Section 4.1, the importance of expectations and understandings instruments is closed and the prior work on related technologies of TEL and of learning and teaching. rules of his game are always to such as Logo is evident. However, Selected technical elements, specific make do with ‘whatever is at hand’, a pivotal element of this case is the pedagogic ideas and desired practices that is to say with a set of tools engagement with and understanding have to be pulled together into and materials which is always of children’s informal learning effectively working bundles, drawing finite and is also heterogeneous practices in programming clubhouses, on contributions from the disparate because what it contains bears no and the later appropriation of social stakeholder communities. All these relation to the current project, or networking practices to support bundles of distinct elements have to indeed to any particular project, social learning. Indeed, the principal be addressed in order for innovation but is the contingent result of all innovation of Scratch lies less in the to take place. This process can be the occasions there have been nature of the programming language characterised as bricolage. to renew or enrich the stock or than in the configuring of Scratch to maintain it with the remains and a social platform to engage 6.2.3 Bricolage of previous constructions or with the informal learning and social The anthropologist Lévi-Strauss destructions. [81:11] networking practices of young people coined the word ‘bricoleur’ to that have emerged in relation to describe someone who makes do Lévi-Strauss described bricolage as computer games. with whatever is at hand. Bricoleurs characteristic of primitive societies do not typically start a project and contrasted it with a scientific Similarly, in the case of iZone (see boxed case study), existing technologies from motor racing, flight simulation, eye-tracking and sports The ‘bricoleur’ is adept at performing a large number of diverse science have been brought together with a particular set of coaching tasks; but, unlike the engineer, he does not subordinate practices and a deep understanding each of them to the availability of raw materials and tools of the practices of racing drivers. conceived and procured or the purpose of the project. There’s as much effort goes into training in this boardroom as

tel.ac.uk 31 6

Case study: iZone Driver Performance

iZone was set up in 2009 to address a change in Fédération Internationale de l’Automobile (FIA) regulations, which reduced racing teams’ testing time. While test equipment and simulators for the testing of cars and components were already used, nothing was available that could replace track time for drivers. Sophisticated simulators with video screens had been developed over the previous 35 years, but much more complex systems, able to provide physical feedback such as g-forces, were required for the development of elite drivers.

iZone addressed this problem by interlinking physiological systems and electromechanical systems. It uses eye-tracking technology to enable coaches to analyse drivers’ performance and assess their control during the simulation. This technology was developed by the company’s simulator designer, John Reid, who was inspired by an article about the use of eye-tracking systems in helicopter gunships.

iZone has links with Cranfield Aerospace that stretch back to the 1980s, when company chairman Alex Hawkridge used the wind tunnel at Cranfield to develop the aerodynamics of Toleman F1 cars. The company now uses the g-force technology from Cranfield’s helicopter trainer and also has PhD students from Cranfield working with the company on aspects of the project. A similar long-term relationship with the Department of Electrical Engineering at the University of Sheffield has also helped with the development of the simulator.

Based on work with racing drivers prior to setting up iZone, the team has created a training regime developed by sports scientists and sport psychologists to offer a complete driver development programme that includes the use of the simulators. The sport psychology input came from Dave Collins, who had developed a name for coaching and mentoring in athletics and football as well as in motorsports.

Most technology businesses are concerned with the protection of intellectual property (IP), but Alex Hawkridge’s view is that, ‘the things that are patentable, we don’t think it would be wise to patent, because you then tell people exactly what you’re doing.’ He considers that the most important way to protect the business’s IP is to keep developing the simulator business. The potential for iZone to run a similar operation at every major racetrack in the world is a real opportunity; a future way forward might include franchising the model in order to maintain its speed of development. approach in modern societies. 6.2.4 Bricolage and ‘persistent A research group is not equipped However, subsequent authors intent’ within the TEL space to take on all the other factors have built on his work to show the Bricolage is a core theme that that are required in order to importance of bricolage to areas has emerged from the Beyond scale, to move from a research of current society as diverse as Prototypes study. The narratives project or pilot to a scaled one scientific endeavour [82], product produced by case studies and […] They are lulled into thinking design, entrepreneurship [83, interviews reveal successful TEL that when they have a successful 84], social entrepreneurship [85], innovators not simply as inventors pilot the next step will be easy. financial economics [86] and the or as scientists proposing and The next step is the hardest enactment of social change [25]. testing hypotheses but also as step of all. When they go to These streams of research have bricoleurs who achieve educational schools with their piece of kit drawn on the notion of bricolage in goals by bringing together and their wonderful technology order to understand how innovation diverse technological elements, [they fail because] other factors and change come about through frameworks and social practices. such as curriculum, professional the creative reinterpretation and The ultimate success of TEL lies in development, sustainability and arrangement of existing social the implementation of the entire appropriateness are not taken practices and resources, and how ‘TEL Complex’ modelled in Figure 1. into consideration [Elliot Soloway, this process enables new possibilities However, much innovative work fails founder of the Center for within the constraints of existing to be implemented, because it does Highly Interactive Computing social systems and institutions. not take the complex into account. in Education]

32 Beyond Prototypes: Enabling innovation in technology-enhanced learning 6

Each element of the complex requires explicit and careful consideration Figure 2: Beyond Prototypes model of the TEL Innovation Process in order to avoid failure and maximise chances of success. This process of TEL bricolage does not take place at a single point in the Vision of process of innovation process but educational extends throughout the process. change TEL innovation, as is typical for

any complex example, can require G N Understanding Practical research I decades for full diffusion and during K ecology of practices and expertise R P that time researchers engage in O E R W bricolage as they work towards their S T I

E S

T evolving vision of the development N

Technology E

N of learning and teaching. This D Technology R&D in

developments in T N BRICOLAGE

educational field

A

I

involves not only the combination of

wider society N

N

T

resources but also the development E O

I

N

T

and assembly of a stock of resources, T A

R in addition to the development of O

L Brokering, Developing practices

a close understanding of the nature P

X covening, influencing E in parallel to and affordances of what is at hand and taking what the format education [87]. This process lies at the heart system will give of the TEL Innovation Process. Co-constructing new practices in formal education

6.3 The TEL Innovation Process Literature based on extensive experience in various business settings by no means a necessary condition. Instead the vision often emerges illustrates that multiple factors and Some elements from that process are and evolves through exploration, issues have to be attended to in order expanded below, along with a design through networking and through for a new technology or practice methodology that encompasses the active engagement in research, to be employed effectively within a entire TEL Innovation Process. development and educational complex. Many authors have observed practice. Visions of educational this fundamental characteristic of the 6.3.1 Vision of educational change change are co-created through implementation of innovation, yet Generating change in educational engagement with different aspects it is nearly always overlooked when practices that is more than local of the TEL Complex. new technologies are developed [See, for example, 14, 73, 88, 89-91]. In the context of innovation in educational multimedia, Van Lieshout and his Vision often emerges and evolves through exploration, through colleagues have identified innovation as a process of ‘social learning’ [92]. networking and through active engagement in research, development and educational practice. Figure 2 presents TEL innovation as a process of bricolage that involves the assembly of technological elements and social practices and temporary is difficult to do and 6.3.2 Pedagogical research to inform a complex process of demands persistent intent over time. and expertise innovation that has the aim of This, in turn, requires a clear vision of Engagement in research into achieving educational goals. As what could and should be achieved. educational technologies and noted above, while the invention However, the Beyond Prototypes pedagogy has an important role of new technological elements or case studies and interviews show that to play in TEL innovation. The pedagogic approaches may be a a clear vision is rarely the starting direct products of this research component of such innovation it is point for the innovation process. are important but so too are the

tel.ac.uk 33 6

means that researchers and practitioners can come, over time, It is important that this TEL Innovation process is aligned to mutual understanding and respect for the ways in which with a research model that supports evaluation of what theory-informed TEL can be has been achieved, and that can build on previous findings. enacted in real classrooms and other educational settings.

A notable consequence of the connections and expertise that learning practices. Increasing public complexity of TEL, representing a are created during the research and political concern about school major developmental opportunity, process. The Beyond Prototypes leavers’ lack of programming skills is that there is scope for ‘user-driven’ study also highlights another crucial has created opportunities to contributions from both teachers form of research. This is research translate the use of Scratch into and students. Making use of these that is aimed at understanding the formal learning contexts. In the contributions requires engagement ecology of practices with which process it becomes something with users and a willingness to accept a particular TEL innovation must new. Scratch as a TEL complex initial proposals that are sufficiently engage. Examples from the case is different when enacted in an unfinished or unpolished to allow for studies included in this report informal learning setting to its effective intervention and ownership include young people’s relationships enactment in a formal learning to take place. Such engagement may to storytelling practices in the setting and the process of also be a necessary condition to Yoza case, financial traders’ trading translating the complex from one properly understand the ecology practices and learning practices setting to another is non-trivial. of practices that will be the context in relation to xDelia, and children’s for any particular TEL innovation. social networking practices in 6.3.4 Co-constructing new practices relation to Scratch. Researchers bring models and In order to ensure that TEL theories of learning to the innovation is evidence driven, it is 6.3.3 Developing practices in innovation process. These are important that this TEL Innovation parallel to formal education refined through working with process is aligned with a research Because it is difficult to achieve rapid teachers and learners, together model that supports evaluation and significant innovation within developing an understanding of of what has been achieved, and formal education sectors, successful how the ecology of the educational that can build on previous findings. innovations may impact first on setting impacts upon these models A core methodology is therefore informal learning practices. In some and theories. This co-construction design-based research. cases, this can provide a platform for translating the innovation into formal education. Table 1: Comparison of design-based research with experimental studies

To take one of the TEL successes Experimental studies Design-based studies described in Section 4 as an example, Laboratory studies Real-world situations that contain Scratch is a complex made up of limitations, complexities and dynamics software, hardware platform(s), informal learning practices, social Aimed at testing hypotheses Aimed at designing new interventions learning practices, as well as many and generating hypotheses other elements. This TEL complex Usually single dependent Multiple dependent variables is enacted within an ecology of variable (though not all are investigated) practices that includes elements such as software security practices Control of variables, through Iterative and flexible revisions of and the friendship networks (online specification of fixed procedures the research design and offline) of the target audience Typically isolated from the Typically involve social interactions in its initial informal learning social world instantiation. In this case, it was Researchers are the Partners contribute to the initially enacted within the context decision makers decision making of a specific ecology of informal

34 Beyond Prototypes: Enabling innovation in technology-enhanced learning 6

6.3.5 Methodology of Design-Based of the research are embedded within toward refining both. Doing Research the system: not only in the design so thus calls for having a solid This methodology has been developed of the software but also in how it is theoretical perspective and over the past two decades and used by a growing and developing for possessing design skills, stems from ground-breaking work by community of practice. two talents rarely found in one Collins [93] and Brown [94]. These individual. This raises the issue researchers developed the idea of Schoenfeld writes of the way in which of design teams as part of the design experiments when they found design experiments work. research endeavor [98]. that traditional laboratory experiments were not sufficient to address the Properly construed, a design Table 1 compares design-based questions of interest to them. experiment consists of the studies with more conventional, creation of an instructional laboratory-based experimental The Design-based Research Collective intervention on the basis of studies. The experimental model asserts that ‘design-based research, a local theory regarding the works well when researchers are which blends empirical educational development of particular able to control the process of the research with the theory-driven understandings. The intervention research and isolate individual design of learning environments, is then examined with regard to variables. The design-based is an important methodology for the accuracy of the underlying approach aligns better with the understanding how, when, and why local theory and the power of model of the TEL innovation process educational innovations work in the intervention, with an eye that is set out in this report. ● practice’ [95]. Barab and Squires explain that

design-based research […] A design experiment consists of the creation of an was introduced with the expectation that researchers instructional intervention on the basis of a local theory would systemically adjust various regarding the development of particular understandings. aspects of the designed context so that each adjustment served as a type of experimentation that allowed the researchers to test and generate theory in naturalistic context [96:3].

Wang and Hannafin describe the approach in more detail as

A systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories [97:6].

In many projects this involves iterative cycles of designing pedagogy and technology, running an inquiry, and then carrying out evaluation and analysis that feed into the next cycle. In this way, some of the key findings

tel.ac.uk 35 7 IMPLICATIONS FOR RESEARCH

Section 7 examines the implications of this report for research. Persistent intent, engagement over time and the use of an appropriate methodology are identified as priorities. Successful research also requires engaging with the practices and stakeholders that must be taken into account if research-informed innovation is to be embedded in practice.

Previous sections have set out how concerned with computer-supported the TEL Innovation Process takes collaborative working, CSCW. I think place, key elements of this process and a lot of the thinking and findings recommendations for the future. This of CSCW research are very relevant section examines the implications of now, because the tools and systems these elements for researchers. to support computer-supported collaborative work are now As significant TEL innovation takes ubiquitous, which they weren’t when place over an extended period of time, the research was being done. When persistent intent on the part of any the research was being done, a lot of research team has been identified as effort had to be put into designing crucial. Imogen Casebourne, Director and sustaining the tools and services of Learning at the Educational in order for the research just to Program Innovations Center, traced happen. Now the communication the evolution of mobile learning from tools and systems are everywhere, initial research in the 1990s, through but the kind of ideas that CSCW the time when her company started research threw up, I think to some working in the area, to the present day: extent have lain dormant, and are not properly utilised. Because lots We started creating mobile learning of people coming to this technology about 10 years ago ourselves, using enhanced learning afresh now don’t PDAs [personal digital assistants]. realise that there’s a back story of It was only a few unusually forward- work that is very important. thinking clients for a very long time, who were interested in exploring it. The eventual line of development, and Whereas, obviously, in more recent the vision of innovation, is not always years it has really taken off. clear at the start. Early research may explore the affordances of a The same extended development set of technologies – such as mobile process has taken place with devices or online conferences – or computer-supported collaborative may investigate how technologies learning. Seb Schmoller, Chief Executive can be used to support a particular of the Association for Learning pedagogic approach. When Seymour Technology between 2003 and 2012, Papert worked in the 1970s and 80s provided an overview of the growth on ‘creating the conditions under of this area, illustrating that the TEL which intellectual models will take innovation process can take decades. root’ [36], there was no technology Significantly, he suggested that the available that could enable the timescale for this process can be so development of a programming extended that original research is language to be used collaboratively forgotten or under-utilised, indicating and synchronously by children a need for persistent engagement and across the world. Nevertheless, his intent over extended periods of time: work on constructionism and on programming with children deeply 15 years ago, there was a research influenced Mitch Resnick, and has field, though ‘field’ is perhaps informed his team’s development of too wide a term for it, which was Scratch at MIT during the last decade.

36 Beyond Prototypes: Enabling innovation in technology-enhanced learning 7 IMPLICATIONS FOR RESEARCH

Another way in which TEL innovation Diana Laurillard, Professor of Learning intelligence that can provoke or extends over time is in the tension with Digital Technologies (LKL): encourage practical action [101]. between creative and practical These analytics help educators and approaches. Early research may You improve the impact of an learners ‘to increase the degree employ ‘catwalk technologies’ that innovation by looking at what are to which our choices are based on demonstrate an exciting new concept the drivers in education, what evidence rather than myth, prejudice but that are also costly, difficult to makes people sit up and worry, and or anecdote’ [102]. The large numbers maintain and often impractical for it’s funding flows, it’s curriculum of participants in MOOCs allow rapid extended use [99]. Successful mass requirements, it’s assessment and cycles of A/B testing in which users exploitation requires a ‘ready-to- it’s quality. are randomly exposed to one of wear’ system that can be used off the two variations of a TEL innovation shelf without problems. Researchers This work on understanding the – control A or treatment B – with therefore need to take into account environment is crucial. Seb Schmoller changes in outcome explained by how this shift will take place, who will of ALT noted that, ‘If your organisation this assignment, leading to insights carry it out, and how it will be funded. wants to do things differently, you need for further development [103]. The to do them differently in a way that will development of one TEL innovation Engaging with user communities work within the rules and frameworks may require the use of many methods. from the start has the potential to that govern the ‘ecosystem’ you are make them genuine stakeholders in in. It’s easy to try to do things that new knowledge. The Epic Learning ultimately just won’t work because they 7.1 Recommendations Group, a global provider of learning do not conform to those rules and for researchers technologies, takes a consultative frameworks’. A technology is unlikely • Research teams should identify, at approach; staff work as partners to result in significant change unless it an early stage, the steps required or advisers with customers in is connected with shifts in pedagogy to enable scalable and sustainable order to develop appropriate and and practice. Steve Vosloo, senior implementation beyond prototypes, affordable solutions. Jeffrey Lins, project officer in mobile learning at so as to enhance learning. head of research and innovation UNESCO, observed • Researchers need to engage at Saxo Bank and a member of the fully with the individuals and xDelia consortium, stresses that it is I have seen many cases where communities that will play a role important for companies to take some computers are put into a school in the implementation process. degree of ownership of research and or into a computer lab and the • Research teams should consider development, so that these are not teaching, the learning, the whole adopting Design-Based Research disconnected from implementation. paradigm, has not changed at all. as a systematic but flexible The only thing that’s different is that methodology for research-led We designed a project [XDelia] it’s being done through a keyboard, innovation, based on collaboration that was aimed at a particular and not on pen and paper. among researchers and ecosystem. We knew it existed; we practitioners in real-world settings. didn’t theorise that it would exist An important role for researchers • The interim and final results from sometime somewhere, or did exist within the TEL innovation process is design-based studies should be somewhere. We knew it existed, we to ensure that research is evidence systematically shared with other knew basically what that ecosystem based. This means employing a researchers so that the process was about and I think we had a methodology that is appropriate of innovation can be compared, clear vision for how we connected to the process. Section 6.3.5 expanded, and continued over into that ecosystem. And that was identified design-based research time. They should also be widely a lot of the power of the project. as a key approach. However, other disseminated to policy makers and [Jeffrey Lins, head of research and new possibilities are opening up practitioners, through events such innovation at Saxo Bank] as technology-enhanced learning as ‘what research says’ meetings. expands its scope. Learning analytics, • Research institutes should set Continuing engagement helps ‘the measurement, collection, analysis up long-lasting collaborations researchers to gain an understanding and reporting of data about learners and consortia, involving schools, of the environment in which their and their contexts, for purposes of museums and other educational work will be implemented, and to be understanding and optimizing learning settings as test-beds, to support clear what has to happen before an and the environments in which it large-scale comparative and cross- innovation can be applied in practice. occurs’ [100], provide actionable cultural investigations. ●

tel.ac.uk 37 8 IMPLICATIONS FOR POLICY AND FOR FUNDING

Section 8 examines the implications of this report for policy and for funding, identifying current problems, particularly in relation to sustainability, and proposing solutions.

Some past policy initiatives have learning and computer-supported supported the TEL innovation collaborative learning, such projects process well. The Microelectronics can build over a period of years into a Education Programme, established body of work that is used to transform in 1980, ran for six years with the learning and teaching. However, this is aim of preparing children for a world an uncertain process of development in which microelectronics would that is not ideal. Impact outside be commonplace and pervasive. academia would be supported by Coupled with an initiative that made changes to current funding models money available for schools to buy in order to support long-term computers, this policy drove a well- engagement and sustainability. structured process of TEL innovation that extended over time and took Seb Schmoller of ALT identified a into account the changes necessary significant problem with externally throughout the TEL complex. During funded projects: the 1990s, the Teaching and Learning Technology Programme provided I will stand more chance of impetus for the adoption of TEL successfully innovating if I try across the university sector. As with to innovate within the general the MEP, this programme took into constraints and parameters under account the different elements of which I and the organisation are the TEL complex and was therefore expected to operate, rather than able to provoke change across the by making use of some temporary educational system. external funding that can be used to stimulate activity; because once the funding dries up the stimulus A project makes a short-term difference but then the funding is removed and the activity ceases. A project makes a short-term runs out. Without support and maintenance, the successful difference but then the funding runs innovation begins to wither. out. The research and development team disbands and moves on to other funded projects. Without support and maintenance, the successful No such system-wide initiative innovation begins to wither. At is currently in place; the closest this stage, it is not clear how the equivalent is the move to introduce work should be taken forward. One programming within schools through option would be to move towards changes to the curriculum. The current commercialisation, but this presents research funding system within the UK several problems. As Section is not aligned with the TEL innovation 5.6 explained, TEL success is not process, though it is well suited to necessarily commercial success. supporting short- and medium- term projects capable of producing At the most basic level, there may be results and academic publications nothing to commercialise. A change that help universities to build a strong in pedagogy or practice is unlikely submission for government audits such to be a marketable commodity; a as the Research Excellence Framework new technology without a change (REF). In some cases, as with mobile in pedagogy or practice is unlikely

38 Beyond Prototypes: Enabling innovation in technology-enhanced learning 8 IMPLICATIONS FOR POLICY AND FOR FUNDING

to have a significant impact. If the A funding package that focuses 8.1 Recommendations for research project does result in a solely on the development of funders and policy makers potentially marketable innovation, it is an artefact cannot adequately • Policy and funding should unclear who should take that forward. support this process. Focused, support innovations in pedagogy The primary role of universities is not persistent intent is needed in order and practice, as well as the to market products. Researchers are to encourage teams of researchers technological developments that not trained in marketing and are not to extend and develop their work will support these. This should recruited for their entrepreneurial over time, with a shared goal in mind. recognize the need to fund vision and spirit. As individuals, Persistent intent has the potential professional development of they may not have the skills, the to focus researchers’ attention on practitioners and evaluation interest, or the legal right to take an the context in which their work takes of the innovation in practice. innovation developed by a team any place, encouraging them to develop • Policy and funding should further. They may also be limited in the skills necessary to work with recognize the importance of their choice of business model by people in different contexts and extended development and provide funders’ requirements that their work bridge the gaps between them. support for scaling and sustaining should be freely and openly available. of innovations, beyond prototypes Knowledge transfer partnerships into educational transformations. A better option is to ensure that the have a role to play here. However, • Policy and funding should innovative development is sustainable. they currently stress ‘the transfer encourage the development In order for this to happen, the of knowledge, technology and skills of skilled, multidisciplinary project must have engaged people to which the company partner teams that are able to complete who are willing and able to support currently has no access’ [104]. There the TEL innovation process. its continued success. The success is less emphasis on the non-financial Recognition and support should of the Scratch project, for example, benefits gained by the university be given to visionary thinking and is strongly connected with its partner. Jeffrey Lins, head of experimentation, to generate fresh development of online and face-to- research and innovation at Saxo insights and achievable visions of face user communities. On a smaller Bank, commented that educational developments. scale, organisations and individuals • There is a need to build research need the sense of ownership that the boundaries have to be blurred, capacity in TEL within the UK. The comes from working closely with and that comes from respect on research councils should give a researchers in order to develop an both sides. Commercial entities clear indication as to where TEL innovation that works in context. need to realise that universities proposals should be submitted, and This may not be an artefact. As Steve actually do understand that things ensure that proposals are reviewed Vosloo, senior project officer in mobile cost money, and how the business by people with appropriate learning at UNESCO, notes: world works and what customers expertise in TEL research. are like and these kind of things, Evaluation criteria should include You can look at other facets of because they do, and they model successful implementation of plans the broader learning experience. these things and they are intensely for scaling and sustainability. There is some administration that interested in understanding them. • Funders should provide support needs to happen, there is some On the other hand, universities to research teams to evaluate assessment that needs to happen, have to understand that there’s their innovations for educational there is content to be delivered. a lot of competent research, impact and transformation, through Perhaps technology can help speed researchers and research appropriate qualitative and/or up the assessment process or the capabilities outside their walls. quantitative methods. New methods administration process. This is not of evaluation, such as learning learning, per se. But it creates a The commercial partner is crucial in analytics should be encouraged. more efficient whole experience providing the ability to contextualise • Findings must be made available to that could allow the teacher or a problem and to understand its other researchers and developers, the learner more time to actually ecology. At the same time, universities including those without access teach and learn. So it’s a kind of offer ways of reframing problems to university libraries, so that system strengthening or efficiency- and identifying new perspectives. the results of research and improving measure. But if you only There is knowledge on both sides development can be used to think of the learner and the learning of the partnership that needs to be continue and complete the experience, you don’t get that. translated and transformed. innovation process. ●

tel.ac.uk 39 9 CHALLENGES AND OPPORTUNITIES

Section 9 focuses on the way forward for TEL research. It identifies a continuing need for sustained building of capacity in TEL, through graduate programmes and investment in national hubs of expertise that share talent and facilities.

Education is a major export for As this report has shown, research the UK economy, estimated to associated with technology and be worth £17.5 billion in 2011 learning has influenced and been [105]. Technology for learning influenced by other sectors, resulting now forms an integral part of in some surprising benefits to the UK that educational export market, economy. Arm Holdings, the major including direct income from British semiconductor and software publishing of e-books, online design company grew out of the work learning, and educational in the 1980s by Acorn computers to software, plus indirect benefits build the BBC Microcomputer as part from competitive advantage in of the Microelectronics Education MOOCs, educational analytics, Programme. During the same period, online learning resources, and thousands of teenagers learned blended and mobile learning. to play and to program games on

Research associated with technology and learning has influenced and been influenced by other sectors, resulting in some surprising benefits to the UK economy.

This innovation needs to be microcomputers such as the Sinclair continually refreshed in order to Spectrum, developing talent that maintain a competitive edge in initiated the UK computer games the combination of technology industry. At the same time, The and pedagogy. A year ago there Open University was developing its was no major UK investment in distinctive approach to supported massive open online learning. online learning that combines Now, 23 UK universities have made computer-delivered materials with a substantial strategic commitment human tutorial support. to the FutureLearn company. They are offering free education to Now there is an opportunity for hundreds of thousands of people similar confluences of research-led worldwide, in part as a means of innovation in learning and technology, attracting overseas students to around massive online learning, register for UK degree courses. mobile learning, haptic technologies The pace of change may be for learning, learning design, learning quickening, as education enters analytics, technology-based science, a similar period of disruptive maths and computing education, innovation to that faced by the interactive e-books and multimedia entertainment and banking sectors educational publishing. Some of ten years ago, and as the nation these are altering traditional sectors develops its understanding of how such as publishing and universities; to develop effective learning with others are opening new business technology at large scale. opportunities in educational software

40 Beyond Prototypes: Enabling innovation in technology-enhanced learning 9 CHALLENGES AND OPPORTUNITIES

The continuing need is for neither abstract grand challenges nor short-term initiatives, but for a sustained building of capacity in technology-enhanced learning, through graduate programmes and investment in national hubs of expertise that share talent and facilities.

applications, and technologies such colleges and museums. The prize as haptic simulators. will be a sustained ability to do ‘big R&D’ that develops substantial These developments are educational systems over a prolonged fundamentally interdisciplinary. period and evaluates them with a They can only take place through range of learners in informal and the combined efforts of topic formal settings. specialists, technologists, and experts in teaching, learning and The focus for future TEL assessment. The UK still lacks research should be on effective expertise in the linking discipline transformation of educational of educational technology. The practices, rather than small continuing need is for neither incremental improvements, abstract grand challenges nor short- and on how these transformations term initiatives, but for a sustained can be scaled and sustained. We building of capacity in technology- need to design new forms of learning enhanced learning, through graduate that people (teachers, students and programmes and investment in informal learners) want to adopt national hubs of expertise that and use. The evidence presented share talent and facilities. in this report shows that research- led innovation in TEL is a complex To compete with other national TEL process; the big successes cannot centres such as SRI International be predicted but they can be in the USA, Nanyang Technological nurtured through a supportive University Singapore, National environment that co-constructs Central University Taiwan, and École learning and technology, supports Polytechnique Fédérale de Lausanne the persistent intent of visionaries, in Switzerland, the UK needs to pool subjects educational innovations to resources across universities active systematic evaluation, and partners in TEL research, involving innovative with innovators in the education companies and partner schools, sector and in creative industries. ●

The focus for future TEL research should be on effective transformation of educational practices, rather than small incremental improvements.

tel.ac.uk 41 BIBLIOGRAPHY

[1] Charmaz, K. Constructing Grounded Theory: A Practical [17] Laurillard, D. Rethinking University Teaching: A Guide through Qualitative Analysis Sage, London, 2006. Conversational Framework for the Effective Use of Learning Technologies. RoutledgeFalmer, New York, 2002. [2] Strauss, A. and Corbin, J. M. Basics of Qualitative Research: Grounded Theory Procedures and Techniques Sage, [18] Katz, D. and Kahn, R. L. The Social Psychology of Newbury Park, 1990. Organizations. John Wiley & Sons, 1978.

[3] Ryan, G. W. and Bernard, H. R. Techniques to identify [19] Scott, W. R. Institutions and Organizations (2nd ed.). Sage, themes. Field Methods, 15, 85-109, 2003. Thousand Oaks, CA, 2001.

[4] Riessman, C. K. Narrative Analysis Sage, Newbury Park, 1993. [20] Fenton-O’Creevy, M., Gooderham, P. and Nordhaug, O. Human resource management in US subsidiaries in Europe [5] Crook, C. and Harrison, C. Web 2.0 Technologies for and Australia: centralisation or autonomy? Journal of Learning at Key Stages 3 and 4: Summary Report. Becta, International Business Studies, 39, 151-166, 2008. Coventry, http://www.becta.org.uk 2008. [21] Fenton-O’Creevy, M., Gooderham, P., Cerdin, J.-L. and [6] Noss, R., Cox, R., Laurillard, D., Luckin, R., Plowman, L., Ronning, R. Bridging roles, social skill, and embedded Scanlon, E. and Sharples, M. System Upgrade: Realising knowing in multinational organisations. In Politics the Vision for UK Education. Available at http://telit.org.uk/, and Power in the Multinational Corporation: The Role http://telit.org.uk/ 2012. of Institutions, Interests and Identities, edited by C. Dorrenbacher and M. Geppert. Cambridge University [7] Gao, N. TEL-isphere 1999: The Caribbean & Technology- Press, Cambridge, 2011. Enhanced Learning. Humanities and Social Sciences Net. http://h-net.msu.edu/cgi-bin/logbrowse.pl?trx=vx&list=ED [22] Fenton-O’Creevy, M. Middle management resistance TECH&month=9904&week=c&msg=TmqV6TPCYpeMPw/ to strategic change initiatives: saboteurs or scapegoats? UHTlrKw 1999. In Managing Strategy Implementation : An Organizational Behaviour Perspective, edited by P. C. Flood, T. [8] Fothergill, R. The Microelectronic Education Programme Dromgoole, T. Carroll and L. Gorman. Blackwell Business, Strategy (April) http://www.naec.org.uk/organisations/ Oxford, UK, 2000. the-microelectronics-education-programme/the- microelectronics-education-programme-strategy. 1981. [23] Ball, S. J., Maguire, M. and Braun, A. How Schools Do Policy: Policy Enactments in Secondary Schools. [9] Her Majesty’s Inspectorate The work of the Routledge, 2012. Microelectronics Education Programme. Studies in Design Education Craft and Technology, 19, 3, 160-163, 1987. [24] Creed, W. E. D., Scully, M. A. and Austin, J. R. Clothes make the person? The tailoring of legitimating accounts and the [10] European Commission. Open Consultation: New Research social construction of identity. Organization Science, 13, 5, Challenges for Technology Supported Learning. Available 475-496, 2002. at http://cordis.europa.eu/ist/telearn/consultation.htm 2001. [25] Fligstein, N. Social skill and the theory of fields. [11] STELLARnet Concept and Project Objectives. http://www. Sociological Theory, 19, 2, 105-125, 2001. stellarnet.eu/about/ 2013. [26] Edxus Group Europe’s 20 Fastest Growing and Most [12] Sutherland, R., Eagle, S. and Joubert, M. A Vision and Innovative e-Learning Companies Named http:// Strategy for Technology Enhanced Learning: Report from edxusgroup.com/europes-20-fastest-growing-and-most- the STELLAR Network of Excellence. STELLAR Network of innovative-e-learning-companies-named-2/ 2013. Excellence 2012. [27] Mayes, T., Morrison, D., Mellar, H., Bullen, P. and Oliver, [13] Vavoula, G. KLeOS: A Knowledge and Learning M. Transforming higher education through Technology Organisation System in Support of Lifelong Learning. PhD, Enhanced Learning The Higher Education Academy, University of Birmingham, Birmingham, UK, 2004. York, http://www.heacademy.ac.uk/assets/documents/ learningandtech/Transforming.pdf 2009. [14] Keeley, L., Pikkel, R., Quinn, B. and Walters, H. Ten Types of Innovation: The Discipline of Building Breakthroughs. [28] Li, Q. and Ma, M. A meta-analysis of the effects of computer Wiley, Hoboken, NJ, 2013. technology on school students’ mathematics learning. Educational Psychology Review, 22, 3, 215-243, 2010. [15] O’Toole, R. Pedagogical strategies and technologies for peer assessment in Massively Open Online Courses (MOOCs). [29] Sharples, M., Arnedillo-Sánchez, I., Milrad, M. and Vavoula, Discussion Paper. University of Warwick, Coventry, 2013. G. Mobile learning: small devices, big issue. In Technology- Enhanced Learning: Principles and Products, edited by N. [16] Dewey, J. Experience and Education. Macmillan, London & Balacheff, S. Ludvisgsen, T. de Jong, A. Lazonder and S. New York, 1938. Barnes. Springer, 2009.

42 Beyond Prototypes: Enabling innovation in technology-enhanced learning BIBLIOGRAPHY

[30] Pilkington, R. M. Measuring the impact of information [44] Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, technology on students’ learning. In International N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, Handbook of Information Technology in Primary and E., SIlver, J., SIlverman, B. and Kafai, Y. B. Scratch: Secondary Education, edited by J. Voogt and G. Knezek. Programming for All. Communications of the ACM, Springer, New York, 2008. 52, 11, 60-67, 2009.

[31] Borgman, C. L., Abelson, H., Dirks, L., Johnson, R., [45] Papert, S. and Solomon, C. NIM: A Game-Playing Koedinger, K. R., Linn, M. C., Lynch, C. A., Oblinger, D. G., Program. AI Memos (1959 - 2004), http://18.7.29.232/ Pea, R. D., Salen, K., Smith, M. S. and Szalay, A. Fostering handle/1721.1/6199?show=full, 1970. Learning in the Networked World: The Cyberlearning Opportunity and Challenge. NSF Taskforce on [46] Papert, S. Teaching Children To Be Mathematicians vs. Cyberlearning, http://www.nsf.gov/pubs/2008/nsf08204/ Teaching about Mathematics. AI Memos (1959 - 2004) nsf08204.pdf 2008. http://dspace.mit.edu/bitstream/handle/1721.1/5837/AIM- 249.pdf?sequence, 1971. [32] Skinner, B. F. Teaching machines. Science, 128, 969-977, 1958. [47] Kukulska-Hulme, A., Sharples, M., Milrad, M., Arnedillo- [33] Engelbart, D. C. Augmenting Human Intellect: A Sánchez, I. and Vavoula, G. Innovation in mobile learning: Conceptual Framework. Contract AF49(638)-1024, a European perspective. International Journal of Mobile Stanford Research Institute: for the Air Force Office and Blended Learning, 1, 1, 13-35, 2009. of Scientific Research, Washington, Menlo Park, California 1962. [48] Walsh, C. and Shaheen, R. English in Action (EIA): mobile phones as an agent of change for large-scale [34] Kay, A. and Goldberg, A. Personal Dynamic Media. teacher professional development and English language Computer, 10, 3, 31-41, 1977. learning in Bangladesh. In Proceedings of American Educational Research Association Annual Conference [35] Negroponte, N. The $100 laptop. In Proceedings of San Francisco, CA., 27 April – 1 May, San Francisco, Globalization and Education: Joint Working Group, the CA., 2013. Pontifical Academy of Sciences, the Pontifical Academy of Social Sciences 16-17 November, Walter de Gruyter, 2007. [49] Liyanagunawardena, T. R., Adams, A. A. and Williams, S. A. MOOCs: A Systematic Study of the Published Literature [36] Papert, S. Mindstorms: Children, Computers and Powerful 2008-2012. 2013. Ideas. Basic Books, Cambridge, MA, 1980. [50] Van Dusen, L. M. and Worthen, B. R. The impact of [37] Berners-Lee, T., Fischetti, M. and Dertouzos, M. L. Weaving integrated learning system implementation on student the Web: The Original Design and Ultimate Destiny of outcomes: implications for research and evaluation. the World Wide Web by Its Inventor. Harper Information, International Journal of Educational Research, 21, 1, 13-24. 2000. [51] Mercer, N., Gillen, J., Kleine Staarman, J., Littleton, K. [38] Traxler, J. Mobile Learning – Confronting Reality In and Twiner, A. Interactive whiteboards: does new Proceedings of the ALT-C Manchester, UK http://altc2012. technology transform teaching? In Learning across Sites: alt.ac.uk/talks/28124, 11-13 September, 2012. New Tools, Infrastructures and Practices, edited by S. R. Ludvigsen, A. Lund, I. Rasmussen and R. Säljö. Routledge, [39] Rieger, R. and Gay, G. Using mobile computing to enhance London, 2011. field study. In Proceedings of CSCL ‘97 Proceedings of the 2nd international conference on Computer support for [52] Dillenbourg, P., Schneider, D. and Synteta, P. Virtual collaborative learning 1997. Learning Environments. In Proceedings of Third Hellenic Conference ‘Information & Communication Technologies [40] Sharples, M. The design of personal mobile technologies in Education’ Greece, Greece, 2002. for lifelong learning. Computers & Education, 34, 177-193, 2000. [53] Sharples, M., McAndrew, P., Weller, M., Ferguson, R., FitzGerald, E., Hirst, T., Mor, Y., Gaved, M. and Whitelock, [41] Kukulska-Hulme, A., Sharples, M., Milrad, M., Arnedillo- D. Innovating Pedagogy 2012: Open University Innovation Sánchez, I. and Vavoula, G. Innovation in mobile learning: a Report No. 1. The Open University, Milton Keynes 2012. European perspective. International Journal of Mobile and Blended Learning, 1, 13-35, 2009. [54] Sharples, M., McAndrew, P., Weller, M., Ferguson, R., FitzGerald, E., Hirst, T. and Gaved, M. Innovating Pedagogy [42] World Innovation Summit for Education BBC Janala. 2013: Open University Innovation Report 2. The Open http://www.wise-qatar.org/content/bbc-janala 2011. University, Milton Keynes 2013.

[43] Wichita State University Palm Pioneers. http://www. [55] World Climate Research Programme WCRP Strategy. education.wichita.edu/m3/help/kellogg2_files/frame.htm http://www.wcrp-climate.org/index.php/about-wcrp/about- 2003. strategy 2013.

tel.ac.uk 43

[56] Leaning, M. The One Laptop Per Child project and the [68] Pane, J. F., Griffin, B. A., McCaffrey, D. F. and Karam, R. problems of technology-led educational development. In Effectiveness of Cognitive Tutor Alegbra I at Scale. RAND High-tech Tots: Childhood in a Digital World, edited by I. R. Working Paper. Available at http://www.rand.org/pubs/ Berson and M. J. Berson. Information Age Publishing, 2010. working_papers/WR984.html 2013.

[57] D f E S . Harnessing Technology: Transforming Learning and [69] WWC. Carnegie Learning Curricula and Cognitive Tutor Children’s Services. Department for Education and Skills, Software. WWC Intervention Report. Available at http:// London 2005. ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_ cogtutor_083110.pdf 2010. [58] HEFCE. HEFCE Strategy for E-learning. Higher Education Funding Council for England, Joint Information Systems [70] Daugherty, L., Phillips, A., Pane, J. F. and Karam, R. Analysis Committee & Higher Education Academy London, http:// of Costs in an Algebra I Curriculum Effectiveness Study. management-class.co.uk/readings/05_12.pdf 2005. RAND, Available at http://www.rand.org/pubs/technical_ reports/TR1171-1.html, 2012. [59] HEFCE. Enhancing Learning and Teaching through the Use of Technology. Higher Education Funding Council [71] Rothwell, R. Towards the fifth-generation innovation for England, http://www.hefce.ac.uk/media/hefce1/pubs/ process. International Marketing Review, 11, 1, 7-31, 1994. hefce/2009/0912/09_12.pdf 2009. [72] Grandin, K., Wormbs, N. and Widmaim, S. (ed.) The [60] Haggard, S. The Maturing of the MOOC. Department for Science-Industry Nexus: History, Policy, Implications. Business Innovation and Skills, London, https://http://www. (Nobel Symposium 123). Science History Publications/USA, gov.uk/government/uploads/system/uploads/attachment_ Sagamore Beach MA., 2004. data/file/240193/13-1173-maturing-of-the-mooc.pdf 2013. [73] Fleck, J. and Howells, J. Technology, the technology complex [61] Leckhart, S. and Cheshire, T. University just got flipped: and the paradox of technological determinism. Technology how online video is opening up knowledge to the world. Analysis and Strategic Management, 13, 4, 523-531, 2001. Wired, 5: May 2012, 2012. [74] Nelson, R. R. and Winter, S. G. In search of useful theory [62] Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C. of innovation. Research Policy, 6, 1, 36-76, 1977. and Schmid, R. F. What forty years of research says about the impact of technology on learning a second-order [75] Teece, D. J. Business models, business strategy and meta-analysis and validation study. Review of Educational innovation. Long Range Planning, 43, 172-194, 2010. Research, 81, 1, 4-28, 2011. [76] Nenonen, S. and Storbacka, K. Business model design: [63] Goldacre, B. Building Evidence into Education. Department conceptualizing networked value co-creation. International for Education, London, https://http://www.gov.uk/ Journal of Quality and Service Sciences, 2, 1, 43-59, 2010. government/uploads/system/uploads/attachment_data/ file/193913/Building_evidence_into_education.pdf 2013. [77] Chesbrough, H. Business model innovation: opportunities and barriers. Long Range Planning, 43, 354-363, 2010. [64] Kirkwood, A. and Price, L. Technology-enhanced learning and teaching in higher education: what is [78] Fleck, J. Configurations: crystallizing contingency. ‘enhanced’ and how do we know? A critical literature International Journal of Human Factors in Manufacturing, review. Learning, Media and Technology. DOI 3, 1, 15-36, 1993. 10.1080/17439884.2013.770404, 2013. [79] Peine, A. Understanding the dynamics of technological [65] Dynarski, M. and Means, B. Effectiveness of Reading and configurations: A conceptual framework and the case Mathematics Software Products: Findings from the First of Smart Homes. Technological Forecasting and Social Student Cohort. US Department of Education, Available at Change, 76, 3, 396-409, 2009. http://ies.ed.gov/ncee/pdf/20074005.pdf 2007. [80] Lévi-Strauss, C. La Pensée Sauvage. Librarie Plon, Paris, [66] Empirical Education. Comparative Effectiveness of France, 1962. Carnegie Learning’s Cognitive Tutor Bridge to Algebra Curricilum: A Report of a Randomised Experiment in [81] Lévi-Strauss, C. Savage Mind. University of Chicago, the Maui School District. Empirical Education Research Chicago, 1962. Summary. Available at http://www.empiricaleducation. com/pdfs/ES-CognitiveTutor2.pdf 2007. [82] Turkle, S. and Papert, S. Epistemological pluralism: styles and voices within the computer culture. Signs, 16, 1, 128- [67] Morgan, P. and Ritter, S. An Experimental Study of 157, 1990. the Effects of Cognitive Tutor Algebra I on Student Knowledge and Attitude. Carnegie Learning. Available [83] Baker, T., Miner, A. S. and Eesley, D. T. Improvising firms: at. http://www.carnegielearning.com/web_docs/morgan_ bricolage, account giving and improvisational competencies ritter_2002.pdf 2002. in the founding process. Research Policy, 32, 2, 255-276, 2003.

44 Beyond Prototypes: Enabling innovation in technology-enhanced learning

[84] Garud, R. and Karnøe, P. Bricolage versus breakthrough: [99] Adams, A., FitzGerald, E. and Priestnall, G. Of catwalk distributed and embedded agency in technology technologies and boundary creatures. ACM Transactions entrepreneurship. Research Policy, 32, 2, 277-300, 2003. of Computer-Human Interaction, 20, 3, 15-40, 2013.

[85] Di Domenico, M., Haugh, H. and Tracey, P. Social [10 0] Siemens, G., Gašević, D., Haythornthwaite, C., Dawson, bricolage: theorizing social value creation in social S., Buckingham Shum, S., Ferguson, R., Duval, E., Verbert, enterprises. Entrepreneurship Theory and Practice, K. and Baker, R. S. J. d. Open Learning Analytics: An 34, 4, 681-703, 2010. Integrated and Modularized Platform (Concept Paper). SOLAR 2011. [86] MacKenzie, D. An equation and its worlds. Bricolage, exemplars, disunity and performativity in financial [101] Fe r g u s o n , R. Learning Analytics for Open and Distance economics. Social Studies of Science, 33, 6, 831-868, Education. Commonwealth Educational Media Centre 2003. for Asia (CEMCA), New Delhi, India, http://cemca.org.in/ ckfinder/userfiles/files/EdTech Notes_LA_Rebecca_15 May. [87] Duymedjian, R. and Rüling, C.-C. Towards a foundation pdf 2013. of bricolage in organization and management theory. Organization Studies, 31, 2, 133-151, 2010. [102] Cooper, A. What Is Analytics? Definition and Essential Characteristics. JISC CETIS, Bolton, http://publications. [88] Utterback, J. M. Mastering the Dynamics of Innovation. cetis.ac.uk/2012/521 2012. Harvard Business School Press, Boston, MA, 1994. [103] Kohavi, R., Henne, R. M. and Sommerfield, D. Practical [89] Swanson, E. B. Information System Implementation: guide to controlled experiments on the web: listen to Bridging the Gap between Design and Utilisation. Irwin, your customers not to the hippo. In Proceedings of 13th Homewood, Illinois, 1988. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2007. [90] Rhodes, E. and Wield, D. Implementing New Technologies: Innovation and the Management of Technology. Blackwell, [10 4] Technology Strategy Board Knowledge Transfer Oxford, 1985. Partnerships: FAQs. http://www.ktponline.org.uk/faqs/ 2013.

[91] Leonard-Barton, D. Wellsprings of Knowledge. Harvard [105] Department for Business Innovation and Skills. Business School Press, Boston, 1985. International Education: Global Growth and Prosperity. Available from http://www.gov.uk/bis 2013. [92] Van Lieshout, M., Egyedi, T. M. and Bijker, W. E. Social Learning Technologies: The Introduction of Multimedia in Education. Ashgate, 2001.

[93] Collins, A. Toward a design science of education. In New Directions in Educational Technology edited by E. Scanlon and T. O’Shea. Springer-Verlag, New York, 1992.

[94] Brown, A. Design experiments: theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2, 2, 141-178, 1992.

[95] Design Based Research Collective Design-based research: an emerging paradigm for educational inquiry. Educational Researcher, 32, 1, 5-8, 2002.

[96] Barab, S. A. and Squire, M. Design based research: putting a stake in the ground. The Journal of the Learning Sciences, 13, 1, 1-14, 2004.

[97] Wang, F. and Hannafin, M. J. Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53, 4, 5-23, 2005.

[98] Schoenfeld, A. H. Bridging the cultures of educational research and design. Educational Designer: Journal of the International Society for Design and Development in Education, 1, 2, 2009.

tel.ac.uk 45 ACKNOWLEDGMENTS

Many thanks to our Research team interviewees: Alex Professor Eileen Scanlon is Regius Professor of Open Education and Hawkridge, Diana Associate Director of Research and Innovation in the Institute of Educational Laurillard, Domic Savage, Technology at The Open University. She has extensive research experience Elliot Soloway, Imogen on educational technology projects in open education, science learning and evaluation. Casebourne, Jeffrey Lins, Jeremy Roschelle, Mitch Professor Mike Sharples is Professor of Educational Technology at The Resnick, Peter Scott, Roger Open University. He has over 30 years of experience in bridging research and exploitation of TEL, through projects with British Telecom, Racal, Kodak, Blamire, Sara de Freitas, Microsoft, Nokia, Sharp and FutureLearn. Seb Schmoller, Steve Vosloo and Thomas C Reeves. Professor Mark Fenton-O’Creevy is Professor of Organisational Behaviour with the Open University Business School. He is a senior fellow of the HEA and a National Teaching Fellow, with a recent track record of successful commercialisation of a major TEL project, the EU FP7-funded xDelia project.

Professor James Fleck is Professor of Innovation Dynamics at the Open University Business School and has researched innovation (most recently in e-learning) for 30 years, focusing specifically on issues of practical implementation. He was a member of the Academic Panel for the DTI/HTM Innovation Policy Review chaired by Lord Sainsbury.

Dr Caroline Cooban is Development Director at The Mighty Creatives, a creative development agency for children and young people, working across formal and informal settings. She also has over 15 years of experience of university research business development and knowledge transfer in the field of educational and learning technologies.

Dr Rebecca Ferguson is a lecturer in the Institute of Educational Technology. She has worked as research lead on educational technology projects including SocialLearn and EnquiryBlogger. Previously, she worked as editor and journalist on a range of publications, including the Radio Times.

Dr Simon Cross works in the Learning and Teaching Development Team in the Institute of Educational Technology. He has worked in research and coordination roles on external and internal projects with a focus on the open online learning space, learning design implementation and practice, badges and assessment.

Peter Waterhouse has worked in a variety of educational and business roles with particular experience in TEL, within both academia and industry. He has graduate qualifications in marketing as well as teaching and learning.

46 Beyond Prototypes: Enabling innovation in technology-enhanced learning ACKNOWLEDGMENTS

beyondprototypes.com TECHNOLOGY ENHANCED LEARNING RESEARCH PROGRAMME

London Knowledge Lab YouTube youtube.com/tlrptel Institute of Education Twitter @TLRPTEL University of London Email [email protected] 23-29 Emerald Street Phone +44 (0)20 7911 5577 London, WC1N 3QS tel.ac.uk