Going with the Grain

Going with the grain? Issues in the evaluation of educational innovations.

Charles Anderson, Kate Day, Jeff Haywood, Ray Land and Hamish Macleod Department of Higher and Further Education, University of Edinburgh.

Paper presented at 8th European Conference for research on Learning and Instruction, Gothenberg, 25-29 August 1999.

Abstract

Whilst considerable attention has been paid in both the evaluation and innovation literatures to the value of setting in context the object of enquiry and viewing it from differing perspectives, relatively little account has been taken of the related and important issue of what constitutes an appropriate grain of description. This article takes up the challenge of raising and exploring a number of distinct issues concerning the 'granularity' of an evaluation, and in the process sets out a framework to guide reflection on these issues. General concerns and points of principle are illustrated with reference to a recent large scale evaluation, conducted by the authors, of a programme promoting the use of ICT for learning and teaching within higher education.

Background

A central theme of the literature on educational evaluation in the last few decades has been the need to ensure that innovations are not assessed solely on the basis of a disembeded set of outcome measures; but studied within the wider contexts that frame them, with an eye to identifying the factors that constrain or enable their immediate uptake and longer-term existence within a school or a whole educational system. This stance concerning evaluation has been argued for from a number of different theoretical perspectives; but it can also be justified on the more pragmatic grounds of pointing to overwhelming evidence from existing studies of the ways in which innovations are powerfully shaped, or indeed 'rejected' by individual contexts. For instance, a recent study reported by Cole (1995, 1996) of the 'life history' of the "Fifth Dimension", a programme of computer-mediated after-school activities for children implemented in different sites, revealed very distinct differences across the sites studied in the sustainability of this particular programme – differences that were clearly linked to the pre- existing purposes, ethos and patterns of interaction in the individual sites. Agreement on the value of examining the wider context within which innovations are embedded can, however, mask important conceptual difficulties surrounding the definition of context and its identification1. Attention to context also confronts an evaluation team with difficult operational decisions in designing and implementing a particular study. How in practice does

1 A very clear overview of the conceptual problems surrounding the definition of context and an argument for a more relational interpretation of context, based around the metaphor of weaving, is provided in Cole (1996). A fruitful reconceptualisation of this construct of context that breaks away from the metaphorical view of context as a 'container' for actions has emerged from the work of scholars such as Lave (1988) and Engeström (1993) who draw on activity theory (Leont'ev, 1978). Viewed from within activity theory contexts can be conceived as integrated, complex systems; and Lave and Engeström stress how current systems of activity cannot be fully understood ahistorically in terms of the immediate actions of the participants but need to be set within a history of interactions and of culturally derived resources and goals. one separate out the 'figure' of an innovation from the 'ground' of its context and then view the relationship between them? To what extent does one focus on illuminating the immediate context of application of an innovation, or also attempt to consider the effects of more distal factors?

Grain of description

This article presents the ways in which, as a research group carrying out evaluations of the use of ICT in higher education, we have attempted to take account of the immediate and wider institutional contexts in which the innovations we have been studying were framed; and our reflections on how our attempt to view innovations from different perspectives affected the detail and type of description that it was possible and/or desirable to achieve. It will be argued, (largely by means of illustrative material from a recent evaluation project), that it may help evaluators to clarify their purposes and communicative intentions, if they direct attention explicitly to the question of the grain of description that is appropriate or feasible when situating innovations in context or presenting the perspectives of different actors on the processes and outcomes of an innovation. On this central theme, it is necessary first to consider the crucial matter of audience requirements.

Audience requirements

The question of providing a well-contextualised assessment of an innovation is complicated by the fact that evaluators are hardly ever, and some would argue should never be, able to aspire to the role of mythical neutral observers independently determining how they will depict the terrain that they are surveying. They must take into account the requirements of the evaluation’s audiences. Professional researchers, policy makers and practitioners have, in Bell and Raffe's useful phrase, different "normative world views of research" ( Bell and Raffe, 1991, p.133). In particular, judgements are likely to vary between these three constituencies as to what makes for a good evaluation report. Researchers will judge a report according to the ‘canons’ of sound professional practice, including the degree to which observations and claims are well supported by a full, clear presentation of evidence. Policy-makers are likely to want a succinct, synoptic report which makes appropriate recommendations and is guided by a realistic appreciation of the constraints within which policies have to be pursued. On the other hand, practitioners will see merit in reporting which focuses closely on the day-to-day challenges involved in implementing an innovation and provides insights and suggestions that are of clear practical relevance. Viewing the needs of practitioners from a different angle, educational psychologists are recognising once again that if the goal of improving practice is to be achieved serious attention needs to be given to the matter of the nature and level of description provided within studies of teaching and learning. For instance, Entwistle, McCune and Walker (2000) recognise that, with an audience of practitioners in mind judgments concerning the level of explanation and the choice of conceptual frameworks need to be guided both by "realism (to what extent is the explanation recognised by participants) and what has been called pedagogical fertility", i.e. the capacity to generate innovations within, or new perspectives on, learning and teaching (Entwistle and Hounsell, 1987). Very often evaluators will wish, or be obliged, to respond to certain at least of these expectations of policy-makers and practitioners in writing up their findings, sometimes by producing separate versions of a report targeted at different audiences. Being alert to the needs of different audiences clearly has implications not only for report writing but for all stages of an evaluation. Figures 1 and 2 present some of the key points concerning nature and level of description that would seem to flow from an attempt to respond to the separate expectations of practitioners and policy makers. The level and nature of description and explanation identified as desirable in Figure 1 could be provided by following one of the illuminative approaches to evaluation that have held such sway since Parlett and Hamilton's (1972) groundbreaking efforts in the early seventies. Similarly the desiderata in terms of type and detail of account highlighted in Figure 2 can be seen to be compatible, to some degree at least, with a systems approach to evaluation. However, there is a clear contrast between the implications for the 'grain' of evaluation identified in Figure 1 and those identified in Figure 2 – a contrast which poses a challenge to evaluators who wish to bear in mind the needs of the different audiences of practitioners, policy makers and peers within the research community and still conduct their study in a principled manner. At the stage of report writing there is also the need to avoid the problems that may arise from employing a “blurred genre” (Geertz, 1980). These challenges are made particularly acute by the fact that evaluators will customarily be working to a contract which imposes, (possibly considerable), constraints on their freedom of manoeuvre. Figure 1: Practitioners' Expectations and Implications for the 'Grain' of an Evaluation

h e l p f u l r e c o g n i s a b l e p r a c t i c a l f r u i t f u l s o u r c e o f p e r s p e c t i v e s r e a l i t y s u g g e s t i o n s g u i d e l i n e s

Implications for the ‘Grain’ of Evaluation

• use of qualitative methods that will allow a rich description of the innovation in its immediate context of use

Analysis and Reporting

• illuminating processes and conveying a sense of reality by including detailed illustration Figure 2: Policy Makers' Expectations and Implications for the 'Grain' of an Evaluation clear appropriate realistic appreciation overview recommendations of constraints

• need for closely focused and more distant views of an innovation to situate it within its immediate and wider contexts

• compressing data, summarising findings; extraction of a clear, main ‘narrative’

Reporting

• providing a succinct, synoptic account

The preceding paragraphs have then established what are likely desiderata held by policy- makers and practitioners concerning the grain of description provided by evaluators; and the general challenges faced in providing information for more than one audience. A later section of the paper will detail the attempts made to meet these challenges in a recent evaluation carried out by our research team. The following section will briefly describe the ICT project whose products were being researched and the key tasks of the evaluation. Having set the scene, attention will then turn to highlighting some of the central questions which arose concerning detail and type of description within this evaluation. Examining these questions will allow us to illustrate general issues that need to be addressed in making principled decisions about what constitutes an appropriate grain of investigation, analysis and reporting in evaluative research.

Detail, Level and Type of Description

The Teaching and Learning Technology Programme

In February 1992, the Universities Funding Council of the United Kingdom launched the first phase of a national Teaching and Learning Technology Programme (TLTP) with the intention of making "teaching and learning [in higher education] more productive and efficient by harnessing modern technology" and of helping "institutions to respond effectively to the current substantial growth in student numbers and to promote and maintain the quality of their provision" (UFC Circular 8/92). In this first phase of the programme, £7.5 million funding was provided each year for three years and academics were invited to bid for monies for projects to develop new methods of learning and teaching using ICT. Of the 43 projects which were funded during this first phase, 11 were concerned with implementing the use of learning technology within mainstream teaching and learning in single universities. The remaining 32 projects involved academics from different universities working as consortia and focused on courseware development. These consortia ranged in size from 2 to 44 member institutions and the projects covered a wide range of disciplines. A second phase of the programme was later announced, under which a further 33 projects were funded. These two first phases spanned the period 1992- 1996, with over £11 million being invested in the programme by the UK Funding Councils, plus investments made by universities themselves.

Range of materials created

The 76 TLTP projects supported under Phase 1 and 2, (some of which received additional continuation funding after the end of the second phase), created a wide range of different products. The majority of the projects produced computer-based materials, with some supporting ‘teacher user guides’. There were, however, a few projects in this group which used video as a major or minor part of their output. Products of the subject-based projects were most likely to be used in academic departments or supplied by central services to students via networks. For institutional and some subject-based projects their aim was mainly to influence the way in which teaching and learning took place rather than to produce courseware, computer- based or otherwise. The majority of projects set out to create tangible materials which could be used within courses (albeit with the same broad aim of affecting teaching practice, but this was not centre stage in their activities).

The First TLTP evaluation

An independent evaluation of these first two phases of TLTP was commissioned from a research team drawn from Coopers & Lybrand, The Institute of Education and the Tavistock Institute; and the report of this evaluation was published in June 1996 (Coopers & Lybrand et al., 1996). This report made recommendations for greater central co-ordination of the programme, progress monitoring of projects and both formative and summative project evaluation. These recommendations became conditions of the TLTP Phase 3 funding announced in May 1998. Although the evaluation that reported in 1996 was able to give a clear view of the management and general direction taken by the TLTP projects, many of the learning materials they had produced had just been released, or were still in the pipeline. It was, therefore, deemed at that point to be premature to take on the task of assessing the extent to which TLTP products, broadly defined, had been disseminated throughout the UK higher education sector and the degree of integration that these products had achieved within day-to-day teaching and learning. This task fell to our own research group.

Evaluating the use of TLTP materials in UK higher education

In January 1998 we were commissioned by the Higher Education Funding Council of England to conduct a study of the use within UK higher education of the learning materials produced by TLTP Phases 1 and 2. The study itself was carried out to a tight schedule between February and August 1998 (Haywood, et al., 1999). Key tasks in this evaluation were:

 to find out which TLTP products were being used, where and how  to examine the pattern of usage of TLTP courseware in relation to other uses of ICT within learning and teaching in UK higher education  to explore how ‘contextual’ factors might have influenced the uptake of TLTP products  to assess the overall impact of TLTP  to conduct a bibliographic search to track existing studies of TLTP use.

These tasks were pursued with surveys of:  all TLTP projects (76),  all teaching departments in all Higher Education institutions in the UK (3854),  courses/modules using TLTP in those departments and  key informants in 102 medium and large Higher Education institutions. (The surveys included a considerable number of open questions that required a qualitative analysis.) Data were also gathered from the TLTP Co-ordinator and the TLTP central collection of materials. Case studies of the implementation and use of TLTP materials were constructed, largely through interviews. A considerable proportion of higher education courses are now delivered through franchising arrangements with further education colleges. For the sake of completeness, it was appropriate to gain information from the further education colleges thought to deliver higher education courses on the extent of their TLTP usage. In carrying out these tasks the research team was faced with a number of difficult, general, methodological challenges which will be the subject of a forthcoming paper. In this present article attention will remain focused on methodological problems and decisions which had very direct implications for the grain of description that could be achieved within this evaluation, starting with the matter of the timescale over which the TLTP learning materials had been produced.

Timescale of the Teaching and Learning Technology Programme The TLTP materials which were the object of our study were created by seventy-six projects over the six year period 1992 to 1998. Thus the timescale over which materials were produced and brought into use was extensive, with some projects delivering their early, or at least prototype, products within a few months of start up and others only just reaching delivery by mid 1998. Accordingly an evaluation taking place in 1998 required some members of staff in universities to think back several years to when TLTP began, whereas others had only recently begun to see relevant products appearing. We thus faced the difficulties common to all retrospective surveys of how to achieve reliable recall as memories fade and to deal with the tendency to view the past through the screen of hindsight and rationalisations. An innovation implemented several years ago that has taken root may be enfolded within ongoing practices, and therefore be harder to separate out as a distinct object in itself. An added complication in our evaluation was the considerable variability within the population in the time period (extent and recency) on which they could comment, and correspondingly in the detail of description that it was reasonable to seek. Technological changes had also occurred over Phases 1 and 2 of TLTP that needed to be taken into account. These difficulties associated with the gathering of retrospective accounts, and in particular the question of the degree of detail that informants could be relied on to provide, were at the forefront of our thinking in the design phase of the evaluation. Judgements concerning the types and specificity of description that informants could give of past actions, played a central part in guiding both our general strategy and the fine shading of individual survey questions. Keeping these difficulties constantly in mind led us to scale down our ambitions for certain aspects of the study; and to be appropriately tentative about the claims that could be made for the whole period of 1992 to 1998 from a ‘snapshot’ taken at the end of the period. Meeting the challenge posed by timescale and variability in informants’ length of experience of using products had to be achieved largely at the level of the design of individual survey items, taking into account the degree of detail that could be achieved on any individual inquiry and assisting informants by directing them to pin their comments to a precise period in time. The problems that we faced in collecting retrospective data in this particular study may have been particularly acute; but they are clearly not unique given that many evaluations of publicly funded programmes of educational innovation are carried out post facto, and therefore are collecting what in effect are historical data. We would suggest from our own experience that directing attention closely to the specificity of the account that informants can be expected to provide of ‘historical’ events, provides a useful focus during the process of design for debating and attempting to resolve the difficulties inherent in researching the past.

Different perspectives, different levels of description

Given these concerns about the attrition of data and changes of view over time, it was imperative to maximise access to information on the use of TLTP products by following different routes into different sources. Independently of this aim to maximise access to available data, it appeared important in terms of ensuring the validity and reliability of our research findings to view the usage of TLTP products from a number of different perspectives. This enabled cross-checks to be carried out on the pattern of findings and a fuller, and thus possibly more nuanced, picture of product use to be constructed. Gaining a view from differently located ‘fixed points’ also allowed information to be gathered at contrasting levels and detail of description. Figure 3 shows the main perspectives that we set out to capture. Figure 3: Perspectives on the use of TLTP materials

University University FE colleges’ wide FE colleges’ wide perspective perspective perspective perspective

Departmental Departmental Projects’ / School Projects’ / School perspective perspective perspective perspective

USE USE OF OF TLTP Course TLTP Course MATERIALS / Module Published MATERIALS / Module Published perspective studies perspective studies

To illustrate the consequences that choosing these different perspectives had for the levels of detail and description that could be achieved, course organisers whom we surveyed and lecturers whom we interviewed in the course of constructing case studies could provide a fine-grained, more narrowly-focused account of the use of TLTP materials. Published articles and reports on the incorporation of TLTP materials into courses were, by and large, written from the viewpoint of developers, enthusiastic adopters and those making innovative use of the products. Existing literature thus also gave a detailed, close-up picture of product use. By contrast informants with an institution-wide perspective could contribute a more broad-brush account, situating the use of TLTP products within the wider context of a university’s overall use of ICT for learning and teaching and in relation to other ICT initiatives and developments. There was thus an inverse relationship between the ‘width’ of view and ’depth’ of description that could be achieved from any of the perspectives that featured in our study as Figure 4 reveals. To fulfil our own ambitions for the evaluation and to meet the needs of different audiences, it was important that we gained both :  a wider-angled, more distant view of TLTP products, that admittedly could only give us a ‘coarser’ grain of definition and  a ’close-up shot’ with a fine grain of definition that would not, however, illuminate objects that fell outside of its narrow field of vision. The need to take both wide-angled, long-distance and close-up shots applied not only to the investigation of the use of TLTP products themselves but also to the task, which is described in the following section, of mapping the territory in which these TLTP products were situated. Figure 4: Perspective metaphors

wide-angled, distant view with a ‘coarser’ grain of definition

WIDER CONTEXTS SITE AND PROCESSES OF USE OF TLTP MATERIALS

RELATIONSHIP TO OTHER ICT INITIATIVES AND DEVELOPMENTS

close-up shot with a fine grain of definition and narrower field of vision

Exploring the context of use

Carrying out the central evaluative task of viewing and understanding the uptake and use of TLTP materials in relation to their immediate and wider contexts required us to build up a sector-wide picture of:

 the use of ICT for learning and teaching in higher education

 the contextual resources and constraints influencing both ICT use and the progress of educational innovation. Surprisingly little published research was available that documented the extent of either general, or very specific, uses of ICT for learning and teaching in UK higher education. Given the pace of technological change, it would also have been dangerous to assume that what data were available accurately represented the current pattern of activity in the sector. Accordingly we needed to build up a wider picture of how ICT was being used for learning and teaching, within which the use of TLTP materials could be viewed2. Measures were taken of departments’ reported levels of general ICT usage and of the extent and nature of its use in learning and teaching. As a result a clear summary picture of current ICT use for teaching and learning in higher education in the UK has emerged from our survey work. A key consideration in pursuing this task of building up the wider picture of the role of ICT in learning and teaching was that of ensuring that the grain of description we gained of this general role of ICT in higher education was proportionate to the level of detail gained from the questions concerning the TLTP materials themselves. Although this may seem a very straightforward point of design, it was not at all trivial to implement given the diversity across the higher education sector in the size, structure and function of the departments/units that we were surveying. Turning to the matter of contextual resources and constraints, the difficulties inherent in both conceptualising contexts and in defining them for the practical purposes of a research study have been referred to earlier in this article. For purely heuristic purposes we found it useful at the design phase and later for communicating to our audiences to conceive of contextual resources in terms of a number of layers, as Figure 5 illustrates. (We are aware, however, of the limitations of this representation of context and as is the case with other handy but imprecise and potentially ‘dangerous’ conceptual tools have recognised the need to deploy it with caution.) One possible important determinant of the uptake of TLTP materials could have been the discipline/subject area taught by a particular department – the degree to which TLTP courseware was seen as consonant with the practices of a discipline and the current level of commitment in that discipline to using ICT in learning and teaching. Course characteristics, including the ones listed in Figure 5 of level, size, mode of delivery and student profile also needed to be taken into consideration as factors which might have influenced the use of TLTP materials. The use of ICT within courses will itself in turn by affected by a department's general use of ICT, its access to ICT resources and its openness to innovation. Finally the characteristics of individual higher education institutions, including their particular strategy for supporting ICT in teaching and learning and the extent to which a defined strategy has been implemented, may have impacted on the use of TLTP courseware and, therefore, required investigation.

2 The importance of doing so had been highlighted by our earlier findings concerning knowledge about, and attitudes towards, the deployment of ICT for learning and teaching within Scottish HEIs, which formed part of an evaluation of the impact of the Learning Technology Dissemination Initiative on Scottish higher education institutions. Day, K., Haywood, J. and Macleod, H. 1997. Evaluation of the Learning Technology Dissemination Initiative. Final report for the Scottish Higher Education Funding Council. [http: //www.flp.ed.ac.uk/LTRG/LTDI.html] Figure 5: TLTP materials in context

INSTITUTIONAL CHARACTERISTICS

DEPARTMENTAL CHARACTERISTICS

COURSE CHARACTERISTICS

DISCIPLINARY

USE OF TLTP MATERIALS

SCOPE AND INTEREST

LEVEL, SIZE, MODE OF DELIVERY, STUDENT PROFILE ACCESS TO ICT RESOURCES, GENERAL USE OF ICT, OPENNESS TO INNOVATION

SIZE, AGE, ICT RESOURCES, STRATEGY

Exploring the contexts within which TLTP materials were being utilised, or were failing to take root, posed very considerable challenges. Many of these challenges revolved round the practical, pressing problem of finding an exact wording for individual survey items that would allow respondents across a diverse sector to answer at a level of description that they would find feasible and enable us to obtain data that would permit reliable, valid comparisons to be made. The difficulties associated with deciding the level of detail at which to pitch particular questions in a large-scale survey to a heterogeneous population can be illustrated by the especially taxing problem of dealing with institutional and departmental differences in course structure. It was not possible to find any completely satisfactory resolution to this particular problem. In the UK, different institutions structure the courses that their students will take during an academic year in quite different ways. In one university, for example, a first year student might take only three courses, whereas in another university a first year student might take a quite large number of individual modules. There was a danger, therefore, that in asking questions of an individual department about the number of its courses/modules that employed TLTP materials, one took a measure not solely of TLTP usage but also of the way in which that department happened to organise its teaching. The wording of individual items in the departmental level and course level questionnaires were designed to manage this problem, as far as was possible; and the course level questionnaire asked respondents who identified a course as using TLTP materials to state: “What percentage of a full-time student’s annual workload does the course/module comprise?” However, it was a problem which could not be side stepped. Accordingly, we have been appropriately cautious in our analysis and conservative in our interpretation of questions which asked about the number of students enrolled on courses/modules using TLTP materials and about the number of a department’s courses/modules which had taken up TLTP materials.

Decisions concerning the ‘grain’ of an evaluation

Returning from this illustration of the difficulties involved in constructing survey items at an appropriate level of detail to focus on the general strategies of the evaluation, as with the use of TLTP materials themselves, we tried to build up a picture of their contexts of use from the different perspectives that are detailed in Figure 3. To pursue one of the evaluation’s key aims of gaining a clear, full picture of how the usage of TLTP products was influenced by contextual resources and constraints, it was necessary to take both sharply-focused near views of the immediate context of TLTP product use and a more distant, wider representation of potentially important contextual factors. Capturing narrowly-focused, fine-grained and wider, ‘broad- brush’ perspectives, and the gathering of both quantitative and qualitative data, also afforded a certain degree of flexibility at the stage of reporting, allowing us to tailor write-ups to meet the needs of different audiences. At the same time the level and detail of description that we could achieve in investigating specific topics was constrained by the timescale of the innovation itself and the diversity within the population that we were studying. Figure 6: determining the ‘grain’ of description

Audience(s) Methodological Approach: general and specific purposes of the evaluation

‘grain’ of description

of innovation and its contexts

Time: innovation Perspective of has been in Diversity/ Informants: existence Homogeneity within population  distant, wide Time: period of being studied informants’ recall  close, narrow Figure 6 represents the interlinked set of chief ‘drivers’ and constraints that determined the grain of description that it was desirable or possible to achieve within this particular evaluation. The features identified in Figure 6 as determining the grain of description in our recent study are likely to figure prominently in any large-scale evaluation:  methodological approach and purposes,  audience design,  history, i.e., time period of the innovation itself and the period that informants are being asked to recall,  diversity or homogeneity of the population that is being studied,  the perspective-point(s) from which informants are able to view the innovation.

While models of evaluation always necessarily contain assumptions, and sometimes provide explicit recommendations, concerning the level and detail of investigation and reporting that should be provided, it is not at all common in the evaluation literature to find the grain of description foregrounded as a topic in its own right. This present article has addressed the topic of the grain of description from the point of view of researchers trying to grapple with the day- to-day tasks of evaluation in a principled way and has highlighted how some of the difficulties that we encountered impacted on the grain of description that we were able to achieve. Considerable attention has also been directed to the way that the needs of audiences powerfully influence decisions about the level and detail of description that should be pursued within an evaluation. It is to be hoped that these reflections from the standpoint of ‘working’ evaluators will prompt others to examine points raised in this article from a more theoretical perspective. A sceptical reader, while conceding that the topic of what makes for an appropriate detail and level of description in an evaluation is clearly of some interest and importance, might challenge this claim that it deserves to achieve a more central place in the methodological literature. For example, it might be pointed out that questions of level and detail of description are intricately intertwined with wider issues concerning the value positions that underpin an evaluation and the approach that is pursued. If there is clarity concerning values and approach surely decisions about the grain of description will take care of themselves? Our response to such a line of argument would involve stressing that an evaluation cannot straightforwardly follow a standard set of guidelines for, and a particular value orientation to, practice. Decisions will have to be made at points between competing directions and values; and accommodations sought to meet real-world constraints. These decisions between competing purposes and strategies for meeting difficulties will very often revolve around the detail of definition within data-gathering, analysis or reporting. Keeping attention closely fixed on the question of what will make for an appropriate grain of description can, therefore, help to make more explicit the tensions between competing purposes and aid the process of making design decisions in a principled manner.

References Bell, C. and Raffe, D. 1991. Working together? Research, Policy and Practice. The experience of the Scottish evaluation of TVEI. In G. Walford (ed.) Doing Educational Research. Routledge: London and New York. Cole, M. 1995. Socio-cultural-historical psychology: some general remarks and a proposal for a new kind of cultural-genetic methodology. In J.V. Wertsch, P. del Río, A. Alvarez (eds.) Sociocultural Studies of Mind. Cambridge: Cambridge University Press. Cole, M. 1996. Cultural Psychology: A Once and Future Discipline. Cambridge, Mass. and London: The Belknap Press of Harvard University Press. Coopers & Lybrand, The Institute of Education, University of London and the Tavistock Institute, Evaluation Development and Review Unit. 1996. Evaluation of the Teaching and Learning Technology Programme Final Report. London. Engeström, Y. 1993. Developmental studies of work as a testbench of activity theory: the case of primary care medical practice. In S. Chaiklin and J. Lave (eds.) Understanding practice. Cambridge: Cambridge University Press. Entwistle, N., McCune, V. and Walker, P. 2000 Conceptions, styles and approaches within higher education: analytic abstractions and everyday experience. In R. J. Sternberg and L-F Zhang, (eds.) Perspectives on Cognitive, Learning and Thinking Styles. Hillsdale, NJ: Erlbaum. Entwistle, N.J. and Hounsell, D. J. 1987. Concepts and communication in student learning and cognitive psychology. Paper given at the British Psychological Society, Cognitive Psychology Section Conference, St. John’s College, Oxford, April, 1987. Geertz, C. 1980. Blurred genres. American Scholar, 49, 165-179. Haywood, J., Anderson, C., Day, K., Haywood, D., Land, R., Macleod, H. 1999. Use of TLTP Materials in UK Higher Education. Bristol: the Higher Education Funding Council for England. Report 39. [htpp://www.flp.ed.ac.uk/LTRG/TLTP.html] Lave, J. 1988. Cognition in practice: mind, mathematics and culture in everyday life. Cambridge: Cambridge University Press. Leont’ev, A. N. 1978. Activity, consciousness and personality. Englewood Cliffs, NJ: Prentice Hall. Parlett, M. and Hamilton, D., 1972. Evaluation as Illumination. Occasional Paper No. 9. University of Edinburgh: Centre for Research in Educational Sciences.