Benchmarking Degrees at Australian universities

Final Report Part 1 Priority Project PP653

Lead institution: University of New England Project team: Wendy Beck & Catherine Clarke

By degrees: Benchmarking archaeology degrees in Australian universities 2007 http://www.australianarchaeologicalassociation.com.au/ANCATL 2008

1 Support for this project has been provided by the Australian Learning and Teaching Council, an initiative of the Australian Government Department of Education, Employment and Workplace Relations. The views expressed in this report do not necessarily reflect the views of the Australian Learning and Teaching Council Ltd. This work is published under the terms of the Creative Commons Attribution- Noncommercial-ShareAlike 2.5 Australia Licence. Under this Licence you are free to copy, distribute, display and perform the work and to make derivative works. Attribution : You must attribute the work to the original authors and include the following statement: Support for the original work was provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government Department of Education, Employment and Workplace Relations. Noncommercial: You may not use this work for commercial purposes. Share Alike. If you alter, transform, or build on this work, you may distribute the resulting work only under a license identical to this one. For any reuse or distribution, you must make clear to others the license terms of this work. Any of these conditions can be waived if you get permission from the copyright holder.

To view a copy of this license, visit http://creativecommons.org/licenses/by/2.5/au/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, , 94105, USA. Requests and inquiries concerning these rights should be addressed to the Australian Learning and Teaching Council, PO Box 2375, Strawberry Hills NSW 2012 or through the website: http://www.altc.edu.au

2008

2 Executive Summary The major challenge identified for archaeology teaching and learning in Australia is the perceived shortcomings in archaeological qualifications, variously seen as inconsistent and/or unsuited to the goals of either students or employers. We argue that academics, not employers, are responsible for developing explicit understandings and standards for the learning outcomes of archaeology degrees, but that there has been little opportunity for Australian academics to discuss university education in archaeology. We review the Benchmarking Archaeology Degrees project which aimed to incorporate the whole disciplinary community, both academic and professional. Overall, forming and maintaining a shared network of understanding in the disciplinary community (a ‘benchmark’) is seen as critical for enhancing teaching and learning outcomes for students, employers and university staff.

Subject benchmarking is valuable both for providing the basis for improved university archaeology teaching and learning, refining curricula, and for throwing light upon continuing issues affecting the discipline. The process of subject benchmarking has many advantages even beyond that of establishing a common set of statements of the knowledge and skills expected of archaeology graduates. This report describes the design, analysis and results of the project, a complementary and interview data collection exercise which formed the basis of two national collaborative workshops where the subject benchmarks were articulated and developed. The benchmark document is now being widely disseminated on the Web.

3 Table of contents

Benchmarking Archaeology Degrees at Australian universities ...... 1 Executive Summary...... 3 Table of contents ...... 4 Project outcomes ...... 5 Description of the approach and methodology ...... 5 Comparing Degrees ...... 5 Benchmarking: The UK example...... 6 Moderation: Comparing Honours moderation and external examination in Australia...... 7 Contributions to knowledge...... 7 Success factors ...... 8 Implementation of findings in other settings ...... 9 Sharing of project outcomes ...... 9 Links between the project and other projects in the ALTC Strategic Priority Areas ...... 10 References cited: ...... 11

4 Project outcomes

The project aimed to: • Enunciate a discipline-based standards framework that included levels of achievement and broad criteria for learning outcomes in Australian undergraduate Archaeology degrees, through a benchmark document, which will be publicly available on the Web. This provides general guidance for the articulation of appropriate teaching and learning outcomes, especially assessment, to achieve comparability between individual courses of study. • Build on current university offerings and their diversity as well as international benchmarks. • Implement processes by which the discipline as a whole, through the direct involvement of many teaching staff, can discuss, endorse and participate in the standards building process as an ongoing discussion. Completion of at least two face-to-face workshops for Archaeology educators to develop collaborative cross-institutional approaches to benchmarking for Honours, including broad articulation of general teaching and learning outcomes, student capabilities and program development criteria, (but not detailed curricula). The processes adopted for sustainable cross-institutional standards development and evaluation in the discipline of Archaeology should prove transferable to other disciplinary contexts. • Determine whether Honours assessment should be more standardized and what other measures, such as assessment moderation, universities could adopt to ensure consistency for all subfields of archaeology. This would enhance national and international comparability of Archaeology Honours degrees. • Provide recommendations for ongoing cross-institutional collaboration in Honours program design and development. • Disseminate widely to stakeholders, including student and employers, the general nature and level of learning outcomes in Archaeology Honours programs through the establishment of an online project site to enable ongoing project reporting as well as the collaborative development and delivery of project documents. Production of academic papers, conference sessions and research reports will also document the process of benchmarking in practice and be widely disseminated.

Description of the approach and methodology

Comparing Degrees This project focused on academic, student and employer expectations of Archaeology graduates, in the light of the identified differences between Honours degrees (Beck and Balme 2005). Moodie (2004: 39) has suggested that there are generally three ways of assuring similar standards between university degrees: 1. External involvement in assessment (or ‘moderation’); 2. A common exam or test; and 3. Building a standards network within each field of study (‘benchmarking’). The second of these options is likely to be the least popular with universities and hence less useful to pursue and most difficult to implement successfully at present in

5 Australia. So the other two options, further explored below, are the focus of this report.

Benchmarking: The UK example Subject benchmark statements in the United Kingdom are part of the Quality Assurance frameworks with the purpose ‘of making more explicit the nature and level of academic standards in higher education and, in turn, providing a foundation for employers, public and others to have confidence in the academic awards of higher education institutions’ (Quality Assurance Agency for Higher Education 2004:3). This is equivalent to the Standards Network envisaged by James et al (2002). Each subject benchmark statement consists of a document that defines the overall components of the undergraduate bachelors Honours degree in terms of 1) generic skills and 2) subject specific skills expected of graduates (Quality Assurance Agency for Higher Education 2000; Quality Assurance Agency for Higher Education 2004). The overall ‘shape’ of the UK undergraduate degree is similar to that currently advocated for Australian arts or science degrees with a core of generic and learning- to-learn skills combined with an approximately equal number of discipline-specific and generic outcomes.

The second aspect of quality assurance is the range and level of achievement. Two different levels are defined in the United Kingdom document as ‘Threshold’ (bare pass) and ‘Typical’. The attributes differ in both quantity and quality. The Typical level (H2A/H2B boundary) includes three additional generic skills (spatial awareness, time management, and applying new ideas) and two additional subject skills, which primarily are concerned with research (‘show an awareness of the issues involved in planning, designing, and executing a programme of field-, laboratory-, or - based study’). In the Typical level there is an additional emphasis on evaluation, critical reflection and application of knowledge as opposed to merely describing or demonstrating knowledge. The generic skills also stress application of skills, and a higher level of independent working. It is these higher level of research skills that are particularly assessed in the archaeology Honours years at Australian universities, through emphasis on the research project and sustained archaeological writing.

The UK Benchmark statement is not prescriptive. It was based on the existing university offerings and methods, rather than a hypothetical set. As well as having an agreed set of expectations about degrees, standards networking or benchmarking (in the British sense of the term) has been found in the United Kingdom experience to be useful for reflecting on teaching practice, for informing students and employers and as a lobbying strength in dealing with government and institutional bureaucracy (Darvill 2003).

We overcome many of the problems of the lack of comparability between Australian universities by articulating a commonly agreed set of archaeology graduate achievements and making them explicit. Similarly we address some of the criticisms of Honours students levelled at us by industry. We adopt the approach that if Archaeology providers in Australia create their own benchmark standards, then they are taking the initiative for scholarly teaching development, rather than waiting for it to be imposed! Benchmarks were created through preparatory surveys and interviews with academic staff, designed to elucidate common themes in graduate outcomes, followed by a three-day workshop. Wider feedback on the draft Benchmark document

6 was then invited from the discipline community. The final Benchmark document was then drafted and disseminated.

Moderation: Comparing Honours moderation and external examination in Australia Ideally the Honours examination process should be an indicator of the acceptance of work in a discipline community, not just in the individual university. The existing convention that a ‘good’ Honours degree from any Australian university is transferable for enrolment in a research degree at any other Australian university (or indeed overseas) is important, and we need to ensure that they are equivalent in standard. One way of attempting this is through moderation.

‘Moderation’ is a term that encompasses a range of possible means for establishing comparability of standards of student performance, and can include statistical moderation of results, inspection of samples of work by visiting or internal staff, external examination, or group moderation (DEST 2004). We argue that the existing convention in use by most providers, that external examiners be engaged to mark dissertations, should be used by all universities. This is also one of the recommendations for good practice in the Australian Vice-Chancellors Committee (AVCC) Honours guidelines (AVCC 1995: 2, Recommendation 2.9). The advantage of this form of standardisation is that the teaching of the program is separate from the assessment. In a trial Archaeology Honours moderation process at the University of Western Australia in 2003, the external assessor visited the university specifically to assess all the Honours theses for that year. The marking was done by the external assessor as well as two internal assessors and all marks were compared prior to the grades being awarded. While the trial demonstrated a proportion (c. 20%) of disagreements between examiners, it was useful in making explicit the goals of awards for the purpose of comparison and we see external moderation as one important complement to the more central priority of benchmark development.

The project held a second workshop, where moderation and good practice examples for implementing the Benchmarks were discussed. A trial of moderation, based on the UWA process, is planned for 2009.

Contributions to knowledge A four year Bachelor degree with Honours in Archaeology is still seen as the fundamental level of academic achievement required to gain entry to the profession and to higher degree research in Archaeology in Australia (Beck and Balme 2005). This project will enhance our knowledge about the teaching and learning of Archaeology in Australia. Changes over the last few years within universities have led some to suggest that Archaeology standards have fallen and that Honours graduates are not necessarily competent Archaeology professionals (Colley 2004). This contention is part of a long running debate (see, for example, Frankel 1980; McBryde 1980) that is difficult to resolve but which this project addresses. Similar concerns about Archaeology graduates are also being expressed outside Australia (Black 2001; McGimsey 2003). International trends in the structuring of Bachelors’ degrees, such as the Bologna Accord, which are making clearer distinctions between Bachelors’ degrees and postgraduate awards, are also likely to increase the move towards a common set of learning outcomes and standards at various levels of award.

7 Benchmarking, which we define as the drawing up of a set of general expectations about award standards by the disciplinary community through dialogue and consensus, (for example, Quality Assurance Agency 2000 and 2007) has the potential to improve outcomes in higher education teaching and learning from a variety of perspectives. In the higher education sector, benchmarking data has been drawn upon to propose more constructive alignment of teaching objectives with actual assessment practices (Yorke 2002); to design classroom strategies to lower attrition rates (Manning and Bostian 2006); as the basis for curriculum renewal and reform (Lemons and Griswold 1998); and to enhance the quality of disciplinary research (Nicholls 2007). Beyond the academy the effects of improved consistency and coherence within and between degree programs experienced by archaeologists offer the possibility of a 'flow-through' effect for developing and supporting practitioners in the wider profession. With better education graduates can expect to be better equipped for their professional roles. The process of collaboration in the Benchmarking project has also had benefits for developing closer institutional co-operation. These potential benefits all added to the considerations that made the recent national two-year project to benchmark archaeology Honours degree a productive one.

This project is significant because it implements measures for addressing concerns about the lack of explicit outcomes of Archaeology degrees for the first time at a national level. For the first time, it brings together university providers of specialist Archaeology education and the profession, to implement and map out cross- institutional approaches to Archaeology standards and assessment. Significantly, all ten specialist provider institutions were involved in this project, together with contributions from non-specialist providers (i.e. those who offer smaller numbers of Archaeology units). These institutions are all contributors who agree with and support the project’s aims. It was agreed that all Australian universities should investigate further collaborative practices, such as joint teaching programs, particularly across specialist sub-fields, as well as the sharing of facilities or equipment where practicable.

Although numerically the discipline is a comparatively small one in Australia, it is a field of practice with integral links to both industry and cultural heritage agencies, as well as a field with generalist humanities education outcomes. By ensuring that Archaeology degrees have agreed standards and assessment approaches, educational goals can be made explicit to employers, the profession and the academy and the relationship between all stakeholders accordingly strengthened and enhanced. This will also ensure the transferability of degrees both nationally and internationally, an important consideration both for graduates seeking specialist education in order to practice Archaeology and for those who wish for a more general archaeological education.

Success factors In a very real sense, the compact nature of the discipline in Australia was part of the very great success of this project, as well as its manageability. Because the project has at its core intra-disciplinary communication, negotiation and collaboration, the relatively small number of participants (10-13) has been a factor in the success of the approach. Face to face meetings have proven to be the best method for joint construction of the Benchmarks and allayed fears of unhealthy competition between the university programs. Discovering unexpected commonalities between different

8 academic approaches has also been a strength of the workshop approach. The major challenge is how to sustain the momentum generated by the project, once this funding has ceased .

Implementation of findings in other settings

‘Standardization’ of Archaeology degrees does not mean prescription (Harding and Johnson 2002), although probably many university teachers’ first reaction to the suggestion of Benchmarks is that it would be a further affront to their autonomy. Students have real choices when selecting a university and universities should be able to teach in their own way on topics that draw upon their research strengths and interests. Benchmarks should support innovation, variety and flexibility in program design. We developed a common set of generic and subject-specific expectations about what graduates should know and can do.

All ten specialist Archaeology provider universities were invited and agreed to participate as benchmark partners, and a further three participated as benchmark associates. These universities differ widely in terms of their approach and the position of archaeology within the institution. The professional Archaeology associations in Australia were also an important audience for the project (i.e.Australasian Society , Australian Institute for , Australian Archaeology Association, Australian Institute for Archaeology, and Australian Association of Consulting Archaeologists).

Sharing of project outcomes

The project results have been reported in two published papers, in an international and a national journal. 2008 Beck, W. and Clarke, C. Archaeology Teaching and Learning in Australia 2003-2008 Perspectives from the Academy. Research in Archaeological Education Journal 1(electronic). http://www.heacademy.ac.uk/hca/archaeology/RAEJournal

Submitted 2008 (Nov) Clarke, C. and Beck, W. The view from the bench: Subject benchmarking and the context of Australian Archaeology. Australian Archaeology.

Poster presentation at Australian Archaeological Association Annual conference by Beck & Clarke ‘Benchmarking Honours in Archaeology’ December 2006.

Invited Seminar at Sydney University in August 2006 by Beck “National perspectives and initiatives of the Joint Interim SubCommittee Archaeology Teaching & Learning (JISCATL)”.

Invited paper to Collaborating To Offer Small Courses/Subjects Colloquium hosted by the University of New England for the Carrick Institute on the 17th and 18th May, 2007. By Beck & Clarke ‘Benchmarking Archaeology Degrees in Australian Universities: Collaboration with discipline-based organisations & supporting disciplinary needs”.

9 The benchmark document is publicly accessible website, as part of the Australian Archaeological Association pages: By degrees: Benchmarking archaeology degrees in Australian universities 2007 http://www.australianarchaeologicalassociation.com.au/ANCATL

Hardcopies will be distributed in a mail out to all members of the Australian Archaeological Association in June 2009, as well as to the Presidents of the relevant archaeology associations, i.e. Australasian Society Historical Archaeology, Australian Institute for Maritime Archaeology, Australian Archaeology Association, Australian Institute for Archaeology, and Australian Association of Consulting Archaeologists Incorporated.

Hardcopies will also be sent to the Pro-Vice-Chancellors (Teaching and Learning) at the ten benchmark partner institutions. Each Archaeology teaching unit has received ten copies for distribution to teaching staff.

Links between the project and other projects in the ALTC Strategic Priority Areas

A number of ALTC forums were attended by the Project team members. These were:

Carrick National Assessment Project Forum Centre for the Study of Higher Education (CSHE), The University of Melbourne on Wednesday 6 December 2006. The forum was facilitated by Professor Dai Hounsell, Professor of Higher Education, The University of Edinburgh.

Collaborating To Offer Small Courses/Subjects Colloquium hosted by the University of New England for the Carrick Institute on the 17th and 18th May, 2007.

The Carrick Institute for Learning and Teaching in Higher Education Project Management Forum 20 August -21 August 2007, Adelaide.

Carrick ATN Assessment Forum held at the Queensland University of Technology from 28 November to 29 November 2007. The forum was facilitated by Professor Dai Hounsell, Professor of Higher Education, The University of Edinburgh.

Carrick Institute teaching-research linkages project forum: issues and purposes of teaching-research linkages in higher education. The University of Melbourne, Centre for the Study of Higher Education, Barbara Falk Room, 19 June 2008.

The purpose of attending these forums was to share ideas and expertise related to Carrick-funded assessment projects, to draw on Professor Dai Hounsell’s views on how to communicate findings and to form collegial networks to enhance the work of the project team members.

10 References cited: Australian Vice-Chancellors Committee 1995 Fourth Year Honours Programs, Guidelines for Good Practice. Retrieved 16 November 2004 from http://www.admin.uwa.edu.au/reg/sec/avcc/document-C.pdf Beck, W. and Balme, J. 2005 Benchmarking for Archaeology Honours degrees in Australian universities. Australian Archaeology 61:32-40. Black, M. S. 2001 Maturing gracefully? Curriculum Standards for History and Archaeology. The Social Studies May/June 2001:103-108. Colley, S. 2004 University-based Archaeology teaching and learning and professionalism in Australia. World Archaeology 36(2):189-202. Darvill, T. 2003 Questions of balance - education, training and experience. Conference Paper Theoretical Archaeology Group Conference 2003. Retrieved November 20, 2004 from http://www.hca.heacademy.ac.uk/Archaeology/confprocs/TAG2003.php . Department of Education, Science and Training (with Language Australia and the Victoria University of Technology) 2004 Principles of Moderation. Retrieved 30 September 2005 from: http://www.staff.vu.edu.au/cgea/principles.html Frankel, D. 1980 Introduction, Education and training in prehistory and Archaeology in Australia. Australian Archaeology 11:69-71.. Harding, A. and Johnson, M. 2002 Archaeology and the QAA subject review. What have we learned? Antiquity 76(294):967. James, R., McInnis, C., Devlin, M. 2002 Options for a national process to articulate and monitor academic standards across Australian universities. Submission to the Higher Education Review across Australian universities’ Submission 11. Lemons, D. E., and J. G. Griswold. 1998. Defining the boundaries of physiological understanding: The benchmarks curriculum model. Advances in Physiology Education 20(1):S35-S45. McBryde, I. 1980 Educational goals of university schools of prehistory and Archaeology: Mechanick trades in the ivory tower? Australian Archaeology 11:72-80 McGimsey, C. R. 2003 The four fields of Archaeology, American Antiquity 68(4): 611-618. Manning, T. M., and B. Bostian. 2006. Using benchmark and assessment data to facilitate institutional change. Benchmarking: An Essential Tool for Assessment, Improvement, and Accountability: New Directions for Community Colleges : 73-81 Moodie, G. 2004 The neglected role of a neglected body: Academic Boards’ role in ensuring ‘equivalent’ standards. Australian Universities Review 47(1):35-41. Nicholls, M. G. 2007. The development of a benchmarking methodology to assist in managing the enhancement of university research quality. Higher Education Quarterly 61(4):539-562. Quality Assurance Agency for Higher Education 2000 Subject Benchmark Statements: Archaeology . Retrieved June 20, 2005 from http://www.qaa.ac.uk/academicinfrastructure/benchmark/Honours/Archaeology.asp Quality Assurance Agency for Higher Education 2004 Recognition Scheme for Subject Benchmark Statements . Gloucester: Quality Assurance Agency for Higher Education Quality Assurance Agency 2007 Archaeology subject benchmark http://www.qaa.ac.uk/academicinfrastructure/benchmark/statements/Archaeology.asp Yorke, M. 2002. Subject benchmarking and the assessment of student learning. Quality Assurance in Education 10(3):155-171

11