Where we’ve been and where we’re going: An expert discussion about Assessment, Measurement, Motivation and Learning

Expert panellists: Dr Ralph Stratton, A/Prof Mary Ainley, Prof Max Smith, & Dr Sue Thompson

Discussion facilitated by: Prof Paul Richardson

Monday 27 November 2017, 4-5.30pm National Ballroom 1, Conference Floor AARE Annual Meeting 2017

The assessment, measurement, motivation and learning disciplines continue to be at the centre of many key developments in the field of educational research. In this retrospective and prospective session, Australian luminaries of the disciplines will discuss the past, present, and future of the discipline. Key points of discussion will include theoretical, methodological, and empirical advances, stagnations, and possibilities in the disciplines. The session will run in a Q and A type format, facilitated by the SIG convenor, who will invite audience participation through questions and discussion with the key scholars. The discussion promises to be lively, informative, and entertaining, and not to be missed by all those involved in research in these disciplines.

Members of the Motivation & Learning SIG, Assessment & Measurement SIG, and other interested conference attendees are invited to submit prospective questions for consideration by the panel. Please email: [email protected]

This special event is made possible by funding from AARE and is a joint initiative of the Motivation & Learning, and Assessment & Measurement Special Interest Groups. Panellist biographies

DR RALPH STRATTON Ralph Straton has been a member of AARE since 1971 and served on the executive committee for 13 years (Secretary, 1972-74; President, 1984) as well as being a member, and also chair, of the Doctoral Thesis Award Committee. He was a member of academic staff of the Department of Education at the University of Sydney (1971-1977) before moving to the School of Education at Murdoch University (1978-2004) as foundation Director of the Institute for Social Program Evaluation, the first evaluation research centre in Australia, and established the MA (Social Research and Evaluation). He also served as Dean of Education (1994-1997). He has conducted over 50 evaluation and other research studies and has numerous books, monographs and other publications and presentations in evaluation and various aspects of education. Dr Straton is a member of the Australian Psychological Society, a Fellow of the Australasian Evaluation Society and has been President of the Society (1999-2001).

A/PROF MARY AINLEY My academic career has largely been concerned with teaching Developmental Psychology and researching motivational processes such as curiosity and interest as they impact students’ learning. A student comes to a new situation whether it is an achievement or a social interaction setting, with a certain combination of traits, dispositions, and well- developed interests. The challenge has been to understand the interaction of person and situation. What processes are triggered by specific features of learning situations? How do these processes play out for different students? Within this area I have used both variable- centred and person-centred analytic approaches and have contributed to the current trend to use micro-analytic techniques to assess motivation processing in learning contexts. Most recently I have co-authored several publications that have used large international data sets to investigate questions concerning student motivation that build on the findings of the microanalytic studies.

There has been an interesting trajectory across the history of motivation assessment and measurement. The focus has shifted from motivation as state, motivation as trait or disposition and back to real-time states in defined contexts. The early learning theorists assessed motivation as drive and measured motivation in terms of deprivation. Behaviourists measured motivation in terms of reinforcement schedules. The counterpoint between intrinsic and extrinsic motivation quickly moved from motivation defined by context to motivation defined by personal orientation or disposition, economically measured through self-report questionnaires. More recently technological innovation has focused attention on ways of accessing motivational states in-the-moment as students engage with learning contexts. How far have we progressed in understanding the energy and activation that supports students’ learning? General state measures, personal orientation or disposition measures, and now opportunities to assess how personal disposition and context generate motivation states that activate learning: Where we’ve been; how far will this take us?

PROF MAX SMITH Max commenced as Professor of Education at the University of Newcastle in 2012 following a 36-year career with the NSW Department of Education including senior positions in quality assurance, assessment and evaluation, school accountability, research management and strategic initiatives. He is a core member of the Teachers and Teaching Research Centre at Newcastle with ongoing research commitments to quality teaching, child development, equity in higher education and innovative public policy. Max advocates for strengthening evidence- based practice, policy entrepreneurship and the dissemination of research as cornerstones for ensuring that our research has impact.

DR SUE THOMPSON For the last 15 years I have been working on large scale international assessments. Starting with PISA 2003 and TIMSS 2003, I have managed every cycle of these studies since - I’m now on my 6th cycle of PISA and 5th cycle of TIMSS. In 2011 I convinced the State and Territory representatives to also participate in PIRLS, which we did in 2011 and then again in 2016 (report released next week). Our reports are widely used with policy makers, our data are used in Productivity Commission annual reporting and we are also called on to conduct bespoke analyses for different jurisdictions and systems.

What I see as being the most important part of these studies is the questionnaire data. While it is important to measure the overall health of a system by understanding achievement in these key areas, both the IEA and the OECD have recognised that the non-cognitive outcomes are also vital outputs of educational systems. These non-cognitive outcomes are defined in the contextual frameworks for each study. They are well-defined, psychometrically sound measures. We consistently collect data on particular measures including those involving engagement and motivation.

Each time we gather data for PISA, which is targeted to 15 year old students, or TIMSS, for Year 4 and Year 8 students, or PIRLS, for Year 4 students, we collect a wealth of rich, clean data that is largely under-utilised. Internationally we are talking about clean data sets for millions of students, all of which are collected under standardised conditions.