Tracking and Assessing Practical Chemistry Skills Development: Practical Skills Portfolios

Home , University of Sheffield

This is a repository copy of Tracking and assessing practical chemistry skills development: practical skills portfolios.

White Rose Research Online URL for this paper: http://eprints.whiterose.ac.uk/139848/

Version: Published Version

Article: Wright, J.S., Read, D., Hughes, O. et al. (1 more author) (2018) Tracking and assessing practical chemistry skills development: practical skills portfolios. New Directions in the Teaching of Physical Sciences, 13 (1). ISSN 1740-9888

10.29311/ndtps.v0i13.2905

Reuse This article is distributed under the terms of the Creative Commons Attribution (CC BY) licence. This licence allows you to distribute, remix, tweak, and build upon the work, even commercially, as long as you credit the authors for the original work. More information and the full terms of the licence here: https://creativecommons.org/licenses/

Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing [email protected] including the URL of the record and the reason for the withdrawal request.

[email protected] https://eprints.whiterose.ac.uk/ 1

COMMUNITY DIRECTIONS

Tracking and assessing practical chemistry skills development: practical skills portfolios

James S. Wright,1,2 David Read*,3, Oliver Hughes3 and Julie Hyde*,1

1Department of Chemistry, University of Sheffield, Brook Hill, Sheffield S3 7HF (UK). 2Guangdong Technion Israel Institute of Technology, Shantou, Guangdong 515063 (PR China) 3School of Chemistry and Centre for Higher Education Practice, University of Southampton, Highfield, Southampton SO17 1BJ (UK)

Corresponding Authors: [email protected] & [email protected]

Keywords: China; Chemistry; Laboratory; Assessment; Reflection

Abstract integral part of this process, given the role of We present the evaluation of the student such skills development in enhancing the response to a novel form of practical employability of science graduates. assessment: the Practical Skills Portfolio (PSP). The PSP is a concise record of a Perhaps the most widely used form of practical activity for the purposes of assessment in practical Chemistry courses is assessment, which prompts students to the post-laboratory write-up, where a script or engage in reflective practice on laboratory scientific report is completed by the student. skills, and provides opportunities for This is, of course, important. Students need to enhanced feedback delivered in a timely write scientific reports to develop their manner. Key goals of this new approach are scientific language skill, and directly to assist students in assimilating the practical assessing their work during a large laboratory skills they are developing during their studies class is logistically difficult. While post- and to support them in developing their ability laboratory reports can, if carefully designed to write the different components of a full (‘constructively aligned’, Biggs (2003)), test laboratory report. the students’ understanding of in-laboratory tasks (Johnstone et. al. 1998, Reid & Shah Introduction 2006), they do not necessarily assess the development of practical skill, even though the The development of skills and attributes in development of practical skills is arguably one chemistry has received increasing attention in of the main aims of such courses. How can recent years (Overton & Garvey, 2017). With we track or assess the practical skills regard to practical skills development, this is development of Chemistry students on a large reflected in recent changes to the “A” Level scale, in a logistically feasible manner? This structure (Read & Barnes 2015; Barnes, issue is being addressed at “A” Level through Laham & Read 2017), the requirements for the introduction of the new ‘practical RSC accreditation of degree programmes (to endorsement’ (OFQUAL, 2016), and a complete a certain amount of practical work), number of universities are currently updating and the demonstration of skills competence in their practical courses in a similar way (Brazil vocational qualifications. Explicitly assessing 2017). and acknowledging the development of

students’ practical skills should form an

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905 2 Tracking and assessing practical chemistry skills development: practical skills portfolios Formative assessment of skills development Chemistry course for their first three years, in practical Chemistry has been previously taught by “flying faculty” UK staff travelling out described by Towns et. al. (2015) (see also to Nanjing. This course consists of 17 Hensiek et. al. 2016), and later Seery (2017), experiments, designed and selected to through digital badging. Students are video- develop a practical skill set from a roughly UK recorded performing a key task, and are “A” level equivalent (in year 1), up to a UK awarded a badge via the virtual learning second-year degree equivalent level (in years environment (VLE) to certify their 2 and 3). competence. In Seery’s modification, students are also exposed to pre-lab videos, which These are all English as Second Language they peer-review, as well as answering a (ESL) students; who in some cases do not number of questions about the work they are have any significant practical Chemistry doing in the video. This formative assessment experience from High School. (Not all was shown to quantitatively improve students’ students had the opportunity to study confidence and experience in selected Chemistry in their final years of High School, practical skills. and indeed those that did, may have had limited practical Chemistry experience.) The applications of digital badging described Therefore, a particular emphasis on building by Seery, Towns, and Hensiek thus far have practical skills will be important for any been limited to a limited range of techniques: university level course they take. Reflection pipetting; preparation of standard solutions; after the initial delivery of the course titration and distillation. To assess the skills suggested that further formative assessment developed over an entire introductory practical tools to gauge students’ development in Chemistry course would necessitate scientific writing and practical skills significant post-laboratory review on the part development, beyond the post-laboratory of the instructor watching the students’ report, were necessary in this abridged recorded video, which may prove challenging course. in terms of workload. Links with the University of Southampton, and Context discussions in 2014 about the Southampton skills assessments led to the PSP model University of Southampton being adapted for use in this unique setting in The Science Foundation Year at the China. In particular, such a portfolio would be University of Southampton provides an entry useful for Chinese students in not only route onto science degree programmes for developing practical skills but also their students who do not have the required chemical language. qualifications for direct entry. Students complete 10 chemistry practicals over the Format, delivery and assessment course of the year, introducing all of the skills of PSPs commonly encountered by “A” Level PSPs were first implemented at Southampton chemistry students. during the 2013/14 academic year and on Sheffield’s course in China in 2014/15, with University of Sheffield (Nanjing Tech small modifications to templates being University (NJTech) Joint Degree) – implemented each year to enhance the impact Practical Skills Portfolio (PSP) on learning. A pro-forma is provided to The University of Sheffield-Nanjing Tech students which they populate with University Degree is a [3+1] joint programme, photographs, reflections and other text or which began in 2011. It is delivered in English diagrams as required by the tasks set. The only and is RSC accredited. Students study PSP pro-formas are generally divided into for three years in Nanjing, China, followed by sections based on ‘technique’, ‘observation’ a year spent in Sheffield, totally integrated and ‘apparatus’. They contain: prompts for with the final-year BSc cohort (Hyde 2014a;b, students to photograph apparatus and key Hyde et. al. 2016). These students take an changes; simple yes/no questions to check abridged, specially designed practical understanding; 5-point ‘confidence’ scales to

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905 3 Tracking and assessing practical chemistry skills development: practical skills portfolios gauge experience; and free-text questions evidencing the acquisition of new skills and based on common difficulties encountered covering all the practical requirements for with new techniques. students to join and be fully integrated in the Level 3 course in Sheffield. However, At Southampton, students submit PSPs online discussion with students about this approach via Turnitin to be marked electronically, with at NJTech suggested that, although they the aim of returning marks and feedback to understood the purpose of the paper-based students within one week of submission i.e. Practical Skills Portfolio, students would prefer before the following chemistry practical. An to submit portfolios online. Therefore, from example of a student’s submission, with 2016-17 the Practical Skills Portfolio pro- feedback added using Turnitin, is shown in formas were instead submitted via Moodle, Figure 1a. removing some logistical challenges for students (e.g. printing photos) and for staff In the case of Sheffield, PSPs were initially (e.g. facilitating assessment and feedback submitted in paper form, with students provision in a more constructive manner). completing them in the laboratory, printing Feedback from 2016-17 onward was provided photos taken on their smartphones and orally in workshop-style classes alongside oral attaching them to the pro-forma. An example and written feedback on their post-lab written of a student’s submission in this format is assignments, as an older version of Moodle shown in Figure 1b. PSPs were then checked was in use at Nanjing Tech which did not by a laboratory teacher and returned to provide a convenient pathway for written students, with written feedback. A collection of feedback. pro-formas constituted a physical ‘portfolio’,

Figure 1a An example of a Southampton student’s PSP including feedback added via Turnitin (used with permission).

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905 4 Tracking and assessing practical chemistry skills development: practical skills portfolios

Figure 1b An example ‘apparatus’ section of a Sheffield-Nanjing student’s PSP, submitted via Moodle - oral feedback given in class (used with permission)

The student response to PSPs A survey of the 2013/14 cohort of students at In view of the fact that PSPs are designed to Southampton indicated that a majority felt that help students assimilate practical and report PSPs improved their understanding of writing skills, it is perhaps more important to techniques and increased their confidence in consider the longer-term impacts on their undertaking practical work. One noteworthy progression to more advanced studies. During comment was that: 2017/18, students who had previously undertaken practical work assessed via PSPs “Skills portfolios have enabled me to were asked to reflect on the impact of this develop my practice. I find it beneficial to approach on their subsequent performance, record each new skill after the practical leading to the data presented below. The and reflect on them at a later date.” figures below illustrate students’ responses to four different questions on a Likert-type scale, Surveys of students on Sheffield’s Nanjing backed up by quotations from students which programme in 2015/16 and 2016/17 explain their response. The quotations suggested that their attitude to the PSP was included were selected on the basis that they generally positive, with a significant majority provided insight into the student’s perceived agreeing with the statement “The Practical value of the PSP. Portfolio is helpful for my studies”.

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905 5 Tracking and assessing practical chemistry skills development: practical skills portfolios

It had a strong positive impact. 6 17 17

It had a small positive impact. 18 25 11

It had no impact at all. 2 2 1

It had a small negative impact. It had a strong negative impact. Soton SFY Sheff Y2 Sheff Y3 0 10 20 30 40 50 60 Number of responses in Agreement (n = 100)

Figure 2 Student responses to the prompt “The skills portfolio required you to collect photographic evidence of you undertaking practical techniques correctly. To what extent did this impact your ability to remember how to perform techniques at a later date?”

The data illustrated in Figure 2 indicates that steps undertook. This truly helped me students have a favourable view of the impact remember the steps.” of the photographs in PSPs on their assimilation of practical techniques. This was “I never looked back at these photos, backed up, anecdotally at least, by staff who but I can remember what I was trying to noted that students showed greater show to get the perfect photo to my proficiency when repeating techniques and skills portfolio.” were less likely to make common errors. Student comments included: “Helpful visual aid for future reference, especially with feedback.” “Going through the techniques I had learned prior to lab work in the first year “It created a memory of a skill that was of my degree and seeing the built upon moving forward!” evidence…gave me confidence.” “I can still remember some techniques “Throughout the experiment we had to about 1 week later, with portfolio it can take pictures of all of nearly all of the be much longer.”

It had a strong positive impact. 5 23 15

It had a small positive impact. 13 17 12

It had no impact at all. 5 3 2

It had a small negative impact.

It had a strong negative impact. Soton SFY Sheff Y2 Sheff Y3

0 10 20 30 40 50 Number of responses in Agreement (n = 98)

Figure 3 Student responses to the prompt “To what extent did the use of photographs in the skills portfolio impact on your ability to self-assess and reflect on your performance in carrying out a technique and formulate advice for other students as stipulated in the portfolio?”

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905 6 Tracking and assessing practical chemistry skills development: practical skills portfolios The data illustrated in Figure 3 indicates that “I think the positive impact came from students found that the photographs also the comments to those pictures…they assisted them in the process of reflecting on allow staff to see the way students their implementation of techniques as required perform the experiment while it is by the PSP. This might help to explain the impossible to do that in the lab for a evidently improved assimilation of techniques large number of students doing the exhibited by students. Student comments same experiment at the same time.” included: “the description texts help me more than “I was able to explain any errors in my the pictures but the photographs help to technique if they had arisen. And (was) memorise it” able to say where I could improve.” “It reminds me of some procedures during the exam.”

It had a strong positive impact. 15

It had a small positive impact. 6

It had no impact at all. 1

It had a small negative impact. Soton SFY

It had a strong negative impact.

0 5 10 15 20 Number of responses in agreement (n = 94)

Figure 4 Student responses to the prompt “The skills portfolio gave you the opportunity to practise writing up sections of lab reports after different practicals e.g. a method, an evaluation, or plotting a graph on which you received feedback. To what extent did this impact on your learning compared with writing a full lab report?” (Note that this question was only pertinent to Southampton SFY students)

The data illustrated in Figure 4 indicates that “It certainly was beneficial to have the students valued the opportunity to practise feedback on writing elements of writing sections of lab reports and that this practical report. It helped me to write had a positive impact on their learning. more concise and accurate reports (as Student comments included: a) result.”

“Broke down larger parts of the lab “I think it helped to improve skills a little report to easier parts and made labs at a time without being overwhelming.” less intense and more approachable.” “Report writing is a lot more difficult at “It helped so much as much of the this stage, but gave a good base on second year (which I am currently which to build on and got us using the completing) involves writing full lab right skills at an early date.” reports. All the guidance and criticism of my foundation year report has helped me go through my work more carefully and lead to high marks in write ups.”

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905 7 Tracking and assessing practical chemistry skills development: practical skills portfolios

It had a strong positive impact. 14 20 16

It had a small positive impact. 6 21 10

It had no impact at all. 3 2 2

It had a small negative impact. 1

It had a strong negative impact. Soton SFY Sheff Y2 Sheff Y3

0 10 20 30 40 50 60 Number of responses in agreement (n = 95)

Figure 5: Student responses to the prompt “Since progressing (in your studies), you will have done a lot of practical work. To what extent did the use of skills portfolios (in your earlier studies) impact on your ability to successfully carry out practical work and complete lab reports/write ups?” The prompt for Southampton students referred to progression to degree programmes and the use of PSPs on the foundation year, while that for Sheffield students referred to progression to Y2 or Y3, and the use of PSPs in China.

The data illustrated in Figure 5 indicates that has drastically improved, although this effort students have a very positive view of the has been in tandem with further scientific impact of PSPs in their earlier studies on their writing training outside of the laboratory, thus ability to perform well in laboratory work. making it difficult to quantify the effect of the Given the fact that in both cases this was a PSP on students’ scientific writing skills. key aim of the implementation of PSPs, this is a very encouraging outcome. Student Discussion and Conclusions comments included: The Practical Skills Portfolio (PSP), a novel form of practical skills assessment, has been “I think it gave me confidence to carry presented in two forms, used both in the UK out techniques from memory correctly (University of Southampton) and China and help others that had not done the (Nanjing Tech University). Student feedback foundation year.” in response to use of the PSP was very positive: students found that the PSP helped “I feel as though I was more prepared to them recall practical techniques, reflect on handle the workload and prepare myself their performance and prepare for future better. Having already had some practical work in later years of study. experience of doing write ups meant that Students’ comments provide strong evidence, I felt more at ease approaching degree albeit based on perception, relating to the level write ups.” features of PSPs that provide value in supporting skills development. In particular, “Very grateful to this module - hugely the students’ self-reported improvement in helped with write ups, throughout my their recall of practical techniques is valuable degree.” for the Nanjing Tech students, as their Sheffield laboratory course is abridged and “Knowing how to do certain procedures runs in only one semester each year. in the lab as well as calculations in the Anecdotally, students preferred the write up has been a very useful skill that submission of an online PSP, because of the I've had to use every lab so far.” logistical difficulties of printing and reprinting the PSP in Nanjing. Anecdotally, it appears that Sheffield/Nanjing students’ quality of English scientific writing

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905 8 Tracking and assessing practical chemistry skills development: practical skills portfolios In summary, PSPs provide a concise approach to the assessment of practical skills Biggs, J. (2003) Teaching for Quality Learning and associated activities, reducing workload at University – What the Student Does (2nd for students and staff alike and paving the Edition) SRHE / Open University Press, way for success in teaching laboratories at Buckingham. more advanced stages of study. Brazil, R. (2017) Curriculums for modern Future developments chemists. Education in Chemistry Nov 2017, The use of a GoPro camera to film first- 22-25. person perspective laboratory work has been shown by Fung (2015) to enhance pre-lab Fung, F.M. (2015) Using First-Person instructional videos. The Sheffield team are Perspective Filming Techniques for a now exploring how a student-worn head- Chemistry Laboratory Demonstration To mounted Go-Pro can be used to capture the Facilitate a Flipped Pre-Lab. Journal of learning of new techniques. A similar method Chemical Education 92, 1518-1521. DOI: has been used to gather evidence relating to 10.1021/ed5009624. meaningful learning in the laboratory (Galloway & Bretz, 2015). This student- Galloway, K.R., & Bretz, S.L. (2016). Video recorded footage is already being episodes and action cameras in the incorporated into pre-lab instructional videos, undergraduate chemistry laboratory: eliciting and investigation is continuing as to how it student perceptions of meaningful learning. can be returned to students for submission as Chemistry Education Research and Practice part of the Practical Skills Portfolio in the 17(1), 139-155. DOI: 10.1039/c5rp00196j. coming academic year. Overton, T. & Garvey, D.J. (2017) At Southampton, the use of a pioneering Development of key skills and attributes in Laboratory Response system, Labdog, is chemistry. Chemistry Education Research being trialled for use in the laboratory as an and Practice 18, 401-402. DOI: interactive laboratory script which prompts 10.1039/c7rp90006f. students to answer questions at key points in a practical procedure, facilitating delivery of Hensiek, S., DeKorver B.K., Harwood, C.J., feedback from staff in real time (Wilson & Fish, J., O’Shea, K. & Towns, M. (2016) Read, 2017). This supports students’ Improving and Assessing Student Hands-On reflections on practical work and helps with Laboratory Skills Through Digital Badging. the completion of PSPs. An evaluation of Journal of Chemical Education 93, 1847- Labdog will be presented in another article. 1854. DOI: 10.1021/acs.jchemed.6b00234.

Acknowledgements Hyde J. (2014a) Taking Laboratory Chemistry to China: A Personal View. New Directions in An award from the University of Sheffield the Teaching of Physical Science 10, 32-35. Deputy Vice President’s Learning & Teaching DOI: 10.29311/ndtps.v0i10.521 Fund for the purchase of the GoPro camera is

gratefully acknowledged. Hyde J. (2014b) Can you teach University

Chemistry Abroad? Chemistry in Sri Lanka The authors also wish to thank the Royal 31, 18-19. Society of Chemistry and Institute of Physics for facilitation of the annual ViCE/PHEC Hyde, J., Cranwell, P. & Page, E. (2016) conferences, which have fostered productive Collaborative chemistry degrees. Education in discussions directly contributing to this work. Chemistry June 2016, 22-25.

References Johnstone, A.H., Watt, A.T. & Zaman, T.U. Barnes, S.M., Laham, A. & Read, D. (2017) (1998) The students’ attitude and cognition Teachers’ views of the impact of the new A change to a physics laboratory. Physics level in chemistry. Available at: Education 33, 22-29. DOI: 10.1088/0031- http://edshare.soton.ac.uk/18928/ 9120/33/1/016.

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905 9 Tracking and assessing practical chemistry skills development: practical skills portfolios (2017) Developing laboratory skills by OFQUAL (2016) GCE Subject Level incorporating peer-review and digital badges. Conditions and Requirements for Science Chemistry Education Research and Practice (Biology, Chemistry, Physics) and Certificate 18, 403-419. DOI: 10.1039/c7rp00003k. Requirements. Available at: https://www.gov.uk/government/publications/g Towns M., Harwood M.J., Robertshaw, M.B., ce-subject-level-conditions-and-requirements- Fish, J. & O’Shea, K. (2015) The Digital for-science Pipetting Badge: A Method To Improve Student Hands-On Laboratory Skills. Journal Read, D. & Barnes, S.M. (2015) Review of A- of Chemical Education 92, 2038-2044. DOI: level content. Available at: 10.1021/acs.jchemed.5b00464. http://edshare.soton.ac.uk/14806/ Wilson, T., & Read, D. (2017). Learning from Reid, N. & Shah, I. (2006) The role of Labdog: Best Practice for Laboratory laboratory work in university chemistry. Response System Questions. In Conference Chemistry Education Research and Practice Proceedings. New Perspectives in Science 8, 172-185. DOI: 10.1039/b5rp90026c. Education (p. 299). libreriauniversitaria. it Edizioni. Seery, M.K., Agustian, H.Y., Doidge, E.D., Kucharski, M.M., O’Connor, H.M. & Price, A.

New Directions in the Teaching of Physical Sciences, Volume 13, Issue 1 (2018) https://doi.org/10.29311/ndtps.v0i13.2905