Assessment Moderation and Validation

Session 4: Administration Procedures, Task Specifications and Rubrics

The VET Development Centre provides this guide as part of the deliverables of the External Validation of Assessment Project funded by the Department of Education and Early Childhood Development. April 2015

2 Contents

Introduction

This guide has been designed by the VET Development Centre as part of the External Validation of Assessment Project funded by the Department of Higher Education and Early Childhood Development. It aims to support RTOs in engaging industry in assessment validation and using industry input for continuous improvement. It is designed to be used in accordance with an RTO's quality procedures and RTOs are invited to customise the templates to meet their needs. It remains an RTO’s responsibility to ensure that the methodology and templates are used in a manner that complies with SNR and/or AQTF requirements Validation of assessment is not only an important compliance requirement, it is also a valuable means by which assessment tools and processes can be continuously improved to achieve greater accuracy in the assessment process and strengthen industry confidence in assessment outcomes. A challenge for many RTOs is to find external industry personnel who are available to make the very important industry contribution required in the validation process. This guide provides a process and advice on successfully engaging industry in the validation of assessment. It is modelled on leveraging the many contacts that RTOs make with industry through the provision of training and assessment services. Three principles underpin the strategy for engaging external industry experts in validation. These principles are reflected in the methodology and templates. The first principle is to separate the information sought from industry and the information sought from educators. Industry experts are asked to comment on assessment issues most relevant to them. Educators address the aspects of assessment for which they have specific accountability. The second principle is that language used in communicating with industry participants focuses on securing industry feedback about industry specific skills and knowledge. The pilot project found that using VET jargon impedes industry participation, as does the expectation that industry participants will be familiar with, or wish to engage with, the intricacies of VET system policy and procedures. The third principle is that respect for industry feedback is demonstrated by closure of the external validation process through communicating directly and clearly to industry participants about how their feedback has improved assessment strategies and outcomes.

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 5 Planning for and achieving industry participation in validation of assessment

The process is separated into three stages, each of which is supported by templates. The templates in this guide are available from the VET Development Centre in an electronic form which can be customised to individual RTO requirements.

Stage Related templates

Sta Initial planning Documents A and B ge 1

Sta Educator validation Documents C, D and E ge session 2

Sta Industry validation Documents D, F and G ge session 3

The steps in planning and conducting validation of assessment, recording required improvements and communicating outcomes to stakeholders are summarised on the next page.

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 6 VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 7 Hints for success

Attending to the following factors is likely to increase the willingness of industry experts to contribute to external validation of assessment processes.

 Prepare a validation schedule early in the year so that industry experts can choose to contribute to those sessions they are most interested in, or are available for. Leave space in the schedule to allow industry experts to nominate their own topic of interest or available time, invite them to do so. If appropriate schedule a validation event to accommodate this interest, if not appropriate thank them for their input and explain why it is not being acted on.

 During initial engagement with industry, extend an invitation to participate in the validation of assessment sessions outlined in the schedule, rather than leaving it until later in the year immediately prior to the validation session to issue the invitation. Present the validation schedule as the opportunity for industry experts to choose the topic and when they wish to contribute.

 Advise industry experts about the work being done by educators in the Educator’s Validation session. Explain this means that Industry Validation sessions will consider a refined and updated assessment tool. Their input will be sought only on industry specific issues.

 Present the validation invitation as an opportunity for input not restricted to a specific time and in a specific place. Multiple industry experts’ involvement in validation of assessment maybe required to achieve comprehensive feedback. This can be achieved through meetings attended by a number of industry experts, conversations with individual industry experts, oral or written input, which may be face to face or conducted via communication technology.

 Consider of clustering units of competency around ‘topics’ or ‘themes’ that logically fit together or breaking units down into topics which relate to workplace roles or activities that are meaningful to industry.

 Address RTO concerns about intellectual property and confidentiality by using Document D to record educators’ and industry experts’ commitment to confidentiality. The templates in this guide are available from the VET Development Centre in an electronic form which can be customised to individual RTO requirements.

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 8 Document A RTO Annual Schedule for Validation of Assessment This form is used to document the RTO validation schedule and record industry experts’ interest in participating.

Manager/Program Manager

Training Package

Qualification(s)

RTO planned validation schedule

Dat Locatio Topics for Unit(s) of competency e n/mode validation of assessment

Record of discussions with industry experts on interest and availability for validation

Topic for Locati D Industry Contact validation of on/Mo at expert details assessment de e (phone & /T email) i m e e.g. Framing

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 9 Note: Invite industry experts to nominate topics and times that are not on the RTO planned schedule above and encourage their involvement by scheduling validation events around these suggests as far as possible.

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 10 Document B Annual Assessment Validation Schedule Confirmation for Industry Experts This form is used to confirm industry experts’ commitment to participate in validation.

Workplace

Phone number

Mobile

Student(s) at the RTO (if relevant)

Assessment validation Date Proposed validation method: phone, email, face to session/topic face meeting or other Safe working at heights

Note: This can be completed after initial contact and discussion and then a copy is provided to industry experts by email to confirm their commitment.

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 11 Document C Assessment Material Coversheet for Validation Chair

This coversheet should be completed by the assessor and attached to the assessment tool(s) for each Unit of Competency being validated. It is to be submitted to the Validation Chair who may remove all identifiers prior to the evidence being reviewed during the meeting.

Unit(s) of Competency:

Assessor:

The following materials are attached or enclosed with this form.  Assessment Tool  Instructions to candidates  Assessor Guide  Competency Mapping Tool  Relevant Unit(s) of Competency  Supporting documentation (e.g. Training and Assessment Strategy, unit outlines)

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 12 Document D Attendance and Confidentiality Form

This form is to record attendance and obtain written agreement of panel members regarding confidentiality.

All participants in the assessment validation group are to sign below as acknowledgement of their attendance and agreement with the confidentiality clause.  I agree to observe the principles of confidentiality with regard to Assessment tools and process documentation shared within the assessment validation meeting.  I agree not to use the information shared at validation for commercial advantage or any other reason not acceptable to the validation group.

Assessment Validation Session ______

Date ______

Name Organisation Signature

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 13 Session 4: Administration Procedures, Task Specifications and Rubrics

Document E Validation Record - Educators

Use this form as a record of the Validation of Assessment Educator Panel’s comments and advice on the assessment presented.

Assessment Validation Session

Validation Topic

Unit(s) of Competency Component Type of information to be included Satisfactory List evidence and record of the tool comments. If no, then also Y/N make recommendations

The Context Qualification(s) Unit(s) of Competency Titles and codes correct?

Competency Are the Unit of Competency assessment Mapping mapping documents provided? Are they complete and correct?

Instructions This component outlines the information to to the be provided to the candidate. candidate Does the assessment tool include complete and clear instructions to the candidate (e.g. parts of competency being assessed, time frame, materials and equipment required and assessment conditions)? Are the instructions in language appropriate to the candidate?

The type of Does the assessment tool include a clear information description of the type of task(s) e.g., oral to be response, written response, product collected and/or demonstration? from the candidate

Instructions Does the assessment provide complete and to the clear instructions to assessors e.g. assessor  What adjustments are reasonable?  Type and amount of intervention or support permitted  Evidence criteria  Answers and examples of typical competent responses?

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 14 Component Type of information to be included Satisfactory List evidence and record of the tool comments. If no, then also Y/N make recommendations

Component Type of information to be included Satisfactory List evidence and record of the tool Y/N comments. If no, then also make recommendations

Principles of Does the assessment comply with the Assessment principle of validity (e.g. uses at least two methods of assessment to gather evidence of competence, gathers sufficient evidence; addresses all relevant elements and performance criteria; uses a simulated or actual workplace)?

Does the assessment comply with the principle of reliability (e.g. will return the same outcomes when applied to different contexts and times, or used by different assessors; demonstrates internal consistency; has clear assessor instructions and marking guide)?

Does the assessment comply with the principle of flexibility (e.g. can be adapted to different contexts and conditions; able to make reasonable adjustments – i.e. verbal or written – to meet learner needs; draws on a range of methods)?

Does the assessment comply with the principle of fairness (e.g. reasonable adjustments can be made for equity purposes to accommodate learners’ geographical/ financial/ social needs; assessment does not require language, literacy or numeracy skills more complex than those required by workplace)?

Rules of Does the assessment produce valid Evidence evidence (e.g. clearly aligns with unit being assessed; addresses the performance requirements of the competency standard and the knowledge, skills and other attributes necessary for a competent performance)?

Does the assessment produce authentic evidence (e.g. not plagiarised; clearly the work of the learner – including RPL evidence)?

Does the assessment produce current evidence (e.g. evidence should be recent; must demonstrate current skills and knowledge based on industry standards)

Component Type of information to be included Satisfactory List evidence and record of the tool comments. If no, then also Y/N make recommendations

Does the assessment produce sufficient evidence (e.g. addresses dimensions of the competency standard; collects evidence in at least two ways to determine whether the performance is repeatable over time)?

Component Type of information to be included Satisfactory List evidence and record of the tool Y/N comments. If no, then also make recommendations

Range and Does the assessment tool include clear task conditions time limit or other specific conditions where required?

Materials Does the assessment tool make it clear and what resources are required by the resources assessor to conduct the assessment? required

Recording Does the assessment tool include clear assessment instructions for recording the assessment outcome outcomes?

Supplementar Is other information that will assist the y information assessor in administering and judging the performance of the candidate clear?

Educator sign-off

Date of Validation ______

Validation Chair Signature ______Name ______

RTO represented ______

Signature ______Name ______

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 16 Signature ______Name ______

Summary Validation and Action Record Use this table to document the actions recommended at the validation meeting, who will complete them and by when. Once completed, the Validation Chair must document when the recommended action is completed.

Improvement Action By By Date wh wh complet om en ed

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 18 Document F Validation Record – Industry Experts

Use this form as a record of the industry experts’ input on the assessment presented for validation.

Date ______Assessment topic ______

Unit of competency ______

This assessment has been reviewed by educators. It adheres to the principles of assessment, rules of assessment evidence and provides adequate coverage of the topic. Industry experts, perspective about the assessments will contribute to the validation of this assessment and the RTO’s continuous improvement process.

Assessment Information Comment

Do the required materials and equipment reflect current industry practice?

Is the sequencing of the task(s) correct?

Does the assessment language and terminology reflect that used in industry?

Do the suggested correct answers and/or examples of required performance reflect current industry standards?

Does the assessment reflect current industry standards?

Does the assessment include clear instructions about the time limits or other conditions that are in line with industry practice?

Any general comments or recommendations for improvements to this assessment?

Are there any particular skills or knowledge that an assessor needs to conduct this assessment, e.g. understanding of new techniques, materials or equipment?

Do you think the assessment will allow the candidate the opportunity to demonstrate relevant non-technical skills such as communication, teamwork, reading, writing and numeracy skills appropriate to the subject?

Validation Chairperson

Signature

Industry Experts

Workplace and Work Role

Signature

Note: This record may be of feedback for multiple industry experts at a validation meeting or of input from individual industry experts at different times and in different modes. All points may be not discussed in every instance and feedback on a number of topics may contribute to the validation of a particular unit. The form facilitates the recording of any discussion about assessment so it is seen to contribute to validation and continuous improvement.

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 20 Summary Validation and Action Record Use this table to document the actions recommended by industry experts when validating, who will complete them and by when. Once completed, the Validation Chair must document when the recommended action is completed.

Improvement Action By whom By when Date completed

Document G Summary Validation and Action: Outcomes Advice to External Industry Experts

The Validation Chair uses this table to communicate the planned improvements to the assessment tools and processes to industry experts. A copy of this form should be emailed to industry experts who have contributes to the validation as promptly as possible.

Improvemen Date completed t Action

Validation Chair Signature ______Name ______

Note: If an improvement suggestion cannot be implemented then the reasons why not should be documented and communicated to the industry experts. This ensures industry experts know their suggestions are always valued and implemented if appropriate.

VET Development Centre ǀ Engaging Industry in External Validation of Assessment ǀ A Guide for RTOs ǀ Page 22