1 Оценка компетенций студентов и выпускников СПО: перед лицом новых вызовов

WorldSkills Approaches to Comparable Skills Assessment in WorldSkills Approaches to Comparable Skills Assessment in Vocational Education / Dudyrev, F., Froumin, I., Maltseva, V., Loshkareva, E. and E. Tatarenko ; National Research University Higher School of Economics, Institute of Education; WorldSkills . Moscow: HSE, 2019. — 44 pp. — Sovremennaja analitika obrazovanija [Modern Analytics of Education], No 8 (30).

About authors: Fedor Dudyrev, Head of the Centre for Skills Development and Vocational Education, Institute of Education HSE; Isak Froumin, Head of the Institute of Education HSE; Vera Maltseva, Research Fellow, Centre for Skills Development and Vocational Education, Institute of Education HSE; Ekaterina Loshkareva, Deputy General Director for Research and Development of WorldSkills Russia; Ekaterina Tatarenko, manager of international programs, R&D Department, WorldSkills Russia.

Substantial contributions to the publication were made by the representatives of the WorldSkills movement in , the and the : Ed Chapman (WorldSkills UK), Jill Goddard (WorldSkills UK), Fred van Koot (WorldSkills Netherlands), Mick Prato (Victorian TAFE Association, Australia), Jos van Zwieten (WorldSkills Netherlands), as well as by the certified experts of WorldSkills Russia Alexey Korniliev, Andrey Mochalkin, Evgeny Petrov, Maxim Pevin, and Andrey Pivinsky.

Our sincere thanks go to Inna Chernoskutova, Dmitry Peskov, Irina Potekhina, Peter Schedrovitsky, and Robert Urazov for their insights and valuable comments.

© National Research University Higher School of Economics, Institute of Education, 2019 © WorldSkills Russia, 2019 3

Contents

Introduction 5

1 TVET systems facing skills agenda challenges 9 1.1 Labor market trends shaping global skills agenda 9 1.2 National TVET systems tackling skills agenda challenges 13

2 Skills assessment in TVET facing new challenges 19 2.1 Assessing technical skills: diversity of measurement tools and patterns 19 2.2 Current skill assessment tools in TVET: looking for new criteria 22

3 Skills assessment within WorldSkills methodology: emerging capabilities 27 3.1 WorldSkills competitions: from an elitist championship to broad education and teaching practice 27 3.2 How are skills competitions embedded into national TVET systems? 29 3.3 Benefits of embedding skills competitions based on WorldSkills standards into vocational education 33 3.4 Emerging roles of WorldSkills: from a blue-collar championship to an inter- national benchmark for skills assessment 34

4 Prospects of an international benchmark for skills assessment in TVET 37

Appendix 39 Assignments and assessment procedures used at the skills competitions in Australia, United Kingdom, Netherlands, and Russia: a comparative study • Mechanical Engineering CAD • Refrigeration and Air Conditioning • Electrical Installation 4 Оценка компетенций студентов и выпускников СПО: перед лицом новых вызовов 5

Introduction

TVET institutions around the globe are level cognitive skills and at the post-secondary responsible for providing youths and adults with level. The same challenges of cross-national relevant skills for employment and developing assessment were faced in projects dealing with career paths1. Vocational education and training vocational education. PISA-VET3, a project for cultivates human capital, promotes inclusive large-scale assessment in eight European Union and sustainable economic growth and full and member states, was an attempt to develop productive employment. a cross-country assessment of TVET student Technical skills are the cornerstone of a learning outcomes, yet this assessment was worker’s profile and human capital capability. never implemented. But TVET systems should also help to develop The WorldSkills movement is an initiative and master transferable (both cognitive and that successfully addresses global challenges of non-cognitive) and digital skills for tomorrow’s skills development and assessment. WorldSkills jobs. This set of transferable technical skills will member organizations are present in 82 coun- shape the success of young people in the VUCA tries, including all G20 states, and cover more world. than two-thirds of the world’s population. Skills assessment is a key issue on the WorldSkills has become a global actor in the post-secondary education agenda. Developing field of skills excellence and assessment that valid assessment methodologies appears to be challenging both intellectually and polit- ically. There are very few cases of effective assessments of TVET learning outcomes imple- mented at the national level. Cross-country 1 United Nations (2015) Transforming our World: the 2030 assessments are even more challenging. The Agenda for Sustainable Development. OECD project AHELO (Assessment of Learning 2 AHELO Main Study. OECD supports the assessment of Outcomes in Higher Education)2 is an attempt to learning outcomes in higher education. http://www.oecd. org/education/skills-beyond-school/ahelo-main-study. assess educational outcomes among bachelor htm degree graduates across countries. The imple- mentation of the AHELO study has uncovered 3 Baethge M., Achtenhagen F., Arends L., Babic E., Baethge-Kinsky V., Weber S. (2006) PISA-VET. Feasibility methodological problems inherent in cross-na- Study. Franz Steiner Verlag. tional research especially when assessing high- 6 Introduction

shapes and drives the policy of national TVET • What are the prospects of using the results systems for building the framework of skills of skills competitions based on WorldSkills development and assessment. standards as a valid measure of the quality This publication aims at producing recom- of vocational education at the international, mendations on improving skills assessments of national and sub-national levels? vocational students and graduates by embed- • What is the institutional design of a skills ding WorldSkills standards. These recom- assessment system that ensures the rele- mendations bring together the experience of vant and comparable measurement of national TVET sectors that employ WorldSkills the skills of youth entering labor markets best practices in skills development and espe- (including TVET graduates)? cially objective skills assessment. This report was drafted by the National This report covers the following key topics: Research University Higher School of Economics at the initiative of the Union “WorldSkills • How does rapid technological change trans- Russia” in partnership with representatives form the demand for skills development and of the WorldSkills movement in Australia, the assessment? Netherlands and the United Kingdom. • What well-established and emerging We hope this paper shall contribute to the approaches to skills and qualification discussion of skills development and objec- assessment are used across different coun- tive skills assessment in TVET and promote tries with both advanced and emerging the dissemination of WorldSkills practices for economies? building international comparative skills assess- • What are the prospects of the WorldSkills ments in TVET. methodology for the objective assessment of learning outcomes in TVET? 7 Оценка компетенций студентов и выпускников СПО: перед лицом новых вызовов 8 Оценка компетенций студентов и выпускников СПО: перед лицом новых вызовов 9

1

TVET systems facing skills agenda challenges

1.1 Labor market trends shaping global skills agenda

National TVET systems are facing challenges and 42%, respectively4. On the other hand, driven by global socio-economic transformation new industries, jobs and roles are emerging as and rapid technological change. We list below a result of technological innovations and the the key challenges resulting from disruptive new division of labor. At the same time, other technological innovations that are currently socio-economic transformations (international- shaping labor market transformations and ization of supply chains, increasing urbanization, guiding demand for new roles and skills of youth growing middle class in emerging economies, and adults. population ageing, promotion of the green economy and sustainable development) are • Dramatic increase in the demand for new also contributing to the dramatic changes in roles with a decreasing demand for tradi- global employment.5 tional ones The key point is that the aggregate demand The global labor market is driven by the for labor is increasing, yet this is a demand for two opposite trends. On the one hand, human new roles and new skills. It is estimated that employment in traditional industries is rapidly technological changes that replace routine work being replaced by machine labor due to auto- have created over 23 million jobs across Europe mation. In 2018, an average of 71% of total from 1999 to 2016, or almost half of the total task hours across 12 industries was performed increase in employment over the same period6. by humans, compared to 29% by machines. By According to the World Economic Forum, 75 2022, this average is expected to shift to 58% million jobs have been displaced by a shift 10 TVET systems facing skills agenda challenges

in the division of labor between humans and tasks that have resisted automation so far machines, with the emergence of 133 million (childcare, etc.) are less vulnerable in the new roles that are more adapted to the new face of automation. However, these workers division of labor between humans, machines can suffer from falling wages as the replaced and algorithms 7. The emerging roles in demand middle-skilled routine workers may enter these mostly represent jobs of the digital economy: jobs instead8. For example, middle-skilled jobs data analysts and scientists, software and shrank by 10% in OECD countries excluding applications developers, and ecommerce and and the Czech Republic in 1995- social media specialists. Moreover, there is an 20159. The most highly automated industries accelerating demand for a variety of brandnew experience more evident job polarization (e.g. roles related to deploying the latest emerging manufacturing, financial services and insurance technologies: AI and machine learning special- sector). ists, big data specialists, information security analysts, and robotics engineers. • Increasing demand for transferable skills and new technical skills • Job polarization and a decrease of middle- The role of transferable skills that shape skill jobs flexibility and the ability to adapt to new Technological change is transforming the working patterns and jobs is increasing in the content and skill profiles of existing jobs in the context of rapidly changing labor markets10. sense that it increases the relative demand Transferable or universal skills include complex for non-routine tasks and decreases demand problem-solving, socio-behavioral and self-reg- for routine roles. Therefore, skilled workers ulation skills11. These transferable skills facil- performing non-routine tasks benefit from itate the development of continuous learning technological change. This group includes not skills. The growing skills instability and the only high-skilled workers that benefit from changing job profiles undermine the relevance the growing number of jobs and wage levels of prior work experience and knowledge. The but also middle-skilled workers performing proportion of core skills required to perform a non-routine tasks. Skilled workers performing job is expected to be about 58%, meaning an routine tasks represent the most vulnerable average shift of 42% in required workforce skills group of workers in the face of labor-replacing over the period 2018-202212. automation. Unskilled and low-skilled jobs Transferable skills, including advanced involving non-routine manual and cognitive cognitive skills and adaptability, are equally applicable in both existing and emerging occu- pations. Advanced cognitive skills such as crit- ical thinking and creativity will increase in value. 4 WEF (2018) The Future of Jobs Report 2018. Centre for Socio-behavioral skills, including leadership, the New Economy and Society, World Economic Forum. initiative, emotional intelligence and communi- 5 Bakhshi H., Downing J., Osborne M., Schneider P. (2017) cation & teamwork as well as lifelong learning The Future of Skills: Employment in 2030. : skills are also increasing in demand13. According Pearson and Nesta. to the World Bank, the share of employment in 6 World Bank (2019) World Development Report 2019: The occupations replete with non-routine cognitive Changing Nature of Work. Washington, DC: World Bank. and socio-behavioral skills has increased since 7 WEF (2018) The Future of Jobs Report 2018. Centre for 2001 from 19 to 23% in emerging economies the New Economy and Society, World Economic Forum. and from 33 to 41% in advanced economies14. 8 WTO (2017) World Trade Report. Trade, Technology and The demand for technical skills is experi- Jobs. Geneva: World Trade Organization. encing two trends. On the one hand, routine- 9 OECD (2018) Good Jobs for All in a Changing World of based, middle-skilled roles are expected to Work: The OECD Jobs Strategy, OECD Publishing, Paris. become increasingly redundant. On the other 11 TVET systems facing skills agenda challenges

hand, the amount of non-routine middle-skilled Employer surveys in advanced economies tasks is increasing (design of individual projects, attest to an increasing demand for transferable troubleshooting). Moreover, tasks in traditional skills. The top skills sought by US employers industries are becoming more technologically include complex problem-solving skills (83% of challenging and require high-level skills in STEM employers mention it), teamwork skills (83%), disciplines and emerging innovative sectors. In communication skills (80.3%) and technical this respect, combinations of high-level tech- skills (60%)17. Employers are less satisfied with nical skills and skills in emerging sectors are in the transferable skills of graduates, especially the greatest demand15. high cognitive and socio-behavioral skills, than with their technical skills. According to the • Changing skills requirements for job seekers survey of leading employers in , grad- and employees uates mostly meet the requirements for tech- Employers agree on the significance of a nical/professional skills (95.7% of companies skills gap problem and regard it as an obstacle agree with this statement), while only 73% to business development and economic growth. of companies are satisfied with the level of Hiring high-skilled workers with a combination socio-behavioral skills18. Employers in emerging of advanced technical and transferable skills is economies, especially in innovative companies, more tough than finding middle and low-skilled also have reported a significant skill gap in the workers due to a lower deficit of routine tech- survey conducted under the World Bank STEP nical skills16. Skills Measurement Program19.

10 World Bank (2019) World Development Report 2019: The 16 World Bank (2018) Measuring skills demanded by Changing Nature of Work. Washington, DC: World Bank. employers: skills toward employment and productivity (STEP) (English). Jobs Notes; issue no. 5. Washington, 11 Froumin I., Dobryakova, M., Barannikov, K., Remorenko D.C. : World Bank Group. I. (2018) Key Competences and New Literacy: From Slogans to School Reality. Рreliminary results of the inter- 17 national report of major trends in the on-going transfor- NACE (2018) NACE Job Outlook 2018. National Associa- mation of school education. Issue 2 (19). Moscow: NRU tion of Colleges and Employers. HSE. 18 Business Council of Canada (2018) Navigating change: 12 WEF (2018) The Future of Jobs Report 2018. Centre for 2018 Business Council Skills Survey. Business Council of the New Economy and Society, World Economic Forum. Canada, Morneau Shepell.

13 Ibid. 19 The STEP Skill Measurement Program. World Bank Micro- 14 World Bank (2019) World Development Report 2019: The data Library. http://microdata.worldbank.org/index.php/ Changing Nature of Work. Washington, DC: World Bank. catalog/step/about

15 Ibid.

13 TVET systems facing skills agenda challenges

1.2 National TVET systems tackling skills agenda challenges

The majority of existing TVET institutions challenge could be addressed by enhancing were established during the industrial era and the engagement of industry at all the levels were targeted at educating workers at the of the TVET system by harmonizing occupa- basic and intermediate occupational levels for tional standards (Russia) or the occupational manufacturing jobs. These TVET programs are framework (Australia) or increasing the involve- offered at three ISCED-201120 levels: upper ment of employers in curriculum development secondary (ISCED 3), post-secondary non-ter- and vocational teaching (e.g. apprenticeship tiary (ISCED 4) and short-cycle tertiary educa- programs in the Netherlands and emerging T tion (ISCED 5). TVET providers range from Levels in the UK). secondary schools to universities. Confronted by the post-industrial economy • Developing vocational programs for framework and increasing tertiarization, emerging occupations TVET schools have to rethink their strategies, For a long time, the TVET system has been promote a new image of vocational jobs and living in a well-established framework of qualifi- build links with industry and other stakeholders. cations and jobs. Today’s rapid changes in occu- Current and upcoming socio-economic trans- pations and job content call for harmonizing the formations will have a huge impact on TVET new qualification framework and TVET educa- national policies and strategies. These chal- tional outcomes. Australia is a case in point of a lenges to the education-employment nexus of sound match between qualifications and TVET rapid technological change are shaping national learning outcomes21. TVET agendas in countries that are promoting the development of human capital. • Contributing to increasing labor productivity Below we give an overview of the key chal- via improved TVET provision lenges facing TVET systems in Australia, the Job creation is directly associated with a rise Netherlands, Russia and the United Kingdom in labor productivity. However, increasing labor (Boxes 1-4). These briefs are based on mate- productivity in traditional industries remains rials specially prepared by national experts an urgent task. In the UK TVET sector (see Box for this report. This overview is also based on 4: “Ensuring that teaching and training in TVET strategic and policy papers and analytical and helps UK industry and business to be more expert reports by different national and inter- productive”), this challenge is tackled through national think-tanks, including UNESCO, OECD, the mandatory provision of digital skills training World Bank, ILO, and CEDEFOP. and the promotion of professional identity. The same strategic approach is relevant for Russian • Modernizing vocational programs to meet vocational education22. Enhancing the quality of new labor market requirements for worker Russian TVET can be achieved by embedding skills WorldSkills standards into TVET programs. The biggest challenge for the national TVET sectors covered in this report is to provide youth with up-to-date skills and ensure their readi- ness for future occupations. Thus, keeping up 20 UNESCO (2015) International Standard Classification with the rapidly changing world of work and of Education ISCED 2011.UNECSO Institute for Statis- tics. http://uis.unesco.org/sites/default/files/docu- catering to the needs of employers and industry ments/international-standard-classification-of-educa- are the most acute issues across countries. This tion-isced-2011-en.pdf . 14 TVET systems facing skills agenda challenges

Box 1. TVET in Australia

ISCED 2-5 4.5 million 9,328 $ TVET programs TVET students PPP total expenditure on upper secondary TVET per student

of upper of lower secondary employment rate 50,5 % secondary 18,6% students are 78% of TVET students are enrolled in TVET graduates23 enrolled in TVET

Challenges facing TVET in Australia:

• Need to update training packages • Making TVET more attractive • Strengthening the role of TVET as faster for school leavers and increasing an institution of social inclusion • Improving the quality of training completion rates in TVET • Balancing the initial training of for tackling skills shortages young people with the re-training of older workers

Box 2. TVET in the Netherlands

ISCED 2-5 0.9 million 14,698 TVET programs TVET students PPP total expenditure on upper secondary TVET per student

of upper secondary employment rate for students are TVET TVET graduates (20- to 66,4 % 84,1% students 34-year-olds)25

Challenges facing TVET in the Netherlands:

• Ensuring access to further learning • Sustaining work-based learning • Providing flexible postsecondary opportunities for TVET graduates as the core element of the upper TVET programs for adults • Building continuous learning secondary TVET system trajectories between vocational • Tackling the expected shortage of programs teachers especially in TVET 15 TVET systems facing skills agenda challenges

Box 3. TVET in Russia

ISCED 3-5 2.9 million 3,664 $ TVET programs TVET students PPP total expenditure on upper secondary TVET per student

of lower of upper employment 47,3 % secondary 20,7 % secondary 79% rate of TVET students are students are graduates26 enrolled in TVET enrolled in TVET

Challenges facing TVET in Russia:

• Providing a skilled workforce for • Increasing capabilities for the • Enhancing the involvement of hi-tech exporting industries professional development of TVET employers and industry to update • Raising the quality of vocational teachers and trainers the curriculum and initiate flexible education by incorporating interna- • Facilitating social inclusion TVET programs for adults tional professional standards by improving youth access to vocational education

Box 4. TVET in the United Kingdom

ISCED 3-5 2.2 million 9,440 $ TVET programs TVET students (63% of whom are adults) PPP total expenditure on upper secondary TVET per student

of upper secondary employment rate for students TVET graduates 42 % (16-17-year-olds) 79% (20- to 34-year-olds)24 study in FE colleges

Challenges facing TVET in the UK:

• Ensuring that teaching and • Maintaining close links with • Improving accessibility and training in TVET helps UK industry industry and business progression in TVET and business to be more productive • Improving the profile and esteem • Tackling the skills shortage and of TVET better meeting employers’ skills • Sustaining high-quality services needs to learners (both young people and adults) and employers 16 TVET systems facing skills agenda challenges

formations, TVET must strengthen its role as • Enhancing the image and attractiveness of an institution of social inclusion, an idea that is vocational education present in the TVET strategies of all four coun- To improve the attractiveness of TVET, coun- tries covered in this survey. tries implement various policies. Some coun- tries provide vocational programs at the tertiary • Strengthening the role of the objective level (e.g. , the Netherlands, ), assessment of learning outcomes in TVET while others, e.g. Russia, strive to promote In recent decades, the assessment of TVET, including the promotion of WorldSkills learning outcomes has received special atten- competitions. Nevertheless, the urgent issue of tion. There are several reasons for this. First expanding the profile and raising the esteem of of all, these assessments could measure the TVET is not limited to the promotion of blue- efficiency of TVET spending, especially in view collar jobs. The real challenge is to enhance of ever-growing public and private expenditures TVET graduates’ employability, standard of on TVET in high-tech industries. Secondly, there living and career prospects. This challenge is a growing need for valid tools for assessing is addressed by the national TVET agendas and certifying qualifications in the context of of most countries covered by the WorldSkills increasing labor migration and mobility. Thirdly, movement. due to the growing internationalization of business activities, companies are interested • Facilitating social inclusion and mitigating in standardized skills assessment for job appli- poverty via TVET cants and employees that would ensure valid Rapid technological development and struc- and comparable measurements. These reasons tural changes in the labor market increase the show why the issue of objective skills assess- risks of unemployment, poverty and inequality. ment is at the top of the list of challenges facing Vocational education could contribute to the national TVET systems. mitigation of these risks. Vocational education has been facilitating social inclusion for such disadvantaged groups as vulnerable youth, migrants, refugees and people with disabilities. In the face of dramatic socio-economic trans- 17 Оценка компетенций студентов и выпускников СПО: передTVET лицом systems новых facing вызовов skills agenda challenges

21 Australian Qualifications Framework. Second Edition. 25 Fazekas M. and I. Litjens (2014) A Skills beyond School January 2013. Australian Qualifications Frame- Review of the Netherlands, OECD Reviews of Vocational work Council. https://www.aqf.edu.au/sites/aqf/files/ Education and Training, OECD Publishing. aqf-2nd-edition-january-2013.pdf CEDEFOP (2018) Developments in vocational educa- tion and training policy in 2015-17: the Netherlands. CEDEFOP monitoring and analysis of VET policies. 22 Presidential Executive Order No. 204 dated May 7, 2018 Smulders H., Cox A., Westerhuis A. (2016)Vocational On National Goals and Strategic Objectives of the Russian education and training in Europe – Netherlands. Federation through to 2024. CEDEFOP ReferNet VET in Europe reports. CEDEFOP (2016) Vocational education and training in the 23 UNESCO-UNEVOC (2018) TVET Country Profile. Australia. Netherlands: short description. : Publications November 2018. UNESCO-UNEVOC. Office. CEDEFOP information series. NCVER (2018) Total VET students and courses 2015. OECD (2018) Education at a Glance: OECD Indicators, https://www.voced.edu.au/content/ngv%3A73628 OECD Publishing, Paris. OECD (2018) Education at a Glance: OECD Indicators, OECD Publishing, Paris. 26 Indicators of the TVET sector in Russia. Available at: 24 CEDEFOP (2018) Developments in vocational educa- http://indicators.miccedu.ru/monitoring/?m=spo tion and training policy in 2015-17: United Kingdom OECD (2018) Education at a Glance: OECD Indicators, (England). CEDEFOP monitoring and analysis of VET OECD Publishing, Paris. policies. Dudyrev F., Romanova O., Travkin P. (2019) Employment OECD (2018) Education at a Glance: OECD Indicators, of Vocational Graduates: Still a Slough or Already a Ford? OECD Publishing, Paris. Voprosy obrazovaniya / Educational Studies Moscow, Abusland, T. (2019) Vocational education and training no.1, pp. 109-136. in Europe: United Kingdom. CEDEFOP ReferNet VET in Europe reports 2018. 18 Оценка компетенций студентов и выпускников СПО: перед лицом новых вызовов 19

2

Skills assessment in TVET facing new challenges

2.1. Assessing technical skills: diversity of measurement tools and patterns

Qualification assessment and recognition assessment tools and tasks for the exam- has become increasingly significant in the ination are also produced by employers and knowledge-based economy. Qualification exam- experts. For instance, the content of the ination is the key tool in assessing skills. Tasks assessment in Germany is developed by for exams are developed based on occupa- Chambers of Crafts and Trades, and Cham- tional standards and descriptions of workplace bers of Commerce and Industry, which also functions and related workplace competen- award qualifications. Qualification exam- cies. Qualification requirements established in ination is typically combined with the final occupational standards are typically linked to certification. national qualification frameworks. Qualification assessments have various institutional designs • In other countries such as the United (or models) depending on the country’s labor Kingdom, Australia, and the , market and level of technological advance- independent credentialing organizations ment. Below is a comparative overview of such are responsible for qualification assess- models in different countries. ment and certification. These organizations are given the right to award qualifications • In Ireland, Germany and Finland qualifica- through legislation, or they are empowered tion assessment is conducted after grad- by a national authority. See Box 5 for two uation. The assessment is carried out by examples of such assessment centres in the industry experts (not TVET institutions), and United Kingdom. 20 Skills assessment in TVET facing new challenges

• In the CIS (Commonwealth of Independent content of tasks are guided by the national States) countries, including Russia, TVET authority, with the involvement of industry institutions are the key players in the qual- experts. According to this model, a graduate ifications assessment system. Based on who does not pass the qualification exam is the results of the final certification, TVET not entitled to perform a job even if he has graduates are awarded a diploma certifying been awarded a diploma. both successful graduation and qualifica- tion. Involvement of industry experts and Despite these differences in institutional employers in qualification assessment is models of qualification assessment, the mode typically unsubstantial, which diminishes of qualification examination and the assess- the reliability of the exam results. Therefore, ment tools themselves are almost the same Russia is gradually moving to the model, in most countries. Generally, examination which combines final certification with a combines a theoretical exam and a practical demonstration exam based on WorldSkills task. The theoretical exam involves completing standards, and an independent qualifica- a test or some written task, while the practical tion assessment carried out in independent part is a work-related practical task in a simu- assessment centres. lated environment, e.g. the student is required to grind a part on a machine, prepare a dish, • The processes of assessment and awarding write a program, etc. The two parts of the exam qualifications in health care, engineering and may be split in time, and the time limit for the other occupations under special government practical task is set by the amount of time it supervision have certain characteristics would have taken to complete in real working in all countries. Qualification assessment conditions (see the case of Finland). happens upon degree completion, and The institutional models of qualification national authorities responsible for a given assessment described above were shaped category of occupations are responsible for through the course of the XIX-XX centuries. assessment. The assessment procedure and They correspond to national institutional and

Box 5. Assessment centres in the UK

EAL is the awarding organization for engineering, manu- The City and Guilds of London Institute is the awarding facturing, building services and related sectors. Over organization in the UK. It offers 37 qualifications in the 100,000 learners embark on an EAL qualification each building services industry, 61 qualifications in business year in schools, colleges, universities, private training services, and others27. facilities and workplaces. Each of EAL’s recognized centres is allocated a dedicated External Verifier (EV). EAL also provides to the centres a range of learning and assessment resources, from learner guides and delivery 27 Gasskov V. (2018) Development Of Occupational Stan- support materials to assessment tools. dards, Qualifications and Skills Assessment Instruments (Stock-taking technical report). Project “Applying the G20 Training Strategy: A Partnership of the ILO and the Russian Federation” (2nd Phase). 21 Skills assessment in TVET facing new challenges

cultural landscapes and enable a smooth tran- opment of intensive tasks, the completion sition into the workplace, confirming the quali- of which requires expensive supplies and fications of both recent graduates and mid-ca- remuneration for experts. Thus, institutions reer workers. responsible for qualification examinations are looking for cost reduction. Some have National assessment systems are facing new looked to replace equipment with VR. Devel- challenges due to the transition to the knowl- oping this kind of substitute for physical edge economy. These challenges are linked to equipment could improve access to qualifi- job polarization and the changing nature of work cation assessment for students from various (these trends are discussed in Section 1): TVET schools.

• Current socio-economic and technological • Another hot topic in skills assessment deals change is transforming job profiles that with measuring transferable or universal were stable during the industrial era, and skills, the demand for which is accelerating. is fueling skill instability. Requirements for The key point is that these skills must be workers are tightening and becoming more assessed in the occupational context rather varied, and standardized skills profiles are than on their own. being replaced by “bespoke” qualifications. In this context, the prime aim when devel- The following section contains an over- oping new assessment tools is identifying view of current skills assessment projects and the core skills for the related qualifications, initiatives in TVET, which illustrates the trends and then measuring those core skills. mentioned above.

• Conducting qualification examinations remains quite costly. It requires the devel- 22 Skills assessment in TVET facing new challenges

2.2 Current skill assessment tools in TVET: looking for new criteria

The need for standardized assessments The issue of objectively assessing the skills applicable to measuring the skills of learners, of new graduates in terms of their employability job-seekers and employees is widely recog- and credentials is significant for emerging econ- nized across governments, think-tanks, poli- omies marked by large informal employment cymakers and experts. On one hand, stake- and a high dropout rate for school children. holders are seeking to improve conventional This is an urgent issue for , an economy standardized assessment and measurements with a big youth population and a high rate of of skills and qualifications. On the other hand, youth unemployment. In an effort to facilitate they are seeking new approaches to skills hiring, Aspiring Minds, an Indian company, assessment and are attempting to identify the has launched the educational project AMCAT core skills that are in-demand, despite growing (Aspiring Minds’ Computer Adaptive Test) in skills instability and the transformation of job 2008 (Box 8). profiles. One of the initiatives that is attempting This glimpse of the current projects of to rethink conventional assessment of learning assessing skills in TVET shows that challenges outcomes is the COMET project (Box 6). remain to be addressed. Individual countries Employers in both advanced and developing and companies have been making efforts in economies report growing demand for transfer- developing new innovative approaches to skills able skills that are applicable to any profession measurement which assess a complex set of and shape professional success. Awareness of skills responsive to the needs of the knowledge the significance of these skills has contributed economy. to the emergence of new assessment tools. The Recently, issues of TVET sector development WorkKeys project launched by the АСТ (Amer- have become one of the most often debated ican College of Testing), one of the leaders in topics of both national and international standardized testing in the U.S, is one such agendas. A key point of contention comes from assessment initiative (Box 7). the fact that until recently, the majority of inter- Striking features of the WorkKeys assess- national labor migrants have been high-skilled ments include tests that are based on situa- workers or employees of multinational corpo- tions in the everyday working world, as well rations, while blue-collar workers have mostly as a built-in function that links the level of found employment on domestic labor markets. skill proficiency with those required in actual occupations. WorkKeys assessments is linked to the ACT JobPro database, which lists the skills profiles of 90% of U.S. jobs. JobPro skills 28 Zhao Z., Zhou Y. (2019) Professional Competence Assess- profiles for various jobs are used to determine ment Diagnostics as an Instrument for Quality Assurance in TVET / S. McGrath et al. (eds.), Handbook of Vocational the skills requirements that are tested by Education and Training. Springer Nature AG. WorkKeys assessments. After passing the test, Rauner F., Heinemann L., Maurer A., Haasler B., Erdwien B., a job-seeker receives his skill proficiency level Martens T. (2013) Competence Development and Assess- ment in TVET (COMET) Theoretical Framework and Empir- and can compare this with the level required in ical Results. Springer Science+Business Media Dordrecht. the chosen occupation. Rauner F., Heinemann L., Hauschildt U., Piening D. (2012) Project Report: COMET-Pilot Test including the Pilot Test Vocational Identity/Occupational Commitment and Results of the QRC Pilot Project. University of Bremen, TVET Research Group. 23 Skills assessment in TVET facing new challenges

Box 6. The COMET project

The German project COMET (Competence Develop- The eight criteria are clarity/presentation, functionality/ ment and Assessment in TVET) is an ambitious effort to operability, sustainability/utility, efficiency/effectiveness, test a methodology for competence diagnostics in TVET. business and work process orientation, social accept- ability, environmental compatibility, and creativity. The The COMET assessment includes four test tasks COMET assessment is expected to compare training which take 120 minutes to complete, test instruments quality across TVET institutions and countries. The measuring vocational identity, and context question- pilot COMET test was successfully run in Germany, naires. The design of the test tasks is based on typical , and South Africa. The COMET assessment model professional tasks, and are domain-specific. has been accepted and is continuing to be developed In order to develop test tasks and assess the test through research and TVET practices in various coun- 28 participants’ solutions, the COMET has established eight tries, especially Germany and China . criteria to evaluate the task solutions.

Box 7. ACT WorkKeys

Since their launch in 1992, the WorkKeys assessments They can earn one of four possible levels of readiness have targeted measuring foundational workplace skills. on the certificate (from to platinum). The mission of ACT WorkKeys is to enable a smooth The ACT NCRC is nationally recognized, with more than transition from school to work and build the work 22 thousand employers recognizing NCRC as a creden- readiness of new graduates. There are 11 WorkKeys tial that certifies foundational workplace skills. More computer-based skills assessments, including applied than 20 million WorkKeys assessments were passed mathematics, graphic literacy, workplace documents, since 1992, and more than 4 million ACT NCRC certifi- business writing, etc. Job-seekers who pass the three cates have been issued since 2006. NCRC is beneficial assessments in applied mathematics, graphic literacy, for employers, as there is a significant decrease in and workplace documents qualify for ACT’s National recruiting costs29. Career Readiness Certificate (NCRC).

Box 8. The AMCAT project

AMCAT is a computer adaptive test, meaning that the AMCAT is targeted at qualified job-seekers, especially test changes based on a user’s answers. The AMCAT in the IT sector. Therefore, besides the five compulsory syllabus encompasses five modules. The assessment modules, AMCAT syllabus provides optional modules measures aptitude in English, quantitative ability, logic, assessing applied skills, mostly in Computer Science. and information gathering. The last part is an overall AMCAT is widely recognized in India. By 2019 more personality assessment of the candidate and is based than 2 million people in India have taken AMCAT, and on the five-factor model of personality. 700+ companies are using it as a compulsory testing mechanism for entry-level rolesв30. 24 Skills assessment in TVET facing new challenges

Due to globalization and a new division of labor 29 Anderson S., Harris R., Fluharty M., McMahon J. (2018), ACT on an international level, blue-collar workers WorkKeys Solutions 101, ACT Workforce Summit Pre-Con- ference. ACT Workforce Summit 2018. are now entering the international labor market. ACT WorkKeys. https://www.act.org/content/act/en/prod- Therefore, the issues of what is required of ucts-and-services/workkeys-for-job-seekers.html middle-skilled workers and the comparability of blue-collar qualifications has put on the interna- 30 AMCAT. https://www.myamcat.com// tional agenda. This calls for developing unified training standards and assessment procedures in TVET across countries.

This is one of the reasons for the growing popularity of the WorldSkills movement, which is covered in the next section of the report. 25 Оценка компетенций студентов и выпускников СПО: перед лицом новых вызовов 26 Оценка компетенций студентов и выпускников СПО: перед лицом новых вызовов 27

3

Skills assessment within WorldSkills methodology: emerging capabilities

3.1 WorldSkills competitions: from an elitist championship to broad education and teaching practice

WorldSkills is one of the most influential execute specific tasks they are studying or international movements aiming to enhance already performing at their workplace. the image of TVET and the prestige of blue- One of the main aspects of the WorldSkills collar occupations through skills competitions. competition is raising the visibility and high- Launched in in 1950 as a competition lighting the importance of professional educa- for artisans, WorldSkills has built an impres- tion as a tool of socio-economic transformation. sive presence in the TVET world arena, with a The event also provides leaders in industry, constantly expanding number of participating government and education with the opportunity countries and skills covered by biennial interna- to exchange information and the best practices tional skills competitions. in contemporary industrial technologies and Held every two years, the WorldSkills professional education. competition is the biggest vocational education WorldSkills competitions are contests event in the world that highlights the best prac- of champions, showcasing skill excellence. tices of modern skill training. Competitors, who WorldSkills standards are regarded as a global are mostly under the age of 23 , are selected reference point both for well-established skills through competitions in many countries and (e.g. hairdressing and automobile technologies) regions. They demonstrate their technical and hi-tech & emerging skills (reverse engi- ability, both individually and collectively, to neering, UAV operation). 28 Skills assessment within WorldSkills methodology: emerging capabilities

At the same time, WorldSkills is not just an of the WorldSkills Movement. Thus the TVET elitist network of champions and an organizer sector has become a key channel for promoting of international skills competitions but has a WorldSkills Standards at the national and much broader mission: raising global bench- subnational levels in WorldSkills countries. marks and improving skills levels and career We give below an overview of the ways in opportunities worldwide. In this respect, the which skills competitions are embedded into TVET sector is the key stakeholder and partner national TVET sectors. 29 Skills assessment within WorldSkills methodology: emerging capabilities

3.2 How are skills competitions embedded into national TVET systems?

Australia

Australia uses a competency-based assess- Test Projects are developed with different ment (CBA) methodology for awarding TVET degrees of complexity that serve as step- qualifications, and competitions based on ping-stones for moving up through the different WorldSkills standards are a means of moving in levels of competition, as follows: this direction. Competitions bring out the best in young people, push them to excel, lift aspira- • Regional Competitions – 6-8 hour duration tions and, if used effectively, enhance learning. (one day), assignments of a basic level of Competitions are conducted over a two-year complexity, intended for a broad range of period: regional competitions leading to a learners with different experience and skill national competition the following year. The levels, encompassing the learning outcomes different states and territories are divided into of the entire qualification, irrelevant of the 34 WorldSkills regions for the purposes of participants’ stages of education; conducting the event (Table 1). • National Competitions – 18-22 hour dura- Competitions are conducted in 55 TVET tion (three days), tasks are more detailed institutions, usually as stand-alone events and complex with the inclusion of some lasting a single day. Assignments (Test Projects) aspects of the international level of the are developed by WorldSkills Australia volun- competition embedded into the test project. teers, most of whom are TVET educators. Test Projects take into account and are associated with competency units within TVET qualifica- tions.

Table 1. Australian Skills Competitions in Numbers

Indicator 2016 2017 2018

Participating institutions 55 59 55

Number of participants n/a >3,000 n/a

Number of competitions n/a 534 n/a

Number of finals 1 – 1

Number of finalists 524 – 490

Preparation hours for students >60,000 n/a >60,000 30 Skills assessment within WorldSkills methodology: emerging capabilities

Netherlands

Skill competitions have become widespread established by WorldSkills Netherlands. This in both vocational education and secondary CRS contains over 400 assignments in different school in the Netherlands. Competitions are disciplines. called Skills Heroes for vocational students and VET institutions use these assignments and Skills Talents for secondary students31 (Table 2). assessment criteria for day-to-day education VET institutions are actively involved in after adjusting them to industry needs. More- preparing their students for the skills contests, over, these tasks are integrating into the skills but this practice does not aim at training cham- assessment procedure at each stage of training. pions. Thus schools use contests as a tool to The key point is that WorldSkills standards, examine all students and not solely top-per- adapted to everyday teaching and exam prepa- forming students. Skill competitions have ration, shape training programs and TVET become an integral part of the curriculum and curricula at the microlevel. Thus the WorldSkills program. In this context, all students and not competition approach has been successfully just contestants benefit from the power of skills. embedded into Dutch vocational education and Let us give a few details on how WorldSkills is making TVET more challenging and effective. practices are disseminated and embedded into Dutch TVET institutions. Assignments and assessment criteria that have been used in previous EuroSkills events 31 van Zwieten, J. (2018) Igniting Education: Making more and Dutch competitions are stored online in and better use of skills competitions. CINOP Advies and WorldSkills Netherlands. the competition registration system (CRS),

Table 2. Dutch Skills Competitions in Numbers

Indicator 2016 2017 2018

Participating institutions 55 55 56

Number of participants >3,000 >6,000 >10,000

Number of competitions 34 60 65

Number of finals 34 40 60

Number of finalists 210 410 640

Preparation hours for students n/a 170,000 >283,000 31 Skills assessment within WorldSkills methodology: emerging capabilities

Russia

In 2013, the Russian Federation took part One of these assignments based on for the first time in the international WorldSkills WorldSkills standards is embedded into the Competition hosted in , Germany. final certification in order to measure TVET Since 2016, WorldSkills practices have been graduates’ qualifications against international embedded into vocational education in Russia benchmarks. The demonstration exam is run through the implementation of the demonstra- by certified experts. After passing the demon- tion exam. stration exam, the student receives an indi- A beta-testing of a demonstration exam- vidual digital SkillsPassport. The data from the ination was run in 13 regions of the Russian SkillsPassport is stored and made available to Federation, and the number of regions and the WorldSkills Russia partner network and to skills covered have expanded year by year. More employers that recognize the demonstration than 50 thousand students from 1,265 TVET exam result as being an independent certifica- institutions in 81 regions have already taken tion of technical and professional skills. the demonstration exam in 2019 (Table 3). By 2024, all graduates of vocational programs in emerging and in-demand occupations are expected to take the demonstration exam and measure their qualifications against WorldSkills benchmarks. Assignments for the demonstration exam are developed by the experts of the Union “WorldSkills Russia” in accordance with assign- ments for national skills competitions.

Table 3. Russia WorldSkills Standards Demonstration Exam in Numbers

Indicator 2017 2018 2019

Participating regions 26 59 81

Participating TVET institutions 244 772 1,265

Number of participants 13,999 30,579 50,718

Registered Demonstration Exam Centers 335 949 1,650 32 Skills assessment within WorldSkills methodology: emerging capabilities

United Kingdom

From the perspective of government, The assignments are developed by special- competitions are a key factor contributing to ists working at Competition Organizing Part- a number of UK national priorities, in partic- ners. The experts who design the test projects ular in the domain of technical education and are currently employed in the corresponding apprenticeships. WorldSkills UK also meets a industry or educational sector and have certified number of broader policy priorities including the up-to-date skills. UK Industrial Strategy, career counselling and The role of competitions in TVET curricula education, and diversity & inclusion. and programs varies between organizations and 87% of UK general FE colleges are engaged regions of the UK. The more advanced use of in WorldSkills UK. A number of training orga- competitions in curricula involves competitions nizations and employers also help to organize being embedded into the curriculum. More- competitions in the UK. Education inspection over, assignments can be used for personal and authorities recognize the value of these compe- professional development of educators and titions in promoting better quality training. trainers. Competitions are also widely employed The government provides significant core to showcase organizations to future students funding to WorldSkills UK. Apprenticeship or apprentices and to engage in dialogue with program funding can also be used to support employers32. the costs of participating in skills competitions. Locally and regionally, competitions contribute to improving the professional development of educators, raising the aspirations and moti- 32 LSIS (2012), Inspiring Excellence: A Guide to Embedding vation of learners, and enhancing employer Skills Competition Activity in Apprenticeships and Voca- tional Learning. Coventry: Learning and Skills Improvement engagement. These features can help local Service. areas and regions meet their economic and social priorities.

Table 4. UK Skills Competitions in Numbers

87 % of general FE colleges are engaged with WorldSkills UK

>15,500 students involved in competitions

60 competition sectors

200 inclusive competitions in 9 sectors 33 Skills assessment within WorldSkills methodology: emerging capabilities

3.3 Benefits of embedding skills competitions based on WorldSkills standards into vocational education

Our study leads to several conclusions on Skills competitions appear to be an effec- the key benefits of WorldSkills practices for tive tool for selecting talented students and improving quality of vocational education: developing these talents further;

• WorldSkills Standards contain consistent • The WorldSkills methodology is a bench- and reliable benchmarks for TVET skills mark of excellence for the professional and qualifications. TVET institutions revise development of TVET teachers and trainers. curriculum and learning assessment meth- TVET teachers gain new professional experi- odologies on the basis of these well-recog- ence and learn new pedagogical approaches nized and reputable standards; while organizing competitions and preparing students for contests, as well as entering • Skills competitions offer real career experi- international professional networks to share ence: students are challenged to achieve a expertise and innovation; level of practice that is professional, in-de- mand and expected, as well as to master • Data and experience, accumulated by communication and teamwork skills. This WorldSkills, could be a solid ground for learning-by-doing approach helps contes- developing national skills systems and inter- tants to make a smooth transition from national comparability, measuring learning school to work and fosters the formation of outcomes and building individual educa- professional identity, independence, and tional trajectories. initiative;

• The WorldSkills methodology supplements the learning process with competitions as a ludic component. This encourages students to realize their potential and study hard; 34 Skills assessment within WorldSkills methodology: emerging capabilities

3.4 Emerging roles of WorldSkills: from a blue-collar championship to an international benchmark for skills assessment

Skills competitions have emerged as a The key findings of this comparative study widespread practice in TVET sectors in various are as follows. countries. This is demonstrated in the overview Skills competitions assignments are based of skills competitions embedded in vocational on WorldSkills Standards Specifications education systems in Australia, Netherlands, (WSSS)33. Member countries adjust interna- Russia and the United Kingdom (section 3.2). tional WorldSkills assignments for their national Tens of thousands of vocational students have skills competitions. These adjustments include, already passed through skills assessments on the one hand, exclusion of individual parts of based on WorldSkills standards. In some the task and reduction of the allotted time for countries, the vast majority of colleges already task completion, and on the other, additional participate in skills competitions (e.g. in the requirements and modified structure. However, United Kingdom). The involvement of TVET experts emphasize that the structure and the institutions in Russia is increasing rapidly. This “core” of the assignments at national skills rapid dissemination of WorldSkills practices competitions adhere to the WorldSkills assign- in Russia leaves little doubt that in a 3-5-year ments from the international championship. perspective the bulk of colleges will be involved Thus, the assessment methodology in assessments of vocational skills based on employed at international WorldSkills champi- WSI standards. onships is an international benchmark for skills The dissemination of skills competitions assessment, which links skills requirements creates new possibilities for measuring the across countries. Following WorldSkills stan- learning outcomes and skills of TVET students. dards is essential for ensuring the compara- WorldSkills member organizations already have bility of learning outcomes and qualifications access to a wealth of data on participants and awarded across TVET schools and nations. their skills, and these valuable datasets are of Therefore, this enables countries to improve great research interest. their global competitiveness not only in terms In order to examine the potential for of training champions but also in terms of a comparative analysis of skills assessment in system-wide improvement of workforce skills. skills competitions, certified WorldSkills experts The WorldSkills methodology encompasses have compared assignments and assessment both components of an assessment procedure: procedures used at national skills competi- what is assessed (abilities and skills by occupa- tions in Australia, United Kingdom, Nether- tion) and how to assess (procedure and criteria). lands, Russia, and the International WorldSkills Its assessment procedure is highly detailed and Championship. The three skills covered in this itemizes each aspect of performance, criteria, comparison study are Mechanical Engineering and sub-criteria with relevant weightings. This CAD, Refrigeration and Air Conditioning, and helps to reduce bias in skills assessment. Electrical Installation. This comparative study is available in appendices.

33 Worldskills Standards Specifications. https://worldskills.org/ what/projects/wsss/ 35 Skills assessment within WorldSkills methodology: emerging capabilities

The WorldSkills methodology shapes New prospects for further unification of assessment procedures and criteria used in skills assessment at international and national national skills competitions – this has become competitions could be reached through full or evident from the comparative analysis. This is partial automation. This is primarily applicable one more argument for the high potential of to skills in engineering, construction, and IT comparability among skills assessments within occupations. Pilot projects of skills assessments skills competitions, and ultimately among with automation have already been launched in learning outcomes in TVET systems. Russia and China in several fields. 36 Оценка компетенций студентов и выпускников СПО: перед лицом новых вызовов 37 4

Prospects of an international bench- mark for skills assessment in TVET

There is a need for tangible evidence on including those that are based on WorldSkills which to ground analysis of the quality of voca- standards. Consideration of these issues tional education systems, the efficiency of requires broad discussion involving researchers, public and private spending on TVET. Data is experts, as well as educators and other practi- also needed for matching graduates’ skills with tioners. employers’ needs. Objective skills assessment WorldSkills has made a remarkable provides solid grounds for shaping policies in contribution to the development of skills TVET. assessments, and is one of many initiatives Skills competitions that use WorldSkills undertaken by individual countries and inter- standards provide invaluable insights for national organizations. Demand for objective policy-makers, and could be leveraged in skills assessment in TVET is accelerating in measuring the learning outcomes of TVET the context of rapid technological change and students and graduates. The assessment associated challenges for the TVET sector. At procedure initially developed for international the same time, the risks associated with a lack skills championships has transformed into a of coordination of initiatives and projects in global benchmark for national and regional skills assessment also are increasing. There is standards, and now guides skills assessments already an example of such a mismatch in the across TVET institutions. Further promotion of development of assessment tools for measuring this global benchmark requires widespread and transferable skills. standardized adoption of WorldSkills practices Tackling these issues requires the active by national TVET sectors. engagement of the international organizations Developing valid and reliable assessment and national think-tanks that shape and guide tools for skills proficiency requires significant the TVET policy agenda. We should join hands research efforts. There remain challenges to in tackling the challenges facing TVET and be addressed. How can we simultaneously fuel the engagement of national and interna- assess multiple skills, including technical and tional stakeholders in the pursuit of developing transferable skills? What are the prospects for international standards for skills assessment measuring digital skills, keeping in mind the in TVET. The ultimate goal of such initiatives is increasing digitalization of all occupations? more inclusive and equal access of youth to the Another issue is associated with the costly labor market. procedures of existing skills assessments, 38 Prospects of an international benchmark for skills assessment in TVET 39

Appendix. Assignments and assessment procedures used at the skills competitions in Australia, United Kingdom, Netherlands, and Russia: a compara- tive study

• Mechanical Engineering CAD • Refrigeration and Air Conditioning • Electrical Installation

Mechanical Engineering CAD

Mechanical Engineering CAD is a well-es- • Module A, “Mechanical assemblies and tablished skills competition among interna- detail drawing for manufacture,” involves tional WorldSkills competitions. Initially, it producing a 3D model of a component was a competition for drafters equipped with according to the assembly drawing. paperboard and paper. Since 2001, all drawings • In Module B, “Mechanical Fabrication,” have been executed with professional design participants model the required sheet metal software. Currently, the key sponsor of skills parts, create frames and various connec- competitions in Mechanical Engineering CAD is tions. AutoDesk Inc., one of the global leaders in 3D • Module C, “Mechanical Design Challenge,” design and engineering software. is dedicated to reviewing a design brief and The CAD competition assignment estab- creating a new version of the design. lishes skills requirements for occupations • Module D, “Reverse Engineering from a related to 3D modelling of components, parts, Physical Model,” involves 3D modelling a metal constructions, and consumer goods. physical part, then producing its drawing. The task includes four modules, each of which Finally, participants produce a rendered requires various skills in mechanical engi- image of the part. neering. 40 Appendix

Member countries make adjustments to The assignment in Mechanical Engineering international WorldSkills assignments when CAD for the national skills competition in the embedding assignments in their national United Kingdom is based on the assignment competitions. These adjustments include, on at WorldSkills São Paulo 2015. However, the the one hand, exclusion of individual parts of assignment modifies Module A with simpli- the task and reduction of the allotted time for fied tasks as compared to the international task completion. On the other hand, national benchmark. Module B of the UK assignment assignments are frequently supplemented with involves tasks from Part C of the assignment at additional requirements and modified structure. WorldSkills Competition 2015 (e.g. building a The assignment in Mechanical Engineering parametric subassembly). Moreover, in Part B CAD for the national skills competition in the participants must produce an animation similar Netherlands involves 3D modelling the product, to the task from Part C, instead of creating two then reviewing and modifying its design. Partic- assembly animations. ipants are not expected to complete the task The CAD assignment used in the demon- included in Module D “Reverse Engineering stration exam in Russia contains Modules A, from a Physical Model.” But aside from model- B and C. The first part of the demonstration ling the product they are also required to exam matches the international assignment produce relevant calculations and a brief cost of Module A both in terms of allotted time and analysis. In the final result, the assignment complexity. The second and third parts of the starts to look like a true engineering project. demonstration exam assignment correspond Its successful completion requires knowledge with Module B and C, but the tasks are simpli- in physics and hydraulics in order to make fied, and the allotted time is reduced compared the extra calculations. Participants modify to the international benchmark. Thus the tasks the design of the product, ensuring its perfor- of these two parts of the demonstration exam mance and taking into account the number are to be completed in 1 day. of units produced. Furthermore, participants must provide grounds for the choice of mate- The key features of the assignments and rials and create a presentation demonstrating marking schemes for the Mechanical Engi- the product and its assembly/disassembly neering CAD sections of national and interna- sequence. They are given 2 days to complete tional WorldSkills competitions are presented in the assignment, instead of 4 as in the interna- table 1. tional competitions.

Table 1. WorldSkills assignments and Marking Schemes in Mechanical Engineering CAD competitions

Competition Number of Allotted time (total hours/ Marks (max) modules hours per module)

WorldSkills 2017 4 22 (6+6+6+4) 100 (25 per module)

Netherlands 2 12 (6+6) 100 (50 per module)

Russian Federation 3 12 (6+6) 50 Module А – 25 Module В – 12 Module С – 13

United Kingdom 2 12 (6+6) 100 (50 per module)

41 Appendix

The CAD Marking Scheme used at interna- • adherence to subassembly structures in an tional WorldSkills competitions for assessing assembly; contestants’ performance measures the • presence and accuracy of dimensions in following abilities and skills: technical drawings; • ability to read technical drawings and build • compliance with the animation scenario and a 3D model, transferring dimensions accu- the quality of assembly animation. rately; • ability to produce digital assemblies and The assessment criteria for the demon- providing a well-performing prototype; stration exam in Russia and the national skills • ability to create usable technical drawings of competition in the United Kingdom are similar parts and assemblies; to the CAD Marking Scheme. The differences are • ability to demonstrate effective communi- in assessing the quality of assembly animations cation and interpersonal skills, and ability and the level of detail required in describing the to clearly present designs to potential users attributes of sheet metal components. Perfor- including high-quality visualizations of a mance at national skills competitions in the product. Netherlands is assessed by the criteria from the WorldSkills Marking Scheme. However, there The key criteria for assessing participants’ are some added criteria due to the specific performance include: content of the assignment. For instance, the • correct dimensions in 3D model compo- calculations are checked additionally, and the nents; grounds for choosing materials used in product • accuracy of placing parts in assemblies; components are assessed.

Refrigeration and Air Conditioning

The Refrigeration and Air Conditioning (RAC) The assignment includes three modules: competition has always attracted a lot of atten- • Module A, “Component fabrication,” involves tion at WorldSkills championships, since RAC fabricating a component of a refrigeration engineers are in demand across counties and system according to specifications. Then this industries. They install and maintain all types component is built into the system and used of cooling systems for residential, commercial, in assembling a refrigeration circuit. and public buildings. These include industrial • Module B, “Electrical installation and and cryogenic cooling systems, refrigeration commissioning,” is dedicated to installing systems, air conditioning systems, and heat and commissioning a circuit. By completing pumps. this task participants demonstrate lock- Refrigeration and Air Conditioning skills are smithing, brazing, and electrical installation becoming increasingly digital as RAC systems skills. are getting smarter. More intelligent cooling • Module C, “Fault find and repair,” involves systems enable higher energy efficiency and maintenance and service of cooling systems. are important in the move towards sustainable Contestants are to identify and repair faults, development. thus performing the tasks of an RAC Service The WorldSkills assignment in RAC encom- Engineer. passes a full list of tasks for an RAC Engineer, from design, installation, and commissioning to The RAC assignment in the Russian demon- service and troubleshooting. stration exam involves all three modules, but is more focused on maintenance of air condi- 42 Appendix

tioning systems and chillers. Brazing skills are The Dutch assignment in RAC is focused tested only in Module A when fabricating a on design and calculations. Two out of three component, while the tasks of Module B are modules correspond to Module B and C, but done without brazing, using flare connections. instead of component fabrication participants The skill of testing using the relevant tools is carry out calculations of a heat exchanger assessed in Module C when identifying faults according to specifications. Then participants and re-commissioning the equipment. These create an air treatment curve and produce a adjustments in the assignment make it possible brief cost analysis. All these calculations are to restore the work area to an appropriate checked by performing the tasks in Module B. condition in a shorter time. The Russian assign- Australia offers the most adjusted RAC ment encompasses all sections of WSSS, but assignment among national skills competitions. the time allotted is limited to 2 days (8 hours The three modules are merged in one, but still per day), instead of the conventional 4 days. assess the whole package of skills of an RAC The British RAC assignment is devel- Engineer. This assignment is more centered on oped in a similar manner. The assignment the service and repair of refrigerating systems, includes simplified versions of modules from which is responsive to industry needs. There the WorldSkills International competition. is more time for completing tasks in Module C, Instead of fabricating a component in Module but the amount of calculations and drawings A, participants fabricate a section of a refriger- is increased. Unlike the previous cases, the ation circuit, which requires more operations Australian assignment involves doing more than at the international competition. Also, operations of the same type in pursuit of prac- the fabricated component is not installed. ticing key skills. Modules B and C are very similar to the inter- The key features of the assignments and national benchmark. The tasks are completed marking scheme for the Refrigeration and Air at different training stands, but assess perfor- Conditioning sections at national and interna- mance of the same set of skills and abilities as tional WorldSkills competitions are presented in at the international championship. Module B table 2. features the use of a compressor-condensing unit that allows participants to start installation directly from a liquid line.

Table 2. WorldSkills assignments and Marking Schemes in Refrigeration and Air Conditioning

Competition Number of Allotted time (total hours/ Marks (max) modules hours per module)

WorldSkills Abu Dhabi 2017 3 20 (3+14+3) 100 (13,5+49,6+36,9)

Australia 3 18 (15+2+2) 100 (11,0+49,0+40,0)

Netherlands 12 (2+8+2) 100 (20,0+55,0+25,0) 12 (2+8+2)

Russian Federation 12 (3+8+5) 87,4 (23,0+46,6+17,8) 12 (3+8+5)

United Kingdom 12 (9+2+1) 100 (59,5+29,0+11,5) 12 (9+2+1)

43 Appendix

The RAC Marking Scheme used at interna- sion an RAC system; tional WorldSkills competitions for assessing • ability to diagnose and rectify faults in elec- the performance of the contestants measures trically operated RAC services and compo- the following abilities and skills: nents. • ability to carry out relevant calculations, appraise, and design RAC systems; The assessment is based on the following • pipework & brazing skills; criteria: • ability to install RAC systems and their • use of appropriate tools to test an RAC components and test the performance system; parameters of an RAC system; • use of appropriate hand tools and burner; • ability to inspect and test an electrically • controller programming with computer; operated RAC system; • efficiency of time-management; • ability to program controllers and commis- • workplace safety.

Electrical Installation

The assignment in Electrical Installation tional benchmark, and corresponds to a level establishes skills requirements for occupations 3 electician’s certification. It includes only related to designing and installing electrical four modules (Measuring and marking out, systems and carrying out maintenance and Testing and inspecting, Fault Finding, and repairs. The task includes six modules, each of Effective health and safety). Participants are which requires various skills in installing and not expected to create a circuit diagram, and wiring electrical power and lighting systems. the number of consumer units for installation These modules consist of the following: Circuit is reduced. The use of programmable logic design, Commissioning and function, Installa- relays (KNX/PLR/PLC) is also not included in the tion of equipment, Installation testing and fault assessment. Finally, the UK assignment doesn’t finding, Programming, and Safety (electrical and contain commissioning tasks. personal). National skills competitions incorporate Assignments in Electrical Installation run the specific needs of domestic industries. For in the Netherlands, Russia and the United instance, participants in the Dutch champion- Kingdom were developed in accordance with ship have to program using the DALI protocol the WorldSkills international assignments. (Digital Addressable Lighting Interface). The Skills competitions in the Netherlands and choice of DALI fits well with the energy effi- the United Kingdom are 3 days long, while the ciency policy in the Netherlands. In general, the Russian demonstration exam takes 2 days. The Dutch assignment is more focused on installing, time allowed for each module varies insignifi- maintenance and repair of intelligent energy cantly. As for complexity, the Russian and Dutch systems. Meanwhile, the British assignment is assignments are consistent with the interna- more centered on assessing abilities around tional one and correspond to a level 4 or 5 installation accuracy, and includes the prede- electrician’s certification. termined positioning of the conductors P, N, and The Russian assignment at the demonstra- PE in a single-phase socket. tion exam encompasses all WSSS sections. The key features of the assignments and However, the number of operations to complete marking scheme for the Electrical Installation the tasks, as well as time allotted, are reduced. section at national and international WorldSkills The UK assignment in Electrical Instal- competitions are presented in table 3. lation is a simplified version of the interna- The Marking Scheme in Electrical Installa- 44 Appendix

Table 3. WorldSkills assignments and Marking Schemes in Electrical Installation

Competition Amount of Allotted time (total hours/ Marks (max) modules hours per module)

WorldSkills Abu Dhabi 2017 8 17 100

Netherlands 15* 100

Russian Federation 8 16 62,25

United Kingdom 4 14 100

* time is not fixed per day

tion used at WorldSkills international competi- measuring equipment, and repair faulty tions measures the following abilities and skills: installations. • ability to read, interpret, and revise layouts, circuit drawings and instructions; The assessment procedure used in the • ability to plan installation work using the Russian demonstration exam matches the drawings and documentation provided; WorldSkills Marking Scheme. Table 4 contains • ability to select and install equipment and details on the differences between the sections wireways, cables, electrical switchboards, in WSSS and the Russian assessment proce- and connect equipment according to the dure. instructions provided; The assessment procedure in the Neth- • ability to test installations and set up equip- erlands also corresponds to the international ment; benchmark, while the British competition • ability to diagnose electrical installations uses the international procedure to assess the and identify problems, test and calibrate simplified tasks of their national assignment.

Table 4. Comparing Marking Schemes in Electrical Installation

Section WSSS, % WS Russia, %

Work organization and management 5 5

Communication and interpersonal skills 5 10

Problem solving, innovation, and creativity 5 10

Planning and design 10 5

Installation 35 30

Testing, reporting, and commissioning 25 25

Maintenance, fault finding, and repair 15 15