Guidelines UK Practical information on testing candidates who may require reasonable adjustments

Laura Cross, R&D Consultant Contents

4 About this document

5 Legislation

5 UK Equality Act 2010

5 Defining ‘’

6 Types of disability discrimination

6 Implications for testing

7 Reasonable adjustments

7 What is a reasonable adjustment?

8 Key considerations

9 Test administration for specific

9 Candidates with visual disabilities

10 Candidates with hearing difficulties

12 Candidates with physical disabilities

13 Candidates with dyslexia

14 Candidates with neurodiverse conditions

16 Online testing and

17 Scoring and interpretation

17 Interpreting test scores

18 Increased error

19 Assessment outcomes

20 References

21 Other useful resources Disclaimer

Test administrators must make reasonable adjustments to ensure candidates and employees with disabilities, or physical or mental health conditions, aren’t substantially disadvantaged when completing a test. Therefore, requests for a reasonable adjustment should be taken very seriously.

When supporting an individual with a reasonable adjustment, the most important source of information is the individual requesting it. We recommend that you work with them to identify any issues and implement such adjustments as are reasonable and possible in the context of the assessment. Adjustments should be considered on a case by case basis, and legal advice sought, where necessary. Test administrators should adopt a consultative approach and work with the individual and test provider to determine the best course of action.

This document is designed to offer some initial guidance around the topic and does not provide an exhaustive list of recommendations or constitute any form of advice, nor should it be interpreted as such. It outlines some of the key challenges that candidates with specific disabilities might face. It is recommended that further research is carried out when offering a reasonable adjustment and, if necessary, legal advice is sought.

3 About this document

The following document is designed to provide introductory information and guidance for administrators of psychometric tests and online assessments. The purpose is to highlight the most important considerations to be made when using these types of tests.

In this document, specific reference is made to UK legislation in the context of making reasonable adjustments in psychometric assessment. It will therefore be most relevant to organisations applying these assessments in the UK. Nonetheless, the general principles and guidance for good practice set out in this document may also be useful for people applying psychometric assessments in other countries. Administrators should always ensure they fully understand and act in accordance with any local applicable legislation in their country relating to equality and to discrimination of people with disabilities.

The usage and appropriate deployment of psychometric tests can vary significantly depending on the type of test, the candidate, how the test is being used and the decisions being made by the organisation using the tests. Therefore, it is important to note that, due to that complexity, there is no set approach that test administrators should follow. Nevertheless, it is good to get a baseline understanding of some of the considerations. The aim of this document is to help with that.

This document will cover the following areas: 1 Legislation Relevant legislation relating to disability discrimination in the workplace and the implications for testing 2 Reasonable adjustments Key considerations when making a reasonable adjustment 3 Test administration for specific disabilities Considerations when testing individuals with specific disabilities 4 Online testing and web accessibility Administering tests online and the guidelines for web accessibility 5 Scoring and interpretation of results Scoring and interpreting test results from accommodated test sessions

The following information has been produced using a number of resources as listed in the References section. Note that these references were assembled in October 2020 and we would recommend all clients to supplement this document with their own research. Please refer to these resources for more information or contact PSI directly for support with any of the information provided within this guide.

4 Legislation UK Equality Act 2010

What is the law?

The UK Equality Act 2010 legally protects people from discrimination in the workplace and wider society. It replaces previous anti-discrimination laws (including the Sex Discrimination Act 1975; Race Relations Act 1976; and Disability Discrimination Act 1995) with a single Act. It sets out the different ways in which it is unlawful to treat someone.19

The Act covers nine protected characteristics: age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex, and sexual orientation.

For full details of the legislation, visit: www.legislation.gov.uk/ukpga/2010/15/contents

Defining ‘Disability’

Disability is one of the nine protected characteristics covered by the Equality Act 2010. The Act defines ‘Disability’ as:

A physical or mental impairment which has a substantial and long-term adverse effect on their ability to carry out normal day-to-day activities.19

This definition can be broken down as follows:15

Impairment – can be physical or mental; or may have elements of both. The impairment may not be easily visible (e.g. in the instance of mental health). It is not necessary to establish a cause for the impairment, and it may or may not result from illness.

Substantial adverse effect – more than minor. A limitation that goes beyond normal differences in ability. This may not always be present and may fluctuate or change.

Long-term – lasting at least a year. May be lifelong or recur.

Normal day-to-day activities – activities that people do on a regular basis that may require co-ordination, speech, hearing, eyesight, mobility, memory, ability to concentrate, learn or understand, etc.

5 Types of disability discrimination

Unlawful discrimination may be ‘direct’ or ‘indirect’:

Direct discrimination – when persons with a particular characteristic are treated differently, and less favourably than others, and the treatment is not a proportionate means of achieving a legitimate aim, e.g. a job applicant being treated less favourably because of something arising in consequence of their disability.15

Indirect discrimination – when a provision, criterion or practice is applied to everyone in the same way, but puts persons with a particular characteristic at a disadvantage, and is not a proportionate means of achieving a legitimate aim, e.g. a job applicant being unfairly disadvantaged during a recruitment process for reasons that are not related to their comparative ability to fulfil the requirements of the job.15

The Act also defines two additional types of discrimination which apply solely to the protected characteristic of disability:

Discrimination arising from disability – when persons are treated unfavourably, not because of the disability itself, but because of something linked with their disability. A disabled person claiming this type of discrimination does not have to compare their treatment to how someone else is treated.1

Failure to make ‘reasonable adjustments’ – the Act imposes a duty for employers to make reasonable adjustments so that disabled persons are not placed at a substantial disadvantage compared to others. The Act defines a ‘reasonable adjustment’ as a change or adaptation to a recruitment process or work environment that has the effect of removing or minimising the impact of the individual’s impairment.1

Implications for testing

What are the implications of the Equality Act 2010 for testing candidates with disabilities?

This Act constitutes a legal obligation for hiring professionals in the UK to ensure that their assessment and selection processes comply with the requirements of the Act.19 The primary obligation here is that selection processes should be fair and not discriminate against any of the nine protected characteristics.

The Act requires employers to take steps to ensure that disabled people are not put at a disadvantage because of their disability by avoiding situations where a provision, criterion, practice, or physical features substantially disadvantage a disabled person.15 Accordingly, hiring professionals must be mindful of their legal obligations as potential employers and of the rights of disabled people when designing their selection processes.

In instances where the testing process may disadvantage disabled job applicants, the employer should consider making reasonable adjustments to the testing process to accommodate the needs of disabled candidates.

6 In the event that a testing process is found to disadvantage individuals with a disability, employers must be able to show that this is a proportionate means of achieving a legitimate aim15 – in this instance, that it tests an attribute that is critical for the job and cannot be met by making reasonable adjustments to the jobholder’s work environment or responsibilities.

To this end, the selection process should be designed with the primary aim of selecting the best applicant for the job, irrespective of whether or not they are disabled.

16% of the working age population in the UK have a disability

Reasonable adjustments

Psychometric tests are inherently designed to be a fair and objective method of assessment.

Notably, their standardisation helps to remove subjective bias and aims to ensure that all candidates are treated equally. However, like any method of assessment, psychometric testing can pose a number of potential barriers for candidates with a disability. Namely, the rigidity from this standardisation can present a number of practical challenges with regards to test administration. It is therefore crucial for employers to be mindful of the practical demands of their selection processes and the reasonable adjustments that can be made to support candidates with specific requirements.

What is a reasonable adjustment?

A reasonable adjustment is a change made to the testing environment which helps remove or reduce disadvantages experienced by candidates with a disability.1 Examples of reasonable adjustment include allowing additional time to complete an assessment, or providing an alternative method of responding.9

Specifically, the Equality Act states that employers have a legal duty to offer reasonable adjustments when a disabled job applicant is placed at a substantial disadvantage compared with a person who is not disabled.9,19 This is particularly important when tests are being used in connection with employment, such as for making selection decisions. This means considering reasonable adjustments to the test environment, where appropriate, to ensure all candidates have an equal opportunity to perform.4

7 What is meant by ‘reasonable’? When deciding – Ask job applicants if they require reasonable whether an adjustment is reasonable, an employer adjustments well in advance of a testing should consider how effective the change will be session.23 This gives both parties time to explore in avoiding the disadvantage that would otherwise the options, and enough time to implement any be experienced; maintaining the integrity of changes to the testing environment. Employers assessment criteria; its practicality; its cost; should be ready to discuss with the applicant the organisation’s resource and size, and the what reasonable adjustment(s) can be put in availability of financial support.15 place.10

– Aim to provide a clear picture of what the testing Key considerations process will look like and what it will involve. This allows applicants to raise any concerns The following section details some key that they may have about the testing process considerations for administering tests and their ability to complete the tasks required. to candidates. Some broad guidelines are Where possible, practice materials can be listed below: used to check whether or not the candidate – Foremost, employers should consider experiences any issues. giving candidates the opportunity to ask for – Adjustments can often be simple and reasonable adjustments.23 Employers should inexpensive, and sometimes cost nothing.1 communicate this in a way that is as easy and According to the law, adjustments need not be visible as possible. This might involve including excessive, they only have to be ‘reasonable’. a statement within communications sent out to job applicants during the recruitment process – Adjustments may include modifications to the to encourage them to get in touch if they need test itself (e.g. extended time, breaks during the any reasonable adjustments. This can be as testing session, auxiliary aids, enlarged print on simple as including a sentence in an email the screen),22 or provision of an alternative form invitation or asking applicants verbally. of assessment (e.g. paper-and pencil).13

– Aim to ensure a clear understanding of the skills – Employers may decide that the test is not and attributes that are being assessed and which appropriate in some instances.18 In this case, the test is trying to measure. Knowing this can disabled candidates may be taken through to help to determine what elements of the test and interview without the need for them to complete its format can be adjusted without compromising a pre-interview test. For example, under the the validity of the test measure.4,24,25 Guaranteed Interview Scheme (GIS).

– Disabilities are potentially broad in scope and – Stress can exacerbate the impact that a scale, and not always visible. It is therefore candidate’s disability has on their performance important for employers to think broadly when in the test. Look at ways to put candidates at considering the possible implications of the ease ahead of the testing session. Providing testing environment and making adjustments. clear test instructions and practice questions Employers should treat candidates on a case can help reduce stress. Ensure that any by case basis, with recognition for the variation accommodations or assistance are offered, within specific types of disability. but not forced.

– Ask the candidate how their disability affects them to try and determine what adjustments may be most suitable.18

8 Test administration for specific disabilities

This section will cover some of the main considerations when testing individuals with different types of disability.

For this purpose, it is broken down into broad categories of disability which may raise similar issues from a testing perspective. Note that there will be huge variation within specific types of disability, and so each candidate must be treated on a case by case basis. Candidates with visual disabilities

‘Visual disabilities’ can be defined as an impairment which causes a decreased ability to see to a degree that causes problems not fixable by usual means, such as glasses. Candidates who are registered or certified blind or partially sighted are legally protected from discrimination when seeking employment. It should not be assumed that a candidate with a visual impairment has no sight at all; visual impairments may include partial or complete loss of central or peripheral vision, blurred vision, light sensitivity, or blindness.7

Any task that presents information visually, whether on paper or computer screen, may cause difficulties for a visually impaired person. This includes the vast majority of psychometric tests. When assessing a person with a visual disability, adjustments to standardised test administration procedures may be required, such as use of alternative forms of materials.

Possible adjustments and considerations could include:

– When inviting candidates to a testing session, ask them prior to the session whether they have any or requests.23 This allows both the candidate and administrator to seek advice on and prepare for any special needs.

– Speak to the individual about what works best for them, including how they best access visual or written material and what adjustments they use in everyday situations. Finding out how a disability affects them functionally can provide useful information regarding reasonable adjustments that can be made to the testing environment.13 Does a blind person read Braille or prefer to have texts read out? Does a partially sighted person need large print or special lighting conditions? Does their impairment affect their reading speed? Do they experience difficulties scanning text, or moving focus? Individuals may be able to tell you about aids, equipment, lighting levels, or any information technology adjustments.

9 They may also be able to tell you about any provisions they have previously received, such as the use of a scribe, reader, extra time, or transcription requirements (braille, moon, thermoform, audio-transcription or large print size etc.).13

– Contact the test provider for advice on best practice recommendations. They may be able to advise on alternative test formats and how to use them.

– Consider the relevance of the test being used for the assessment purpose.25 Is the test assessing a skill or attribute that is not relevant to a visually impaired person, i.e. is it something that would not be expected of them in role that can be adjusted or assigned to someone else? Consider instances where it may be best to omit the test from the recruitment process.

– If administering the test remotely online, you may consider the use of screen readers, audio versions of the test, adjustments to screen display settings (font size, contrast settings, typefaces, colour combinations), additional time allowances, etc.7,12,22

– If administering the test in-person, you may consider using assistants to read out text and record answers, braille versions of the tests, large print versions, screen readers, audio versions of the test, adjustments to screen display settings (font size, contrast settings, typefaces, colour combinations), magnification screens and devices, additional time allowances, etc.7,12,22

– Where appropriate, try to provide practice materials in advance of the testing session so that test takers can check if they would have difficulty completing any of the tasks.7 This may involve providing them with practice materials across alternative formats so that they can determine which is the most accessible format for them.

– Also consider adjustments to other aspects of the testing process. Can any communications and instructions prior to the testing session be provided in an accessible format?7 Is the testing venue accessible? Will the test taker be able to use a standard answer sheet, or does it require modification? How does the person usually use a calculator if one is needed for the test? Is there a requirement for a talking calculator?

Candidates with hearing difficulties

‘Hearing difficulties’ can be defined as hearing loss that prevents people from receiving sounds through the ear. This encompasses individuals who are deaf, deafened, or have mild to severe hearing loss.8 Candidates who are registered or certified deaf, or have impaired hearing, are legally protected from discrimination when seeking employment. In some cases, these hearing difficulties are permanent, and other times they may fluctuate, depending on the nature and reason for the deafness.

10 Any task involving auditory stimuli may cause difficulty for a person with hearing difficulties. Whilst psychometric tests are not typically delivered in this way, there are still a number of considerations to bear in mind with regards to the test administration.

Possible adjustments and considerations could include:

– When inviting candidates to a testing session, ask them prior to the session whether they have any special needs or requests.23 This allows both the candidate and administrator to seek advice on and prepare for any special needs.

– Ask the individual about what works best for them and their preferred administration method, including how they best access such materials and what adjustments they use in everyday situations and consider how to replicate these in the test situation. Finding out how their disability affects them functionally can provide useful information regarding reasonable adjustments that can be made to the testing process.13 Are they a lip reader? Do they use sign language/require the services of an interpreter (‘signer’)?22 Can they hear effectively with a hearing aid or an inductive loop? Does their impairment affect their reading speed? Individuals may be able to tell you about aids, equipment, or any information technology adjustments. They may also be able to tell you about any provisions they have previously received, for example, use of a signer, extra time or transcription requirements.

– Determine whether the candidate’s first language is English or British Sign Language (BSL).8 If English is their second language to BSL, they may be disadvantaged on tests requiring reading (e.g. verbal reasoning tests) and writing/typing.22 Where appropriate, consider the use of BSL interpreters who can translate English into BSL and vice versa. In instances where the format of test is not quite appropriate for candidates with BSL as their first language, consider alternative forms of assessment.

– Contact the test provider for advice on best practice recommendations. They may be able to advise of possible test accommodations and alternative formats.

– If administering the test remotely online, determine if there are any auditory elements to the test that may require adjustment. If using multimedia assessments, consider whether any audio content could be represented graphically using either subtitles, or signing on video content.22

– If administering the test in-person, consider using an alternative format for verbal instructions. This may involve providing written instructions if the candidate is able to read, or using a sign language interpreter.22 Check that the testing venue is accessible. Consider one-to-one testing. If the candidate prefers to lipread, support this for example, by looking at the candidate when speaking so that your facial movements are openly visible.8

– Where appropriate, try to provide practice materials in advance of the testing session to help determine any potential difficulties.8 This may involve providing the candidate with practice materials across alternative formats to help them determine which is the most accessible format for them.

11 Candidates with physical disabilities

‘Physical disabilities’ can be defined as a limitation to a person’s physical functioning, mobility, dexterity, or stamina. Examples of physical disability include brain or spinal cord injuries, multiple sclerosis, cerebral palsy, respiratory disorders, epilepsy and chronic pain.

One major issue for testing candidates with physical disabilities is ensuring that they are able to access the testing centre. If the test is being administered remotely, accommodations may be more limited. Moreover, any tasks requiring motor skills, movement, or actions of the muscles, may cause difficulties depending on the type and severity of the disability. For example, the majority of psychometric tests will require the candidate to interact with test materials and mark their answer.

Possible adjustments and considerations could include:

– Ask the candidate prior to the session whether they have any special needs or requests.23 This allows both the candidate and administrator to seek advice on and prepare for any special needs.

– Ask the individual about what works best for them and their preferred administration method, including what adjustments they use in every- day situations. Finding out how their disability affects them functionally can provide useful information regarding reasonable adjustments that can be made to the testing process. Individuals may be able to tell you about aids, equipment, or any information technology adjustments. The candidate may have their own equipment which eases accessibility which you can try to integrate in to the testing process.

– Contact the test provider for advice on best practice recommendations. They may be able to advise of possible test accommodations and alternative formats.

– If administering the test remotely online, consider whether the candidate is able to use a mouse and keyboard. If not, can they navigate the page with ease using alternative means, such as tabbing or auto- completion forms?

– If administering the test in-person, determine whether they will experience any difficulties accessing the testing room.22 Consider whether they may experience any challenges handling the test materials.22 Consider alternative formats of the test materials, and alternative means of providing answers which may be more easily accessible. If administering tests which are not timed, such as personality questionnaires or most situational judgement tests, consider inviting candidates to take breaks during the testing session.

– Where appropriate, try to provide practice materials in advance of the testing session to help determine any potential difficulties. This may involve providing practice materials across alternative formats to help the candidate determine which is the most accessible format for them.

12 Candidates with dyslexia

‘Dyslexia’ is a common learning difficulty that can cause problems with reading, writing and spelling. Dyslexia is a specific learning difficulty, but unlike a , intelligence is not affected. It is estimated that as many as 1 in every 10 people in the UK has some degree of dyslexia. Dyslexia may be characterised by reading and writing slowly; confusing the order of letters in words; poor or inconsistent spelling; difficulties understanding written information; and difficulties with planning and organisation.5 The extent of these difficulties can range from mild to severe and can be a legally recognised disability depending on the severity.5

Candidates with dyslexia may experience difficulties with tasks requiring verbal comprehension and functional literacy.3,6 This will impact the majority of psychometric tests, which often require the candidate to read and interpret written information in the form of instructions and/or test content.

Possible adjustments and considerations could include:

– Inform candidates in advance of what the testing process will involve and invite them to contact you if they have a disability or any condition that might affect their performance on the tests. This allows for both the administrator and test taker to seek advice on and prepare for any special requirements.

– Ask the individual about what works best for them and their preferred administration method, including what adjustments they use in everyday situations. Finding out how their disability affects them can be informative and may provide useful information regarding reasonable adjustments that can be made in a testing situation.6 Individuals may be able to tell you about aids, equipment, or any information technology adjustments. The candidate may have their own equipment which eases accessibility which you can try to integrate in to the testing process.

– Ask if their condition has been diagnosed and documented.6 If so, the results can provide useful information for consideration when determining what adjustments may be most suitable. Ensure that any information shared is kept in confidence and only used to provide adjustments.

– You may contact the test provider for advice on best practice recommendations. They will be able to advise of possible test accommodations and alternative formats.

– Note that stress may exacerbate the impact of dyslexia.3 Therefore, it is advisable to take a calm and understanding approach when delivering the testing session. Consider administering the test one-to-one to allow for a less formal approach where the candidate may feel more comfortable reading test instructions at their own pace and asking any questions.

– Consider what types of verbal skills are being measured and whether these are relevant to the role.6,25 Would the candidate be reasoning from written materials in the job, or would it often be from other sources of information? Are reading and spelling critical to the role? Could the tasks entailing these skills be performed by other members of staff?23

13 – If verbal skill is critical to the role, but the Candidates with neurodiverse format of the test is not quite appropriate for conditions candidates with learning difficulties, consider alternative real work-based exercises that ‘Neurodiverse conditions’ can be defined measure that skill, such as work samples.10 as cognitive functioning which deviates With this, it is important to predetermine from what is seen as ‘normal’.2 This is a broad what an acceptable result would be so that term which encompasses a wide spectrum you can assess to these criteria. of neurodevelopmental and mental health conditions, such as Autism Spectrum Disorders, – If administering the test remotely online, you Attention Deficit Disorders, and Personality may consider using alternative formats, such disorders.11 Consequently, neurodiverse as paper and pencil, or having audio-recorded conditions may be characterised by any number instructions, or synthesised speech.6,12 A of characteristics, including communication common modification for dyslexic candidates difficulties, sensory sensitivity, attention is to allow additional time to complete the test. difficulties, and detrimental beliefs and feelings. The amount of extra time required will depend on the way the dyslexia manifests itself and Given this variation across neurological conditions, its severity, and so supports a case by candidates may face very different difficulties case approach. upon completing psychometric tests.11 Here we will look to cover some of the broad areas for – If administering the test in-person, consider consideration based on research in the area having the instructions read out by an adminis- and reported experiences of test takers.2 trator, or providing a recording of the instructions which can be listened to at the candidate’s own Possible adjustments and considerations pace.6,12,22 This can help support candidates with could include: reading difficulties, or with short term memory – Ask candidates prior to the testing session and/or sequencing. To ensure fair testing, ensure whether they have any special needs or that all candidates understand the task before requests.23 If they disclose a neurodiverse starting the test. You may also decide to have condition, ask how their condition affects them candidates complete some example questions to determine what administration method would before they begin to check that the instructions be preferable for them.11 Ask about adjustments are clear. Consider additional time allowances they use in everyday situations. This can provide where applicable. useful information in respect of reasonable – Where appropriate, consider providing practice adjustments that can be made in a testing materials in advance of the testing session situation.2 Individuals may be able to tell you to help determine any potential difficulties6. about aids, equipment, or any information This may involve providing them with practice technology adjustments. The candidate materials across alternative formats so that may have their own equipment which eases the candidate can determine which is the most accessibility which you can try to integrate accessible format for them. in to the testing process.

– Also consider adjustments to other aspects – You may contact the test provider for advice on of the testing process such as minimising best practice recommendations. They should be the length of sentences and paragraphs in able to advise of possible test accommodations test instructions and content; reviewing the and alternative formats. complexity of vocabulary used and whether – Ensure clarity of communication through the this is appropriate for the job role; making assessment process to help support candidates spelling checkers and word prediction available; with communication difficulties.11 Ensure test and administering paper version tests on instructions and content is easy to understand, cream paper.12

14 and that any tests used have a clearly stated purpose. Be ready to support candidates who require clarification around the testing process and provide contact information when administering tests remotely, so that candidates have somewhere to send any questions.10

– If administering the test remotely online, consider making the test available in alternative formats.11 For example, providing recorded instructions, or the use of synthesised speech. Consider adjustments that can be made for candidates who are hypersensitive to sensory stimuli, such as adjustments to display and contrast settings.13 You may also consider offering additional time allowances.

– If administering the test in-person, you may consider inviting the candidate to bring a supporter along with them to be present through the testing session.11 You may consider using alternative assessment formats, or allowing additional time. You may consider presenting test instructions in a different format, such as having them read aloud by a test administrator. You may also consider using alternative assessments entirely which assess the same skills, such as work sample tests.

– Where appropriate, try to provide practice materials in advance of the testing session to help determine any potential difficulties. This may involve providing practice materials across alternative formats so that the candidate can determine which is the most accessible format for them.

– Consider the relevance of the test being used for the assessment purpose.25 Is the test assessing a skill or attribute that is not relevant, i.e. is it something that would not be expected of them in a role that could be adjusted or assigned to someone else? Consider instances where it may be preferable to omit the test from the recruitment process.

15 Online testing and web accessibility

The use of online testing has increased significantly over the last few years, with it now being the most prevalent mode of administration for psychometric testing.

With the increased growth of remote testing, comes a number of new implications for the delivery of psychometric tests. In addition to the accessibility principles denoted within the previous sections, there are also more general web accessibility guidelines that are also useful to consider.10,29

To understand this, it is important to bear in mind the core accessibility principles which include having a user interface which is perceivable through sight, hearing or touch; operable through navigation controls, such as a keyboard or mouse; easy to understand in its presentation and format; and robust in its functioning across browsers, assistive technologies and mobile devices.

The World Wide Web Consortium (W3C), launched the Web Accessibility Initiative (WAI) in 1997 to help address the need for universal accessibility across online platforms. They have published Web Content Accessibility Guidelines (WCAG), including WCAG 2.0 and WCAG 2.1, which offer recommendations in the form of ‘checkpoints’ for design practices.30 These checkpoints are ranked into three categories, defined by the WAI as Priorities 1, 2 or 3, according to its view of their relative importance.29 Conformance qualifies a site for the designation ‘Conformance Level A, AA, or AAA’ respectively.

Some of the key implications of applying these standards to online testing include: – Ensuring website content uses the clearest and simplest language possible. – Providing alternative formats, e.g. text equivalents for non-text elements. – Including headers for tables. – Having adjustable display settings – ensuring that all information conveyed with colour is available without colour, with foreground and background colour combinations that provide sufficient contrast. – Using style sheets to control layout and presentation. – Using header elements to convey page structure. – Consistent use of navigation mechanisms. – Creating a logical tab order through links, form controls and objects. – Keyboard shortcuts to important links.

Full details of the W3C Web Accessibility Guidelines can be found at: www.w3.org/TR/WCAG21

Note that you can also contact test providers for more information about the accessibility features of their online assessment platforms.

16 Scoring and interpretation

The standardised nature of psychometric tests is one of the main contributors to their effectiveness and objectivity. Thus, modifications to the test or administration procedure could cause challenges for the scoring and interpretation of results.

In this section, we will explore some of the key challenges associated with scoring and interpreting the results of candidates who have completed a modified version of a test.

Interpreting test scores

One important issue is whether test scores can be interpreted in the same way when adjustments have been made to the testing session.

Some key considerations in determining the comparability of results:

– Tests which have been modified will typically be scored in the same way.17 This will either be done automatically if delivered online, or manually if administered in paper version. Note that extra care should be taken for tests scored manually due to the potential for error.28 For this reason, it may be beneficial to have a second assessor corroborate the results.

– The Standards for Educational and Psychological Testing details the challenges associated with comparing test scores for candidates with a disability who have completed a modified version of the test, and candidates who have completed the standard version.4,20 Whilst there are no definitive rules on how to best interpret accommodated test results, test results are typically interpreted in the same way, though with some additional caution.14,16,17 This means that test users should try to consider the candidate’s level of performance compared with the requirements of the job, as opposed to numerically comparing them with other candidates.17

– Where the test time allowance has been adjusted, consider the amount of extra time allowed and the significance of this.21,27 Note that the type of assessment may also affect the level of variation seen in performance across different time limits.26 For example, tests that have been designed to measure speed of processing are likely to show a greater improvement in performance, as opposed to power tests which have more generous time limits. Consider the time demands within the role itself. Is speed of processing a critical factor within the role itself?23

– In instances where the test is administered by personnel who are separate to the assessment process, the administrator must decide whether or not to identify candidates who have received accommodations to the hiring personnel. Test administrators must balance the candidate’s right to confidentiality, with the need to flag test scores which should be interpreted with caution.24,26,28

17 The Standards for Educational and Psychological Testing advise that in rare instances where the modified test has evidence of score comparability, the score should not be flagged, but in all other instances, information about the nature of the modification should be provided to enable test users to properly interpret and act on test scores.4,24,26,28

– Test users may seek advice from the test publisher regarding the potential impact of the type and degree of the modification.24,28 This way they can share any insight they may have regarding the comparative validity of different test modifications.4,14,20,21,24

Increased error

Some of the key considerations concerning the potential for increased error:

– Any score obtained on a test is susceptible to a degree of error of measurement, also known as the ‘Standard Error of Measurement (SEM)’. This means that a candidate’s test score may deviate slightly from their true performance. Sources of error may include fatigue, anxiety, motivations, degree of correct guessing, and attributes of the test, such as the quality of the test items, clarity of directions, misalignment between test difficulty and candidate ability, and scoring errors.

– Research has found that accommodations can contribute to error in the testing process. For example, using Braille versions, and increasing time allowances, may increase fatigue, as marked by a larger SEM.5,17

– Candidates with a disability may also show increased error in their test performance due to the wider effects associated with their condition, beyond those accommodated for.16,17 This may occur as the result of differences in the cognitive process of interpreting the information and responding. For example, reading Braille is slower than reading ordinary text, and candidates who are deaf often experience challenges with tests with lots of verbal information.

– However, the extent to which test accommodations increase the level of error is likely to vary depending on the nature of the test and accommodations made,5,16,26 with some research finding the level of error to be virtually identical between standard and modified versions (e.g. ETS studies).28 Test users may contact the test provider for advice regarding accommodations and scoring.

18 Assessment outcomes

It is crucial to consider these issues of scoring and interpreting modified test results when determining assessment outcomes. Ultimately, employers must ensure that they are complying with the law when reaching recruitment decisions.9

Some of the key considerations when interpreting modified test results to reach assessment outcomes:

– Wherever possible use multiple criteria when using psychometric tests to reach an assessment outcome, such as a hiring decision.17,26 This extends beyond testing disabled candidates to reflect general best practice in testing. Batteries of assessments offer incremental validity above and beyond the use of singular assessments and allow you to make better informed recruitment decisions.4

– With respect to accommodated testing, using multiple criteria can partially account for additional error in test results.17,26 This helps account for instances where particular types of assessment may disadvantage the candidate to a greater or lesser degree.20

– It is also typically advised to avoid using score composites, where possible. These can be problematic because they lack transparency, richness in information, and do not allow for consideration and differential weighting of individual assessment elements. Candidates’ individual test results should instead be considered in relation to the requirements of the role, and relative weighting given to skills and attributes that are more or less critical to performing the job.23

– You may also consider instances when assessing candidates with disabilities where applying ‘cut-off’ scores may not be appropriate in the same way.16 Consider results within the context of the recruitment process and the knowledge, skills and abilities (KSAs) that are essential for successful performance in the role.23

19 15 Equality and Human Rights Commission (2011). Equality References Act 2010 code of practice: Employment statutory code of practice. Retrieved from: www.equalityhumanrights.com/ 1 ACAS (2017). Disability discrimination: Key points for the sites/default/files/employercode.pdf workplace. Retrieved from: www.acas.org.uk/media/4504/ 16 Fuchs, L. S., Fuchs, D., & Capizzi, A. M. (2005). Disability-discrimination-key-points-for-the-workplace/ Identifying appropriate test accommodations for students pdf/Disability_discrim_keypoints_workplace_Nov.pdf with learning disabilities. Focus on Exceptional Children, 2 ACAS (2016). Neurodiversity at work. Retrieved from: 37(6). www.acas.org.uk/media/4655/Neurodiversity-at-work/ 17 Geisinger, K. F. (1994). Psychometric issues in testing pdf/Neurodiversity_at_work_0916(2).pdf students with disabilities. Applied Measurement in 3 Achieveability (2018). Neurodiverse voices: Opening Education, 7(2), 121-140. doors to employment. Retrieved from: www.achieveability. 18 International test commission (2013). ITC guidelines org.uk/files/1516612947/wac-neurodiverse-voices-opening- on test use. Retrieved from: www.intestcom.org/files/ doors-to-employment-report_2018_interactive.pdf guideline_test_use.pdf 4 American Psychological Association (2014). Standards 19 Legislation.GOV.UK (2010). Equality Act 2010, c.15. for educational and psychological testing. Retrieved from: www.legislation.gov.uk/ukpga/2010/15/ 5 Banerjee, M., & Brinckerhoff, L. C. (2002). Assessing contents student performance in distance education courses: 20 Lewandowski, L. J., Lovett, B. J., & Rogers, C. L. (2008). Implications for testing accommodations for students Extended time as a testing accommodation for students with learning disabilities. Assessment for Effective with reading disabilities: Does a rising tide lift all ships? Intervention, 27(3), 25-35. Journal of Psychoeducational Assessment, 26(4), 315-324. 6 The British Psychological Society (2014). Dyslexia 21 Lovett, B. (2010). Extended Time Testing and occupational testing. Retrieved from: https://ptc.bps. Accommodations for Students With Disabilities: Answers org.uk/sites/ptc.bps.org.uk/files/guidance_documents/ to Five Fundamental Questions. Review of Educational dyslexia_and_occupational_testing.pdf Research, 80(4), 611-638. 7 The British Psychological Society (2016). Visual 22 The Open University (n.d.). Psychometric and online impairment and psychometric testing. Retrieved from: tests: Psychometric and online tests if you have a disability. https://ptc.bps.org.uk/sites/ptc.bps.org.uk/files/ Retrieved from: https://help.open.ac.uk/psychometric- guidance_documents/ptc23_visual_impairment_0.pdf and-online-tests-if-you-have-disability 8 The British Psychological Society (2017). Hearing 23 Personnel Today (2014). Disability discrimination: loss, deafness and psychometric testing. Retrieved from: 12 common reasonable adjustments in recruitment. https://ptc.bps.org.uk/sites/ptc.bps.org.uk/files/ Retrieved from: www.personneltoday.com/hr/disability- guidance_documents/ptc25_hearing_loss_and_ discrimination-12-common-reasonable-adjustments- deafness_web_0.pdf in-recruitment 9 Brodies (2017). Recruitment: Psychometric tests and 24 Pitoniak, M. J., & Royer, J. M. (2001). Testing disability discrimination. Retrieved from: https://brodies. accommodations for examinees with disabilities: A review com/blog/employment/recruitment-psychometric-tests- of psychometric, legal, and social policy issues. Review of and-disability-discrimination Educational Research, 71(1), 53-104. 10 Business Disability Forum (2016). A Business 25 Psychometric Tests (n.d.). Discrimination. Retrieved Disability Forum guide to best practice for recruiting from: www.psychometrictest.org.uk/discrimination and retaining disabled people. Retrieved from: https://businessdisabilityforum.org.uk/media_manager/ 26 Sireci, S. G. (2005). Unlabeling the disabled: A public/261/Retention_Toolkit_Teaser_FINAL.pdf perspective on flagging scores from accommodated test administrations. Educational Researcher, 34(1), 3-12. 11 CIPD (2018). Neurodiversity at work. Retrieved from: www.cipd.co.uk/Images/neurodiversity-at-work_2018_ 27 Sireci, S. G., Scarpati, S. E., & Li, S. (2005). Test tcm18-37852.pdf accommodations for students with disabilities: An analysis of the interaction hypothesis. Review of Educational 12 Cook, R. A., & Gladhart, M. A. (2002). A survey of online Research, 75(4), 457-490. instructional issues and strategies for postsecondary students with learning disabilities. Information Technology 28 Thurlow, M. L., Ysseldyke, J. E., & Silverstein, B. (1995). and Disabilities, 8(1). Testing accommodations for students with disabilities. Remedial and , 16(5), 260-270. 13 Disability Rights UK (2019). Adjustments for disabled students: Disability Rights UK factsheet. Retrieved from: 29 World Wide Consortium (W3C) Web Accessibility www.disabilityrightsuk.org/adjustments-disabled- Initiative (1999). Checklist of checkpoints for web content students accessibility guidelines 1.0. Retrieved from: www.w3.org/ TR/WAI-WEBCONTENT/full-checklist.html 14 Elliott, S. N., Kratochwill, T. R., & McKevitt, B. C. (2001). Experimental analysis of the effects of testing 30 World Wide Consortium (W3C) Web Accessibility accommodations on the scores of students with and Initiative (2019). Accessibility Fundamentals. Retrieved without disabilities. Journal of School Psychology, from: www.w3.org/WAI/fundamentals/accessibility-intro 39(1), 3-24.

20 Other useful resources

Useful organisations

Access to Work www.gov.uk/access-to-work Advisory, Conciliation and Arbitration Service (ACAS) www.acas.org.uk The British Association for Supported Employment (BASE) www.base-uk.org The British Psychological Society’s Psychological Testing Centre (PTC) https://ptc.bps.org.uk The Business Disability Forum (BDF) https://businessdisabilityforum.org.uk Chartered Institute of Personnel and Development (CIPD) www.cipd.co.uk Citizens Advice www.citizensadvice.org.uk Disability Rights Commission www.drc.org.uk Equality Advisory and Support Service (EASS) www.equalityadvisoryservice.com Equality and Human Rights Commission www.equalityhumanrights.com/en/about-us GOV.UK www.gov.uk The International Test Commission (ITC) www.intestcom.org National Center for Fair and Open Testing (FairTest) www.fairtest.org/facts/mctfcat.html Remploy www.remploy.co.uk

Disability charities

Action for Hearing Loss www.actiononhearingloss.org.uk British Dyslexia Association www.bdadyslexia.org.uk Disability Rights UK www.disabilityrightsuk.org Mencap www.mencap.org.uk Mind www.mind.org.uk National Autistic Society www.autism.org.uk The Royal National Institute of Blind People (RNIB) www.rnib.org.uk Scope www.scope.org.uk Sense www.sense.org.uk

© 2020 PSI Services LLC. All rights reserved.

Other than for the purposes of using PSI’s electronic assessment service, no portion of this publication may be translated or reproduced in whole or in part, stored in a retrieval system,  or transmitted in any form or by any means (electronic, mechanical, photocopying, recording or otherwise) without the prior written permission of the copyright owner. This publication may not be resold, rented, lent, leased, exchanged, given or otherwise disposed of to third parties. Neither the purchaser nor any individual test user employed by or otherwise contracted to  the purchaser may act as agent, distribution channel or supplier for this publication. PSI and the PSI logo are registered trademarks of PSI Services LLC. 21 PSI Talent Management psionline.com/talent

@PSIServicesLLC

PSI Services LLC