ASPPB 2016 JTA Report

Association of State and Provincial Psychology Boards Examination for Professional Practice in Psychology (EPPP) Job Task Analysis Report May 19, 2016–September 30, 2016

Prepared by: Pearson VUE November 2016

Table of Contents Scope of Work ...... 1 Glossary ...... 1 Executive Summary ...... 2 Job Task Analysis ...... 3 Expert Panel Meeting ...... 4 Developing the Survey ...... 4 Piloting the Survey ...... 6 Administering the Survey ...... 6 Analyzing the Survey ...... 7 Test Specifications Meeting ...... 9 Summary ...... 12 References ...... 13 Appendix A. Meeting Participants ...... 14 Appendix B. Knowledge Statements Surveyed ...... 15 Appendix C. Competencies and Behavioral Exemplars Surveyed ...... 19 Appendix D. Demographic Questions Surveyed ...... 24 Appendix E. Rating Scales Utilized in Survey ...... 29 Appendix F. Emails Distributed to Jurisdictions ...... 30 Appendix G. Email Distributed to Participants ...... 32 Appendix H. Demographic Results: Overall ...... 33 Appendix I. Demographic Results: EPPP Step 1 Respondents ...... 42 Appendix J. Demographic Results: EPPP Step 2 Respondents ...... 51 Appendix K. Responses to the Knowledge Scales ...... 60 Appendix L. Responses to the Competency Scales ...... 84 Appendix M. Responses to the Behavioral Exemplar Scales ...... 96 Appendix N. Post-Survey Revisions to the ASPPB Competency Model ...... 125 Appendix O. Final EPPP Knowledge Scales Statement Weights ...... 135 Appendix P. Final EPPP Competency Scales Statement Weights ...... 141 Appendix Q. Final EPPP Behavioral Exemplar Scales Statement Weights ...... 143 Appendix R. Final EPPP Step 1 Blueprint ...... 148 Appendix S. Final EPPP Step 2 Blueprint ...... 151

Tables and Figures Table A1. JTATF Members ...... 14 Table A2. JTAAC Members ...... 14 Table A3. ASPPB Staff Members ...... 14 Table A4. Pearson VUE Staff Members ...... 14 Table H1. Licensed or Registered for Independent Practice as a Psychologist ..... 33 Table H2. Country Where Licensed/Registered ...... 33 Table H3. Jurisdiction(s) From Which You Received the Link to This Survey ...... 33 Table H4 . Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level ...... 34 Table H5. Highest Level of Education ...... 35 Table H6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated ...... 35 Table H7. Major Area of Training ...... 35 Table H8. Participation in Formal Postdoctoral Training Program ...... 36 Table H9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology ...... 36 Table H10. Current Major Area(s) of Practice ...... 36 Table H11. Certified by ABPP ...... 37 Table H12. Area(s) in Which You are Certified by ABPP ...... 37 Table H13. Employment Setting(s) ...... 37 Table H14. Primary Theoretical Orientation ...... 38 Table H15. Secondary Theoretical Orientation ...... 38 Table H16. Area(s) of Expertise ...... 38 Table H17. Experiences You Have Had as a Psychologist During the Past Three Years ...... 40 Table H18. Gender ...... 40 Table H19. Racial Background ...... 41 Table H20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation ...... 41 Table H21. What Type of Disability ...... 41 Table I1. Responses to the EPPP Step 1 Survey ...... 42 Table I2. Country Where Licensed/Registered ...... 42 Table I3. Jurisdiction(s) From Which You Received the Link to This Survey ...... 42 Table I4. Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level ...... 43 Table I5. Highest Level of Education ...... 44

Table I6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated ...... 44 Table I7. Major Area of Training ...... 44 Table I8. Participation in Formal Postdoctoral Training Program ...... 45 Table I9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology ...... 45 Table I10. Current Major Area(s) of Practice ...... 45 Table I11. Certified by ABPP ...... 46 Table I12. Area(s) in Which You are Certified by ABPP ...... 46 Table I13. Employment Setting(s) ...... 46 Table I14. Primary Theoretical Orientation ...... 47 Table I15. Secondary Theoretical Orientation ...... 47 Table I16. Area(s) of Expertise ...... 47 Table I17. Experiences You Have Had as a Psychologist During the Past Three Years ...... 49 Table I18. Gender ...... 49 Table I19. Racial Background ...... 50 Table I20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation ...... 50 Table I21. What Type of Disability ...... 50 Table J1. Responses to the EPPP Step 2 Survey ...... 51 Table J2. Country Where Licensed/Registered ...... 51 Table J3. Jurisdiction(s) From Which You Received the Link to this Survey ...... 51 Table J4. Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level ...... 52 Table J5. Highest Level of Education ...... 53 Table J6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated ...... 53 Table J7. Major Area of Training ...... 53 Table J8. Participation in Formal Postdoctoral Training Program ...... 54 Table J9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology ...... 54 Table J10. Current Major Area(s) of Practice...... 54 Table J11. Certified by ABPP ...... 55 Table J12. Area(s) in Which You are Certified by ABPP ...... 55 Table J13. Employment Setting(s) ...... 55 Table J14. Primary Theoretical Orientation ...... 56 Table J15. Secondary Theoretical Orientation ...... 56

Table J16. Area(s) of Expertise ...... 56 Table J17. Experiences You Have Had as a Psychologist During the Past Three Years ...... 58 Table J18. Gender ...... 58 Table J19. Racial Background ...... 59 Table J20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation ...... 59 Table J21. What Type of Disability ...... 59 Table K1. EPPP Knowledge Scales Frequency Ratings ...... 60 Table K2. EPPP Knowledge Scales Criticality Ratings ...... 68 Table K3. EPPP Knowledge Scales Importance Ratings ...... 76 Table L1. EPPP Part 2 Competency Scales Frequency Ratings ...... 84 Table L2. EPPP Part 2 Competency Scales Criticality Ratings ...... 87 Table L3. EPPP Part 2 Competency Scales Importance Ratings ...... 90 Table L4. EPPP Part 2 Competency Scales Acquisition Ratings ...... 93 Table M1. EPPP Part 2 Behavioral Exemplar Scales Frequency Ratings ...... 96 Table M2. EPPP Part 2 Behavioral Exemplar Scales Criticality Ratings ...... 103 Table M3. EPPP Part 2 Behavioral Exemplar Scales Importance Ratings ...... 110 Table M4. EPPP Part 2 Behavioral Exemplar Scales Acquisition Ratings ...... 117 Table N1. Post-Survey Revisions to the ASPPB Competency Model ...... 125 Table O1. Final EPPP Knowledge Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method ...... 135 Table P1. Final EPPP Competency Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method ...... 141 Table Q1. Final EPPP Behavioral Exemplar Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method ...... 143

Scope of Work

The Association of State and Provincial Psychology Boards (ASPPB) is the alliance of state, provincial, and territorial agencies responsible for the licensure and certification of psychologists throughout the United States and Canada. ASPPB was formed to serve psychology boards in the two countries. Currently, the psychology boards of all fifty states of the United States, the District of Columbia, the U.S. Virgin Islands, Puerto Rico, Guam and all ten provinces of Canada are members of ASPPB. ASPPB maintains examinations used by psychology licensing boards to establish the readiness of applicants for initial licensure or certification as psychologists in the United States and Canada. This report documents the procedures and results of the job task analysis (JTA) conducted by Pearson VUE on behalf of ASPPB for the Examination for Professional Practice in Psychology (EPPP) Step 1 and Step 2 examinations. Pearson’s involvement in the JTA process was initiated on May 19, 2016 at the expert panel meeting, and completed on September 30, 2016 at the end of the test specifications meeting.

Glossary

Blueprint – Sometimes called the test specifications; provides a listing of the major content areas intended to be included on each test form.

Domain – Psychology subject area assessed by the EPPP.

EPPP – The Examination for Professional Practice in Psychology. The test administered for initial licensure in psychology in the United States and Canada.

EPPP Step 1 – The knowledge-based portion of the EPPP.

EPPP Step 2 – The skills-based portion of the EPPP.

JTA - A job task analysis (JTA) is a systematic, documented process for obtaining information about the professional competency of individuals with a credential or for whom a credential is designed. This includes the actual knowledge, skills, and abilities (KSA) necessary for credentialing.

JTAAC – A committee of psychologists responsible for management/oversight of the JTA process

JTATF – A task force of psychologists (which includes the JTAAC) responsible for developing the survey and deciding on the final blueprint based on results of the survey

JTA Rating Scales - Acquisition – When the skill is acquired, pre- or post-licensure. - Criticality – How critical to protecting the patient/client/public from harm. - Frequency – How frequently used in practice. - Importance – How important to practice.

Jurisdiction – A U.S. or Canadian state, territory or province.

Knowledge Statement – Areas of psychology that are assessed on the EPPP within each domain.

Pearson VUE Confidential Page 1 Licensed – Having an official psychology license to practice independently in a U.S. state or territory, or registration to practice independently in a Canadian province or territory.

Pearson VUE – The test vendor contracted by ASPPB to assist in the development and administration of the EPPP.

SurveyMonkey – An online survey software and questionnaire tool.

Test Specifications – Sometimes called the blueprint; provides a listing of the major content areas intended to be included on each test form.

Executive Summary

A job task analysis (JTA) is a systematic, documented process for obtaining information about the professional competency of individuals with a credential or for whom a credential is designed. This includes the actual knowledge, skills, and abilities (KSA) necessary for initial licensure. The purpose of a JTA is to determine the structure of an exam and ultimately construct (or revise) the exam blueprint. The exam blueprint impacts the types of items that can be written, as well as the number of items in each content area that can be assigned to a test.

The JTA used to update the EPPP Step 1 (knowledge exam) and to develop EPPP Step 2 (skills exam) consisted of three major elements. Initially, two expert panels were recruited by ASPPB, the JTA Advisory Committee (JTAAC) and JTA Task Force (JTATF). Both panels were asked to review and update the major knowledge and skills areas for psychologists registered or licensed for independent practice, as well as review and update the knowledge statements, skills statements, and behavioral exemplars associated with each of those knowledge or competency areas. The expert panels also updated demographic questions and rating scales for inclusion on a survey of independent practitioners. Next, using SurveyMonkey®, Pearson VUE constructed a survey, which was revised and edited by the expert panel. This survey was designed to ask licensed psychologists about the: (a) frequency of performing a behavior or recalling a piece of knowledge, (b) the importance of a behavior or knowledge to independent practice, (c) the criticality of whether not performing the behavior or recalling the knowledge adequately would endanger public safety, and (d) whether the behavior was acquired prior to or after being licensed for independent practice as a psychologist. The survey also included several demographic questions to evaluate whether the survey respondents were comparable to the general population of licensed psychologists. Finally, the survey was analyzed, and the expert panel was reconvened to review the survey findings and to revise and construct the new test specifications.

After reviewing the JTA process, JTAAC and JTATF members approved test specifications for the EPPP Step 1 and Step 2 that reflect current and relevant practice in the field of Psychology. The final test specifications are included at the end of this report in the Appendices.

Pearson VUE Confidential Page 2 Job Task Analysis

Job task analyses (JTA); also known as job analyses, practice analyses, role delineation studies, or task analyses, are used to validate certification examinations and provide a basis for defending the appropriateness of examination content (Kane, 1997). The purpose of a JTA is to determine the structure of an examination and ultimately construct (or revise) the test specifications. By assessing what psychology professionals do in their practice, the JTA ensures that the test specifications for the EPPP Step 1 and Step 2 are reflective of current practice.

Performing a JTA ensures that the examination’s content remains valid with respect to the purpose of the examination. The Standards for Educational and Psychological Testing (2014), prepared jointly by the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME), states that “to identify the knowledge and skills necessary for competent practice, it is important to complete an analysis of the actual work performed and then document the tasks and responsibilities that are essential to the occupation or profession of interest” (p. 175) and that “validation of credentialing tests depend mainly on content- related evidence, often in the form of judgments that the test adequately represents the content domain associated with the occupation or specialty being considered” (p. 175). In other words, JTAs are necessary to validate the assessment of professional knowledge and skills of people for whom a credential is designed. Similarly, the Institute for Credentialing Excellence (ICE) or the American National Standards Institute (ANSI) cite job analyses as a necessary foundation for building a testing program (e.g., NCCA Standards 10 and 11).

The ultimate goal of a JTA is to delineate or update the test specifications, ensuring that the testing program, item content, and examination structure reflect current practice. Conducting a JTA for the EPPP Step 1 and Step 2 ensures that the test specifications are aligned with ICE Standard 1100 Requirement 7.5:

The [certification] provider shall employ a procedure to demonstrate that the assessment is valid for its intended purpose. At a minimum, this procedure shall include documentation of the linkage between the assessment and the intended learning outcomes (e.g., a table listing the knowledge, skills, and/or competencies needed for participants to achieve the intended learning outcomes and identifying how the specified knowledge, skills, and/or competencies are covered by the assessment). A job/practice analysis shall be conducted for high-stakes [certification] programs when their scope is sufficiently broad to support such a study. (ICE, 2010)

The JTA study described in this report was undertaken to ensure the validity of the content and structure of the EPPP Step 1 and Step 2. Test specifications based on a practice analysis encourage the development of test items that reflect up-to-date practice in the field of psychology.

The previous JTA, conducted by ProExam in 2009, updated and refined the knowledge required for the practice of psychology and created a foundation for a competency-based model of practice, separate from the knowledge-based model. The purpose of the current JTA was to update the EPPP Step 1 test specifications (which contains the knowledge statements) due to changes in practice, as well as compile test specifications for the development of the new EPPP Step 2 (based on the competency model).

Pearson VUE Confidential Page 3 The 2016 JTA was conducted by Pearson VUE on behalf of ASPPB for the purpose of refining (for the EPPP Step 1) and constructing (for the EPPP Step 2) test specifications that reflect current standards of practice in the field of psychology. Participants in the JTA meetings included the six-member JTAAC as well as the ten-member JTATF. The Advisory Committee included experts responsible for guiding the development of the ASPPB examinations. The task force included members of the JTAAC, and licensed psychologists who represent different regions and expertise, as well as a variety of other demographic attributes.

The JTAAC and the JTA process were approved by the ASPPB Board of Directors in October 2015. The 2016 JTA consisted of four phases. On February 26-27, 2016, the JTAAC met to select the additional members of the JTATF. On May 19-20, a survey development meeting was held with the JTATF, ASPPB staff, and two psychometricians from Pearson VUE. Based on that meeting, a survey was developed using SurveyMonkey® and sent to the JTATF and JTAAC for testing and refinement. The final survey was live from July 11 to August 31. On September 29-30, a test specification meeting was held with the JTATF, ASPPB staff, and two psychometricians from Pearson VUE. During this meeting, the blueprint for the EPPP Step 1 was revised, and a blueprint for the EPPP Step 2 was developed based on the results from the survey. Members of the JTATF and the JTAAC are shown in Appendix A.

Expert Panel Meeting

Prior to the survey development meeting, JTATF members reviewed relevant and current professional literature regarding competency and competency assessment, and the JTA process. The JTATF also reviewed:

(1) the current test specifications for the EPPP Step 1, including the overall domain descriptions as well as the knowledge statements; (2) the 2014 ASPPB competency model approved by the ASPPB Board of Directors; and (3) the demographic questionnaire administered at the last JTA.

The purpose of the expert panel meeting was to create a survey for distribution to licensed psychologists throughout Canada and the United States. The requirements for taking the EPPP Step 1, as well as the prospective purpose of the EPPP Step 2, were discussed.

The JTATF worked to verify the overall knowledge areas, as well as the particular knowledge statements tied to each of the areas. The Task Force verified that the eight major knowledge domains from the previous blueprint should remain the same for the purposes of the survey. The JTATF edited the individual knowledge statements to prepare them for the survey. The knowledge statements included in the survey are presented in Appendix B.

The focus then shifted to the revision of the ASPPB Competency Model (competencies and behavioral exemplars) that will serve as the foundation for the prospective EPPP Step 2. The competencies and behavioral exemplars included in the survey are presented in Appendix C. The demographic questions and the rating scales that were developed at the meeting are presented in Appendices D and E, respectively.

Developing the Survey

Pearson VUE constructed the survey on the SurveyMonkey® platform. The initial survey was developed between May 20, 2016 and June 15, 2016. The survey link was sent electronically to prospective respondents. A disqualification question was asked to ensure that respondents were licensed or registered for independent practice. Following the disqualification question, respondents were asked two required questions, as follows:

Pearson VUE Confidential Page 4 (1) the country of a respondent’s license; and (2) the jurisdiction(s) in which they are licensed.

These questions were included to evaluate whether the results were representative of the population of licensed psychologists from each of the regulatory jurisdictions.

After the initial set of questions, respondents were randomly assigned to respond to questions from one of two survey blocks. Half of the respondents would be asked to rate knowledge statements from the EPPP Step 1. The other half of the respondents would be asked to rate competency and behavioral exemplar statements from the EPPP Step 2. The knowledge statements contained the following scales:

1. Frequency: How frequently did you call upon this particular knowledge in your practice during the past year? a. Never or very rarely (e.g., Annually) b. Infrequently (e.g., Monthly or Quarterly) c. Frequently (e.g., Weekly) d. Very Frequently (e.g., Daily) 2. Criticality: How critical is possessing this knowledge to protecting the patient/client/public from harm? a. Not critical b. Minimally critical c. Moderately critical d. Highly critical 3. Importance: How important was the knowledge to your practice as a psychologist during the past year? a. Not important b. Minimally important c. Moderately important d. Highly important

The competency and behavioral exemplar statements contained the following scales:

1. Frequency: How frequently did you perform the competency or behavior in your practice during the past year? a. Never or very rarely (e.g., Annually) b. Infrequently (e.g., Monthly or Quarterly) c. Frequently (e.g., Weekly) d. Very Frequently (e.g., Daily) 2. Criticality: How critical is possessing the competency or behavior to protecting the patient/client/public from harm? a. Not critical b. Minimally critical c. Moderately critical d. Highly critical 3. Importance: How important was performance of the competency or behavior to your practice as a psychologist during the past year? a. Not important b. Minimally important c. Moderately important d. Highly important 4. Acquisition: At what point where you able to demonstrate this competency or behavior?

Pearson VUE Confidential Page 5 a. Prior to independent practice as a Psychologist b. After independent practice as a Psychologist

Note that the first survey contained one set of statements (i.e., knowledge) to be rated in three scales (i.e., frequency, criticality, and importance), whereas the second survey contained two sets of clusters (i.e., competency and behavior) to be rated in four scales (i.e., frequency, criticality, importance, and acquisition). Therefore, answering questions to the second survey was a bit more time intensive than answering questions to the first survey. Within each survey, the content domains were randomized to eliminate fatigue or order effects.

After rating individual knowledge, competency, or behavior statements, respondents were asked to rank the eight knowledge or six competency domains based on a global rating of importance. The overall survey concluded with a set of optional demographic questions, a question relating to changes in the field of psychology over the next 5-10 years, and a question soliciting contact information for a drawing for one of twenty $100 gift cards. These final sets of questions were the same regardless of respondent and did not depend on the set of statements to which the respondent was assigned.

Piloting the Survey

Members of the JTATF pilot tested the survey. The purposes of piloting the survey were:

 to ensure that the instructions were clear,  to verify that there were no typographical errors,  to review and clarify the rating scales,  to determine how much time the survey would take to complete,  to assure that both surveys were randomly administered approximately 50% of the time, and  to make any final changes to the survey tasks and demographic questions.

Based on the pilot test, appropriate revisions were made to the survey. A blank copy of the final survey, as presented in SurveyMonkey®, is shown in an attachment to this report. Although all task force members agreed that the survey was time intensive, most believed that the survey could be completed by most respondents in 30 to 45 minutes.

Administering the Survey

The survey remained active between July 11, 2016 and August 31, 2016.

The introduction to the survey provided information about the purpose of the survey as well as the estimated time for completion. The introduction also assured the respondent that the survey was appropriate for all psychologists licensed or registered for independent practice, regardless of years of experience or educational level. If respondents indicated that they were not licensed or registered for independent practice, SurveyMonkey® took respondents to a disqualification page rather than proceeding to the first question.

A few non-licensed respondents answered survey questions. The data from those respondents was not included in the development of the final content weights.

Initially, ASPPB sent the survey to individual licensing boards for distribution to their licensees. ASPPB solicited help from other organizations, including the American Board of Professional Psychology (ABPP), the Association of Psychology Postdoctoral and Internship

Pearson VUE Confidential Page 6 Centers (APPIC), the Canadian Psychological Association (CPA), and the American Psychological Association Practice Organization (APAPO). The emails soliciting assistance with the distribution of the survey are included in Appendix F, and the email that was sent to prospective survey respondents is included in Appendix G.

The final data consisted of 9,599 overall responses to the qualification question, 9,116 (95.0%) of whom indicated that they were licensed or registered for independent practice as a psychologist. Of those, 8,449 (92.7%) respondents indicated the country in which they are licensed or registered and indicated the jurisdiction from which they received the survey. Of the respondents who indicated their jurisdiction, 6,010 (71.1%) respondents provided at least one rating to the JTA rating scales (3,274 corresponding to the knowledge scales, and 2,736 corresponding to the competency and behavioral exemplar scales). Finally, of the respondents who provided at least one response to a JTA rating scale, 2,929 (48.7%) completed ratings to all of the statements (1,899 corresponding to the knowledge scales – 58.0%, and 1,030 corresponding to the competency and behavioral exemplar scales – 37.6%). The difference in response rate is likely due to the difference in time it took to complete each version of the survey. Based on discussions at the expert panel meeting, all responses were included in the final ratings.

Analyzing the Survey

After the survey closed, Pearson VUE extracted the data from SurveyMonkey® and analyzed responses to the survey in accordance with standard practice. Demographic descriptions of respondents to the survey are presented in Appendices H-J (with Appendix H describing survey respondents overall, Appendix I describing survey respondents who provided at least one rating on the EPPP Step 1 questionnaire, and Appendix J describing survey respondents who provided at least one rating on the EPPP Step 2 questionnaire). Moreover, a summary of responses to the knowledge scales are presented in Appendix K, responses to the competency scales are presented in Appendix L, and responses to the behavioral exemplar scales are presented in Appendix M. These tables indicate the percentage of respondents who marked each of the scale levels as well as the number of respondents who responded to that particular statement and a few statistics describing the typical rating.

Determining Respondent Rating Weights

During the expert panel meeting, participants discussed the possibility of respondent ratings being weighted so that the overall weight for a particular jurisdiction is equal to the percentage of licensed or registered practitioners in that jurisdiction. After consideration of the actual data relative to the proportion of licensed psychologists across jurisdictions, the decision was made that the ratings of all respondents would be weighted equally. This resulted in an effective sample size equal to the number of respondents who rated at least one statement for a given survey.

Constructing Overall Ratings

Pearson VUE met with the JTAAC on August 1, 2016 to discuss the relative importance of each of the rating scales. Based on Kane (1982; also see Kane, Kingsbury, Colton, & Estes, 1989), who indicated that, as licensing should protect the public, critical skills are imperative for effective practice, as well as a discussion regarding how behavioral exemplars and competencies should be removed if they are acquired after licensure, it was decided that the following aggregation of ratings would be used to establish the final blueprint weights. This aggregation was based on the hierarchical scale ordering method proposed by Spray and Huang (2000).

Pearson VUE Confidential Page 7 The method of Spray and Huang (2000) for combining scale weights maps a set of ordinal scales into an ordinal overall rating and “requires only a subjective judgment of the relative ordering of the scales” (p. 193). By mapping a set of ordinal ratings into an overall ordinal value, one can avoid treating ordinal ratings as though they were interval scales.

The following scales were ranked with criticality being most essential, then importance, then frequency. As shown in the following table, frequency is nested in importance that is nested in criticality. As decided by the JTATF, this model was used for the analysis of data for the EPPP Step 1.

Table 1. Example Rankings Using Frequency, Importance, and Criticality Criticality Importance Frequency Rank 3. Very frequently 63 2. Frequently 62 3. Highly 1. Infrequently 61 0. Never or very rarely 60 3. Very frequently 59 2. Frequently 58 2. Moderately 1. Infrequently 57 0. Never or very rarely 56 1. Highly 3. Very frequently 55 2. Frequently 54 1. Minimally 1. Infrequently 53 0. Never or very rarely 52 3. Very frequently 51 2. Frequently 50 0. Not important 1. Infrequently 49 0. Never or very rarely 48 3. Very frequently 47 2. Frequently 46 3. Highly 1. Infrequently 45 0. Never or very rarely 44 3. Very frequently 43 2. Moderately 2. Frequently 42 2. Moderately 1. Infrequently 41

0. Never or very rarely 40

3. Very frequently 39

2. Frequently 38 1. Minimally 1. Infrequently 37 0. Never or very rarely 36 3. Very frequently 35 2. Moderately 2. Frequently 34 (cont.) 0. Not important 1. Infrequently 33 0. Never or very rarely 32 3. Very frequently 31 2. Frequently 30 3. Highly 1. Infrequently 29 1. Minimally 0. Never or very rarely 28 3. Very frequently 27 2. Moderately 2. Frequently 26 1. Infrequently 25

Pearson VUE Confidential Page 8 Criticality Importance Frequency Rank 0. Never or very rarely 24 3. Very frequently 23 2. Frequently 22 1. Minimally 1. Infrequently 21 0. Never or very rarely 20 3. Very frequently 19 2. Frequently 18 0. Not important 1. Infrequently 17 0. Never or very rarely 16 3. Very frequently 15 2. Frequently 14 3. Highly 1. Infrequently 13 0. Never or very rarely 12 3. Very frequently 11 2. Frequently 10 2. Moderately 1. Infrequently 9 0. Never or very rarely 8 0. Not critical 3. Very frequently 7 2. Frequently 6 1. Minimally 1. Infrequently 5 0. Never or very rarely 4 3. Very frequently 3 2. Frequently 2 0. Not important 1. Infrequently 1 0. Never or very rarely 0

Frequency: How frequently used in practice Criticality: How critical to protecting the patient/client/public from harm Importance: How important for one’s practice

The competency survey, unlike the knowledge survey, had an additional rating scale, acquisition, assessing whether a behavior was acquired before or after licensure for independent practice. The acquisition scale was discussed with respect to the competency survey but not incorporated into the final weight calculations.

Test Specifications Meeting

EPPP Step 1 (Knowledge) Blueprint

The JTATF first considered individual statements comprising each of the domains to determine whether any statement should be excluded from a domain. The JTATF decided that any statement with a mean criticality rating greater than 1.0 should automatically be included in the final test specifications. This cutoff helped prioritize a discussion of content deemed as not critical to job performance by the survey respondents. The JTATF decided to keep all of the statements with lower criticality in the final blueprint. The rationale for keeping all of the statements in the blueprint was that those statements with low criticality either related to the APA criteria for training, or related to technological innovations that were expected to increase in the coming years. The panelists also decided that the content domain names for the EPPP Step 1 should not change from the previous blueprint, either in terms of classification of knowledge statements or domain titles.

Pearson VUE Confidential Page 9 The JTATF then discussed which survey analysis method the panelists should consider when determining the final domain weights. It was decided that the results should be presented without considering jurisdiction in the final weights.

The JTATF reflected on the methods for combining each of the scales into a final, overall rating. As described above, panelists agreed that criticality outweighs importance, so that the frequency nested within importance nested within criticality aggregation method should be used for determining overall ratings. Finally, panelists debated the benefits and drawbacks of using the Item Response Theory method or classical test theory method of turning overall ratings into final percentages. The panelists unanimously agreed to choose the classical method due, mostly, to the drawbacks of the IRT method that could artificially affect the final percentage weights.

Panelists finally considered how much emphasis, in terms of the percentage of total test items, each of the content domains should receive on the EPPP Step 1. To generate discussion and facilitate consensus, panelists were asked to consider multiple methods for constructing the final set of exam weights.

One method used the sum of the knowledge statement weights in each domain for the blueprint weights. Another method used the average ranking of each domain, as rated by respondents, scaled so that the weights summed to 100. The expert panel decided to take the average of these two methods, which is shown in column 1 of Table 2.

Table 2. EPPP Step 1 Exam Domain Weights Average of knowledge statement Previous Final Domain weights blueprint Weights and weights (rounded) domain rankings Biological Bases of Behavior 8.0 12.0 10 Cognitive-Affective Bases of Behavior 12.8 13.0 13 Social and Cultural Bases of Behavior 10.5 12.0 11 Growth and Lifespan Development 11.7 12.0 12 Assessment and Diagnosis 17.5 14.0 16 Treatment, Intervention, and Prevention and Supervision 16.3 14.0 15 Research Methods and Statistics 6.1 8.0 7 Ethical/Legal/Professional Issues 17.4 15.0 16

The JTATF agreed that neither the domain areas represented by the survey or the requirements of a professional psychologist have changed significantly enough since the last JTA to justify more than a few percentage points change from the previous test specification weights. A final weighting was unanimously agreed upon by panelists to make slight adjustments by taking into account the previous blueprint weights, and the average of Table 2 column 1 results. Subsequent to the meeting, the JTAAC revised the wording of some knowledge statements for grammatical consistency. The final knowledge statement weightings are presented in Appendix O, and the final blueprint for the EPPP Step 1 exam is presented in Appendix R.

Pearson VUE Confidential Page 10 EPPP Step 2 (Competency) Blueprint

The development of the blueprint for the EPPP Step 2 followed a similar process. As when revising the EPPP Step 1 blueprint, task force members first considered individual competency or behavioral exemplar statements for exclusion. Unlike the knowledge survey, the competency survey had an additional rating scale assessing whether a behavior was acquired before or after licensure for independent practice. JTATF members decided at the expert panel meeting that competencies or behaviors typically acquired after licensure could be considered for removal from the final blueprint.

The JTATF decided on the final composition of the domains as they moved several of the competencies in the Scientific Orientation domain to either the Assessment or Intervention sub-competency areas in the Professional Practice domain, due to relevance. In addition, the panelists agreed to move the Consultation and Supervision sub-competency area to the Systems Thinking competency area, and renamed the updated competency area Collaboration, Consultation, and Supervision. Appendix N contains the post-survey revisions to the ASPPB Competency Model, listing the old and new sub-domains competency areas corresponding to each of the competency and behavioral exemplar statements in the survey.

After agreeing on the final construction of the EPPP Step 2 domains, the JTATF unanimously decided to use the same survey analysis method employed for the EPPP Step 1 exam in order to maintain consistency. Thus, the panelists would consider results unweighted by jurisdiction, let criticality be weighted more than importance and importance weighted more than frequency, and aggregate the results using the classical method rather than the IRT method. Moreover, panelists unanimously decided to base domain weights on the sum of results from the behavioral exemplar scale rather than the competency scale. Panelists believed that the behavioral exemplar scale better reflected what respondents actually performed in their job. Because of the major changes to the content domains, the final weights were based on the sum of task weights within each domain with slight adjustment for the Assessment and Intervention domain weight to be one-third of the total exam to take into account the practices in the field. The final domain weights on the EPPP Step 2 are presented in Table 3.

Table 3. EPPP Step 2 Domain Weights Sum of behavioral Rounded Final Domain exemplar weights weights statement weights Scientific Orientation 5.7 6 6 Assessment, and Intervention 31.1 31 33 Relational Competence 16.9 17 16 Professionalism 11.3 11 11 Ethical Practice 17.6 18 17 Collaboration, Consultation, and 17.5 18 17 Supervision

Pearson VUE Confidential Page 11

When considering the EPPP Step 2 blueprint, the JTATF agreed to limit the adjustment for the test domains to two percentage points or less. After necessary adjustments, they unanimously agreed to accept the final weightings. Subsequent to the meeting, the JTAAC revised the wording of some of the competency and behavioral statements for grammatical consistency. The final competency statement weights are presented in Appendix P, the final behavioral exemplar statement weights are presented in Appendix Q, and the final blueprint for the EPPP Step 2 is presented in Appendix S.

Summary

The goal of the current JTA was to update the examination blueprint for the EPPP Step 1, and construct the examination blueprint for the EPPP Step 2. The previous JTA for the EPPP Step 1 had been completed in 2009. The final EPPP Step 1 blueprint appears in Appendix R, and the final EPPP Step 2 blueprint appears in Appendix S.

Pearson VUE Confidential Page 12 References AERA, APA, & NCME (2014). Standards for Educational and Psychological Testing.

Washington, D.C.: American Educational Research Association.

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika,

43, 561-573.

ICE (2010). ICE 1100: 2010(E) - Standard for Assessment-Based Certificate Programs.

Washington, D.C.: Institute for Credentialing Excellence.

Kane, M. T. (1982). The validity of licensure examinations. American Psychologist, 37, 911-

918.

Kane, M. T. (1997). Model-based practice analysis and test specifications. Applied

Measurement in Education, 10(1).

Kane, M. T., Kingsbury, C., Colton, D., & Estes, C. (1989). Combining data on criticality and

frequency in developing plans or licensure and certification examinations. Journal of

Educational Measurement, 26, 17–27.

Linacre, J. M. (2016). Winsteps® (Version 3.92.1) [Computer software]. Beaverton, OR:

Winsteps.com.

Spray, J. A., & Huang, C. (2000). Obtaining test blueprint weights from job analysis

surveys. Journal of Educational Measurement, 37-3, 187–201.

Pearson VUE Confidential Page 13 Appendix A. Meeting Participants

Table A1. JTATF Members First name Last name Jurisdiction Meeting participation Scott Cypers CO Both Kenneth Drude OH Both Georita Frierson NJ Both Sara Hagstrom ON Both Joel Kamper FL Both Gregory Keilin TX Both Rodney Lowman CA Both Kathryn Macapagal IL Both Rosemarie Manfriedi PA Both Melissa Robinson-Brown NY Both Emil Rodolfa CA Both John Hunsley ON Both Barry Edelstein WV Both Donald Meck GA Both Karen Messer-Engel SK Both Amy Hilson GA Both

Table A2. JTAAC Members First name Last name Jurisdiction Meeting Participation Emil Rodolfa CA Both John Hunsley ON Both Barry Edelstein WV Both Donald Meck GA Both Karen Messer-Engel SK Both Amy Hilson GA Both

Table A3. ASPPB Staff Members First name Last name Meeting Participation Jamie Sherrill Both Matthew Turner Both

Table A4. Pearson VUE Staff Members First name Last name Meeting Participation Julie Miles Expert Panel Meeting Steven Nydick Both Xin Li Test Specifications Meeting

Pearson VUE Confidential Page 14 Appendix B. Knowledge Statements Surveyed

Domain 1: Biological Bases of Behavior 1. Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and affect in normal, acute and chronic neurobehavioral disease processes; and/or acute and chronic disease comorbidities 2. Drug classification, mechanisms of action, and desired/adverse effects of therapeutic agents, drugs of abuse, and complementary or alternative remedies 3. Results from major trials and general guidelines for pharmacological, psychotherapeutic, and combined treatment of psychological disorders 4. Behavioral genetics, transmission and expression of genetic information and its modification and the role and limitations of this information in understanding disorders 5. Applications of structural and functional brain imaging methods, electrophysiological methods, therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness

Domain 2: Cognitive-Affective Bases of Behavior 6. Major research-based theories and models of intelligence and their application 7. Major research-based theories, models, and principles of learning and their application 8. Major research-based theories and models of memory and their application 9. Major research-based theories and models of motivation and their application 10. Major research-based theories and models of emotion and their application 11. Additional elements of cognition, including sensation and perception, attention, language, information processing, visual-spatial processing, executive functioning 12. Relations among cognitions/beliefs, behavior, affect, temperament, and mood 13. Influence of psychosocial factors on cognitions/beliefs and behaviors

Domain 3: Social and Cultural Bases of Behavior 14. Social cognition (e.g., theories, person perception, development of stereotypes, prejudice) 15. Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non‐verbal communication, internet communication, mate selection, empathy) 16. Group and systems processes (e.g., school, work, and family systems, job satisfaction, team functioning, conformity, persuasion) and social influences on functioning 17. Personality theories (e.g., psychodynamic, humanistic/existential, cognitive, behavioral, trait theory, interpersonal) 18. Cultural and sociopolitical psychology (e.g., privilege/oppression, cross‐cultural comparisons, political differences, international and global awareness, religion and spirituality, acculturation) 19. Identity diversity and intersectionality (e.g., psychological impact of diversity on individuals, families, and systems; conceptual models; assumptions) 20. Causes, manifestations, and effects of oppression

Pearson VUE Confidential Page 15 Domain 4: Growth and Lifespan Development 21. Normal growth and development across the lifespan (e.g., biological, physical, sexual, cognitive, perceptual, social, personality, moral, emotional, career) 22. Influence of individual‐environment interaction over time on development (e.g., the relationship between the individual and the social, academic, work, community environment) 23. Major research‐based theories of development 24. Influence of diverse identities on development 25. Family development, configuration, and functioning and its impact on the individual across the lifespan 26. Life events that can influence the normal course of development across the lifespan 27. Risk/protective factors that may impact a developmental course (e.g., nutrition, prenatal care, health care, social support, socioeconomic status, abuse/victimization/resiliency) 28. Disorders/diseases that impact the expected course of development over the lifespan

Domain 5: Assessment and Diagnosis 29. Psychometric theory, generalizability theory, item and test characteristics, test construction and standardization procedures, reliability and validity, sensitivity and specificity, and test fairness and bias 30. Assessment theories and models (e.g., developmental, behavioral, ecological, neuropsychological) 31. Assessment methods (e.g., self‐report, multi-informant reports, psychophysiological measures, work samples, assessment centers, direct observation, structured and semi‐structured interviews) and their strengths and limitations 32. Commonly used instruments for the measurement of characteristics and behaviors of individuals (e.g., psychosocial functioning; cognitive and personality; achievement; psychopathology; risk assessment) and the appropriate use of these tests with various populations 33. Issues of differential diagnosis and integration of non‐psychological information into psychological assessment 34. Instruments and methods appropriate for the assessment of groups and organizations (e.g., program evaluation, needs assessment, organizational and personnel assessment) 35. Criteria for selection and adaptation of assessment methods including evidenced- based knowledge of assessment limitations, cultural appropriateness, trans‐cultural adaptation, and language accommodations 36. Classification systems and their underlying rationales and limitations for evaluating client functioning; dimensional vs. categorical approaches to diagnosis 37. Factors influencing interpretation of data and decision‐making (e.g., base rates, group differences, cultural biases and differences, heuristics, evidence base) 38. Constructs of epidemiology and base rates of psychological conditions and behavioral disorders in clinical or demographic populations 39. Major research-based theories and models of psychopathology 40. Measurement of outcomes and changes due to prevention or intervention efforts with individuals, couples, families, groups, and organizations 41. Use of technology in implementing tests, surveys, and other forms of assessment and diagnostic evaluation; validity, cost effectiveness, consumer acceptability

Pearson VUE Confidential Page 16 Domain 6: Treatment, Intervention, and Prevention and Supervision 42. Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client or patient with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change) 43. Contemporary theories/models of treatment/intervention/prevention and their evidence base 44. Treatment techniques/interventions and the evidence for their comparative efficacy and effectiveness 45. Methods and their evidence base for prevention, intervention and/or rehabilitation with diverse or special populations 46. Interventions to enhance growth and performance of individuals, couples, families, groups, systems, and organizations 47. Consultation models and processes 48. Models of vocational and career development in the provision of psychological services 49. Technology‐assisted psychological services, including telepsychology 50. Healthcare systems, structures, and economics, and how these impact intervention choice 51. Approaches to health promotion, risk reduction, resilience, and wellness 52. Contemporary theories/models of supervision and their evidence base

Domain 7: Research Methods and Statistics 53. Sampling and data collection methods and issues 54. Design of case studies, correlational, quasi‐experimental and experimental studies 55. Analytic methods, including qualitative (e.g., thematic, phenomenological) and quantitative (e.g., probability theory; descriptive, inferential, and parametric statistics; meta-analysis, exploratory and confirmatory factor analysis, non‐ parametric statistics, and causal modeling) 56. Statistical interpretation (e.g., power, effect size, causation vs. association, sensitivity and specificity, generalizability, clinical versus statistical significance) 57. Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to generalizations, threats to internal and external validity, design flaws, level of evidence) 58. Evaluation strategies and techniques (e.g., needs assessment, process/implementation evaluation, formative and summative assessment program evaluation, outcome evaluation, cost‐benefit analysis) 59. Considerations regarding community involvement and participation in research, particularly for under-represented populations 60. Dissemination and presentation of research findings

Domain 8 - Ethical/Legal/Professional Issues 61. Ethical principles and codes of conduct for psychologists (APA, CPA) 62. Professional standards and relevant guidelines for the practice of psychology (e.g., practice guidelines for providers of psychological services, standards for educational and psychological testing) 63. Awareness of applicable laws/statutes and/or judicial decisions that affect psychological practice 64. Identifying and managing potential ethical issues 65. Models of ethical decision‐making 66. Approaches for continuing professional development

Pearson VUE Confidential Page 17 67. Consideration of emerging social, legal, ethical, and policy issues and their impact on psychology 68. Client’s/patient’s rights 69. Ethical issues in the conduct of research 70. Ethical issues in supervision 71. Ethical issues in technology assisted psychological services

Pearson VUE Confidential Page 18 Appendix C. Competencies and Behavioral Exemplars Surveyed

The competencies are indicated in the first indent level, and the behavioral exemplars are indicated in the second indent level. Behavioral exemplars tied to a particular competency are listed directly below that competency.

Competency Area 1: Scientific Orientation 1. Select relevant research literature and critically review its assumptions, conceptualization, methodology, interpretation, and generalizability 1. Critically evaluate and apply research findings to practice, with attention to its applicability and generalizability 2. Interpret and communicate empirical research results in a manner that is easily understood by non-scientific audiences 2. Interpret, evaluate, and integrate results of assessment activities within the context of scientific/professional knowledge to formulate and reformulate working hypotheses, conceptualizations, and recommendations 3. Identify and collect applicable assessment data and integrate with theoretical models to develop working hypotheses 4. Synthesize client-specific and scientific data with contextual factors to refine working hypotheses and develop conclusions and recommendations across a range of problems 5. Reformulate working hypotheses and recommendations based on emerging data 3. Articulate a rationale for decisions and psychological services that rely on objective supporting data (e.g., research results, base rates, epidemiological data) 6. Articulate evidence-based rationale for decisions, recommendations, and opinions to clients, professionals, and the public 7. Make and implement decisions and recommendations that integrate client- specific, social/contextual, and scientific data 8. Identify areas where biases and heuristics may interfere with effective services 4. Acquire and disseminate knowledge in accord with scientific and ethical principles 9. Critically evaluate the literature relevant to professional practice 10. Share psychological knowledge with diverse groups (e.g., students, colleagues, clients, other professionals, the public) within professional settings in an unbiased manner 5. Select and use evidence-based assessment methods and instruments 11. Select tests based upon the constructs to be assessed, the psychometric properties of the tests, and the client population 12. Integrate and analyze client needs, practical constraints, and assessment questions into the selection of assessment methods 6. Select and use evidence-based interventions 13. Conceptualize interventions for client presentations guided by research and theory 14. Regularly evaluate effectiveness of interventions

Competency Area 2: Professional Practice 7. Apply knowledge of individual and diversity characteristics in assessment and diagnosis 15. Integrate knowledge of client characteristics in formulating assessment questions and understanding the reason for assessment

Pearson VUE Confidential Page 19 16. Select assessment methods and instruments based on available normed data and/or criterion-reference standards, and address any limitations in that selection 17. Ensure that professional opinions, recommendations, and case formulations adequately reflect consideration of client characteristics 8. Demonstrate effective interviewing skills 18. Adapt interview questions and behaviors in light of the characteristics of the interviewer and interviewee 19. Demonstrate flexible, empathic, and accurate use of a broad range of interview techniques 20. Consider contextual information (e.g., reason for assessment, possible legal or forensic considerations) in conducting an interview 9. Administer and score instruments following current guidelines and research 21. Administer, score, and interpret a range of commonly used standardized assessment methods and instruments 22. Adapt relevant guidelines in situations requiring non-standard administration, scoring, interpretation, or communication of assessment results 10. Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, interviewees, collateral sources of information) following current guidelines and research 23. Interpret and integrate results from standardized tests and interviews following established guidelines and, as appropriate, multiple applicable norm sets 24. Identify the strengths and limitations of various types of assessment data 25. Reconcile or explain discrepancies between various sources of data and suggest alternative interpretations or explanations in light of any limitations of assessment instruments 11. Formulate diagnoses, recommendations, and/or professional opinions using relevant criteria and considering all assessment data 26. Use multi-modal assessment data and theory to formulate an understanding of the client 27. Formulate diagnoses using current taxonomies 28. Provide recommendations that incorporate client and contextual factors 12. Communicate assessment results to clients, referral sources, and other professionals in an integrative manner 29. Write assessment reports in a timely and understandable manner 30. Verbally communicate results from assessments in a timely and understandable manner 13. Evaluate service or program effectiveness across a variety of contexts 31. Develop plans for evaluating service or program effectiveness 32. Assess outcome effectiveness in an ongoing way 14. Select interventions for clients based on ongoing assessment and research evidence as well as contextual and diversity factors 33. Conceptualize intervention or treatment on the basis of evidenced-based literature 34. Integrate client/stakeholder opinions, preferences, readiness for change, and potential for improvement into intervention plan 15. Apply and modify interventions based on ongoing assessment, research, contextual factors, client characteristics, and situational and environmental variables 35. Continually evaluate, modify, and assess the effectiveness of interventions, considering all relevant variables 36. Consult with qualified peers when facing the need to modify interventions in unfamiliar situations

Pearson VUE Confidential Page 20 16. Consult and collaborate within and across professions 37. Tailor consultation requests and provision of information based on knowledge of others’ professional needs and viewpoints 38. Use evidence-based psychological theories, decision-making strategies, and interventions when consulting 39. Continually evaluate, modify, and assess the effectiveness of consultation, considering all relevant variables 17. Ensure compliance with policies and procedures of the practice/organization, the profession, and the jurisdiction 40. Provide a supervision plan that details the supervisory relationship and the policies and procedures of supervision, including procedures to manage high- risk situations 41. Identify responsibilities of supervisees towards clients, including informed consent and supervisory status 18. Monitor, evaluate, and accurately and sensitively communicate supervisee performance to the supervisee, the organization, and the jurisdiction as needed 42. Regularly provide behaviorally anchored feedback about supervisee strengths and areas that need further development 43. Assure that supervisees who are trainees practice within the scope of supervisor’s competence and license and promote professional development 19. Create and maintain a supportive environment in which effective supervision occurs for trainees and other professionals being supervised 44. Attend to the interpersonal process between supervisor and supervisees 45. Monitor possible multiple roles or conflicts of interest and work toward resolution if required

Competency Area 3: Relational Competence 20. Integrate and apply theory, research, professional guidelines, and personal understanding about social contexts to work effectively with diverse clients 46. Recognize, understand, and monitor the impact of one’s own identities in professional situations 47. Engage in respectful interactions with an awareness of individual, community, and organizational differences 48. Modify one’s own behavior based on self-reflection and an understanding of the impact of social, cultural, and organizational contexts 49. Follow professional guidelines and the scientific literature, when available, for providing professional services to diverse populations 50. Apply culturally appropriate skills, techniques, and behaviors with an appreciation of individual differences 21. Work effectively with individuals, families, groups, communities, and/or organizations 51. Use relational skills to engage, establish and maintain working relationships with a range of clients 52. Communicate respectfully, showing empathy for others 53. Collaborate effectively in professional interactions 22. In all areas of professional practice, demonstrate respect for others, including those with differing viewpoints 54. Consider differing viewpoints held by clients and others 55. Respond to differing viewpoints by seeking clarification to increase understanding before taking action 23. Identify and manage interpersonal conflict between self and others

Pearson VUE Confidential Page 21 56. Manage difficult and complex interpersonal relationships between self and others 57. Seek experienced peers’ feedback to examine, and modify if necessary, one’s own reactions and behavior when managing interpersonal conflict

Comptency Area 4: Professionalism 24. Identify and observe boundaries of competence in all areas of professional practice 58. Identify limits of professional competence by recognizing strengths and weaknesses in practice areas 59. Use knowledge of personal strengths and weaknesses to guide scope of practice 60. Seek appropriate consultation when unsure about one’s competence and additional needs for training and development 61. Seek additional knowledge, training, and supervision when expanding scope of practice 62. Update knowledge and skills relevant to psychological practice on an ongoing basis 25. Critically evaluate one’s own professional practice through self-reflection and feedback from others 63. Engage in systematic and ongoing self-assessment and skills development 64. Accept personal responsibility when mistakes or oversights occur and take appropriate corrective action if needed 65. Maintain awareness of personal factors that may impact professional functioning

Competency Area 5: Ethical and Legal Practice 26. Demonstrate and promote values and behaviors commensurate with standards of practice, including ethics codes, laws, and regulations 66. Demonstrate integration and application of ethics codes and laws in all professional interactions 67. Communicate ethical and legal standards in professional interactions 68. Seek professional consultation on ethical or legal issues when needed 69. Discuss with peers or organizations any ethical concerns with their behavior 70. Take appropriate steps to resolve conflicts between laws/rules and codes of ethics in one’s practice 27. Accurately represent and document work performed in professional practice and scholarship 71. Maintain complete and accurate records 72. Report research results accurately, avoiding personal biases 73. Ensure adequate and appropriate credit is given to trainees and collaborators in scholarship 28. Implement ethical practice management 74. Develop a plan commensurate with laws, ethical guidelines, and fiscal constraints to manage professional activities 75. Practice in a fiscally and ethically sound manner 29. Establish and maintain a process that promotes ethical decision-making 76. Systematically identify the ethical and legal issues and conflicts that occur in one’s professional practice 77. Consult with peers to aid in ethical decision-making when appropriate 78. Engage in critical analysis of identified ethical issues and proactively address them 79. Model ethical decision-making to others

Pearson VUE Confidential Page 22 Comptency Area 6: Systems Thinking 30. Work effectively within organizations and systems 80. Recognize the organizational and systemic factors that affect delivery of psychological services 81. Utilize knowledge of organizations and systems to optimize delivery of psychological services 31. Demonstrate interdisciplinary collaborations 82. Collaborate with various healthcare professions to meet common client goals 83. With other healthcare professionals, integrate services to meet common client goals 84. Collaborate with representatives from other professional groups or systems in the provision of psychological services

Pearson VUE Confidential Page 23 Appendix D. Demographic Questions Surveyed

1. Country where licensed/registered as a Psychologist: a. US (States or Territories) b. Canada c. Both 2. Jurisdiction(s) from which you received the link to this survey: (check all that apply) 3. Number of years practicing as a psychologist licensed (US) or registered (Canada) at the independent (autonomous) level: ______4. Highest level of education: a. Master’s degree b. Graduate work beyond Master’s degree (at least 12 semester hours or equivalent) c. Coursework for Doctorate completed d. PsyD e. EdD f. PhD g. Other Degree (please specify): ______5. Was your doctoral program APA or CPA accredited at the time you graduated? a. Yes b. No c. Don’t know 6. Major area of training: a. Child Clinical Psychology b. Clinical Psychology c. Clinical Geropsychology d. Clinical Neuropsychology e. Cognitive Psychology f. Community Psychology g. Comparative Psychology h. Consulting Psychology i. Counseling Psychology j. Developmental Psychology k. Educational Psychology l. Environmental Psychology m. Experimental Psychology n. Forensic Psychology o. General Psychology p. Health Psychology q. Industrial/Organizational Psychology r. Neurosciences s. Perception/Learning t. Personality Psychology u. Physiological Psychology/Psychobiology v. Psychopharmacology w. Quantitative/Mathematical/Psychometrics/Statistics x. Rehabilitation Psychology y. School Psychology z. Social Psychology aa. Sports Psychology bb. Other (please specify): ______

Pearson VUE Confidential Page 24 7. Participation in formal postdoctoral training program: a. Yes b. No 8. Participation in a formal respecialization program in clinical, counseling, or school psychology: a. Yes b. No 9. Current major area(s) of practice: (check all that apply) a. Child Clinical Psychology b. Clinical Psychology c. Clinical Geropsychology d. Clinical Neuropsychology e. Cognitive Psychology f. Community Psychology g. Comparative Psychology h. Consulting Psychology i. Counseling Psychology j. Developmental Psychology k. Educational Psychology l. Environmental Psychology m. Experimental Psychology n. Forensic Psychology o. General Psychology p. Health Psychology q. Industrial/Organizational Psychology r. Medical Psychology s. Neurosciences t. Perception/Learning u. Personality Psychology v. Physiological Psychology/Psychobiology w. Psychopharmacology x. Quantitative/Mathematical/Psychometrics/Statistics y. Rehabilitation Psychology z. School Psychology aa. Social Psychology bb. Sports Psychology cc. Other (please specify): ______10. Certified by ABPP: a. Yes b. No 11. Area(s) in which you are certified by ABPP: (check all that apply) a. Behavioral & Cognitive b. Clinical c. Clinical Child & Adolescent d. Clinical Health e. Clinical Neuropsychology f. Pediatric Clinical Neuropsychology Subspecialty g. Counseling h. Couple & Family i. Forensic j. Geropsychology k. Group l. Organizational & Business Consulting

Pearson VUE Confidential Page 25 m. Police & Public Safety n. Psychoanalysis o. Rehabilitation p. School 12. Employment Setting: (check all that apply) a. University setting (not medical school) b. Four-year college c. Medical school d. Other academic setting e. School/other educational settings f. VA Medical Center g. Military hospital h. Psychiatric hospital i. Other hospital setting j. Independent practice k. Counseling center l. Community clinic m. Other human service setting n. Business/government setting o. Consulting practice p. Not currently employed q. Other setting (please specify): ______13. Primary theoretical orientation: a. Behavioral b. Cognitive/Behavioral c. Dialectical Behavioral d. Existential/Humanistic e. Interpersonal f. Psychodynamic g. Social learning h. Systems i. Integrative j. Person-Centered k. Other orientation l. Not relevant 14. Secondary theoretical orientation: a. Behavioral b. Cognitive/Behavioral c. Dialectical Behavioral d. Existential/Humanistic e. Interpersonal f. Psychodynamic g. Social learning h. Systems i. Integrative j. Person-Centered k. Other orientation l. Not relevant 15. Area(s) of expertise: (check all that apply) a. Academic Practice b. Adult Psychology c. Assessment/Evaluation d. Behavioral Psychology

Pearson VUE Confidential Page 26 e. Career/Vocational Psychology f. Child/Adolescent Psychology g. Clinical Geropsychology/Aging h. Clinical Neuropsychology-Adult i. Clinical Neuropsychology-Child j. Clinical Psychology k. Cognitive Psychology l. Community Psychology m. Comparative Psychology n. Consultation o. Consulting Psychology p. Consumer Psychology q. Counseling Psychology r. Developmental Psychology s. Educational Psychology t. Environmental Psychology u. Ethical/Legal/Professional Issues v. Experimental Psychology w. Family Psychology x. Forensic Psychology y. General Psychology/Methods & Systems z. Growth and Lifespan Development aa. Health Psychology bb. Industrial/Organizational Psychology cc. Interdisciplinary Systems dd. Lesbian/Gay/Bisexual/Transgender Psychology ee. Management ff. Marriage and Family Psychology gg. Intellectual/Developmental Disabilities hh. Multicultural Psychology/Ethnic Minority Psychology ii. Neuroscience jj. Pain Management kk. Pediatric Psychology ll. Personality Psychology mm. Physiological Psychology/Psychobiology nn. Prevention oo. Psychoanalysis pp. Psychology of Women qq. Psychopharmacology rr. Quantitative/Mathematical/Psychometrics/Statistics ss. Rehabilitation Psychology tt. Research and Evaluation uu. Research Methods and Statistics vv. Serious/Chronic Mental Illness ww. School Psychology xx. Social and Multicultural Basis of Behavior yy. Social Psychology zz. Specialized Assessment Techniques aaa. Sports Psychology bbb. Substance Abuse ccc. Supervision ddd. Telepsychology

Pearson VUE Confidential Page 27 16. Experiences you have had as a psychologist during the past three years: (check all that apply) a. Served as the principal investigator or co-investigator on a research grant/contract b. Provided health/mental health services to a client pro bono (free of charge) c. Provided other services (e.g., expert witness, research, teaching, organization consultation) on a pro bono basis d. Served as a paid consultant to another institution, business, government agency, or organization (exclude direct clinical) e. Served as a consultant on a grant/contract f. Served as an expert witness in a court proceeding g. Appeared in media (e.g., television, radio, web-based platform) as a psychologist h. Used social media for practice purposes i. Served on a review group for a government agency or private foundation, to review grant proposals for research, training j. Been involved in legislative/lobbying activities k. Presented at an international psychology convention l. Participated in any professional psychology association activities as committee, board, or other governance group member m. Served on a licensure or regulatory board n. Served as a clinical supervisor to trainee o. Served as a clinical supervisor to other licensed professional p. Served as an administrative supervisor q. Provided telepsychological services 17. Gender a. Male b. Female c. Transgender Male d. Transgender Female e. Not listed (please specify): ______18. Racial Background: a. Asian including South, Southeast and West Asia b. Black or African American c. White d. Hispanic or Latino e. Native American or Indigenous People f. Native Hawaiian or Pacific Islander g. More than one race h. Not listed (please specify): ______19. Do you consider yourself to have a disability as defined by US Americans with Disabilities Act/Canadian Human Rights legislation? a. Yes b. No 20. What type of Disability? (check all that apply) a. Blind or visually impaired b. Deaf or hard of hearing c. Physical or Orthopedic d. Learning e. Cognitive f. Psychological g. Not listed (please specify): ______

Pearson VUE Confidential Page 28 Appendix E. Rating Scales Utilized in Survey

The knowledge statements contained the following scales:

The competency and behavioral exemplar statements contained the following scales:

Pearson VUE Confidential Page 29 Appendix F. Emails Distributed to Jurisdictions

Preliminary Email

Greetings Executive Officers, Registrars, and Board Members:

The Association of State and Provincial Psychology Boards (ASPPB) has undertaken an important initiative, a job task analysis, to validate the knowledge statements used in the Examination for Professional Practice in Psychology (EPPP) and to validate the competency model that will be used in the development of the EPPP Step 2. ASPPB last conducted a practice analysis (now called ‘job task analysis’, or JTA) in 2009 and published the results in 2010. It is time to update that review.

As in 2009, ASPPB needs your help to conduct this survey. On July 11, we will send you a letter containing the link to the Job Task Analysis survey. We ask that you forward this letter to all the licensees in your jurisdiction. It is critical to the future validity of the EPPP and EPPP Step 2 that our results are based on the largest sample of licensed psychologists possible practicing in each jurisdiction in the United States and Canada.

We thank you for emailing the request letter to all your licensees when you receive it from ASPPB.

You may direct any questions or concerns about the job task analysis survey to [email protected].

Thanks very much for your assistance in this critical project.

Sincerely, Emil Rodolfa, PhD Chair, Job Task Analysis Task Force [email protected]

Steve DeMers, EdD ASPPB Chief Executive Officer

Don Crowder, PhD President, ASPPB

Pearson VUE Confidential Page 30 Final Email

Dear Colleagues:

The Association of State and Provincial Psychology Boards (ASPPB) is conducting a job task analysis (JTA) for the Examination for Professional Practice in Psychology (EPPP). The JTA will be used to validate the examination to ensure that it assesses the knowledge (EPPP) and skills (EPPP Step 2) required at the point of licensure.

ASPPB needs your assistance in notifying your licensees/registrants of the JTA survey and encouraging their participation. We want to ensure representation from every jurisdiction, so we ask that you please send out the attached invitation to your licensees/registrants via email. We also ask that you email Jamie Sherrill ([email protected]) when you have sent the invitation to your licensees/registrants.

ASPPB would like to survey as many licensed/registered psychologists in the United States and Canada as possible to insure that the responses to this survey provide an accurate reflection of what psychologists do. When your licensees receive our request, they will be randomly routed to one of two online surveys, developed by licensed/registered subject matter experts, assessing the knowledge base or skills required for the practice of psychology. Thus the questions on the survey will examine the knowledge or the skills needed to practice psychology. Responses will be gathered anonymously.

It is anticipated that the survey will take approximately 30 minutes for respondents to complete. The survey will be open for response on July 11, 2016 and will close on August 22, 2016.

As a regulator for the profession you are essential to the success of the JTA study as you provide a very necessary and important link to members of the profession. On behalf of ASPPB, thank you for your support in this effort and your ongoing work to protect the public by ensuring that those practicing are competent and ethical to do so.

Sincerely, Emil Rodolfa, PhD, Chair Job Task Analysis Advisory Committee

Pearson VUE Confidential Page 31 Appendix G. Email Distributed to Participants

Dear Licensed or Registered Psychologist:

The Association of State and Provincial Psychology Boards (ASPPB) is asking you as a licensed/registered psychologist to participate in an important job task analysis. This study will serve as the foundation to validate the content of the Examination for Professional Practice in Psychology (“EPPP”, the knowledge exam for entry to practice) and the Examination for Professional Practice in Psychology Step 2 (“EPPP Step 2”, the skills exam for entry to practice). Your response is crucial to ensuring that this study accurately represents contemporary psychology practice. Please note that you will receive this letter from every state and province in which you hold a psychology license, but you only need to complete the job task analysis survey only once.

Participants will be randomly routed to one of two surveys assessing the knowledge or skills required for the practice of psychology. The questions on the survey will examine your view of the knowledge base or the skills needed to practice psychology competently. Your responses will be anonymous (i.e. we will have no way to identify any of your responses). The survey will take approximately 30 minutes to complete, and you may access it by clicking on the link below or copying the link into any of the following browsers (the most recent version will be best): Chrome, Firefox, Safari, or Internet Explorer.

http://www.research.net/r/EPPPJobTaskAnalysis

We hope that you will be able to complete the survey in one sitting. When you complete the survey before the deadline of August 31, 2016, you will be included in a random drawing for one of twenty $100 gift cards. Because the job task analysis survey is anonymous, you will be asked to supply your contact information for the $100 gift card drawing via a separate survey link, which will be accessible upon completion of the job task analysis survey.

You may direct any questions or concerns about the job task analysis survey to [email protected].

On behalf of the ASPPB, the profession of psychology, and the future consumers of psychological services, thank you in advance for your assistance with this very important project.

Sincerely,

Emil Rodolfa, Ph.D., Chair ASPPB Job Task Analysis Advisory Committee

Pearson VUE Confidential Page 32 Appendix H. Demographic Results: Overall

Note. The totals for the tables in this appendix vary because not all items were answered by all respondents. The percentage of each Table is based on the total number who responded to that item. Rules of rounding were applied.

Table H1. Licensed or Registered for Independent Practice as a Psychologist Licensed N % Yes 9,116 95.0% No 483 5.0% Total 9,599 100.0%

Table H2. Country Where Licensed/Registered Country N % US 7,182 85.0% Canada 1,239 14.7% Both 28 0.3% Total 8,449 100.0%

Table H3. Jurisdiction(s) From Which You Received the Link to This Survey Jurisdiction N % Alabama 45 0.5% Alaska 4 0.0% Alberta 378 4.5% Arizona 227 2.7% Arkansas 60 0.7% British Columbia 151 1.8% California 1,881 22.3% Colorado 137 1.6% Connecticut 44 0.5% Delaware 18 0.2% District of Columbia 78 0.9% Florida 344 4.1% Georgia 94 1.1% Guam 0 0.0% Hawaii 39 0.5% Idaho 13 0.2% Illinois 175 2.1% Indiana 78 0.9% Iowa 23 0.3% Kansas 48 0.6% Kentucky 139 1.6% Louisiana 39 0.5% Maine 13 0.2% Manitoba 56 0.7% Maryland 380 4.5% Massachusetts 162 1.9% Michigan 490 5.8% Minnesota 377 4.5% Mississippi 24 0.3%

Pearson VUE Confidential Page 33 Jurisdiction N % Missouri 191 2.3% Montana 38 0.4% Nebraska 16 0.2% Nevada 25 0.3% New Brunswick 46 0.5% New Hampshire 12 0.1% New Jersey 70 0.8% New Mexico 21 0.2% New York 305 3.6% Newfoundland 34 0.4% North Carolina 114 1.3% North Dakota 8 0.1% Nova Scotia 26 0.3% Ohio 296 3.5% Oklahoma 72 0.9% Ontario 509 6.0% Oregon 205 2.4% Pennsylvania 169 2.0% Prince Edward Island 7 0.1% Puerto Rico 48 0.6% Quebec 11 0.1% Rhode Island 27 0.3% Saskatchewan 75 0.9% South Carolina 115 1.4% South Dakota 3 0.0% Tennessee 57 0.7% Texas 274 3.2% US Virgin Islands 2 0.0% Utah 42 0.5% Vermont 17 0.2% Virginia 372 4.4% West Virginia 73 0.9% Washington 110 1.3% Wisconsin 197 2.3% Wyoming 26 0.3% Total 8,449 100.0%

Table H4 . Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level Years N % < 1 81 2.1% [1, 4) 910 23.4% [4,10) 994 25.5% [10,20) 747 19.2% ≥ 20 1,165 29.9% Total 3,897 100.0%

Pearson VUE Confidential Page 34 Table H5. Highest Level of Education Education N % Master’s degree 291 7.2% Graduate work beyond Master’s 53 1.3% degree Coursework for Doctorate completed 30 0.7% PsyD 1,074 26.4% EdD 57 1.4% PhD 2,491 61.3% Other Degree 67 1.6% Total 4,063 100.0%

Table H6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated APA or CPA accredited N % Yes 3,248 83.5% No 521 13.4% Don’t know 122 3.1% Total 3,891 100.0%

Table H7. Major Area of Training Major Area of training N % Child Clinical Psychology 326 8.0% Clinical Geropsychology 39 1.0% Clinical Neuropsychology 194 4.8% Clinical Psychology 2,270 55.7% Cognitive Psychology 11 0.3% Community Psychology 8 0.2% Consulting Psychology 3 0.1% Counseling Psychology 587 14.4% Developmental Psychology 39 1.0% Educational Psychology 60 1.5% Experimental Psychology 7 0.2% Forensic Psychology 72 1.8% General Psychology 18 0.4% Health Psychology 67 1.6% Industrial/Organizational Psychology 16 0.4% Neurosciences 6 0.1% Perception/Learning 2 0.0% Personality Psychology 9 0.2% Physiological 3 0.1% Psychology/Psychobiology Psychopharmacology 5 0.1% Quantitative/Mathematical/ 3 0.1% Psychometrics/Statistics Rehabilitation Psychology 25 0.6% School Psychology 176 4.3% Social Psychology 10 0.2% Sports Psychology 3 0.1% Other 115 2.8% Total 4,074 100.0%

Pearson VUE Confidential Page 35 Table H8. Participation in Formal Postdoctoral Training Program Training program N % Yes 2,031 50.1% No 2,026 49.9% Total 4,057 100.0%

Table H9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology Respecialization program N % Yes 431 10.7% No 3,605 89.3% Total 4,036 100.0%

Table H10. Current Major Area(s) of Practice Major area of practice N % Child Clinical Psychology 1,121 27.6% Clinical Geropsychology 279 6.9% Clinical Neuropsychology 463 11.4% Clinical Psychology 2,815 69.2% Cognitive Psychology 370 9.1% Community Psychology 180 4.4% Comparative Psychology 2 0.0% Consulting Psychology 319 7.8% Counseling Psychology 873 21.5% Developmental Psychology 326 8.0% Educational Psychology 246 6.1% Environmental Psychology 12 0.3% Experimental Psychology 30 0.7% Forensic Psychology 601 14.8% General Psychology 427 10.5% Health Psychology 735 18.1% Industrial/Organizational Psychology 121 3.0% Medical Psychology 348 8.6% Neurosciences 132 3.2% Perception/Learning 104 2.6% Personality Psychology 237 5.8% Physiological 78 1.9% Psychology/Psychobiology Psychopharmacology 119 2.9% Quantitative/Mathematical/ 93 2.3% Psychometrics/Statistics Rehabilitation Psychology 216 5.3% School Psychology 354 8.7% Social Psychology 71 1.7% Sports Psychology 51 1.3% Other 346 8.5% Total 4,065 100.0%

Pearson VUE Confidential Page 36 Table H11. Certified by ABPP ABPP certified N % Yes 456 11.4% No 3,529 88.6% Total 3,985 100.0%

Table H12. Area(s) in Which You are Certified by ABPP ABPP certified areas N % Behavioral & Cognitive 29 6.7% Clinical 190 43.6% Clinical Child & Adolescent 58 13.3% Clinical Health 23 5.3% Clinical Neuropsychology 66 15.1% Pediatric Clinical Neuropsychology 3 0.7% Subspecialty Counseling 66 15.1% Couple & Family 19 4.4% Forensic 25 5.7% Geropsychology 10 2.3% Group 11 2.5% Organizational & Business 6 1.4% Consulting Police & Public Safety 8 1.8% Psychoanalysis 13 3.0% Rehabilitation 23 5.3% School 29 6.7% Total 436 100.0%

Table H13. Employment Setting(s) Employment setting N % University setting 662 17.2% Four-year college 78 2.0% Medical school 349 9.1% Other academic setting 69 1.8% School/other educational settings 291 7.6% VA Medical Center 0 0.0% Military hospital 54 1.4% Psychiatric hospital 264 6.9% Other hospital setting 499 13.0% Independent practice 1,686 43.8% Counseling center 258 6.7% Community clinic 444 11.5% Other human service setting 101 2.6% Business/government setting 136 3.5% Consulting practice 244 6.3% Not currently employed 30 0.8% Other setting 564 14.6% Total 3,850 100.0%

Pearson VUE Confidential Page 37 Table H14. Primary Theoretical Orientation Primary orientation N % Behavioral 167 4.1% Cognitive/Behavioral 1,962 48.6% Existential/Humanistic 175 4.3% Interpersonal 190 4.7% Psychodynamic 473 11.7% Systems 129 3.2% Integrative 577 14.3% Person-Centered 115 2.8% Other orientation 110 2.7% Not relevant 140 3.5% Total 4,038 100.0%

Table H15. Secondary Theoretical Orientation Secondary orientation N % Behavioral 444 11.2% Cognitive/Behavioral 809 20.4% Existential/Humanistic 301 7.6% Interpersonal 461 11.6% Psychodynamic 311 7.8% Systems 369 9.3% Integrative 433 10.9% Person-Centered 317 8.0% Other orientation 174 4.4% Not relevant 346 8.7% Total 3,965 100.0%

Table H16. Area(s) of Expertise Area of expertise N % Academic Practice 523 13.0% Adult Psychology 2,049 50.9% Assessment/Evaluation 2,024 50.3% Behavioral Psychology 1,003 24.9% Career/Vocational Psychology 196 4.9% Child/Adolescent Psychology 1,511 37.5% Clinical Geropsychology/Aging 405 10.1% Clinical Neuropsychology-Adult 411 10.2% Clinical Neuropsychology-Child 238 5.9% Clinical Psychology 2,770 68.8% Cognitive Psychology 515 12.8% Community Psychology 185 4.6% Comparative Psychology 3 0.1% Consultation 659 16.4% Consulting Psychology 210 5.2% Consumer Psychology 5 0.1% Counseling Psychology 868 21.6% Developmental Psychology 477 11.8% Educational Psychology 277 6.9% Environmental Psychology 12 0.3%

Pearson VUE Confidential Page 38 Area of expertise N % Ethical/Legal/Professional Issues 501 12.4% Experimental Psychology 56 1.4% Family Psychology 518 12.9% Marriage and Family Psychology 623 15.5% Forensic Psychology 584 14.5% General Psychology/Methods & 162 4.0% Systems Growth and Lifespan Development 348 8.6% Health Psychology 750 18.6% Industrial/Organizational Psychology 115 2.9% Intellectual/Developmental 627 15.6% Disabilities Interdisciplinary Systems 256 6.4% Lesbian/Gay/Bisexual/Transgender 433 10.8% Psychology Management 173 4.3% Pain Management 398 9.9% Multicultural Psychology/Ethnic 359 8.9% Minority Psychology Neuroscience 178 4.4% Pediatric Psychology 307 7.6% Personality Psychology 330 8.2% Physiological 83 2.1% Psychology/Psychobiology Prevention 217 5.4% Psychoanalysis 148 3.7% Psychology of Women 351 8.7% Psychopharmacology 145 3.6% Quantitative/Mathematical/ 127 3.2% Psychometrics/Statistics Rehabilitation Psychology 264 6.6% Research and Evaluation 379 9.4% Research Methods and Statistics 333 8.3% School Psychology 396 9.8% Serious/Chronic Mental Illness 629 15.6% Social and Multicultural Basis of 187 4.6% Behavior Social Psychology 104 2.6% Specialized Assessment Techniques 209 5.2% Sports Psychology 76 1.9% Substance Abuse 570 14.2% Supervision 995 24.7% Telepsychology 175 4.3% Total 4,027 100.0%

Pearson VUE Confidential Page 39 Table H17. Experiences You Have Had as a Psychologist During the Past Three Years Psychological experiences N % Appeared in media as a psychologist 686 18.5% Been involved in legislative/lobbying 397 10.7% activities Presented at an international 622 16.8% psychology convention Provided health/mental health 1,989 53.7% services to a client pro bono Provided telepsychological services 761 20.5% Served as a clinical supervisor to 2,362 63.7% trainee Served as a consultant on a 417 11.2% grant/contract Served as an administrative 1,024 27.6% supervisor Served as an expert witness in a 780 21.0% court proceeding Served as the principal investigator or co-investigator on a research 695 18.7% grant/contract Served on a licensure or regulatory 123 3.3% board Served on a review group for a government agency or private 321 8.7% foundation, to review grant proposals for research, training Used social media for practice 458 12.4% purposes Participated in any professional psychology association activities as 940 25.4% committee, board, or other governance group member Provided other services on a pro 1,286 34.7% bono basis Served as a clinical supervisor to 1,150 31.0% other licensed professional Served as a paid consultant to another institution, business, 915 24.7% government agency, or organization Total 3,707 100.0%

Table H18. Gender Gender N % Male 1,255 31.1% Female 2,770 68.6% Transgender Male 1 0.0% Transgender Female 2 0.0% Not listed 9 0.2% Total 4,037 100.0%

Pearson VUE Confidential Page 40 Table H19. Racial Background Race N % Asian including South, Southeast 151 3.8% and West Asia Black or African American 106 2.6% White 3,433 85.4% Hispanic or Latino 146 3.6% Native American or Indigenous 19 0.5% People Native Hawaiian or Pacific Islander 1 0.0% More than one race 114 2.8% Not listed 49 1.2% Total 4,019 100.0%

Table H20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation Disability N % Yes 199 4.9% No 3,830 95.1% Total 4,029 100.0%

Table H21. What Type of Disability Disability type N % Blind or visually impaired 16 8.1% Deaf or hard of hearing 25 12.6% Physical or Orthopedic 67 33.8% Learning 15 7.6% Cognitive 13 6.6% Psychological 34 17.2% None 5 2.5% Not listed 45 22.7% Total 198 100.0%

Pearson VUE Confidential Page 41 Appendix I. Demographic Results: EPPP Step 1 Respondents

Note. The totals for the tables in this appendix vary because not all items were answered by all respondents. The percentage of each Table is based on the total number who responded to that item.

Table I1. Responses to the EPPP Step 1 Survey Status N % Completed Survey 1,899 58.0% Started Survey 1,375 42.0% Total 3,274 100.0%

Table I2. Country Where Licensed/Registered Country N % US 2,742 83.8% Canada 519 15.9% Both 13 0.4% Total 3,274 100.0%

Table I3. Jurisdiction(s) From Which You Received the Link to This Survey Jurisdiction N % Alabama 18 0.5% Alaska 3 0.1% Alberta 160 4.9% Arizona 72 2.2% Arkansas 20 0.6% British Columbia 61 1.9% California 708 21.6% Colorado 50 1.5% Connecticut 15 0.5% Delaware 9 0.3% District of Columbia 34 1.0% Florida 133 4.1% Georgia 36 1.1% Guam 0 0.0% Hawaii 16 0.5% Idaho 4 0.1% Illinois 63 1.9% Indiana 25 0.8% Iowa 7 0.2% Kansas 18 0.5% Kentucky 65 2.0% Louisiana 18 0.5% Maine 5 0.2% Manitoba 27 0.8% Maryland 145 4.4% Massachusetts 58 1.8% Michigan 190 5.8% Minnesota 150 4.6% Mississippi 10 0.3%

Pearson VUE Confidential Page 42 Jurisdiction N % Missouri 76 2.3% Montana 17 0.5% Nebraska 7 0.2% Nevada 12 0.4% New Brunswick 19 0.6% New Hampshire 6 0.2% New Jersey 25 0.8% New Mexico 10 0.3% New York 115 3.5% Newfoundland 14 0.4% North Carolina 44 1.3% North Dakota 4 0.1% Nova Scotia 8 0.2% Ohio 117 3.6% Oklahoma 32 1.0% Ontario 208 6.4% Oregon 69 2.1% Pennsylvania 63 1.9% Prince Edward Island 4 0.1% Puerto Rico 21 0.6% Quebec 1 0.0% Rhode Island 8 0.2% Saskatchewan 41 1.3% South Carolina 40 1.2% South Dakota 1 0.0% Tennessee 24 0.7% Texas 109 3.3% US Virgin Islands 0 0.0% Utah 19 0.6% Vermont 10 0.3% Virginia 144 4.4% West Virginia 28 0.9% Washington 53 1.6% Wisconsin 71 2.2% Wyoming 5 0.2% Total 3,274 100.0%

Table I4. Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level Years N % < 1 43 1.9% [1, 4) 551 23.8% [4,10) 573 24.7% [10,20) 443 19.1% ≥ 20 707 30.5% Total 2,317 100.0%

Pearson VUE Confidential Page 43 Table I5. Highest Level of Education Education N % Master's degree 179 7.4% Graduate work beyond Master's 30 1.2% degree Coursework for Doctorate completed 16 0.7% PsyD 646 26.7% EdD 33 1.4% PhD 1,469 60.8% Other Degree 43 1.8% Total 2,416 100.0%

Table I6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated APA or CPA accredited N % Yes 1,932 83.9% No 308 13.4% Don’t know 64 2.8% Total 2,304 100.0%

Table I7. Major Area of Training Major area of training N % Child Clinical Psychology 196 8.1% Clinical Geropsychology 20 0.8% Clinical Neuropsychology 110 4.5% Clinical Psychology 1,350 55.8% Cognitive Psychology 8 0.3% Community Psychology 6 0.2% Consulting Psychology 1 0.0% Counseling Psychology 343 14.2% Developmental Psychology 21 0.9% Educational Psychology 40 1.7% Experimental Psychology 3 0.1% Forensic Psychology 35 1.4% General Psychology 8 0.3% Health Psychology 44 1.8% Industrial/Organizational Psychology 13 0.5% Neurosciences 3 0.1% Perception/Learning 2 0.1% Personality Psychology 6 0.2% Physiological 0 0.0% Psychology/Psychobiology Psychopharmacology 5 0.2% Quantitative/Mathematical/ 3 0.1% Psychometrics/Statistics Rehabilitation Psychology 15 0.6% School Psychology 107 4.4% Social Psychology 5 0.2% Sports Psychology 0 0.0% Other 76 3.1% Total 2,420 100.0%

Pearson VUE Confidential Page 44 Table I8. Participation in Formal Postdoctoral Training Program Training program N % Yes 1,204 49.9% No 1,210 50.1% Total 2,414 100.0%

Table I9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology Respecialization program N % Yes 260 10.8% No 2,139 89.2% Total 2,399 100.0%

Table I10. Current Major Area(s) of Practice Major area of practice N % Child Clinical Psychology 666 27.6% Clinical Geropsychology 174 7.2% Clinical Neuropsychology 287 11.9% Clinical Psychology 1,661 68.7% Cognitive Psychology 241 10.0% Community Psychology 114 4.7% Comparative Psychology 2 0.1% Consulting Psychology 210 8.7% Counseling Psychology 520 21.5% Developmental Psychology 205 8.5% Educational Psychology 167 6.9% Environmental Psychology 10 0.4% Experimental Psychology 20 0.8% Forensic Psychology 354 14.6% General Psychology 256 10.6% Health Psychology 450 18.6% Industrial/Organizational Psychology 84 3.5% Medical Psychology 215 8.9% Neurosciences 90 3.7% Perception/Learning 78 3.2% Personality Psychology 149 6.2% Physiological 56 2.3% Psychology/Psychobiology Psychopharmacology 87 3.6% Quantitative/Mathematical/ 68 2.8% Psychometrics/Statistics Rehabilitation Psychology 132 5.5% School Psychology 214 8.9% Social Psychology 56 2.3% Sports Psychology 31 1.3% Other 229 9.5% Total 2,417 100.0%

Pearson VUE Confidential Page 45 Table I11. Certified by ABPP ABPP certified N % Yes 259 10.9% No 2,107 89.1% Total 2,366 100.0%

Table I12. Area(s) in Which You are Certified by ABPP ABPP certified areas N % Behavioral & Cognitive 17 6.9% Clinical 100 40.3% Clinical Child & Adolescent 37 14.9% Clinical Health 11 4.4% Clinical Neuropsychology 37 14.9% Pediatric Clinical Neuropsychology 2 0.8% Subspecialty Counseling 34 13.7% Couple & Family 10 4.0% Forensic 13 5.2% Geropsychology 5 2.0% Group 6 2.4% Organizational & Business 5 2.0% Consulting Police & Public Safety 6 2.4% Psychoanalysis 8 3.2% Rehabilitation 13 5.2% School 19 7.7% Total 248 100.0%

Table I13. Employment Setting(s) Employment setting N % University setting 383 16.7% Four-year college 38 1.7% Medical school 210 9.2% Other academic setting 41 1.8% School/other educational settings 183 8.0% VA Medical Center 0 0.0% Military hospital 35 1.5% Psychiatric hospital 143 6.2% Other hospital setting 287 12.5% Independent practice 1,002 43.8% Counseling center 150 6.6% Community clinic 254 11.1% Other human service setting 55 2.4% Business/government setting 78 3.4% Consulting practice 168 7.3% Not currently employed 17 0.7% Other setting 347 15.2% Total 2,288 100.0%

Pearson VUE Confidential Page 46 Table I14. Primary Theoretical Orientation Primary orientation N % Behavioral 87 3.6% Cognitive/Behavioral 1,209 50.3% Existential/Humanistic 104 4.3% Interpersonal 101 4.2% Psychodynamic 278 11.6% Systems 76 3.2% Integrative 326 13.6% Person-Centered 68 2.8% Other orientation 73 3.0% Not relevant 81 3.4% Total 2,403 100.0%

Table I15. Secondary Theoretical Orientation Secondary orientation N % Behavioral 283 12.0% Cognitive/Behavioral 463 19.6% Existential/Humanistic 170 7.2% Interpersonal 255 10.8% Psychodynamic 179 7.6% Systems 222 9.4% Integrative 272 11.5% Person-Centered 196 8.3% Other orientation 107 4.5% Not relevant 213 9.0% Total 2,360 100.0%

Table I16. Area(s) of Expertise Area of expertise N % Academic Practice 300 12.5% Adult Psychology 1,220 50.8% Assessment/Evaluation 1,246 51.9% Behavioral Psychology 603 25.1% Career/Vocational Psychology 122 5.1% Child/Adolescent Psychology 901 37.5% Clinical Geropsychology/Aging 241 10.0% Clinical Neuropsychology-Adult 247 10.3% Clinical Neuropsychology-Child 139 5.8% Clinical Psychology 1,659 69.1% Cognitive Psychology 320 13.3% Community Psychology 116 4.8% Comparative Psychology 3 0.1% Consultation 398 16.6% Consulting Psychology 119 5.0% Consumer Psychology 5 0.2% Counseling Psychology 513 21.4% Developmental Psychology 280 11.7% Educational Psychology 180 7.5% Environmental Psychology 11 0.5%

Pearson VUE Confidential Page 47 Area of expertise N % Ethical/Legal/Professional Issues 314 13.1% Experimental Psychology 34 1.4% Family Psychology 305 12.7% Marriage and Family Psychology 380 15.8% Forensic Psychology 343 14.3% General Psychology/Methods & 87 3.6% Systems Growth and Lifespan Development 246 10.2% Health Psychology 456 19.0% Industrial/Organizational Psychology 78 3.2% Intellectual/Developmental 381 15.9% Disabilities Interdisciplinary Systems 146 6.1% Lesbian/Gay/Bisexual/Transgender 253 10.5% Psychology Management 106 4.4% Pain Management 257 10.7% Multicultural Psychology/Ethnic 226 9.4% Minority Psychology Neuroscience 112 4.7% Pediatric Psychology 190 7.9% Personality Psychology 207 8.6% Physiological 53 2.2% Psychology/Psychobiology Prevention 137 5.7% Psychoanalysis 84 3.5% Psychology of Women 205 8.5% Psychopharmacology 95 4.0% Quantitative/Mathematical/ 94 3.9% Psychometrics/Statistics Rehabilitation Psychology 157 6.5% Research and Evaluation 235 9.8% Research Methods and Statistics 223 9.3% School Psychology 245 10.2% Serious/Chronic Mental Illness 392 16.3% Social and Multicultural Basis of 124 5.2% Behavior Social Psychology 69 2.9% Specialized Assessment Techniques 130 5.4% Sports Psychology 40 1.7% Substance Abuse 338 14.1% Supervision 602 25.1% Telepsychology 123 5.1% Total 2,400 100.0%

Pearson VUE Confidential Page 48 Table I17. Experiences You Have Had as a Psychologist During the Past Three Years Psychological experiences N % Appeared in media as a psychologist 403 18.3% Been involved in legislative/lobbying 225 10.2% activities Presented at an international 357 16.2% psychology convention Provided health/mental health 1,155 52.3% services to a client pro bono Provided telepsychological services 489 22.1% Served as a clinical supervisor to 1,397 63.3% trainee Served as a consultant on a 256 11.6% grant/contract Served as an administrative 609 27.6% supervisor Served as an expert witness in a 458 20.7% court proceeding Served as the principal investigator or co-investigator on a research 402 18.2% grant/contract Served on a licensure or regulatory 84 3.8% board Served on a review group for a government agency or private 201 9.1% foundation, to review grant proposals for research, training Used social media for practice 276 12.5% purposes Participated in any professional psychology association activities as 549 24.9% committee, board, or other governance group member Provided other services on a pro 736 33.3% bono basis Served as a clinical supervisor to 700 31.7% other licensed professional Served as a paid consultant to another institution, business, 537 24.3% government agency, or organization Total 2,208 100.0%

Table I18. Gender Gender N % Male 732 30.4% Female 1,669 69.3% Transgender Male 0 0.0% Transgender Female 1 0.0% Not listed 5 0.2% Total 2,407 100.0%

Pearson VUE Confidential Page 49 Table I19. Racial Background Race N % Asian including South, Southeast 96 4.0% and West Asia Black or African American 64 2.7% White 2,042 85.2% Hispanic or Latino 87 3.6% Native American or Indigenous 8 0.3% People Native Hawaiian or Pacific Islander 1 0.0% More than one race 69 2.9% Not listed 29 1.2% Total 2,396 100.0%

Table I20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation Disability N % Yes 109 4.5% No 2,289 95.5% Total 2,398 100.0%

Table I21. What Type of Disability Disability type N % Blind or visually impaired 8 7.4% Deaf or hard of hearing 13 12.0% Physical or Orthopedic 41 38.0% Learning 8 7.4% Cognitive 7 6.5% Psychological 20 18.5% None 1 0.9% Not listed 24 22.2% Total 108 100.0%

Pearson VUE Confidential Page 50 Appendix J. Demographic Results: EPPP Step 2 Respondents

Table J1. Responses to the EPPP Step 2 Survey Status N % Completed Survey 1,030 37.6% Started Survey 1,706 62.4% Total 2,736 100.0%

Table J2. Country Where Licensed/Registered Country N % US 2,305 84.2% Canada 423 15.5% Both 8 0.3% Total 2,736 100.0%

Table J3. Jurisdiction(s) From Which You Received the Link to this Survey Jurisdiction N % Alabama 11 0.4% Alaska 0 0.0% Alberta 132 4.8% Arizona 75 2.7% Arkansas 21 0.8% British Columbia 50 1.8% California 580 21.2% Colorado 52 1.9% Connecticut 9 0.3% Delaware 3 0.1% District of Columbia 21 0.8% Florida 108 3.9% Georgia 34 1.2% Guam 0 0.0% Hawaii 11 0.4% Idaho 5 0.2% Illinois 58 2.1% Indiana 29 1.1% Iowa 11 0.4% Kansas 18 0.7% Kentucky 35 1.3% Louisiana 9 0.3% Maine 4 0.1% Manitoba 16 0.6% Maryland 126 4.6% Massachusetts 59 2.2% Michigan 150 5.5% Minnesota 113 4.1% Mississippi 10 0.4% Missouri 73 2.7% Montana 12 0.4% Nebraska 3 0.1% Nevada 9 0.3%

Pearson VUE Confidential Page 51 Jurisdiction N % New Brunswick 19 0.7% New Hampshire 4 0.1% New Jersey 25 0.9% New Mexico 6 0.2% New York 78 2.9% Newfoundland 12 0.4% North Carolina 36 1.3% North Dakota 3 0.1% Nova Scotia 10 0.4% Ohio 92 3.4% Oklahoma 26 1.0% Ontario 182 6.7% Oregon 78 2.9% Pennsylvania 54 2.0% Prince Edward Island 0 0.0% Puerto Rico 11 0.4% Quebec 6 0.2% Rhode Island 12 0.4% Saskatchewan 18 0.7% South Carolina 42 1.5% South Dakota 2 0.1% Tennessee 21 0.8% Texas 90 3.3% US Virgin Islands 0 0.0% Utah 13 0.5% Vermont 3 0.1% Virginia 127 4.6% West Virginia 27 1.0% Washington 27 1.0% Wisconsin 70 2.6% Wyoming 12 0.4% Total 2,736 100.0%

Table J4. Number of Years Practicing as a Psychologist Licensed or Registered at the Independent Level Years N % < 1 38 2.4% [1, 4) 357 22.7% [4,10) 421 26.7% [10,20) 303 19.2% ≥ 20 456 29.0% Total 1,575 100.0%

Pearson VUE Confidential Page 52 Table J5. Highest Level of Education Education N % Master's degree 112 6.8% Graduate work beyond Master's 23 1.4% degree Coursework for Doctorate completed 14 0.9% PsyD 426 26.0% EdD 24 1.5% PhD 1,018 62.1% Other Degree 23 1.4% Total 1,640 100.0%

Table J6. Was Your Doctoral Program APA or CPA Accredited at the Time You Graduated APA or CPA accredited N % Yes 1,311 82.9% No 213 13.5% Don’t know 57 3.6% Total 1,581 100.0%

Table J7. Major Area of Training Major area of training N % Child Clinical Psychology 130 7.9% Clinical Geropsychology 19 1.2% Clinical Neuropsychology 83 5.0% Clinical Psychology 916 55.6% Cognitive Psychology 3 0.2% Community Psychology 2 0.1% Consulting Psychology 2 0.1% Counseling Psychology 244 14.8% Developmental Psychology 18 1.1% Educational Psychology 20 1.2% Experimental Psychology 4 0.2% Forensic Psychology 37 2.2% General Psychology 10 0.6% Health Psychology 23 1.4% Industrial/Organizational Psychology 3 0.2% Neurosciences 3 0.2% Perception/Learning 0 0.0% Personality Psychology 3 0.2% Physiological 3 0.2% Psychology/Psychobiology Psychopharmacology 0 0.0% Quantitative/Mathematical/ 0 0.0% Psychometrics/Statistics Rehabilitation Psychology 10 0.6% School Psychology 68 4.1% Social Psychology 4 0.2% Sports Psychology 3 0.2% Other 39 2.4% Total 1,647 100.0%

Pearson VUE Confidential Page 53 Table J8. Participation in Formal Postdoctoral Training Program Training program N % Yes 821 50.2% No 815 49.8% Total 1,636 100.0%

Table J9. Participation in a Formal Respecialization Program in Clinical, Counseling, or School Psychology Respecialization program N % Yes 170 10.4% No 1,460 89.6% Total 1,630 100.0%

Table J10. Current Major Area(s) of Practice Major area of practice N % Child Clinical Psychology 450 27.4% Clinical Geropsychology 105 6.4% Clinical Neuropsychology 174 10.6% Clinical Psychology 1,150 70.1% Cognitive Psychology 128 7.8% Community Psychology 66 4.0% Comparative Psychology 0 0.0% Consulting Psychology 108 6.6% Counseling Psychology 352 21.5% Developmental Psychology 120 7.3% Educational Psychology 79 4.8% Environmental Psychology 2 0.1% Experimental Psychology 10 0.6% Forensic Psychology 246 15.0% General Psychology 169 10.3% Health Psychology 283 17.2% Industrial/Organizational Psychology 37 2.3% Medical Psychology 131 8.0% Neurosciences 41 2.5% Perception/Learning 25 1.5% Personality Psychology 88 5.4% Physiological 22 1.3% Psychology/Psychobiology Psychopharmacology 31 1.9% Quantitative/Mathematical/ 25 1.5% Psychometrics/Statistics Rehabilitation Psychology 83 5.1% School Psychology 139 8.5% Social Psychology 15 0.9% Sports Psychology 20 1.2% Other 116 7.1% Total 1,641 100.0%

Pearson VUE Confidential Page 54 Table J11. Certified by ABPP ABPP certified N % Yes 196 12.1% No 1,418 87.9% Total 1,614 100.0%

Table J12. Area(s) in Which You are Certified by ABPP ABPP certified areas N % Behavioral & Cognitive 11 5.9% Clinical 89 47.6% Clinical Child & Adolescent 20 10.7% Clinical Health 12 6.4% Clinical Neuropsychology 29 15.5% Pediatric Clinical Neuropsychology 1 0.5% Subspecialty Counseling 31 16.6% Couple & Family 9 4.8% Forensic 12 6.4% Geropsychology 5 2.7% Group 5 2.7% Organizational & Business 1 0.5% Consulting Police & Public Safety 2 1.1% Psychoanalysis 4 2.1% Rehabilitation 10 5.3% School 10 5.3% Total 187 100.0%

Table J13. Employment Setting(s) Employment setting N % University setting 278 17.9% Four-year college 40 2.6% Medical school 137 8.8% Other academic setting 28 1.8% School/other educational settings 108 6.9% VA Medical Center 0 0.0% Military hospital 19 1.2% Psychiatric hospital 121 7.8% Other hospital setting 209 13.4% Independent practice 682 43.9% Counseling center 108 6.9% Community clinic 190 12.2% Other human service setting 46 3.0% Business/government setting 58 3.7% Consulting practice 76 4.9% Not currently employed 12 0.8% Other setting 216 13.9% Total 1,555 100.0%

Pearson VUE Confidential Page 55 Table J14. Primary Theoretical Orientation Primary orientation N % Behavioral 80 4.9% Cognitive/Behavioral 749 46.0% Existential/Humanistic 71 4.4% Interpersonal 89 5.5% Psychodynamic 193 11.8% Systems 53 3.3% Integrative 251 15.4% Person-Centered 47 2.9% Other orientation 37 2.3% Not relevant 59 3.6% Total 1,629 100.0%

Table J15. Secondary Theoretical Orientation Secondary orientation N % Behavioral 159 9.9% Cognitive/Behavioral 345 21.6% Existential/Humanistic 131 8.2% Interpersonal 206 12.9% Psychodynamic 132 8.3% Systems 147 9.2% Integrative 159 9.9% Person-Centered 120 7.5% Other orientation 67 4.2% Not relevant 133 8.3% Total 1,599 100.0%

Table J16. Area(s) of Expertise Area of expertise N % Academic Practice 223 13.7% Adult Psychology 826 50.9% Assessment/Evaluation 775 47.8% Behavioral Psychology 398 24.5% Career/Vocational Psychology 74 4.6% Child/Adolescent Psychology 605 37.3% Clinical Geropsychology/Aging 164 10.1% Clinical Neuropsychology-Adult 164 10.1% Clinical Neuropsychology-Child 98 6.0% Clinical Psychology 1,107 68.2% Cognitive Psychology 193 11.9% Community Psychology 69 4.3% Comparative Psychology 0 0.0% Consultation 260 16.0% Consulting Psychology 91 5.6% Consumer Psychology 0 0.0% Counseling Psychology 354 21.8% Developmental Psychology 196 12.1% Educational Psychology 97 6.0% Environmental Psychology 1 0.1%

Pearson VUE Confidential Page 56 Area of expertise N % Ethical/Legal/Professional Issues 187 11.5% Experimental Psychology 22 1.4% Family Psychology 213 13.1% Marriage and Family Psychology 242 14.9% Forensic Psychology 241 14.9% General Psychology/Methods & 75 4.6% Systems Growth and Lifespan Development 102 6.3% Health Psychology 294 18.1% Industrial/Organizational Psychology 37 2.3% Intellectual/Developmental 246 15.2% Disabilities Interdisciplinary Systems 110 6.8% Lesbian/Gay/Bisexual/Transgender 180 11.1% Psychology Management 66 4.1% Pain Management 141 8.7% Multicultural Psychology/Ethnic 133 8.2% Minority Psychology Neuroscience 66 4.1% Pediatric Psychology 116 7.2% Personality Psychology 123 7.6% Physiological 29 1.8% Psychology/Psychobiology Prevention 77 4.7% Psychoanalysis 63 3.9% Psychology of Women 146 9.0% Psychopharmacology 49 3.0% Quantitative/Mathematical/ 33 2.0% Psychometrics/Statistics Rehabilitation Psychology 107 6.6% Research and Evaluation 144 8.9% Research Methods and Statistics 110 6.8% School Psychology 150 9.2% Serious/Chronic Mental Illness 236 14.5% Social and Multicultural Basis of 63 3.9% Behavior Social Psychology 35 2.2% Specialized Assessment Techniques 79 4.9% Sports Psychology 36 2.2% Substance Abuse 232 14.3% Supervision 393 24.2% Telepsychology 52 3.2% Total 1,622 100.0%

Pearson VUE Confidential Page 57 Table J17. Experiences You Have Had as a Psychologist During the Past Three Years Psychological experiences N % Appeared in media as a psychologist 283 18.9% Been involved in legislative/lobbying 170 11.4% activities Presented at an international 265 17.7% psychology convention Provided health/mental health 832 55.7% services to a client pro bono Provided telepsychological services 272 18.2% Served as a clinical supervisor to 963 64.4% trainee Served as a consultant on a 161 10.8% grant/contract Served as an administrative 413 27.6% supervisor Served as an expert witness in a 321 21.5% court proceeding Served as the principal investigator or co-investigator on a research 293 19.6% grant/contract Served on a licensure or regulatory 39 2.6% board Served on a review group for a government agency or private 119 8.0% foundation, to review grant proposals for research, training Used social media for practice 181 12.1% purposes Participated in any professional psychology association activities as 389 26.0% committee, board, or other governance group member Provided other services on a pro 549 36.7% bono basis Served as a clinical supervisor to 450 30.1% other licensed professional Served as a paid consultant to another institution, business, 378 25.3% government agency, or organization Total 1,495 100.0%

Table J18. Gender Gender N % Male 521 32.1% Female 1,096 67.5% Transgender Male 1 0.1% Transgender Female 1 0.1% Not listed 4 0.2% Total 1,623 100.0%

Pearson VUE Confidential Page 58 Table J19. Racial Background Race N % Asian including South, Southeast 54 3.3% and West Asia Black or African American 41 2.5% White 1,387 85.8% Hispanic or Latino 59 3.7% Native American or Indigenous 10 0.6% People Native Hawaiian or Pacific Islander 0 0.0% More than one race 45 2.8% Not listed 20 1.2% Total 1,616 100.0%

Table J20. Do You Consider Yourself to Have a Disability as Defined by US Americans With Disabilities Act/Canadian Human Rights Legislation Disability N % Yes 89 5.5% No 1,535 94.5% Total 1,624 100.0%

Table J21. What Type of Disability Disability type N % Blind or visually impaired 8 9.0% Deaf or hard of hearing 12 13.5% Physical or Orthopedic 26 29.2% Learning 7 7.9% Cognitive 6 6.7% Psychological 13 14.6% None 4 4.5% Not listed 21 23.6% Total 89 100.0%

Pearson VUE Confidential Page 59 Appendix K. Responses to the Knowledge Scales

Table K1. EPPP Knowledge Scales Frequency Ratings Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Biological Bases of Behavior Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and 1 17.0% 30.6% 31.9% 20.5% 2,702 1.6 1.0 affect in normal, acute and chronic neurobehavioral disease processes; and/or acute and chronic disease comorbidities Drug classification, mechanisms of action, and desired/adverse effects of 2 10.5% 27.3% 42.1% 20.1% 2,704 1.7 0.9 therapeutic agents, drugs of abuse, and complementary or alternative remedies Results from major trials and general guidelines for pharmacological, 3 23.0% 38.7% 27.6% 10.7% 2,699 1.3 0.9 psychotherapeutic, and combined treatment of psychological disorders Behavioral genetics, transmission and expression of genetic information and its 4 modification and the role and limitations 34.3% 40.8% 19.6% 5.35 2,701 1.0 0.9 of this information in understanding disorders Applications of structural and functional brain imaging methods, electrophysiological methods, 5 50.5% 31.5% 11.4% 6.6% 2,699 0.7 0.9 therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness Cognitive-Affective Bases of Behavior

Pearson VUE Confidential Page 60 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Major research‐based theories and 6 models of intelligence and their 22.3% 36.7% 26.8% 14.3% 2,699 1.3 1.0 application Major research-based theories, models, 7 and principles of learning and their 14.5% 33.5% 33.2% 18.8% 2,692 1.6 1.0 application Major research‐based theories and 8 20.0% 40.4% 27.0% 12.5% 2,678 1.3 0.9 models of memory and their application Major research‐based theories and 9 models of motivation and their 13.6% 33.7% 36.8% 16.0% 2,684 1.6 0.9 application Major research-based theories and 10 8.9% 25.4% 39.8% 25.9% 2,674 1.8 0.9 models of emotion and their application Additional elements of cognition, including sensation and perception, 11 attention, language, information 13.4% 29.3% 34.5% 22.8% 2,691 1.7 1.0 processing, visual-spatial processing, executive functioning Relations among cognitions/beliefs, 12 behavior, affect, temperament, and 3.3% 11.2% 35.2% 50.4% 2,691 2.3 0.8 mood Influence of psychosocial factors on 13 2.6% 10.5% 41.2% 45.6% 2,685 2.3 0.8 cognitions/beliefs and behaviors Social and Cultural Bases of Behavior Social cognition (e.g., theories, person 14 perception, development of stereotypes, 14.5% 38.5% 33.9% 13.1% 2,659 1.5 0.9 prejudice) Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non‐ 15 11.4% 25.6% 36.2% 26.8% 2,650 1.8 1.0 verbal communication, internet communication, mate selection, empathy) Group and systems processes (e.g., school, work, and family systems, job 16 satisfaction, team functioning, 14.9% 28.1% 37.4% 19.6% 2,649 1.6 1.0 conformity, persuasion) and social influences on functioning

Pearson VUE Confidential Page 61 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Personality theories (e.g., psychodynamic, humanistic/existential, 17 8.0% 23.4% 35.6% 33.0% 2,645 1.9 0.9 cognitive, behavioral, trait theory, interpersonal) Cultural and sociopolitical psychology (e.g., privilege/oppression, cross‐cultural 18 comparisons, political differences, 13.7% 30.0% 35.5% 20.8% 2,648 1.6 1.0 international and global awareness, religion and spirituality, acculturation) Identity diversity and intersectionality (e.g., psychological impact of diversity 19 14.5% 31.0% 35.0% 19.4% 2,648 1.6 1.0 on individuals, families, and systems; conceptual models; assumptions) Causes, manifestations, and effects of 20 27.1% 36.9% 23.8% 12.2% 2,649 1.2 1.0 oppression Growth and Lifespan Development Normal growth and development across the lifespan (e.g., biological, physical, 21 4.9% 17.0% 41.6% 36.4% 2,694 2.1 0.9 sexual, cognitive, perceptual, social, personality, moral, emotional, career) Influence of individual‐environment interaction over time on development 22 (e.g., the relationship between the 5.5% 17.1% 40.5% 36.8% 2,694 2.1 0.9 individual and the social, academic, work, community environment) Major research‐based theories of 23 17.9% 41.6% 30.6% 9.9% 2,693 1.3 0.9 development Influence of diverse identities on 24 15.4% 34.4% 34.3% 15.9% 2,658 1.5 0.9 development Family development, configuration, and 25 functioning and its impact on the 8.3% 22.7% 39.5% 29.4% 2,688 1.9 0.9 individual across the lifespan Life events that can influence the normal 26 course of development across the 2.7% 14.2% 44.2% 38.9% 2,690 2.2 0.8 lifespan

Pearson VUE Confidential Page 62 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Risk/protective factors that may impact a developmental course (e.g., nutrition, 27 prenatal care, health care, social 5.1% 17.95 41.5% 35.5% 2,693 2.1 0.9 support, socioeconomic status, abuse/victimization/resiliency) Disorders/diseases that impact the 28 expected course of development over 7.5% 28.3% 36.6% 27.6% 2,691 1.8 0.9 the lifespan Assessment and Diagnosis Psychometric theory, generalizability theory, item and test characteristics, test construction and standardization 29 26.7% 30.5% 21.7% 21.2% 2,701 1.4 1.1 procedures, reliability and validity, sensitivity and specificity, and test fairness and bias Assessment theories and models (e.g., 30 developmental, behavioral, ecological, 18.5% 30.8% 31.1% 19.7% 2,698 1.5 1.0 neuropsychological) Assessment methods (e.g., self‐report, multi-informant reports, psychophysiological measures, work 31 samples, assessment centers, direct 10.3% 19.7% 32.6% 37.4% 2,689 2.0 1.0 observation, structured and semi‐ structured interviews) and their strengths and limitations Commonly used instruments for the measurement of characteristics and behaviors of individuals (e.g., psychosocial functioning; cognitive and 32 10.8% 20.95 30.2% 38.2% 2,692 2.0 1.0 personality; achievement; psychopathology; risk assessment) and the appropriate use of these tests with various populations Issues of differential diagnosis and integration of non‐psychological 33 5.7% 12.7% 33.8% 47.9% 2,687 2.2 0.9 information into psychological assessment

Pearson VUE Confidential Page 63 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Instruments and methods appropriate for the assessment of groups and 34 organizations (e.g., program evaluation, 57.7% 26.5% 10.1% 5.8% 2,676 0.6 0.9 needs assessment, organizational and personnel assessment) Criteria for selection and adaptation of assessment methods including evidenced-based knowledge of 35 assessment limitations, cultural 23.1% 34.8% 26.7% 15.4% 2,673 1.3 1.0 appropriateness, trans‐cultural adaptation, and language accommodations Classification systems and their underlying rationales and limitations for 36 evaluating client functioning; 24.4% 32.7% 28.8% 14.1% 2,673 1.3 1.0 dimensional vs. categorical approaches to diagnosis Factors influencing interpretation of data and decision‐making (e.g., base rates, 37 22.4% 31.3% 28.8% 17.5% 2,673 1.4 1.0 group differences, cultural biases and differences, heuristics, evidence base) Constructs of epidemiology and base rates of psychological conditions and 38 26.5% 38.8% 25.85 8.9% 2,663 1.2 0.9 behavioral disorders in clinical or demographic populations Major research-based theories and 39 10.7% 29.3% 38.5% 21.5% 2,643 1.7 0.9 models of psychopathology Measurement of outcomes and changes due to prevention or intervention efforts 40 21.5% 31.3% 30.0% 17.2% 2,665 1.4 1.0 with individuals, couples, families, groups, and organizations Use of technology in implementing tests, surveys, and other forms of assessment 41 37.8% 32.3% 20.5% 9.3% 2,672 1.0 1.0 and diagnostic evaluation; validity, cost effectiveness, consumer acceptability Treatment, Intervention, and Prevention and Supervision

Pearson VUE Confidential Page 64 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client 42 7.0% 14.0% 34.2% 44.7% 2,619 2.2 0.9 or patient with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change) Contemporary theories/models of 43 treatment/intervention/prevention and 5.5% 18.1% 42.1% 34.4% 2,618 2.1 0.9 their evidence base Treatment techniques/interventions and 44 the evidence for their comparative 4.9% 20.2% 40.6% 34.3% 2,606 2.0 0.9 efficacy and effectiveness Methods and their evidence base for prevention, intervention and/or 45 9.7% 30.9% 36.5% 22.8% 2,614 1.7 0.9 rehabilitation with diverse or special populations Interventions to enhance growth and performance of individuals, couples, 46 12.7% 25.3% 35.4% 26.6% 2,607 1.8 1.0 families, groups, systems, and organizations 47 Consultation models and processes 29.2% 36.0% 24.7% 10.0% 2,598 1.2 1.0 Models of vocational and career 48 development in the provision of 57.2% 31.3% 8.4% 3.0% 2,591 0.6 0.8 psychological services Technology‐assisted psychological 49 50.7% 29.3% 14.5% 5.5% 2,606 0.7 0.9 services, including telepsychology Healthcare systems, structures, and 50 economics, and how these impact 25.7% 31.1% 28.1% 15.1% 2,608 1.3 1.0 intervention choice Approaches to health promotion, risk 51 12.6% 26.0% 37.6% 23.8% 2,607 1.7 1.0 reduction, resilience, and wellness Contemporary theories/models of 52 32.2% 33.4% 23.0% 11.4% 2,606 1.1 1.0 supervision and their evidence base Research Methods and Statistics Sampling and data collection methods 53 48.8% 30.6% 13.1% 7.5% 2,718 0.8 0.9 and issues

Pearson VUE Confidential Page 65 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Design of case studies, correlational, 54 quasi‐experimental and experimental 55.2% 29.4% 9.5% 5.9% 2,685 0.7 0.9 studies Analytic methods, including qualitative (e.g., thematic, phenomenological) and quantitative (e.g., probability theory; descriptive, inferential, and parametric 55 54.2% 27.5% 10.7% 7.6% 2,713 0.7 0.9 statistics; meta-analysis, exploratory and confirmatory factor analysis, non‐ parametric statistics, and causal modeling) Statistical interpretation (e.g., power, effect size, causation vs. association, 56 sensitivity and specificity, 38.4% 32.8% 19.2% 9.6% 2,712 1.0 1.0 generalizability, clinical vs. statistical significance) Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to 57 29.7% 33.8% 23.7% 12.8% 2,705 1.2 1.0 generalizations, threats to internal and external validity, design flaws, level of evidence) Evaluation strategies and techniques (e.g., needs assessment, process/implementation evaluation, 58 39.0% 33.9% 18.7% 8.4% 2,713 1.0 1.0 formative and summative assessment program evaluation, outcome evaluation, cost‐benefit analysis) Considerations regarding community involvement and participation in 59 52.6% 30.6% 11.9% 4.9% 2,705 0.7 0.9 research, particularly for under- represented populations Dissemination and presentation of 60 47.0% 29.9% 15.4% 7.7% 2,700 0.8 1.0 research findings Ethical/Legal/Professional Issues Ethical principles and codes of conduct 61 1.7% 15.7% 38.1% 44.5% 2,711 2.3 0.8 for psychologists (APA, CPA)

Pearson VUE Confidential Page 66 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Professional standards and relevant guidelines for the practice of psychology 62 (e.g., practice guidelines for providers of 3.0% 20.3% 39.2% 37.5% 2,708 2.1 0.8 psychological services, standards for educational and psychological testing) Awareness of applicable laws/statutes 63 and/or judicial decisions that affect 4.45 30.6% 35.5% 29.5% 2,722 1.9 0.9 psychological practice Identifying and managing potential 64 3.2% 26.7% 38.7% 31.4% 2,712 2.0 0.8 ethical issues 65 Models of ethical decision‐making 17.9% 44.9% 25.2% 12.0% 2,683 1.3 0.9 Approaches for continuing professional 66 9.3% 36.0% 39.7% 15.1% 2,699 1.6 0.9 development Consideration of emerging social, legal, 67 ethical, and policy issues and their 10.6% 42.8% 34.8% 11.8% 2,712 1.5 0.8 impact on psychology 68 Client’s/patient’s rights 3.3% 17.5% 35.4% 43.8% 2,712 2.2 0.8 69 Ethical issues in the conduct of research 53.2% 27.9% 10.1% 8.7% 2,689 0.7 1.0 70 Ethical issues in supervision 27.0% 27.4% 25.8% 19.8% 2,704 1.4 1.1 Ethical issues in technology assisted 71 38.9% 35.8% 17.3% 8.0% 2,713 0.9 0.9 psychological services Note. All domains/knowledge areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 67 Table K2. EPPP Knowledge Scales Criticality Ratings Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Biological Bases of Behavior Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and 1 9.2% 24.9% 41.3% 24.6% 2,574 1.8 0.9 affect in normal, acute and chronic neurobehavioral disease processes; and/or acute and chronic disease comorbidities Drug classification, mechanisms of action, and desired/adverse effects of 2 5.3% 17.3% 46.3% 31.2% 2,577 2.0 0.8 therapeutic agents, drugs of abuse, and complementary or alternative remedies Results from major trials and general guidelines for pharmacological, 3 10.6% 26.3% 40.4% 22.7% 2,572 1.8 0.9 psychotherapeutic, and combined treatment of psychological disorders Behavioral genetics, transmission and expression of genetic information and its 4 modification and the role and limitations 19.7% 39.1% 31.8% 9.3% 2,573 1.3 0.9 of this information in understanding disorders Applications of structural and functional brain imaging methods, electrophysiological methods, 5 28.7% 36.8% 25.0% 9.6% 2,575 1.2 0.9 therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness Cognitive-Affective Bases of Behavior Major research‐based theories and 6 models of intelligence and their 12.7% 30.2% 37.9% 19.2% 2,586 1.6 0.9 application Major research-based theories, models, 7 and principles of learning and their 9.9% 29.3% 40.5% 20.3% 2,580 1.7 0.9 application

Pearson VUE Confidential Page 68 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number

Major research‐based theories and 8 12.1% 35.0% 38.6% 14.3% 2,567 1.6 0.9 models of memory and their application

Major research‐based theories and 9 models of motivation and their 9.6% 32.9% 40.8% 16.8% 2,573 1.6 0.9 application Major research-based theories and 10 6.5% 23.6% 43.8% 26.1% 2,561 1.9 0.9 models of emotion and their application Additional elements of cognition, including sensation and perception, 11 attention, language, information 9.1% 26.8% 42.5% 21.6% 2,578 1.8 0.9 processing, visual-spatial processing, executive functioning Relations among cognitions/beliefs, 12 behavior, affect, temperament, and 3.0% 13.4% 40.8% 42.8% 2,579 2.2 0.8 mood Influence of psychosocial factors on 13 2.4% 13.0% 42.6% 42.0% 2,578 2.2 0.8 cognitions/beliefs and behaviors Social and Cultural Bases of Behavior Social cognition (e.g., theories, person 14 perception, development of stereotypes, 9.2% 29.1% 41.7% 20.1% 2,553 1.7 0.9 prejudice) Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non‐ 15 7.8% 26.2% 41.9% 24.2% 2,545 1.8 0.9 verbal communication, internet communication, mate selection, empathy) Group and systems processes (e.g., school, work, and family systems, job 16 satisfaction, team functioning, 10.6% 29.2% 41.8% 18.4% 2,547 1.7 0.9 conformity, persuasion) and social influences on functioning Personality theories (e.g., psychodynamic, humanistic/existential, 17 7.9% 26.7% 35.8% 29.6% 2,544 1.9 0.9 cognitive, behavioral, trait theory, interpersonal)

Pearson VUE Confidential Page 69 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Cultural and sociopolitical psychology (e.g., privilege/oppression, cross‐cultural 18 comparisons, political differences, 7.7% 22.9% 39.7% 29.8% 2,547 1.9 0.9 international and global awareness, religion and spirituality, acculturation) Identity diversity and intersectionality (e.g., psychological impact of diversity 19 8.2% 24.3% 41.4% 26.1% 2,547 1.9 0.9 on individuals, families, and systems; conceptual models; assumptions) Causes, manifestations, and effects of 20 13.0% 23.9% 37.9% 25.2% 2,546 1.8 1.0 oppression Growth and Lifespan Development Normal growth and development across the lifespan (e.g., biological, physical, 21 3.6% 15.7% 42.3% 38.3% 2,605 2.2 0.8 sexual, cognitive, perceptual, social, personality, moral, emotional, career) Influence of individual‐environment interaction over time on development 22 (e.g., the relationship between the 4.9% 18.2% 42.7% 34.3% 2,604 2.1 0.8 individual and the social, academic, work, community environment) Major research‐based theories of 23 12.8% 34.9% 38.9% 13.5% 2,602 1.5 0.9 development Influence of diverse identities on 24 10.5% 26.2% 41.2% 22.1% 2,575 1.7 0.9 development Family development, configuration, and 25 functioning and its impact on the 6.0% 21.5% 44.2% 28.3% 2,595 1.9 0.9 individual across the lifespan Life events that can influence the normal 26 course of development across the 2.3% 15.1% 46.1% 36.5% 2,600 2.2 0.8 lifespan Risk/protective factors that may impact a developmental course (e.g., nutrition, 27 prenatal care, health care, social 2.8% 12.6% 41.8% 42.7% 2,599 2.2 0.8 support, socioeconomic status, abuse/victimization/resiliency) Disorders/diseases that impact the 28 expected course of development over 4.2% 19.5% 44.8% 31.4% 2,600 2.0 0.8 the lifespan

Pearson VUE Confidential Page 70 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Assessment and Diagnosis Psychometric theory, generalizability theory, item and test characteristics, test construction and standardization 29 13.9% 19.6% 33.3% 33.2% 2,579 1.9 1.0 procedures, reliability and validity, sensitivity and specificity, and test fairness and bias Assessment theories and models (e.g., 30 developmental, behavioral, ecological, 11.0% 26.6% 38.0% 24.4% 2,573 1.8 0.9 neuropsychological) Assessment methods (e.g., self‐report, multi-informant reports, psychophysiological measures, work 31 samples, assessment centers, direct 6.6% 17.6% 37.8% 38.0% 2,567 2.1 0.9 observation, structured and semi‐ structured interviews) and their strengths and limitations Commonly used instruments for the measurement of characteristics and behaviors of individuals (e.g., psychosocial functioning; cognitive and 32 5.3% 14.8% 36.6% 43.3% 2,567 2.2 0.9 personality; achievement; psychopathology; risk assessment) and the appropriate use of these tests with various populations Issues of differential diagnosis and integration of non‐psychological 33 3.0% 10.2% 34.1% 52.7% 2,565 2.4 0.8 information into psychological assessment Instruments and methods appropriate for the assessment of groups and 34 organizations (e.g., program evaluation, 33.2% 34.6% 24.4% 7.8% 2,557 1.1 0.9 needs assessment, organizational and personnel assessment)

Pearson VUE Confidential Page 71 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Criteria for selection and adaptation of assessment methods including evidenced-based knowledge of 35 assessment limitations, cultural 10.4% 19.4% 36.9% 33.4% 2,555 1.9 1.0 appropriateness, trans‐cultural adaptation, and language accommodations Classification systems and their underlying rationales and limitations for 36 evaluating client functioning; 16.2% 30.9% 33.8% 19.1% 2,551 1.6 1.0 dimensional vs. categorical approaches to diagnosis Factors influencing interpretation of data and decision‐making (e.g., base rates, 37 11.7% 23.8% 36.1% 28.4% 2,554 1.8 1.0 group differences, cultural biases and differences, heuristics, evidence base) Constructs of epidemiology and base rates of psychological conditions and 38 16.2% 33.0% 36.6% 14.3% 2,543 1.5 0.9 behavioral disorders in clinical or demographic populations Major research-based theories and 39 7.9% 23.0% 40.5% 28.6% 2,528 1.9 0.9 models of psychopathology Measurement of outcomes and changes due to prevention or intervention efforts 40 11.0% 24.7% 40.8% 23.6% 2,546 1.8 0.9 with individuals, couples, families, groups, and organizations Use of technology in implementing tests, surveys, and other forms of assessment 41 24.8% 35.6% 28.5% 11.0% 2,554 1.3 1.0 and diagnostic evaluation; validity, cost effectiveness, consumer acceptability Treatment, Intervention, and Prevention and Supervision Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client 42 4.3% 11.4% 32.9% 51.4% 2,530 2.3 0.8 or patient with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change)

Pearson VUE Confidential Page 72 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Contemporary theories/models of 43 treatment/intervention/prevention and 3.8% 14.2% 41.4% 40.6% 2,530 2.2 0.8 their evidence base Treatment techniques/interventions and 44 the evidence for their comparative 3.0% 13.5% 40.0% 43.5% 2,525 2.2 0.8 efficacy and effectiveness Methods and their evidence base for prevention, intervention and/or 45 5.2% 16.4% 43.6% 34.8% 2,525 2.1 0.8 rehabilitation with diverse or special populations Interventions to enhance growth and performance of individuals, couples, 46 8.4% 22.7% 39.9% 29.0% 2,521 1.9 0.9 families, groups, systems, and organizations 47 Consultation models and processes 19.7% 36.7% 32.5% 11.1% 2,509 1.4 0.9 Models of vocational and career 48 development in the provision of 38.2% 39.6% 18.5% 3.8% 2,508 0.9 0.8 psychological services Technology‐assisted psychological 49 25.1% 31.3% 31.2% 12.3% 2,518 1.3 1.0 services, including telepsychology Healthcare systems, structures, and 50 economics, and how these impact 14.2% 28.2% 36.3% 21.3% 2,522 1.6 1.0 intervention choice Approaches to health promotion, risk 51 6.4% 21.0% 45.2% 27.4% 2,518 1.9 0.9 reduction, resilience, and wellness Contemporary theories/models of 52 19.0% 29.0% 36.4% 15.6% 2,519 1.5 1.0 supervision and their evidence base Research Methods and Statistics Sampling and data collection methods 53 27.5% 27.1% 30.2% 15.2% 2,593 1.3 1.0 and issues Design of case studies, correlational, 54 quasi‐experimental and experimental 31.2% 29.0% 26.9% 12.9% 2,563 1.2 1.0 studies

Pearson VUE Confidential Page 73 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Analytic methods, including qualitative (e.g., thematic, phenomenological) and quantitative (e.g., probability theory; descriptive, inferential, and parametric 55 32.2% 29.9% 25.3% 12.7% 2,590 1.2 1.0 statistics; meta-analysis, exploratory and confirmatory factor analysis, non‐ parametric statistics, and causal modeling) Statistical interpretation (e.g., power, effect size, causation vs. association, 56 sensitivity and specificity, 22.8% 27.7% 29.3% 20.1% 2,591 1.5 1.1 generalizability, clinical vs. statistical significance) Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to 57 17.7% 22.9% 32.2% 27.2% 2,593 1.7 1.1 generalizations, threats to internal and external validity, design flaws, level of evidence) Evaluation strategies and techniques (e.g., needs assessment, process/implementation evaluation, 58 23.3% 32.3% 30.4% 14.0% 2,592 1.4 1.0 formative and summative assessment program evaluation, outcome evaluation, cost‐benefit analysis) Considerations regarding community involvement and participation in 59 25.2% 27.3% 32.2% 15.3% 2,591 1.4 1.0 research, particularly for under- represented populations Dissemination and presentation of 60 28.4% 28.0% 28.7% 15.0% 2,585 1.3 1.0 research findings Ethical/Legal/Professional Issues Ethical principles and codes of conduct 61 0.4% 2.3% 13.5% 83.8% 2,616 2.8 0.5 for psychologists (APA, CPA) Professional standards and relevant guidelines for the practice of psychology 62 (e.g., practice guidelines for providers of 0.9% 5.8% 24.4% 68.8% 2,618 2.6 0.6 psychological services, standards for educational and psychological testing)

Pearson VUE Confidential Page 74 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Awareness of applicable laws/statutes 63 and/or judicial decisions that affect 1.4% 7.0% 28.9% 62.7% 2,629 2.5 0.7 psychological practice Identifying and managing potential 64 0.4% 4.4% 19.1% 76.0% 2,624 2.7 0.6 ethical issues 65 Models of ethical decision‐making 7.8% 23.1% 37.6% 31.5% 2,601 1.9 0.9 Approaches for continuing professional 66 8.8% 26.7% 41.6% 23.0% 2,614 1.8 0.9 development Consideration of emerging social, legal, 67 ethical, and policy issues and their 5.0% 24.4% 45.2% 25.4% 2,622 1.9 0.8 impact on psychology 68 Client’s/patient’s rights 0.9% 4.8% 23.7% 70.7% 2,621 2.6 0.6 69 Ethical issues in the conduct of research 17.9% 17.0% 26.1% 39.0% 2,605 1.9 1.1 70 Ethical issues in supervision 9.7% 12.9% 33.4% 44.0% 2,611 2.1 1.0 Ethical issues in technology assisted 71 12.2% 21.1% 35.9% 30.8% 2,621 1.9 1.0 psychological services Note. All domains/knowledge areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 75 Table K3. EPPP Knowledge Scales Importance Ratings Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Biological Bases of Behavior Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and 1 10.3% 24.7% 35.5% 29.4% 2,572 1.8 1.0 affect in normal, acute and chronic neurobehavioral disease processes; and/or acute and chronic disease comorbidities Drug classification, mechanisms of action, and desired/adverse effects of 2 6.5% 20.1% 41.0% 32.4% 2,576 2.0 0.9 therapeutic agents, drugs of abuse, and complementary or alternative remedies Results from major trials and general guidelines for pharmacological, 3 12.5% 29.2% 37.1% 21.2% 2,571 1.7 0.9 psychotherapeutic, and combined treatment of psychological disorders Behavioral genetics, transmission and expression of genetic information and its 4 modification and the role and limitations 21.0% 39.4% 29.0% 10.6% 2,575 1.3 0.9 of this information in understanding disorders Applications of structural and functional brain imaging methods, electrophysiological methods, 5 32.0% 35.8% 21.9% 10.3% 2,572 1.1 1.0 therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness Cognitive-Affective Bases of Behavior Major research‐based theories and 6 models of intelligence and their 14.7% 29.1% 33.6% 22.5% 2,587 1.6 1.0 application Major research-based theories, models, 7 and principles of learning and their 9.9% 26.5% 37.6% 26.0% 2,574 1.8 0.9 application

Pearson VUE Confidential Page 76 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number

Major research‐based theories and 8 12.6% 32.8% 36.4% 18.1% 2,566 1.6 0.9 models of memory and their application

Major research‐based theories and 9 models of motivation and their 8.8% 28.5% 39.8% 22.9% 2,571 1.8 0.9 application Major research-based theories and 10 6.1% 20.6% 41.2% 32.1% 2,557 2.0 0.9 models of emotion and their application Additional elements of cognition, including sensation and perception, 11 attention, language, information 8.0% 23.9% 39.6% 28.4% 2,578 1.9 0.9 processing, visual-spatial processing, executive functioning Relations among cognitions/beliefs, 12 behavior, affect, temperament, and 2.4% 9.0% 34.7% 53.8% 2,579 2.4 0.8 mood Influence of psychosocial factors on 13 2.2% 8.7% 37.5% 51.6% 2,570 2.4 0.7 cognitions/beliefs and behaviors Social and Cultural Bases of Behavior Social cognition (e.g., theories, person 14 perception, development of stereotypes, 9.6% 30.0% 39.4% 21.0% 2,552 1.7 0.9 prejudice) Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non‐ 15 7.9% 23.2% 38.7% 30.1% 2,545 1.9 0.9 verbal communication, internet communication, mate selection, empathy) Group and systems processes (e.g., school, work, and family systems, job 16 satisfaction, team functioning, 11.3% 25.0% 39.9% 23.8% 2,544 1.8 0.9 conformity, persuasion) and social influences on functioning Personality theories (e.g., psychodynamic, humanistic/existential, 17 6.5% 22.0% 34.2% 37.3% 2,541 2.0 0.9 cognitive, behavioral, trait theory, interpersonal)

Pearson VUE Confidential Page 77 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Cultural and sociopolitical psychology (e.g., privilege/oppression, cross‐cultural 18 comparisons, political differences, 8.2% 23.9% 36.8% 31.1% 2,548 1.9 0.9 international and global awareness, religion and spirituality, acculturation) Identity diversity and intersectionality (e.g., psychological impact of diversity 19 8.7% 25.5% 37.9% 27.9% 2,546 1.9 0.9 on individuals, families, and systems; conceptual models; assumptions) Causes, manifestations, and effects of 20 14.5% 28.5% 32.7% 24.3% 2,544 1.7 1.0 oppression Growth and Lifespan Development Normal growth and development across the lifespan (e.g., biological, physical, 21 3.5% 11.6% 38.5% 46.5% 2,603 2.3 0.8 sexual, cognitive, perceptual, social, personality, moral, emotional, career) Influence of individual‐environment interaction over time on development 22 (e.g., the relationship between the 3.9% 15.3% 38.8% 42.0% 2,598 2.2 0.8 individual and the social, academic, work, community environment) Major research‐based theories of 23 11.9% 33.0% 39.0% 16.1% 2,597 1.6 0.9 development Influence of diverse identities on 24 10.8% 26.0% 39.3% 23.8% 2,572 1.8 0.9 development Family development, configuration, and 25 functioning and its impact on the 5.6% 18.4% 41.3% 34.7% 2,590 2.1 0.9 individual across the lifespan Life events that can influence the normal 26 course of development across the 2.0% 12.4% 42.7% 42.9% 2,595 2.3 0.7 lifespan Risk/protective factors that may impact a developmental course (e.g., nutrition, 27 prenatal care, health care, social 3.0% 12.4% 39.8% 44.8% 2,596 2.3 0.8 support, socioeconomic status, abuse/victimization/resiliency) Disorders/diseases that impact the 28 expected course of development over 4.5% 19.3% 40.5% 35.6% 2,595 2.1 0.9 the lifespan

Pearson VUE Confidential Page 78 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Assessment and Diagnosis Psychometric theory, generalizability theory, item and test characteristics, test construction and standardization 29 16.2% 21.9% 28.1% 33.8% 2,575 1.8 1.1 procedures, reliability and validity, sensitivity and specificity, and test fairness and bias Assessment theories and models (e.g., 30 developmental, behavioral, ecological, 12.1% 25.9% 35.1% 26.9% 2,570 1.8 1.0 neuropsychological) Assessment methods (e.g., self‐report, multi-informant reports, psychophysiological measures, work 31 samples, assessment centers, direct 7.1% 16.4% 34.2% 42.3% 2,562 2.1 0.9 observation, structured and semi‐ structured interviews) and their strengths and limitations Commonly used instruments for the measurement of characteristics and behaviors of individuals (e.g., psychosocial functioning; cognitive and 32 6.0% 15.2% 31.7% 47.1% 2,562 2.2 0.9 personality; achievement; psychopathology; risk assessment) and the appropriate use of these tests with various populations Issues of differential diagnosis and integration of non‐psychological 33 3.5% 9.9% 29.9% 56.7% 2,560 2.4 0.8 information into psychological assessment Instruments and methods appropriate for the assessment of groups and 34 organizations (e.g., program evaluation, 40.3% 31.6% 19.2% 8.9% 2,557 1.0 1.0 needs assessment, organizational and personnel assessment)

Pearson VUE Confidential Page 79 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Criteria for selection and adaptation of assessment methods including evidenced-based knowledge of 35 assessment limitations, cultural 14.5% 23.1% 34.0% 28.4% 2,549 1.8 1.0 appropriateness, trans‐cultural adaptation, and language accommodations Classification systems and their underlying rationales and limitations for 36 evaluating client functioning; 17.9% 30.6% 31.8% 19.7% 2,547 1.5 1.0 dimensional vs. categorical approaches to diagnosis Factors influencing interpretation of data and decision‐making (e.g., base rates, 37 14.8% 25.1% 32.5% 27.6% 2,555 1.7 1.0 group differences, cultural biases and differences, heuristics, evidence base) Constructs of epidemiology and base rates of psychological conditions and 38 17.7% 33.2% 33.7% 15.4% 2,541 1.5 1.0 behavioral disorders in clinical or demographic populations Major research-based theories and 39 7.8% 22.3% 39.3% 30.6% 2,527 1.9 0.9 models of psychopathology Measurement of outcomes and changes due to prevention or intervention efforts 40 13.8% 24.2% 36.0% 26.0% 2,541 1.7 1.0 with individuals, couples, families, groups, and organizations Use of technology in implementing tests, surveys, and other forms of assessment 41 26.5% 33.7% 27.2% 12.6% 2,552 1.3 1.0 and diagnostic evaluation; validity, cost effectiveness, consumer acceptability Treatment, Intervention, and Prevention and Supervision Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client 42 5.1% 11.1% 27.8% 56.0% 2,521 2.3 0.9 or patient with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change)

Pearson VUE Confidential Page 80 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Contemporary theories/models of 43 treatment/intervention/prevention and 4.1% 13.9% 36.7% 45.3% 2,518 2.2 0.8 their evidence base Treatment techniques/interventions and 44 the evidence for their comparative 3.7% 13.8% 36.0% 46.5% 2,516 2.3 0.8 efficacy and effectiveness Methods and their evidence base for prevention, intervention and/or 45 6.5% 19.0% 38.5% 35.9% 2,514 2.0 0.9 rehabilitation with diverse or special populations Interventions to enhance growth and performance of individuals, couples, 46 9.0% 21.9% 36.3% 32.8% 2,512 1.9 1.0 families, groups, systems, and organizations 47 Consultation models and processes 19.8% 34.0% 31.1% 15.1% 2,500 1.4 1.0 Models of vocational and career 48 development in the provision of 39.0% 38.2% 18.2% 4.5% 2,496 0.9 0.9 psychological services Technology‐assisted psychological 49 32.0% 30.1% 26.5% 11.4% 2,510 1.2 1.0 services, including telepsychology Healthcare systems, structures, and 50 economics, and how these impact 17.1% 28.7% 33.2% 21.0% 2,514 1.6 1.0 intervention choice Approaches to health promotion, risk 51 7.7% 21.1% 41.2% 30.0% 2,512 1.9 0.9 reduction, resilience, and wellness Contemporary theories/models of 52 20.9% 27.7% 34.1% 17.3% 2,511 1.5 1.0 supervision and their evidence base Research Methods and Statistics Sampling and data collection methods 53 31.9% 28.6% 25.5% 13.9% 2,585 1.2 1.0 and issues Design of case studies, correlational, 54 quasi‐experimental and experimental 36.6% 29.8% 22.6% 11.0% 2,555 1.1 1.0 studies

Pearson VUE Confidential Page 81 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Analytic methods, including qualitative (e.g., thematic, phenomenological) and quantitative (e.g., probability theory; descriptive, inferential, and parametric 55 36.3% 30.6% 21.1% 12.0% 2,584 1.1 1.0 statistics; meta-analysis, exploratory and confirmatory factor analysis, non‐ parametric statistics, and causal modeling) Statistical interpretation (e.g., power, effect size, causation vs. association, 56 sensitivity and specificity, 25.4% 28.9% 26.7% 19.1% 2,587 1.4 1.1 generalizability, clinical vs. statistical significance) Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to 57 19.4% 25.9% 30.0% 24.6% 2,586 1.6 1.1 generalizations, threats to internal and external validity, design flaws, level of evidence) Evaluation strategies and techniques (e.g., needs assessment, process/implementation evaluation, 58 26.9% 31.0% 27.2% 14.9% 2,584 1.3 1.0 formative and summative assessment program evaluation, outcome evaluation, cost‐benefit analysis) Considerations regarding community involvement and participation in 59 33.4% 29.8% 24.2% 12.5% 2,584 1.2 1.0 research, particularly for under- represented populations Dissemination and presentation of 60 31.6% 27.8% 25.3% 15.3% 2,576 1.2 1.1 research findings Ethical/Legal/Professional Issues Ethical principles and codes of conduct 61 0.4% 2.8% 14.3% 82.5% 2,608 2.8 0.5 for psychologists (APA, CPA) Professional standards and relevant guidelines for the practice of psychology 62 (e.g., practice guidelines for providers of 1.0% 6.0% 24.8% 68.3% 2,615 2.6 0.6 psychological services, standards for educational and psychological testing)

Pearson VUE Confidential Page 82 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Awareness of applicable laws/statutes 63 and/or judicial decisions that affect 1.4% 9.2% 26.8% 62.7% 2,630 2.5 0.7 psychological practice Identifying and managing potential 64 0.4% 5.3% 19.8% 74.5% 2,622 2.7 0.6 ethical issues 65 Models of ethical decision‐making 8.0% 23.8% 35.4% 32.8% 2,602 1.9 0.9 Approaches for continuing professional 66 6.1% 24.6% 42.0% 27.3% 2,612 1.9 0.9 development Consideration of emerging social, legal, 67 ethical, and policy issues and their 5.0% 24.7% 43.8% 26.5% 2,619 1.9 0.8 impact on psychology 68 Client’s/patient’s rights 1.5% 6.1% 24.5% 67.8% 2,622 2.6 0.7 69 Ethical issues in the conduct of research 28.8% 22.9% 21.3% 27.1% 2,605 1.5 1.2 70 Ethical issues in supervision 14.4% 14.4% 28.5% 42.8% 2,607 2.0 1.1 Ethical issues in technology assisted 71 19.6% 25.7% 29.9% 24.8% 2,616 1.6 1.1 psychological services Note. All domains/knowledge areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 83 Appendix L. Responses to the Competency Scales

Table L1. EPPP Part 2 Competency Scales Frequency Ratings Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Scientific Orientation Select relevant research literature and critically review its assumptions, 1 15.3% 40.4% 30.8% 13.5% 1,958 1.4 0.9 conceptualization, methodology, interpretation, and generalizability Interpret, evaluate, and integrate results of assessment activities within the context of scientific/professional 2 12.3% 23.4% 34.9% 29.5% 1,941 1.8 1.0 knowledge to formulate and reformulate working hypotheses, conceptualizations, and recommendations Articulate a rationale for decisions and psychological services that rely on 3 12.1% 27.8% 38.0% 22.1% 1,942 1.7 0.9 objective supporting data (e.g., research results, base rates, epidemiological data) Acquire and disseminate knowledge in 4 accord with scientific and ethical 8.2% 22.7% 39.4% 29.7% 1,940 1.9 0.9 principles Select and use evidence-based 5 10.4% 18.0% 33.7% 38.0% 1,931 2.0 1.0 assessment methods and instruments Select and use evidence-based 6 6.4% 11.2% 37.8% 44.6% 1,926 2.2 0.9 interventions Professional Practice Apply knowledge of individual and 7 diversity characteristics in assessment 3.2% 11.4% 42.0% 43.4% 1,861 2.3 0.8 and diagnosis 8 Demonstrate effective interviewing skills 1.3% 3.8% 20.3% 74.5% 1,853 2.7 0.6 Administer and score instruments 9 17.7% 22.3% 27.5% 32.4% 1,846 1.7 1.1 following current guidelines and research Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, 10 9.0% 17.5% 31.5% 42.1% 1,850 2.1 1.0 interviewees, collateral sources of information) following current guidelines and research

Pearson VUE Confidential Page 84 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Formulate diagnoses, recommendations, and/or professional opinions using 11 2.7% 7.8% 31.1% 58.3% 1,848 2.5 0.8 relevant criteria and considering all assessment data Communicate assessment results to 12 clients, referral sources, and other 8.7% 18.3% 36.5% 36.4% 1,842 2.0 0.9 professionals in an integrative manner Evaluate service or program 13 29.2% 39.7% 22.0% 9.0% 1,836 1.1 0.9 effectiveness across a variety of contexts Select interventions for clients based on ongoing assessment and research 14 5.3% 14.2% 40.3% 40.2% 1,839 2.2 0.9 evidence as well as contextual and diversity factors Apply and modify interventions based on ongoing assessment, research, 15 contextual factors, client characteristics, 6.4% 15.3% 38.4% 39.9% 1,835 2.1 0.9 and situational and environmental variables Consult and collaborate within and 16 2.6% 15.6% 42.4% 39.3% 1,834 2.2 0.8 across professions Ensure compliance with policies and 17 procedures of the practice/organization, 3.8% 12.6% 34.4% 49.1% 1,830 2.3 0.8 the profession, and the jurisdiction Monitor, evaluate, and accurately and sensitively communicate supervisee 18 performance to the supervisee, the 31.0% 18.4% 29.6% 21.0% 1,827 1.4 1.1 organization, and the jurisdiction as needed Create and maintain a supportive environment in which effective 19 28.0% 13.9% 26.0% 32.1% 1,826 1.6 1.2 supervision occurs for trainees and other professionals being supervised Relational Competence Integrate and apply theory, research, professional guidelines, and personal 20 2.6% 12.9% 43.4% 41.1% 1,956 2.2 0.8 understanding about social contexts to work effectively with diverse clients

Pearson VUE Confidential Page 85 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Work effectively with individuals, 21 families, groups, communities, and/or 1.7% 4.8% 26.7% 66.8% 1,955 2.6 0.7 organizations In all areas of professional practice, 22 demonstrate respect for others, 0.3% 1.5% 21.0% 77.3% 1,955 2.8 0.5 including those with differing viewpoints Identify and manage interpersonal 23 8.8% 43.8% 28.0% 19.4% 1,951 1.6 0.9 conflict between self and others Professionalism Identify and observe boundaries of 24 competence in all areas of professional 0.8% 11.5% 41.0% 46.7% 2,017 2.3 0.7 practice Critically evaluate one’s own professional 25 practice through self-reflection and 0.9% 14.0% 46.4% 38.7% 2,019 2.2 0.7 feedback from others Ethical and Legal Practice Demonstrate and promote values and behaviors commensurate with standards 26 0.5% 2.5% 25.2% 71.9% 1,986 2.7 0.5 of practice, including ethics codes, laws, and regulations Accurately represent and document work 27 performed in professional practice and 1.9% 4.8% 24.5% 68.8% 1,980 2.6 0.7 scholarship 28 Implement ethical practice management 2.4% 6.7% 29.3% 61.5% 1,963 2.5 0.7 Establish and maintain a process that 29 2.1% 10.6% 35.4% 51.9% 1,965 2.4 0.8 promotes ethical decision-making Systems Thinking Work effectively within organizations and 30 5.7% 9.8% 26.8% 57.7% 2,036 2.4 0.9 systems Demonstrate interdisciplinary 31 3.8% 15.0% 35.4% 45.8% 2,025 2.2 0.8 collaborations Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 86 Table L2. EPPP Part 2 Competency Scales Criticality Ratings Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Scientific Orientation Select relevant research literature and critically review its assumptions, 1 5.2% 24.2% 42.0% 28.6% 1,854 1.9 0.9 conceptualization, methodology, interpretation, and generalizability Interpret, evaluate, and integrate results of assessment activities within the context of scientific/professional 2 4.1% 12.6% 36.7% 46.6% 1,842 2.3 0.8 knowledge to formulate and reformulate working hypotheses, conceptualizations, and recommendations Articulate a rationale for decisions and psychological services that rely on 3 5.2% 17.0% 41.8% 36.0% 1,840 2.1 0.9 objective supporting data (e.g., research results, base rates, epidemiological data) Acquire and disseminate knowledge in 4 accord with scientific and ethical 4.0% 13.4% 38.0% 44.6% 1,841 2.2 0.8 principles Select and use evidence-based 5 4.4% 8.8% 32.9% 54.0% 1,837 2.4 0.8 assessment methods and instruments Select and use evidence-based 6 3.2% 7.8% 34.2% 54.8% 1,838 2.4 0.8 interventions Professional Practice Apply knowledge of individual and 7 diversity characteristics in assessment 0.7% 5.0% 29.3% 65.0% 1,770 2.6 0.6 and diagnosis 8 Demonstrate effective interviewing skills 0.3% 4.2% 23.7% 71.8% 1,766 2.7 0.6 Administer and score instruments 9 4.5% 10.2% 27.5% 57.8% 1,758 2.4 0.8 following current guidelines and research Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, 10 2.1% 6.9% 27.6% 63.4% 1,759 2.5 0.7 interviewees, collateral sources of information) following current guidelines and research

Pearson VUE Confidential Page 87 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Formulate diagnoses, recommendations, and/or professional opinions using 11 0.6% 2.0% 21.1% 76.2% 1,757 2.7 0.5 relevant criteria and considering all assessment data Communicate assessment results to 12 clients, referral sources, and other 1.9% 5.6% 31.1% 61.5% 1,757 2.5 0.7 professionals in an integrative manner Evaluate service or program 13 9.8% 25.2% 42.8% 22.2% 1,740 1.8 0.9 effectiveness across a variety of contexts Select interventions for clients based on ongoing assessment and research 14 1.5% 5.8% 34.7% 58.0% 1,752 2.5 0.7 evidence as well as contextual and diversity factors Apply and modify interventions based on ongoing assessment, research, 15 contextual factors, client characteristics, 1.1% 5.9% 34.2% 58.8% 1,750 2.5 0.7 and situational and environmental variables Consult and collaborate within and 16 1.1% 9.4% 42.2% 47.3% 1,748 2.4 0.7 across professions Ensure compliance with policies and 17 procedures of the practice/organization, 1.5% 7.7% 30.3% 60.4% 1,745 2.5 0.7 the profession, and the jurisdiction Monitor, evaluate, and accurately and sensitively communicate supervisee 18 performance to the supervisee, the 7.5% 9.7% 32.8% 50.0% 1,727 2.3 0.9 organization, and the jurisdiction as needed Create and maintain a supportive environment in which effective 19 6.4% 8.8% 31.3% 53.5% 1,723 2.3 0.9 supervision occurs for trainees and other professionals being supervised Relational Competence Integrate and apply theory, research, professional guidelines, and personal 20 0.7% 6.5% 33.5% 59.3% 1,876 2.5 0.7 understanding about social contexts to work effectively with diverse clients

Pearson VUE Confidential Page 88 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Work effectively with individuals, 21 families, groups, communities, and/or 0.6% 4.4% 24.7% 70.2% 1,875 2.6 0.6 organizations In all areas of professional practice, 22 demonstrate respect for others, 0.6% 3.1% 18.6% 77.7% 1,878 2.7 0.5 including those with differing viewpoints Identify and manage interpersonal 23 1.1% 10.6% 35.3% 53.0% 1,872 2.4 0.7 conflict between self and others Professionalism Identify and observe boundaries of 24 competence in all areas of professional 0.4% 2.5% 20.7% 76.3% 1,928 2.7 0.5 practice Critically evaluate one’s own professional 25 practice through self-reflection and 0.5% 5.2% 32.9% 61.3% 1,926 2.6 0.6 feedback from others Ethical and Legal Practice Demonstrate and promote values and behaviors commensurate with standards 26 0.3% 2.1% 11.9% 85.7% 1,898 2.8 0.4 of practice, including ethics codes, laws, and regulations Accurately represent and document work 27 performed in professional practice and 1.4% 6.0% 26.8% 65.8% 1,895 2.6 0.7 scholarship 28 Implement ethical practice management 0.9% 2.5% 19.7% 76.9% 1,875 2.7 0.6 Establish and maintain a process that 29 0.5% 3.2% 22.3% 74.1% 1,881 2.7 0.6 promotes ethical decision-making Systems Thinking Work effectively within organizations and 30 3.0% 14.6% 38.6% 43.8% 1,942 2.2 0.8 systems Demonstrate interdisciplinary 31 2.3% 10.9% 41.7% 45.1% 1,934 2.3 0.8 collaborations Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 89 Table L3. EPPP Part 2 Competency Scales Importance Ratings Survey Survey Statement Minimally Moderately Very Statement Not important N M SD Text important important important Number Scientific Orientation Select relevant research literature and critically review its assumptions, 1 6.3% 23.4% 39.8% 30.5% 1,852 1.9 0.9 conceptualization, methodology, interpretation, and generalizability Interpret, evaluate, and integrate results of assessment activities within the context of scientific/professional 2 5.3% 14.8% 31.5% 48.4% 1,840 2.2 0.9 knowledge to formulate and reformulate working hypotheses, conceptualizations, and recommendations Articulate a rationale for decisions and psychological services that rely on 3 5.8% 18.7% 37.5% 38.0% 1,838 2.1 0.9 objective supporting data (e.g., research results, base rates, epidemiological data) Acquire and disseminate knowledge in 4 accord with scientific and ethical 5.2% 14.8% 34.5% 45.5% 1,840 2.2 0.9 principles Select and use evidence-based 5 5.9% 12.5% 27.7% 53.9% 1,837 2.3 0.9 assessment methods and instruments Select and use evidence-based 6 4.4% 9.4% 30.6% 55.6% 1,835 2.4 0.8 interventions Professional Practice Apply knowledge of individual and 7 diversity characteristics in assessment 1.4% 6.6% 28.6% 63.4% 1,763 2.5 0.7 and diagnosis 8 Demonstrate effective interviewing skills 0.7% 2.2% 17.4% 79.7% 1,763 2.8 0.5 Administer and score instruments 9 8.0% 13.7% 24.9% 53.4% 1,754 2.2 1.0 following current guidelines and research Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, 10 3.5% 9.6% 25.0% 61.9% 1,758 2.5 0.8 interviewees, collateral sources of information) following current guidelines and research

Pearson VUE Confidential Page 90 Survey Survey Statement Minimally Moderately Very Statement Not important N M SD Text important important important Number Formulate diagnoses, recommendations, and/or professional opinions using 11 i.1% 2.9% 19.5% 76.5% 1,758 2.7 0.6 relevant criteria and considering all assessment data Communicate assessment results to 12 clients, referral sources, and other 3.7% 9.9% 25.0% 61.3% 1,754 2.4 0.8 professionals in an integrative manner Evaluate service or program 13 13.7% 26.6% 37.4% 22.2% 1,739 1.7 1.0 effectiveness across a variety of contexts Select interventions for clients based on ongoing assessment and research 14 3.0% 7.9% 32.1% 57.0% 1,749 2.4 0.8 evidence as well as contextual and diversity factors Apply and modify interventions based on ongoing assessment, research, 15 contextual factors, client characteristics, 3.1% 7.6% 32.3% 56.9% 1,749 2.4 0.8 and situational and environmental variables Consult and collaborate within and 16 1.1% 8.3% 36.6% 53.9% 1,749 2.4 0.7 across professions Ensure compliance with policies and 17 procedures of the practice/organization, 1.3% 8.4% 30.6% 59.7% 1,746 2.5 0.7 the profession, and the jurisdiction Monitor, evaluate, and accurately and sensitively communicate supervisee 18 performance to the supervisee, the 15.4% 11.0% 26.7% 46.9% 1,726 2.1 1.1 organization, and the jurisdiction as needed Create and maintain a supportive environment in which effective 19 14.5% 8.0% 24.9% 52.7% 1,722 2.2 1.1 supervision occurs for trainees and other professionals being supervised Relational Competence Integrate and apply theory, research, professional guidelines, and personal 20 1.5% 8.3% 32.2% 58.0% 1,873 2.5 0.7 understanding about social contexts to work effectively with diverse clients

Pearson VUE Confidential Page 91 Survey Survey Statement Minimally Moderately Very Statement Not important N M SD Text important important important Number Work effectively with individuals, 21 families, groups, communities, and/or 1.1% 4.0% 21.7% 73.2% 1,872 2.7 0.6 organizations In all areas of professional practice, 22 demonstrate respect for others, 0.3% 1.9% 15.6% 82.2% 1,875 2.8 0.5 including those with differing viewpoints Identify and manage interpersonal 23 1.8% 10.1% 32.2% 56.0% 1,869 2.4 0.7 conflict between self and others Professionalism Identify and observe boundaries of 24 competence in all areas of professional 0.7% 4.1% 24.5% 70.6% 1,929 2.7 0.6 practice Critically evaluate one’s own professional 25 practice through self-reflection and 0.6% 5.0% 28.4% 66.0% 1,925 2.6 0.6 feedback from others Ethical and Legal Practice Demonstrate and promote values and behaviors commensurate with standards 26 0.3% 2.4% 13.3% 84.0% 1,896 2.8 0.5 of practice, including ethics codes, laws, and regulations Accurately represent and document work 27 performed in professional practice and 1.4% 4.5% 22.7% 71.4% 1,894 2.6 0.6 scholarship 28 Implement ethical practice management 1.7% 3.8% 18.8% 75.7% 1,875 2.7 0.6 Establish and maintain a process that 29 0.7% 4.6% 21.0% 73.6% 1,883 2.7 0.6 promotes ethical decision-making Systems Thinking Work effectively within organizations and 30 3.3% 9.5% 32.3% 54.9% 1,943 2.4 0.8 systems Demonstrate interdisciplinary 31 2.2% 8.7% 35.1% 54.0% 1,933 2.4 0.7 collaborations Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 92 Table L4. EPPP Part 2 Competency Scales Acquisition Ratings Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Scientific Orientation Select relevant research literature and critically review its assumptions, 1 16.0% 84.0% 1,810 0.8 0.4 conceptualization, methodology, interpretation, and generalizability Interpret, evaluate, and integrate results of assessment activities within the context of scientific/professional 2 24.7% 75.3% 1,802 0.8 0.4 knowledge to formulate and reformulate working hypotheses, conceptualizations, and recommendations Articulate a rationale for decisions and psychological services that rely on 3 33.4% 66.6% 1,796 0.7 0.5 objective supporting data (e.g., research results, base rates, epidemiological data) Acquire and disseminate knowledge in 4 accord with scientific and ethical 26.8% 73.2% 1,796 0.7 0.4 principles Select and use evidence-based 5 22.9% 77.1% 1,797 0.8 0.4 assessment methods and instruments Select and use evidence-based 6 29.2% 70.8% 1,797 0.7 0.5 interventions Professional Practice Apply knowledge of individual and 7 diversity characteristics in assessment 26.0% 74.0% 1,720 0.7 0.4 and diagnosis 8 Demonstrate effective interviewing skills 17.7% 82.3% 1,720 0.8 0.4 Administer and score instruments 9 13.6% 86.4% 1,705 0.9 0.3 following current guidelines and research Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, 10 22.9% 77.1% 1,709 0.8 0.4 interviewees, collateral sources of information) following current guidelines and research

Pearson VUE Confidential Page 93 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Formulate diagnoses, recommendations, and/or professional opinions using 11 24.0% 76.0% 1,711 0.8 0.4 relevant criteria and considering all assessment data Communicate assessment results to 12 clients, referral sources, and other 28.2% 71.8% 1,709 0.7 0.5 professionals in an integrative manner Evaluate service or program 13 61.1% 38.9% 1,674 0.4 0.5 effectiveness across a variety of contexts Select interventions for clients based on ongoing assessment and research 14 34.0% 66.0% 1,702 0.7 0.5 evidence as well as contextual and diversity factors Apply and modify interventions based on ongoing assessment, research, 15 contextual factors, client characteristics, 39.2% 60.8% 1,699 0.6 0.5 and situational and environmental variables Consult and collaborate within and 16 38.3% 61.7% 1,701 0.6 0.5 across professions Ensure compliance with policies and 17 procedures of the practice/organization, 42.6% 57.4% 1,695 0.6 0.5 the profession, and the jurisdiction Monitor, evaluate, and accurately and sensitively communicate supervisee 18 performance to the supervisee, the 73.6% 26.4% 1,666 0.3 0.4 organization, and the jurisdiction as needed Create and maintain a supportive environment in which effective 19 73.9% 26.1% 1,659 0.3 0.4 supervision occurs for trainees and other professionals being supervised Relational Competence Integrate and apply theory, research, professional guidelines, and personal 20 32.1% 67.9% 1,825 0.7 0.5 understanding about social contexts to work effectively with diverse clients

Pearson VUE Confidential Page 94 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Work effectively with individuals, 21 families, groups, communities, and/or 29.2% 70.8% 1,825 0.7 0.5 organizations In all areas of professional practice, 22 demonstrate respect for others, 16.1% 83.9% 1,830 0.8 0.4 including those with differing viewpoints Identify and manage interpersonal 23 28.6% 71.4% 1,822 0.7 0.5 conflict between self and others Professionalism Identify and observe boundaries of 24 competence in all areas of professional 30.6% 69.4% 1,884 0.7 0.5 practice Critically evaluate one’s own professional 25 practice through self-reflection and 33.2% 66.8% 1,884 0.7 0.5 feedback from others Ethical and Legal Practice Demonstrate and promote values and behaviors commensurate with standards 26 17.5% 82.5% 1,850 0.8 0.4 of practice, including ethics codes, laws, and regulations Accurately represent and document work 27 performed in professional practice and 20.4% 79.6% 1,847 0.8 0.4 scholarship 28 Implement ethical practice management 36.0% 64.0% 1,827 0.6 0.5 Establish and maintain a process that 29 35.8% 64.2% 1,838 0.6 0.5 promotes ethical decision-making Systems Thinking Work effectively within organizations and 30 40.8% 59.2% 1,895 0.6 0.5 systems Demonstrate interdisciplinary 31 36.4% 63.6% 1,890 0.6 0.5 collaborations Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 95 Appendix M. Responses to the Behavioral Exemplar Scales

Table M1. EPPP Part 2 Behavioral Exemplar Scales Frequency Ratings Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Scientific Orientation Critically evaluate and apply research 1 findings to practice, with attention to its 12.0% 35.0% 36.8% 16.1% 1,935 1.6 0.9 applicability and generalizability Interpret and communicate empirical 2 research results in a manner that is easily 19.6% 32.3% 30.8% 17.3% 1,932 1.5 1.0 understood by non-scientific audiences Identify and collect applicable assessment 3 data and integrate with theoretical models 19.0% 24.9% 33.1% 23.1% 1,930 1.6 1.0 to develop working hypotheses Synthesize client-specific and scientific data with contextual factors to refine 4 working hypotheses and develop 13.0% 18.2% 38.8% 30.0% 1,921 1.9 1.0 conclusions and recommendations across a range of problems Reformulate working hypotheses and 5 10.2% 25.7% 40.9% 23.3% 1,919 1.8 0.9 recommendations based on emerging data Articulate evidence-based rationale for 6 decisions, recommendations, and opinions 8.0% 22.6% 42.9% 26.5% 1,918 1.9 0.9 to clients, professionals, and the public Make and implement decisions and recommendations that integrate client- 7 6.1% 13.6% 43.5% 36.9% 1,914 2.1 0.9 specific, social/contextual, and scientific data Identify areas where biases and heuristics 8 9.4% 33.8% 40.0% 16.8% 1,911 1.6 0.9 may interfere with effective services Critically evaluate the literature relevant to 9 10.1% 37.3% 36.7% 15.9% 1,909 1.6 0.9 professional practice Share psychological knowledge with diverse groups (e.g., students, colleagues, 10 clients, other professionals, the public) 8.9% 26.4% 38.2% 26.4% 1,914 1.8 0.9 within professional settings in an unbiased manner Select tests based upon the constructs to 11 be assessed, the psychometric properties 22.8% 23.0% 26.1% 28.2% 1,907 1.6 1.1 of the tests, and the client population

Pearson VUE Confidential Page 96 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Integrate and analyze client needs, practical constraints, and assessment 12 17.1% 22.0% 32.7% 28.2% 1,906 1.7 1.1 questions into the selection of assessment methods Conceptualize interventions for client 13 presentations guided by research and 9.4% 18.4% 41.9% 30.3% 1,907 1.9 0.9 theory Regularly evaluate effectiveness of 14 7.2% 17.3% 43.3% 32.2% 1,911 2.0 0.9 interventions Professional Practice Integrate knowledge of client characteristics in formulating assessment 15 8.0% 13.1% 37.3% 41.6% 1,821 2.1 0.9 questions and understanding the reason for assessment Select assessment methods and instruments based on available normed 16 data and/or criterion-reference standards, 19.9% 25.3% 28.5% 26.2% 1,818 1.6 1.1 and address any limitations in that selection Ensure that professional opinions, recommendations, and case formulations 17 2.3% 8.1% 38.8% 50.8% 1,820 2.4 0.7 adequately reflect consideration of client characteristics Adapt interview questions and behaviors in 18 light of the characteristics of the 2.9% 11.6% 39.3% 46.2% 1,817 2.3 0.8 interviewer and interviewee Demonstrate flexible, empathic, and 19 accurate use of a broad range of interview 1.8% 7.6% 35.5% 55.1% 1,812 2.4 0.7 techniques Consider contextual information (e.g., reason for assessment, possible legal or 20 5.2% 14.5% 35.3% 45.0% 1,809 2.2 0.9 forensic considerations) in conducting an interview Administer, score, and interpret a range of 21 commonly used standardized assessment 22.1% 22.6% 23.4% 31.9% 1,807 1.7 1.1 methods and instruments Adapt relevant guidelines in situations requiring non-standard administration, 22 33.6% 36.8% 19.7% 9.9% 1,803 1.1 1.0 scoring, interpretation, or communication of assessment results

Pearson VUE Confidential Page 97 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Interpret and integrate results from standardized tests and interviews following 23 25.0% 23.1% 27.5% 24.4% 1,798 1.5 1.1 established guidelines and, as appropriate, multiple applicable norm sets Identify the strengths and limitations of 24 20.5% 27.9% 32.3% 19.3% 1,796 1.5 1.0 various types of assessment data Reconcile or explain discrepancies between various sources of data and suggest 25 alternative interpretations or explanations 20.6% 28.7% 32.5% 18.2% 1,796 1.5 1.0 in light of any limitations of assessment instruments Use multi-modal assessment data and 26 theory to formulate an understanding of 15.4% 21.4% 35.4% 27.9% 1,797 1.8 1.0 the client Formulate diagnoses using current 27 6.0% 10.4% 34.0% 49.6% 1,786 2.3 0.9 taxonomies Provide recommendations that incorporate 28 2.4% 6.5% 40.3% 50.9% 1,793 2.4 0.7 client and contextual factors Write assessment reports in a timely and 29 18.9% 14.7% 27.1% 39.3% 1,787 1.9 1.1 understandable manner Verbally communicate results from 30 assessments in a timely and 16.7% 20.1% 32.1% 31.2% 1,784 1.8 1.1 understandable manner Develop plans for evaluating service or 31 38.3% 36.6% 16.7% 8.4% 1,785 1.0 0.9 program effectiveness Assess outcome effectiveness in an 32 17.4% 28.9% 34.3% 19.4% 1,774 1.6 1.0 ongoing way Conceptualize intervention or treatment on 33 7.5% 16.5% 44.9% 31.0% 1,779 2.0 0.9 the basis of evidenced-based literature Integrate client/stakeholder opinions, preferences, readiness for change, and 34 8.1% 13.8% 42.7% 35.3% 1,778 2.1 0.9 potential for improvement into intervention plan Continually evaluate, modify, and assess 35 the effectiveness of interventions, 7.7% 14.2% 43.3% 34.8% 1,772 2.1 0.9 considering all relevant variables Consult with qualified peers when facing 36 the need to modify interventions in 11.2% 40.0% 33.3% 15.5% 1,773 1.5 0.9 unfamiliar situations

Pearson VUE Confidential Page 98 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Tailor consultation requests and provision 37 of information based on knowledge of 12.2% 34.3% 37.2% 16.4% 1,765 1.6 0.9 others’ professional needs and viewpoints Use evidence-based psychological theories, 38 decision-making strategies, and 12.2% 23.0% 40.8% 24.0% 1,773 1.8 1.0 interventions when consulting Continually evaluate, modify, and assess 39 the effectiveness of consultation, 20.4% 31.6% 32.7% 15.3% 1,768 1.4 1.0 considering all relevant variables Provide a supervision plan that details the supervisory relationship and the policies 40 40.7% 24.8% 21.4% 13.1% 1,760 1.1 1.1 and procedures of supervision, including procedures to manage high-risk situations Identify responsibilities of supervisees 41 towards clients, including informed consent 36.6% 21.1% 22.9% 19.4% 1,761 1.3 1.1 and supervisory status Regularly provide behaviorally anchored 42 feedback about supervisee strengths and 37.1% 20.0% 27.8% 15.1% 1,762 1.2 1.1 areas that need further development Assure that supervisees who are trainees practice within the scope of supervisor’s 43 37.5% 14.5% 24.3% 23.6% 1,761 1.3 1.2 competence and license and promote professional development Attend to the interpersonal process 44 34.7% 18.1% 27.4% 19.9% 1,760 1.3 1.1 between supervisor and supervisees Monitor possible multiple roles or conflicts 45 of interest and work toward resolution if 17.8% 35.3% 30.9% 16.1% 1,772 1.5 1.0 required Relational Competence Recognize, understand, and monitor the 46 impact of one’s own identities in 1.8% 14.7% 48.3% 35.2% 1,958 2.2 0.7 professional situations Engage in respectful interactions with an 47 awareness of individual, community, and 1.0% 7.7% 38.4% 52.9% 1,956 2.4 0.7 organizational differences Modify one’s own behavior based on self- reflection and an understanding of the 48 2.1% 21.3% 45.0% 31.6% 1,956 2.1 0.8 impact of social, cultural, and organizational contexts

Pearson VUE Confidential Page 99 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Follow professional guidelines and the scientific literature, when available, for 49 3.1% 16.4% 46.0% 34.6% 1,952 2.1 0.8 providing professional services to diverse populations Apply culturally appropriate skills, 50 techniques, and behaviors with an 1.7% 12.5% 42.7% 43.2% 1,957 2.3 0.7 appreciation of individual differences Use relational skills to engage, establish 51 and maintain working relationships with a 1.2% 2.7% 20.3% 75.8% 1,953 2.7 0.6 range of clients Communicate respectfully, showing 52 0.2% 0.3% 11.2% 88.4% 1,952 2.9 0.3 empathy for others Collaborate effectively in professional 53 0.7% 7.7% 37.5% 54.2% 1,941 2.5 0.7 interactions Consider differing viewpoints held by 54 0.3% 3.8% 37.6% 58.3% 1,943 2.5 0.6 clients and others Respond to differing viewpoints by seeking 55 clarification to increase understanding 1.0% 12.8% 46.7% 39.5% 1,948 2.2 0.7 before taking action Manage difficult and complex interpersonal 56 6.1% 31.7% 36.2% 26.0% 1,943 1.8 0.9 relationships between self and others Seek experienced peers’ feedback to examine, and modify if necessary, one’s 57 8.7% 44.4% 33.0% 13.9% 1,951 1.5 0.8 own reactions and behavior when managing interpersonal conflict Professionalism Identify limits of professional competence 58 by recognizing strengths and weaknesses 0.8% 15.4% 52.9% 30.9% 2,037 2.1 0.7 in practice areas Use knowledge of personal strengths and 59 1.1% 11.5% 50.0% 37.3% 2,035 2.2 0.7 weaknesses to guide scope of practice Seek appropriate consultation when unsure 60 about one’s competence and additional 4.2% 40.8% 36.6% 18.4% 2,036 1.7 0.8 needs for training and development Seek additional knowledge, training, and 61 supervision when expanding scope of 11.5% 34.3% 35.3% 18.8% 2,034 1.6 0.9 practice Update knowledge and skills relevant to 62 0.7% 15.9% 54.6% 28.8% 2,034 2.1 0.7 psychological practice on an ongoing basis Engage in systematic and ongoing self- 63 4.6% 30.2% 45.4% 19.8% 2,032 1.8 0.8 assessment and skills development

Pearson VUE Confidential Page 100 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Accept personal responsibility when 64 mistakes or oversights occur and take 6.6% 34.0% 35.9% 23.5% 2,032 1.8 0.9 appropriate corrective action if needed Maintain awareness of personal factors that 65 1.7% 13.5% 45.6% 39.3% 2,037 2.2 0.7 may impact professional functioning Ethical and Legal Practice Demonstrate integration and application of 66 ethics codes and laws in all professional 1.1% 7.5% 35.0% 56.5% 1,968 2.5 0.7 interactions Communicate ethical and legal standards in 67 2.8% 27.1% 40.8% 29.3% 1,970 2.0 0.8 professional interactions Seek professional consultation on ethical or 68 8.3% 45.8% 32.1% 13.8% 1,959 1.5 0.8 legal issues when needed Discuss with peers or organizations any 69 31.4% 48.6% 13.3% 6.7% 1,965 1.0 0.8 ethical concerns with their behavior Take appropriate steps to resolve conflicts 70 between laws/rules and codes of ethics in 29.7% 48.7% 13.2% 8.4% 1,953 1.0 0.9 one’s practice 71 Maintain complete and accurate records 0.7% 1.5% 13.2% 84.6% 1,960 2.8 0.5 Report research results accurately, 72 46.5% 21.2% 14.3% 18.0% 1,933 1.0 1.2 avoiding personal biases Ensure adequate and appropriate credit is 73 given to trainees and collaborators in 52.0% 19.9% 13.6% 14.5% 1,934 0.9 1.1 scholarship Develop a plan commensurate with laws, 74 ethical guidelines, and fiscal constraints to 21.9% 32.1% 26.7% 19.3% 1,942 1.4 1.0 manage professional activities Practice in a fiscally and ethically sound 75 2.8% 4.4% 24.3% 68.5% 1,946 2.6 0.7 manner Systematically identify the ethical and legal 76 issues and conflicts that occur in one’s 7.7% 34.6% 38.5% 19.3% 1,946 1.7 0.9 professional practice Consult with peers to aid in ethical 77 6.9% 37.5% 38.4% 17.3% 1,951 1.7 0.8 decision-making when appropriate Engage in critical analysis of identified 78 10.5% 43.8% 32.5% 13.1% 1,943 1.5 0.9 ethical issues and proactively address them 79 Model ethical decision-making to others 3.5% 14.8% 39.8% 41.9% 1,951 2.2 0.8 Systems Thinking Recognize the organizational and systemic 80 factors that affect delivery of psychological 6.4% 17.7% 42.8% 33.1% 2,040 2.0 0.9 services

Pearson VUE Confidential Page 101 Survey Never or Very statement Survey statement text Infrequently Frequently N M SD very rarely frequently number Utilize knowledge of organizations and 81 systems to optimize delivery of 8.5% 24.7% 40.8% 26.0% 2,038 1.8 0.9 psychological services Collaborate with various healthcare 82 5.4% 20.5% 39.4% 34.7% 2,040 2.0 0.9 professions to meet common client goals With other healthcare professionals, 83 integrate services to meet common client 8.3% 27.3% 35.6% 28.8% 2,040 1.8 0.9 goals Collaborate with representatives from other 84 professional groups or systems in the 14.5% 31.4% 33.8% 20.3% 2,041 1.6 1.0 provision of psychological services Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 102 Table M2. EPPP Part 2 Behavioral Exemplar Scales Criticality Ratings Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Scientific Orientation Critically evaluate and apply research 1 findings to practice, with attention to its 4.6% 20.5% 43.4% 31.5% 1,833 2.0 0.8 applicability and generalizability Interpret and communicate empirical 2 research results in a manner that is easily 9.2% 22.3% 40.0% 28.5% 1,829 1.9 0.9 understood by non-scientific audiences Identify and collect applicable assessment 3 data and integrate with theoretical models 7.8% 18.0% 39.6% 34.6% 1,825 2.0 0.9 to develop working hypotheses Synthesize client-specific and scientific data with contextual factors to refine 4 working hypotheses and develop 5.8% 13.7% 37.6% 43.0% 1,818 2.2 0.9 conclusions and recommendations across a range of problems Reformulate working hypotheses and 5 4.4% 10.8% 40.4% 44.5% 1,819 2.2 0.8 recommendations based on emerging data Articulate evidence-based rationale for 6 decisions, recommendations, and opinions 3.8% 16.4% 38.4% 41.4% 1,821 2.2 0.8 to clients, professionals, and the public Make and implement decisions and recommendations that integrate client- 7 2.0% 9.7% 37.9% 50.4% 1,815 2.4 0.7 specific, social/contextual, and scientific data Identify areas where biases and heuristics 8 3.0% 13.6% 39.1% 44.3% 1,815 2.2 0.8 may interfere with effective services Critically evaluate the literature relevant to 9 4.8% 20.5% 42.5% 32.2% 1,813 2.0 0.8 professional practice Share psychological knowledge with diverse groups (e.g., students, colleagues, 10 clients, other professionals, the public) 5.7% 20.3% 38.7% 35.3% 1,816 2.0 0.9 within professional settings in an unbiased manner Select tests based upon the constructs to 11 be assessed, the psychometric properties 6.7% 9.1% 30.3% 53.9% 1,810 2.3 0.9 of the tests, and the client population

Pearson VUE Confidential Page 103 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Integrate and analyze client needs, practical constraints, and assessment 12 5.0% 11.7% 36.6% 46.7% 1,807 2.2 0.9 questions into the selection of assessment methods Conceptualize interventions for client 13 presentations guided by research and 3.8% 11.5% 40.8% 44.0% 1,811 2.2 0.8 theory Regularly evaluate effectiveness of 14 2.4% 5.5% 36.2% 55.8% 1,818 2.5 0.7 interventions Professional Practice Integrate knowledge of client characteristics in formulating assessment 15 2.0% 5.8% 32.8% 59.4% 1,728 2.5 0.7 questions and understanding the reason for assessment Select assessment methods and instruments based on available normed 16 data and/or criterion-reference standards, 5.5% 9.3% 33.6% 51.5% 1,715 2.3 0.9 and address any limitations in that selection Ensure that professional opinions, recommendations, and case formulations 17 0.3% 3.7% 29.4% 66.5% 1,732 2.6 0.6 adequately reflect consideration of client characteristics Adapt interview questions and behaviors in 18 light of the characteristics of the 1.7% 7.7% 37.8% 52.9% 1,724 2.4 0.7 interviewer and interviewee Demonstrate flexible, empathic, and 19 accurate use of a broad range of interview 0.7% 8.0% 33.3% 58.0% 1,723 2.5 0.7 techniques Consider contextual information (e.g., reason for assessment, possible legal or 20 1.6% 6.1% 34.5% 57.8% 1,718 2.5 0.7 forensic considerations) in conducting an interview Administer, score, and interpret a range of 21 commonly used standardized assessment 6.4% 11.6% 31.9% 50.0% 1,709 2.3 0.9 methods and instruments Adapt relevant guidelines in situations requiring non-standard administration, 22 8.5% 18.9% 39.5% 33.1% 1,697 2.0 0.9 scoring, interpretation, or communication of assessment results

Pearson VUE Confidential Page 104 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Interpret and integrate results from standardized tests and interviews following 23 5.8% 11.3% 33.9% 49.0% 1,693 2.3 0.9 established guidelines and, as appropriate, multiple applicable norm sets Identify the strengths and limitations of 24 4.9% 14.2% 37.2% 43.7% 1,694 2.2 0.9 various types of assessment data Reconcile or explain discrepancies between various sources of data and suggest 25 alternative interpretations or explanations 5.4% 11.3% 36.7% 46.6% 1,692 2.2 0.9 in light of any limitations of assessment instruments Use multi-modal assessment data and 26 theory to formulate an understanding of 5.2% 12.7% 39.2% 43.0% 1,698 2.2 0.9 the client Formulate diagnoses using current 27 3.1% 10.5% 34.5% 51.9% 1,702 2.4 0.8 taxonomies Provide recommendations that incorporate 28 0.6% 3.6% 33.5% 62.3% 1,706 2.6 0.6 client and contextual factors Write assessment reports in a timely and 29 3.7% 7.7% 31.2% 57.4% 1,694 2.4 0.8 understandable manner Verbally communicate results from 30 assessments in a timely and 3.7% 8.6% 32.7% 55.0% 1,688 2.4 0.8 understandable manner Develop plans for evaluating service or 31 11.5% 25.3% 43.6% 19.6% 1,678 1.7 0.9 program effectiveness Assess outcome effectiveness in an 32 5.1% 15.5% 47.0% 32.4% 1,680 2.1 0.8 ongoing way Conceptualize intervention or treatment on 33 3.3% 11.8% 38.6% 46.3% 1,691 2.3 0.8 the basis of evidenced-based literature Integrate client/stakeholder opinions, preferences, readiness for change, and 34 2.4% 9.7% 39.3% 48.5% 1,688 2.3 0.8 potential for improvement into intervention plan Continually evaluate, modify, and assess 35 the effectiveness of interventions, 1.7% 7.9% 40.0% 50.4% 1,686 2.4 0.7 considering all relevant variables Consult with qualified peers when facing 36 the need to modify interventions in 1.8% 8.6% 38.4% 51.2% 1,681 2.4 0.7 unfamiliar situations

Pearson VUE Confidential Page 105 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Tailor consultation requests and provision 37 of information based on knowledge of 5.9% 23.0% 45.0% 26.1% 1,667 1.9 0.8 others’ professional needs and viewpoints Use evidence-based psychological theories, 38 decision-making strategies, and 4.4% 17.7% 42.1% 35.9% 1,678 2.1 0.8 interventions when consulting Continually evaluate, modify, and assess 39 the effectiveness of consultation, 6.4% 23.1% 43.5% 26.9% 1,668 1.9 0.9 considering all relevant variables Provide a supervision plan that details the supervisory relationship and the policies 40 9.0% 9.4% 30.8% 50.9% 1,652 2.2 1.0 and procedures of supervision, including procedures to manage high-risk situations Identify responsibilities of supervisees 41 towards clients, including informed consent 7.1% 7.1% 26.7% 59.1% 1,651 2.4 0.9 and supervisory status Regularly provide behaviorally anchored 42 feedback about supervisee strengths and 7.8% 11.1% 34.5% 46.6% 1,650 2.2 0.9 areas that need further development Assure that supervisees who are trainees practice within the scope of supervisor’s 43 6.3% 4.4% 21.3% 67.9% 1,646 2.5 0.8 competence and license and promote professional development Attend to the interpersonal process 44 8.1% 15.6% 37.7% 38.6% 1,650 2.1 0.9 between supervisor and supervisees Monitor possible multiple roles or conflicts 45 of interest and work toward resolution if 3.8% 11.8% 34.3% 50.1% 1,672 2.3 0.8 required Relational Competence Recognize, understand, and monitor the 46 impact of one’s own identities in 1.7% 11.0% 39.8% 47.5% 1,875 2.3 0.7 professional situations Engage in respectful interactions with an 47 awareness of individual, community, and 1.0% 7.6% 33.2% 58.2% 1,877 2.5 0.7 organizational differences Modify one’s own behavior based on self- reflection and an understanding of the 48 1.0% 9.0% 36.4% 53.7% 1,875 2.4 0.7 impact of social, cultural, and organizational contexts

Pearson VUE Confidential Page 106 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Follow professional guidelines and the scientific literature, when available, for 49 1.0% 7.1% 33.5% 58.4% 1,876 2.5 0.7 providing professional services to diverse populations Apply culturally appropriate skills, 50 techniques, and behaviors with an 0.5% 5.4% 31.1% 63.0% 1,877 2.6 0.6 appreciation of individual differences Use relational skills to engage, establish 51 and maintain working relationships with a 0.3% 3.4% 19.9% 76.5% 1,873 2.7 0.5 range of clients Communicate respectfully, showing 52 0.2% 1.9% 13.1% 84.9% 1,875 2.8 0.4 empathy for others Collaborate effectively in professional 53 1.4% 7.7% 38.4% 52.5% 1,866 2.4 0.7 interactions Consider differing viewpoints held by 54 0.4% 5.4% 30.2% 64.0% 1,866 2.6 0.6 clients and others Respond to differing viewpoints by seeking 55 clarification to increase understanding 0.9% 7.6% 34.5% 57.0% 1,867 2.5 0.7 before taking action Manage difficult and complex interpersonal 56 1.4% 9.3% 35.2% 54.2% 1,859 2.4 0.7 relationships between self and others Seek experienced peers’ feedback to examine, and modify if necessary, one’s 57 2.4% 11.9% 40.3% 45.4% 1,868 2.3 0.8 own reactions and behavior when managing interpersonal conflict Professionalism Identify limits of professional competence 58 by recognizing strengths and weaknesses 0.5% 3.5% 28.9% 67.2% 1,946 2.6 0.6 in practice areas Use knowledge of personal strengths and 59 0.4% 5.6% 34.5% 59.5% 1,948 2.5 0.6 weaknesses to guide scope of practice Seek appropriate consultation when unsure 60 about one’s competence and additional 0.5% 5.0% 29.3% 65.2% 1,945 2.6 0.6 needs for training and development Seek additional knowledge, training, and 61 supervision when expanding scope of 1.6% 4.1% 27.0% 67.3% 1,944 2.6 0.6 practice Update knowledge and skills relevant to 62 0.5% 4.8% 36.6% 58.1% 1,950 2.5 0.6 psychological practice on an ongoing basis Engage in systematic and ongoing self- 63 1.1% 13.7% 44.3% 40.8% 1,942 2.2 0.7 assessment and skills development

Pearson VUE Confidential Page 107 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Accept personal responsibility when 64 mistakes or oversights occur and take 0.4% 3.5% 23.2% 72.9% 1,948 2.7 0.6 appropriate corrective action if needed Maintain awareness of personal factors that 65 0.2% 4.4% 27.8% 67.6% 1,945 2.6 0.6 may impact professional functioning Ethical and Legal Practice Demonstrate integration and application of 66 ethics codes and laws in all professional 0.3% 3.5% 22.1% 74.0% 1,884 2.7 0.5 interactions Communicate ethical and legal standards in 67 1.0% 9.1% 33.5% 56.4% 1,883 2.5 0.7 professional interactions Seek professional consultation on ethical or 68 0.6% 4.4% 25.0% 69.9% 1,882 2.6 0.6 legal issues when needed Discuss with peers or organizations any 69 1.8% 8.7% 33.7% 55.8% 1,872 2.4 0.7 ethical concerns with their behavior Take appropriate steps to resolve conflicts 70 between laws/rules and codes of ethics in 1.9% 9.2% 28.2% 60.7% 1,864 2.5 0.7 one’s practice 71 Maintain complete and accurate records 0.7% 3.6% 20.7% 75.0% 1,877 2.7 0.6 Report research results accurately, 72 9.0% 9.8% 24.9% 56.3% 1,839 2.3 1.0 avoiding personal biases Ensure adequate and appropriate credit is 73 given to trainees and collaborators in 14.4% 15.3% 23.7% 46.7% 1,830 2.0 1.1 scholarship Develop a plan commensurate with laws, 74 ethical guidelines, and fiscal constraints to 7.0% 16.7% 36.2% 40.1% 1,854 2.1 0.9 manage professional activities Practice in a fiscally and ethically sound 75 1.3% 5.5% 22.2% 70.9% 1,864 2.6 0.7 manner Systematically identify the ethical and legal 76 issues and conflicts that occur in one’s 1.6% 8.3% 30.7% 59.4% 1,859 2.5 0.7 professional practice Consult with peers to aid in ethical 77 1.0% 5.4% 29.5% 64.1% 1,864 2.6 0.6 decision-making when appropriate Engage in critical analysis of identified 78 1.7% 7.8% 30.1% 60.3% 1,863 2.5 0.7 ethical issues and proactively address them 79 Model ethical decision-making to others 1.3% 7.8% 28.1% 62.8% 1,864 2.5 0.7 Systems Thinking Recognize the organizational and systemic 80 factors that affect delivery of psychological 3.1% 17.4% 44.4% 35.1% 1,947 2.1 0.8 services

Pearson VUE Confidential Page 108 Survey Minimally Moderately statement Survey statement text Not critical Highly critical N M SD critical critical number Utilize knowledge of organizations and 81 systems to optimize delivery of 3.8% 20.8% 45.6% 29.9% 1,945 2.0 0.8 psychological services Collaborate with various healthcare 82 1.3% 7.9% 39.9% 50.9% 1,950 2.4 0.7 professions to meet common client goals With other healthcare professionals, 83 integrate services to meet common client 1.6% 11.6% 42.2% 44.6% 1,947 2.3 0.7 goals Collaborate with representatives from other 84 professional groups or systems in the 5.3% 21.1% 44.3% 29.3% 1,945 2.0 0.8 provision of psychological services Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 109 Table M3. EPPP Part 2 Behavioral Exemplar Scales Importance Ratings Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Scientific Orientation Critically evaluate and apply research 1 findings to practice, with attention to its 5.7% 22.1% 39.7% 32.6% 1,831 2.0 0.9 applicability and generalizability Interpret and communicate empirical 2 research results in a manner that is easily 9.6% 22.8% 36.7% 30.9% 1,830 1.9 1.0 understood by non-scientific audiences Identify and collect applicable assessment 3 data and integrate with theoretical models 10.1% 18.7% 35.9% 35.2% 1,825 2.0 1.0 to develop working hypotheses Synthesize client-specific and scientific data with contextual factors to refine 4 working hypotheses and develop 7.7% 13.1% 34.9% 44.3% 1,818 2.2 0.9 conclusions and recommendations across a range of problems Reformulate working hypotheses and 5 6.0% 12.1% 37.9% 43.9% 1,821 2.2 0.9 recommendations based on emerging data Articulate evidence-based rationale for 6 decisions, recommendations, and opinions 4.6% 14.7% 38.6% 42.2% 1,822 2.2 0.8 to clients, professionals, and the public Make and implement decisions and recommendations that integrate client- 7 2.9% 10.1% 37.6% 49.4% 1,814 2.3 0.8 specific, social/contextual, and scientific data Identify areas where biases and heuristics 8 4.9% 16.6% 38.2% 40.3% 1,813 2.1 0.9 may interfere with effective services Critically evaluate the literature relevant to 9 4.9% 21.2% 41.2% 32.7% 1,813 2.0 0.9 professional practice Share psychological knowledge with diverse groups (e.g., students, colleagues, 10 clients, other professionals, the public) 4.9% 17.3% 38.6% 39.2% 1,815 2.1 0.9 within professional settings in an unbiased manner Select tests based upon the constructs to 11 be assessed, the psychometric properties 10.2% 14.0% 26.3% 49.6% 1,808 2.2 1.0 of the tests, and the client population

Pearson VUE Confidential Page 110 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Integrate and analyze client needs, practical constraints, and assessment 12 8.1% 15.3% 31.8% 44.8% 1,807 2.1 1.0 questions into the selection of assessment methods Conceptualize interventions for client 13 presentations guided by research and 5.8% 12.1% 37.8% 44.3% 1,811 2.2 0.9 theory Regularly evaluate effectiveness of 14 4.2% 8.0% 32.7% 55.0% 1,815 2.4 0.8 interventions Professional Practice Integrate knowledge of client characteristics in formulating assessment 15 4.4% 8.2% 29.2% 58.1% 1,727 2.4 0.8 questions and understanding the reason for assessment Select assessment methods and instruments based on available normed 16 data and/or criterion-reference standards, 10.0% 14.9% 27.7% 47.4% 1,715 2.1 1.0 and address any limitations in that selection Ensure that professional opinions, recommendations, and case formulations 17 1.2% 4.2% 29.4% 65.3% 1,730 2.6 0.6 adequately reflect consideration of client characteristics Adapt interview questions and behaviors in 18 light of the characteristics of the 2.4% 7.3% 33.4% 57.0% 1,724 2.4 0.7 interviewer and interviewee Demonstrate flexible, empathic, and 19 accurate use of a broad range of interview 1.3% 7.0% 30.5% 61.1% 1,722 2.5 0.7 techniques Consider contextual information (e.g., reason for assessment, possible legal or 20 2.9% 9.2% 31.3% 56.6% 1,717 2.4 0.8 forensic considerations) in conducting an interview Administer, score, and interpret a range of 21 commonly used standardized assessment 10.6% 15.1% 25.8% 48.5% 1,707 2.1 1.0 methods and instruments Adapt relevant guidelines in situations requiring non-standard administration, 22 14.9% 22.6% 33.0% 29.5% 1,696 1.8 1.0 scoring, interpretation, or communication of assessment results

Pearson VUE Confidential Page 111 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Interpret and integrate results from standardized tests and interviews following 23 12.2% 15.5% 28.5% 43.8% 1,693 2.0 1.0 established guidelines and, as appropriate, multiple applicable norm sets Identify the strengths and limitations of 24 10.0% 16.8% 33.0% 40.2% 1,692 2.0 1.0 various types of assessment data Reconcile or explain discrepancies between various sources of data and suggest 25 alternative interpretations or explanations 10.2% 15.3% 31.5% 43.1% 1,691 2.1 1.0 in light of any limitations of assessment instruments Use multi-modal assessment data and 26 theory to formulate an understanding of 8.2% 16.5% 31.9% 43.3% 1,698 2.1 1.0 the client Formulate diagnoses using current 27 4.3% 11.8% 31.4% 52.6% 1,702 2.3 0.8 taxonomies Provide recommendations that incorporate 28 1.5% 4.3% 30.9% 63.3% 1,705 2.6 0.6 client and contextual factors Write assessment reports in a timely and 29 9.0% 9.0% 22.0% 60.0% 1,693 2.3 1.0 understandable manner Verbally communicate results from 30 assessments in a timely and 8.8% 11.0% 26.7% 53.6% 1,688 2.2 1.0 understandable manner Develop plans for evaluating service or 31 17.6% 26.7% 35.5% 20.1% 1,678 1.6 1.0 program effectiveness Assess outcome effectiveness in an 32 8.5% 17.6% 39.9% 33.9% 1,680 2.0 0.9 ongoing way Conceptualize intervention or treatment on 33 4.8% 13.0% 36.7% 45.4% 1,690 2.2 0.9 the basis of evidenced-based literature Integrate client/stakeholder opinions, preferences, readiness for change, and 34 4.4% 10.4% 34.8% 50.4% 1,686 2.3 0.8 potential for improvement into intervention plan Continually evaluate, modify, and assess 35 the effectiveness of interventions, 4.2% 8.5% 36.6% 50.6% 1,685 2.3 0.8 considering all relevant variables Consult with qualified peers when facing 36 the need to modify interventions in 4.8% 13.7% 35.0% 46.5% 1,679 2.2 0.9 unfamiliar situations

Pearson VUE Confidential Page 112 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Tailor consultation requests and provision 37 of information based on knowledge of 7.1% 22.7% 41.6% 28.6% 1,666 1.9 0.9 others’ professional needs and viewpoints Use evidence-based psychological theories, 38 decision-making strategies, and 7.3% 18.3% 38.8% 35.6% 1,679 2.0 0.9 interventions when consulting Continually evaluate, modify, and assess 39 the effectiveness of consultation, 10.1% 23.6% 39.4% 26.9% 1,666 1.8 0.9 considering all relevant variables Provide a supervision plan that details the supervisory relationship and the policies 40 19.5% 12.4% 25.3% 42.9% 1,649 1.9 1.2 and procedures of supervision, including procedures to manage high-risk situations Identify responsibilities of supervisees 41 towards clients, including informed consent 18.0% 9.3% 22.5% 50.2% 1,648 2.1 1.1 and supervisory status Regularly provide behaviorally anchored 42 feedback about supervisee strengths and 18.6% 10.1% 26.9% 44.4% 1,648 2.0 1.1 areas that need further development Assure that supervisees who are trainees practice within the scope of supervisor’s 43 18.6% 7.1% 19.3% 54.9% 1,647 2.1 1.2 competence and license and promote professional development Attend to the interpersonal process 44 17.3% 11.6% 29.9% 41.2% 1,649 1.9 1.1 between supervisor and supervisees Monitor possible multiple roles or conflicts 45 of interest and work toward resolution if 7.1% 14.7% 34.0% 44.2% 1,669 2.2 0.9 required Relational Competence Recognize, understand, and monitor the 46 impact of one’s own identities in 1.2% 10.4% 37.6% 50.8% 1,873 2.4 0.7 professional situations Engage in respectful interactions with an 47 awareness of individual, community, and 0.9% 5.9% 31.3% 62.0% 1,878 2.5 0.6 organizational differences Modify one’s own behavior based on self- reflection and an understanding of the 48 1.0% 9.2% 35.1% 54.7% 1,871 2.4 0.7 impact of social, cultural, and organizational contexts

Pearson VUE Confidential Page 113 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Follow professional guidelines and the scientific literature, when available, for 49 1.6% 9.0% 34.1% 55.3% 1,876 2.4 0.7 providing professional services to diverse populations Apply culturally appropriate skills, 50 techniques, and behaviors with an 0.8% 7.0% 29.9% 62.3% 1,876 2.5 0.7 appreciation of individual differences Use relational skills to engage, establish 51 and maintain working relationships with a 0.7% 2.5% 17.7% 79.1% 1,872 2.8 0.5 range of clients Communicate respectfully, showing 52 0.1% 0.4% 10.6% 88.9% 1,876 2.9 0.3 empathy for others Collaborate effectively in professional 53 0.2% 4.3% 32.8% 62.7% 1,869 2.6 0.6 interactions Consider differing viewpoints held by 54 0.3% 4.4% 29.5% 65.8% 1,870 2.6 0.6 clients and others Respond to differing viewpoints by seeking 55 clarification to increase understanding 0.6% 6.9% 35.1% 57.5% 1,868 2.5 0.7 before taking action Manage difficult and complex interpersonal 56 1.4% 8.6% 34.2% 55.8% 1,864 2.4 0.7 relationships between self and others Seek experienced peers’ feedback to examine, and modify if necessary, one’s 57 2.0% 12.9% 37.4% 47.7% 1,867 2.3 0.8 own reactions and behavior when managing interpersonal conflict Professionalism Identify limits of professional competence 58 by recognizing strengths and weaknesses 0.7% 5.4% 30.3% 63.6% 1,944 2.6 0.6 in practice areas Use knowledge of personal strengths and 59 0.6% 6.2% 35.3% 58.0% 1,946 2.5 0.6 weaknesses to guide scope of practice Seek appropriate consultation when unsure 60 about one’s competence and additional 0.9% 8.2% 30.9% 60.0% 1,944 2.5 0.7 needs for training and development Seek additional knowledge, training, and 61 supervision when expanding scope of 3.7% 9.6% 27.5% 59.2% 1,943 2.4 0.8 practice Update knowledge and skills relevant to 62 0.3% 5.0% 35.7% 59.0% 1,950 2.5 0.6 psychological practice on an ongoing basis Engage in systematic and ongoing self- 63 1.2% 12.4% 43.4% 42.9% 1,943 2.3 0.7 assessment and skills development

Pearson VUE Confidential Page 114 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Accept personal responsibility when 64 mistakes or oversights occur and take 0.7% 6.5% 21.9% 71.0% 1,945 2.6 0.6 appropriate corrective action if needed Maintain awareness of personal factors that 65 0.9% 6.2% 28.0% 64.9% 1,948 2.6 0.6 may impact professional functioning Ethical and Legal Practice Demonstrate integration and application of 66 ethics codes and laws in all professional 0.6% 3.8% 21.2% 74.3% 1,882 2.7 0.6 interactions Communicate ethical and legal standards in 67 1.2% 9.4% 33.1% 56.3% 1,878 2.4 0.7 professional interactions Seek professional consultation on ethical or 68 1.1% 7.9% 24.8% 66.3% 1,879 2.6 0.7 legal issues when needed Discuss with peers or organizations any 69 4.1% 14.9% 31.5% 49.5% 1,870 2.3 0.9 ethical concerns with their behavior Take appropriate steps to resolve conflicts 70 between laws/rules and codes of ethics in 4.7% 13.8% 27.4% 54.0% 1,863 2.3 0.9 one’s practice 71 Maintain complete and accurate records 0.8% 2.2% 15.2% 81.7% 1,882 2.8 0.5 Report research results accurately, 72 19.1% 12.6% 19.9% 48.3% 1,835 2.0 1.2 avoiding personal biases Ensure adequate and appropriate credit is 73 given to trainees and collaborators in 22.6% 11.9% 19.3% 46.2% 1,830 1.9 1.2 scholarship Develop a plan commensurate with laws, 74 ethical guidelines, and fiscal constraints to 9.5% 17.6% 33.8% 39.2% 1,854 2.0 1.0 manage professional activities Practice in a fiscally and ethically sound 75 1.9% 4.5% 22.4% 71.2% 1,865 2.6 0.7 manner Systematically identify the ethical and legal 76 issues and conflicts that occur in one’s 2.6% 11.2% 29.6% 56.5% 1,860 2.4 0.8 professional practice Consult with peers to aid in ethical 77 1.4% 9.0% 29.1% 60.4% 1,863 2.5 0.7 decision-making when appropriate Engage in critical analysis of identified 78 2.4% 12.8% 29.9% 54.9% 1,860 2.4 0.8 ethical issues and proactively address them 79 Model ethical decision-making to others 1.6% 8.2% 26.6% 63.6% 1,863 2.5 0.7 Systems Thinking Recognize the organizational and systemic 80 factors that affect delivery of psychological 4.0% 14.9% 40.9% 40.3% 1,946 2.2 0.8 services

Pearson VUE Confidential Page 115 Survey Minimally Moderately Very statement Survey statement text Not important N M SD important important important number Utilize knowledge of organizations and 81 systems to optimize delivery of 4.7% 18.4% 42.7% 34.2% 1,944 2.1 0.8 psychological services Collaborate with various healthcare 82 1.7% 9.3% 37.0% 52.0% 1,952 2.4 0.7 professions to meet common client goals With other healthcare professionals, 83 integrate services to meet common client 2.5% 13.1% 38.8% 45.6% 1,947 2.3 0.8 goals Collaborate with representatives from other 84 professional groups or systems in the 5.8% 21.7% 40.6% 31.9% 1,944 2.0 0.9 provision of psychological services Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 116 Table M4. EPPP Part 2 Behavioral Exemplar Scales Acquisition Ratings Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Scientific Orientation Critically evaluate and apply research 1 findings to practice, with attention to its 28.1% 71.9% 1,790 0.7 0.4 applicability and generalizability Interpret and communicate empirical research results in a manner that is 2 33.3% 66.7% 1,784 0.7 0.5 easily understood by non-scientific audiences Identify and collect applicable assessment data and integrate with 3 25.9% 74.1% 1,779 0.7 0.4 theoretical models to develop working hypotheses Synthesize client-specific and scientific data with contextual factors to refine 4 working hypotheses and develop 34.6% 65.4% 1,773 0.7 0.5 conclusions and recommendations across a range of problems Reformulate working hypotheses and 5 recommendations based on emerging 38.4% 61.6% 1,774 0.6 0.5 data Articulate evidence-based rationale for decisions, recommendations, and 6 41.0% 59.0% 1,778 0.6 0.5 opinions to clients, professionals, and the public Make and implement decisions and recommendations that integrate client- 7 37.6% 62.4% 1,773 0.6 0.5 specific, social/contextual, and scientific data Identify areas where biases and 8 heuristics may interfere with effective 41.1% 58.9% 1,772 0.6 0.5 services Critically evaluate the literature relevant 9 26.7% 73.3% 1,771 0.7 0.4 to professional practice

Pearson VUE Confidential Page 117 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Share psychological knowledge with diverse groups (e.g., students, 10 colleagues, clients, other professionals, 42.3% 57.7% 1,775 0.6 0.5 the public) within professional settings in an unbiased manner Select tests based upon the constructs to be assessed, the psychometric 11 22.9% 77.1% 1,766 0.8 0.4 properties of the tests, and the client population Integrate and analyze client needs, practical constraints, and assessment 12 30.0% 70.0% 1,764 0.7 0.5 questions into the selection of assessment methods Conceptualize interventions for client 13 presentations guided by research and 29.3% 70.7% 1,769 0.7 0.5 theory Regularly evaluate effectiveness of 14 37.5% 62.5% 1,774 0.6 0.5 interventions Professional Practice Integrate knowledge of client characteristics in formulating assessment 15 26.9% 73.1% 1,683 0.7 0.4 questions and understanding the reason for assessment Select assessment methods and instruments based on available normed 16 data and/or criterion-reference 23.2% 76.8% 1,676 0.8 0.4 standards, and address any limitations in that selection Ensure that professional opinions, recommendations, and case formulations 17 29.4% 70.6% 1,692 0.7 0.5 adequately reflect consideration of client characteristics Adapt interview questions and behaviors 18 in light of the characteristics of the 34.1% 65.9% 1,687 0.7 0.5 interviewer and interviewee Demonstrate flexible, empathic, and 19 accurate use of a broad range of 33.2% 66.8% 1,683 0.7 0.5 interview techniques

Pearson VUE Confidential Page 118 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Consider contextual information (e.g., reason for assessment, possible legal or 20 38.5% 61.5% 1,677 0.6 0.5 forensic considerations) in conducting an interview Administer, score, and interpret a range 21 of commonly used standardized 15.1% 84.9% 1,664 0.8 0.4 assessment methods and instruments Adapt relevant guidelines in situations requiring non-standard administration, 22 39.3% 60.7% 1,645 0.6 0.5 scoring, interpretation, or communication of assessment results Interpret and integrate results from standardized tests and interviews 23 following established guidelines and, as 22.7% 77.3% 1,645 0.8 0.4 appropriate, multiple applicable norm sets Identify the strengths and limitations of 24 24.7% 75.3% 1,646 0.8 0.4 various types of assessment data Reconcile or explain discrepancies between various sources of data and 25 suggest alternative interpretations or 31.1% 68.9% 1,645 0.7 0.5 explanations in light of any limitations of assessment instruments Use multi-modal assessment data and 26 theory to formulate an understanding of 27.6% 72.4% 1,651 0.7 0.4 the client Formulate diagnoses using current 27 24.1% 75.9% 1,661 0.8 0.4 taxonomies Provide recommendations that 28 28.8% 71.2% 1,666 0.7 0.5 incorporate client and contextual factors Write assessment reports in a timely and 29 19.8% 80.2% 1,649 0.8 0.4 understandable manner Verbally communicate results from 30 assessments in a timely and 21.9% 78.1% 1,645 0.8 0.4 understandable manner Develop plans for evaluating service or 31 59.5% 40.5% 1,618 0.4 0.5 program effectiveness

Pearson VUE Confidential Page 119 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Assess outcome effectiveness in an 32 51.1% 48.9% 1,631 0.5 0.5 ongoing way Conceptualize intervention or treatment 33 on the basis of evidenced-based 32.7% 67.3% 1,650 0.7 0.5 literature Integrate client/stakeholder opinions, preferences, readiness for change, and 34 42.5% 57.5% 1,646 0.6 0.5 potential for improvement into intervention plan Continually evaluate, modify, and assess 35 the effectiveness of interventions, 41.3% 58.7% 1,643 0.6 0.5 considering all relevant variables Consult with qualified peers when facing 36 the need to modify interventions in 36.2% 63.8% 1,634 0.6 0.5 unfamiliar situations Tailor consultation requests and provision of information based on 37 54.2% 45.8% 1,621 0.5 0.5 knowledge of others’ professional needs and viewpoints Use evidence-based psychological 38 theories, decision-making strategies, and 48.9% 51.1% 1,633 0.5 0.5 interventions when consulting Continually evaluate, modify, and assess 39 the effectiveness of consultation, 58.8% 41.2% 1,617 0.4 0.5 considering all relevant variables Provide a supervision plan that details the supervisory relationship and the 40 policies and procedures of supervision, 73.5% 26.5% 1,595 0.3 0.4 including procedures to manage high- risk situations Identify responsibilities of supervisees 41 towards clients, including informed 67.2% 32.8% 1,594 0.3 0.5 consent and supervisory status Regularly provide behaviorally anchored 42 feedback about supervisee strengths and 71.4% 28.6% 1,590 0.3 0.5 areas that need further development

Pearson VUE Confidential Page 120 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Assure that supervisees who are trainees practice within the scope of supervisor’s 43 70.8% 29.2% 1,589 0.3 0.5 competence and license and promote professional development Attend to the interpersonal process 44 66.4% 33.6% 1,594 0.3 0.5 between supervisor and supervisees Monitor possible multiple roles or 45 conflicts of interest and work toward 52.7% 47.3% 1,627 0.5 0.5 resolution if required Relational Competence Recognize, understand, and monitor the 46 impact of one’s own identities in 35.4% 64.6% 1,831 0.6 0.5 professional situations Engage in respectful interactions with an 47 awareness of individual, community, and 26.9% 73.1% 1,835 0.7 0.4 organizational differences Modify one’s own behavior based on self- reflection and an understanding of the 48 31.9% 68.1% 1,826 0.7 0.5 impact of social, cultural, and organizational contexts Follow professional guidelines and the scientific literature, when available, for 49 27.5% 72.5% 1,828 0.7 0.4 providing professional services to diverse populations Apply culturally appropriate skills, 50 techniques, and behaviors with an 28.4% 71.6% 1,834 0.7 0.5 appreciation of individual differences Use relational skills to engage, establish 51 and maintain working relationships with 20.9% 79.1% 1,830 0.8 0.4 a range of clients Communicate respectfully, showing 52 10.9% 89.1% 1,832 0.9 0.3 empathy for others Collaborate effectively in professional 53 27.8% 72.2% 1,823 0.7 0.4 interactions Consider differing viewpoints held by 54 18.2% 81.8% 1,821 0.8 0.4 clients and others

Pearson VUE Confidential Page 121 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Respond to differing viewpoints by 55 seeking clarification to increase 27.5% 72.5% 1,820 0.7 0.4 understanding before taking action Manage difficult and complex 56 interpersonal relationships between self 36.2% 63.8% 1,823 0.6 0.5 and others Seek experienced peers’ feedback to examine, and modify if necessary, one’s 57 31.1% 68.9% 1,826 0.7 0.5 own reactions and behavior when managing interpersonal conflict Professionalism Identify limits of professional 58 competence by recognizing strengths 36.4% 63.6% 1,905 0.6 0.5 and weaknesses in practice areas Use knowledge of personal strengths and 59 42.1% 57.9% 1,908 0.6 0.5 weaknesses to guide scope of practice Seek appropriate consultation when unsure about one’s competence and 60 28.8% 71.2% 1,903 0.7 0.5 additional needs for training and development Seek additional knowledge, training, and 61 supervision when expanding scope of 43.1% 56.9% 1,902 0.6 0.5 practice Update knowledge and skills relevant to 62 psychological practice on an ongoing 44.4% 55.6% 1,906 0.6 0.5 basis Engage in systematic and ongoing self- 63 43.3% 56.7% 1,900 0.6 0.5 assessment and skills development Accept personal responsibility when 64 mistakes or oversights occur and take 31.7% 68.3% 1,908 0.7 0.5 appropriate corrective action if needed Maintain awareness of personal factors 65 31.9% 68.1% 1,907 0.7 0.5 that may impact professional functioning Ethical and Legal Practice Demonstrate integration and application 66 of ethics codes and laws in all 29.8% 70.2% 1,838 0.7 0.5 professional interactions

Pearson VUE Confidential Page 122 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice Communicate ethical and legal standards 67 34.0% 66.0% 1,833 0.7 0.5 in professional interactions Seek professional consultation on ethical 68 27.6% 72.4% 1,834 0.7 0.4 or legal issues when needed Discuss with peers or organizations any 69 49.0% 51.0% 1,823 0.5 0.5 ethical concerns with their behavior Take appropriate steps to resolve 70 conflicts between laws/rules and codes 50.0% 50.0% 1,815 0.5 0.5 of ethics in one’s practice 71 Maintain complete and accurate records 19.3% 80.7% 1,831 0.8 0.4 Report research results accurately, 72 14.8% 85.2% 1,785 0.9 0.4 avoiding personal biases Ensure adequate and appropriate credit 73 is given to trainees and collaborators in 26.9% 73.1% 1,772 0.7 0.4 scholarship Develop a plan commensurate with laws, 74 ethical guidelines, and fiscal constraints 60.3% 39.7% 1,798 0.4 0.5 to manage professional activities Practice in a fiscally and ethically sound 75 51.7% 48.3% 1,816 0.5 0.5 manner Systematically identify the ethical and 76 legal issues and conflicts that occur in 51.4% 48.6% 1,814 0.5 0.5 one’s professional practice Consult with peers to aid in ethical 77 30.1% 69.9% 1,823 0.7 0.5 decision-making when appropriate Engage in critical analysis of identified 78 ethical issues and proactively address 40.2% 59.8% 1,813 0.6 0.5 them 79 Model ethical decision-making to others 41.5% 58.5% 1,820 0.6 0.5 Systems Thinking Recognize the organizational and 80 systemic factors that affect delivery of 53.7% 46.3% 1,901 0.5 0.5 psychological services Utilize knowledge of organizations and 81 systems to optimize delivery of 58.6% 41.4% 1,896 0.4 0.5 psychological services Collaborate with various healthcare 82 39.8% 60.2% 1,902 0.6 0.5 professions to meet common client goals

Pearson VUE Confidential Page 123 Survey After Prior to statement Survey statement text independent independent N M SD number practice practice With other healthcare professionals, 83 integrate services to meet common 44.2% 55.8% 1,897 0.6 0.5 client goals Collaborate with representatives from 84 other professional groups or systems in 51.2% 48.8% 1,885 0.5 0.5 the provision of psychological services Note. All domains/competency areas, statement numbers, and statement text are presented as they appeared in the survey and do not necessarily represent how they are organized for the final test specifications.

Pearson VUE Confidential Page 124 Appendix N. Post-Survey Revisions to the ASPPB Competency Model

Note. aCombined with behavioral exemplar 16. bMoved to competency 10. cCombined with behavioral exemplar 25.

Table N1. Post-Survey Revisions to the ASPPB Competency Model Survey Survey Survey Revised competency behavioral exemplar competency area competency area 1. Select relevant research 1. Critically evaluate and apply literature and critically review its research findings to practice, with assumptions, conceptualization, Scientific Orientation attention to its applicability and methodology, interpretation, and generalizability generalizability 1. Select relevant research 2. Interpret and communicate literature and critically review its empirical research results in a assumptions, conceptualization, Scientific Orientation Scientific Orientation manner that is easily understood methodology, interpretation, and by non-scientific audiences generalizability 2. Interpret, evaluate, and integrate results of assessment activities within the context of a3. Identify and collect applicable scientific/professional knowledge assessment data and integrate Scientific Orientation Deleted to formulate and reformulate with theoretical models to working hypotheses, develop working hypotheses conceptualizations, and recommendations 2. Interpret, evaluate, and b4. Synthesize client-specific and integrate results of assessment scientific data with contextual activities within the context of factors to refine working scientific/professional knowledge Assessment and hypotheses and develop Scientific Orientation to formulate and reformulate Intervention conclusions and working hypotheses, recommendations across a range conceptualizations, and of problems recommendations 2. Interpret, evaluate, and integrate results of assessment activities within the context of c5. Reformulate working scientific/professional knowledge hypotheses and Scientific Orientation Deleted to formulate and reformulate recommendations based on working hypotheses, emerging data conceptualizations, and recommendations

Pearson VUE Confidential Page 125 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 3. Articulate a rationale for 6. Articulate evidence-based decisions and psychological rationale for decisions, services that rely on objective Assessment and recommendations, and opinions Scientific Orientation supporting data (e.g., research Intervention to clients, professionals, and the results, base rates, public epidemiological data) 3. Articulate a rationale for 7. Make and implement decisions decisions and psychological and recommendations that services that rely on objective Assessment and integrate client-specific, Scientific Orientation supporting data (e.g., research Intervention social/contextual, and scientific results, base rates, data epidemiological data) 3. Articulate a rationale for decisions and psychological 8. Identify areas where biases services that rely on objective Assessment and and heuristics may interfere with Scientific Orientation supporting data (e.g., research Intervention effective services results, base rates, epidemiological data) 4. Acquire and disseminate 9. Critically evaluate the knowledge in accord with literature relevant to professional Scientific Orientation Scientific Orientation scientific and ethical principles practice 10. Share psychological knowledge with diverse groups 4. Acquire and disseminate (e.g., students, colleagues, knowledge in accord with Scientific Orientation Scientific Orientation clients, other professionals, the scientific and ethical principles public) within professional settings in an unbiased manner a11. Select tests based upon the 5. Select and use evidence-based constructs to be assessed, the assessment methods and Scientific Orientation Deleted psychometric properties of the instruments tests, and the client population 12. Integrate and analyze client 5. Select and use evidence-based needs, practical constraints, and Assessment and assessment methods and Scientific Orientation assessment questions into the Intervention instruments selection of assessment methods 13. Conceptualize interventions 6. Select and use evidence-based for client presentations guided by Scientific Orientation Deleted interventions research and theory 6. Select and use evidence-based 14. Regularly evaluate Scientific Orientation Deleted interventions effectiveness of interventions 15. Integrate knowledge of client 7. Apply knowledge of individual characteristics in formulating Assessment and and diversity characteristics in assessment questions and Professional Practice Intervention assessment and diagnosis understanding the reason for assessment

Pearson VUE Confidential Page 126 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 16. Select assessment methods and instruments based on 7. Apply knowledge of individual available normed data and/or Assessment and and diversity characteristics in Professional Practice criterion-reference standards, Intervention assessment and diagnosis and address any limitations in that selection 17. Ensure that professional 7. Apply knowledge of individual opinions, recommendations, and Assessment and and diversity characteristics in case formulations adequately Professional Practice Intervention assessment and diagnosis reflect consideration of client characteristics 18. Adapt interview questions 8. Demonstrate effective and behaviors in light of the Assessment and Professional Practice interviewing skills characteristics of the interviewer Intervention and interviewee 19. Demonstrate flexible, 8. Demonstrate effective empathic, and accurate use of a Assessment and Professional Practice interviewing skills broad range of interview Intervention techniques 20. Consider contextual information (e.g., reason for 8. Demonstrate effective Assessment and assessment, possible legal or Professional Practice interviewing skills Intervention forensic considerations) in conducting an interview 21. Administer, score, and 9. Administer and score interpret a range of commonly Assessment and instruments following current Professional Practice used standardized assessment Intervention guidelines and research methods and instruments 22. Adapt relevant guidelines in 9. Administer and score situations requiring non-standard Assessment and instruments following current administration, scoring, Professional Practice Intervention guidelines and research interpretation, or communication of assessment results 10. Interpret and synthesize results from multiple sources 23. Interpret and integrate (e.g., multiple methods of results from standardized tests assessment, written and interviews following Assessment and Professional Practice documentation, interviewees, established guidelines and, as Intervention collateral sources of information) appropriate, multiple applicable following current guidelines and norm sets research

Pearson VUE Confidential Page 127 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 10. Interpret and synthesize results from multiple sources (e.g., multiple methods of 24. Identify the strengths and assessment, written Assessment and limitations of various types of Professional Practice documentation, interviewees, Intervention assessment data collateral sources of information) following current guidelines and research 10. Interpret and synthesize 25. Reconcile or explain results from multiple sources discrepancies between various (e.g., multiple methods of sources of data and suggest assessment, written Assessment and alternative interpretations or Professional Practice documentation, interviewees, Intervention explanations in light of any collateral sources of information) limitations of assessment following current guidelines and instruments research 11. Formulate diagnoses, recommendations, and/or 26. Use multi-modal assessment Assessment and professional opinions using data and theory to formulate an Professional Practice Intervention relevant criteria and considering understanding of the client all assessment data 11. Formulate diagnoses, recommendations, and/or 27. Formulate diagnoses using Assessment and professional opinions using Professional Practice current taxonomies Intervention relevant criteria and considering all assessment data 11. Formulate diagnoses, recommendations, and/or 28. Provide recommendations Assessment and professional opinions using that incorporate client and Professional Practice Intervention relevant criteria and considering contextual factors all assessment data 12. Communicate assessment 29.Write assessment reports in a results to clients, referral sources, Assessment and timely and understandable Professional Practice and other professionals in an Intervention manner integrative manner 12. Communicate assessment 30.Verbally communicate results results to clients, referral sources, Assessment and from assessments in a timely and Professional Practice and other professionals in an Intervention understandable manner integrative manner 13. Evaluate service or program Collaboration, 31.Develop plans for evaluating effectiveness across a variety of Professional Practice Consultation, and service or program effectiveness contexts Supervision 13. Evaluate service or program Collaboration, 32. Assess outcome effectiveness effectiveness across a variety of Professional Practice Consultation, and in an ongoing way contexts Supervision

Pearson VUE Confidential Page 128 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 14. Select interventions for clients based on ongoing 33. Conceptualize intervention or Assessment and assessment and research treatment on the basis of Professional Practice Intervention evidence as well as contextual evidenced-based literature and diversity factors 14. Select interventions for 34. Integrate client/stakeholder clients based on ongoing opinions, preferences, readiness Assessment and assessment and research for change, and potential for Professional Practice Intervention evidence as well as contextual improvement into intervention and diversity factors plan 15. Apply and modify interventions based on ongoing 35. Continually evaluate, modify, assessment, research, contextual and assess the effectiveness of Assessment and Professional Practice factors, client characteristics, and interventions, considering all Intervention situational and environmental relevant variables variables 15. Apply and modify interventions based on ongoing 36. Consult with qualified peers assessment, research, contextual when facing the need to modify Assessment and Professional Practice factors, client characteristics, and interventions in unfamiliar Intervention situational and environmental situations variables 37. Tailor consultation requests and provision of information Collaboration, 16. Consult and collaborate within based on knowledge of others’ Professional Practice Consultation, and and across professions professional needs and Supervision viewpoints 38. Use evidence-based Collaboration, 16. Consult and collaborate within psychological theories, decision- Professional Practice Consultation, and and across professions making strategies, and Supervision interventions when consulting 39. Continually evaluate, modify, Collaboration, 16. Consult and collaborate within and assess the effectiveness of Professional Practice Consultation, and and across professions consultation, considering all Supervision relevant variables 40. Provide a supervision plan 17. Ensure compliance with that details the supervisory Collaboration, policies and procedures of the relationship and the policies and Professional Practice Consultation, and practice/organization, the procedures of supervision, Supervision profession, and the jurisdiction including procedures to manage high-risk situations 17. Ensure compliance with 41. Identify responsibilities of Collaboration, policies and procedures of the supervisees towards clients, Professional Practice Consultation, and practice/organization, the including informed consent and Supervision profession, and the jurisdiction supervisory status

Pearson VUE Confidential Page 129 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 18. Monitor, evaluate, and 42. Regularly provide accurately and sensitively behaviorally anchored feedback Collaboration, communicate supervisee about supervisee strengths and Professional Practice Consultation, and performance to the supervisee, areas that need further Supervision the organization, and the development jurisdiction as needed 18. Monitor, evaluate, and 43. Assure that supervisees who accurately and sensitively are trainees practice within the Collaboration, communicate supervisee scope of supervisor’s competence Professional Practice Consultation, and performance to the supervisee, and license and promote Supervision the organization, and the professional development jurisdiction as needed 19. Create and maintain a supportive environment in which 44. Attend to the interpersonal Collaboration, effective supervision occurs for process between supervisor and Professional Practice Consultation, and trainees and other professionals supervisees Supervision being supervised 19. Create and maintain a supportive environment in which 45. Monitor possible multiple Collaboration, effective supervision occurs for roles or conflicts of interest and Professional Practice Consultation, and trainees and other professionals work toward resolution if required Supervision being supervised 20. Integrate and apply theory, 46. Recognize, understand, and research, professional guidelines, monitor the impact of one’s own Relational Relational and personal understanding about identities in professional Competence Competence social contexts to work effectively situations with diverse clients 20. Integrate and apply theory, 47. Engage in respectful research, professional guidelines, interactions with an awareness of Relational Relational and personal understanding about individual, community, and Competence Competence social contexts to work effectively organizational differences with diverse clients 20. Integrate and apply theory, 48. Modify one’s own behavior research, professional guidelines, based on self-reflection and an Relational Relational and personal understanding about understanding of the impact of Competence Competence social contexts to work effectively social, cultural, and with diverse clients organizational contexts 20. Integrate and apply theory, 49. Follow professional guidelines research, professional guidelines, and the scientific literature, when Relational Relational and personal understanding about available, for providing Competence Competence social contexts to work effectively professional services to diverse with diverse clients populations

Pearson VUE Confidential Page 130 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 20. Integrate and apply theory, 50. Apply culturally appropriate research, professional guidelines, skills, techniques, and behaviors Relational Relational and personal understanding about with an appreciation of individual Competence Competence social contexts to work effectively differences with diverse clients 21. Work effectively with 51. Use relational skills to individuals, families, groups, engage, establish and maintain Relational Relational communities, and/or working relationships with a Competence Competence organizations range of clients 21. Work effectively with individuals, families, groups, 52. Communicate respectfully, Relational Relational communities, and/or showing empathy for others Competence Competence organizations 21. Work effectively with individuals, families, groups, 53. Collaborate effectively in Relational Relational communities, and/or professional interactions Competence Competence organizations 22. In all areas of professional practice, demonstrate respect for 54. Consider differing viewpoints Relational Relational others, including those with held by clients and others Competence Competence differing viewpoints 55. Respond to differing 22. In all areas of professional viewpoints by seeking practice, demonstrate respect for Relational Relational clarification to increase others, including those with Competence Competence understanding before taking differing viewpoints action 23. Identify and manage 56. Manage difficult and complex Relational Relational interpersonal conflict between self interpersonal relationships Competence Competence and others between self and others 57. Seek experienced peers’ 23. Identify and manage feedback to examine, and modify Relational Relational interpersonal conflict between self if necessary, one’s own reactions Competence Competence and others and behavior when managing interpersonal conflict 58. Identify limits of professional 24. Identify and observe competence by recognizing boundaries of competence in all Professionalism Professionalism strengths and weaknesses in areas of professional practice practice areas 24. Identify and observe 59. Use knowledge of personal boundaries of competence in all strengths and weaknesses to Professionalism Professionalism areas of professional practice guide scope of practice 60. Seek appropriate consultation 24. Identify and observe when unsure about one’s boundaries of competence in all Professionalism Professionalism competence and additional needs areas of professional practice for training and development

Pearson VUE Confidential Page 131 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 24. Identify and observe 61. Seek additional knowledge, boundaries of competence in all training, and supervision when Professionalism Professionalism areas of professional practice expanding scope of practice 24. Identify and observe 62. Update knowledge and skills boundaries of competence in all relevant to psychological practice Professionalism Professionalism areas of professional practice on an ongoing basis 25. Critically evaluate one’s own 63. Engage in systematic and professional practice through self- ongoing self-assessment and Professionalism Professionalism reflection and feedback from skills development others 25. Critically evaluate one’s own 64. Accept personal responsibility professional practice through self- when mistakes or oversights Professionalism Professionalism reflection and feedback from occur and take appropriate others corrective action if needed 25. Critically evaluate one’s own 65. Maintain awareness of professional practice through self- personal factors that may impact Professionalism Professionalism reflection and feedback from professional functioning others 26. Demonstrate and promote 66. Demonstrate integration and values and behaviors application of ethics codes and Ethical and Ethical and commensurate with standards of laws in all professional Legal Practice Legal Practice practice, including ethics codes, interactions laws, and regulations 26. Demonstrate and promote values and behaviors 67. Communicate ethical and Ethical and Ethical and commensurate with standards of legal standards in professional Legal Practice Legal Practice practice, including ethics codes, interactions laws, and regulations 26. Demonstrate and promote values and behaviors 68. Seek professional Ethical and Ethical and commensurate with standards of consultation on ethical or legal Legal Practice Legal Practice practice, including ethics codes, issues when needed laws, and regulations 26. Demonstrate and promote values and behaviors 69. Discuss with peers or Ethical and Ethical and commensurate with standards of organizations any ethical Legal Practice Legal Practice practice, including ethics codes, concerns with their behavior laws, and regulations 26. Demonstrate and promote 70. Take appropriate steps to values and behaviors resolve conflicts between Ethical and Ethical and commensurate with standards of laws/rules and codes of ethics in Legal Practice Legal Practice practice, including ethics codes, one’s practice laws, and regulations

Pearson VUE Confidential Page 132 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 27. Accurately represent and document work performed in 71. Maintain complete and Ethical and Ethical and professional practice and accurate records Legal Practice Legal Practice scholarship 27. Accurately represent and 72. Report research results document work performed in Ethical and Ethical and accurately, avoiding personal professional practice and Legal Practice Legal Practice biases scholarship 27. Accurately represent and 73. Ensure adequate and document work performed in appropriate credit is given to Ethical and Ethical and professional practice and trainees and collaborators in Legal Practice Legal Practice scholarship scholarship 74. Develop a plan 28. Implement ethical practice commensurate with laws, ethical Ethical and Ethical and management guidelines, and fiscal constraints Legal Practice Legal Practice to manage professional activities 28. Implement ethical practice 75. Practice in a fiscally and Ethical and Ethical and management ethically sound manner Legal Practice Legal Practice 76. Systematically identify the 29. Establish and maintain a ethical and legal issues and Ethical and Ethical and process that promotes ethical conflicts that occur in one’s Legal Practice Legal Practice decision-making professional practice 29. Establish and maintain a 77. Consult with peers to aid in Ethical and Ethical and process that promotes ethical ethical decision-making when Legal Practice Legal Practice decision-making appropriate 29. Establish and maintain a 78. Engage in critical analysis of Ethical and Ethical and process that promotes ethical identified ethical issues and Legal Practice Legal Practice decision-making proactively address them 29. Establish and maintain a 79. Model ethical decision-making Ethical and process that promotes ethical Deleted to others Legal Practice decision-making 80. Recognize the organizational Collaboration, 30. Work effectively within and systemic factors that affect Systems Thinking Consultation, and organizations and systems delivery of psychological services Supervision 81. Utilize knowledge of Collaboration, 30. Work effectively within organizations and systems to Systems Thinking Consultation, and organizations and systems optimize delivery of psychological Supervision services 82. Collaborate with various Collaboration, 31. Demonstrate interdisciplinary healthcare professions to meet Systems Thinking Consultation, and collaborations common client goals Supervision 83. With other healthcare 31. Demonstrate interdisciplinary professionals, integrate services Systems Thinking Deleted collaborations to meet common client goals

Pearson VUE Confidential Page 133 Survey Survey Survey Revised competency behavioral exemplar competency area competency area 84. Collaborate with representatives from other Collaboration, 31. Demonstrate interdisciplinary professional groups or systems in Systems Thinking Consultation, and collaborations the provision of psychological Supervision services

Pearson VUE Confidential Page 134 Appendix O. Final EPPP Knowledge Scales Statement Weights

Table O1. Final EPPP Knowledge Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method

Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items

Survey statement Survey statement text Weight Rank Quartile number Biological Bases of Behavior 6.20 Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and 1 1.41 35 2 affect in normal, acute and chronic neurobehavioral disease processes; and/or acute and chronic disease comorbidities Drug classification, mechanisms of action, and desired/adverse effects of 2 1.57 21 2 therapeutic agents, drugs of abuse, and complementary or alternative remedies Results from major trials and general guidelines for pharmacological, 3 1.34 45 3 psychotherapeutic, and combined treatment of psychological disorders Behavioral genetics, transmission and expression of genetic information and its 4 modification and the role and limitations 1.01 62 4 of this information in understanding disorders Applications of structural and functional brain imaging methods, electrophysiological methods, 5 0.88 69 4 therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness Cognitive-Affective Bases of Behavior 11.56 Major research‐based theories and 6 models of intelligence and their 1.27 51 3 application Major research-based theories, models, 7 and principles of learning and their 1.34 44 3 application Major research‐based theories and 8 1.21 53 3 models of memory and their application Major research‐based theories and 9 models of motivation and their 1.30 49 3 application Major research-based theories and 10 1.49 24 2 models of emotion and their application

Pearson VUE Confidential Page 135 Survey statement Survey statement text Weight Rank Quartile number Additional elements of cognition, including sensation and perception, 11 attention, language, information 1.39 37 3 processing, visual-spatial processing, executive functioning Relations among cognitions/beliefs, 12 behavior, affect, temperament, and 1.77 9 1 mood Influence of psychosocial factors on 13 1.77 8 1 cognitions/beliefs and behaviors Social and Cultural Bases of Behavior 9.83 Social cognition (e.g., theories, person 14 perception, development of stereotypes, 1.34 46 3 prejudice) Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non‐ 15 1.44 33 2 verbal communication, internet communication, mate selection, empathy) Group and systems processes (e.g., school, work, and family systems, job 16 satisfaction, team functioning, 1.32 48 3 conformity, persuasion) and social influences on functioning Personality theories (e.g., psychodynamic, humanistic/existential, 17 1.49 25 2 cognitive, behavioral, trait theory, interpersonal) Cultural and sociopolitical psychology (e.g., privilege/oppression, cross‐cultural 18 comparisons, political differences, 1.48 26 2 international and global awareness, religion and spirituality, acculturation) Identity diversity and intersectionality (e.g., psychological impact of diversity 19 1.44 32 2 on individuals, families, and systems; conceptual models; assumptions) Causes, manifestations, and effects of 20 1.34 47 3 oppression Growth and Lifespan Development 12.46 Normal growth and development across the lifespan (e.g., biological, physical, 21 1.70 14 1 sexual, cognitive, perceptual, social, personality, moral, emotional, career) Influence of individual‐environment interaction over time on development 22 (e.g., the relationship between the 1.63 16 1 individual and the social, academic, work, community environment) Major research‐based theories of 23 1.20 55 4 development Influence of diverse identities on 24 1.36 42 3 development

Pearson VUE Confidential Page 136 Survey statement Survey statement text Weight Rank Quartile number Family development, configuration, and 25 functioning and its impact on the 1.54 22 2 individual across the lifespan Life events that can influence the normal 26 course of development across the 1.71 13 1 lifespan Risk/protective factors that may impact a developmental course (e.g., nutrition, 27 prenatal care, health care, social 1.75 10 1 support, socioeconomic status, abuse/victimization/resiliency) Disorders/diseases that impact the 28 expected course of development over 1.59 20 2 the lifespan Assessment and Diagnosis 17.77 Psychometric theory, generalizability theory, item and test characteristics, test construction and standardization 29 1.42 34 2 procedures, reliability and validity, sensitivity and specificity, and test fairness and bias Assessment theories and models (e.g., 30 developmental, behavioral, ecological, 1.36 41 3 neuropsychological) Assessment methods (e.g., self‐report, multi-informant reports, psychophysiological measures, work 31 samples, assessment centers, direct 1.62 17 1 observation, structured and semi‐ structured interviews) and their strengths and limitations Commonly used instruments for the measurement of characteristics and behaviors of individuals (e.g., psychosocial functioning; cognitive and 32 1.70 15 1 personality; achievement; psychopathology; risk assessment) and the appropriate use of these tests with various populations Issues of differential diagnosis and integration of non‐psychological 33 1.85 6 1 information into psychological assessment Instruments and methods appropriate for the assessment of groups and 34 organizations (e.g., program evaluation, 0.80 70 4 needs assessment, organizational and personnel assessment) Criteria for selection and adaptation of assessment methods including evidenced-based knowledge of 35 assessment limitations, cultural 1.46 31 2 appropriateness, trans‐cultural adaptation, and language accommodations

Pearson VUE Confidential Page 137 Survey statement Survey statement text Weight Rank Quartile number Classification systems and their underlying rationales and limitations for 36 evaluating client functioning; 1.21 54 4 dimensional vs. categorical approaches to diagnosis Factors influencing interpretation of data and decision‐making (e.g., base rates, 37 1.39 38 3 group differences, cultural biases and differences, heuristics, evidence base) Constructs of epidemiology and base rates of psychological conditions and 38 1.15 56 4 behavioral disorders in clinical or demographic populations Major research-based theories and 39 1.48 29 2 models of psychopathology Measurement of outcomes and changes due to prevention or intervention efforts 40 1.36 40 3 with individuals, couples, families, groups, and organizations Use of technology in implementing tests, surveys, and other forms of assessment 41 0.97 66 4 and diagnostic evaluation; validity, cost effectiveness, consumer acceptability Treatment, Intervention, and Prevention and 14.98 Supervision Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client 42 1.81 7 1 or patient with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change) Contemporary theories/models of 43 treatment/intervention/prevention and 1.71 12 1 their evidence base Treatment techniques/interventions and 44 the evidence for their comparative 1.74 11 1 efficacy and effectiveness Methods and their evidence base for prevention, intervention and/or 45 1.61 19 2 rehabilitation with diverse or special populations Interventions to enhance growth and performance of individuals, couples, 46 1.48 28 2 families, groups, systems, and organizations 47 Consultation models and processes 1.06 59 4 Models of vocational and career 48 development in the provision of 0.67 71 4 psychological services Technology‐assisted psychological 49 0.98 65 4 services, including telepsychology

Pearson VUE Confidential Page 138 Survey statement Survey statement text Weight Rank Quartile number Healthcare systems, structures, and 50 economics, and how these impact 1.26 52 3 intervention choice Approaches to health promotion, risk 51 1.51 23 2 reduction, resilience, and wellness Contemporary theories/models of 52 1.15 57 4 supervision and their evidence base Research Methods and Statistics 8.25 Sampling and data collection methods 53 1.00 63 4 and issues Design of case studies, correlational, 54 quasi‐experimental and experimental 0.91 67 4 studies Analytic methods, including qualitative (e.g., thematic, phenomenological) and quantitative (e.g., probability theory; descriptive, inferential, and parametric 55 0.89 68 4 statistics; meta-analysis, exploratory and confirmatory factor analysis, non‐ parametric statistics, and causal modeling) Statistical interpretation (e.g., power, effect size, causation vs. association, 56 sensitivity and specificity, 1.12 58 4 generalizability, clinical vs. statistical significance) Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to 57 1.29 50 3 generalizations, threats to internal and external validity, design flaws, level of evidence) Evaluation strategies and techniques (e.g., needs assessment, process/implementation evaluation, 58 1.03 60 4 formative and summative assessment program evaluation, outcome evaluation, cost‐benefit analysis) Considerations regarding community involvement and participation in 59 1.02 61 4 research, particularly for under- represented populations Dissemination and presentation of 60 0.99 64 4 research findings Ethical/Legal/Professional Issues 18.96 Ethical principles and codes of conduct 61 2.17 1 1 for psychologists (APA, CPA) Professional standards and relevant guidelines for the practice of psychology 62 (e.g., practice guidelines for providers of 2.02 4 1 psychological services, standards for educational and psychological testing) Awareness of applicable laws/statutes 63 and/or judicial decisions that affect 1.95 5 1 psychological practice

Pearson VUE Confidential Page 139 Survey statement Survey statement text Weight Rank Quartile number Identifying and managing potential 64 2.08 2 1 ethical issues 65 Models of ethical decision‐making 1.48 27 2 Approaches for continuing professional 66 1.41 36 2 development Consideration of emerging social, legal, 67 ethical, and policy issues and their 1.48 30 2 impact on psychology 68 Client’s/patient’s rights 2.04 3 1 69 Ethical issues in the conduct of research 1.35 43 3 70 Ethical issues in supervision 1.61 18 1 Ethical issues in technology assisted 71 1.37 39 3 psychological services Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items

Pearson VUE Confidential Page 140 Appendix P. Final EPPP Competency Scales Statement Weights

Table P1. Final EPPP Competency Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items Survey statement Survey statement text Weight Rank Quartile number Scientific Orientation to Practice 5.85 Select relevant research literature and critically review its assumptions, 1 2.72 28 4 conceptualization, methodology, interpretation, and generalizability Acquire and disseminate knowledge in 4 accord with scientific and ethical 3.13 25 4 principles Assessment, Diagnosis, and Intervention 34.85 Articulate a rationale for decisions and psychological services that rely on 3 2.93 27 4 objective supporting data (e.g., research results, base rates, epidemiological data) Select and use evidence-based 5 3.31 20 3 assessment methods and instruments Apply knowledge of individual and 7 diversity characteristics in assessment 3.63 10 2 and diagnosis 8 Demonstrate effective interviewing skills 3.81 6 1 Administer and score instruments 9 3.29 21 3 following current guidelines and research Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, 10 3.52 13 2 interviewees, collateral sources of information) following current guidelines and research Formulate diagnoses, recommendations, and/or professional opinions using 11 3.84 3 1 relevant criteria and considering all assessment data Communicate assessment results to 12 clients, referral sources, and other 3.51 15 2 professionals in an integrative manner Select interventions for clients based on ongoing assessment and research 14 3.49 17 3 evidence as well as contextual and diversity factors Apply and modify interventions based on ongoing assessment, research, 15 contextual factors, client characteristics, 3.51 16 3 and situational and environmental variables Relational Competence 14.52 Integrate and apply theory, research, professional guidelines, and personal 20 3.53 12 2 understanding about social contexts to work effectively with diverse clients

Pearson VUE Confidential Page 141 Survey statement Survey statement text Weight Rank Quartile number Work effectively with individuals, 21 families, groups, communities, and/or 3.75 8 1 organizations In all areas of professional practice, 22 demonstrate respect for others, 3.89 2 1 including those with differing viewpoints Identify and manage interpersonal 23 3.35 18 3 conflict between self and others Professionalism 7.42 Identify and observe boundaries of 24 competence in all areas of professional 3.82 5 1 practice Critically evaluate one’s own professional 25 practice through self-reflection and 3.60 11 2 feedback from others Ethical and Legal Practice 15.29 Demonstrate and promote values and behaviors commensurate with standards 26 3.99 1 1 of practice, including ethics codes, laws, and regulations Accurately represent and document work 27 performed in professional practice and 3.66 9 2 scholarship 28 Implement ethical practice management 3.84 4 1 Establish and maintain a process that 29 3.79 7 1 promotes ethical decision-making Collaboration, Consultation, and Supervision 22.08 Evaluate service or program 13 2.44 29 4 effectiveness across a variety of contexts Consult and collaborate within and 16 3.35 19 3 across professions Ensure compliance with policies and 17 procedures of the practice/organization, 3.52 14 2 the profession, and the jurisdiction Monitor, evaluate, and accurately and sensitively communicate supervisee 18 performance to the supervisee, the 3.08 26 4 organization, and the jurisdiction as needed Create and maintain a supportive environment in which effective 19 3.19 24 4 supervision occurs for trainees and other professionals being supervised Work effectively within organizations and 30 3.21 23 4 systems Demonstrate interdisciplinary 31 3.28 22 3 collaborations Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items

Pearson VUE Confidential Page 142 Appendix Q. Final EPPP Behavioral Exemplar Scales Statement Weights

Table Q1. Final EPPP Behavioral Exemplar Statement Weights Unweighted by Jurisdiction, Ordering Criticality Above Importance Above Frequency, and Using the Classical Aggregation Method Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items Survey statement Survey statement text Weight Rank Quartile number Scientific Orientation to Practice 5.66 Critically evaluate and apply research 1 findings to practice, with attention to its 1.12 70 4 applicability and generalizability Interpret and communicate empirical research results in a manner that is 2 1.05 76 4 easily understood by non-scientific audiences Synthesize client-specific and scientific data with contextual factors to refine 4 working hypotheses and develop 1.22 58 3 conclusions and recommendations across a range of problems Critically evaluate the literature relevant 9 1.13 69 4 to professional practice Share psychological knowledge with diverse groups (e.g., students, 10 colleagues, clients, other professionals, 1.15 65 4 the public) within professional settings in an unbiased manner Assessment, Diagnosis, and Intervention 31.07 Articulate evidence-based rationale for decisions, recommendations, and 6 1.22 56 3 opinions to clients, professionals, and the public Make and implement decisions and recommendations that integrate client- 7 1.32 37 2 specific, social/contextual, and scientific data Identify areas where biases and 8 heuristics may interfere with effective 1.24 52 3 services Integrate and analyze client needs, practical constraints, and assessment 12 1.24 50 3 questions into the selection of assessment methods Integrate knowledge of client characteristics in formulating assessment 15 1.39 23 2 questions and understanding the reason for assessment Select assessment methods and instruments based on available normed 16 data and/or criterion-reference 1.26 47 3 standards, and address any limitations in that selection

Pearson VUE Confidential Page 143 Survey statement Survey statement text Weight Rank Quartile number Ensure that professional opinions, recommendations, and case formulations 17 1.46 7 1 adequately reflect consideration of client characteristics Adapt interview questions and behaviors 18 in light of the characteristics of the 1.36 29 2 interviewer and interviewee Demonstrate flexible, empathic, and 19 accurate use of a broad range of 1.40 20 1 interview techniques Consider contextual information (e.g., reason for assessment, possible legal or 20 1.38 24 2 forensic considerations) in conducting an interview Administer, score, and interpret a range 21 of commonly used standardized 1.24 51 3 assessment methods and instruments Adapt relevant guidelines in situations requiring non-standard administration, 22 1.06 74 4 scoring, interpretation, or communication of assessment results Interpret and integrate results from standardized tests and interviews 23 following established guidelines and, as 1.23 53 3 appropriate, multiple applicable norm sets Identify the strengths and limitations of 24 1.20 59 4 various types of assessment data Reconcile or explain discrepancies between various sources of data and 25 suggest alternative interpretations or 1.22 54 3 explanations in light of any limitations of assessment instruments Use multi-modal assessment data and 26 theory to formulate an understanding of 1.22 57 3 the client Formulate diagnoses using current 27 1.32 38 2 taxonomies Provide recommendations that 28 1.44 12 1 incorporate client and contextual factors Write assessment reports in a timely and 29 1.34 33 2 understandable manner Verbally communicate results from 30 assessments in a timely and 1.31 39 2 understandable manner Conceptualize intervention or treatment 33 on the basis of evidenced-based 1.27 45 3 literature Integrate client/stakeholder opinions, preferences, readiness for change, and 34 1.31 42 3 potential for improvement into intervention plan Continually evaluate, modify, and assess 35 the effectiveness of interventions, 1.33 36 2 considering all relevant variables

Pearson VUE Confidential Page 144 Survey statement Survey statement text Weight Rank Quartile number Consult with qualified peers when facing the need to modify interventions in 36 1.30 43 3 unfamiliar situations

Relational Competence 16.86 Recognize, understand, and monitor the 46 impact of one’s own identities in 1.31 40 3 professional situations Engage in respectful interactions with an 47 awareness of individual, community, and 1.40 19 1 organizational differences Modify one’s own behavior based on self- reflection and an understanding of the 48 1.36 30 2 impact of social, cultural, and organizational contexts Follow professional guidelines and the scientific literature, when available, for 49 1.39 22 2 providing professional services to diverse populations Apply culturally appropriate skills, 50 techniques, and behaviors with an 1.43 13 1 appreciation of individual differences Use relational skills to engage, establish 51 and maintain working relationships with 1.54 2 1 a range of clients Communicate respectfully, showing 52 1.60 1 1 empathy for others Collaborate effectively in professional 53 1.38 25 2 interactions Consider differing viewpoints held by 54 1.45 10 1 clients and others Respond to differing viewpoints by 55 seeking clarification to increase 1.39 21 2 understanding before taking action Manage difficult and complex 56 interpersonal relationships between self 1.35 31 2 and others Seek experienced peers’ feedback to examine, and modify if necessary, one’s 57 1.27 46 3 own reactions and behavior when managing interpersonal conflict Professionalism 11.32 Identify limits of professional 58 competence by recognizing strengths 1.46 9 1 and weaknesses in practice areas Use knowledge of personal strengths and 59 1.41 16 1 weaknesses to guide scope of practice Seek appropriate consultation when unsure about one’s competence and 60 1.42 14 1 additional needs for training and development Seek additional knowledge, training, and 61 supervision when expanding scope of 1.42 15 1 practice

Pearson VUE Confidential Page 145 Survey statement Survey statement text Weight Rank Quartile number Update knowledge and skills relevant to 62 psychological practice on an ongoing 1.41 18 1 basis Engage in systematic and ongoing self- 63 1.26 49 3 assessment and skills development Accept personal responsibility when 64 mistakes or oversights occur and take 1.48 5 1 appropriate corrective action if needed Maintain awareness of personal factors 65 1.46 8 1 that may impact professional functioning Ethical and Legal Practice 17.58 Demonstrate integration and application 66 of ethics codes and laws in all 1.51 4 1 professional interactions Communicate ethical and legal standards 67 1.37 26 2 in professional interactions Seek professional consultation on ethical 68 1.45 11 1 or legal issues when needed Discuss with peers or organizations any 69 1.31 41 3 ethical concerns with their behavior Take appropriate steps to resolve 70 conflicts between laws/rules and codes 1.34 35 2 of ethics in one’s practice 71 Maintain complete and accurate records 1.53 3 1 Report research results accurately, 72 1.22 55 3 avoiding personal biases Ensure adequate and appropriate credit 73 is given to trainees and collaborators in 1.10 72 4 scholarship Develop a plan commensurate with laws, 74 ethical guidelines, and fiscal constraints 1.15 64 4 to manage professional activities Practice in a fiscally and ethically sound 75 1.48 6 1 manner Systematically identify the ethical and 76 legal issues and conflicts that occur in 1.36 27 2 one’s professional practice Consult with peers to aid in ethical 77 1.41 17 1 decision-making when appropriate Engage in critical analysis of identified 78 ethical issues and proactively address 1.36 28 2 them Collaboration, Consultation, and Supervision 17.51 Develop plans for evaluating service or 31 0.93 77 4 program effectiveness Assess outcome effectiveness in an 32 1.14 66 4 ongoing way Tailor consultation requests and provision of information based on 37 1.07 73 4 knowledge of others’ professional needs and viewpoints Use evidence-based psychological 38 theories, decision-making strategies, and 1.16 63 4 interventions when consulting

Pearson VUE Confidential Page 146 Survey statement Survey statement text Weight Rank Quartile number Continually evaluate, modify, and assess 39 the effectiveness of consultation, 1.05 75 4 considering all relevant variables Provide a supervision plan that details the supervisory relationship and the 40 policies and procedures of supervision, 1.19 61 4 including procedures to manage high- risk situations Identify responsibilities of supervisees 41 towards clients, including informed 1.27 44 3 consent and supervisory status Regularly provide behaviorally anchored 42 feedback about supervisee strengths and 1.19 62 4 areas that need further development Assure that supervisees who are trainees practice within the scope of supervisor’s 43 1.34 34 2 competence and license and promote professional development Attend to the interpersonal process 44 1.13 68 4 between supervisor and supervisees Monitor possible multiple roles or 45 conflicts of interest and work toward 1.26 48 3 resolution if required Recognize the organizational and 80 systemic factors that affect delivery of 1.19 60 4 psychological services Utilize knowledge of organizations and 81 systems to optimize delivery of 1.13 67 4 psychological services Collaborate with various healthcare 82 1.34 32 2 professions to meet common client goals Collaborate with representatives from 84 other professional groups or systems in 1.10 71 4 the provision of psychological services Note. Quartile 1 – highest ranked items, Quartile 2 – second highest ranked items, Quartile 3 – third highest ranked items, Quartile 4 –lowest ranked items

Pearson VUE Confidential Page 147 Appendix R. Final EPPP Step 1 Blueprint

Test Length: 4 hr, 175 items

Domain 1: Biological Bases of Behavior (10%) K1. Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and affect in normal, acute and chronic neurobehavioral disease processes and disease comorbidities K2. Drug classification, mechanisms of action, and desired/adverse effects of therapeutic agents, drugs of abuse, and complementary or alternative agents K3. Results from major trials and general guidelines for pharmacological, psychotherapeutic, and combined treatment of psychological disorders K4. Behavioral genetics, transmission and expression of genetic information and its modification, and the role and limitations of this information in understanding disorders K5. Applications of structural and functional brain imaging methods, electrophysiological methods, therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness

Domain 2: Cognitive-Affective Bases of Behavior (13%) K6. Major research-based theories and models of intelligence and their application K7. Major research-based theories, models, and principles of learning and their application K8. Major research-based theories and models of memory and their application K9. Major research-based theories and models of motivation and their application K10. Major research-based theories and models of emotion and their application K11. Elements of cognition, including sensation and perception, attention, language, information processing, visual-spatial processing, executive functioning K12. Relations among cognitions/beliefs, behavior, affect, temperament, and mood K13. Influence of psychosocial factors on cognitions/beliefs and behaviors

Domain 3: Social and Cultural Bases of Behavior (11%) K14. Major research-based theories and models of social cognition (e.g., person perception, development of stereotypes, prejudice) K15. Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non-verbal communication, internet communication, mate selection, empathy) K16. Group and systems processes (e.g., school, work, and family systems, job satisfaction, team functioning, conformity, persuasion) and social influences on functioning K17. Major research-based personality theories and models K18. Cultural and sociopolitical psychology (e.g., privilege, cross-cultural comparisons, political differences, international and global awareness, religiosity and spirituality, acculturation) K19. Identity diversity and intersectionality (e.g., psychological impact of diversity on individuals, families, and systems) K20. Causes, manifestations, and effects of oppression

Pearson VUE Confidential Page 148 Domain 4: Growth and Lifespan Development (12%) K21. Normal growth and development across the lifespan K22. Influence of individual-environment interaction on development over time (e.g., the relationship between the individual and the social, academic, work, community environment) K23. Major research-based theories and models of development K24. Influence of diverse identities on development K25. Family development, configuration, and functioning and their impact on the individual across the lifespan K26. Life events that can influence the course of development across the lifespan K27. Risk and protective factors that may impact a developmental course (e.g., nutrition, prenatal care, health care, social support, socioeconomic status, abuse, victimization, and resiliency) K28. Disorders and diseases that impact the expected course of development over the lifespan

Domain 5: Assessment and Diagnosis (16%) K29. Psychometric theories, item and test characteristics, test construction and standardization procedures, reliability and validity, sensitivity and specificity, and test fairness and bias K30. Assessment theories and models (e.g., developmental, behavioral, ecological, neuropsychological) K31. Assessment methods and their strengths and limitations (e.g., self-report, multi- informant reports, psychophysiological measures, work samples, assessment centers, direct observation, structured and semi-structured interviews) K32. Commonly used instruments for the measurement of characteristics and behaviors of individuals and their appropriate use with various populations K33. Issues of differential diagnosis and integration of non-psychological information into psychological assessment K34. Instruments and methods appropriate for the assessment of groups and organizations (e.g., program evaluation, needs assessment, organizational and personnel assessment) K35. Criteria for selection and adaptation of assessment methods (e.g., evidenced-based knowledge of assessment limitations, cultural appropriateness, trans-cultural adaptation, and language accommodations) K36. Classification systems and their underlying rationales and limitations for evaluating client functioning; dimensional vs. categorical approaches to diagnosis K37. Factors influencing evidence-based interpretation of data and decision-making (e.g., base rates, group differences, cultural biases and differences, heuristics) K38. Constructs of epidemiology and base rates of psychological and behavioral disorders K39. Major research-based theories and models of psychopathology K40. Measurement of outcomes and changes due to prevention or intervention efforts with individuals, couples, families, groups, and organizations K41. Use of technology in implementing tests, surveys, and other forms of assessment and diagnostic evaluation (e.g., validity, cost-effectiveness, consumer acceptability)

Domain 6: Treatment, Intervention, and Prevention and Supervision (15%) K42. Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client or patient

Pearson VUE Confidential Page 149 with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change) K43. Contemporary research-based theories and models of treatment, intervention, and prevention K44. Treatment techniques and interventions and the evidence for their comparative efficacy and effectiveness K45. Methods and their evidence base for prevention, intervention, and rehabilitation with diverse and special populations K46. Interventions to enhance growth and performance of individuals, couples, families, groups, systems, and organizations K47. Research-based consultation models and processes K48. Research-based models of vocational and career development K49. Telepsychology and technology-assisted psychological services K50. Healthcare systems, structures, and economics, and how these impact intervention choice K51. Approaches to health promotion, risk reduction, resilience, and wellness K52. Contemporary theories and models of supervision and their evidence base

Domain 7: Research Methods and Statistics (7%) K53. Sampling and data collection methods K54. Design of case, correlational, quasi-experimental and experimental studies K55. Analytic methods, including qualitative (e.g., thematic, phenomenological) and quantitative (e.g., probability theory; descriptive, inferential, and parametric statistics; meta-analysis; factor analysis; causal modeling) K56. Statistical interpretation (e.g., power, effect size, causation vs. association, clinical vs. statistical significance) K57. Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to generalizability, threats to internal and external validity, design flaws, level of evidence) K58. Evaluation strategies and techniques (e.g., needs assessment, process and implementation evaluation, formative and summative program evaluation, outcome evaluation, cost-benefit analysis) K59. Considerations regarding community involvement and participation in research K60. Dissemination and presentation of research findings

Domain 8: Ethical/Legal/Professional Issues (16%) K61. Current ethical principles and codes for psychologists (APA, CPA) K62. Professional standards and relevant guidelines for the practice of psychology (e.g., standards for educational and psychological testing) K63. Laws, statutes, and judicial decisions that affect psychological practice K64. Identification and management of potential ethical issues K65. Models of ethical decision-making K66. Approaches for continuing professional development K67. Emerging social, legal, ethical, and policy issues and their impact on psychological practice K68. Client and patient rights K69. Ethical issues in the conduct of research K70. Ethical issues in supervision K71. Ethical issues in technology-assisted psychological services

Pearson VUE Confidential Page 150 Appendix S. Final EPPP Step 2 Blueprint

The competencies (C) are indicated in the first indent level, and the behavioral exemplars (B) are indicated in the second indent level. Behavioral exemplars tied to a particular competency are listed directly below that competency.

Test Length: 4 hr

Domain 1: Scientific Orientation to Practice (6%) C1. Select relevant research literature and critically review its assumptions, conceptualization, methodology, interpretation, and generalizability B1. Critically evaluate and apply research findings to practice, with attention to its applicability and generalizability B2. Interpret and communicate empirical research results in a manner that is easily understood by non-scientific audiences C2. Acquire and disseminate knowledge in accord with scientific and ethical principles B3. Critically evaluate the literature relevant to professional practice B4. Share psychological knowledge with diverse groups (e.g., students, colleagues, clients, other professionals, the public) within professional settings in an unbiased manner

Domain 2: Assessment and Intervention (33%) C3. Apply knowledge of individual and diversity characteristics in assessment and diagnosis B5. Integrate knowledge of client characteristics in formulating assessment questions and understanding the reason for assessment B6. Select assessment methods and instruments based on psychometric properties, available normed data and/or criterion-referenced standards, and address any limitations in that selection B7. Ensure that professional opinions, recommendations, and case formulations adequately reflect consideration of client characteristics C4. Demonstrate effective interviewing skills B8. Adapt interview questions and behaviors in light of the characteristics of the interviewer and interviewee B9. Demonstrate flexible, empathic, and appropriate use of a broad range of interview techniques B10. Consider contextual information (e.g., reason for assessment, possible legal or forensic considerations) in conducting an interview C5. Administer and score instruments following current guidelines and research B11. Administer, score, and interpret a range of commonly used standardized assessment instruments B12. Adapt relevant guidelines in situations requiring non-standard administration, scoring, interpretation, or communication of assessment results C6. Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, interviewees, collateral sources of information) following current guidelines and research B13. Interpret and integrate results from standardized tests and interviews following established guidelines and, as appropriate, multiple applicable norm sets B14. Identify the strengths and limitations of various types of assessment data

Pearson VUE Confidential Page 151 B15. Reconcile or explain discrepancies between various sources of data and suggest alternative interpretations or explanations in light of any limitations of assessment instruments B16. Synthesize client-specific and scientific data with contextual factors to refine working hypotheses and develop conclusions and recommendations across a range of problems C7. Formulate and communicate diagnoses, recommendations, and/or professional opinions using relevant criteria and considering all assessment data B17. Formulate diagnoses using current taxonomies B18. Provide recommendations that incorporate client and contextual factors, including diagnoses B19. Communicate assessment results to clients, referral sources, and other professionals in an integrative manner C8. Select interventions for clients based on ongoing assessment and research evidence as well as contextual and diversity factors B20. Conceptualize intervention or treatment on the basis of evidenced-based literature B21. Integrate client or stakeholder opinions, preferences, readiness for change, and potential for improvement into intervention plan C9. Apply and modify interventions based on ongoing assessment, research, contextual factors, client characteristics, and situational and environmental variables B22. Articulate evidence-based rationale for decisions, recommendations, and opinions to clients and others as indicated B23. Continually evaluate, modify, and assess the effectiveness of interventions, considering all relevant variables including biases and heuristics B24. Consult with qualified peers when facing the need to modify interventions in unfamiliar situations

Domain 3: Relational Competence (16%) C10. Integrate and apply theory, research, professional guidelines, and personal understanding about social contexts to work effectively with diverse clients B25. Recognize, understand, and monitor the impact of one’s own identities in professional situations B26. Engage in respectful interactions with an awareness of individual, community, and organizational differences B27. Modify one’s own behavior based on self-reflection and an understanding of the impact of social, cultural, and organizational contexts B28. Follow professional guidelines and the scientific literature, when available, for providing professional services to diverse populations B29. Apply culturally appropriate skills, techniques, and behaviors with an appreciation of individual differences C11. Work effectively with individuals, families, groups, communities, and/or organizations B30. Use relational skills to engage, establish, and maintain working relationships with a range of clients B31. Communicate respectfully, showing empathy for others B32. Collaborate effectively in professional interactions C12. Demonstrate respect for others in all areas of professional practice B33. Consider differing viewpoints held by clients and others B34. Respond to differing viewpoints by seeking clarification to increase understanding before taking action C13. Identify and manage interpersonal conflict between self and others

Pearson VUE Confidential Page 152 B35. Manage difficult and complex interpersonal relationships between self and others B36. Consult with peers to examine and address one’s own reactions and behavior when managing interpersonal conflict

Domain 4: Professionalism (11%) C14. Identify and observe boundaries of competence in all areas of professional practice B37. Identify limits of professional competence B38. Use knowledge of professional competence to guide scope of practice B39. Seek appropriate consultation when unsure about one’s competence and additional needs for training and professional development B40. Seek additional knowledge, training, and supervision when expanding scope of practice B41. Update knowledge and skills relevant to psychological practice on an ongoing basis C15. Critically evaluate one’s own professional practice through self-reflection and feedback from others B42. Engage in systematic and ongoing self-assessment and skill development B43. Accept responsibility for one’s own professional work and take appropriate corrective action if needed B44. Maintain awareness of personal factors that may impact professional functioning

Domain 5: Ethical Practice (17%) C16. Demonstrate and promote values and behaviors commensurate with standards of practice, including ethics codes, laws, and regulations B45. Demonstrate integration and application of ethics codes and laws in all professional interactions B46. Communicate ethical and legal standards in professional interactions as necessary B47. Seek professional consultation on ethical or legal issues when needed B48. Discuss with peers or collaborators any ethical concerns with their behavior B49. Take appropriate steps to resolve conflicts between laws or rules and codes of ethics in one’s professional practice C17. Accurately represent and document work performed in professional practice and scholarship B50. Maintain complete and accurate records B51. Report research results accurately, avoiding personal biases B52. Ensure adequate and appropriate credit is given to trainees and collaborators in scholarship C18. Implement ethical practice management B53. Practice in a manner commensurate with laws, ethical standards, practice guidelines, and organizational constraints B54. Manage billing practices in an ethical manner C19. Establish and maintain a process that promotes ethical decision-making B55. Systematically identify the ethical and legal issues and conflicts that occur in professional practice B56. Consult with peers to aid in ethical decision-making when needed B57. Proactively address identified ethical issues

Pearson VUE Confidential Page 153 Domain 6: Collaboration, Consultation, and Supervision (17%)

C20. Work effectively within organizations and systems B58. Recognize the organizational and systemic factors that affect delivery of psychological services B59. Utilize knowledge of organizations and systems to optimize delivery of psychological services C21. Demonstrate interdisciplinary collaborations B60. Collaborate with various professionals to meet client goals C22. Consult and collaborate within and across professions B61. Tailor consultation requests and provision of information based on knowledge of others’ professional needs and viewpoints B62. Use evidence-based psychological theories, decision-making strategies, and interventions when consulting B63. Continually evaluate, modify, and assess the effectiveness of consultation, considering all relevant variables C23. Evaluate service or program effectiveness across a variety of contexts B64. Develop plans for evaluating service or program effectiveness B65. Assess outcome effectiveness in an ongoing way C24. Ensure supervisee compliance with policies and procedures of the setting, the profession, and the jurisdiction B66. Provide a supervision plan that details the supervisory relationship and the policies and procedures of supervision, including procedures to manage high-risk situations B67. Identify responsibilities of supervisees towards clients, including informed consent and supervisory status C25. Monitor, evaluate, and accurately and sensitively communicate supervisee performance to the supervisee, the organization, and the jurisdiction as needed B68. Regularly provide behaviorally anchored feedback about supervisee strengths and areas that need further development B69. Assure that supervisees who are trainees practice within the scope of supervisor’s competence and license C26. Create and maintain a supportive environment in which effective supervision occurs for trainees and other professionals being supervised B70. Attend to the interpersonal process between supervisor and supervisees B71. Monitor possible multiple roles or conflicts of interest, and work toward resolution, if needed

Pearson VUE Confidential Page 154 ASPPB 2016 Job Task Analysis Final Survey

Qualification

* Are you licensed or registered for independent practice as a Psychologist?

Yes

Introduction

Survey of the Practice of Psychology

Thank you for participating in this survey of the practice of psychology conducted on behalf of The Association of State and Provincial Psychology Boards (ASPPB) by Pearson VUE. ASPPB is the alliance of state, provincial, and territorial agencies responsible for the licensure and certification of psychologists throughout the United States and Canada.

This job task analysis survey is the first and most important step in the development process for the Examination for Professional Practice in Psychology (EPPP). It is being distributed to all licensed psychologists in the United States and Canada to adequately reflect the views of psychologists currently practicing.

Results from the survey will be used to revise the test specifications for the knowledge-based EPPP and to develop test specifications for the new skills-based exam, the EPPP Step 2. You will be randomly assigned to answer questions about practice as related to either knowledge or competencies, and basic demographic and professional questions, which will be used to assess sample representativeness.

Participation in the survey is voluntary. Your answers are anonymous, and responses will be only reported in the aggregate.

The survey may take 30-40 minutes to complete. Please try to complete the survey in one sitting. If you need more than one session to complete the survey, close the browser between survey sessions. Do NOT click "Click to Submit and Exit" before completing the full survey. If you do click "Click to Submit and Exit", you will not be allowed to return to the survey. Please complete the survey by August 31, 2016.

ASPPB and the psychology regulatory agencies throughout the United States and Canada sincerely thank you for your contribution to the study of Psychology and for your time and assistance in ensuring that the EPPP is reflective of the current practice of psychology.

Demographic Information

This information is being collected to ensure sample representativeness.

* Country where licensed/registered as a Psychologist:

US (States or Territories)

Canada

Both * Jurisdiction(s) from which you received the link to this survey: (check all that apply)

Alabama (AL) Maine (ME) Ontario (ON)

Alaska (AK) Manitoba (MB) Oregon (OR)

Alberta (AB) Maryland (MD) Pennsylvania (PA)

Arizona (AZ) Massachusetts (MA) Prince Edward Island (PE)

Arkansas (AR) Michigan (MI) Puerto Rico (PR)

British Columbia (BC) Minnesota (MN) Quebec (PQ)

California (CA) Mississippi (MS) Rhode Island (RI)

Colorado (CO) Missouri (MO) Saskatchewan (SK)

Connecticut (CT) Montana (MT) South Carolina (SC)

Delaware (DE) Nebraska (NE) South Dakota (SD)

District of Columbia (DC) Nevada (NV) Tennessee (TN)

Florida (FL) New Brunswick (NB) Texas (TX)

Georgia (GA) New Hampshire (NH) US Virgin Islands (VI)

Guam (GU) New Jersey (NJ) Utah (UT)

Hawaii (HI) New Mexico (NM) Vermont (VT)

Idaho (ID) New York (NY) Virginia (VA)

Illinois (IL) Newfoundland (NL) Washington (WA)

Indiana (IN) North Carolina (NC) West Virginia (WV)

Iowa (IA) North Dakota (ND) Wisconsin (WI)

Kansas (KS) Nova Scotia (NS) Wyoming (WY)

Kentucky (KY) Ohio (OH)

Louisiana (LA) Oklahoma (OK)

Knowledge Statements Introduction

Knowledge Statements

You have been randomly selected to answer questions about the knowledge required for the practice of psychology. The overall domains containing the knowledge statements are presented in a random order, but the knowledge statements within a domain are always presented in the same order. The numbering of the knowledge statements represents the original order as constructed. Due to the randomization of knowledge domains, this numbering might appear to be out of order.

Please rate each of the knowledge statements using the scales displayed to the best of your ability and as related to your practice of psychology. Knowledge Statements

Please rate each knowledge statement using the following three scales:

1. Frequency How frequently did you call upon this particular knowledge in your practice during the past year? (Never or very rarely (e.g., Annually); Infrequently (e.g., Monthly or Quarterly); Frequently (e.g., Weekly); Very Frequently (e.g., Daily)) 2. Criticality How critical is possessing this knowledge to protecting the patient/client/public from harm? (Not critical; Minimally critical; Moderately critical; Highly critical) 3. Importance How important was the knowledge to your practice as a psychologist during the past year? (Not important; Minimally important; Moderately important; Very important)

Biological Bases of Behavior

Frequency Criticality Importance

(K.01) Functional correlates and determinants of the neurobiological and genetic bases of behavior pertaining to perception, cognition, personality, and mood and affect in normal, acute and chronic neurobehavioral disease processes; and/or acute and chronic disease comorbidities

(K.02) Drug classification, mechanisms of action, and desired/adverse effects of therapeutic agents, drugs of abuse, and complementary or alternative remedies

(K.03) Results from major trials and general guidelines for pharmacological, psychotherapeutic, and combined treatment of psychological disorders

(K.04) Behavioral genetics, transmission and expression of genetic information and its modification and the role and limitations of this information in understanding disorders

(K.05) Applications of structural and functional brain imaging methods, electrophysiological methods, therapeutic drug monitoring methods, and genetic screening methodologies, and the evidence for their effectiveness

Knowledge Statements Please rate each knowledge statement using the following three scales:

Cognitive-Affective Bases of Behavior

Frequency Criticality Importance

(K.06) Major research‐based theories and models of intelligence and their application

(K.07) Major research-based theories, models, and principles of learning and their application

(K.08) Major research‐based theories and models of memory and their application

(K.09) Major research‐based theories and models of motivation and their application

(K.10) Major research-based theories and models of emotion and their application

(K.11) Additional elements of cognition, including sensation and perception, attention, language, information processing, visual-spatial processing, executive functioning

(K.12) Relations among cognitions/beliefs, behavior, affect, temperament, and mood

(K.13) Influence of psychosocial factors on cognitions/beliefs and behaviors

Knowledge Statements

Please rate each knowledge statement using the following three scales: 1. Frequency How frequently did you call upon this particular knowledge in your practice during the past year? (Never or very rarely (e.g., Annually); Infrequently (e.g., Monthly or Quarterly); Frequently (e.g., Weekly); Very Frequently (e.g., Daily)) 2. Criticality How critical is possessing this knowledge to protecting the patient/client/public from harm? (Not critical; Minimally critical; Moderately critical; Highly critical) 3. Importance How important was the knowledge to your practice as a psychologist during the past year? (Not important; Minimally important; Moderately important; Very important)

Social and Cultural Bases of Behavior

Frequency Criticality Importance

(K.14) Social cognition (e.g., theories, person perception, development of stereotypes, prejudice)

(K.15) Social interaction and relationships (e.g., attraction, aggression, altruism, organizational justice, verbal and non‐verbal communication, internet communication, mate selection, empathy)

(K.16) Group and systems processes (e.g., school, work, and family systems, job satisfaction, team functioning, conformity, persuasion) and social influences on functioning

(K.17) Personality theories (e.g., psychodynamic, humanistic/existential, cognitive, behavioral, trait theory, interpersonal)

(K.18) Cultural and sociopolitical psychology (e.g., privilege/oppression, cross‐cultural comparisons, political differences, international and global awareness, religion and spirituality, acculturation)

(K.19) Identity diversity and intersectionality (e.g., psychological impact of diversity on individuals, families, and systems; conceptual models; assumptions)

(K.20) Causes, manifestations, and effects of oppression

Knowledge Statements Please rate each knowledge statement using the following three scales:

Growth and Lifespan Development

Frequency Criticality Importance

(K.21) Normal growth and development across the lifespan (e.g., biological, physical, sexual, cognitive, perceptual, social, personality, moral, emotional, career)

(K.22) Influence of individual‐ environment interaction over time on development (e.g., the relationship between the individual and the social, academic, work, community environment)

(K.23) Major research‐based theories of development

(K.24) Influence of diverse identities on development

(K.25) Family development, configuration, and functioning and its impact on the individual across the lifespan

(K.26) Life events that can influence the normal course of development across the lifespan

(K.27) Risk/protective factors that may impact a developmental course (e.g., nutrition, prenatal care, health care, social support, socioeconomic status, abuse/victimization/resiliency)

(K.28) Disorders/diseases that impact the expected course of development over the lifespan

Knowledge Statements Please rate each knowledge statement using the following three scales:

Assessment and Diagnosis

Frequency Criticality Importance

(K.29) Psychometric theory, generalizability theory, item and test characteristics, test construction and standardization procedures, reliability and validity, sensitivity and specificity, and test fairness and bias

(K.30) Assessment theories and models (e.g., developmental, behavioral, ecological, neuropsychological)

(K.31) Assessment methods (e.g., self‐ report, multi-informant reports, psychophysiological measures, work samples, assessment centers, direct observation, structured and semi‐ structured interviews) and their strengths and limitations

(K.32) Commonly used instruments for the measurement of characteristics and behaviors of individuals (e.g., psychosocial functioning; cognitive and personality; achievement; psychopathology; risk assessment) and the appropriate use of these tests with various populations

(K.33) Issues of differential diagnosis and integration of non‐psychological information into psychological assessment

(K.34) Instruments and methods appropriate for the assessment of groups and organizations (e.g., program evaluation, needs assessment, organizational and personnel assessment) Frequency Criticality Importance

(K.35) Criteria for selection and adaptation of assessment methods including evidenced-based knowledge of assessment limitations, cultural appropriateness, trans‐cultural adaptation, and language accommodations

(K.36) Classification systems and their underlying rationales and limitations for evaluating client functioning; dimensional vs. categorical approaches to diagnosis

(K.37) Factors influencing interpretation of data and decision‐ making (e.g., base rates, group differences, cultural biases and differences, heuristics, evidence base)

(K.38) Constructs of epidemiology and base rates of psychological conditions and behavioral disorders in clinical or demographic populations

(K.39) Major research-based theories and models of psychopathology

(K.40) Measurement of outcomes and changes due to prevention or intervention efforts with individuals, couples, families, groups, and organizations

(K.41) Use of technology in implementing tests, surveys, and other forms of assessment and diagnostic evaluation; validity, cost effectiveness, consumer acceptability

Knowledge Statements

Please rate each knowledge statement using the following three scales:

Frequency Criticality Importance

(K.42) Factors related to treatment or intervention decision-making (e.g., relevant research, matching treatment to assessment/diagnosis, matching client or patient with psychologist characteristics, knowledge and use of allied services, cost and benefit, readiness to change)

(K.43) Contemporary theories/models of treatment/intervention/prevention and their evidence base

(K.44) Treatment techniques/interventions and the evidence for their comparative efficacy and effectiveness

(K.45) Methods and their evidence base for prevention, intervention and/or rehabilitation with diverse or special populations

(K.46) Interventions to enhance growth and performance of individuals, couples, families, groups, systems, and organizations

(K.47) Consultation models and processes

(K.48) Models of vocational and career development in the provision of psychological services

(K.49) Technology‐assisted psychological services, including telepsychology

(K.50) Healthcare systems, structures, and economics, and how these impact intervention choice

(K.51) Approaches to health promotion, risk reduction, resilience, and wellness

(K.52) Contemporary theories/models of supervision and their evidence base

Knowledge Statements

Please rate each knowledge statement using the following three scales:

Research Methods and Statistics

Frequency Criticality Importance

(K.53) Sampling and data collection methods and issues

(K.54) Design of case studies, correlational, quasi‐experimental and experimental studies

(K.55) Analytic methods, including qualitative (e.g., thematic, phenomenological) and quantitative (e.g., probability theory; descriptive, inferential, and parametric statistics; meta-analysis, exploratory and confirmatory factor analysis, non‐ parametric statistics, and causal modeling)

(K.56) Statistical interpretation (e.g., power, effect size, causation vs. association, sensitivity and specificity, generalizability, clinical versus statistical significance)

(K.57) Critical appraisal and application of research findings (e.g., adequacy of design and statistics, limitations to generalizations, threats to internal and external validity, design flaws, level of evidence)

(K.58) Evaluation strategies and techniques (e.g., needs assessment, process/implementation evaluation, formative and summative assessment program evaluation, outcome evaluation, cost‐benefit analysis)

(K.59) Considerations regarding community involvement and participation in research, particularly for under-represented populations

(K.60) Dissemination and presentation of research findings

Knowledge Statements Please rate each knowledge statement using the following three scales:

Ethical/Legal/Professional Issues

Frequency Criticality Importance

(K.61) Ethical principles and codes of conduct for psychologists (APA, CPA)

(K.62) Professional standards and relevant guidelines for the practice of psychology (e.g., practice guidelines for providers of psychological services, standards for educational and psychological testing)

(K.63) Awareness of applicable laws/statutes and/or judicial decisions that affect psychological practice

(K.64) Identifying and managing potential ethical issues

(K.65) Models of ethical decision‐ making

(K.66) Approaches for continuing professional development

(K.67) Consideration of emerging social, legal, ethical, and policy issues and their impact on psychology

(K.68) Client’s/patient’s rights

(K.69) Ethical issues in the conduct of research

(K.70) Ethical issues in supervision

(K.71) Ethical issues in technology assisted psychological services

Knowledge Statements Overall In the previous sections, you provided ratings for individual knowledge statements. We also want you to rank the overall domains in terms of importance in your work as a Psychologist. In the following question, please rank the overall domains in terms of importance to your practice. You can review the knowledge statements in each of the domains before you make your ratings.

You can either rank the statements separately via the dropdown or drag the statements into the desired order (with 1 or the top position representing "most important" and 8 or the bottom position representing "least important"). No two tasks can have the same rank.

Overall Domain Ratings

Biological Bases of Behavior

Cognitive-Affective Bases of Behavior

Social and Cultural Bases of Behavior

Growth and Lifespan Development

Assessment and Diagnosis

Treatment, Intervention, and Prevention and Supervision

Research Methods and Statistics

Ethical/Legal/Professional Issues

To what extent did the statements listed on the previous pages cover all of the knowledge required to perform your work as a Psychologist?

Completely Covered

Adequately Covered

Inadequately Covered

Competencies Introduction

Competencies

You have been randomly selected to answer questions about the competencies and behavioral exemplars required for the practice of psychology. The competency clusters are presented in a random order. The numbering of the competencies and behavioral exemplars represents the original order as constructed. Due to the randomization of competency clusters, this numbering might appear to be out of order.

Please rate each of the competencies and behavioral exemplars using the scales displayed to the best of your ability and as related to your practice of psychology. Competencies

Please rate each competency and behavioral exemplar statement using the following four scales:

1. Frequency How frequently did you perform the competency or behavior in your practice during the past year? (Never or very rarely (e.g., Annually); Infrequently (e.g., Monthly or Quarterly); Frequently (e.g., Weekly); Very Frequently (e.g., Daily)) 2. Criticality How critical is possessing the competency or behavior to protecting the patient/client/public from harm? (Not critical; Minimally critical; Moderately critical; Highly critical) 3. Importance How important was performance of the competency or behavior to your practice as a psychologist during the past year? (Not important; Minimally important; Moderately important; Very important) 4. Acquisition At what point were you able to demonstrate this competency or behavior? (Prior to independent practice as a Psychologist; After independent practice as a Psychologist) Scientific Orientation Competencies

Frequency Criticality Importance Acquisition

(C.01) Select relevant research literature and critically review its assumptions, conceptualization, methodology, interpretation, and generalizability

(C.02) Interpret, evaluate, and integrate results of assessment activities within the context of scientific/professional knowledge to formulate and reformulate working hypotheses, conceptualizations, and recommendations

(C.03) Articulate a rationale for decisions and psychological services that rely on objective supporting data (e.g., research results, base rates, epidemiological data)

(C.04) Acquire and disseminate knowledge in accord with scientific and ethical principles

(C.05) Select and use evidence-based assessment methods and instruments

(C.06) Select and use evidence-based interventions

Scientific Orientation Behavioral Exemplars

Frequency Criticality Importance Acquisition

(T.01) Critically evaluate and apply research findings to practice, with attention to its applicability and generalizability Frequency Criticality Importance Acquisition

(T.02) Interpret and communicate empirical research results in a manner that is easily understood by non-scientific audiences

(T.03) Identify and collect applicable assessment data and integrate with theoretical models to develop working hypotheses

(T.04) Synthesize client-specific and scientific data with contextual factors to refine working hypotheses and develop conclusions and recommendations across a range of problems

(T.05) Reformulate working hypotheses and recommendations based on emerging data

(T.06) Articulate evidence-based rationale for decisions, recommendations, and opinions to clients, professionals, and the public

(T.07) Make and implement decisions and recommendations that integrate client-specific, social/contextual, and scientific data

(T.08) Identify areas where biases and heuristics may interfere with effective services Frequency Criticality Importance Acquisition

(T.09) Critically evaluate the literature relevant to professional practice

(T.10) Share psychological knowledge with diverse groups (e.g., students, colleagues, clients, other professionals, the public) within professional settings in an unbiased manner

(T.11) Select tests based upon the constructs to be assessed, the psychometric properties of the tests, and the client population

(T.12) Integrate and analyze client needs, practical constraints, and assessment questions into the selection of assessment methods

(T.13) Conceptualize interventions for client presentations guided by research and theory

(T.14) Regularly evaluate effectiveness of interventions

Competencies

Please rate each competency and behavioral examplar statement using the following four scales:

Professional Practice Competencies

Frequency Criticality Importance Acquisition

(C.07) Apply knowledge of individual and diversity characteristics in assessment and diagnosis

(C.08) Demonstrate effective interviewing skills

(C.09) Administer and score instruments following current guidelines and research

(C.10) Interpret and synthesize results from multiple sources (e.g., multiple methods of assessment, written documentation, interviewees, collateral sources of information) following current guidelines and research

(C.11) Formulate diagnoses, recommendations, and/or professional opinions using relevant criteria and considering all assessment data Frequency Criticality Importance Acquisition

(C.12) Communicate assessment results to clients, referral sources, and other professionals in an integrative manner

(C.13) Evaluate service or program effectiveness across a variety of contexts

(C.14) Select interventions for clients based on ongoing assessment and research evidence as well as contextual and diversity factors

(C.15) Apply and modify interventions based on ongoing assessment, research, contextual factors, client characteristics, and situational and environmental variables

(C.16) Consult and collaborate within and across professions

(C.17) Ensure compliance with policies and procedures of the practice/organization, the profession, and the jurisdiction

(C.18) Monitor, evaluate, and accurately and sensitively communicate supervisee performance to the supervisee, the organization, and the jurisdiction as needed

(C.19) Create and maintain a supportive environment in which effective supervision occurs for trainees and other professionals being supervised Professional Practice Behavioral Exemplars

Frequency Criticality Importance Acquisition

(T.15) Integrate knowledge of client characteristics in formulating assessment questions and understanding the reason for assessment.

(T.16) Select assessment methods and instruments based on available normed data and/or criterion- reference standards, and address any limitations in that selection.

(T.17) Ensure that professional opinions, recommendations, and case formulations adequately reflect consideration of client characteristics

(T.18) Adapt interview questions and behaviors in light of the characteristics of the interviewer and interviewee

(T.19) Demonstrate flexible, empathic, and accurate use of a broad range of interview techniques

(T.20) Consider contextual information (e.g., reason for assessment, possible legal or forensic considerations) in conducting an interview Frequency Criticality Importance Acquisition

(T.21) Administer, score, and interpret a range of commonly used standardized assessment methods and instruments

(T.22) Adapt relevant guidelines in situations requiring non- standard administration, scoring, interpretation, or communication of assessment results

(T.23) Interpret and integrate results from standardized tests and interviews following established guidelines and, as appropriate, multiple applicable norm sets

(T.24) Identify the strengths and limitations of various types of assessment data

(T.25) Reconcile or explain discrepancies between various sources of data and suggest alternative interpretations or explanations in light of any limitations of assessment instruments

(T.26) Use multi- modal assessment data and theory to formulate an understanding of the client

(T.27) Formulate diagnoses using current taxonomies Frequency Criticality Importance Acquisition

(T.28) Provide recommendations that incorporate client and contextual factors

(T.29) Write assessment reports in a timely and understandable manner

(T.30) Verbally communicate results from assessments in a timely and understandable manner

(T.31) Develop plans for evaluating service or program effectiveness

(T.32) Assess outcome effectiveness in an ongoing way

(T.33) Conceptualize intervention or treatment on the basis of evidenced-based literature

(T.34) Integrate client/stakeholder opinions, preferences, readiness for change, and potential for improvement into intervention plan

(T.35) Continually evaluate, modify, and assess the effectiveness of interventions, considering all relevant variables

(T.36) Consult with qualified peers when facing the need to modify interventions in unfamiliar situations Frequency Criticality Importance Acquisition

(T.37) Tailor consultation requests and provision of information based on knowledge of others’ professional needs and viewpoints

(T.38) Use evidence-based psychological theories, decision- making strategies, and interventions when consulting

(T.39) Continually evaluate, modify, and assess the effectiveness of consultation, considering all relevant variables

(T.40) Provide a supervision plan that details the supervisory relationship and the policies and procedures of supervision, including procedures to manage high-risk situations

(T.41) Identify responsibilities of supervisees towards clients, including informed consent and supervisory status

(T.42) Regularly provide behaviorally anchored feedback about supervisee strengths and areas that need further development Frequency Criticality Importance Acquisition

(T.43) Assure that supervisees who are trainees practice within the scope of supervisor’s competence and license and promote professional development

(T.44) Attend to the interpersonal process between supervisor and supervisees

(T.45) Monitor possible multiple roles or conflicts of interest and work toward resolution if required

Competencies

Please rate each competency and behavioral examplar statement using the following four scales:

Frequency Criticality Importance Acquisition

(C.20) Integrate and apply theory, research, professional guidelines, and personal understanding about social contexts to work effectively with diverse clients

(C.21) Work effectively with individuals, families, groups, communities, and/or organizations

(C.22) In all areas of professional practice, demonstrate respect for others, including those with differing viewpoints

(C.23) Identify and manage interpersonal conflict between self and others

Relational Competence Behavioral Exemplars

Frequency Criticality Importance Acquisition

(T.46) Recognize, understand, and monitor the impact of one’s own identities in professional situations Frequency Criticality Importance Acquisition

(T.47) Engage in respectful interactions with an awareness of individual, community, and organizational differences

(T.48) Modify one’s own behavior based on self- reflection and an understanding of the impact of social, cultural, and organizational contexts

(T.49) Follow professional guidelines and the scientific literature, when available, for providing professional services to diverse populations

(T.50) Apply culturally appropriate skills, techniques, and behaviors with an appreciation of individual differences

(T.51) Use relational skills to engage, establish and maintain working relationships with a range of clients Frequency Criticality Importance Acquisition

(T.52) Communicate respectfully, showing empathy for others

(T.53) Collaborate effectively in professional interactions

(T.54) Consider differing viewpoints held by clients and others

(T.55) Respond to differing viewpoints by seeking clarification to increase understanding before taking action

(T.56) Manage difficult and complex interpersonal relationships between self and others

(T.57) Seek experienced peers’ feedback to examine, and modify if necessary, one’s own reactions and behavior when managing interpersonal conflict

Competencies

Please rate each competency and behavioral examplar statement using the following four scales: 1. Frequency How frequently did you perform the competency or behavior in your practice during the past year? (Never or very rarely (e.g., Annually); Infrequently (e.g., Monthly or Quarterly); Frequently (e.g., Weekly); Very Frequently (e.g., Daily)) 2. Criticality How critical is possessing the competency or behavior to protecting the patient/client/public from harm? (Not critical; Minimally critical; Moderately critical; Highly critical) 3. Importance How important was performance of the competency or behavior to your practice as a psychologist during the past year? (Not important; Minimally important; Moderately important; Very important) 4. Acquisition At what point were you able to demonstrate this competency or behavior? (Prior to independent practice as a Psychologist; After independent practice as a Psychologist)

Professionalism Competencies

Frequency Criticality Importance Acquisition

(C.24) Identify and observe boundaries of competence in all areas of professional practice

(C.25) Critically evaluate one’s own professional practice through self- reflection and feedback from others

Professionalism Behavioral Exemplars

Frequency Criticality Importance Acquisition

(T.58) Identify limits of professional competence by recognizing strengths and weaknesses in practice areas Frequency Criticality Importance Acquisition

(T.59) Use knowledge of personal strengths and weaknesses to guide scope of practice

(T.60) Seek appropriate consultation when unsure about one’s competence and additional needs for training and development

(T.61) Seek additional knowledge, training, and supervision when expanding scope of practice

(T.62) Update knowledge and skills relevant to psychological practice on an ongoing basis

(T.63) Engage in systematic and ongoing self- assessment and skills development

(T.64) Accept personal responsibility when mistakes or oversights occur and take appropriate corrective action if needed Frequency Criticality Importance Acquisition

(T.65) Maintain awareness of personal factors that may impact professional functioning

Competencies

Please rate each competency and behavioral examplar statement using the following four scales:

Frequency Criticality Importance Acquisition

(C.26) Demonstrate and promote values and behaviors commensurate with standards of practice, including ethics codes, laws, and regulations

(C.27) Accurately represent and document work performed in professional practice and scholarship

(C.28) Implement ethical practice management

(C.29) Establish and maintain a process that promotes ethical decision- making

Ethical and Legal Practice Behavioral Exemplars

Frequency Criticality Importance Acquisition

(T.66) Demonstrate integration and application of ethics codes and laws in all professional interactions

(T.67) Communicate ethical and legal standards in professional interactions Frequency Criticality Importance Acquisition

(T.68) Seek professional consultation on ethical or legal issues when needed

(T.69) Discuss with peers or organizations any ethical concerns with their behavior

(T.70) Take appropriate steps to resolve conflicts between laws/rules and codes of ethics in one’s practice

(T.71) Maintain complete and accurate records

(T.72) Report research results accurately, avoiding personal biases

(T.73) Ensure adequate and appropriate credit is given to trainees and collaborators in scholarship

(T.74) Develop a plan commensurate with laws, ethical guidelines, and fiscal constraints to manage professional activities

(T.75) Practice in a fiscally and ethically sound manner Frequency Criticality Importance Acquisition

(T.76) Systematically identify the ethical and legal issues and conflicts that occur in one’s professional practice

(T.77) Consult with peers to aid in ethical decision- making when appropriate

(T.78) Engage in critical analysis of identified ethical issues and proactively address them

(T.79) Model ethical decision- making to others

Competencies

Please rate each competency and behavioral examplar statement using the following four scales:

Frequency Criticality Importance Acquisition

(C.30) Work effectively within organizations and systems

(C.31) Demonstrate interdisciplinary collaborations

Systems Thinking Behavioral Exemplars

Frequency Criticality Importance Acquisition

(T.80) Recognize the organizational and systemic factors that affect delivery of psychological services

(T.81) Utilize knowledge of organizations and systems to optimize delivery of psychological services

(T.82) Collaborate with various healthcare professions to meet common client goals

(T.83) With other healthcare professionals, integrate services to meet common client goals

(T.84) Collaborate with representatives from other professional groups or systems in the provision of psychological services Competency Tasks Overall

In the previous sections, you provided ratings for individual competencies and task statements. We also want you to rank the overall domains in terms of importance in your work as a Psychologist. In the following question, please rank (with 1 or the top position representing "most important" and 6 or the bottom position representing "least important") the overall domains in terms of importance to your practice. You can review the task statements in each of the domains before you make your ratings.

You can either rank the statements separately via the dropdown or drag the statements into the desired order (with top representing "most important" and bottom representing "least important"). No two tasks can have the same rank.

Overall Domain Ratings

Scientific Orientation

Professional Practice

Relational Competence

Professionalism

Ethical and Legal Practice

Systems Thinking

To what extent did the statements listed on the previous pages cover all of the tasks required to perform your work as a Psychologist?

Completely Covered

Adequately Covered

Inadequately Covered

Demographic and Professional Information

This information is being collected for statistical analysis and to ensure sample representativeness.

Number of years practicing as a psychologist licensed (US) or registered (Canada) at the independent (autonomous) level: Highest level of education:

Master's degree

Graduate work beyond Master's degree (at least 12 semester hours or equivalent)

Coursework for Doctorate completed

PsyD

EdD

PhD

Other Degree (please specify)

Was your doctoral program APA or CPA accredited at the time you graduated?

Yes

Don't know

Major area of training:

Child Clinical Psychology General Psychology

Clinical Psychology Health Psychology

Clinical Geropsychology Industrial/Organizational Psychology

Clinical Neuropsychology Neurosciences

Cognitive Psychology Perception/Learning

Community Psychology Personality Psychology

Comparative Psychology Physiological Psychology/Psychobiology

Consulting Psychology Psychopharmacology

Counseling Psychology Quantitative/Mathematical/Psychometrics/Statistics

Developmental Psychology Rehabilitation Psychology

Educational Psychology School Psychology

Environmental Psychology Social Psychology

Experimental Psychology Sports Psychology

Forensic Psychology

Other (please specify)

Participation in a formal postdoctoral training program:

Yes

No Participation in a formal respecialization program in clinical, counseling, or school psychology:

Yes

Demographic and Professional Information

This information is being collected for statistical analysis and to ensure sample representativeness.

Current major area(s) of practice: (check all that apply)

Child Clinical Psychology General Psychology

Clinical Psychology Health Psychology

Clinical Geropsychology Industrial/Organizational Psychology

Clinical Neuropsychology Medical Psychology

Cognitive Psychology Neurosciences

Community Psychology Perception/Learning

Comparative Psychology Personality Psychology

Consulting Psychology Physiological Psychology/Psychobiology

Counseling Psychology Psychopharmacology

Developmental Psychology Quantitative/Mathematical/Psychometrics/Statistics

Educational Psychology Rehabilitation Psychology

Environmental Psychology School Psychology

Experimental Psychology Social Psychology

Forensic Psychology Sports Psychology

Other (please specify)

Certified by ABPP:

Yes

Demographic and Professional Information

This information is being collected for statistical analysis and to ensure sample representativeness. Area(s) in which you are certified by ABPP: (check all that apply)

Behavioral & Cognitive

Clinical

Clinical Child & Adolescent

Clinical Health

Clinical Neuropsychology

Pediatric Clinical Neuropsychology Subspecialty

Counseling

Couple & Family

Forensic

Geropsychology

Group

Organizational & Business Consulting

Police & Public Safety

Psychoanalysis

Rehabilitation

School

Demographic and Professional Information

This information is being collected for statistical analysis and to ensure sample representativeness.

Employment setting: (check all that apply)

University setting (not medical school) Other hospital setting

Four-year college Independent practice

Medical school Counseling center

Other academic setting Community clinic

School/other educational settings Other human service setting

VA Medical Center Business/government setting

Military hospital Consulting practice

Psychiatric hospital Not currently employed

Other setting (please specify) Primary theoretical orientation:

Behavioral Systems

Cognitive/Behavioral Integrative

Existential/Humanistic Person-Centered

Interpersonal Other orientation

Psychodynamic Not relevant

Secondary theoretical orientation:

Behavioral Systems

Cognitive/Behavioral Integrative

Existential/Humanistic Person-Centered

Interpersonal Other orientation

Psychodynamic Not relevant

Demographic and Professional Information

This information is being collected for statistical analysis and to ensure sample representativeness. Area(s) of expertise: (check all that apply)

Academic Practice Interdisciplinary Systems

Adult Psychology Lesbian/Gay/Bisexual/Transgender Psychology

Assessment/Evaluation Management

Behavioral Psychology Marriage and Family Psychology

Career/Vocational Psychology Intellectual/Developmental Disabilities

Child/Adolescent Psychology Multicultural Psychology/Ethnic Minority Psychology

Clinical Geropsychology/Aging Neuroscience

Clinical Neuropsychology-Adult Pain Management

Clinical Neuropsychology-Child Pediatric Psychology

Clinical Psychology Personality Psychology

Cognitive Psychology Physiological Psychology/Psychobiology

Community Psychology Prevention

Comparative Psychology Psychoanalysis

Consultation Psychology of Women

Consulting Psychology Psychopharmacology

Consumer Psychology Quantitative/Mathematical/Psychometrics/Statistics

Counseling Psychology Rehabilitation Psychology

Developmental Psychology Research and Evaluation

Educational Psychology Research Methods and Statistics

Environmental Psychology Serious/Chronic Mental Illness

Ethical/Legal/Professional Issues School Psychology

Experimental Psychology Social and Multicultural Basis of Behavior

Family Psychology Social Psychology

Forensic Psychology Specialized Assessment Techniques

General Psychology/Methods & Systems Sports Psychology

Growth and Lifespan Development Substance Abuse

Health Psychology Supervision

Industrial/Organizational Psychology Telepsychology Experiences you have had as a psychologist during the past three years: (check all that apply)

Served as the principal investigator or co-investigator on a research grant/contract

Provided health/mental health services to a client pro bono (free of charge)

Provided other services (e.g. expert witness, research, teaching, organization consultation) on a pro bono basis

Served as a paid consultant to another institution, business, government agency, or organization (exclude direct clinical)

Served as a consultant on a grant/contract

Served as an expert witness in a court proceeding

Appeared in media (e.g. television, radio, web-based platform) as a psychologist

Used social media for practice purposes

Served on a review group for a government agency or private foundation, to review grant proposals for research, training

Been involved in legislative/lobbying activities

Presented at an international psychology convention

Participated in any professional psychology association activities as committee, board, or other governance group member

Served on a licensure or regulatory board

Served as a clinical supervisor to trainee

Served as a clinical supervisor to other licensed professional

Served as an administrative supervisor

Provided telepsychological services

Demographic and Professional Information

This information is being collected for statistical analysis and to ensure sample representativeness.

Gender:

Male Transgender Male

Female Transgender Female

Not listed (please specify) Racial Background:

Asian including South, Southeast and Hispanic or Latino More than one race West Asia Native American or Indigenous People Black or African American Native Hawaiian or Pacific Islander White

Not listed (please specify)

Do you consider yourself to have a disability as defined by US Americans with Disabilities Act/Canadian Human Rights legislation?

Yes

Demographic and Professional Information

This information is being collected for statistical analysis and to ensure sample representativeness.

What type of Disability? (check all that apply)

Blind or visually impaired

Deaf or hard of hearing

Physical or Orthopedic

Learning

Cognitive

Psychological

None

Not listed (please specify)

Changes

The next few questions provide you space to share your thoughts as to any potential future changes to the knowledge or competencies required of psychologists in the next 5-10 years. Describe one long-range change that you see occurring in the practice of psychology over the next 5-10 years.

If you responded to the previous question, please classify the change to one of the eight listed knowledge areas and/or one of the six listed competency areas.

Knowledge Area

Biological Bases of Behavior

Cognitive-Affective Bases of Behavior

Social and Cultural Bases of Behavior

Growth and Lifespan Development

Assessment and Diagnosis

Treatment, Intervention, and Prevention and Supervision

Research Methods and Statistics

Ethical/Legal/Professional Issues

None

Not listed (please specify)

Competency Area

Scientific Orientation

Professional Practice

Relational Competence

Professionalism

Ethical and Legal Practice

Systems Thinking

None

Not listed (please specify)

Thank You

Thank you for participating in the ASPPB test development process. We appreciate the time you have taken out of your busy schedule to help us in validating the examination process for psychology licensure. If you would like to be included in a drawing for one of twenty $100 gift cards, please provide your contact information below. Your personal information will not be exported with or linked in any way to your survey responses.

Contact Information

First Name

Last Name

Email Address