Applying the Mathematical Work of Teaching Framework to Develop a Computer Science Pedagogical Content Knowledge Assessment Yvonne Kao Katie D’Silva Aleata Hubbard WestEd WestEd WestEd 400 Seaport Court, Suite 222 400 Seaport Court, Suite 222 400 Seaport Court, Suite 222 Redwood City, CA Redwood City, CA Redwood City, CA [email protected] [email protected] [email protected] Joseph Green Kimkinyona Cully WestEd WestEd 400 Seaport Court, Suite 222 400 Seaport Court, Suite 222 Redwood City, CA Redwood City, CA [email protected] [email protected] ABSTRACT KEYWORDS Pedagogical content knowledge (PCK) is specialized knowledge assessment; high school; pedagogical content knowledge; teacher necessary to teach a subject. PCK integrates subject-ma!er preparation content knowledge with knowledge of students and of teaching strategies so that teachers can perform the daily tasks of teaching. Studies in mathematics education have found ACM Reference format: correlations between measures of PCK and student learning. Kao, Y., D’Silva, K., Hubbard, A., Green, J., and Cully, K. 2018. Finding robust, scalable ways for developing and measuring Applying the Mathematical Work of Teaching Framework to computer science (CS) teachers’ PCK is particularly important in Develop and Computer Science Pedagogical Content Knowledge CS education in the United States, given the lack of formal CS Assessment. In SIGCSE ’18: 49th ACM Technical Symposium on teacher preparation programs and certi"cations. However, Computer Science Education, Feb. 21–24, 2018, Baltimore, MD, measuring pedagogical content knowledge is a challenge for all USA. ACM, NY, NY, USA, 6 pages. subject areas. It can be di#cult to write assessment items that https://doi.org/10.1145/3159450.3159521 elicit the di$erent aspects of PCK and there are o%en multiple appropriate pedagogical choices in any given teaching scenario. In this paper, we describe a framework and pilot data from a 1 INTRODUCTION questionnaire intended to elicit PCK from teachers of high Shulman introduced the idea of pedagogical content knowledge school introductory CS courses and we propose future directions (PCK) as a special category of knowledge for teaching [21]. for this work. Whereas subject-ma!er content knowledge is necessary for anyone who seeks to master a subject, PCK is only necessary for CCS CONCEPTS those who are teaching it. Aspects of PCK include knowledge of • Social and professional topics~Computer science how students think about concepts, knowledge of what makes education learning concepts di#cult, and knowledge of how to use di$erent content representations to make concepts understandable for others [21]. &is paper describes an e$ort to develop a framework for Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or understanding computer science teaching knowledge, drawing distributed for profit or commercial advantage and that copies bear this notice and from similar e$orts in mathematics education, and to develop the full citation on the first page. Copyrights for components of this work owned items that will elicit this knowledge from computer science by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, teachers. &e results reported here are a subset of the data requires prior specific permission and/or a fee. Request permissions collected for a larger project to document knowledge change in from [email protected]. SIGCSE '18, February 21–24, 2018, Baltimore, MD, USA novice computer science teachers. © 2018 Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-5103-4/18/02…$15.00 https://doi.org/10.1145/3159450.3159521 1.1 Pedagogical Content Knowledge in framework of CS PCK that focuses on both content knowledge Mathematics Education and the processes of teaching. CS education researchers have employed a variety of &ere is a large body of work in mathematics education to re"ne methods to study PCK. For example, several scholars used the the concept of PCK and develop measures for it. &is project CoRe (Content Representation) instrument to elicit educators’ aimed to adapt those methods into a computer science education ideas about teaching CS [4,7,13,18]. Interviews have been used to context. &e Learning Mathematics for Teaching Project [1] understand instructional strategies, the impact of lessons on elaborated and re"ned Shulman’s concept of PCK in teacher knowledge development, and how teachers judge the mathematics by analyzing the work of mathematics teachers and di#culty of topics for their students [2,11]. Ohrndorf and identifying how mathematical knowledge is used. &is paper Schubert [15,16] developed an assessment that asked teachers to focuses on four of these categories: explain possible student responses and misconceptions. Saeli et al. [19] employed a document analysis approach to analyze the 1.! Common content knowledge (CCK): the mathematical PCK information contained in CS textbooks. Doukakis and knowledge and skill that is common across disciplines that colleagues [6] developed an assessment for in-service CS use mathematics, such as performing calculations or teachers based on the Technological Pedagogical and Content otherwise solving mathematics problems. 1 Knowledge (TPACK) framework that included two items 2.! Specialized content knowledge (SCK): the mathematical targeting PCK [14]. In another paper, we described the knowledge and skill unique to teaching, such as evaluating development of think-aloud tasks we created to drive discussion whether a student’s non-standard approach to a problem is about teachers’ PCK [10]. While scholars have employed a sufficiently general. variety of methods to study and de"ne PCK in CS, the existing 3.! Knowledge of content and students (KCS): knowledge that body of research does not provide strong empirical evidence combines knowing about students and knowing about linking speci"c pedagogical practices to student learning mathematics, such as knowing what students will find outcomes [17]. Furthermore, there has yet to be a validated motivating or confusing. assessment of CS PCK that can be used to evaluate teachers’ 4.! Knowledge of content and teaching (KCT): knowledge that preparedness to teach an introductory CS course. combines knowing about teaching and knowing about &is lack of validated assessments of CS PCK presents a mathematics, such as understanding how to sequence topics challenge for evaluating the numerous e$orts around the United or choose representations for instruction. States to train more K-12 computer science teachers, especially given the lack of formal CS teacher preparation and certi"cation Hill and colleagues [8] then used this framework to develop programs [5]. How can we evaluate the PCK content of such assessments of Mathematics Knowledge for Teaching (MKT). professional learning programs? How can we tell when a teacher &e MKT focuses on teachers’ knowledge of content (both is equipped to teach computer science? A validated CS PCK common and specialized) and knowledge of students and assessment could help us address these questions and be!er content. Teachers’ performance on these assessments was found gauge the community’s e$orts at increasing the CS teacher to predict student learning gains in "rst and third grades, a%er workforce. controlling for students’ absence rates and demographic characteristics, classroom characteristics, and teachers’ 2 DEVELOPING AN ASSESSMENT OF CSPCK: professional backgrounds [9]. FRAMEWORK AND CONTENT 1.2 Pedagogical Content Knowledge in Developing measures of PCK that can be used to evaluate Computer Science Education individual teachers is a challenging proposition—even the &e concept of PCK has recently been applied to CS education Mathematics Knowledge for Teaching assessments are intended research. For example, in a study of experienced computing only for assessing groups of teachers for research purposes [20]. teachers learning to teach a new programming paradigm, In conducting this preliminary foray into creating such an assessment, we were inspired and guided by the Mathematical Liberman et al. [12] explored the connection between CS PCK Work of Teaching framework [20]. Selling and colleagues [20] and content knowledge. &ey found teachers entered a state of identi"ed four areas of di#culty for PCK assessment developers regressed expertise where they displayed elements of both novice and created a framework for navigating them. &e four areas of teaching and expert teaching. Buchholz et al. [4] provided di#culty are: anecdotal evidence that pre-service CS teachers progress through stages of complexity in their PCK when developing 1.! Developing items that measure SCK instead of exclusively teaching modules. Baxter’s [2] case study of two experienced CS measuring CCK, teachers, one formally trained and the other self-taught, demonstrated how PCK could vary across teachers with similar levels of expertise and how PCK was enacted di$erently in their 1 classrooms. Lastly, the KUI research project [3] created a TPACK is another extension of Shulman’s categories of teacher knowledge that incorporates teacher knowledge of technology and how to teach with technology. 2.! Developing items that measure SCK without touching on understand about the topic during an introductory lesson. &e pedagogical decision-making, inclusion of this