Challenges to Computer Science Education Research Vicki L
Total Page:16
File Type:pdf, Size:1020Kb
Challenges to Computer Science Education Research Vicki L. Almstrum Orit Hazzan (moderator) Department of Computer Sciences Department of Education in Technology & Science The University of Texas at Austin Technion – Israel Institute of Technology [email protected] [email protected] Mark Guzdial Marian Petre College of Computing Faculty of Mathematics and Computing Georgia Institute of Technology The Open University [email protected] [email protected] Categories and Subject Descriptions: K.3.0 PANELISTS’ STATEMENTS [Computers and Education] - general. Vicki L. Almstrum A key challenge that prevents many computing educators from General Terms: Human Factors. attempting CSE research is isolation. Isolation can result from Keywords: research in computer science education, research existing in an atmosphere where colleagues regularly convey the attitude that educational research is not real research. Isolation methods, qualitative methods, quantitative methods. can come from uncertainty about where to begin digging into the literature from computer science education as well as from a wide SUMMARY variety of related and supporting fields. Isolation can be like a In recent years, the theme of research in computer science void into which ideas for research questions, designs, and education (CSE) has received a relatively large share of attention analyses simply disappear without the benefit of collegial in the CSE community [1, 2, 3, 4, 5]. Some specific activities discussion and brainstorming. A sense of isolation can emerge conducted in this direction include: from frustrating struggles while dealing with special situations, • such as small sample sizes, the need for specialized methods, or 12/02: Nell Dale's summary of her SIGCSE proceedings difficulty in locating or creating appropriate instruments. review: "Beginning in 1998, there is a definite increase in CS Ed research related papers." [2] Combating this sense of isolation requires patience, persistence, and creativity. Creating a receptive support network outside of • 06/03: Launch of the Scaffolding Research in CS Education one's own department can help. For example, partnering with an hands-on workshop, aimed at introducing higher-education academic from a different department at one's own institution faculty to research in CSE (depts.washington.edu/srcse/). (such as communication, psychology, or mathematics education) • 03/04: Publication of Sally Fincher and Marian Petre's edited can provide inspiration and productive intermixing of specialties book Computer Science Education Research [3]. — if an appropriate partner can be found. Setting up collaborations with computing educators (or folks in other areas) • 12/04: Publication of the special issue of Computer Science at other institutions can result in anything from a one-on-one Education about import/export relationships to CSE research interaction to a multi-institutional cooperative endeavor. On-line [1], focusing on mutual relationships of CSE research with forums can also help pull an isolated researcher into richer research in other educational fields. interactions. With some imagination, practitioners in isolated situations can find research approaches leading to doable studies Based on our experience with research in CSE, we suggest that that add to the collection of knowledge about what works in the community of CSE researchers should consider future computing education. challenges. The panelists will present four challenges, illustrating each with specific cases and research studies. Open discussion with the audience will follow the panelists' short presentations. Mark Guzdial The real challenge to computing education is to avoid the temptation to re-invent the wheel. Computers are a revolutionary human invention, so we might think that teaching and learning about computers requires a new kind of education. That’s completely false: The basic mechanisms of human learning haven’t changed in the last 50 years. Too much of the research in computing education ignores the hundreds of years of education, cognitive science, and learning sciences research that have gone before us. We know that student opinions are an unreliable measure of learning or teaching quality. We know that meta-analyses are very hard to do with any sort of Copyright is held by the author/owner(s). rigor, so there are careful, formal procedures for doing them right. SIGCSE'05, February 23-27, 2005, St. Louis, Missouri, USA. We know that the educational value of algorithm animations is ACM 1-58113-997-7/05/0002. 191 subtle, and we’re not going to be able to measure the potential wish to address and by an understanding of the nature of evidence. benefit from whole-class, quantitative studies. How we borrow it should be shaped by vigilant striving for the greatest possible rigour. If we want our research to have any value to the researchers that come after us, if we want to grow a longstanding field that There are some key requirements in borrowing and employing contributes to the improvement of computing education, then we methods, among them: have to “stand on the shoulders of giants,” as Newton put it, and stop erecting ant hills that provide too little insight. understanding the method in its context: It is essential, in borrowing methods, that we understand the epistemology, focus, and assumptions that underpin them. It’s not enough to borrow a Orit Hazzan method without understanding how it is applied – and constrained While the theme of research in CS education has recently received – in its discipline of origin, and how that tradition shapes the a lot of attention, most of the research conducted in CSE uses method and the evidence it can yield. quantitative research tools. My presentation addresses the challenge of diversifying the research methods we employ. seeking rigour: Weak or sloppy evidence does no one any good. Specifically, I illustrate how qualitative research, which has been And so we must concern ourselves with repeatability of studies. used extensively in other educational research fields, may be We must be vigilant against bias. We must address issues of applied in CSE research. I address the nature of qualitative representativeness and generalisation explicitly. We must consider research, its fitness for use in the research in CSE, and its research alternative accounts and potentially contradictory evidence. tools. Whatever the method, it requires full and honest reporting, with articulation of the reasoning that connects question to data to Qualitative research can be used to investigate environments, interpretation to conclusions, so that the whole ‘audit trail’ is situations, and processes for which quantitative data cannot exposed to scrutiny and potentially to repetition. adequately describe their complexity. For example, how can numerical data describe mental processes involved in learners' accumulating evidence: It’s crucial to remember that no one study comprehension of object-oriented concepts, or student interaction stands on its own; one must also understand how the study fits in software development processes? I claim that though into the body of existing work. How might the results of one study quantitative data can shed light on several aspects of such accumulate with those of other studies, how might findings be processes, they cannot provide a full picture of what goes on in compared to other evidence, and how might findings be validated? such situations. I do not argue that qualitative data analysis can describe such complex processes entirely; I do argue, however, REFERENCES that qualitative data analysis highlights additional important [1] Almstrum, V., Hazzan, O. and Ginat, D. (in press, December aspects of these processes. 2004). Special Issue on Import/Export Relationships to Qualitative research has different objectives than those of Computer Science Education Research, Computer Science quantitative research. Accordingly, its data-gathering tools are Education 15(3). different. Specifically, since the products of qualitative research [2] Dale, N. (2002). Increasing interest in CS ED research, are descriptive, the data gathering tools, such as interviews and SIGCSE Bulletin inroads 34(4), pp. 16-17. observations, are verbal as well; because qualitative research aims at describing a given situation, rather than at generalizing a [3] Fincher, S. and Petre, M. (2004). Computer Science theory, in qualitative research it is sufficient to focus on a small Education Research, Routledge Falmer. number of individuals who participate in the research field. [4] Goldweber, M., Clark, M. and Fincher, S. (2004). The relationships between CS education research and the My presentation will be accompanied with examples of research SIGCSE community. SIGCSE Bulletin inroads 36(1), pp. works that illustrate the use of different qualitative research tools. 147-148. Marian Petre [5] Valentine, D. W. (2004) CS Educational Research: A meta- analysis of SIGCSE Technical Symposium proceedings. In the absence of a driving theory and an established SIGCSE Bulletin inroads 35(1), pp. 255-259. methodology, we borrow research methods from other disciplines. I take a ‘horses for courses’ view of research design for CS education. What we borrow should be shaped by the questions we 192.