Learning Efficiency Correlates of Using Supermemo with Specially Crafted Flashcards in Medical Scholarship
Total Page:16
File Type:pdf, Size:1020Kb
Learning efficiency correlates of using SuperMemo with specially crafted Flashcards in medical scholarship. Authors: Jacopo Michettoni, Alexis Pujo, Daniel Nadolny, Raj Thimmiah. Abstract Computer-assisted learning has been growing in popularity in higher education and in the research literature. A subset of these novel approaches to learning claim that predictive algorithms called Spaced Repetition can significantly improve retention rates of studied knowledge while minimizing the time investment required for learning. SuperMemo is a brand of commercial software and the editor of the SuperMemo spaced repetition algorithm. Medical scholarship is well known for requiring students to acquire large amounts of information in a short span of time. Anatomy, in particular, relies heavily on rote memorization. Using the SuperMemo web platform1 we are creating a non-randomized trial, inviting medical students completing an anatomy course to take part. Usage of SuperMemo as well as a performance test will be measured and compared with a concurrent control group who will not be provided with the SuperMemo Software. Hypotheses A) Increased average grade for memorization-intensive examinations If spaced repetition positively affects average retrievability and stability of memory over the term of one to four months, then consistent2 users should obtain better grades than their peers on memorization-intensive examination material. B) Grades increase with consistency There is a negative relationship between variability of daily usage of SRS and grades. 1 https://www.supermemo.com/ 2 Defined in Criteria for inclusion: SuperMemo group. C) Increased stability of memory in the long-term If spaced repetition positively affects knowledge stability, consistent users should have more durable recall even after reviews of learned material have ceased. Study design This non-randomized controlled intervention study will be conducted during the 2021-2022 anatomy class at the Padua Faculty of medicine in Italy. The study is divided into two phases. At the time of writing, the authors are waiting for Padua university's thesis committee to arbitrate which data are permissible in this study. To accommodate this uncertainty, we define several scenarios in a later chapter. Phase one In parallel with their normal scholarship, students will be distributed into two groups. The SuperMemo Group (SG) will study their anatomy course material using flashcards3 that have been collaboratively prepared by the students4 of this group and published on the SuperMemo (SM) web platform. The Traditional Group (TG) or control group will study anatomy course material using traditional methods of learning5. All students of the 2021-2022 anatomy class will participate in the study, and each student will have the choice of joining either the TG or the SG. A presentation will be given before the beginning of the course’s term to introduce SuperMemo, the principles of spaced repetition, and the objectives of the study. Depending on the scenario, the authors of this study may conduct additional simulation tests involving some or all of the students participating in phase one. These simulation tests will cover the same material as the official examinations given by the Padua university. If conducted, these tests will provide a more accurate and detailed measure of the students’ understanding of the course material. Samples from the TG and SG will be analysed and compared to test hypotheses A and B. The sample design for phase one is a non-randomized, two-group, unpaired relative comparison. No blinding will be involved. 3 A card bearing information, each with a question and an answer field. 4 Defined in Course material. 5 Defined in Traditional learning. Phase two In the aftermath of their 2020-2021 scholarship, students participating in phase one will be offered the opportunity to participate in a follow-up test. The test will cover the same questions as the simulation tests conducted in phase one. Samples from the TG and SG will be analysed and compared to test hypothesis C. The sample design for phase one is a non-randomized, two-group, paired relative comparison with repeated measures. Considerations about samples The target population (TP) of this study are medicine students who employ traditional learning methods. We will infer properties about the TP from one population and two sub-populations: 1. Population 1: all students from the 2021-2022 anatomy class (phase one), 2. Sub-population 1: a subset of students from population 1 participating in the simulation tests, 3. Sub-population 2: a subset of students from population 1 participating in the follow-up test (phase 2). Within these populations we define an additional dimension for enrolment within either the SuperMemo group or the traditional group. For the remainder of this paper, we define the terminology: • Sampled population or sub-population (SP) refers to the entire group of students participating in the test under consideration, • Traditional group (TG) and SuperMemo group (SG) refer to the subset of students within a SP filtered by their membership in either group. We use this compartmentalization to systematically consider errors introduced by each layer of the selection process. For example, the selection process for student scholarships differs significantly across academic institutions. Prestigious universities often exercise strict control on surface criteria such as academic success or financial means, which may correlate with features such as personal motivations or social status. Therefore, it is not evident that students of a given institution are representative of the general population. Furthermore, any subsequent sample would inherit the same attributes and likely produce biased results. We address these concerns in the later chapters about the representativity of samples. We begin by defining the qualities of TP and SP, as well as SG and TG. Each of these sections will also discuss the possible biases introduced at each stage of the study. Sample Population Selection Units Measurement Inferred Samples data Estimation Target population, sampled populations and sub-populations SP 1 consists of the entire 2021-2022 anatomy class. Samples from this population will be in the form of two sets of grades, corresponding to exams taken in July and September 2021. They are conducted by the Padua faculty of medicine and are mandatory for graduation. SP 2 consists of a subset of students from SP 1 that are participating in the simulation tests. Samples from this population will be in the form of grades corresponding to the successive simulation test. Depending on the scenario, the simulation tests may occur 3 to 21 days before or after Padua university's official tests. Questions will cover the same corpus of knowledge as the official examinations. The simulation tests do not have any direct6 influence on the graduation process. SP 3 consists of a subset of students from SP1 participating in a follow-up test. Participation will be voluntary. Samples from this population will be in the form of a single set of grades, and 6 No influence other than helping students assess their readiness for the official test, if taken beforehand. questions will target the entire corpus of knowledge from the July and September exams. It is conducted by authors of this study, and it will not have any influence on the students’ graduation process. All of these sampled populations and sub-populations are further subdivided within two groups: the TG and the SG. TARGET POPULATION Sampled population 1 Faculty’s anatomy exams Not included in Sub-population 1 sampling frame Simulation tests Sub-population 2 Students not part of 6 months follow- Padua’s 2021-2022 up test anatomy class. Size: Unknown Size: Unknown Size: 336 students Size: more than 90000 The size of the TP is a conservative estimate based on a paper by (Curtoni & Sutnick, 1995): There is approximately one first-year medical student per 10,000 people in Europe, compared with one per 15,000 people in the United States. Assuming at least similar proportions of first-year students, given the 1995 populations of 267 million in the U.S. and 728 million in Europe, and given the strict growth of their respective populations after 1995 based on public data for years 1995-2020, we calculate: 267000000 728000000 푐푎푟푑(푇푃) > + > 90,600 15000 10000 We expect yet larger numbers for the TP when factoring in countries and continents other than America and Europe. Moreover, this estimate only includes first-year students. Medical scholarships often last many years, therefore the number of students who could benefit from the learning treatment at any given time would be significantly greater than 90,600. At the time of writing, the sizes of SP 2 and 3 are unknown. If no form of incentive is offered in exchange for participation, we expect their levels to be low. Representativity of SP 1 To evaluate the representativity of SP 1 with respect to our hypotheses and the TP, we consider the following factors and whether they contribute to sampling and non-sampling error7: Non-sampling error: curriculum, teaching methods, studied discipline, geographic location, and randomness of scholarship. Curriculum may differ between medicine faculties. Namely, the Padua faculty of medicine is a prestigious university, and its curriculum may differ significantly from other institutions. Therefore, the efficacy of the learning treatment may differ as well. The applicability of spaced repetition as defined in this paper may vary by discipline. However, medical scholarship relies primarily on rote learning, and the learning treatment may generalize well to other classes. Padua is one of the wealthiest regions in Italy with a rich cultural history. The Padua faculty is a highly ranked medical universities in Italy, and tuition fees average around 2500€ per year. If data about the demographics background of students studying at the Faculty are made available for this study, we may examine how these parameters affect representativity. Ideally, the sampled population would have students from many schools.