Connecting Symbolic Integrals to Physical Meaning in Introductory Physics

Dissertation

Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University

By

Nathaniel R. Amos, B.S., M.S.

Graduate Program in Physics

The Ohio State University

2017

Dissertation Committee:

Andrew Heckler, Advisor Lei Bao Robert Perry Richard Hughes c Copyright by Nathaniel R. Amos

2017 Abstract

This dissertation presents a series of studies pertaining to introductory physics students’ abilities to derive physical meaning from symbolic integrals (e.g., vdt) R and their components, namely di↵erentials and di↵erential products (e.g., dt and vdt, respectively). Our studies focus on physical meaning in the form of interpretations

(e.g., “the total displacement of an object”) and units (e.g., “meters”).

Our first pair of studies independently attempted to identify introductory-level mechanics students’ common conceptual diculties with and unproductive interpre- tations of physics integrals and their components, as well as to estimate the frequencies of these diculties. Our results confirmed some previously-observed incorrect inter- pretations, such as the notion that di↵erentials are physically meaningless; however, we also uncovered two new conceptualizations of di↵erentials, the “rate” (di↵eren- tials are “rates” or “derivatives”) and “instantaneous value” (di↵erentials are values of physical variables “at an instant”) interpretations, which were exhibited by more than half of our participants at least once.

Our next study used linear regression analysis to estimate the strengths of the inter-connections between the abilities to derive physical meaning from each of dif- ferentials, di↵erential products, and integrals in both first- and second-semester, calculus-based introductory physics. As part of this study, we also developed a highly reliable, multiple choice assessment designed to measure students’ abilities to connect

ii symbolic di↵erentials, di↵erential products, and integrals with their physical interpre- tations and units. Findings from this study were consistent with statistical mediation via di↵erential products. In particular, students’ abilities to extract physical meaning from di↵erentials were seen to be strongly related to their abilities to derive physical meaning from di↵erential products, and similarly di↵erential products to integrals; there was seen to be almost no direct connection between the abilities to derive physical meaning from di↵erentials and the abilities to derive physical meaning from integrals.

Our final pair of studies intended to implement and quantitatively assess the e- cacy of specially-designed instructional tutorials in controlled experiments (with sev- eral treatment factors that may impact performance, most notably the e↵ect of feed- back during training) for the purpose of promoting better connection between sym- bolic di↵erentials, di↵erential products, and integrals with their corresponding phys- ical meaning. Results from both experiments consistently and conclusively demon- strated that the ability to connect verbal and symbolic representations of integrals and their components is greatly improved by the provision of electronic feedback during training. We believe that these results signify the first instance of a large, controlled experiment involving introductory physics students that has yielded sig- nificantly stronger connection of physics integrals and their components to physical meaning, compared to untrained peers.

iii Dedicated to Jessica Amos and Nancy McArthur.

iv Acknowledgments

The number of people to whom I owe gratitude is too large to list here. My wife

(Jessica), my parents (Nancy, Hugh, Ronnie, and Paula), my grandparents (Jack,

Willena, Harvey, Nancy, Hugh, and Betsy), my siblings (Ben and Jada), my uncle

(Rob), and my wife’s family (Frances and Rosel) all shaped who I am today; I was lucky to be guided by their steadfast support, care, and wisdom. I am also grateful to my advisor, Andrew Heckler, for his personal and professional investment in my education and career; it has been an extremely valuable and influential four years.

Finally, my thanks are due to my fellow graduate students (most of whom have already gone before me), for their assistance, advice, mentorship, and camaraderie:

Brendon, Abby, Ryan, and Dan.

Everyone listed above, and many more, deserve to be acknowledged by far more than a single page in this dissertation. In this small way, I hope they recognize their importance to me in my life.

v Vita

5May2012...... B.S.Summa Cum Laude Physics, Uni- versity of Florida, Gainesville, FL 20 August 2014 ...... M.S. Physics, The Ohio State Univer- sity, Columbus, OH August 2013-December 2015 ...... Graduate Teaching Assistant, Depart- ment of Physics, The Ohio State University. August 2014-Present ...... Graduate Research Assistant, Depart- ment of Physics, The Ohio State University.

Publications

Research Publications

N. Amos and A. Heckler “Spatial Reasoning and the Construction of Integrals in Physics”. PERC Conference Proceedings,2014.

N. Amos and A. Heckler “Student Understanding and Construction of Di↵erentials in Introductory Physics”. PERC Conference Proceedings,2015.

Fields of Study

Major Field: Physics

vi Table of Contents

Page

Abstract...... ii

Dedication ...... iv

Acknowledgments...... v

Vita...... vi

List of Tables ...... xi

ListofFigures ...... xiii

1. Introduction ...... 1

1.1 Current Issues in Integration from Mathematics and Physics Educa- tion Research ...... 2 1.2 Research Goals & Contributions ...... 4 1.2.1 Goal 1: Investigating, Discovering, & Estimating Frequencies of Conceptual Obstacles from Unproductive Interpretations of Physics Integrals & Their Components ...... 4 1.2.2 Goal 2: Investigating Mediating & Moderating Relationships Between the Abilities to Derive Physical Meaning from Dif- ferentials, Di↵erential Products, & Integrals ...... 5 1.2.3 Goal 3: Designing, Implementing, & Assessing Ecacy of Instructional Intervention Tutorials for Improving Students’ Abilities to Derive Physical Meaning from Symbolic Di↵er- entials, Di↵erential Products, & Integrals ...... 6 1.3 Thesis Organization ...... 7

vii 2. Theoretical Motivation ...... 9

2.1 Student Diculties Extracting Meaning from Mathematical Symbols inPhysics...... 9 2.1.1 “WhatDoesthisSymbolMeanPhysically?” ...... 9 2.1.2 “What Kind of Quantity is this Physical Symbol?” . . . . . 11 2.2 Procedural vs. Conceptual Abilities in Integration...... 13 2.3 Student Conceptual Diculties with Physics Integration ...... 16 2.4 Theoretical Instructional Frameworks ...... 21 2.4.1 Process-Object Layers of Integration ...... 22 2.4.2 “Change” and “Amount” Di↵erentials ...... 23 2.5 Summary ...... 24

3. Discovery of Common Conceptual Diculties and Their Frequencies in Connecting Symbolic Physics Integrals and Their Components to Physical Meaning ...... 26

3.1 Introduction ...... 26 3.2 Physics Integration Exploratory Study ...... 27 3.2.1 Study Design & Research Methods ...... 27 3.2.2 Results & Discussion ...... 32 3.3 OnlinePhysicsIntegralAssessmentStudy ...... 37 3.3.1 Study&AssessmentDesign ...... 38 3.3.2 Results & Discussion ...... 41 3.4 Conclusions & Outlook ...... 46

4. The Mediating Relationship of Di↵erential Products in Understanding Physics Integration ...... 48

4.1 Introduction ...... 48 4.1.1 Motivation ...... 49 4.1.2 Regression-Based Statistical Mediation ...... 50 4.2 Methods & Design ...... 53 4.2.1 Study Participants ...... 53 4.2.2 Study&AssessmentDesign ...... 54 4.2.3 AssessmentValidity&Reliability ...... 56 4.3 Results & Discussion ...... 58 4.3.1 DescriptivesStatistics ...... 58 4.3.2 Mediation from Di↵erential Products ...... 60 4.3.3 Possible Moderation from Context-Familiarity ...... 63 4.4 Conclusions ...... 68

viii 5. Instructional Interventions for Student Understanding of Physics Di↵er- entials and Di↵erentialProducts...... 70

5.1 Introduction to Di↵erentials Training ...... 70 5.1.1 Motivation ...... 70 5.1.2 Instructional Tutorials & Feedback from Computer-Based In- struction ...... 71 5.2 Experimental Design & Methods ...... 72 5.2.1 Participant Demographics ...... 72 5.2.2 EstimatingNecessarySampleSize ...... 73 5.2.3 Di↵erentials Tutorials ...... 75 5.2.4 Di↵erentials Post-Test ...... 79 5.3 Experimental Results & Discussion ...... 84 5.3.1 The E↵ect of Physical Context ...... 85 5.3.2 The E↵ect of Feedback ...... 87 5.4 Conclusions ...... 92

6. Physics Integration Paths to Understanding Instructional Interventions . 94

6.1 Introduction to Integration Paths to Understanding Experiment . . 94 6.1.1 Motivation & Research Questions ...... 94 6.2 Experimental Methods ...... 99 6.2.1 Study Participants ...... 99 6.2.2 Experimental Design ...... 101 6.2.3 InstructionalInterventionTutorials ...... 104 6.2.4 Physics Di↵erentials, Di↵erential Products, Integrals Assess- ment...... 115 6.3 Results & Discussion ...... 116 6.3.1 Descriptive Statistics ...... 116 6.3.2 The E↵ect of Instructional Path ...... 118 6.3.3 The E↵ectofUnits...... 119 6.3.4 Electronic Feedback vs. Interactive Engagement ...... 122 6.4 Possible Limitations ...... 125 6.4.1 Baseline Ceiling E↵ects ...... 125 6.4.2 Possible Performance from Working Memory ...... 126 6.4.3 Missing Interaction E↵ects ...... 126 6.5 Conclusions ...... 127

7. Conclusions and Implications ...... 129

ix 7.1 Investigating, Discovering, & Estimating Frequencies of Conceptual Obstacles from Unproductive Interpretations of Physics Integrals & Their Components ...... 129 7.2 Investigating Mediating & Moderating Relationships Between the Abilities to Derive Physical Meaning from Di↵erentials, Di↵erential Products, & Integrals ...... 131 7.3 Designing, Implementing, & Assessing Ecacy of Instructional In- tervention Tutorials for Improving Students’ Abilities to Derive Phys- ical Meaning from Symbolic Di↵erentials, Di↵erential Products, & Integrals ...... 133

Appendices 136

A. Sample Introductory Physics Di↵erentials, Di↵erential Products, and In- tegralsAssessmentItems...... 136

A.1 SamplePhysics1AssessmentItems...... 136 A.1.1 Physics1InterpretationItems...... 136 A.1.2 Physics1UnitsItems ...... 138 A.2 SamplePhysics2AssessmentItems...... 139 A.2.1 Physics2InterpretationItems...... 139 A.2.2 Physics2UnitsItems ...... 140

B. Di↵erentials Training Post-Test Items ...... 143

Bibliography ...... 145

x List of Tables

Table Page

3.1 Frequencies of some common diculties. Frequencies reflect the per- centage of students (out of 83 participants) who responded in a man- ner consistent with each conception at least once. Errors represent one standard error for binomial proportions...... 33

3.2 Physical quantity categories and physical contexts of assessment items. Each context in the second column corresponds to two assessment items: a physical interpretation question and an explicit units ques- tion, both employing identical physical scenarios. For examples of individualitems,seeAppendixA...... 39

3.3 Observed frequencies of some common diculties from online assess- ment. Frequencies reflect the percentage of students (out of 1102 par- ticipants) who responded in a manner consistent with each incorrect conceptualization at least once (second column) or more than once (third column). Errors represent one standard error for binomial pro- portions...... 41

4.1 Categories and physical contexts of assessment items for each sample. Each quantity in the Physics 1 and 2 columns corresponds to two assessment items: a physical interpretation question and an explicit units question, both employing identical physical scenarios...... 55

4.2 Assessment reliability statistics. Cronbach’s ↵ is shown for both the Physics 1 and Physics 2 samlpes for each set of assessment items. Below each Cronbach’s ↵, we show the number of items for each Physical Quantity from both assessments...... 57

4.3 Assessment descriptive statistics. Mean scores with one standard error are shown for both the Physics 1 and Physics 2 samples for each set of assessment items...... 58

xi 4.4 Linear regression model coecients. Means with one standard error are shown for both the Physics 1 and Physics 2 samples for each regression coecientfromFigure4.1...... 61

4.5 Indirect E↵ects ab estimated with one standard error for each context separately. These Indirect E↵ects were computed for each context individually, which demonstrates that some contexts present stronger evidence of mediation than others. Familiarity with each context (or lackthereof)mayexplainsomeofthisvariation...... 64

5.1 Di↵erentials/Di↵erential Products experimental treatment design. . . 75

5.2 Mean di↵erentials post-test scores for all items (vertical) across all conditions (horizontal). Mean total di↵erentials post-test scores are displayed for each condition in the last row. Errors are one standard error of the mean...... 84

6.1 Integration paths-to-understanding experimental design...... 101

6.2 Categories and physical contexts of assessment items for Physics 2 assessment. Each symbolic quantity corresponds to two assessment items: a physical interpretation question and an explicit units question, bothemployingidenticalphysicalscenarios...... 116

6.3 Assessment descriptive statistics. Mean scores with one standard error are shown for all experimental conditions for each set of assessment items...... 117

6.4 Assessment descriptive statistics for units items only. Mean units scores with one standard error are shown for all experimental con- ditions for each set of assessment items...... 120

xii List of Figures

Figure Page

3.1 Reference table of symbols, names, and units provided to students completing our exploratory research task...... 28

3.2 Sample Symbolic Comparison subtask from our exploratory research task...... 29

3.3 “Student A vs. Student B” item from our exploratory research task. This item was designed to estimate the frequency of the conceptual- ization that di↵erentials within integrals are only labels...... 30

3.4 Discussion item from our exploratory research task. This item was intended to estimate the frequency of the notion that integrals and derivatives are necessarily inverse operations...... 31

3.5 Written response sample from a participant in our exploratory research study. This student indicates that di↵erentials do not have physical meaning within an integral; the student claims that they are simply labels...... 34

3.6 Written response sample from a participant in our exploratory research study. This student may perceive the di↵erential dt as a kind of “in- stantaneous value” operator...... 35

3.7 Written response sample from a participant in our exploratory research study. This student interprets the di↵erential dt as a rate or “deriva- tive.” Note that the student included “rate-like” units...... 37

3.8 Sample Physics Di↵erentials, Di↵erential Products, Integrals Assess- ment Item. This item assessed the ability to interpret the physical t2 meaning of the integral t1 vdt ...... 40 R xiii 4.1 TOP: A visual representation of the Total E↵ect of a simple mediation model. Mathematically, this is shown in Equation 4.1. BOTTOM: AvisualrepresentationoftheIndirectandDirectE↵ects of a simple mediation model. Mathematically, this is shown in Equations 4.2 and 4.3. 51

4.2 TOP: A visual representation of the Total E↵ect of our simple medi- ation model. The regression coecient estimating the Total E↵ect is shown for both Physics 1 and Physics 2 samples as c1 and c2, respec- tively, with one standard error. BOTTOM: A visual representation of the Direct and Indirect E↵ects of our simple mediation model. The re- gression coecients are shown for both Physics 1 and Physics 2 samples with subscripts “1” and “2”, respectively, with one standard error. . . 62

4.3 HORIZONTAL: Mean self-reported familiarity with each physical con- text from Table 4.5. VERTICAL: Completely standardized Indirect E↵ect for each physical context from Table 4.5. One standard error shown for both dimensions. Blue symbols represent contexts presented to Physics 1 students. Red symbols represent contexts presented to Physics 2 students. Note: Familiarity scores were obtained from a separate sample of Physics 2 students in a later semester...... 66

5.1 Sampletrainingitem,fromearlyintheFXtutorial...... 77

5.2 Second sample training item, from later in the FX tutorial...... 78

5.3 Mean total di↵erentials post-test scores separated by training context. Error bars are one standard error of the mean...... 86

5.4 Mean total di↵erentials post-test separated by feedback. Error bars are one standard error of the mean...... 88

5.5 Mean di↵erentials post-test scores, separated for v0dt, Fkdx items (scar- let) and all other post-test items (gray). Feedback (VTE, FXE), No Feedback (VT, FX), and Control conditions along horizontal axis. Er- rorbarsareonestandarderrorofthemean...... 89

5.6 Mean di↵erentials post-test scores, separated for Fkdx, P Adx items (scarlet) and all other post-test items (gray). Feedback (FXE, PVE), No Feedback (FX, PV), and Control conditions along horizontal axis. Error bars are one standard error of the mean...... 91

xiv 6.1 A simplified version of the Von Kor↵ and Rebello process-object layered framework for physics integration. Scarlet arrows: Riemann path to understanding. Gray arrows: Di↵erential path to understanding. . . . 95

6.2 FirstKEYINFOfeatureofRiemannpathtutorial...... 104

6.3 Macroscopic quantity layer sample subtasks from Riemann path tutorial.105

6.4 Summation layer subtask from Riemann path tutorial...... 107

6.5 Limiting procedure discussion question from Riemann path tutorial. . 108

6.6 Integral definition KEY INFO from Riemann path tutorial...... 108

6.7 Practice conceptual table, connecting symbols, interpretations, and unitsfromRiemannpathtutorial...... 110

6.8 FinalsubtaskfromRiemannpathtutorial...... 111

6.9 First KEY INFO feature of Di↵erential path tutorial...... 112

6.10 Integral definition KEY INFO from Di↵erential path tutorial. . . . . 114

6.11 Practice conceptual table, connecting symbols, interpretations, and units from Di↵erentialpathtutorial...... 114

6.12 Mean integral interpretations scores for all conditions. Di↵erential path conditions shown in gray. Riemann path condition shown in scarlet. Control condition shown in white. Error bars represent one standard error of the mean...... 123

6.13 Mean abstract integral ( AdB)interpretationsscoresforallcondi- tions. Di↵erential path conditionsR shown in gray. Riemann path con- dition shown in scarlet. Control condition shown in white. Error bars represent one standard error of the mean...... 124

xv Chapter 1: Introduction

Students enrolled in calculus-based introductory physics courses inevitably en- counter homework and exam problems that require integration, especially in the second semester of the standard introductory sequence. Their instructors may fre- quently (implicitly or explicitly) expect students to be able to answer the question,

“What does a symbolic integral physically mean?” Some authors have explicitly claimed that interpreting and making sense of contextualized integrals is the optimal

first step towards understanding and working with them in physics and engineer- ing courses (e.g., Jones, 2015) [28]. This translation from symbols (e.g., vdt)to R verbal/physical meaning (e.g., “the total displacement of an object”) may not be a task for which calculus courses adequately prepare incoming physics students, even though they are often capable of completing integral operations. In fact, making sense of the concept of an integral in an introductory physics course is arguably more important than the ability to actually calculate one. Unfortunately, the ability to derive physical/conceptual meaning from symbolic integrals remains elusive to many introductory physics students. Existing research has attempted to highlight this is- sue and to propose instructional remedies, but there remain significant questions as to the nature and causes of the student diculties and a noticeable lack of large, quantitative experiments showing e↵ective treatment.

1 To address this need, this thesis focuses on the following research goals (detailed in Section 1.2) to help improve students’ abilities to make physical sense of symbolic integrals and their components (di↵erentials and di↵erential products):

1. Investigate, discover, and estimate the frequencies of conceptual obstacles that

are based on unproductive interpretations of physics integrals and their compo-

nents

2. Investigate mediating and moderating relationships between the abilities to de-

rive physical meaning from di↵erentials, di↵erential products, and integrals

3. Design, implement, and assess the ecacy of instructional intervention tutorials

in quantitative experiments (with several treatment factors that may impact

performance, including the e↵ect of feedback during training and the e↵ect

of varied physical training contexts) for the purpose of improving introductory

physics students’ abilities to derive physical meaning from symbolic di↵erentials,

di↵erential products, and integrals

Before elaborating on each of these goals, we will first survey the current state of research findings on this topic in the following section.

1.1 Current Issues in Integration from Mathematics and Physics Education Research

For several decades, mathematics and physics education research have shed light on a conceptual gap in integral understanding among introductory-level students.

Specifically, university-level calculus students have demonstrated proficiency in per- forming Riemann sum and integral calculations, but have also shown an inability

2 to explain the fundamental meaning or purpose behind such operations (Orton,

1983) [38]. Ideally, our students would interpret integration as a sum of products, which is a physically productive and flexible conceptualization, but even in particu- larly advantageous situations, students tend not to view integration this way (Jones,

2015) [28]. In reality, students tend to look for potentially unproductive cues alert- ing them to integrate, such as the dependence of one variable on another (Meredith and Marrongelle, 2008) [35]. More recently, research has emphasized the importance of the di↵erential and di↵erential product (e.g., dt and vdt, respectively) within the integral as a potentially key stumbling block for physics students (e.g., Nguyen &

Rebello, 2011; Hu & Rebello, 2013) [36] [27].

In light of the conceptual obstacles that impede understanding of physics inte- grals, a variety of attempts have been made to identify successful reasoning strategies in order to guide student thinking. Some publications have supported future research of integration understanding by developing structural, conceptual frameworks that establish theoretical foundations for how students may successfully learn integration, as well as supplying the research community with a richer vocabulary of relevant con- cepts (Sealy, 2014; Von Kor↵ & Rebello, 2012; Jones, 2015) [45] [52] [28]. Frameworks have been proposed and tested for ecacy in both mathematics and physics educa- tion research (with large overlap) to facilitate this cause, but most of these studies are qualitative, featuring small groups of students. To this point, there has been a clear deficiency of large-scale, quantitative studies assessing the utility of these conceptual frameworks designed to help students make physical sense of integration.

3 1.2 Research Goals & Contributions

Having briefly outlined the current state of conceptual understanding of integra- tion within mathematics and physics education research, this section will proceed with the enumeration and explanation of our research goals for this dissertation. Ul- timately, we were guided by the ambition to help introductory physics students make sense of symbolic integrals and their components. To that end, we discuss the goals that this dissertation accomplishes in the proceeding subsections.

1.2.1 Goal 1: Investigating, Discovering, & Estimating Fre- quencies of Conceptual Obstacles from Unproductive Interpretations of Physics Integrals & Their Compo- nents

We designed and conducted multiple studies intended to illuminate some of the various ways that first-semester, introductory calculus-based physics students inter- pret and conceptualize di↵erentials, di↵erential products, and integrals in a variety of physical contexts, including interpretations that are counter-productive to expert- like physical understanding. Our results confirmed some previously-known diculties; however, we also uncovered novel incorrect interpretations that may provide new in- sight into students’ ideas about di↵erentials in physics, such as the incorrect notion that a di↵erential is a “rate.” These studies also attempted to estimate the frequen- cies of many common diculties encountered by students when discussing physics integrals and their components, which is a luxury not a↵orded to previous experi- ments with much smaller sample sizes. The results of these studies are discussed in

Chapter 3.

4 1.2.2 Goal 2: Investigating Mediating & Moderating Re- lationships Between the Abilities to Derive Physical Meaning from Di↵erentials, Di↵erential Products, & Integrals

Von Kor↵ and Rebello developed a conceptual framework intended to guide physics integration instruction, in which the di↵erential product is situated between di↵er- entials and integrals (e.g., dt vdt vdt)(VonKor↵ &Rebello,2012)[52]. ! ! R We hypothesized that this relationship was an indication of statistical mediation. In other words, students’ abilities to extract physical meaning from di↵erentials would be strongly related to their abilities to derive physical meaning from di↵erential prod- ucts, and likewise di↵erential products to integrals; however, this hypothesis also implied that we should expect to find little direct relationship between the abilities to derive physical meaning from di↵erentials and those analogous abilities pertaining to integrals. In this way, the understanding of physics di↵erential products would act as a mediator between the understanding of physics di↵erentials and integrals.

As part of our study, we designed a reliable, multiple choice assessment to gauge students’ abilities to relate symbolic integrals and their components to corresponding physical interpretations and units across many contexts. Scores from this assessment were used in a multiple linear regression analysis that revealed strong evidence sup- porting the hypothesis that students’ abilities to glean physical meaning from physics di↵erentials and integrals are statistically mediated by their capacities to do the same from di↵erential products. Details of this study are presented in Chapter 4.

5 1.2.3 Goal 3: Designing, Implementing, & Assessing Ecacy of Instructional Intervention Tutorials for Improving Students’ Abilities to Derive Physical Meaning from Symbolic Di↵erentials, Di↵erential Products, & Inte- grals

To help students better associate symbolic integrals and their components with correct physical interpretations and units, we conducted two controlled experiments utilizing paper- and computer-based training tutorials. Experimental factors of inter- est included the use of electronic feedback, interactive engagement with peers, the use of varied physical contexts, the instruction of units/dimensional analysis, and tutorial design following more traditional or physically-intuitive instruction.

The first experiment, detailed in Chapter 5, developed and implemented training tutorials to encourage expert-like interpretation of physics di↵erentials and di↵eren- tial products. Experimental conditions received individual training from tutorials on paper (without feedback during training) or on a computer (with feedback during training); the physical contexts of these tutorials were varied by condition to de- termine their e↵ect on training. We assessed the ecacy of our tutorials and the e↵ects of our experimental factors with a “di↵erentials post-test” that we designed to gauge students’ abilities to make physical sense of di↵erentials and di↵erential prod- ucts. For comparison, an untrained control group served as a baseline estimate of the population’s performance.

In the second experiment (provided in Chapter 6), we designed and deployed new training tutorials to promote better extraction of physical meaning from symbolic integrals. Primarily, we intended to determine if significant di↵erences in performance were observable when training students along an instructional path more commonly

6 n seen in mathematics courses (t vt i vit limn i vit vdt)or ! ! ! !1 ! P P R one that may be more physically-intuitive (dt vdt vdt). We also compared the ! ! R training e↵ects of electronic feedback and interactive engagement by providing some students with an individual computer-based tutorial (with electronic feedback during training) and others with interactive, three-person-group, paper-based tutorials (with inter-personal feedback from each other during training, not guided by the proctor).

Finally, the e↵ect of training with units/dimensional analysis was tested by including units questions on tutorials for some conditions, but ignoring any discussion of units on tutorials for a di↵erent condition. The e↵ects of these factors were gauged using our multiple choice physics di↵erentials, di↵erential products, and integrals assessment; an untrained control condition was employed for comparison.

Our findings from both experiments indicate that the ability to physically interpret symbolic representations of integrals and their components is greatly improved by the provision of authoritative, electronic feedback. To our knowledge, these results mark the first instance of a large, quantitative experiment involving introductory physics students that has demonstrated significantly stronger connection of physics integrals and their components to physical meaning, compared to untrained peers. The results and instructional implications of these studies are discussed in Chapters 5 and 6.

1.3 Thesis Organization

In Chapter 2, we will examine in greater detail results of previous studies per- taining to integration and the interpretation of mathematical symbols in physics.

Furthermore, several valuable theoretical frameworks used in this dissertation will be

7 presented to enhance the research lexicon and provide guidance for instruction and intervention.

After motivation and a more thorough overview of the current state of integra- tion in physics, Chapters 3 - 6 will lay out the unique contributions made to the understanding of introductory-level physics integration with presentations of original studies discussed above. Each chapter will introduce the main problem to be solved, as well as an explanation of methods, results, and overall conclusions. Chapter 7 will provide a summary of all of our findings, implications for instruction, and possible future improvements to research presented in this dissertation.

8 Chapter 2: Theoretical Motivation

In this chapter, we will review existing literature relevant to our research and motivate the experiments to be discussed later in this dissertation by providing some theoretical and contextual considerations.

2.1 Student Diculties Extracting Meaning from Mathemat- ical Symbols in Physics

2.1.1 “What Does this Symbol Mean Physically?”

“What does this symbol mean physically?” On its surface, this seems a straight- forward question, and the pursuit of conceptual understanding in physics problem- solving requires we ask it of ourselves. Consider the physical meaning of several mundane symbols in a first semester, calculus-based introductory physics course, all with units of meters: traditionally, x0 is a specific, non-varying location; L is often understood to be a length, though frequently it is used as a scaling parameter of some characteristic/physical significance; x, on the other hand, is usually a variable posi- tion, which is not necessarily the same as an “unknown,” although it certainly could be, if it is the sought-after quantity in a typical introductory physics problem. Clearly, these distinctions are subtle, but they are also crucial to making sense of problems with a variety of quantities and types of quantities. Needless to say, the answer to

9 the question, “What does this symbol mean physically?,” may be heavily dependent on perspective and experience and it may not be as obvious as we (physicists) would like to think.

As it pertains to our research, it is an essential question that helped us navigate the ways in which students perceive physics integrals and how they might verbalize this perception. Because mathematics courses usually expose students to algebraic symbols first (or simultanesouly), e↵ort should be made (and is being made) by physicists to recognize and bridge the divide that exists between our own field and that of mathematicians (and thereby their students) regarding the use of symbolic quantities (Dray and Manogue, 2005) [12].

As an example, Redish surveyed di↵erences between the uses of algebraic symbols by mathematicians and physicists (Redish, 2005) [42]. Among his considerations was the use of labeling constants and variables. In a traditional mathematics class, there are typically only a few variables students will ever encounter: x, y, z, and occasionally t. Likewise, constants tend to be the same set of easily identifiable symbols: a, b, c, and sometimes k. In general, mathematics courses tend to o↵er relatively bare equations, stripped of context, on which students perform routine operations (Meel, 1998) [34].

However, in the first month of first semester, introductory physics, students may see xf ,xi,vi,vf ,t,t0,ax,g,etc. in a single homework problem, or perhaps even in a single equation (Halliday, Resnick, & Walker, 2013) [17]. This symbolic overload “blurs” the recognizable lines between variables and constants or knowns and unknowns (Redish,

2005). If one’s only experience with the symbol x is that it represents the requested quantity to solve a problem (i.e., “the unknown,” as in, 3x +4=7),thenweshould

10 be unsurprised that our students do not immediately view it with the same instinctive recognition and attention that we do as physicists.

Specifically regarding integration, this remains a challenge even in calculus courses, well-beyond the level of introductory algebra. Calculus students are peppered with nightly homework problems that may involve computing integrals featuring a variety

1+x of physically uncommon (or unrealizable) functional forms, such as 1+x2 dx (Stewart, R 2008) [49]. The integrals may change very little between problems, by merely adding computational complexity and altering the variable of integration from x to t or ✓

(Yeatts and Hundhausen, 1992) [53]. Whereas in algebra, x is simply “the unknown,” in calculus, x is just “the variable of integration.” That novice physics students may struggle to identify the meaning of physical symbols based on their lack of prior experience with the way physicists use them is a consideration that drives this research, specifically pertaining to physics integrals.

2.1.2 “What Kind of Quantity is this Physical Symbol?”

In addition to the question of “What does this symbol mean physically?,” an- other important facet to deriving meaning from physical symbols is recognizing what kinds of quantities they represent depending on the contextual clues presented. In the language of Dray and Manogue, “what sort of a beast is it?” (Manogue et al.,

2006) [32]. Unfortunately, because students tend to have much less experience with the notion that mathematics represents real, physical things, they may not instinc- tively ask themselves questions about the symbols they see and use (Manogue et al.,

2006). For example, an expert might easily discern t from T (time and tempera- ture, respectively), in the context of thermal physics. While these two symbols may

11 appear trivially di↵erent to a novice, a physicist sees them as possessing entirely dif- ferent physical dimensions/units - they are completely di↵erent “kinds” of quantities.

This represents part of understanding what a quantity means physically, and while it is not wholly independent from the question posed by Section 2.1.1 (e.g., knowing that x0 represents a specific location likely has strong overlap with knowing that it is measured in meters), it does provide another piece of the puzzle that physicists use automatically, unlike novices.

When we ask the question, “What kind of quantity is it?” - operationally, we are asking, “what are its physical dimensions/units?” Physics tends to load meaning (or

“metadata”) onto symbols, such as physical context or dimensional/unit considera- tions (Redish, 2005) [42]. For an example illustrated by Redish, consider the question,

“A(x, y)=k(x2 + y2), what is A(r, )?” Mathematicians may make the obvious sub- stitution A(r, )=k(r2 + 2), whereas physicists are more likely to instinctively answer with a preconceived notion of the expression x2 + y2, writing A(r, )=kr2.

Physicists might decry the improper combination of units in the mathematician’s ex- pression r2 +2, but this further demonstrates the point: physical symbols bear extra information visible for physics experts that may be inapparent to novices, who may not even be aware that such information exists.

Redish observes that students are strongly inclined to substitute numerical values in their physics problems as quickly as possible to “[make] the equations look more like the equations in their math classes and [make] them seem more familiar.” He notes that this has the e↵ect of destroying the “metadata” encoded in the physical symbols, though students often see this as unimportant - that this data can be retrieved by simply appending the units to the final answer; as Redish phrases it, “I’ll put them

12 in at the end since I know how they have to come out.” However, as we will see in Section 2.3, students who are not accustomed to extracting information about the units/dimensions of physical symbols may find this an impossible task when the units of the integral di↵er from those of the integrand, as is often the case. As an example, note that Fdx has units of Newtons meters, not simply Newtons as · R many students may be inclined to believe from their mathematics courses. Indeed,

b apopularcalculustextbookdefinesthedefiniteintegral a f(x)dx for the first time

R b by claiming, “For now, the symbol dx has no meaning by itself; a f(x)dx is all R one symbol. The dx simply indicates that the independent variable is x,” (Stewart,

2008) [49]. This represents a profound fracture between the priorities of a physicist and a mathematician regarding the units (and interpretation) of the components of an integral, and introductory physics students may bear the burden of this conceptual gap.

2.2 Procedural vs. Conceptual Abilities in Integration

Hiebert and Lefevre developed a widely-used framework for discussing two broad categories of mathematical knowledge (Hiebert & Lefevre, 1986) [25]. They defined conceptual knowledge as that which is “rich in relationships.” They present it as a network, in which a single unit of conceptual knowledge cannot be isolated from other information. As an example of conceptual knowledge, they discuss the learning of decimals: position values to the right of the decimal have names corresponding to their order of magnitude, like “tenths” or “hundredths;” furthermore, lining up the decimal points is a precursor to adding or subtracting decimals. These two pieces of conceptual knowledge are linked during addition, in which the position values to the right of

13 the decimal add when the points are aligned. Procedural knowledge was defined as the mathematical form of symbol representation and the “rules [or] procedures” for solving mathematical problems. They provide an example of procedural knowledge as the adding of two fractions of unlike denominators.

Within the subject of integration, existing research from mathematics education provides great insight into students’ procedural knowledge in comparison to their con- ceptual knowledge (e.g., Orton, 1983; Artigue, 1991; Rasslan & Tall, 2002; Grund- meier et al., 2006; Mahir, 2009; Serhan, 2015) [38] [5] [41] [15] [31] [46]. Among the earliest investigations of student conceptual knowledge of integration is Orton, who examined 110 British students ages 16-22 on such topics as as the limit of a Riemann sum, providing the area beneath a graphical function, the linearity property of inte- grals, and obtaining a volume by revolving a function about an axis (Orton, 1983).

Among his findings, he noted that when directly asked to compute a limit of a se- quence, the majority of participants were able to complete the calculation, especially when prompted. Students were also explicitly asked to compute an indefinite inte- gral, a task which proved less challenging than discussing the conceptual meaning of their calculations. In particular, Orton indicated that his participants often failed to connect the limit of the Riemann sum with the exact value of the integral, treating it more as an approximation. Notably, these students generally had no trouble actually evaluating the limit of the Riemann sum, despite their poor understanding of its pur- pose. Orton concluded that the most significant diculties among his participants pertained to their lack of understanding that an integral is the limit of a sum.

This conceptual-procedural gap was also uncovered by Artigue, who examined cal- culus students’ comprehension of di↵erentiation and integration concepts (Artigue,

14 1991) [5]. Once again, though the majority of participants were able to perform rou- tine, procedural computations in integration, such as finding the area under a curve, very few were capable of explaining rationale behind the calculations. Interestingly, some students were not even fully aware of the reason for the calculations at all.

In a study of British high school students in their final year, all of whom had been exposed to the concepts of integration in the past, Rasslan and Tall assessed the con-

b ceptual knowledge of their participants regarding the simple definite integral a f(x)dx R (Rasslan and Tall, 2002)[41]. The students in their participant group exceeded na- tional examination averages and had been enrolled in a calculus course designed to be more experiential and conceptual. Nevertheless, the authors succinctly concluded,

“The majority do not write meaningfully about the definition of definite integral, and have diculty interpreting problems calculating areas and definite integrals in wider contexts.”

Furthermore, a survey developed by Grundmeier et al. was provided to calculus students with prior experience in integration (Grundmeier et al., 2006) [15]. The sur- vey sought to study integral procedural and conceptual knowledge of its participants, including their understanding of the definition of definite integrals and their skill in evaluating definite integrals. Similarly to findings discussed above, students failed to make adequate connections between the techniques of integral computation and the conceptual meaning behind them.

While not intended to provide an exhaustive overview of the topic, we have demon- strated the existence of evidence that points to a conceptual-procedural gap among calculus students regarding integration. Naturally, within the context of introduc- tory physics, integration is often most valuable as a concept, rather than a technique,

15 where calculations frequently reduce to simple multiplication. For example, while stu- dents may see that mechanical work is defined in their textbooks as W = Fdx,this R is often stripped of its complexity by ensuring that the force is constant in homework problems: W = Fdx = F dx = F x. What e↵ect might the poor conceptual un- R R derstanding of integration discussed in this Chapter have when it must be translated to a physics course? In Section 2.3, we will explore existing research that identifies some of the most common specific diculties that students face when attempting to reason about physics integrals.

2.3 Student Conceptual Diculties with Physics Integration

In Section 2.2, we provided evidence that shows a disconnect between the inte- gration techniques students learn in a calculus course and their ability to make sense of and find meaning from those very same operations. In this section, we will at- tempt to illustrate how students conceptualize integration and its components (e.g., the di↵erential, the integrand, etc.), especially in physical contexts, and what sort of cues prompt students to employ integration when solving physics problems. In doing so, we will see how integration presents conceptual obstacles that are dicult for mathematics and physics students to overcome.

In addition to comparing proficiency in calculus operations with conceptual un- derstanding discussed in Section 2.2, much research has helped to identify specific deficiencies in comprehension of integrals in both mathematics and physics (e.g.,

Yeatts and Hundhausen, 1992; Cui et al., 2006; Sealey, 2006; Thompson and Silver- man, 2008; Nguyen and Rebello, 2011a; Nguyen and Rebello, 2011b; Hu and Rebello,

2013a; Hu and Rebello, 2013b; Von Kor↵ et al., 2013; Jones, 2015) [53] [11] [44] [50]

16 [36] [37] [27] [26]. A common result among many existing studies is many students may be unaware that an integral is a) a sum, and in particular b) a sum of products.

Yeatts and Hundhausen published early findings from a combined calculus and physics course (Yeatts and Hundhausen, 1992) [53]. The authors categorized student diculties as: Notation & Symbolism, Distraction Factor, and Compartmentalization of Knowledge. Of particular interest in this discussion is the first category, Notation &

Symbolism, in which student errors arose from “their dependence on specific symbols or specific verbiage to jog their memory.” As an example, the authors reference what they call “xy syndrome,” which occurs when students are heavily reliant on the familiarity of the symbols x and y as their independent and dependent variables from their calculus course. Because they are less likely to be exposed to symbols with physical meaning in a pure mathematics class, they are prone to ignoring important details, especially within integration, such as mg sin ✓ds = mg cos ✓s. This type R of mistake suggests a failure to recognize the meaning of the di↵erential ds. The authors also mention the physical value of the Leibniz notation for di↵erentiation

dy and integration, such as dx and f(x)dx, which makes use of the d symbol and R suggests the notion that a derivative is a quotient and that an integral involves a product. By contrast, mathematics courses often utilize other notations, especially for derivatives, such as f 0(x), which may obfuscate physical meaning.

Later, Sealey conducted an instructional experiment in a calculus workshop to attempt to clarify what it means to “understand definite integrals” (Sealey, 2006)[44].

She found that students were sometimes capable of relating the area under the curve to the integral, such as in the case of energy from a F vs x diagram, but were unable to explain why the area under the curve of a Force/displacement diagram equated

17 to energy. She surmised that this inability to explain their reasoning stemmed from a lack of understanding of the Riemann structure of an integral, namely that it is comprised of a sum of products. She states, “Several times throughout the video, the students in this group incorrectly said that the ‘summation of forces equals energy.’

It is not just the summation of forces that equals energy, but it is the summation of the products of force and distance that equals energy. The students were attending to the summation layer of the definite integral, but did not include the product layer.”

She concluded that area under the curve is not a “bad” representation of an integral; however, she found it to be insucient for understanding, and considered it to be most useful for students who already made sense of the underlying sum-of-products structure of the integral.

To emphasize the understanding of two independent aspects of integral structure

(the product and the sum), Jones recently developed the Multiplicatively Based Sum- mation (MBS) conception, which he defined as: (1) the product of the integrand and the di↵erential to create a new quantity, and (2) the notion of adding up tiny amounts of the resultant product through tiny intervals of the domain to accumulate the total quantity (Jones, 2015) [28]. In spite of this, Jones found that some students were still not likely to spontaneously use the MBS conception to make sense of physics integrals, even in particularly helpful contexts, such as pressure integrated over an area to obtain a total force, F = PdA, for which area under the curve would be R a less productive avenue of reasoning. Jones suggests that what many researchers have labeled “misconceptions” or (more generally) “diculties” may actually be in- stances of competing cognitive “resources” (small-scale elements of knowledge used in sense making; Hammer, 2000 [19]). Jones asserts that “student diculties might not

18 necessarily arise from lack of knowledge, but from the activation of less-productive cognitive resources over others.”

In addition to the research on student diculties with integration, Meredith and

Marrongelle broadened the existing literature by investigating which cues prompt stu- dents to use an integral in the context of electrostatics (Meredith and Marrongelle,

2008) [35]. Their findings indicate that students are most commonly cued to use an integral by dependence (utilized by 8 out of 10 students in their study), such as recog- nizing that a non-constant density ⇢(x) depends on the location of the measurement x from the origin. While this cue does not necessarily preclude the possibility of suc- cessful setup of the physics integral, the authors observed occasions in which it can be misleading. For example, several students misapplied the di↵erential to an electric

q dq field calculation by writing dE = k r2 dr, instead of dE = k r2 ; the authors noted that this mistake was likely a result of the dependence cue, in which students interpreted the spatially-varying field strength as a prompt to integrate, but without considering precisely how the pieces should be accumulated, and therefore appended a simple dr to the end of the integrand. In other instances, some students were prompted to in- tegrate because they remembered similar problems in which integration was needed, which the authors called the “recall” cue. Finally, the most productive and highly

flexible cue was deemed, “parts of a whole,” which most closely resembles the MBS conception discussed above. The “parts of a whole” cue was characterized by the recognition that students would be adding up “small pieces” to produce an overall physical result.

As evidence of student diculty in understanding physics integration mounts, recent research has also investigated the roles of the di↵erential and the di↵erential

19 product in integral conception (Nguyen and Rebello, 2011; Von Kor↵ and Rebello,

2012; Hu and Rebello, 2013; Von Kor↵ and Rebello, 2014) [36] [52] [26] [30].

Nguyen and Rebello found that mid-level electrostatics students encountered sig- nificant obstacles solving and explaining several integration problems, often due to

“incorrect expression for the infinitesimal quantity and/or accumulating the infinites- imal quantities in an inappropriate manner” (Nguyen and Rebello, 2011)[36]. The authors further commented that the most serious conceptual impedances were “due to students’ inability to understand the infinitesimal term in the integral and failure to understand the notion of accumulation of an infinitesimal quantity.” The authors recommended tutorials to emphasize the meaning of the infinitesimal quantity (e.g., dx, dr, d✓, etc.)withintheintegralanditsaccumulationprocess.

Expounding on this result, Hu and Rebello characterized students’ use of di↵eren- tials into several categories, some of which resulted in greater integral setup success than others (Hu and Rebello, 2013)[26]. Their study identified and named several commonly-used resources: “small piece/segment,” in which the di↵erential refers to asmallamountofaphysicalvariable;“point,”whichisinvokedwhenphysicalsize can be neglected (e.g., “dq is the charge at a point); and “di↵erentiation,” which suggested that students viewed d as a mathematical operator to be “pulled out of nowhere” - with little formal explanation - as a cue to di↵erentiate a quantity (e.g.,

R = k ⇢ dR = k d[⇢]=k d⇢). · ! · · On this note, di↵erentials may be an awkward concept for students in introduc- tory calculus courses because their usage as intuitive, manipulable elements in calcu- lus operations has been largely purged from the field of mathematics within the past century (Dray and Manogue, 2010) [14]. In a 1952 editorial comment in American

20 Mathematical Monthly, Allendoerfer summarizes the prevailing viewpoint of math- ematics: “There is a discredited view that di↵erentials are some sort of ‘infinitely small quantities.’ Of course, this is nonsense.” (Allendoerfer, 1952) [1]. It should be noted that this perspective was subsequently challenged by Robinson in his textbook,

Non-Standard Analysis, in which the author writes, “In coming centuries it will be considered a great oddity in the history of mathematics that the first exact theory of infinitesimals was developed 300 years after the invention of the di↵erential calcu- lus” (Robinson, 1996) [43]. Further support for the use of di↵erentials in calculus and physics courses is expressed by Dray and Manogue [13], though the authors state that

“little e↵ort has been made to restore the original role of di↵erentials in calculus.”

(Dray and Manogue, 2010) [14]

In summary, we have provided a brief (though not exhaustive) overview of existing literature regarding student diculties with integration. These diculties often relate to the di↵erential and di↵erential product, especially in regards to their summation to compute the integral. Students may draw on conflicting and sometimes unproductive resources, even when they have the knowledge to do otherwise. And many also struggle to see why integration is needed, or how to use it in physics contexts. In

Section 2.4, we will discuss previously developed conceptual frameworks to provide insight and language to address the aforementioned problems.

2.4 Theoretical Instructional Frameworks

To better cement a theoretical foundation for our research, we will briefly out- line several important instructional frameworks in this section. These frameworks

21 were used to assist in experimental design and to clarify/standardize analysis and discussion.

2.4.1 Process-Object Layers of Integration

Among the most insightful and valuable constructs we will employ is the “process- object layered framework” for understanding physics integration. According to Sfard, some mathematical elements can be understood as both processes and (later) as struc- tural objects (Sfard, 1991) [47]. For an historical example, consider the development of subtraction (a process), which later produced the concept of negative numbers

(objects). This was translated to calculus when Zandieh introduced several layers of process-objects in understanding the derivative (Zandieh, 2000) [54]. Zandieh pro- posed a process-object layered framework for conceptualizing derivatives, which in- cluded a Ratio layer (at its most simplified level, a derivative is division - a quotient), aLimitlayer(thequotientislimitedtoshrinkthedenominator),andaFunctionlayer

(to yield analytical functions as derivatives). Each additional stage can therefore be understood as both a verb and a noun, hence “process-object layers.”

In much the same way, integration can also be dissected into process-object layers.

In her development of a process-object layered framework for Riemann Sums, Sealey devised a four-stage Riemann-oriented system for constructing an integral (Sealey,

2014) [45]. The framework consisted of: (1) Product, f(x ) x; (2) Sum, f(x )x; i · i i n n P (3) Limit, limn i f(xi)x; and (4) Function, f(b) = limn i f(xi)x. Ul- !1 !1 P P timately, she found that students encountered the most diculty setting up finite products analogous to the (infinitesimal) di↵erential product of an integral.

22 In this dissertation, we will utilize an expanded version of Sealey’s integration lay- ers developed by Von Kor↵ and Rebello (Von Kor↵ and Rebello, 2012) [52]. Unlike that of Sealey, this framework was created specifically for use in physics. It o↵ers a va- riety of instructional paths for guiding student learning towards understanding physics integrals. The authors introduced a “network” of layers that di↵ers from Sealey’s by also o↵ering strictly “microscopic” (infinitesimal) layers, such as di↵erentials and dif- ferential products, which can be summed to produce “macroscopic” (finite) results, in addition to all of the macroscopic layers discussed by Sealey. Their model asserts that physics students can be taught that “an integral is ‘the sum of many infinites- imally small quantities,’ without harm to their understanding.” According to Von

Kor↵ and Rebello’s model, along the microscopic path to understanding integrals, the di↵erential product lies between the di↵erential and integral (e.g., dt vdt vdt) ! ! R 2.4.2 “Change” and “Amount” Di↵erentials

Von Kor↵ and Rebello further developed an integration framework to sca↵old student understanding by distinguishing between several types of di↵erentials and in- tegrals (Von Kor↵ and Rebello, 2014) [30]. Their framework was designed to connect to students’ prior knowledge; be within their “Zone of Proximal Development (ZPD),” named for the set of tasks that students can complete with some assistance; address the most common and significant diculties that confuse students; and help students build expert-like understanding. The authors’ framework provides an organized col- lection of names and examples to aid in conceptual understanding for “Amount” (e.g., dM, dA, dq)and“Change”(e.g., dx, dt, dv)di↵erentials. They also developed the

“Static Object” (e.g., r2dM, ⇢dV )and“Trajectory”(e.g., vdt, Fdx)integral R R R R

23 types to accompany them. These delineations draw clear distinctions in the language used to discuss di↵erent physics di↵erentials and integrals, depending on the physical context. To a physics expert, referring to dM as a “change” in mass may be awkward and it could negatively impact student understanding. By contrast, helping students to recognize potential contextual di↵erences in what is being accumulated and how we might conceptualize it may prove beneficial, by empowering students with a richer vocabulary to talk about di↵erentials and di↵erential products. The authors recom- mend that, “any assessment of students’ understanding of integration must take into account the diversity of infinitesimal and integral types. It would be a mistake to test students’ understanding of trajectory integrals and then claim that they under- stand static object integrals.” This prescription, and the framework overall, directly influenced research presented in the following chapters.

2.5 Summary

In closing, this chapter presented evidence that mathematics and physics stu- dents often lack the conceptual understanding to formally reason about integration.

Even when they possess the knowledge, competing resources were seen to sometimes impede success. The gap between conceptual and procedural knowledge of integra- tion was noticeable in physics contexts, when real-world applications are needed for techniques learned in calculus courses, but simple computation of pre-constructed in- tegrals is insucient. The notion of adding up small products to produce an overall,

“total” quantity was seen to be highly flexible and appeared to be a strong predictor of success, especially in physics, although many students were seen to be less likely to

24 utilize this conception compared to other, less productive lines of reasoning. Further- more, several instructional frameworks were discussed that enable elevated discussion of integration and di↵erentials, which can also provide guidelines for more expert-like student understanding of these topics.

Looking forward, the following chapters will present new contributions to this

field of study. We begin with Chapter 3, in which we discuss our findings from an exploratory study to better understand how introductory physics students conceptu- alize physics di↵erentials and integrals in a variety of contexts, and in what ways do they di↵er from experts.

25 Chapter 3: Discovery of Common Conceptual Diculties and Their Frequencies in Connecting Symbolic Physics Integrals and Their Components to Physical Meaning

3.1 Introduction

In Chapter 2.3, we examined existing evidence that has identified common dicul- ties students have when conceptualizing integrals, especially in physics. To contribute to this literature, we will present results from two studies from the first semester of calculus-based introductory physics that support and extend previous findings. The

first study was conducted in the Autumn 2014 semester, and sought to explore how students understand physics integrals and their components (e.g., di↵erentials and di↵erential products) with open-ended questions in their own words. The second study took place in the Autumn 2016 semester; it helped establish baseline (without research intervention), end-of-course, student abilities to connect symbolic integrals and their components to verbal interpretations and units. This multiple-choice as- sessment was administered electronically at the end of the semester via an online assignment for course credit to a large sample of students. In Section 3.2, we will discuss results from the former study, followed by a presentation of the latter study in Section 3.3.

26 3.2 Physics Integration Exploratory Study

In an early attempt to ascertain the common diculties students may encounter when describing physics integrals, we designed a short, paper-based research task to be administered to first-semester introductory physics students. The task attempted to elicit students’ di↵ering conceptions of di↵erentials, di↵erential products, and in- tegrals, as well as contrasts between macroscopic and microscopic quantities (e.g.,

t vs. dt). As another measure of conceptual understanding in addition to “in- terpretation” of symbolic quantities, students were also occasionally asked to discuss units/dimensions. Our findings uncovered several known unproductive conceptualiza- tions for these quantities and the proportions of students who demonstrate reasoning consistent with them. However, we may have also identified new, as-yet undiscovered diculties. In Section 3.2.1, we will present a more detailed description of the study, its participants, and the research task.

3.2.1 Study Design & Research Methods Student Participant Demographics

This exploratory study was conducted during the Autumn 2014 semester using student participants. All students were enrolled in the first semester of a calculus- based, introductory physics course at The Ohio State University. Students received course credit for their participation or an alternative assignment if they declined the study. A total of 1136 students enrolled in the course participated in at least some PER investigations during the semester, of whom about 74% self-reported as engineering or physics majors. In this particular exploratory study, N =83ofthose students were randomly selected for participation over a period of about one week

27 during the month of October. The average final physics course grade from our sample of 83 students was 72% 2%, which di↵ered from the population of all participating ± students by less than one tenth of a standard deviation. Each participant completed a variety of computer and/or paper tasks in a quiet testing room for no more than

55 minutes.

Exploratory Research Task

Our study utilized a short, paper-based research task that most students com- pleted in under 20 minutes. The task consisted of five items, and each item contained between one and four questions to be answered in an open-ended format. Students were provided with a small reference table to familiarize themselves with the symbols, symbol names, and symbol units that would be most frequently used during the task.

This reference table is seen in Figure 3.1.

Figure 3.1: Reference table of symbols, names, and units provided to students com- pleting our exploratory research task.

The largest component of the task was the first item, “Symbolic Comparisons,” for which there were a total of four subtasks to complete. One of the four Symbolic

Comparison subtasks is shown in Figure 3.2 as a sample. Each of the four Symbolic

28 Comparisons subtasks provided students with a short contextual scenario, followed by fill-in-the-blank tables to compare two symbolic representations of di↵erential, di↵erential product, or integral quantities. The Symbolic Comparison tasks were designed to assess several student abilities shown below:

“Interpretation” - Briefly (a few words) interpret physical meaning of physics • di↵erentials, di↵erential products, and integrals

“Units” - Identify physical units of physics di↵erentials, di↵erential products, • and integrals

“Di↵erences” - Verbally contrast two similar but distinct symbolic physical • quantities, at least one of which is a physics di↵erential, di↵erential product, or

integral

Figure 3.2: Sample Symbolic Comparison subtask from our exploratory research task.

The Interpretation and Units skills were chosen because they represent two dis- tinct measures of conceptual understanding of symbolic quantities. The Di↵erences

29 skill was included to better determine students’ recognition of other relevant features of interest, such as the relative size of microscopic and macroscopic quantities or the significance of the di↵erential within the integral. The following Symbolic Compar-

x dx 1 1 isons were used in this research task: t vs. dt; t vs. dt ; m F t vs. m Fdt;and Fdx vs. Fdt. R R In addition to the Symbolic Comparisons, another item asked students to read a disagreement between two students (“Student A/Student B”) regarding the physical nature of the di↵erential within an integral. Students were instructed to choose one argument with which they most agreed and explain their reasoning. This item is displayed in Figure 3.3.

Figure 3.3: “Student A vs. Student B” item from our exploratory research task. This item was designed to estimate the frequency of the conceptualization that di↵erentials within integrals are only labels.

The Student A/Student B item sought to determine the proportion of students who believe that the di↵erential within an integral is an artifact of notation and contains no physical information. Student A argues that dt represents an actual physical quantity multiplying v within vdt. This understanding of dt is consistent R with the Riemann-structure of the integral, namely that a physics integral is a sum of products. We consider this to be the more expert-like view of the role of dt. By contrast, Student B sees dt as little more than instruction - it informs the student

30 that the integral will be calculated “with respect to” the variable t. Students who perceive the di↵erential as relatively meaningless may also misattribute the units of the integrand to the resultant quantity when computing the integral, such as claiming that vdt has units of meters/second. Indeed, they may not even recognize the R resultant quantity as being physically distinct from the integrand at all.

Finally, our task also featured an item intended to ascertain the percentage of students who believe that any integral “cancels” any derivative. In their introductory calculus courses, students are introduced to the word “anti-derivative,” which they compute in the form of indefinite integrals of a single variable, f(x)dx. As discussed R in Chapter 2.1, the variable of integration is rarely emphasized as a salient feature of traditional, end-of-chapter, calculus homework problems involving integrals, and students may not attend to its importance when dealing with physical quantities.

Figure 3.4: Discussion item from our exploratory research task. This item was in- tended to estimate the frequency of the notion that integrals and derivatives are necessarily inverse operations.

As a result, we chose the item displayed in Figure 3.4 to better understand the extent to which students recognize that derivatives and integrals are not necessarily

“inverse operations” when the variable of integration is not the variable of di↵eren- tiation. This is yet another measure of students’ conceptual understanding of the integrand and its relationship to the di↵erential, as we expected that students who

31 were able to make physical sense out of the di↵erentials (in both the integral and the derivative contained therein) would disagree that the units of the overall quantity would match the units of the variable p. On the other hand, students who believe that

“any integral negates any derivative” may be more likely to see the overall quantity as equivalent to p or at least possessing the same units, which may hinder conceptual understanding in physics.

In general, this research task was not intended to comprehensively or reliably assess student understanding of physics integrals or their components. However, it was able to provide valuable information regarding the existence of some common, previously-established student diculties, including estimates of the proportion of students who encounter them. Furthermore, several unforeseen and unproductive conceptualization of di↵erentials and di↵erential products were also uncovered. These results are discussed below in Section 3.2.2.

3.2.2 Results & Discussion

To analyze the data provided by our exploratory research task, we began by categorizing incorrect student responses into similar groups (“common diculties”) based on the apparent reasoning employed. We elected to count individual instances of each incorrect conceptualization as “present” for each student if the student indicated a response to an item that was consistent with that conception at least once in the task. We can assert that a student provided responses consistent with a given incorrect conceptualization at times, if not regularly, and these frequencies between students can be estimated. These frequencies are displayed for each identified common diculty in Table 3.1. Errors represent one standard error for binomial proportions,

32 Table 3.1: Frequencies of some common diculties. Frequencies reflect the percentage of students (out of 83 participants) who responded in a manner consistent with each conception at least once. Errors represent one standard error for binomial proportions.

Common Diculty Observed Frequency Di↵erential products are meaningless outside of integrals 11% 3% Di↵erentials are unitless 18% ± 4% Di↵erentials are “instantaneous values” 27% ± 5% Integrals necessarily “cancel” derivatives 39% ± 5% Di↵erentials are rates, slopes, or derivatives 56% ± 5% Di↵erentials in integrals are “just labels” 72% ± 5% ±

p (1 p) · p = N . In this equation, p represents the proportion of students who answered q in a manner consistent with a common diculty, 1 p represents the proportion who did not answer in that way, and N refers to the sample size.

Some of these unproductive conceptualizations of di↵erentials have been observed among the existing literature, most notably that di↵erentials o↵er little information beyond mere notation (Artigue et al., 1990)[4]. This general notion is specifically ex- pressed by several items in Table 3.1, including “Di↵erential products are meaningless outside of integrals,” “Di↵erentials are unitless,” and the most widely-observed, “Dif- ferentials in integrals are just labels.” These conceptions were seen in all three items of our research task, though this was especially apparent in responses to the Student

A/Student B in Figure 3.3, in which most students explicitly refuted the meaning of dt provided by Student A. We demonstrate an example of this conceptualization provided by one of our participants in Figure 3.5.

Additionally, the lack of perceived meaning from di↵erentials alone may also con- tribute to the idea that “Integrals necessarily cancel derivatives.” From the task item

33 Figure 3.5: Written response sample from a participant in our exploratory research study. This student indicates that di↵erentials do not have physical meaning within an integral; the student claims that they are simply labels.

shown in Figure 3.4, we found that more than one third of participants agreed that

dp integrating the quantity dt would return physical units equivalent to those of p, even though the di↵erential within that integral was dx, rather than dt. It is important to note that this task did not ask students to compute an integral, nor did it ask students to identify the units of the integral; students were only instructed to determine if the units of the integral matched those of the quantity p.

Interestingly, we also observed evidence of two unexpected incorrect conceptions, namely that di↵erentials are “instantaneous values” or they are “rates” (or “slopes” or “derivatives”). A review of existing literature did not alert us to their existence and we are unaware of previously-published instances of these notions. In the case of the former, 27% of students provided at least one response to our Symbolic Comparisons subtasks that indicated that they believed that di↵erentials represent the value of a function “at an instant.” This is reminiscent of the “Point Resource” discussed by Hu and Rebello, in which students referred to a di↵erential amount of charge dq as being

“at a point” (Hu and Rebello, 2013b) [26]. However, we believe our results di↵er

34 subtly from this conceptualization. For example, consider the Symbolic Comparison

1 subtask displayed in Figure 3.2. Some students referred to the quantity m Fdt, which Newton’s Second Law would demonstrate to be an extremely small change in velocity dv, as the “velocity at an instant.” An example of this interpretation from one of our students is provided in Figure 3.6.

Figure 3.6: Written response sample from a participant in our exploratory research study. This student may perceive the di↵erential dt as a kind of “instantaneous value” operator.

Taken literally, these students are describing the quantity v(t)(orsimplyv)- the velocity of an object at a chosen moment in time t. Although these students

1 demonstrate a recognition of the units of the quantity m Fdt, their interpretation of its physical meaning cannot be considered correct. It is unclear if these students

1 perceive m Fdt to be a “change in” velocity and are not able to express this in words, or if they actually view it as the value of some velocity. To explain this phenomenon, we hypothesize that students who provided responses consistent with the notion that di↵erentials are “instantaneous values” may associate the symbol “d”withtheword

35 “instantaneous” or the phrase “at an instant.” This could arise from calculus and/or physics instruction that highlights the di↵erences between average and instantaneous

x dx rates, such as t vs. dt . In the absence of the more expert-like “small amount” conception, this interpretation is not an unreasonable stretch from a student’s per- spective.

Furthermore, without a firm understanding of what a di↵erential means, many students may draw upon the more familiar cognitive resource of rates to assist in interpreting them. This was observed at least once from more than half of our par-

dx ticipating students. Students who recall that the symbols dt represent a “rate” may connect the symbol “d”withtheconceptofderivativesorslope.Indeed,thisissimilar to the “Di↵erentiation Resource” (Hu and Rebello, 2013b) [26] that some students activate when writing y = k f(x) dy = d (k f(x)) = k df (x). However, rather · ! · · than a form of notational convenience to express an operation, we found evidence to suggest that at least some of these students believed that the di↵erential was actually arate.Forexample,somestudentswereexplicitthatthequantitydt was the “rate at which time changed,” even going so far as to ascribe to it rate-like units, “1/seconds.”

For an example of this, see Figure 3.7. Not all students were consistent in this regard across all items of our research task, employing this conception selectively at times.

Nevertheless, this particular interpretation was unforeseen and we are not aware of its prior documentation; it provided new insight into how a large proportion of our participants conceptualized di↵erential quantities.

Because this task permitted open-ended answers from participants, we catalogued their responses for future tasks to use as convincing distractors. In Section 3.3, we will examine the design of a physics di↵erential, di↵erential product, and integral

36 Figure 3.7: Written response sample from a participant in our exploratory research study. This student interprets the di↵erential dt as a rate or “derivative.” Note that the student included “rate-like” units.

assessment that was directly influenced by the outcomes of this exploratory study.

This instrument was intended to assess a student’s ability to connect the symbolic representation of a physics integral (or components) to its verbal meaning and units.

The assessment allowed us to better quantitatively establish the prevalence of some of the diculties discussed above by utilizing a very large sample of introductory physics student participants, which will be discussed in Section 3.3.2.

3.3 Online Physics Integral Assessment Study

Two years after the original exploratory study discussed in Section 3.2, in the

Autumn 2016 semester, we obtained new data as part of a larger experiment that permitted more conclusive estimates for the frequencies of some of the previously mentioned incorrect conceptions. This study featured an assessment designed to gauge connections between symbolic and verbal representations of physics integrals and their components, as well as their units/dimensions. Our investigation opens

37 with the design of this assessment in Section 3.3.1, followed by results from its im- plementation in Section 3.3.2.

3.3.1 Study & Assessment Design

Students who participated in this study were enrolled in the first semester of the calculus-based, introductory physics (“Physics 1”). Research data was gathered from students at the end of the semester (late November 2016) via an online assignment for course credit, with full credit for participation. All students enrolled in the course were required to complete the assignment from which the data was taken; however, only students who consented to the use of their results for research purposes were included in this study’s data. In total, N =1102studentscomprisethissample.More than 80% of enrolled students chose to participate in this study. No time limit was imposed for the active duration of this research task, but all students were provided the same five-day period within which to complete it. Students were instructed to answer questions without seeking assistance and were not permitted to return to any previously completed items after submitting a response.

To establish relationships between introductory physics students’ understanding of di↵erentials, di↵erential products, and integrals, this assignment was administered through an online assessment instrument developed earlier for research purposes. The instrument consisted of two parts, with of total of 26 multiple choice items.

The first half of the task consisted of “physical interpretation” items presenting a given symbolic representation of a di↵erential, di↵erential product, or integral (e.g., dt) and asking the student to identify the correct physical description (e.g., “An ex- tremely short duration of time”). Note that we used such conceptual and physical

38 Table 3.2: Physical quantity categories and physical contexts of assessment items. Each context in the second column corresponds to two assessment items: a physi- cal interpretation question and an explicit units question, both employing identical physical scenarios. For examples of individual items, see Appendix A.

PhysicalQuantity AssessmentItemContext Di↵erentials dt, dx, dM Di↵erential Products vdt, adt, Fdx, PdA, AdB Integrals vdt, adt, Fdx, PdA, AdB R R R R R descriptions as “extremely short” rather than “infinitesimally small,” because student interviews from an earlier study revealed that the latter phrase was highly attractive due to recall from prior learning experiences, such as in calculus class; however, stu- dents were generally unable to explain why they chose it, hence our avoidance of the phrase. The “interpretation” questions that comprised the first half of the assess- ment varied in physical context, incorporating both concrete (“ vdt”) and abstract R (“ AdB”) contextual scenarios. R In particular, the first half of the assessment began with three items to assess the ability to interpret di↵erentials in three physical contexts; this was followed by

five questions relating to di↵erential products in a variety of contexts; finally, the last five questions pertained to integrals, also in a variety of contexts. Note that for each di↵erential product item (e.g., vdt), there was also a corresponding integral item

(e.g., vdt)ofidenticalphysicalcontextlaterintheassessment.Foranexample, R see Figure 3.8. Other sample assessment interpretations questions can be found in

Appendix A.1.1. In addition, all physical contexts/symbolic representations from each item are shown in Table 3.2.

39 Figure 3.8: Sample Physics Di↵erentials, Di↵erential Products, Integrals Assessment Item. This item assessed the ability to interpret the physical meaning of the integral t2 t1 vdt R

The second half of the instrument consisted of “units” items. These items were identical to the items in the first half, except students were asked to identify the correct units for a given di↵erential, di↵erential product, or integral. Several examples are displayed in Appendix A.1.2. Because students were not permitted to return to previous questions after answering, they were unable to cross-reference interpretations and units between identical contexts, and perhaps change the answers from the first half to match those of the second half of the assessment.

This assessment (and an analogous version crafted for the second semester of calculus-based introductory physics) served as a rich data source for several studies, including those discussed in Chapters 4 and 6. For the purposes of this chapter, we

40 Table 3.3: Observed frequencies of some common diculties from online assessment. Frequencies reflect the percentage of students (out of 1102 participants) who re- sponded in a manner consistent with each incorrect conceptualization at least once (second column) or more than once (third column). Errors represent one standard error for binomial proportions.

Common Diculty At least once More than once Di↵erentials are “rates” 62% 1% 45% 1% Di↵erential Products are “rates” 67% ± 1% 54% ± 2% Di↵erentials are “instantaneous values” 13% ± 1% 1.7% ±0.4% Di↵erential Products are “instantaneous values” 56% ± 1% 34%± 1% Integrals accumulate integrands 49% ± 2% 15% ± 1% Di↵erentials are unitless 19% ± 1% 7% ±1% Di↵erential Products are unitless 20% ± 1% 6% ± 1% Di↵erential Products are meaningless without 22% ± 1% 5% ± 1% ± ± R will restrict our focus to the common diculties with which these data are consis- tent. However, a fuller dissection of the assessment, including reliability statistics and descriptive statistics for performance will be presented in Chapter 4.

3.3.2 Results & Discussion

Our investigation of Physics 1 students’ potentially unproductive conceptualiza- tions of physics di↵erentials, di↵erential products, and integrals via our online as- sessment proved to be informative. Compared to the aforementioned paper-based, exploratory study in Section 3.2, we were able to much more confidently determine the proportions of students who appear to answer in many of the incorrect patterns already discussed. A very large sample size of N =1102wasespeciallybeneficial in a↵ording improved resolution. The results from this study are summarized in

Table 3.3.

41 An immediate observation from Table 3.3 is the high percentage of students who have interpreted a symbolic di↵erential or di↵erential product as a rate, at least once

(as shown in the second column) or multiple times (as seen in the third column). Our assessment provided three di↵erentials and five di↵erential products to be interpreted by our participants, which was displayed in Table 3.2. With this consideration in mind, we can see that nearly half (45% 1%) of our N =1102studentsinterpreted ± at least two of our three di↵erentials (dt, dx,anddM)asrates.Thepercentageof students who selected rate interpretations for multiple di↵erential products is more than half (54% 2%). As discussed in Section 3.2.2, it is not clear why so many ± students perceive quantities such as vdt to be rates (or “derivatives”), but this could

d d be a grasp at the more familiar dx and dt operators seen from their calculus courses, in the absence of a firmer understanding of di↵erentials as used by physicists. That is to say, students may not know precisely what the symbol d means in isolation, but having seen it among rates, they may simply conceive of it as a “rate operator.”

Also seen in Table 3.3 are the frequencies of answer selection consistent with the belief that di↵erentials and di↵erential products are “instantaneous values” of a func- tion. As noted in Section 3.2.2, we were previously unaware of the existence of this conceptualization of di↵erentials and di↵erential products, but having encountered it in our exploratory study, it later became a featured distractor for many of our assess- ment items. Our results indicate that, in fact, few students ever choose di↵erentials interpretations consistent with this notion (13% 1%). However, as we found in the ± 1 exploratory study with the quantity m Fdt,di↵erential products o↵ered a much more enticing temptation for this interpretation. Among di↵erential products, more than half of students (56% 1%) selected at least one answer claiming that a di↵erential ±

42 product was a value of some physical variable at an “instant,” and a third chose this interpretation on multiple occasions from our five di↵erential products interpretation questions.

We should note that not all of these “at an instant” answer choices were isomor- phic between items. For example, the di↵erential product vdt permitted two such interpretations, “the velocity of the car at a single moment” and “the position of the car at a single moment.” Only one of these choices actually demonstrates a recogni- tion of the correct units of vdt, which has dimensions of length. Due to contextual di↵erences, not every di↵erential product item could feature multiple “instantaneous value” interpretations. However, in two cases (vdt and adt), multiple “instantaneous values” answer choices were present, and we observed that more students preferred the “instantaneous value” interpretation with the incorrect units than the correct units. Indeed, when making sense of vdt, 200 students labeled this quantity “the velocity of the car at a single moment,” while only 118 students considered it “the position of the car at a single moment.” Likewise, for adt, 177 students selected the answer choice “the acceleration of the toy at a single moment,” compared to the 137 who referred to adt as the “velocity of the toy at a single moment.” This is a notable result among students in the “instantaneous value” category, because it suggests a preference to see the di↵erential as a kind of “instantaneous operator” rather than aquantity(oroperator)possessingphysicalunitsofitsown.Still,thereisevidence that this may not be a consistently employed interpretation: our results indicate that only 35 of the 200 students who claimed that vdt was the “the velocity of the car at a single moment” also chose dt to be unitless; contrastingly, 71 of those same 200

43 students claimed that vdt had units of “meters/second.” In other words, more stu- dents are claiming that vdt has units of v than are claiming that dt has no units. Do these students see the solitary dt as a discernibly di↵erent entity than the quantity appended to the end of v in the product vdt? What implications does this uncer- tainty hold for their ability to interpret physics integrals? This point remains unclear without more extensive, one-on-one student interviewing, but it is apparent that this unexpected conceptualization of di↵erentials and di↵erential products belongs in fu- ture discussions of the subject, and we are currently unaware of its mention among existing research.

Specifically within the area of unitless di↵erentials and di↵erential products, we also found a noteworthy proportion of students answering in a manner consistent with this belief. Table 3.3 shows that approximately one fifth of our participants interpreted both di↵erentials and di↵erential products as unitless at least once (19% ± 1% and 20% 1%, respectively). We found that this was not likely to occur multiple ± times. This units selection was most common for items dt and AdB; the latter was our abstract context, devoid of specific physical meaning. Students may have been tempted to declare this quantity unitless because the symbols A and B did not represent actual physical variables; however, we were careful to point out that these variables had physical units of A and B in the text of the question. Still, if this instruction was missed, this could explain some of the variance we observed.

Similarly, about one fifth of students ( 22% 1%) answered questions in a manner ± that suggests they believe di↵erential products cannot be interpreted without an integral symbol, “summa” . As in the case of unitless di↵erential products, this R conceptionualization also dwindled in frequency of multiple observations per student.

44 Surprisingly, we found less overlap between these two possible diculties than one might anticipate. It is reasonable to expect that “meaningless” di↵erential products may also be unitless. Nevertheless, in consideration of a representative example, Fdx, we observed that 146 students declared this quantity meaningless without the symbol

, but only 14 of those students also believed it to be unitless. In some sense, it R appears to have at least some physical meaning among the vast majority of students who considered it meaningless. Note that 67 of those 146 students attributed to

Fdx the correct units of Newtons meter, while another 35 chose the “rate-like” units · of Newtons/meter, though they did not interpret Fdx as a rate when a↵orded the opportunity.

Finally, we turn our attention to a commonly-observed issue that is central to our question of how students make sense of symbolic physics integrals and how to improve their abilities to do so: “integrals accumulate integrands.” This understanding of integration is characterized by the belief that integrals are sums, but not sums of products. For example, students who chose answers consistent with this interpretation might claim that the meaning of our abstract integral AdB was, “the accumulation R of A,” or that Fdx represented the “total force” on an object. This was observed R in about half of our participants at least once (49% 2%), though it was much less ± likely to occur twice or more (15% 1%) for a single student. While not necessarily ± a consistently demonstrated misunderstanding across all contexts, it does signify the importance of stressing the Riemann-structure of integrals, specifically that integrals represent a sum of products between the integrand and the di↵erential. Students who view an integral this way have been shown to find more success in integral setup than

45 students who do not, as we illustrated in Chapter 2.3. This particular interpretation of physics integrals motivated future work to be discussed in Chapter 6.

3.4 Conclusions & Outlook

In this chapter, we have quantitatively substantiated some known diculties that students have when conceptualizing physics integrals and their components. In ad- dition, we believe we also have evidence for some previously-unknown diculties.

Among the most pervasive of these types of unproductive reasoning appear to be the ignorance of (or inattention to) the physical meaning of di↵erentials and di↵erential products, and the belief that these quantities are rates or derivatives by themselves.

The former is an over-arching theme of many student diculties with physics inte- grals, and it may be one of the crucial, missing pieces for interpreting them as a physicist would.

As instructors, it is important for us to be aware of the pitfalls that students may encounter when accessing this subject from the perspective of a physicist for the first time; in particular, we must acknowledge that students’ prior conceptions of physics integrals and integral components may have implications for how best to introduce them instructionally. Furthermore, many of the conceptual complexities on which our students stumble may be dependent on the physical context in which they are presented - and not all of these issues can be so conveniently and easily addressed in the same manner across all contexts, because students may not consistently draw on the resources that give rise to them.

Looking ahead, Chapter 4 will seek to establish some inter-relationships between the understanding of physics di↵erentials, di↵erential products, and integrals. The

46 assessment presented above allowed us to find connections between scores on these three skills that suggest an instructional way forward to help overcome some of the struggles uncovered in this chapter. Of principal importance will be the physics di↵erential product, apparently situated at the instructional junction of di↵erentials and integrals, and its role in guiding understanding towards our ultimate goal of connecting the symbolic physics integral to its verbal meaning.

47 Chapter 4: The Mediating Relationship of Di↵erential Products in Understanding Physics Integration

4.1 Introduction

In order to find ways to better address the fundamental diculties with integration seen in Chapter 3, we developed an assessment first introduced in Section 3.3. This assessment allowed us to measure more precisely our skill of interest: the ability to extract physical meaning from symbolic integrals and their components within the context of introductory physics. However, an additional benefit of this instrument was the opportunity to uncover potential relationships between each of its assessed skills (i.e., deriving physical meaning from di↵erentials, from di↵erential products, and from integrals). The interrelationships of these skills may have implications for the most productive method and order of instruction. For example, if performance on di↵erentials items is statistically uncorrelated with performance on integration items, it may be unreasonable to teach students about physics di↵erentials while expecting improvement for their understanding of integrals. Due to its design, our assessment could reveal these hidden connections.

Using our di↵erentials, di↵erential products, and integrals assessment, this Chap- ter will:

48 1. Present validity and reliability data supporting the design of our assessment

2. Identify interrelationships (including statistical mediation) between the abilities

to make sense of physics di↵erentials, di↵erential products, and integrals within

introductory physics

3. Discover evidence to suggest possible moderation due to student-familiarity with

the physical contexts in the assessment

4.1.1 Motivation

In Chapter 2.4.1, we discussed the process-object layered framework for under- standing integrals (specifically in the context of physics) developed by Von Kor↵ and

Rebello. It o↵ers a variety of instructional paths for guiding student learning towards understanding physics integrals. Its “network” of layers o↵ers strictly “microscopic”

(infinitesimal) layers, such as di↵erentials and di↵erential products, which can be summed to produce “macroscopic” (finite) results. The authors assert that physics students can be taught that “an integral is ‘the sum of many infinitesimally small quantities,’ without harm to their understanding.”

According to their model, along the microscopic path to understanding integrals, the di↵erential product lies between the di↵erential and integral (e.g., dt vdt ! ! vdt). In our study, we hypothesize that that this chain suggests that conceptual un- R derstanding of di↵erential products “mediates” understanding of integrals. Although probably intuitive and credible to experts, the veracity of such a relationship has not been quantitatively established among a population of student participants, and its potential existence could pose implications for physics integration instruction. In the next section we describe how mediation may be formally defined and measured.

49 4.1.2 Regression-Based Statistical Mediation

In an early and influential work, Baron and Kenny stated that “a given variable may be said to function as a mediator to the extent that it accounts for the relation between the predictor and the criterion” (Baron and Kenny, 1986) [7]. In simple mediation, a third variable exists as an explanatory mechanism or process that un- derlies an observed relationship between an independent variable X and a dependent variable Y (see Figure 4.1). By definition, a mediator variable M exists causally between X and Y , such that changes in X a↵ect M, whose changes thereby a↵ect

Y . Baron and Kenny argued for a set of criteria to establish the existence of a me- diating relationship between these three variables, which is frequently referred to as the “Causal Steps Approach” (Hayes and Rockwood, 2016) [22]. The following set of linear regression models is used to assess whether mediation exists for these variables via Ordinary Least Squares (OLS) regression:

ˆ Y = iY1 + cX (4.1)

Mˆ = iM + aX (4.2)

ˆ Y = iY2 + bM + c0X (4.3)

The symbols iY1 , iM ,andiY2 are constants (or “intercepts”), while the symbols a, b, c,andc0 are the regression coecients to be determined by OLS regression. These equations are visually represented by the diagrams in Figure 4.1. In the “Causal

Steps Approach,” a significant “Total E↵ect” is obtained when a regression analysis returns a value for the coecient c in Equation 4.1 that is statistically di↵erent from zero. This establishes an overall relationship between variables X and Y . Next, a

50 statistically significant value of a in Equation 4.2 suggests a relationship between the independent variable X and the suspected mediator M. Finally, by Equation 4.3, if the coecient b is significantly di↵erent from zero, while c0 (known as the “Direct

E↵ect”) is closer to zero than c (or statistically indistinguishable from zero), then mediation is said to exist via M between X and Y . The final step demonstrates the diminished direct relationship between X and Y while controlling for the e↵ect of M.

Figure 4.1: TOP: A visual representation of the Total E↵ect of a simple mediation model. Mathematically, this is shown in Equation 4.1. BOTTOM: A visual represen- tation of the Indirect and Direct E↵ects of a simple mediation model. Mathematically, this is shown in Equations 4.2 and 4.3.

51 This intuitive and straightforward recipe for determining the existence of medi- ation has been popular for several decades, but carries with it several drawbacks making it undesirable for use as an analysis tool in our study. Hayes and Rockwood provide an excellent argument (Hayes and Rockwood, 2016) [22]. To summarize: philosophically, an empirical claim should be supported quantitatively by the e↵ect most directly pertinent to that claim, particularly in as few inferential steps as nec- essary to minimize compounding uncertainty. In the case of mediation, establishing the existence of a non-zero “Indirect E↵ect” of X on Y through M is sucient; this is achieved by finding a statistically significant, non-zero product of regression coecients ab, which define the visual path from X to M to Y as seen in Figure 4.1.

Furthermore, while the Causal Steps Approach may still be informative in media- tion analysis, it cannot convey an expression of confidence or uncertainty in the princi- pal quantity of interest, the Indirect E↵ect ab. By contrast, the bootstrap resampling technique (or “bootstrapping”) calculated by the PROCESS macro for the SPSS sta- tistical software can provide a confidence interval for the Indirect E↵ect (Hayes, 2013;

Preacher and Hayes, 2004) [21] [40]. Bootstrapping employs several thousand itera- tions of random sampling with replacement from an existing data sample to model inference about a population. Through successive resampling simulations, measures of accuracy can be obtained for sample estimates, including confidence intervals for the Indirect E↵ect ab in mediation. Moreover, unlike other popular statistical meth- ods, the reliability of bootstrapping is not contingent on normally distributed samples.

This is a crucial benefit, as the sampling distribution of an Indirect E↵ect is not likely to be normal (Hayes and Scharkow, 2013) [23].

52 4.2 Methods & Design

4.2.1 Study Participants

Data from this experiment was obtained from two di↵erent student participant samples at the Ohio State University during the Autumn 2016 semester. The first group of students were enrolled in the second semester of the standard calculus- based, introductory physics sequence (“Physics 2”). Students in this group received course credit for their participation in our research or an alternative assignment if they declined consent for the study. Over 80% of students enrolled in the course participated in at least some PER investigations; N2 =65ofthoseparticipantswere randomly selected for this particular study. Each participant completed paper tasks in a quiet testing room for no more than 55 minutes. Data was obtained for a duration of two weeks with approximately one-third of the semester remaining.

Students in our second group were enrolled in the first semester of the stan- dard calculus-based, introductory physics sequence (“Physics 1”). Research data was gathered from students at the end of the semester via an online assignment for course credit, with full credit for participation. These data were also used in a separate study to determine frequencies of common diculties, seen in Chapter 3.3. While all enrolled students were required to complete the assignment from which the data was taken, only students who consented to the use of their results for research pur- poses were included in this study’s data. In total, N1 =1102studentscomprised the Physics 1 group. More than 90% of enrolled students chose to participate in this study. No time limit was imposed for completion of the research task for this group.

Because these data were obtained from both Physics 1 and Physics 2 samples during the same semester, there was no overlap in participation across groups.

53 4.2.2 Study & Assessment Design

As discussed in Chapter 3.3, to help establish relationships between introductory physics students’ understanding of di↵erentials, di↵erential products, and integrals, two assessment instruments were developed and administered to each sample. The two instruments were very similar; only the physical contexts were adjusted to best suit the physics topics relevant for each course. The instruments consisted of two parts, with a total of 26 multiple choice items for Physics 1 and 30 multiple choice items for Physics 2.

The assessments were divided into two halves: “interpretations” and “units.” The

first half of the task provided participants with a symbolic representation of a di↵er- ential, di↵erential product, or integral and requested a correct verbal interpretation from a list of several choices. Interpretations items varied in physical context, includ- ing concrete and abstract (“ AdB”). Examples of assessment interpretation items R are shown in Appendices A.1.1 and A.2.1 for the Physics 1 and Physics 2 assessments, respectively. The second half of the assessment was identical in both format and phys- ical contexts presented, except students were now asked to identify the correct units for each di↵erential, di↵erential product, and integral. Several examples are displayed in Appendices A.1.2 and A.2.2 for Physics 1 and Physics 2 assessments, respectively.

Furthermore, in Table 4.1, we display the set of all physical contexts for each section of both assessments.

Participants were not able to refer to their units answers to assist in answering the interpretations questions, which was observed in an earlier pilot study. For our

Physics 1 sample, this was managed electronically through the online quiz system on which the assessment was administered; students were unable to return to any prior

54 Table 4.1: Categories and physical contexts of assessment items for each sample. Each quantity in the Physics 1 and 2 columns corresponds to two assessment items: a physical interpretation question and an explicit units question, both employing identical physical scenarios.

Physical Quantity Physics 1 Physics 2 Di↵erentials dt, dx, dM dt, dx, dq

Di↵erential Products vdt, adt, Fdx, PdA, AdB vdt, adt, qEdx, dx, ⇢⇡R2dx, AdB

Integrals vdt, adt, Fdx, vdt, adt, qEdx, 2 R PdAR , AdBR Rdx, R⇢⇡R dxR , AdB R R R R R question after answering. For our Physics 2 sample, this restriction was imposed manually, as these students completed the task on paper; the first half of the assess- ment (interpretations) was physically retrieved before students received the second half (units).

Students were assigned a single score for each of the three categories of physical quantities assessed: di↵erentials, di↵erential products, and integrals. Each category score was calculated by averaging performance on all items (interpretations and units; all physical contexts equally weighted) pertaining to that category. For example, a student who correctly answered all di↵erentials units questions, but incorrectly an- swered all di↵erentials interpretations questions, would receive a di↵erentials score of

50%. There were two reasons for choosing this scoring method. The first is theoretical and conceptual: each assessment dimension (interpretations and units) represents an aspect of conceptual understanding of the quantity, namely, “what kind of quantity it is” and “what it physically means.” Both represent operational means of assessing

55 conceptual understanding of the symbolic representations of these quantities. The second reason is empirical: for a given quantity the scores between the two dimen- sions were highly correlated, thus supporting the validity of the scoring method, as is shown in the next section.

4.2.3 Assessment Validity & Reliability

The assessment instruments were developed from (and based on) prior work, in- cluding several semesters of design iterations, implementation and item analysis, stu- dent interviews, as well as adjustment, addition, or removal of items based on the results. Physical contexts and quantities were chosen based on calculus expressions commonly found in introductory calculus-based physics courses; student interviews verified the relevance of the expressions. Important distractors were derived from free response questions pertaining to interpretations of di↵erentials and di↵erential prod- ucts, such as the incorrect notion that di↵erentials are “rates” or “derivatives” (Amos and Heckler, 2015) [3]. In addition, other incorrect answer choices were produced from questions relating to interpretations of integrals, such as the misconception that the di↵erential at the end of an integral is merely a label that contributes neither meaning nor units to the integrand (Amos and Heckler, 2014) [2]. These conceptualizations and the tasks that yielded them are presented in Chapter 3.

In both Physics 1 and Physics 2 studies, the assessment instruments were found to be statistically reliable, both as a total score and as individual category scores (e.g.,

“di↵erentials score”). In Table 4.2, we display the reliability measure Cronbach’s

↵ (a metric of the internal consistency of the instrument’s measurement; Cronbach,

56 Table 4.2: Assessment reliability statistics. Cronbach’s ↵ is shown for both the Physics 1 and Physics 2 samlpes for each set of assessment items. Below each Cron- bach’s ↵, we show the number of items for each Physical Quantity from both assess- ments.

Physical Quantity Physics 1 Physics 2

Di↵erentials 0.843 0.807 Items: 6 Items: 6

Di↵erential Products 0.863 0.877 Items: 10 Items: 12

Integrals 0.790 0.826 Items: 10 Items: 12

All 0.913 0.909 Items: 26 Items: 30

1951 [10]) for di↵erentials questions, di↵erential products questions, integral ques- tions, and all questions (combined) for both samples of participants. Within each set of questions pertaining to a single physical quantity (e.g.,di↵erential products), we found good internal consistency. In Kline’s “Handbook of Psychological Test- ing,” the author states that a reliability of ↵ =0.7isa“minimumforagoodtest”

(Kline, 2000) [29]; every category of our assessment exceeds this value, including the assessment’s reliability value as a whole. Kline adds, “if a reliable 30-item test is split into two parallel 15-item tests, each will be itself reliable, although less so than the composite,” which accounts for the increased Cronbach’s ↵ values for the overall assessment. Finally, we also note that no single assessment item caused noticeably dif- ferent/outlying results if removed, which represents yet another inspection of internal consistency.

57 Table 4.3: Assessment descriptive statistics. Mean scores with one standard error are shown for both the Physics 1 and Physics 2 samples for each set of assessment items.

Physical Quantity Physics 1 (N =1102) Physics2(N =65)

Di↵erentials 47% 1% 73% 4% ± ± Di↵erential Products 32% 1% 59% 4% ± ± Integrals 51% 1% 71% 3% ± ±

4.3 Results & Discussion

4.3.1 Descriptives Statistics

Descriptive statistics for each of the three scores (di↵erentials, di↵erential prod- ucts, and integrals) for each sample are displayed in Table 4.3. Participants from the

Physics 2 population were seen to significantly outperform students from Physics 1 in all three sets of assessment items pertaining to each physical quantity: Di↵erentials,

F (1, 1165) = 31.4,p < 0.001,d =0.81; Di↵erential Products, F (1, 1165) = 50.5,p <

0.001,d =0.89; Integrals, F (1, 1165) = 32.5,p < 0.001,d =0.79. Note that we use the symbol d in reference to Cohen’s d (Cohen, 1977) [8], a unitless measure of the size of an e↵ect, equal to the di↵erence in sample means divided by their pooled standard deviation, d =(¯x x¯ )/, where = ((N 1) 2 +(N 1) 2) / (N + N 2). 1 2 1 1 2 2 1 2 q The di↵erence between Physics 1 and Physics 2 scores could be explained by several factors, including population di↵erences between Physics 1 and Physics 2 due to attrition from grades or major requirements, as well as mathematics experience

58 and background. In any case, this result is presented for informative purposes only, and will not be a principal focus in the following sections.

A more interesting feature was observed upon analyzing Table 4.3 between rows.

By inspection, it is clear that mean scores on the set of di↵erential products ques- tions were lower in both samples than di↵erentials and integral scores, which were comparable to one another within each course. In the absence of other competing factors, and with a significant Indirect E↵ect (that is large compared to the Direct

E↵ect) from our hypothesized mediation model, one might reasonably predict that integral scores would be lower than (or at best equal to) di↵erential products scores, which would be in turn lower than di↵erential scores. This expected cascading e↵ect of scores could be explained by the fact that many students may successfully demon- strate understanding of di↵erentials, but only some subset of those students could successfully answer di↵erential products questions, and even fewer could demonstrate success on all three topics. To the contrary, our results do not show this.

To verify this apparent contradiction of our expectation, a paired-samples t- test was performed using a within-student design on di↵erential products scores and integral scores. From this, we observed that integral scores significantly ex- ceeded di↵erential products scores (in both samples of participants): Physics 1, t(1101) = 25.7,p<0.001,d=0.77; Physics 2, t(64) = 3.2,p=0.002,d=0.40.

This surprising result prompted us to consider instances in which students could correctly answer questions about an integral but not its corresponding di↵erential

t2 product (within identical contexts) (e.g., correctly interpreting t1 vdt but not vdt,

x2 R or correctly identifying the units of x1 Fdx but not the units of Fdx, etc.). We R found that this possible inconsistency with our proposed model occurred in 27% of

59 responses from Physics 1 students, and 20% of responses from Physics 2 students.

In other words, a fifth or more of student responses in both samples demonstrated conceptual understanding of an integral, but not its di↵erential product. These results could be seen to run counter to our mediation model, at least for the case of zero or very small Direct E↵ects. In Section 4.3.3, we will present evidence supporting apossibleexplanation:forsomecontextsthatareapparentlycommonandfamiliar, students may be able to succeed on integral questions in spite of lower di↵erential products performance.

4.3.2 Mediation from Di↵erential Products

The ability of students to derive physical meaning from di↵erential products (with variable name “DP”) is hypothesized to act as a mediator between the physical un- derstanding of di↵erentials (“D”) and integrals (“Int”) in the context of introductory physics. To assess the existence of this mediation, the following linear regression models were selected, based on those shown in Equations 4.1 4.3:

Intˆ = i1 + cD(4.4)

DPˆ = i2 + aD(4.5)

Intˆ = i3 + bDP + c0D(4.6)

Using the PROCESS macro for the SPSS statistical software discussed in Sec- tion 4.1.2 and k =10000bias-correctedbootstrapsamples,astatisticallysignif- icant Indirect E↵ect was observed in both samples at the ↵ =0.05 level. For

N =1102participantsinthePhysics1group,weobservedanIndirectE↵ect of ab =(0.55) (0.53) = 0.29, 95% CI [0.25, 0.33]. Similarly, for N =65participantsin · 60 Table 4.4: Linear regression model coecients. Means with one standard error are shown for both the Physics 1 and Physics 2 samples for each regression coecient from Figure 4.1.

Model Coecient Physics 1 (Sig.) Physics 2 (Sig.)

a 0.55 0.02 (p<0.001) 0.60 0.10 (p<0.001) ± ± b 0.53 0.03 (p<0.001) 0.46 0.11 (p<0.001) ± ± c 0.38 0.02 (p<0.001) 0.20 0.10 (p =0.040) ± ±

c0 0.09 0.02 (p<0.001) 0.07 0.11 (p =0.521) ± ±

the Physics 2 group, we observed an Indirect E↵ect of ab =(0.60) (0.46) = 0.27, 95% · CI [0.14, 0.48]. Due to the bootstrap confidence intervals (neither of which include zero in their range), we are 95% confident that the Indirect E↵ect in both samples is greater than zero, and thus that mediation exists for this set of variables among both participant groups. The regression model coecients for both samples are displayed in Table 4.4.

Among Physics 1 participants, due to large sample size, all regression coecients were seen to be significant at the ↵ =0.05 level. From this, we identified a significant

Total E↵ect (c =0.38 0.02) and a significant Direct E↵ect (c0 =0.09 0.02), ± ± although the latter is a considerably smaller e↵ect than the Indirect E↵ect (ab =0.29

0.02) mentioned above. Apparently, as suggested by these data, improvement in ± di↵erentials understanding is not directly associated with large improvements in stu- dents’ abilities to understand integrals, relatively speaking. Instead, there is strong

61 evidence that increased performance on di↵erentials is related to increased perfor- mance on di↵erential products, which is thereby related to gains in integral score.

Figure 4.2: TOP: A visual representation of the Total E↵ect of our simple media- tion model. The regression coecient estimating the Total E↵ect is shown for both Physics 1 and Physics 2 samples as c1 and c2, respectively, with one standard error. BOTTOM: A visual representation of the Direct and Indirect E↵ects of our simple mediation model. The regression coecients are shown for both Physics 1 and Physics 2sampleswithsubscripts“1”and“2”,respectively,withonestandarderror.

In Physics 2, the sample size was only 6% of the former group. Consequently, we did not identify a significant Direct E↵ect (c0 = 0.07 0.11); if one exists, there was ± insucient statistical power to discover it. Therefore, we cannot conclude from this sample that improvement in di↵erentials understanding has any direct relationship

62 with integral understanding. However, a significant Total E↵ect (c =0.20 0.10) was ± observed among Physics 2 participants. Again, it is the Indirect E↵ect (ab =0.27

0.09) that dominates this model. The data suggest that di↵erentials gains are ± associated with di↵erential products gains; likewise, higher di↵erential products scores are associated with higher integral scores.

In Figure 4.2, we summarize our findings by displaying the visual representation of the model discussed above, including regression coecients.

4.3.3 Possible Moderation from Context-Familiarity

During the analysis process, we observed that the Indirect E↵ects for several in- dividual physical contexts appeared to di↵er (nominally, if not significantly) from each other depending on the scenario provided in the problem statement. In other words, the same linear regression models shown in Equations 4.4 4.6 can be used to compute the Indirect E↵ect ab for a single individual context featured on our as- sessment (see Table 4.1). For example, we can estimate the Indirect E↵ect ab for assessment items pertaining to the context of velocity vs. time, in which the di↵er- ential, di↵erential product, and integral assessed were dt, vdt,and vdt, respectively. R Among physical contexts featuring integrals with which students were likely very familiar (e.g., vdt), we generally observed lower Indirect E↵ects. Contrastingly, R physical contexts involving integrals that were presumed less familiar to students

(e.g., ⇢⇡R2dx) tended to have slightly higher Indirect E↵ects. This information is R summarized in Table 4.5.

The notion that familiarity may play some role in how students arrive at their understanding of a physical integral is not unreasonable. After all, for a familiar

63 Table 4.5: Indirect E↵ects ab estimated with one standard error for each context separately. These Indirect E↵ects were computed for each context individually, which demonstrates that some contexts present stronger evidence of mediation than others. Familiarity with each context (or lack thereof) may explain some of this variation.

Model/Context Physics 1 Physics 2

dt vdt vdt 0.12 0.01 0.11 0.09 ! ! ± ± R dt adt adt 0.12 0.01 0.08 0.08 ! ! ± ± R dx Fdx Fdx 0.18 0.02 0.24 0.10 ! ! ± ± R dx dx dx 0.16 0.09 ! ! ± R dA PdA PdA 0.25 0.02 ! ! ± R dx ⇢⇡R2dx ⇢⇡R2dx 0.38 0.13 ! ! ± R

context, students might access other cognitive resources to choose an answer instead of using the chain of reasoning from di↵erentials to di↵erential products to integrals studied here. For example, some students may simply directly recall from their current or past math and physics courses that the commonly used collection of symbols

“ vdt” represents a displacement when written in that order, but they may not be R able to correctly interpret the meaning of the di↵erential product vdt, perhaps because it is less commonly encountered without the integral symbol, “ .” This type of R performance could weaken the Indirect E↵ect in contexts where familiarity is already sucient for interpretation of the integrals, but not the di↵erential products. On the other hand, some physical contexts and their respective integrals may be completely foreign to students, such as ⇢⇡R2dx. In this case, students would find more success R 64 comprehending the meaning of the di↵erential product in order to make sense of the integral, which could strengthen the Indirect E↵ect for less-familiar questions.

In a later semester, we polled N =106Physics2studentsintheirfirstmonthof the course, that we might better understand prior experience and familiarity with the integrals assessed in our study from the perspective of introductory physics students.

These students were not previously participants in our Physics 2 mediation study; however, their mean familiarity scores to the prompt, “How would you rank your familiarity with the following quantity? (1 = ‘Not at all familiar with this quantity’ and 5 = ‘Very familiar with this quantity’)” were used to rank the student-perceived familiarity of the integrals used in the assessments. This between-students design is not suited to perform robust statistical tests on context as a moderating factor, but it can provide some qualitative insight into the role of familiarity as an influencing factor in our mediation model. We therefore display Figure 4.3, which pairs mean self-reported familiarity of our assessment integrals with the Indirect E↵ects shown in Table 4.5, measured from the mediation study. Due to the large size of the error bars (each shown as one standard error), particularly among the Physics 2 mediation sample, broad claims cannot be made, but nominal Indirect E↵ect values generally align with our hypothesis that the Indirect E↵ect decreases with increasing familiarity.

To further support the claim that high familiarity seems to reduce the role of di↵erential products as a mediator in correctly answering the conceptual questions on integrals, we returned to the data described in Section 4.3.1 from our mediation study to compare individual physical contexts in which students demonstrated conceptual understanding of a given integral, but not its di↵erential product.

65 Figure 4.3: HORIZONTAL: Mean self-reported familiarity with each physical con- text from Table 4.5. VERTICAL: Completely standardized Indirect E↵ect for each physical context from Table 4.5. One standard error shown for both dimensions. Blue symbols represent contexts presented to Physics 1 students. Red symbols represent contexts presented to Physics 2 students. Note: Familiarity scores were obtained from a separate sample of Physics 2 students in a later semester.

In particular, we found that only 14% of Physics 2 responses were correct for

⇢⇡R2dx items and simultaneously incorrect for ⇢⇡R2dx items (rated the least fa- R miliar Physics 2 assessment context in our poll), whereas 25% of Physics 2 responses were correct for vdt items and simultaneously incorrect for vdt items (rated the R most familiar Physics 2 assessment context in our poll). A chi-squared test showed that this di↵erence was significant at the ↵ =0.05 level for Physics 2 students:

66 2(4, 65) = 19.3,p=0.001, with an e↵ect size of (Cram´er’s) V =0.39 - a large-sized e↵ect.

Likewise, we also found that only 14% of Physics 1 responses were correct for

PdA items and simultaneously incorrect for PdA items (rated the least familiar R Physics 1 assessment context in our poll); but again, this was contrasted by 44% of Physics 1 responses that suggested understanding of vdt without understanding R vdt.Oncemore,achi-squaredtestshowedthatthisdi↵erence was significant at the

↵ =0.05 level for Physics 1 students: 2(4, 1102) = 37.8,p < 0.001, with an e↵ect size of (Cram´er’s) V =0.13 - a small-to-medium-sized e↵ect.

To summarize, our data illustrate that for the least familiar physical contexts, integral success is most paired with understanding of its corresponding di↵erential product; the opposite (which is counter to our mediation model) appears to be true in the most familiar contexts.

In statistical terms, whereas mediation attempts to address how an independent variable X a↵ects a dependent variable Y , the hint of familiarity may be better understood as “moderation” - when and to what degree X a↵ects Y . A detailed discussion of moderation analysis is provided by Hayes and Rockwood (Hayes and

Rockwood, 2016) [22]. The data presented in this study do not permit a rigorous moderation analysis that might conclusively determine the e↵ect of familiarity on our simple mediation model. Instead, a within-student design including familiarity ratings is needed. Nevertheless, our results do support the hypothesis that such moderation exists and warrants further study.

67 4.4 Conclusions

Von Kor↵ and Rebello devised a process-object layered framework with a net- work of possible paths to understanding of physics integrals. Along the “microscopic path” (using di↵erentials to build integrals; e.g., dt vdt vdt), di↵erential ! ! R products are situated between di↵erentials and integrals. We hypothesized that me- diation via understanding of di↵erential products may be present in the relationship between the assessment scores of di↵erentials and integrals; this model was illustrated by Figure 4.2. Among independent student populations (Physics 1 and Physics 2), our results were seen to be consistent with mediation via di↵erential products under- standing. Specifically, neither of the 95% confidence intervals estimating the Indirect

E↵ect ab included zero; this was shown in Section 4.3.2.

If mediation exists along the “microscopic path” to understanding physics inte- grals in the manner suggested by our results, on average, students would find more success from instruction that follows the progression formulated by Von Kor↵ and

Rebello (i.e., dt vdt vdt). In particular, because of the very small (possibly ! ! R non-existent) Direct E↵ect (c0)fromTable4.4,simplyemphasizingthemeaningand units of the di↵erential may have no direct impact on students’ ability to make sense of the integral. However, comprehension of physics di↵erentials may prove benefi- cial in promoting understanding of the di↵erential product, whose instruction may ultimately lead to better integral understanding.

We also find evidence that familiarity can play an important role in understanding integrals. Specifically, our results support the idea that for novel, unfamiliar integrals, instruction emphasizing the microscopic path mediated by the ability to make physical sense of di↵erential products may be an e↵ective method. For more familiar integrals,

68 it is clear that some students may have bypassed important understanding of the components of the integral while still having a general idea of the whole integral itself. This finding may also be useful for designing instruction, noting that the instructional method may benefit from adapting to the student familiarity with the integral of interest.

Finally, it is worth noting that ultimately, mediation models imply an assumed causal relationship. While our results are consistent with mediation via di↵erential products understanding, we would also recommend a controlled experiment involving treatments in the form of instructional interventions to measure the Indirect, Direct, and Total E↵ects discussed in our simple mediation model. Such a study could probe the causal nature of the model, which was an assumption made in this study (in which all students were assessed on all topics without instructional intervention), as demonstrated by the direction of the arrows shown in Figure 4.2.

69 Chapter 5: Instructional Interventions for Student Understanding of Physics Di↵erentials and Di↵erential Products

5.1 Introduction to Di↵erentials Training

5.1.1 Motivation

After conducting an exploratory study to identify and estimate the frequencies of some of the most glaring deficiencies in student understanding of the physical mean- ing of integrals and their components (Chapter 3.2), our next goal was to determine if short, contextual training could lead to improved reasoning and interpretation of physics di↵erentials for introductory physics students (compared with no training).

Existing literature (Chapter 2.3) and our exploration of diculties indicated that physics di↵erentials and di↵erential products imposed significant barriers to under- standing the integral as a whole, and thus became targets for instructional interven- tion. Indeed, although not yet known at the time, we later found evidence to suggest potential conceptual benefits from training along an instructional path beginning with di↵erentials, followed by di↵erential products, and ending with integrals, especially in unfamiliar contexts (Chapter 4). Taken together, a clear picture resolves: training introductory physics students on the meaning of physics di↵erentials and di↵erential

70 products was a worthy endeavor along the road to improving their ability to make sense of integrals.

5.1.2 Instructional Tutorials & Feedback from Computer- Based Instruction

Research-based instructional tutorials have been a fixture of Physics Education

Research for decades (Van Heuvelen, 1991; McDermott et al.,1995;Sokolo↵ et al.,

2011; etc.) [51] [33] [48]. As an example, Real Time Physics by Sokolo↵ et al. engaged students in computer-based, laboratory activities. It also provided immediate feed- back to help relate physical motions and graphical representations. Overall, research has demonstrated that similar Computer Based Instruction (CBI) has yielded gen- erally positive e↵ects, especially when paired with feedback (Hattie and Timperley,

2007) [20].

To take advantage of these results, our experimental approach also utilized tuto- rials, both paper-based without feedback and electronically with immediate feedback from a computer terminal. What are the advantages to using this strategy to at- tempt to produce conceptual gains? Both the paper-based and electronic tutorials mimic recitation, homework, or laboratory-style environments. Consequently, any empirically-established success with this implementation could be fairly easily trans- lated to actual classrooms. The paper-based tutorials were designed to be completed by a single student in a brief time window; similarly, the electronic feedback tuto- rials require only a computer terminal. If improvement is seen with electronic feed- back, this particular di↵erentials intervention could be injected into a physics course without significant disruption to existing curriculum by nature of its simplicity and compatibility.

71 Before continuing, it is important to note that research has demonstrated that the ecacy of electronic feedback depends on a variety of factors, including the knowledge to be learned, student confidence, and the type of feedback given (Hattie and Timper- ley, 2007) [20]. In the case of our physics di↵erentials tutorials, this was considered an empirical question: we compared the trained performances of students receiving elec- tronic feedback with those of students who had roughly identical paper-based training but lacking feedback. A variety of possible outcomes existed with this experiment, including: identical gains between feedback and no feedback conditions; better per- formance from students receiving feedback than paper-based, no feedback tutorials; or perhaps even no statistically significant improvement from either treatment group when compared to untrained control students. Although this experiment represented only our first attempt to help bridge the conceptual gap in physics integration seen in

Chapter 2.3, we nevertheless anticipated that the e↵ect of training would make some noticeable impact.

In the following section, we will examine in more precise detail the methods em- ployed in this study to promote better physical sense-making of symbolic di↵erentials and di↵erential products within introductory physics.

5.2 Experimental Design & Methods

5.2.1 Participant Demographics

Our study participants were sampled from the population of second semester in- troductory calculus-based physics students in the Spring 2015 semester. In total, 90% of the Npop = 885 students enrolled in the course participated in some PER investi- gations during this semester. A sample of 185 students participated in this particular

72 experiment and were randomly sorted into groups: 78 students were placed in two experimental conditions to be trained using paper tasks that did not permit any feed- back (called “VT, FX” and “FX, PV”); 64 students were placed in two experimental conditions (called “VTE, FXE” and “FXE, PVE”) to be trained using computer tasks that provided electronic feedback (correct answers and a brief explanation were given after each question; training was structured identically to paper-based condi- tions, except for the inclusion of feedback from the computer screen); and 43 students received no training prior to assessment (called “Control”).

The average course grade for our sample of students was 79.1% 0.8%, which ± di↵ered from the population average (76.3%) by only one fifth of a standard deviation.

Of our 185 participants, 158 (approximately 85%) self-reported their major discipline as a branch of engineering or physics. Students participated in this experiment from mid-February 2015 until mid-March 2015.

5.2.2 Estimating Necessary Sample Size

By design, our study intended to feature between 30 and 40 students in each of our five conditions (two Paper/No Feedback conditions, two Computer/Feedback conditions, and one No-Training/Control condition). This comes from a pre-selected statistical power 1 =0.8 (a standard power level in social science experiments). Note: Power is defined as the probability of correctly rejecting the null hypothesis when it is false. In our experiment, the null hypothesis H0 can be stated as, “the mean di↵erentials post-test scores of all five conditions are equal,” whereas the alternative hypothesis H1 is, “not all mean di↵erentials post-test scores of all five conditions are

73 equal.” A more detailed discussion of the di↵erentials post-test referenced here can be found in Section 5.2.4, later in this chapter.

To assure an 80% probability of correctly rejecting H0 when it is actually false without the benefit of prior research to provide guiding estimates for reasonable sta- tistical quantities (e.g.,thesamplestandarddeviationofourdi↵erentials post-test scores could not be estimated before our experiment), some rules of thumb must be employed to begin quantifying necessary sample sizes. Thankfully, this task has been greatly facilitated for several decades by Cohen and others (Cohen, 1992) [9].

In his influential primer on statistical power, Cohen provides operational definitions for “small,” “medium,” and “large” treatment e↵ects depending on the experimental design and statistical tests to be used. We had reason to believe that some e↵ect of training would be apparent, most probably from the conditions receiving instructional feedback based on previous literature, so we surmised a “medium” e↵ect size due to training to better identify an appropriate sample size for our experiment. Cohen’s prescription for a “medium”-sized e↵ect with an ↵-level of 0.05, five experimental conditions, and power-level of 0.8 is n =39participantspercondition.Thisnumber is not arbitrary, but it does make some important assumptions, most notably the size of the e↵ect we expect to find. However, it represents the best estimate for the given circumstances, and it was able to provide sucient power to discover several e↵ects to be discussed in Section 5.3.

74 Table 5.1: Di↵erentials/Di↵erential Products experimental treatment design.

Condition Feedback TrainingContexts Participants, N VT, FX No vdt, Fdx 41 FX, PV No Fdx, PdV 37 VTE, FXE Yes vdt, Fdx 33 FXE, PVE Yes Fdx, PdV 31 Control N/A No training 43

5.2.3 Di↵erentials Tutorials

All training conditions received two identically-structured, consecutively-administered training tasks, or “tutorials.” Each tutorial was based on exactly one pair of phys- ical variables, an independent and a dependent variable, which together formed a di↵erential product. For example, students receiving the “VT” tutorial responded to training questions that pertained to velocity v,timet,andthedi↵erential product vdt. This training design is summarized in Table 5.1.

It is important to note that we elected to provide each participant with two sep- arate tutorials in a sequential pair for a variety of reasons, including alternating independent variables (e.g., x, t, etc.), which could help students’ abilities to general- ize across multiple di↵erentials and di↵erential products, as well as varying students’ familiarity of certain contexts, according to our perception of what is more commonly seen in an introductory physics curriculum.

There were no di↵erences in instructional treatment administered to students who received paper-based “VT, FX” training compared to those who received the computer-based “VTE, FXE,” except that the latter received feedback. The same is true for students receiving “FX, PV” training compared to the electronic “FXE,

75 PVE” condition: no di↵erences existed in the structure of the training except that the latter was given feedback by the computer on which their training tasks were completed. Furthermore, no di↵erences existed in the design of the training from the

first tutorial to the second. Every tutorial featured identical questions with only the physical contexts changed.

Our tutorials included items of the following varieties: “Student A vs. Student

B”argumentquestions(similartooneseeninChapter3.2),inwhichparticipants selected the explanation about physics di↵erentials with which they most agreed, based on a debate of competing views; dimensional analysis questions to identify the units of physics di↵erentials; as well as open-ended questions in which the participant was asked to describe di↵erences between finite interval (“macroscopic”) quantities

(e.g., t)anddi↵erential quantities.

Each tutorial gradually tightened the scope of its questions around the notion that physics di↵erentials are extremely small physical amounts/intervals with units.

While this restricted the size of the answer “phase-space” within which students could respond organically, we hypothesized that this would eventually encourage students to reconcile their prior understanding of di↵erentials and di↵erential products with the tutorial’s ever-narrowing description of their properties with each subsequent question. For example, an early “Student A vs. Student B”tutorialitemfeatureda disagreement regarding the fundamental properties of a di↵erential product (“What kind of quantity is this physical symbol?”; Chapter 2.1.2). A sample item from an

FX tutorial, whose arguments were influenced strongly by our exploratory study from

Chapter 3.2, illustrates this in Figure 5.1.

76 Figure 5.1: Sample training item, from early in the FX tutorial.

The broad range of disagreement seen in Figure 5.1 is contrasted with a later tutorial item of the same type, which portrayed a much narrower argument over the subtler attributes of di↵erential products, such as what one precisely means and how it di↵ers from a macroscopic product (“What does this physical symbol mean?”;

Chapter 2.1.1). This is demonstrated by a sample item from the same FX tutorial, shown in Figure 5.2. Students in this debate no longer disagree about “what kind of quantity is Fdx,” (i.e., it is an amount of energy, measured in Newton meters); · instead, their dispute resolves to one of size, where the distinguishing feature of Fdx from F x is that the former is extremely (possibly “infinitesimally”) small.

Although the scope of possible answers to multiple choice items narrowed through- out the tutorials, such as the series of “Student A vs. Student B” questions mentioned above, we were careful to provide participants with opportunities to answer these

77 Figure 5.2: Second sample training item, from later in the FX tutorial.

questions in an open-ended format before moving on to multiple-choice items. Imme- diately preceding a “Student A vs. Student B” question, our participants were asked to first formulate their own interpretations of the symbolic quantities to be debated in the next item. We believed that this tutorial design would serve an important purpose: taking into consideration our goal of encouraging productive interpretations of symbolic physics di↵erentials and di↵erential products, participants needed to be prepared to confront any potential underlying misunderstandings by first examining and verbalizing their own beliefs about such quantities. In principle, this might open the door to re-examination if their prior conceptions were challenged in the next item.

In all, students were trained for about 10 minutes for each tutorial, which re- sulted in approximately 20 minutes of total training. Following training, all trained conditions were given an unrelated interim task for about 10 to 15 minutes to avoid the possibility that performance on our post-test was based solely on their working

78 memory (the brain’s “temporary storage system” under current attentional control - not capable of storing longterm instructional information; Baddeley, 2007) [6]. This was followed by our physics di↵erentials post-test, a description of which is provided in the next section.

5.2.4 Di↵erentials Post-Test

To assess the e↵ect of physics di↵erentials training, we constructed a short post- test. The post-test items were mostly open-ended questions, and many physical contexts were included. The items used to calculate the post-test score are shown in

Appendix B (and discussed in detail below), though other items were included in the post-test that were not used for this particular experiment, as they did not pertain to physics di↵erentials. To compute the post-test total score, each item was weighted equally and scored as “0,” “0.5,” or “1.” The post-test total score was calculated as the mean score of all items.

Abstract Context Post-Test Item

The first post-test item resembled the Student A vs. Student B debates from the tutorials. In this item, several hypothetical students argued over the fundamental physical meaning of the abstract di↵erential product AdB, with unknown physical variables A and B. Students “B” and “C” advocated for incorrect interpretations, which were based on the “instantaneous value” and “rate” interpretations observed in

Chapter 3.2. Because this post-test item was multiple choice, its scoring was indepen- dent of researcher judgment of students’ responses. Therefore, it had the potential to provide perhaps the “cleanest” signal of improvement over untrained control results.

Additionally, it represented pure contextual abstraction, which was not a component

79 of any tutorial training, and we regarded this question as demonstration of “transfer” of learning (the process and extent to which prior experiences a↵ect performance in novel scenarios; see: Perkins and Salomon, 1992) [39].

Velocity vs. Time Context Post-Test Item

The next post-test item requested that participants write an equation showing the distance a bicyclist travels at a speed v0 during an “infinitesimal amount of time.” This open-ended response item would ideally be completed with the equa- tion “dx = v0dt,” although students were not told that the distance traveled during an extremely short duration must also be extremely short. In principle, this could reveal which students were aware that the distance must be dx, however, in prac- tice, this goal was limited by the variety of student responses. This item was scored with values of “0” (for equations not of the form displacement = velocity time or ⇥ expressions that were unrelated to velocity time), “0.5” (for the expression v dt ⇥ 0 without an equation), and “1” (for the correct equation dx = v0dt or an equation indicating displacement = v0dt). Unfortunately, it was not possible to equally judge the intent of all students regarding the symbol dx or the meaning of the word “dis- placement,” which was sometimes inscribed as a full English word on one side of the equation. Some participants simply denoted the displacement “x,” rather than

x or dx. Ultimately, a score of “1” was given to students who responded with an equation indicating displacement (of some form) = v0dt. Whether all of these students viewed the left-hand side of the equation as an actual physics di↵erential cannot be known without one-on-one interviews, and thus, this allowance was made when coding responses.

80 Force vs. Displacement Context Post-Test Item

The third item of our di↵erentials post-test assessed the meaning of the quantity

Fkdx during the braking of a bicycle by friction force Fk. Responses consistent with the interpretation, “a tiny (or ‘infinitesimal’) amount of work” were scored as “1.”

Those responses that indicated that Fkdx represented “work at an instant,” “work at an instant of position,” and “instantaneous work” were given a score of “0.5.”

While this may sound emblematic of the “instantaneous value” misconception from

Chapter 3, there is some ambiguity in the context of work that does not appear in many other contexts, such as velocity. The phrase, “work at an instant of position” could be interpreted to mean, “the work [done] at an instant of position.” Depending on the student’s conception of this phrase, this might mean dW , which is the tiny amount of work done across the distance dx. This is certainly reminiscent of Hu and

Rebello’s “point resource” for the meaning of dx (Hu and Rebello, 2013) [26], but it may not be technically incorrect, unlike the phrase “velocity at an instant of time,” which is absolutely distinct from dv. Consequently, responses of this form were given ascoreof“0.5,”asthisisnotideal,expert-likelanguage,butitdoessuggestthe possibility of some understanding of the quantity Fkdx. Students who deviated from both our preferred interpretation and the latter interpretation received a score of “0.”

Such responses often referred to Fkdx as an amount of force, such as “the force at an instant” or “the force across dx.”

Acceleration vs. Time Context Post-Test Item

1 Next, we asked students to provide the units of the quantity m Fdt. Responses received a score of “1” if they were technically correct, including meters/second

81 and Newton seconds/kilogram, although the latter does not necessarily demonstrate · an understanding that the quantity is a tiny change in velocity dv. However, stu- dents may have been uncomfortable with the definition of a Newton in MKS units

(kilogram meter/second2), and this was an allowance made for coding, because it is · consistent with the understanding that dt represents an amount of time, rather than mere notation or a rate. All other responses were scored as “0.” Scores of “0.5” were not needed for this item.

Pressure vs. Volume Context Post-Test Item

The fifth post-test item asked students to construct an equation representing the work done by an air pump with pressure P and cross-sectional area A,fora compression distance dx. Scoring methods were similar to that of the dx = v0dt item described above: “0,” “0.5,” and “1” were assigned to each response. Once again, many students did not write the symbol dW , instead choosing to include the symbol

W or the word “work” in their equation. As before, we cannot know if they perceive the work done as tiny or macroscopic, but these students were scored as “1” for this item if they indicated that the work (in some form) was equal to P Adx. Scores of

“0.5” were most often given to the solitary expression P Adx (or “PdV”) that did not include an equals sign or any indication of work. All other responses received a score of “0.”

Mass Density vs. Volume Context Post-Test Item

Following the PdV context, participants received a mass-density item that asked,

“How much mass is contained in an infinitesimally small length” of a cylinder with aradiusR and mass-density ⇢(x) that varied only along its length. Responses of

82 the form ⇢(x)⇡R2dx, including ⇢(x)Adx and ⇢(x)dV , were scored as “1.” Much lati- tude was given for correct expressions in this item, due to the potentially-ambiguous wording of the question, “How much mass is contained...?,” which does not necessar- ily require the need for a mathematical expression. As a result, some students simply wrote that a “small amount of mass” or an “infinitesimally small amount of mass” was present. Although this does not precisely answer the question, it is technically correct, and we scored these responses as “0.5.” All other responses were scored as

“0.”

Linear Charge Density vs. Length Context Post-Test Item

Finally, we asked students to provide an expression for the amount of charge contained in an infinitesimally small length of wire dl. The wire had total length L and a uniformly-distributed total charge Q. A full “1” point was awarded to responses of

Q L dl or dl, though the latter does not directly demonstrate that students understand

Q that = L ; furthermore, the symbol was a student-invention, as we did not use the symbol in the item text, and precisely how those students interpreted the symbol is not known. “0.5” Points were given to responses of dl, because this symbol may arise from conflation with the symbol . Do students truly believe that a surface charge- density is needed? We find it more likely that they failed to remember the standard symbol for linear charge density, but this too remains unknown without interviews.

The same “0.5” was placed on responses that simply stated “infinitesimally small charge” or “dq”withnofurtherelaboration.Otherresponsesweregivenascoreof

Q “0,” such as L , Qdl, QLdl,orIdl.

83 Table 5.2: Mean di↵erentials post-test scores for all items (vertical) across all con- ditions (horizontal). Mean total di↵erentials post-test scores are displayed for each condition in the last row. Errors are one standard error of the mean.

Post-Test VT, FX FX, PV VTE, FXE FXE, PVE Control Item (N =41) (N =37) (N =33) (N =31) (N =43) AdB 71% 7% 63% 8% 97% 3% 84% 7% 40% 8% v dt 78% ± 7% 50% ± 8% 91% ± 5% 75% ± 8% 33% ± 7% 0 ± ± ± ± ± Fkdx 68% 7% 62% 8% 71% 7% 81% 6% 26% 6% 1 ± ± ± ± ± m Fdt 34% 8% 41% 8% 33% 8% 29% 8% 28% 8% P Adx 39% ± 8% 36% ± 8% 36% ± 9% 48% ± 9% 21% ± 6% ⇢(x)⇡R2dx 33% ± 7% 16% ± 6% 32% ± 9% 23% ± 8% 21% ± 6% ± ± ± ± ± Q dl 32% 7% 31% 8% 52% 9% 47% 9% 42% 8% L ± ± ± ± ± Total 50% 4% 42% 4% 58% 4% 55% 4% 29% 4% ± ± ± ± ±

5.3 Experimental Results & Discussion

The results of our experiment are tabulated in Table 5.2 for each condition and each post-test item. A total score is also displayed at the bottom of this table. It is evident from the total post-test scores that most training conditions showed marked improvement overall when compared to the untrained Control condition. Indeed, a one-way ANOVA confirmed that significant di↵erences existed in total post-test scores between groups: F (4, 180) = 8.4,p < 0.001. A Tukey post-hoc analysis illuminated these di↵erences in total post-test score as: VT, FX > Control (p =0.002, d =0.8);

VTE, FXE > Control (p<0.001, d =1.2); FXE, PVE > Control (p<0.001, d =1.1).

The only condition that revealed no significant improvement in mean total post-test score over untrained Control students was the no feedback FX, PV condition. This was our first sign that, all else equal, the contexts in which our participants were trained may have a↵ected their learning.

84 To better understand the e↵ects of physical context and feedback on our training tutorials, we performed a two-way (factorial) ANOVA. A factorial ANOVA extends the analysis of variance of a single categorical variable to multiple, independent cate- gorical variables on one continuous dependent variable. This type of analysis assesses the main e↵ects of each factor and any potential interactions between them. In our study, a two-way ANOVA revealed the following results: an insignificant main e↵ect of physical training context (F (1, 180) = 1.5,p =0.221); a significant main e↵ect of feedback (F (1, 180) = 6.8,p =0.010); and no evidence of interaction between them

(F (1, 180) = 0.3,p =0.567). Note that our Control condition does not feature mul- tiple levels for each factor, unlike our treatment conditions (e.g., students trained in velocity vs. time and force vs. displacement contexts are further separated by no feedback and feedback levels; this type of separation is not possible for Control students who were not trained in any way). Therefore, in the following two subsec- tions, we will discuss these main e↵ects separately as one-way ANOVAs to facilitate comparison to Control performance.

5.3.1 The E↵ect of Physical Context

As alluded to in the previous section, students trained in VT, FX contexts sig- nificantly outperformed Control, even without receiving feedback. Students receiving our (otherwise identical) FX, PV tutorials showed no significant gains on average over

Control in total post-test score. Indeed, when comparing performance on our abstract post-test item, AdB, the same pattern emerges. Only FX, PV failed to significantly out-gain Control at the ↵ =0.05 level for this item, according to a Tukey post-hoc analysis (though this was marginal): AdB: FX, PV Control (p =0.095, d =0.5). ⇠

85 Figure 5.3: Mean total di↵erentials post-test scores separated by training context. Error bars are one standard error of the mean.

However, these indications alone do not provide sucient evidence to claim that students trained in FX, PV and FXE, PVE contexts performed significantly worse than students trained in VT, FX and VTE, FXE contexts. In fact, when grouping participants in this way (FX, PV & FXE, PVE vs. VT, FX & VTE, FXE), no statistical di↵erences in mean score were seen on any single post-test item, except for v0dt, but this is attributable to the fact that VT, FX and VTE, FXE conditions received direct instruction in this context. Furthermore, students trained in FX, PV

& FXE, PVE showed no significant detriment compared to VT, FX & VTE, FXE

86 participants in overall post-test score (48% 3% vs. 53% 3%) according to a one- ± ± way ANOVA: F (1, 140) = 1.6,p=0.210. Cohen’s d shows that the size of this e↵ect on overall post-test score is d =0.2. This is shown visually in Figure 5.3. This is not alargee↵ect, and if a true di↵erence exists, we lack the statistical power to detect it with our sample size. Likewise, no significant di↵erences were observed between FX,

PV & FXE, PVE vs. VT, FX & VTE, FXE in our abstract context post-test item:

AdB: F (1, 140) = 1.6,p =0.202. Therefore, by grouping participants according to the contexts of their tutorial pairs, we cannot claim that students trained in VT, FX contexts showed significant gains over students trained in FX, PV contexts for either our abstract item or total score.

5.3.2 The E↵ect of Feedback

In this section, we will group conditions by Feedback (VTE, FXE & FXE, PVE),

No Feedback (VT, FX & FX, PV), and Control. Our two-way ANOVA previously indicated that this main e↵ect of electronic feedback over paper-based, no feedback training was significant, and we will examine it in detail while also including compar- ison to untrained Control students.

Overall, an ANOVA revealed significant di↵erences between mean total post-test scores of Feedback (57% 3%), No Feedback (46% 3%) , and Control (29% ± ± ± 4%) conditions (F (2, 182) = 15.8,p < 0.001). This is displayed graphically by Fig- ure 5.4, from which the mean total post-test score di↵erences are visibly evident.

In confirmation, a Tukey post-hoc analysis showed that all conditions significantly di↵ered from each other (pairwise). The use of electronic feedback was seen to sig- nificantly improve overall performance on our di↵erentials post-test when compared

87 Figure 5.4: Mean total di↵erentials post-test separated by feedback. Error bars are one standard error of the mean.

with both no training and training without feedback: Feedback > Control (p<0.001, d =1.2); Feedback > No Feedback (p =0.028, d =0.4). These data, as shown in Figure 5.4 support the conclusion that the introduction of immediate electronic feedback in our (otherwise identical) di↵erentials tutorials yielded a statistically sig- nificant gain of four-tenths of a standard deviation over paper-based training without feedback. However, we also observed an increase in mean total post-test score from paper-training without feedback compared to no training: No Feedback > Control

88 (p =0.001,d =0.6). This suggests that our tutorials - even without feedback - may confer some conceptual benefit as measured by our physics di↵erential post-test.

Figure 5.5: Mean di↵erentials post-test scores, separated for v0dt, Fkdx items (scarlet) and all other post-test items (gray). Feedback (VTE, FXE), No Feedback (VT, FX), and Control conditions along horizontal axis. Error bars are one standard error of the mean.

The E↵ect of Feedback within Trained and Transfer Contexts

Our results also allow us to examine the extent to which feedback benefits stu- dents on post-test items in contexts for which they received training (“Trained”), as well as all of the remaining post-test items in contexts for which they received no

89 training (“Transfer”). In Figures 5.5 and 5.6, we display the mean post-test scores for Feedback, No Feedback, and Control conditions trained in VT, FX and FX, PV contexts, respectively. These mean scores are split into categories of post-test items whose contexts pertained to training (in scarlet) and post-test items whose contexts di↵ered from training (in gray). For comparison, Control scores are also displayed for those same post-test questions in each Figure.

As visualized in Figure 5.5, an ANOVA revealed significant di↵erences among

Trained Contexts (in scarlet; F (2, 114) = 31.0,p < 0.001). By inspection of the scarlet bars in Figure 5.5, it is apparent that participants in VTE, FXE and VT, FX conditions posted significantly better scores on v0dt and Fkdx post-test items than their peers in the Control condition. This was armed by a Tukey post-hoc test: v0dt and Fkdx: VTE, FXE > Control (p<0.001,d =1.7); v0dt and Fkdx: VT, FX

> Control (p<0.001,d=1.3).

Although not as visually discernible, the gray bars of Figure 5.5 also indicate that statistical di↵erences exist between the groups. For VT, FX and VTE, FXE conditions, the “Transfer” post-test items were all those for which they received no

1 2 Q training: AdB, m Fdt, P Adx, ⇢(x)⇡R dx, L dl. An ANOVA demonstrated a di↵erence between mean post-test scores for these Transfer items (in gray; F (2, 114)) = 5.0,p=

0.008). However, only the Feedback condition VTE, FXE achieved a significantly higher mean post-test score on its Transfer items compared to Control, according to post-hoc analysis (Transfer: VTE, FXE > Control: p =0.006,d =0.7). The

No Feedback condition VT, FX was not seen to have performed significantly better than Control at the ↵ =0.05 level on Transfer items: (Transfer: VT, FX Control: ⇠ p =0.168,d=0.4).

90 Figure 5.6: Mean di↵erentials post-test scores, separated for Fkdx, P Adx items (scar- let) and all other post-test items (gray). Feedback (FXE, PVE), No Feedback (FX, PV), and Control conditions along horizontal axis. Error bars are one standard error of the mean.

Similarly, from the FX, PV data shown in Figure 5.6, we once again found signifi- cant di↵erences among mean post-test scores on Trained (F (2, 108) = 16.2,p<0.001) and Transfer items (F (2, 108)) = 4.6,p =0.012). For clarity, in Figure 5.6, “Feed- back” and “No Feedback” now refer to FXE, PVE and FX, PV conditions, respec- tively. Likewise, “Trained” and “Transfer” refer to those post-test items for which

1 these students were trained (Fkdx and P Adx)ornottrained(AdB, v0dt, m Fdt,

2 Q ⇢(x)⇡R dx, L dl). For Trained contexts, Feedback showed significant improvement

91 over Control (p<0.001, d =1.5), as did No Feedback (p =0.002, d =0.8). This di↵ers from the mean post-test scores among Transfer contexts, where Feedback per- formed significantly better than Control (p =0.008, d =0.7), but No Feedback did not (p =0.465, d =0.3). These results mimic those seen in Figure 5.5. Apparently, as measured by our di↵erentials post-test, immediate feedback was statistically more beneficial for instruction in physics di↵erentials contexts that directly pertained to training, as well as those that are untrained; however, we do not have sucient evi- dence to make this broad of a claim for our paper-based tutorials that do not feature feedback. The No Feedback conditions performed significantly higher than Control only on those contexts for which they received direct training.

5.4 Conclusions

Based on results from our di↵erentials post-test, we found that we were able to produce significant improvement over untrained Control participants by implementing pairs of training tutorials with varying physical contexts. We also discovered some evidence that the context of training may play a role in the tutorial’s success; however, this remained inconclusive. Furthermore, a significant main e↵ect of Feedback was seen, which boosted performance on our post-test over Control. Feedback was also seen to yield superior results over No Feedback tutorials. This was also evident when separating post-test results by Trained and Transfer contexts, whereupon we observed that Feedback conditions consistently outperformed Control in both sets of post-test items, while No Feedback could not make this claim. However, our

No Feedback tutorials still presented a noticeable overall benefit when compared to untrained Control participants, which implies that students may have gained better

92 conceptual understanding of physics di↵erentials and di↵erential products simply by receiving our training, even when not informed of their performance on the tutorials.

Looking ahead, the experiment presented in Chapter 6 will attempt to incorporate many of the results from this chapter and those prior to help empirically establish a path to better understanding physics integrals and connecting them with their ver- bal meanings. It will also attempt to answer questions about the e↵ect of feedback

(both electronically and from peer-interaction), the e↵ect of training with and with- out dimensional analysis, as well as the advantages or disadvantages to traditional

Riemann-instruction of integration versus the more physics-oriented di↵erentials ap- proach.

93 Chapter 6: Physics Integration Paths to Understanding Instructional Interventions

6.1 Introduction to Integration Paths to Understanding Ex- periment

This experiment represented the culmination of several years of inspiration, discov- ery, revision, and iteration. Although the data occasionally countered our intuitions, and not every result matched our initial optimism, this study remains the high-water mark of our e↵orts to help introductory physics students derive physical meaning from symbolic integrals and their components.

6.1.1 Motivation & Research Questions The Riemann path vs. the Di↵erential path

The physics integration process-object layered framework, developed by Von Kor↵ and Rebello, strongly motivated the design of this experiment (Von Kor↵ and Rebello,

2012) [52]. It provided a network of possible instructional “paths to understanding” physics integrals, but two paths were especially influential. We refer to them as the

“Riemann” and “Di↵erential” paths. For a simplified visualization of their framework illustrating these two paths, see Figure 6.1. Note that our visualization di↵ers from the original in several ways: we have removed all but one of the vertical connection arrows

94 illustrating other possible transitions from macroscopic and microscopic quantities; we removed a fourth set of elements (“Sum function” and “Integral function”); and we refer to the “infinitesimal” quantities as “di↵erentials.” The first and second changes were made to reduce the visual complexity and highlight the most relevant features to our experiment; the third change was made to adhere to our preference for the word

“di↵erential,” which allows that the quantity be interpreted as merely “extremely small,” rather than the more restricted definition of “infinitesimally small.”

Figure 6.1: A simplified version of the Von Kor↵ and Rebello process-object layered framework for physics integration. Scarlet arrows: Riemann path to understanding. Gray arrows: Di↵erential path to understanding.

Mathematics Education Research has emphasized the importance of the Riemann- structure of the integral, and has developed its own process-object layered frame- works for encouraging understanding of the integral, such as that by Sealey (Sealey,

2014) [45]. However, the reliance on the more traditional Riemann path for integral understanding may come at the expense of a more intuitive Di↵erential path, which may be especially valuable and relevant in physics. Indeed, as we alluded in Chap- ter 2.4.1, Von Kor↵ and Rebello claimed that physics students can be taught that “an integral is ‘the sum of many infinitesimally small quantities,’ without harm to their

95 understanding.” This naturally invites the question: for the purposes of introduc- tory physics, does instruction along either the more traditional, mathematics-based

Riemann path, or the more physically-intuitive Di↵erential path present evidence of better conceptual understanding of physics integrals? To address that inquiry, we must establish and clarify several further questions:

1. What constitutes “better conceptual understanding of physics integrals?”

2. “Better” conceptual understanding compared to whom/what?

3. By what metrics shall we measure “conceptual understanding of physics inte-

grals?”

It is possible to provide a variety of answers (with great complexity) to Question 1, but a core tenet of our work has always been that merely the ability to make physical sense of symbolic integrals represents a necessary (though perhaps not sucient) step along the road to expert-like understanding. This view is echoed by Jones,

first mentioned in Chapter 1 (Jones, 2015) [28]. Therefore, we will define “better conceptual understanding of physics integrals” as “the measurement of a stronger ability to derive physical meaning (interpretation and units) from symbolic integrals.”

Furthermore, regarding Question 2, a primary goal of this experiment was to compare students trained along the Riemann path with those trained along the Dif- ferential path. However, we also intended to compare these groups to a baseline - a sample of untrained peers to serve as a Control. More discussion of the design of our experiment follows in Section 6.2.

Finally, we answer Question 3 with the physics di↵erentials, di↵erential products, and integrals assessment specially crafted to help us address Question 1, as introduced

96 in Chapters 3 and 4. This assessment was seen to be highly statistically reliable in

Chapter 4.2.3 and serves as a standard measuring device for the skill in question, including in this experiment.

Units & Dimensional Analysis

In addition to the central question of superior instructional paths to understanding physics integrals, this experiment also explored the role of units and dimensional analysis in integration instruction. We have previously treated the topic of physical units as a self-evident component of conceptual understanding, without justification for its inclusion among our various training tasks and assessments.

This experiment sought to empirically determine the e↵ect of units instruction on our training by directly measuring it. One of our experimental conditions received

Di↵erential path instruction for physics integration with regular prompts to identify and discuss units of di↵erential and integral quantities, whereas a second Di↵erential path condition completely avoided any discussion of units during training, but was otherwise identical. Section 6.2 explores this in greater detail.

Electronic Feedback vs. Interactive Engagement

As a final consideration, this experiment also intended to gauge the ecacy of our integration path-to-understanding tutorials within an “interactive engagement” environment as compared to electronic feedback from a computer terminal. Physics

Education Research has consistently shown that instruction through interactive en- gagement can produce higher scores than traditional instruction alone (even by the best lecturers) on instruments of conceptual understanding (Hake, 1998) [16]. Hake defines interactive engagement (IE) as methods “designed at least in part to promote

97 conceptual understanding through [the] interactive engagement of students in heads- on (always) and hands-on (usually) activities which yield immediate feedback through discussion with peers and/or instructors” (Hake, 1998). In his landmark survey study encompassing the pre-/post-test results of more than 6000 students across 62 physics courses, he concluded, “the classroom use of [interactive engagement] methods can increase mechanics-course e↵ectiveness well beyond that obtained in traditional prac- tice,” as measured by the Halloun-Hestenes Mechanics Diagnostic test (Halloun and

Hestenes, 1985) [18] and the Force Concept Inventory (Hestenes et al.,1992)[24].

As in Chapter 5, we have already seen the benefits of electronic feedback when compared to no feedback: in our di↵erentials training experiment, students either received immediate, authoritative responses from a computer during training, or al- ternatively, they received no criticism or armation of any kind, and worked in complete isolation to outside information, apart from the tutorial. The fact that electronic feedback during training produced overall better results than no feedback at all was not especially surprising. However, given PER’s history of consistently- demonstrated instructional success from interactive engagement with other students, we elected to compare the e↵ect of electronic feedback during our integration path-to- understanding tutorials with identical training in three-person groups without elec- tronic feedback. For practical purposes, if our training hypothetically yielded equal

(or more) benefit from paper-based tutorials in three-person groups as from a com- puter terminal with electronic feedback, it may be more desirable for use in an actual physics classroom. More students could receive simultaneous training with fewer re- sources required. Section 6.2 will provide information about how these experimental conditions were structured, trained, and assessed.

98 Summary of Experimental Questions

Our integration paths-to-understanding experiment will attempt to answer several questions by taking advantage of experimental design and our reliable assessment:

1. Does tutorial instruction along either the Riemann path or the Di↵erential path

provide evidence of better conceptual understanding compared to each other

and an untrained Control group?

2. What benefit (if any) does the inclusion of units instruction confer on students’

abilities to connect symbolic integrals to their verbal, physical meanings?

3. Does tutorial instruction using either electronic feedback or interactive engage-

ment provide evidence of better conceptual understanding compared to each

other and an untrained Control group?

6.2 Experimental Methods

6.2.1 Study Participants Participant Demographics

Participants in this experiment were sampled from second semester, calculus- based introductory physics (“Physics 2”) students at The Ohio State University in the

Autumn 2016 semester. More than 90% of all enrolled Physics 2 students participated in at least some PER investigations during the semester. For our experiment, N =318 were randomly selected after obtaining consent to participate in research.

As in previous experiments with samples from similar populations, more than 80% of our 318 participants self-reported an engineering or physics major discipline. These

99 participants posted a mean final course grade of 80.8% 0.6%. For reference, the pop- ± ulation mean final course grade was 79.0% for all 620 Physics 2 students; our sample of participants exceeded the population in mean course grade by only 0.15 standard deviations. Furthermore, as part of our experimental design, our study participants were randomly sorted into five conditions, detailed in Section 6.2.3. However, an

ANOVA revealed no significant di↵erences in mean final course grade between these

five conditions: F (4, 312) = 0.8; p =0.544.

Estimating Necessary Sample Size

In Chapter 5.2.2, we discussed the methods by which the sample size and number of participants per condition n were selected. By Cohen’s prescription at an ↵-level of 0.05, n =39perconditionwouldprovideapowerlevelof1 =0.8 for discovery of “medium”-sized e↵ects with five experimental conditions. However, due to the simplified, non-factorial structure of our design (see Section 6.2.2) and the possibility of small e↵ects (e.g., the e↵ect of integration instructional path may not be large), we instead chose to use all available N =318studentsdividedroughlyevenly(n 60) ⇡ per condition. While we utilized a total of five conditions, many of our experimental questions can be answered by comparisons of two or three conditions, testing the e↵ect of a single factor; Cohen’s prescription for two- and three-condition ANOVA studies

(with the aforementioned circumstances) detecting “medium”-sized e↵ects with 80% power is n2 =64andn3 =52,respectively(Cohen,1992)[9].Moreover,largersample sizes provide finer granularity of salient features within subpopulations. With these considerations in mind, and given the structure of our study, we were comfortable with our available n 60 participants per condition. ⇡

100 Table 6.1: Integration paths-to-understanding experimental design.

Experimental Condition Short Type of Units Participants Name Name Feedback Instruction N

Di↵erentialPath “DP” IE No 62

Riemann Path Units “RPU” IE Yes 62

Di↵erential Path Units “DPU” IE Yes 65

Di↵erential Path Units “DPUE” Electronic Yes 60 Electronic (Feedback)

Control “Control” N/A N/A 65

6.2.2 Experimental Design

As we alluded above, this experiment was not a true “factorial” design, in which the number of conditions is equal to the product of the number of levels of all fac- tors. In our case, with three factors, such a design might (more rigorously) involve

Units Instruction (Yes, No) Feedback (Electronic, Interactive Engagement, None) ⇥ Instructional Path (Riemann, Di↵erential, None), which equates to 2 3 3=18 ⇥ ⇥ ⇥ separate experimental conditions. This is wholly untenable for a variety of reasons.

In fact, it may not even be necessary when design shortcuts are available that still enable us to make direct comparisons within the factors of interest, but do not require unreasonable numbers of participants and data-acquisition logistics.

In light of this, we opted for a reduced-complexity design that could still answer our central questions from Section 6.1.1. We chose five experimental conditions, shown

101 in Table 6.1. A minimum number of conditions was chosen, where each condition was selected to enable a specific comparison for relevant factors.

To determine the e↵ect of training along our two instructional paths, we used the Di↵erential Path Units (“DPU”) and Riemann Path Units (“RPU”) conditions.

Both used interactive engagement environments, in which three students collabora- tively work through a tutorial, and both conditions’ tutorials employed frequent units questions. In addition, both conditions’ tutorials also used the same physical contexts during instruction and were structured with similar numbers and types of subtasks; to the extent that it was possible, these tutorials matched. The most notable design di↵erence between their tutorials was that the DPU tutorial followed the gray Dif- ferential path of instruction seen in Figure 6.1, whereas the RPU tutorial followed the scarlet Riemann path. Note that the Riemann path features one extra step (a downward arrow) that connects the sum to the integral, which we call the “limiting procedure.” Because the Di↵erential path represents a literal “shortcut” (in terms of the number of layers necessary to reach the symbolic integral from the beginning of the framework), there is an unavoidable, structural di↵erence in the DPU and RPU tutorials, where several extra subtasks appear in the RPU condition to establish this limiting procedure; these subtasks could not have appeared in the DPU condition.

Next, we observed the e↵ect of units instruction by comparing the Di↵erential

Path (“DP”) and Di↵erential Path Units conditions. These tutorials were intention- ally identical, except that the DPU condition received regular prompts as part of their training to identify and discuss units of di↵erential, di↵erential product, and in- tegral quantities. The DP condition saw no units (nor even the word “units”) during their tutorial, though units were still tested as part of our assessment after training.

102 Occasionally, students in the DP condition made original and unprompted mention of units to their group members, but this was neither common nor required.

Finally, we compared the assessment results of the Di↵erential Path Units and

Di↵erential Path Units Electronic (Feedback) (“DPUE”) conditions. This comparison enabled us to examine the benefits of electronic feedback from a computer with those of feedback received from interactive engagement (IE) through peer-interaction. As described in Section 6.1.1, this analysis has practical implications for implementation of our training tutorials within real physics curricula. Students in both conditions worked on identical tutorials, DPUE completing training alone on a computer with an indication of the correct answer and a brief explanation, and DPU collaborating in a group of three students on paper in a separate testing room.

For every factor and experimental comparison addressed above, our Control condi- tion served as a baseline performance estimate for the Physics 2 population. Control students received no training of any kind; their only participation in this study was the solitary completion of our assessment in a quiet testing room. Similarly, all par- ticipants in training conditions also completed our paper-based assessment, without assistance from a computer or other students, following the completion of their re- spective tutorials.

The following section will explore in greater depth the structure and content of the tutorials. First, we will introduce the only Riemann path tutorial, assigned to the RPU condition. Afterwards, we will dissect the several Di↵erential path tutorials, all of which were identically structured to one another with minor adjustments per condition.

103 6.2.3 Instructional Intervention Tutorials Riemann Path Tutorial (RPU)

The RPU condition received a paper-based tutorial in an interactive engagement environment. This tutorial (the only Riemann instructional path tutorial in this experiment) sequentially followed the layers of our simplified version of the Von Kor↵ and Rebello framework, shown in scarlet in Figure 6.1.

To begin the tutorial, every group of three participants read a short paragraph labeled “KEY INFO,” explaining the meaning of the symbol ;thisdescriptionwas chosen by us, intended for an audience of introductory physics students. The “KEY

INFO” feature was a recurring theme during the tutorials, designed to attract the reader’s attention to important instructional information which might otherwise be overlooked in haste. To see the first KEY INFO encountered by RPU students, see

Figure 6.2.

Figure 6.2: First KEY INFO feature of Riemann path tutorial.

After familiarizing participants with our description of the symbol ,thetutorial proceeded to the first layer of the framework: the macroscopic quantity layer. In general, the subtasks in this layer were conceived to challenge students’ thinking regarding the subtle meaning of the symbol ,andtoshowthatquantitiessuchas

104 t can be conceptualized as amounts of a physical variable, rather than simply the

“di↵erence operator” (t = t t ). As in previous experiments, a primary tool of all f i of our tutorials was the use of “Student A vs. Student B” arguments, influenced by student responses from our exploratory study in Chapter 3.2 and other resources. In the macroscopic quantity layer, RPU students debated the merits of two arguments regarding the symbols t and x. As in Chapter 5, the first such argument was broad and fundamental, whereas the second was more nuanced and focused. After discussing the first debate, but before proceeding to the second, RPU participants were asked to discuss the units of the quantity t. Units were a regular component o training throughout this tutorial. See Figure 6.3 for a sample of the subtasks featured in this layer.

Figure 6.3: Macroscopic quantity layer sample subtasks from Riemann path tutorial.

The next set of subtasks in the Riemann path tutorial focused on the macroscopic product layer. The context featured during this layer was vt, superficially referring

105 to the constant speed v of a charged particle (although this detail was only included to increase the apparent relevance to a Physics 2 audience, and likely had little e↵ect on the nature of the discussion). A “Student A vs. Student B” argument debated the physical meaning of the quantity vt. However, this time, we allowed partic- ipants to label each hypothetical argument as “Correct,” “Incorrect,” or “Partially

Correct.” In this debate, imaginary students argued that vt was either the “average velocity” during t, the product of v and t (the distance traveled by the particle, in meters), or the “average position” of the particle (but still with units of meters).

Ideally, participants would recognize that the third interpretation was approximately correct due to the ascribed units, but incorrect in verbal meaning, thus earning the distinction “Partially Correct.” Surprisingly, although we considered the interpreta- tion of vt fairly elementary to a second-semester introductory physics population, vigorous debate occasionally surfaced among RPU participants due to the subtlety of the hypothetical disagreements in the subtask.

The third stage of the framework along our Riemann path was the summation layer. Our Riemann path tutorial contextualized this layer as an accumulation of charge in a capacitor with time-varying current, Q = i Iit. Participants were pro- P vided with a “data-table,” however only symbolic values were provided. For reference, see Figure 6.4. This subtask asked students to describe in words the Multiplicatively

Based Summation (MBS) conception (i.e., a sum of products: I1t1 + I2t2 + ...; see

Chapter 2.3) to estimate the accumulated charge on the capacitor. Following their own attempts to estimate Q, participants were provided a Student A vs. Student

B argument about this precise calculation. Once again, our tutorials made frequent use of these types of questions to redirect the aim of the conversation towards our

106 preferred interpretation of physical quantities after allowing students the opportunity to consider it independently (or in this condition, collaboratively) first.

Figure 6.4: Summation layer subtask from Riemann path tutorial.

At this stage in the tutorial, participants had progressed through several sub- tasks encouraging them to discuss and interpret the meaning and units of t and x

(macroscopic quantities), vt (macroscopic product), and i Iit (sum of macro- P scopic products). To establish the distinguishing characteristics of an integral, the summation layer had to undergo the “limiting procedure.” Within the same con- text of a charging capacitor, our RPU participants received a second KEY INFO to introduce the sigma notation of a sum. This was immediately followed by a dis- P cussion/writing prompt to improve the estimate for Q without changing the overall duration of data acquisition. A hint was given to focus on n, the “number of rows” in the table shown in Figure 6.4. Ideally, students would recognize that increasing the number of rows in the data table (i.e., taking more measurements per second)

107 produces a better estimate. Students were prohibited from using the words “integral” or any of its forms as a shortcut to avoid explanation and elaboration of the concept.

This is seen in Figure 6.5.

Figure 6.5: Limiting procedure discussion question from Riemann path tutorial.

After this discussion, RPU participants were finally introduced to a KEY INFO feature that associated the formal limit of the summation with the integral (the final layer of the framework). This KEY INFO took extra precaution to note that the limit was taken such that the number of products in the summation approached infinity

(n ). See Figure 6.6. !1

Figure 6.6: Integral definition KEY INFO from Riemann path tutorial.

108 There is an alternative and physically equivalent formulation to this limit, in which the size of each interval approaches zero (t 0). However, this is virtually indistin- ! guishable from the concept of an infinitesimal, where some physicists may informally write t dt. We sought to entirely avoid prompting this conceptualization, lest ! we “cross-contaminate” the two instructional paths that principally motivated this experiment. The purity of instruction along the Riemann path was preserved, in our opinion, by restricting the definition of the integral to that of a summation with an arbitrarily large number of product terms.

To be mathematically precise, this may not be an optimal or formal definition for an integral from a summation, because the limit is perhaps more appropriately taken on the closed form expression for the sum as a function of the number of terms. In- deed, according to Jones, “the [true] Riemann integral constructs a sequence of finite summations and then considers the limit of this sequence” (Jones, 2015) [28]. How- ever, for the physical intuition that our conceptualization provides, we assert that it was a sucient interpretation for distinguishing an integral from an ordinary sum of products, while not veering too closely to the concept of di↵erentials. Mathematicians may bristle at this allowance, but we viewed it as more physically realizable. Fur- thermore, our overall goal was to remain faithful to the Riemann instructional path as a whole (as may be traditionally taught in a physics or combined physics/calculus course), not necessarily a strict adherence to the formal, mathematical subtleties of its transitions between layers, to which our engineering and physics students may never attend anyway.

Following the KEY INFO shown in Figure 6.6, our RPU tutorial featured a fill- in-the-blank table for students to practice similar types of symbols, interpretations,

109 and units to those encountered throughout the tutorial. Multiple contexts were pre- sented. Students were given the first completed row as an example. See Figure 6.7 for reference. The table attempted to highlight the MBS conception of integrals and clarify their units.

Figure 6.7: Practice conceptual table, connecting symbols, interpretations, and units from Riemann path tutorial.

Having traversed the layers along the length of the Riemann instructional path,

RPU participants were then given a final subtask intended to help assimilate and consolidate their conceptual understanding of the Riemann-structure of an integral thus far. The subtask featured a hypothetical student who required assistance mak- ing sense of the necessary components of a physical integral. The context was the volume mass density of a solid cylinder that varied only along the cylinder’s length.

Participants were asked to perform the following: conceptualize a representative slice

110 of the cylinder Vn; construct the mass of that specific slice Mn = ⇢nVn; choose

2 the correct expression and units of Vn = ⇡R xn from a set of possible choices; discuss the meaning of the symbol xn; and, fill in the complete sum and integral using the symbols provided and the expressions discovered, and prove its units. The last step is shown in Figure 6.8. This concluded training for participants in the RPU condition.

Figure 6.8: Final subtask from Riemann path tutorial.

Di↵erential Path Tutorials (DP, DPU, DPUE)

Above, we introduced the tutorial for the RPU condition, which instructs par- ticipants about the meaning and units of physics integrals along the Riemann path from Figure 6.1. In this section, we will examine the tutorials provided to the three

Di↵erential path conditions. As much as possible, the tutorials were kept identical across conditions to facilitate comparison of assessment results. We will therefore restrict this discussion to highlighting the di↵erences posed by the Di↵erential path conditions. Di↵erences between the DP and DPU tutorials were limited to items and

111 subtasks that explicitly reference units; all other aspects of the DPU task remain in tact in the DP tutorial. There were e↵ectively no di↵erences between the tuto- rials given to DPU and DPUE participants, aside from the fact that the latter was completed on a computer with feedback rather than on paper collaboratively.

All Di↵erential path tutorials began with a KEY INFO feature discussing our preferred interpretation of the physics di↵erential. We were careful to not explic- itly interpret any specific physics di↵erential in this passage. When assessed after training, our participants’ interpretations of specific di↵erentials would thereby be authentically obtained from either their interaction with peers, electronic feedback, or their own reasoning about the tutorial. The first Di↵erential path KEY INFO is shown in Figure 6.9.

Figure 6.9: First KEY INFO feature of Di↵erential path tutorial.

Matching the structure of the Riemann path tutorial above, students proceeded to the first layer of the Di↵erential path (shown in gray in Figure 6.1): the di↵erential quantity layer. Student A vs. Student B subtasks were again posed to our partici- pants to examine the fundamental nature of the di↵erentials dt and dx,particularly against the “rate” interpretation and the “di↵erentials are just labels” conception from Chapter 3. Wedged between these two debates were requests for the units of dt for DPU and DPUE participants; these students also received a question asking how

112 di↵erentials could even have units if they were merely “labels,” and what those units should actually be. The DP condition saw no units questions.

In the second layer of the framework, our participants were faced with the di↵eren- tial product vdt in a Student A vs. Student B debate (just as RPU students discussed vt at this layer). This hypothetical argument pitted a correct interpretation for vdt

(i.e., “a tiny amount of distance traveled during dt”) against several known alterna- tive conceptualizations, such as the “rate” interpretation and two di↵erent versions of the “instantaneous value” interpretation, where each incorrect interpretation was ac- companied by a declaration of its corresponding incorrect units (for DPU and DPUE participants).

To approach the third layer of the framework (the physics integral), students were required to conceptualize accumulating many di↵erential products of charge on a capacitor, Idt = dq. Students were asked to explain this process in words;

DPU/DPUE groups were also asked to discuss the units of I, dt,andIdt. They then faced a Student A vs. Student B item that attempted to highlight the need for this method of accumulation, instead of simply computing the product Q = I (t t ), · f i which does not return a meaningful expression due to the variable nature of the symbol I. At this stage, Di↵erential path participants were given the final KEY

INFO to introduce the formal notation of an integral and to associate it with the accumulation process just discussed. See Figure 6.10 for reference.

In the same fashion as the Riemann path tutorial, the Di↵erential path tutorials proceeded to a fill-in-the-blank table to immediately practice the information from the KEY INFO above. This is demonstrated in Figure 6.11. The first row was again completed by default to serve as an example. For students in the DP condition,

113 Figure 6.10: Integral definition KEY INFO from Di↵erential path tutorial.

the column labeled “Has units of:” and its adjacent column requesting units were removed. Both the Riemann path and Di↵erential path tutorials intentionally left one cell blank in the column that reads, “And is an infinite sum of products of:,” which was occasionally overlooked by participants as seemingly insignificant. This cell was presumed vital to their understanding of the MBS conception of integration, though many students disregarded its importance due to the formulaic nature of the column.

Figure 6.11: Practice conceptual table, connecting symbols, interpretations, and units from Di↵erential path tutorial.

114 To close the tutorial, DP/DPU/DPUE conditions completed the same mass- density subtask as featured on the Riemann path tutorial. However, every instance of macroscopic quantities was replaced with di↵erential quantities (e.g., V dV ). n ! The remainder of the subtask was isomorphic or identical to that of RPU students.

The final item requested was the complete mass integral, M = dM = ⇢(x)⇡R2dx; R R however, unlike in the Riemann path tutorial, the limiting summation notation was intentionally avoided. We note that some students voluntarily wrote M = dM =

L 2 R 0 ⇢(x)⇡R dx, with correct integral limits as dictated by the subtask, though this was R not requested. This activity concluded the Di↵erential Path tutorial, although DPUE students were shown the correct integral in its entirety on their computer screens at this stage.

6.2.4 Physics Di↵erentials, Di↵erential Products, Integrals Assessment

As discussed above, to measure the e↵ects of each factor of interest, we employed our specially designed physics di↵erentials, di↵erential products, and integrals as- sessment. Of principal importance to our experiment was the mean performance of each condition on integral items, especially those pertaining to our abstract context,

AdB, which represents a clean signal of transfer, as it was never a component of R training. As a reminder, we display the categories and contexts of the Physics 2 assessment in Table 6.2. For sample questions, refer to Appendix A.2. Reliability statistics can be found in Table 4.2 from Chapter 4.2.3.

In the following section, we will inspect the results from this experiment for each of the three central questions that motivated its execution.

115 Table 6.2: Categories and physical contexts of assessment items for Physics 2 as- sessment. Each symbolic quantity corresponds to two assessment items: a physical interpretation question and an explicit units question, both employing identical phys- ical scenarios.

Physical Quantity Contexts

Di↵erentials dt, dx, dq

Di↵erential Products vdt, adt, qEdx, dx, ⇢⇡R2dx, AdB

Integrals vdt, adt, qEdx, dx, ⇢⇡R2dx, AdB R R R R R R

6.3 Results & Discussion

6.3.1 Descriptive Statistics

Descriptive statistics for each of the three scores (di↵erentials, di↵erential prod- ucts, and integrals) for each condition are displayed in Table 6.3. Immediately visible from these data is the superiority of the DPUE feedback condition in every category, including the total score. It is also clear from examining the “Integrals” row, in particular, that e↵ects due to units instruction or instructional path were virtually non-existent.

Although not surprising given the instructional content, we also observed evidence that performance on di↵erentials (though not di↵erential products) items was likely a↵ected by instructional path. Specifically, an ANOVA revealed that significance di↵erences existed between mean di↵erentials scores of RPU, DPU, and Control stu- dents: F (2, 190) = 5.0; p =0.007. Post-hoc analysis showed the DPU performance on di↵erentials items significantly surpassed Control students (p =0.008,d =0.5), and

116 Table 6.3: Assessment descriptive statistics. Mean scores with one standard error are shown for all experimental conditions for each set of assessment items.

Physical DP RPU DPU DPUE Control Quantity (N =62) (N =62) (N =65) (N =60) (N =65)

Di↵erentials 88% 3% 76% 3% 87% 3% 96% 1% 73% 4% ± ± ± ± ± Di↵erential 71% 4% 67% 3% 72% 3% 82% 3% 59% 4% Products ± ± ± ± ±

Integrals 75% 3% 73% 3% 73% 3% 84% 2% 71% 3% ± ± ± ± ±

Total 76% 3% 72% 3% 75% 3% 85% 2% 67% 3% ± ± ± ± ±

was on the statistical cusp of exceeding that of RPU students (p =0.052,d =0.5); while not technically significant at the ↵ =0.05 level (which is arbitrarily agreed upon by most social scientists), this result is a strong indicator that the Di↵erential path tutorial may be better preparing students for making sense of di↵erential quan- tities than its Riemann analogue. This is expected, because Riemann path students received no di↵erentials instruction, whereas the Di↵erential path was founded on it. It is especially evident in students’ ability to assign verbal meanings to symbolic di↵erential quantities. An ANOVA showed significant di↵erences in mean score for di↵erentials interpretations items between DPU, RPU, and Control: Di↵erentials In- terpretation: F (2, 190) = 6.7,p =0.002. Post-hoc analysis confirmed that RPU and

Control scores were statistically inferior to those of DPU: Di↵erentials Interpretation:

DPU > RPU (p =0.001,d=0.7); DPU > Control (p =0.041,d=0.5).

117 6.3.2 The E↵ect of Instructional Path

Comparisons between DPU, RPU, and Control conditions showed no significant di↵erences for integral scores based on a one-way ANOVA: F (2, 184) = 0.2,p=0.833.

This was also observed when splitting mean integral scores of these three condi- tions into interpretations and units subscores: Integrals Interpretation: F (2, 184) =

0.6,p =0.527; Integrals Units: F (2, 184) = 0.4,p =0.674. As discussed above in

Section 6.3.1, statistical gains from the Di↵erential path tutorials over the Riemann path tutorial were largely attributable to a demonstration of improved understanding of di↵erential quantities. This e↵ect was not seen to translate to improved di↵erential products or integrals scores.

On only one integral assessment item (the interpretation of the quantity vdt)did R our results indicate the existence of significant di↵erences between the mean scores of

DPU, RPU, and Control conditions: vdt Interpretation: F (2, 184) = 4.4,p=0.014. R Post-hoc analysis revealed that this was due to better performance from the DPU condition over the others: DPU > RPU (p =0.050,d =0.5); DPU > Control (p =

0.019,d =0.5). We attempted to discover the source of this di↵erence and found that it may lie in the pattern of common answers provided by these three conditions.

We observed that DPU students incorrectly interpreted vdt as “the total change in R velocity” with a frequency of 2% 1%, where we have used one standard binomial ± p (1 p) · error for proportions, p = N . By contrast, RPU and Control students chose q this interpretation with frequencies 8% 3% and 14% 4%, respectively. Mean ± ± scores on this item were comparable to that of DPU for the other Di↵erential path tutorial conditions, DP and DPUE. It is unclear why a larger proportion of RPU and

Control participants interpreted vdt as a change in velocity; none of the paper-based R 118 interactive engagement conditions (DP, DPU, RPU) was significantly more likely to ascribe to the quantity vdt units of meters/second. R Still, this represents the lone instance in which Di↵erential path tutorials demon- strated statistically higher mean scores on any single integral assessment item or subscore compared to the RPU condition, and we thereby retain the null hypothe- sis that there are no significant di↵erences in overall mean integral scores between instructional paths given our methods of training. Note: the fact that we found in- sucient evidence to reject this hypothesis could be due to ceiling e↵ects, which will be discussed in greater detail in the following section.

6.3.3 The E↵ect of Units

Our secondary question from Section 6.1.1 also produced mostly null results. Anal- ysis of variance results comparing the scores of DP, DPU, and Control conditions yielded insucient evidence to claim an overall e↵ect due to units instruction. We found no significant di↵erences between these three conditions when comparing their mean integral scores: F (2, 184) = 0.3,p =0.708. Significant di↵erences were seen when examining di↵erentials (F (2, 184) = 7.1,p =0.001) and di↵erential products

(F (2, 184) = 3.6,p=0.030) scores; however, this signal was strongly driven by statis- tically weaker performance by Control compared to DP and DPU students; DP/DPU scores were virtually identical to one another in these categories.

An analysis of every assessment item (followed by a Tukey post-hoc analysis for direct comparison between conditions) demonstrated no single item or subscore in which DPU students posted a statistically advantageous mean score compared to DP students. Even on the half of the assessment that directly tests understanding of

119 Table 6.4: Assessment descriptive statistics for units items only. Mean units scores with one standard error are shown for all experimental conditions for each set of assessment items.

Physical DP RPU DPU DPUE Control Quantity (N =62) (N =62) (N =65) (N =60) (N =65)

Di↵erentials 89% 4% 92% 3% 90% 3% 95% 2% 78% 5% ± ± ± ± ± Di↵erential 76% 4% 84% 3% 83% 3% 89% 3% 72% 5% Products ± ± ± ± ±

Integrals 80% 4% 83% 3% 78% 4% 86% 3% 80% 3% ± ± ± ± ±

Total 80% 3% 85% 3% 82% 3% 89% 2% 76% 3% ± ± ± ± ±

units for di↵erentials, di↵erential products, and integrals, we found no evidence that either DP, DPU, or Control students di↵ered significantly in score: Units Total Score

(all items combined): F (2, 184) = 1.1,p=0.339.

Several explanations exist for this unexpected null result, including insucient

“dosage” of units training and baseline ceiling e↵ects. The former refers to the quan- tity and quality of units instruction received by our DPU students. It is possible that the units questions regularly interspersed during the tutorial were simply not frequent, challenging, and/or engaging enough to a↵ect change in their assessment performance.

Regarding the possibility of baseline ceiling e↵ects, we display Table 6.4, which illustrates plainly why uncovering potential e↵ects due to units instruction may be impossible in our experiment (as we conducted it). This table displays mean units

120 scores for each assessment category and each condition; no single mean score is less than 72%. These results are characteristic of “ceiling,” where our estimates of (un- trained) baseline mean scores may already be too high to provide “room” to detect di↵erences between groups, and/or too close to students’ maximum achievement (on average) to push further performance via training.

We observed a hint of some benefit of units instruction in the di↵erential products units score when comparing DPU and DP conditions; however, with standard errors of the mean between 3%-4%, this experiment is severely underpowered to detect what might be a real e↵ect. These mean scores, even among untrained Control students, are pressed firmly against the statistical ceiling for improvement, and with so little discerning room between them already, it is unlikely that an experiment with n 60 ⇡ participants per condition would be capable of finding e↵ects of the size that this small amount of room permits. To observe a real, “small”-sized e↵ect under otherwise unchanged circumstances with an 80% probability of detection, Cohen recommends roughly three to four times more participants per condition (Cohen, 1992) [9], which was not possible for this experiment.

Furthermore, it is generally dicult to find any e↵ect of units instruction (even nominally), as measured by comparisons of mean scores of DP and DPU conditions, when observing Tables 6.3 and 6.4. For example, the “Integrals” row in each table was of primary interest, and nominal indications are that the advantage in score lies with the DP condition, not DPU. It is doubtful that this represents any real e↵ect; however, it underscores the need to preserve the null hypothesis, namely that there is no apparent overall di↵erence in mean scores between DP and DPU conditions.

121 6.3.4 Electronic Feedback vs. Interactive Engagement

Several noteworthy null outcomes resulted from the previous two sections; how- ever, when assessing the e↵ect of electronic feedback versus interactive engagement, we have strong evidence that the computer-trained students of the DPUE condition were much more able to make sense of physics integrals than their peers in the paper- based DPU condition. Indeed, a one-way ANOVA comparing DPUE, DPU, and

Control groups determined significant di↵erences exist between their mean integral scores: F (2, 185) = 5.1,p=0.007. Post-hoc analysis showed that the electronic feed- back condition DPUE obtained higher scores than both DPU and Control: Integrals:

DPUE > DPU (p =0.040,d =0.4); DPUE > Control (p =0.009,d =0.6). The mean integral scores in reference are shown in Table 6.3.

As seen in Table 6.4, mean units scores for each category were consistently, natu- rally high, even among Control students, who did not receive instructional training.

This ceiling e↵ect was less troublesome among mean interpretation scores, where stu- dents typically performed worse than on units items. Consequently, this provided more opportunity to discover experimental e↵ects. In fact, we observed that elec- tronic feedback significantly boosted the mean integral interpretation score for the

DPUE condition over those of the DPU and Control conditions: Integral Interpreta- tions: DPUE > DPU (p =0.040,d =0.4); DPUE > Control (p =0.009,d =0.6).

The mean integral interpretation scores for all five conditions are shown graphically in Figure 6.12.

In Figure 6.12, Di↵erential path conditions are shown in gray, while the Riemann path condition is shown in scarlet; the Control condition is shown in white. From

122 Figure 6.12: Mean integral interpretations scores for all conditions. Di↵erential path conditions shown in gray. Riemann path condition shown in scarlet. Control condition shown in white. Error bars represent one standard error of the mean.

this figure, it is visually apparent that the DPUE condition was a statistical outlier in positive performance, whereas all other conditions showed comparable scores.

Further, we also show the ANOVA results from the specific comparison of mean abstract integral ( AdB) interpretation scores between DPUE, DPU, and Control R conditions: F (1, 185) = 4.5,p=0.013. Again, a post-hoc analysis reveals that DPUE produced significantly higher scores than Control, although only nominally higher than the DPU condition: AdB Interpretation: DPUE > Control (p =0.009,d = R 0.5). The DPUE condition was the only training condition to significantly exceed the mean Control score for this assessment item. This does not imply that mean abstract integral interpretation scores are significantly di↵erent between DPUE and

123 DPU conditions at the ↵ =0.05 level, though it does draw further attention to the exceptionally strong performance of the electronic feedback condition compared to all others. The mean abstract integral interpretation scores are displayed in Figure 6.13.

Figure 6.13: Mean abstract integral ( AdB)interpretationsscoresforallconditions. Di↵erential path conditions shown inR gray. Riemann path condition shown in scarlet. Control condition shown in white. Error bars represent one standard error of the mean.

Due to the evidence seen in this section, we can conclude that the use of electronic feedback yielded significantly higher integral scores (and several higher subscores, in- cluding integral interpretations) than interactive engagement. Specifically, electronic feedback produced an e↵ect size of approximately 0.4 standard deviations of improve- ment over interactive engagement for integral scores; it also resulted in 0.6 standard deviations of improvement over untrained Control.

124 6.4 Possible Limitations

In the previous section, we displayed and analyzed the results of the Integration

Paths to Understanding Experiment. This section will attempt to briefly critically examine some possible limitations and drawbacks to the study. These considerations will be discussed to provide qualifying context for the conclusions drawn at the end of the chapter.

6.4.1 Baseline Ceiling E↵ects

As noted in Sections 6.3.2 and 6.3.3, this experiment encountered diculty from ceiling e↵ects on the scores of most assessment categories. This e↵ect was less con- sequential when assessing students sampled from a Physics 1 population, as seen in

Chapter 4; however, using the Physics 2 sample in this experiment, untrained Control scores were potentially too high to induce statistical separation through our training.

The fact that we observed results consistent with ceiling e↵ects among our Physics 2 sample and not our Physics 1 sample from Chapter 4 could be caused by a selection e↵ect, whereby those students who enroll in Physics 2 are necessarily more knowl- edgeable than many Physics 1 students, both due to more experience and the filtering e↵ect of the first semester course. Without lower baseline scores, it would be much more dicult to ascertain the true e↵ects intended to be studied in this experiment.

If they exist, it cannot be presently known without a much larger sample size (or more e↵ective treatment) to increase resolution into the small window a↵orded by our high Control scores.

125 6.4.2 Possible Performance from Working Memory

Due to time constraints, this experiment did not feature an interim task designed to cleanse the working memory of our participants. This was a component of our

Di↵erentials training experiment seen in Chapter 5. Instead, students completed the tutorials immediately before proceeding to the assessment. Due to the length of the training phase, insucient time was available to administer a short, unrelated task.

It could be argued that students may not have “learned” the concepts encountered in training and could have been completing the assessment by using only their working memory. Still, some observed results are not likely attributable to this e↵ect, such as the statistical gain observed by the DPUE condition over Control in a completely unfamiliar, abstract, and untrained context (seen in Figure 6.13).

6.4.3 Missing Interaction E↵ects

Finally, we recall that the structure of our experiment was not a full “factorial” design, as noted in Section 6.2.2. We stated that this design was beneficial for practical purposes and that it had the potential to answer our central questions of this chapter.

However, the advantage of a full factorial design lies in its ability to inform us about possible interaction e↵ects. For example, it may be that the use of electronic feedback is greatly diminished when applied to the Riemann path tutorial; alternatively, it is also possible that the e↵ect of units instruction may have been visible when comparing the RPU condition with a hypothetical RP condition that lacked units. These e↵ects cannot be observed due to the design of our experiment, and if they exist, more research would be required to discover them.

126 6.5 Conclusions

The Integration Paths to Understanding Experiment was designed to answer sev- eral important questions, seen in Section 6.1.1. Based on our results discussed in

Section 6.3, we now answer those questions here:

1. Does tutorial instruction along either the Riemann path or the Di↵erential path

provide evidence of better conceptual understanding compared to each other

and an untrained Control group?

Evidence suggests that students trained along the Di↵erential path were better

equipped to extract physical meaning from di↵erentials than those who were

trained along the Riemann path. Because RPU students were not exposed to

the concept of di↵erentials during training, this is not unexpected. More im-

portantly to our goals, there is insucient evidence to claim that either of these

instructional paths resulted in better overall assessment scores, or in particular,

astrongerabilitytoobtainphysicalmeaningfromsymbolicintegrals,evenwhen

compared to untrained Control students. Mean integral scores were statistically

indistinguishable across the RPU, DPU, and Control conditions.

2. What benefit (if any) does the inclusion of units instruction confer on students’

abilities to derive physical meaning from symbolic integrals?

To our surprise, we found that the inclusion of units instruction among the

tutorials produced no observable e↵ect on mean assessment scores. The DPU

condition exceeded neither DP nor even Control in mean integral score. In addi-

tion, no discernible di↵erences were noted among mean units scores. We believe

this may be an artifact of the naturally high baseline units scores observed from

127 the Control condition. It is also possible that units were insuciently empha-

sized during training.

3. Does tutorial instruction using either electronic feedback or interactive engage-

ment provide evidence of better conceptual understanding compared to each

other and an untrained Control group?

Electronic feedback appears to have produced a noticeable e↵ect on students’

abilities to derive physical meaning from symbolic integrals, compared to in-

teractive engagement of three-person group tutorials. Moreover, participants

trained with electronic feedback posted statistically higher mean scores on the

assessment item pertaining to our untrained abstract integral ( AdB)thanstu- R dents in our Control condition; this was a claim that no other condition could

make.

In light of the evidence presented, we can state that the implementation of

electronic feedback in our Di↵erential path tutorial was e↵ective in achieving

our goal: helping introductory physics students extract physical meaning from

symbolic integrals. This was shown by the DPUE condition, which significantly

outperformed an untrained sample of students intended to estimate the abilities

of the Physics 2 population at large.

128 Chapter 7: Conclusions and Implications

The ultimate aim of this dissertation was the improved ability of introductory physics students to derive physical meaning from di↵erentials, di↵erential products, and integrals. To help guide this undertaking, we mapped a set of goals in Chapter 1.

In this chapter, we will revisit each objective, discuss its major conclusions, and consider their implications for the future.

7.1 Investigating, Discovering, & Estimating Frequencies of Conceptual Obstacles from Unproductive Interpretations of Physics Integrals & Their Components

In Chapter 3, we presented two independently conducted studies intended to dis- cover conceptual diculties students may have with physics integration and integral components, as well as to estimate the frequencies of these diculties. Existing re- search had already documented a variety of ways that students perceive and use dif- ferentials, di↵erential products, and integrals, and how these perceptions may impact their problem-solving success, as seen in Chapter 2.3.

In our “Exploratory Study,” we provided Physics 1 participants with a research task designed to elicit some of these incorrect conceptions. Of particular interest were the comparisons between macroscopic and di↵erential quantities and the physical

129 meaning of the di↵erential within an integral. Our findings aligned with some well- known results, such as the notion that di↵erentials are physically meaningless labels that instruct the reader to integrate with respect to a specific variable. However, we also uncovered two unexpected and possibly heretofore unknown conceptualizations of di↵erentials: the “rate” and “instantaneous value” interpretations. In the case of the former, the di↵erential is seen as a derivative or slope, sometimes even with

“rate-like” units (e.g., students may ascribe to the quantity dt units of 1/seconds).

The latter is characterized by a belief that the di↵erential endows physical quantities with the quality of being “at an instant,” and this was seen to be especially prevalent for di↵erential products. These interpretations were surprising, but quite common: as many as half of our participants provided responses consistent with the “rate” interpretation, and a quarter answered in a manner indicative of the “instantaneous value” interpretation.

Our second study, conducted later, took advantage of the physics di↵erentials, di↵erential products, and integrals assessment designed to measure students’ abilities to make physical sense of these symbolic quantities. A large sample of Physics 1 par- ticipants completed this assessment online, which allowed us to better estimate the frequencies of some of our previously-discovered diculties. Di↵erentials and di↵eren- tial products were interpreted as rates and/or instantaneous values by approximately half or more of the Physics 1 population. In addition, about half of our participants perceived integrals as accumulations of their integrands on at least one occasion.

Due to their apparent ubiquity at this educational level, it is our belief that instructors should be aware of these conceptualizations and should work to directly confront them during physics integration instruction. Opportunities ought to be

130 provided for students to reconcile their ideas about the meaning of the symbol dt in isolation with the symbol dt inside the integral vdt.Moreover,thesediculties are R not always expressed consistently, so it is recommended to promote generalization of the preferred conception across multiple physical contexts; simply noting that a student appears to understand the meaning of dt is not a guarantee that they will also comprehend dx.

7.2 Investigating Mediating & Moderating Relationships Be- tween the Abilities to Derive Physical Meaning from Dif- ferentials, Di↵erential Products, & Integrals

Following our inspection of some common diculties encountered in physics in- tegration, we considered the statistical relationships between the understanding of physics integrals and the understanding of their components in Chapter 4. We pre- sented the results of a study whose purpose was to examine and establish the possible inter-relationships between students’ successes assigning physical meaning to sym- bolic di↵erentials, di↵erential products, and integrals, as measured by our reliable assessment (sampled in Appendix A). Using a linear regression analysis, this study found strong evidence that the understanding of the physics di↵erential product medi- ates the understanding of di↵erentials and integrals; this finding was consistent across samples of Physics 1 and Physics 2 participants. This result has instructional implica- tions: students would find more success from instruction that follows the progression formulated by Von Kor↵ and Rebello (i.e., dt vdt vdt). In particular, because ! ! R of the very small (or possibly non-existent) Direct E↵ect (c0)fromTable4.4,simply emphasizing the meaning and units of the di↵erential may have no e↵ect on students’ ability to make sense of the integral. However, comprehension of physics di↵erentials

131 may prove beneficial in promoting understanding of the di↵erential product, whose instruction may ultimately lead to better integral understanding.

We were not able to make a formal statistical claim of moderation due to design limitations of the study; however, we presented evidence that suggests that familiarity of a symbolic integral’s physical context may a↵ect the strength of the indirect e↵ect of our model. This is to say, prior experience with the physical context of an integral may permit the avoidance of the line of reasoning promoted by the Von Kor↵ and

Rebello process-object layered framework, dt vdt vdt (Von Kor↵ and Rebello, ! ! R 2012) [52]. In such cases, informed students may be able to correctly interpret the physical meaning of an integral with little understanding of its constituent pieces.

The opposite may be true for less familiar physical contexts, in which our simple me- diation model was seen to more accurately describe the relationships of understanding between di↵erentials, di↵erential products, and integrals. We therefore recommend that instruction should follow the microscopic path mediated by understanding dif- ferential products, especially for integrals of novel and unfamiliar context.

Finally, as noted in Chapter 4.4, mediation models assume causality. Our results from the study presented in Chapter 4 are wholly consistent with statistical mediation via di↵erential products understanding. Still, ordinary least squares regression analy- sis does not definitively and conclusively demonstrate the assumed causal relationship between the three skills in reference. Therefore, a controlled experiment testing in- structional interventions that enable the measurement of the Indirect, Direct, and

Total E↵ects of our model is recommended.

132 7.3 Designing, Implementing, & Assessing Ecacy of In- structional Intervention Tutorials for Improving Students’ Abilities to Derive Physical Meaning from Symbolic Dif- ferentials, Di↵erential Products, & Integrals

To begin the work of achieving our goal of helping introductory physics students better associate symbolic integrals with their physical meanings, we presented two instructional intervention experiments in Chapters 5 and 6. The first study more narrowly focused on di↵erentials and di↵erential products instruction. Although we were not yet aware of any statistical relationships between the understanding of dif- ferentials, di↵erential products, and integrals (as seen in Chapter 4), our exploratory study of student diculties had already inspired our interest in training on the com- ponents of physics integrals. To do this, we designed a between-students experiment with pairs of short, contextual tutorials, either on paper or on a computer, the latter featuring electronic feedback. Two major factors of interest were tested: the e↵ect of the physical contexts used during training and the e↵ect of electronic feedback. We did not find an e↵ect of training context that was significant at the ↵ =0.05 level; however, nominal di↵erences exist that suggest the possibility of a small e↵ect of con- text, which might have been detected with more statistical power. Electronic feedback was seen to significantly boost performance on our di↵erentials post-test (shown in

Appendix B) compared to no-feedback training conditions. Still, our paper-based, no-feedback tutorials were seen to statistically improve performance on our post-test compared to our untrained Control condition. This study showed that it was possi- ble, with no more than 30 minutes of contextual training (even without feedback), to

133 significantly improve students’ abilities to extract physical meaning from di↵erentials and di↵erential products.

These encouraging results prompted the experiment discussed in Chapter 6, which would be our final study, called “Physics Integration Paths to Understanding.” The study was predicated on the Von Kor↵ and Rebello framework for physics integration instruction. Two instructional paths, which we called “Riemann” and “Di↵erential,” were used to design paper-based and electronic tutorials. We intended to assess and compare the ecacy of these instructional paths in improving physics integration conceptual understanding. Three experimental factors motivated this study: the ef- fect of instructional path; the e↵ect of units instruction; and the e↵ect of electronic feedback vs. interactive engagement. To measure the e↵ects of each of these factors, we deployed our physics di↵erentials, di↵erential products, and integrals assessment, which was not yet previously available in our Di↵erentials Training experiment from

Chapter 5. Our results showed no significant di↵erences between mean assessment integral scores from the Riemann and Di↵erential path tutorials. Furthermore, the inclusion of units instruction during tutorials was also not observed to statistically a↵ect mean assessment integral scores. Feedback, however, was highly influential in producing stronger average integral scores than interactive feedback conditions, whose tutorials were conducted in three-person collaborative groups. Our condition receiving electronic feedback was the only experimental group to significantly out- perform our Control condition in both mean integral score and our abstract integral interpretation item, which asked students to physically interpret the meaning of the quantity AdB. R

134 Unfortunately, our capability to detect notable signals within the data was ham- pered by ceiling e↵ects due to strong untrained, baseline scores. To better conduct this experiment a second time, any of the following alterations to experimental design would likely be beneficial: more participants per condition; more challenging physical contexts during assessment, with pilot-testing before deployment; more experimental conditions to discover interaction e↵ects; stronger dosage of units instruction for con- ditions receiving it; and an interim task to cleanse working memory. Nevertheless, in spite of these existing limitations, for the first time since we began our exploration of physics integration, the experiment detailed in Chapter 6 fulfilled our original goal of positively a↵ecting students’ abilities to associate symbolic integrals with physical meaning; the success of short, contextual training with feedback for physics integra- tion has now been quantitatively observed. And although we were unable to uncover discernible advantages to either of our physics integration instructional paths, we can confidently recommend the inclusion of feedback during instruction, the e↵ects of which were demonstrable in multiple experiments.

135 Appendix A: Sample Introductory Physics Di↵erentials, Di↵erential Products, and Integrals Assessment Items

A.1 Sample Physics 1 Assessment Items

A.1.1 Physics 1 Interpretation Items

Examples of assessment interpretation questions are shown below. Each question presents participants with a physics di↵erential, di↵erential product, or integral to correctly interpret from several possible choices.

1. A ball is dropped. While the ball falls, a stopwatch is used to measure time t

(in seconds). Which of the following best describes the quantity dt?

(a) The time on the stopwatch at some moment during the ball-drop.

(b) The time on the stopwatch at the moment the ball is dropped.

(c) An extremely small change in time at some moment during the ball-drop.

(d) A change in time large enough to measure on the stopwatch

(e) The rate (at a single moment) at which the position changes.

(f) The rate (at a single moment) at which time passes

136 2. A toy is suddenly pulled across the floor. At time t (measured in seconds),

the acceleration of the toy is a (measured in meters/second2). Which of the

following best describes the quantity adt?

(a) The velocity of the toy at a single moment

(b) The rate (at a single moment) at which the toy’s acceleration changes as

time passes

(c) The acceleration of the toy at a single moment

(d) An extremely small change in the toy’s velocity

(e) A change in the toy’s velocity large enough to measure

(f) The rate (at a single moment) at which the toy’s velocity changes as time

passes

3. A car is traveling in a straight line. Between times t1 and t2 (in seconds), it has

t2 velocity v (in meters/second). Which of the following best describes t1 vdt? R

(a) The total velocity of the car between t1 and t2.

(b) The total change in the car’s position between t1 and t2.

(c) The average velocity of the car between t1 and t2.

(d) The total change in the velocity of the car between t1 and t2.

(e) The average position of the car between t1 and t2.

(f) The anti-derivative of the car’s velocity.

137 A.1.2 Physics 1 Units Items

Examples of assessment units questions are shown below. Each question presents participants with a physics di↵erential, di↵erential product, or integral whose units they must correctly identify from several possible choices.

1. A ball is dropped. While the ball falls, a stopwatch is used to measure time t

(in seconds). Which of the following shows possible units for dt?

(a) meters/second

(b) No units

(c) seconds/meter

(d) 1/second

(e) seconds

2. A toy is suddenly pulled across the floor. At time t (measured in seconds),

the acceleration of the toy is a (measured in meters/second2). Which of the

following best describes the quantity adt?

(a) meters/second2

(b) meters/second3

(c) meters/second

(d) meters

(e) No units

138 3. A car is traveling in a straight line. Between times t1 and t2 (in seconds), it has

velocity v (in meters/second). Which of the following shows possible units for

t2 t1 vdt? R (a) No units

(b) meters/second

(c) meters

(d) meters seconds · (e) meters/second2

A.2 Sample Physics 2 Assessment Items

A.2.1 Physics 2 Interpretation Items

Examples of assessment interpretation questions are shown below. Each question presents participants with a physics di↵erential, di↵erential product, or integral to correctly interpret from several possible choices.

1. A stopwatch is used to measure time t (in seconds) as a car moves. Which of

the following best describes dt?

(a) The time on the stopwatch at the moment the car starts moving.

(b) The rate (at a single moment) that the car’s position changes as time

passes.

(c) The rate (at a single moment) that time passes.

(d) An extremely small change in time.

(e) A change in time large enough to measure on the stopwatch.

139 2. An electron in a magnetic field experiences an acceleration a (in meters/second2)

at time t (in seconds). Which of the following best describes adt?

(a) The acceleration of the electron at a single moment.

(b) The rate (at a single moment) the electron’s acceleration is changing as

time passes.

(c) The rate (at a single moment) the electron’s velocity is changing as time

passes.

(d) An extremely small change in the electron’s velocity

(e) The velocity of the electron at a single moment.

3. A car is traveling in a straight line. Between times t1 and t2 (in seconds), it has

t2 velocity v (in meters/second). Which of the following best describes t1 vdt? R

(a) The total velocity of the car between t1 and t2.

(b) The total change in the car’s position between t1 and t2.

(c) The average velocity of the car between t1 and t2.

(d) The total change in the velocity of the car between t1 and t2.

(e) The average position of the car between t1 and t2.

A.2.2 Physics 2 Units Items

Examples of assessment units questions are shown below. Each question presents participants with a physics di↵erential, di↵erential product, or integral whose units they must correctly identify from several possible choices.

140 1. A stopwatch is used to measure time t (in seconds) as a car moves. Which of

the following shows possible units for dt?

(a) meters/second

(b) No units

(c) seconds/meter

(d) 1/second

(e) seconds

2. An electron in a magnetic field experiences an acceleration a (in meters/second2)

at time t (in seconds). Which of the following shows possible units for adt?

(a) meters/second2

(b) meters/second3

(c) meters/second

(d) meters

(e) No units

3. A car is traveling in a straight line. Between times t1 and t2 (in seconds), it has

velocity v (in meters/second). Which of the following shows possible units for

t2 t1 vdt? R (a) No units

(b) meters/second

(c) meters

141 (d) meters seconds · (e) meters/second2

142 Appendix B: Di↵erentials Training Post-Test Items

Post-test items from the physics di↵erentials tutorials experiment (Chapter 5) are shown below. These items were used to calculate the post-test score for this experiment.

1. Several students in a physics class are presented with unfamiliar symbols, “AdB.”

They attempt to interpret the meaning of “AdB” without knowing the context.

All students assume that A and B are physical quantities with some meaning.

Indicate the student with whom you most agree.

(a) Student A: “This must be some sort of product between A and an in-

finitesimally small amount of B. I guess it has units of A Bbecauseit’sa · product, but I don’t know what ‘A’ and ‘B’ are.”

(b) Student B: “That doesn’t sound right to me. The dB just means that

we want the value of A at an instant of B. The dB doesn’t have its own

units because it’s not really a physical quantity like A or B -it’stelling

you that the quantity is instantaneous. The units are A, whatever ‘A’ is.”

(c) Student C: “It’s not a product and it’s not just A at some instant of B.

It’s the rate of A with respect to B. It’s like a derivative. Notice the dB?

A Its units are B .” 143 2. A bicyclist is currently traveling at a speed v0. Write an equation (with an

equals sign) for the distance the biker travels in an infinitesimal amount of

time.

3. The bicyclist applies the brake and skids to a stop. The braking happens for a

distance x due to a friction force Fk as the tires slide on the ground. What is

[physically] meant by Fkdx?

4. Suppose the bike and the rider have a combined mass of m. What are the units

1 of m Fkdt?

5. After coming to a stop, the bicyclist decides to pump up one of the bike tires.

The pump is an air-filled cylinder with a piston. The air inside the cylinder is

at pressure P and the cylinder has a cross-sectional area A. Write an equation

(with an equals sign) for the work done by the cyclist when compressing the

pump by dx.

6. A cylinder of radius R has a volume mass density ⇢(x) (measured in kilograms/meter3)

that varies only along the length of the cylinder. How much mass is contained

in an infinitesimally small length of the cylinder dx?

7. A line of length L contains charge Q spread uniformly along its length. A

student [mistakenly] claims that in an infinitesimally small length dl of the line,

there is “zero charge, because dl is basically zero length.” Find an expression

for the amount of charge contained in a length dl along the line.

144 Bibliography

[1] CB Allendoerfer. Editorial [di↵erentials]. American Mathematical Monthly, 59:403–406, 1952.

[2] Nathaniel Amos and Andrew F. Heckler. Spatial reasoning ability and the con- struction of integrals in physics. In Physics Education Research Conference 2014, PER Conference, pages 23–26, Minneapolis, MN, July 30-31 2014.

[3] Nathaniel Amos and Andrew F. Heckler. Student understanding of di↵erentials in introductory physics. In Physics Education Research Conference 2015, PER Conference, pages 35–38, College Park, MD, July 29-30 2015.

[4] M Artigue, J Menigaux, and L Viennot. Some aspects of students’ conceptions and diculties about di↵erentials. European Journal of Physics,11(5):262,1990.

[5] Mich`ele Artigue. Analysis, pages 167–198. Springer Netherlands, Dordrecht, 1991.

[6] Alan Baddeley. Working memory, thought, and action, volume 45. OUP Oxford, 2007.

[7] Reuben M Baron and David A Kenny. The moderator–mediator variable dis- tinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of personality and social psychology, 51(6):1173, 1986.

[8] Jacob Cohen. Chapter 2 - the t test for means. In Jacob Cohen, editor, Statistical Power Analysis for the Behavioral Sciences (Revised Edition), pages 19 – 74. Academic Press, revised edition edition, 1977.

[9] Jacob Cohen. A power primer. Psychological bulletin, 112(1):155, 1992.

[10] Lee J. Cronbach. Coecient alpha and the internal structure of tests. Psychome- trika,16(3):297–334,1951.

[11] Lili Cui, N. Sanjay Rebello, and Andrew G. Bennett. College students transfer from calculus to physics. AIP Conference Proceedings,818(1):37–40,2006.

145 [12] Tevian Dray and Corinne A. Manogue. Bridging the Gap between Mathemat- ics and the Physical Sciences. In Diane Smith and Elisabeth Swanson, edi- tors, Preparing Future Science and Mathematics Teachers, pages 39–41. Montana State University, Bozeman, 2005.

[13] Tevian Dray and Corinne A. Manogue. Bridging the Gap between Mathematics and the Physical Sciences, 2010. Presentation: American Association of Physics Teachers: 2010 Summer Meeting - Portland, Oregon.

[14] Tevian Dray and Corinne A. Manogue. Putting Di↵erentials back into Calculus. College Math. J., 41:90–100, 2010.

[15] Todd A. Grundmeier, Jennifer Hansen, and Emily Sousa. An exploration of definition and procedural fluency in integral calculus. PRIMUS,16(2):178–191, 2006.

[16] Richard R. Hake. Interactive-engagement versus traditional methods: A six- thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1):64–74, 1998.

[17] D. Halliday, R. Resnick, and J. Walker. Fundamentals of Physics. Wiley, 10 edition, 2013.

[18] Ibrahim Abou Halloun and David Hestenes. The initial knowledge state of college physics students. American journal of Physics,53(11):1043–1055,1985.

[19] David Hammer. Student resources for learning introductory physics. American Journal of Physics,68(S1):S52–S59,2000.

[20] John Hattie and Helen Timperley. The power of feedback. Review of educational research,77(1):81–112,2007.

[21] Andrew F. Hayes. Introduction to Mediation, Moderation, and Conditional Pro- cess Analysis: A Regression-Based Approach. The Guilford Press, 5 2013.

[22] Andrew F. Hayes and Nicholas J. Rockwood. Regression-based statistical medi- ation and moderation analysis in clinical research: Observations, recommenda- tions, and implementation. Behaviour Research and Therapy, pages –, 2016.

[23] Andrew F. Hayes and Michael Scharkow. The relative trustworthiness of infer- ential tests of the indirect e↵ect in statistical mediation analysis. Psychological Science, 24(10):1918–1927, Aug 2013.

[24] David Hestenes, Malcolm Wells, and Gregg Swackhamer. Force concept inven- tory. The physics teacher,30(3):141–158,1992.

146 [25] J. Hiebert and P. Lefevre. Conceptual and procedural knowledge in mathematics: an introductory analysis. In Conceptual and Procedural Knowledge: The Case of Mathematics. Lawrence Erlbaum, 1986.

[26] D. Hu and N.S. Rebello. Characterizing student use of di↵erential resources in physics integration problems. AIP Conference Proceedings,1513(1):186–189, 2013.

[27] Dehui Hu and N. Sanjay Rebello. Using conceptual blending to describe how students use mathematical integrals in physics. Phys. Rev. ST Phys. Educ. Res., 9:020118, Nov 2013.

[28] Steven R. Jones. Areas, anti-derivatives, and adding up pieces: Definite integrals in pure mathematics and applied science contexts. The Journal of Mathematical Behavior, 38:9–28, Jun 2015.

[29] Paul Kline. The Handbook of Psychological Testing. Psychology Press, 2000.

[30] Joshua Von Kor↵ and N. Sanjay Rebello. Distinguishing between ‘change’ and ‘amount’ infinitesimals in first-semester calculus-based physics. American Jour- nal of Physics,82(7):695–705,2014.

[31] Nevin Mahir. Conceptual and procedural performance of undergraduate students in integration. International Journal of Mathematical Education in Science and Technology,40(2):201–211,2009.

[32] Corinne A. Manogue, Kerry Browne, Tevian Dray, and Barbara Edwards. Why is amp`ere’ss law so hard? a look at middle-division physics. American Journal of Physics,74(4):344–350,2006.

[33] Lillian C McDermott et al. Physics by inquiry, volume 1. Physics by Inquiry, Volume 1, by Lillian C. McDermott, Physics Education Group, Univ. of Wash- ington, pp. 400. ISBN 0-471-14440-1. Wiley-VCH, August 1995.,page400,1995.

[34] David E. Meel. Honors students’ calculus understandings: Comparing calcu- lus&mathematica and traditional calculus students. In Research in Collegiate Mathematics Education III, pages 163–215. American Mathematical Society, 1998.

[35] Dawn C. Meredith and Karen A. Marrongelle. How students use mathematical resources in an electrostatics context. American Journal of Physics,76(6):570– 578, 2008.

[36] Dong-Hai Nguyen and N. Sanjay Rebello. Students’ diculties with integration in electricity. Phys. Rev. ST Phys. Educ. Res., 7:010113, Jun 2011.

147 [37] Dong-Hai Nguyen and N Sanjay Rebello. Students’ understanding and applica- tion of the area under the curve concept in physics problems. Physical Review Special Topics-Physics Education Research, 7(1):010112, 2011.

[38] A. Orton. Students’ understanding of integration. Educational Studies in Math- ematics, 14(1):1–18, 1983.

[39] David N Perkins, Gavriel Salomon, et al. Transfer of learning. International encyclopedia of education, 2:6452–6457, 1992.

[40] Kristopher J. Preacher and Andrew F. Hayes. Spss and sas procedures for esti- mating indirect e↵ects in simple mediation models. Behavior Research Methods, Instruments, & Computers,36(4):717–731,2004.

[41] Shaker Rasslan and David Tall. Definitions and images for the definite integral concept. In PME CONFERENCE, volume 4, pages 4–089, 2002.

[42] Edward F Redish. Problem solving and the use of math in physics courses. arXiv preprint physics/0608268,2005.

[43] Abraham Robinson. Non-standard Analysis. Princeton University Press, 1996.

[44] Vicki Sealey. Definite integrals, riemann sums, and area under a curve: What is necessary and sucient? In Psychology of Mathematics Education, page 46, 2006.

[45] Vicki Sealey. A framework for characterizing student understanding of riemann sums and definite integrals. The Journal of Mathematical Behavior,33:230– 245, 2014.

[46] Derar Serhan. Students’ understanding of the definite integral concept. Interna- tional Journal of Research in Education and Science, 1(1):84–88, 2015.

[47] Anna Sfard. On the dual nature of mathematical conceptions: Reflections on processes and objects as di↵erent sides of the same coin. Educational Studies in Mathematics, 22(1):1–36, 1991.

[48] David R Sokolo↵, Ronald K Thornton, and Priscilla W Laws. RealTime Physics Active learning laboratories, Module 1: Mechanics. John Wiley & Sons, 2011.

[49] James Stewart. Calculus: Early Transcendentals. Thomson Brooks/Cole, 6 edition, 2008.

[50] Patrick W. Thompson and Jason Silverman. The concept of accumulation in calculus, pages 43–52. Mathematical Association of America, 1 2008.

148 [51] Alan Van Heuvelen. Overview, case study physics. American Journal of Physics, 59(10):898–907, 1991.

[52] J. Von Kor↵ and N.S. Rebello. Teaching integration with layers and representa- tions: A case study. Phys. Rev. ST Phys. Educ. Res., 8:010125, May 2012.

[53] F. Richard Yeatts and Joan R. Hundhausen. Calculus and physics: Challenges at the interface. American Journal of Physics,60(8):716–721,1992.

[54] Michelle Zandieh. A theoretical framework for analyzing student understanding of the concept of derivative. Research in Collegiate Mathematics Education IV, 8:103–127, 2000.

149