MAKIN’ MATH MOVE: A FULL BODY INTERACTIVE LEARNING ENVIRONMENT FOR PRE-ALGEBRAIC PRACTICE

By TIFFANIE R. SMITH

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2019 © 2019 Tiffanie R. Smith I dedicate this to my entire support system of family, friends and loved ones. Without you all, I would not and could not have made it through. This is also dedicated to all of the future Black STEM scholars. It is possible. We can do it. ACKNOWLEDGMENTS First, I would like to give all glory and honor to God, because without Him none of this would have been possible. Secondly, I’d like to thank Yolanda D. Smith and Ollie S. Williams, my mother and grandmother, for instilling perseverance within me and their tremendous amount of love and support. I’d also like to thank my sister, brother, family members, New Light Baptist Church family, and friends for their constant prayers and support along this journey. Thank you to my committee members and advisor for helping to shape and reshape my ideas along the way. Thank you to the North Carolina A&T State University for my undergraduate career, foundation, and inspiration to pursue my doctoral degree. I’d also like to thank the National Science Foundation Graduate Research and Ford Dissertation Fellowship Programs for funding my education. I would also like to thank Mr. Curtis Peterson and the Caring and Sharing Learning School, as well as the Hopewell City Public Schools. Last, but certainly not least, I would like to thank the second family I’ve gained by way of the Human Experience Research Lab and at the University of Florida. I appreciate all of the support, friendship, laughs and memories.

4 TABLE OF CONTENTS page ACKNOWLEDGMENTS ...... 4 LIST OF TABLES ...... 9 LIST OF FIGURES ...... 11 LIST OF SYMBOLS ...... 13 ABSTRACT ...... 14

CHAPTER 1 INTRODUCTION ...... 16 Background ...... 16 Motivation ...... 17 Research Goals, Research Questions and Thesis ...... 18 Overview of Approach ...... 19 Contributions ...... 20 Organization of the Dissertation ...... 20 2 LITERATURE REVIEW ...... 22 Algebraic Misconceptions ...... 22 Culturally Relevant Pedagogy ...... 24 Culturally Relevant Pedagogy in Math Education ...... 25 Nine Interrelated Dimensions of African-American Culture ...... 28 Embodied Cognition ...... 29 Gestures in Education ...... 30 Gesture Based Educational Technologies ...... 31 Interface Design ...... 33 Natural User Interface Design ...... 33 Designing of gesture sets ...... 36 Gesture classification and taxonomies ...... 37 Designing for Culture ...... 37 Designing for Children ...... 39 Digital Game Based Learning and Gamification ...... 42 Math DGBL ...... 46 Gesture Based DGBL ...... 48 What’s Missing: Significance of Current Research ...... 50 3 DESIGN AND DEVELOPMENT OF Makin’ MATH MOVE: A CULTURALLY RELEVANT FULL BODY INTERACTIVE LEARNING ENVIRONMENT FOR PRE-ALGERBRAIC PRACTICE ...... 51

5 Use of System and Rationale ...... 51 Content ...... 55 Interface Design ...... 57 System Architecture and Implementation ...... 60 4 GESTURE ELICITATION ...... 64 Purpose ...... 64 Participants ...... 64 Instruments and Measures ...... 64 Materials ...... 64 Procedure ...... 65 Results ...... 65 Gesture Taxonomy ...... 65 Agreement ...... 68 User Defined Gesture Set ...... 71 Survey Results ...... 72 Discussion and Next Steps ...... 72 Predictions vs. Actual Gestures ...... 72 Limitations ...... 74 Next Steps ...... 74 Summary and Conclusion ...... 74 5 INITIAL EVALUATION OF MAKIN’ MATH MOVE’S USABILITY ...... 76 Purpose ...... 76 Participants ...... 76 Materials and Setup ...... 77 Measures ...... 77 Procedure ...... 77 Results ...... 78 Task Completion and Gestural Accuracy ...... 78 Usability ...... 80 Cultural Relevance ...... 80 Additional Results from Survey, Interview and Observations ...... 81 Discussion and Next Steps ...... 82 Gestural Accuracy and Task Completion ...... 82 Usability ...... 82 Cultural Relevance ...... 83 Conclusion ...... 83 6 COMPLETE USABILITY STUDY ...... 85 Purpose ...... 85 Participants ...... 85 Hypotheses ...... 85 Results ...... 86

6 Usability ...... 86 Task Completion and Gestural Accuracy ...... 86 Cultural Relevance ...... 88 Additional Results from Survey, Interview and Observations ...... 89 Revisiting the Hypotheses ...... 90 Summary, Discussion and Next Steps ...... 90 Gestural Accuracy and Task Completion ...... 90 Usability ...... 92 Cultural Relevance ...... 93 Conclusion ...... 93 7 EVALUATION OF MAKIN’ MATH MOVE FOR IMPACT ON MOTIVATION AND ACADEMIC ACHIEVEMENT ...... 95 Purpose ...... 95 Participants ...... 95 Materials and Study Set Up ...... 97 Measures ...... 98 Hypotheses ...... 100 Procedure ...... 100 Before the Study ...... 100 Study Logistics ...... 100 Pre-Test and Post Test ...... 101 The Experiment ...... 101 Results ...... 102 Raw Score ...... 103 Weighted Score ...... 107 Motivation Score-Overall ...... 107 Motivation: Attention ...... 107 Motivation: Relevance ...... 108 Motivation: Confidence ...... 109 Motivation: Satisfaction ...... 109 Makin’ Math Move Performance Data ...... 109 Observations and Post-Survey Responses ...... 111 Discussion ...... 114 Summary ...... 116 8 SUMMARY AND FUTURE WORK ...... 117 Summary and Research Questions ...... 117 Contributions ...... 118 Discussion, Limitations and Future Work ...... 118 Conclusion ...... 122

APPENDIX A TAXONOMIC BREAKDOWN BY INDIVIDUAL REFERENTS ...... 124

7 B CULTURAL RELEVANCE ...... 127 B.1 Questionnaire for Stimuli Preference (Verve) ...... 127 B.2 Child Activity Questionnaire : Movement and Music Subscale ...... 128 C MATH BENCHMARK ...... 129 C.1 Course Motivation Survey ...... 132 D ADDITIONAL SIGNIFICANT FINDINGS FROM EVALUATION ...... 138 REFERENCE LIST ...... 141 BIOGRAPHICAL SKETCH ...... 161

8 LIST OF TABLES Table page 2-1 Gesture Conditions ...... 30 2-2 Embodied Mapping Metaphors ...... 32 2-3 Section of Mapping between Cultural Dimensions and Design Components ..... 39 2-4 Educational Design Principles ...... 44 2-5 Types of Motivation and Corresponding Theories ...... 45 3-1 2017 FSA Performance by Predominantly African American Schools in Gainesville . 55 3-2 Strands Tested on SOLs by Grade Level and Percentage ...... 56 3-3 Strands Tested on FSA by Grade Level and Percentage ...... 57 3-4 Standards highlighted in Makin’ Math Move ...... 57 4-1 Taxonomy of Full Body Gestures for a Culturally Appropriate Educational Technology 66 4-2 Taxonomy for Form Dimension ...... 67 4-3 Gesture Consensus for Each Referent ...... 71 5-1 Average Task Completion by Grade ...... 79 5-2 Average Task Completion by Gender ...... 79 5-3 Average Task Completion by Kinect Use ...... 79 5-4 Task Completion ...... 79 5-5 Average SUS Score by Grade ...... 80 5-6 Average SUS Score by Gender ...... 80 5-7 Average SUS Score by Kinect Use ...... 80 5-8 Descriptive Statistics for Movement/Music Mosaic Subscale ...... 81 5-9 Descriptive Statistics for the Questionnaire of Stimuli Preference ...... 81 6-1 Average SUS Score by Grade ...... 86 6-2 Average SUS Score by Gender ...... 86 6-3 Average SUS Score by Kinect Use ...... 87 6-4 Average Task Completion by Grade ...... 87

9 6-5 Average Task Completion by Gender ...... 87 6-6 Average Task Completion by Kinect Use ...... 87 6-7 Task Completion ...... 88 6-8 Gestural Recognition Accuracy ...... 88 6-9 Descriptive Statistics for Movement/Music Mosaic Subscale ...... 89 6-10 Descriptive Statistics for the Questionnaire of Stimuli Preference ...... 89 7-1 ARCS Components of Motivation and Their Foundational Theories ...... 99 7-2 Problem Types Featured in Makin’ Math Move ...... 110

10 LIST OF FIGURES Figure page 2-1 AADMLSS ...... 27 2-2 Cornrow Curves ...... 27 2-3 Math DGBL: a) Mathmazing, b) MeMa Pads and c) Jumpido ...... 49 3-1 Modes ...... 52 3-2 Step 1: Problem Shown to Student ...... 53 3-3 Step 2: Selecting the First Operand ...... 53 3-4 Step 3: Performing the Appropriate Gesture ...... 54 3-5 Step 4: Selecting the Second Operand and Solving ...... 54 3-6 Screenshot of Dance Central interface ...... 58 3-7 Screenshot of Just Dance interface ...... 58 3-8 Login Page of Makin’ Math Move ...... 58 3-9 Mock-up of Problem Solving Screen ...... 59 3-10 Implemented Problem Solving Screen ...... 59 3-11 Background and Music Settings ...... 59 3-12 Avatar Settings ...... 59 3-13 Updated Background and Music Settings ...... 60 3-14 Updated Avatar Settings ...... 60 3-15 System Architecture of Makin’ Math Move ...... 61 3-16 Screenshot of VGB software ...... 62 4-1 Hand Position Guide ...... 67 4-2 Taxonomic Breakdown of the 180 Elicited Gestures ...... 68 4-3 Agreement Scores for the Elicited Gestures ...... 70 4-4 User Defined Gestures and Optional Choices for Divide and Help Referents ..... 73 7-1 Control Group’s Interface ...... 97 7-2 Experiment Setup ...... 98

11 7-3 Group * Grade Interaction ...... 104 7-4 Gender * Group * Grade Interaction - Female ...... 105 7-5 Gender * Group * Grade Interaction - Male ...... 105 7-6 Group * Gender * Time Interaction - Female ...... 106 7-7 Group * Gender * Time Interaction - Male ...... 106 7-8 AchieveWeighted (Performance Level) * Group Interaction ...... 108 7-9 Group * Gender Interaction ...... 109 7-10 Means and Standard Deviations - Problem Type C ...... 111 7-11 Variance Equality Test- Problem Type C ...... 111 7-12 t-Test - Problem Type C ...... 111 A-1 Breakdown of Add Gestures ...... 124 A-2 Breakdown of Subtract Gestures ...... 124 A-3 Breakdown of Multiply Gestures ...... 124 A-4 Breakdown of Divide Gestures ...... 125 A-5 Breakdown of Input Gestures ...... 125 A-6 Breakdown of Home Gestures ...... 125 A-7 Breakdown of Help Gestures ...... 126 A-8 Breakdown of Pause Gestures ...... 126 A-9 Breakdown of Next Gestures ...... 126 D-1 Gender * Grade Interaction ...... 138 D-2 Time * Gender Interaction ...... 139 D-3 Gender * AchieveWeighted (Performance Level) Interaction ...... 140 D-4 Gender * Time Interaction ...... 140

12 LIST OF SYMBOLS, NOMENCLATURE, OR ABBREVIATIONS

CCSS Common Core State Standards

CMS Course Motivation Survey

DGBL Digital Game Based Learning

FSA Florida Standards Assessments

HCI Human Computer Interaction

NAEP National Assessment of Educational Progress

NCTM National Council of Teachers of Mathematics

NUI Natural User Interface

SOL Standards of Learning

STEM Science, Technology, Engineering and Math

SUS System Usability Scale

UI User Interface

VGB Visual Gesture Builder

13 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy MAKIN’ MATH MOVE: A FULL BODY INTERACTIVE LEARNING ENVIRONMENT FOR PRE-ALGEBRAIC PRACTICE By Tiffanie R. Smith December 2019 Chair: Juan E. Gilbert Major: Human Centered Computing Dancing and movement has been and continues to play a significant role in the African-American culture. It has a transformative and engaging power of bringing joy to people and bringing people together. This research attempts to harness this power and the enjoyment of dance and use it to impact the academic engagement and math achievement of pre-algebraic African-American students. Despite Algebra being considered the gatekeeper to higher-level math courses and a key requirement for entrance into STEM career fields, many students nationwide are not proficient in this subject. Research suggests that thisis due to a weakened arithmetic foundation needed before the transition to algebraic thinking. African-Americans, in particular, are underperforming in Algebra, and many other levels of math. Lack of culture in mainstream schooling has been attributed to weak academic performance for this and other minority demographics. This research explores the intersection of African-American culture and educational technology to strengthen the foundation of this demographic by improving pre-algebraic skills necessary for success in Algebra 1. In particular, this research leverages two of the nine interrelated dimensions of African-American culture, movement expressiveness and verve. The two dimensions were combined in the form of a full body gestural based educational technology, entitled Makin’ Math Move, that also leverages effective gamification properties used in digital game based learning. Similar to gamessuch as Dance Central, this technology harnesses gesture mapping and recognition capabilities. The system maps specified gestures to mathematical operations which serve as input for

14 solving pre-algebraic problems. This research invoked a user centered design. Participation from 6th and 7th grade African-American students was utilized throughout the design, via a gesture elicitation study, implementation and evaluation of this novel technology. The gesture elicitation study resulted in a partial user-defined gesture set that was finalized by the usability phase of the research. The usability portion of the research suggested that students were able to use the system, despite low scores on the System Usability Scale, and that more training of the gestures was required before final evaluation. An evaluation of Makin’ Math Moveon its impact on math performance and motivation resulted in a non-significant increase in the Attention component of the motivation measure. However, there was no other significant findings between the students in the experimental and control groups. Despite the findings in both the usability and evaluation studies, most students enjoyed using the system and some mentioned that the moving/dancing made their experience enjoyable. Contributions of this research includes the system, Makin’ Math Move, the gesture database created to recognize the students’ gestures which can be incorporated into other math Kinect based gestural technologies and findings that suggest the creation of an alternative usability measurement that matches reading comprehension abilities of middle-school students.

15 CHAPTER 1 INTRODUCTION 1.1 Background

Algebra has been labeled the ”gatekeeper” to success in educational and career endeavors [16, 89]. Due to its importance, 32 states have made it a requirement for high school graduation [49]; the remaining states define the amount of math credits required to graduate, but do not specify which courses must be taken. Although required by most states, the U.S. struggles in the subject nationwide. While there is no national standard or standardized test to measure Algebra competency, most states offer their own end of course assessments. Additionally, there are national assessments and reports that gauge some level of algebraic performance of students in the 8th and 12th grades, which often occurs before many have taken the course and then after Algebra I and additional math courses have been taken. The National Assessment of Educational Progress (NAEP) is one such evaluation. 30% of the 8th grade test covers Algebra topics, while 35% of the 12th grade test adheres to the topic [115]. Although not tailored toward Algebra specifically, on average, the majority of students are not at the proficiency level for either grade; 33% of 8th graders and 25% of 12th graders were at or above the proficiency level187 [ ]. Proficiency, which requires varying levels of algebraic knowledge for both grades, requires a score of 299 out of 500 and 176 out of 300 for eighth and twelfth graders, respectively. In 2015, 8th graders averaged 282 on the entire math assessment and 12th grade students averaged 152 points. Students experience difficulties in varying content areas within the subject. Ina longitudinal study with 21,000 ninth graders enrolled in Algebra 1 in 2009, results indicated that most students found success, measured by proficiency, in the lower levels of Algebra involving algebraic expressions (86%) and multiplicative and proportional thinking (59%) [85]. Proficiency decreased as skill complexity increased in the remaining three levels studied, which were algebraic equivalents, systems of equations, and linear functions. Only 9% of the students in the study were found to be proficient in the highest level. Students often struggle

16 in higher levels of Algebra due to the shift in mathematical thinking; they are transitioning from arithmetic to algebraic thinking [172] and are not practicing adequately enough to make this transition. 1.2 Motivation

Although students across the nation struggle when switching to Algebra, minority students are underperforming in Algebra, and all other levels of math, when compared to their counterparts [187]. In 2015, African-Americans, as a group, had the lowest percentage of students meeting the math proficiency level when compared to students in other racial/ethnic groups as assessed by the NAEP. Researchers have suggested several underlying factors that attribute to the lackluster math performance of African-American students. First, African-Americans often underperform in mathematics, specifically Algebra, due to inadequate and unequal opportunities to learn [183]. African-American students from low income families are most likely to be in low achieving/ ability math classrooms [106]. For those who do perform highly at a young age, they are presented an unequal opportunity to be enrolled in college-dependent courses such as Algebra, which is pertinent to their success [179]. The Early Childhood Longitudinal Study, Kindergarten Class of 1998-1999 (ECLS-K) followed a group of students until the reached the eighth grade in the 2006-2007 school year [193]. It was found that of the students who performed highly in the fifth grade, only 35% of the exceptional African-American students were enrolled in Algebra 1 in the eighth grade; however, 63% of the White students who performed well three years prior were enrolled in the course in eighth grade. A secondary study utilizing the same data revealed that teacher evaluations also severely impacted the placement of Black students in Algebra classes [63]. Secondly, stereotypical images associated with successful representatives of the STEM fields, such as White ”nerdy” males aid in discouraging math excellence and perpetuating the notion that math and related subjects are not fields in which African-Americans can succeed106 [ ]. These images and negative perceptions hinder African-American students from pursuing higher levels of math educations and careers in the STEM fields40 [ ].

17 Also attributed to the poor performance is the disconnect between the context and language utilized in African-American students’ everyday experiences and mainstream schooling[106, 143, 180]. Traditional math curriculum, including assessments, have been found to be more in line with the experience of White middle class Americans, which is not always fully relatable to African-American students [91, 106]. This research is motivated to address this cause of underperformance by providing a tool designed to incorporate aspects of African-American culture into the learning experience. This incorporation is the basis for the ideology behind culturally relevant pedagogy, which will be discussed further in Section 2.2. 1.3 Research Goals, Research Questions and Thesis

This research aims to find a meaningful way to address the low success rate for African-Americans in Algebra by attacking the problem before it occurs. This research will approach this idea by focusing on improving the knowledge and retention rate of some pre-algebraic concepts that are necessary to be successful in the gatekeeper course. Students are not simply failing the course because they cannot grasp the concepts; their foundational knowledge is not sufficient171 [ ]. They are not able to build upon and transform their arithmetic ways of thinking into algebraic reasoning because of the weakness of their mathematical foundation. The areas essential to having strong success in algebraic subject matter will be discussed in detail in Section 2.1. The main goal of this research is to design and develop a culturally relevant educational tool to improve the math performance of 6th and 7th grade African-American students. These grades were targeted because students are typically enrolled in Algebra I in the 8th or 9th grade, which would be too late to intervene. This technology aims to provide a means of building the algebraic foundation by utilizing cultural and familiar aspects to limit the disconnect otherwise experienced in mainstream schooling. This research addresses the following questions:

18 • To what extent is the proposed system usable, in terms of system usability and gesture recognition accuracy, by 6th and 7th grade African-American students?

• Is the system found to be culturally relevant by 6th and 7th grade African-American students?

• To what extent are 6th and 7th grade African-American learners more motivated in pre-algebra problem solving when using Makin’ Math Move as compared to traditional practice?

• To what extent do 6th and 7th grade African-American learners improve pre-algebra problem solving performance after using Makin’ Math Move as compared to traditional practice? The thesis statement for this research is as follows: Using a culturally relevant, full body interactive learning environment will cause an increase in measures of math performance and motivation for African-American sixth and seventh grade students that is the same or greater than any improvement the same measures of the African-American sixth and seventh grade students who do not utilize the system. 1.4 Overview of Approach

This research is going to apply a culturally relevant approach in designing the proposed math educational technology by incorporating the cultural aspects of music and dance, which also relate to cultural dimensions of verve and movement. The technology will utilize dance movements and other gestures as input to manipulate and practice pre-algebra problems in an engaging, stimulating and interactive format with gamification features. To investigate this thesis, the research was broken down into three different phases. First, the content for the system was selected by analyzing standardized testing scores of African-American students from one district in Florida and one district in Virginia. The content area in which students were underperforming and which aligned with one of nine areas deemed necessary for Algebra I success was selected. Next, a gesture elicitation study was conducted to garner input for nine different functions within the system. Twenty students from the targeted demographic were paired and asked to create two distinct gestures for each of the nine functions. The gestures were analyzed for frequency and agreement and employed in the initial design of the system.

19 Using information gathered from the elicitation study and research on culturally relevant pedagogy, interface design and digital game based learning, a culturally relevant gestural learning technology was designed and developed. The initial system was tested for usability and cultural relevancy by twenty sixth and seventh grade African-American students. After modifications were made, the system was evaluated by eight African-American students ofthe same grade levels and their results were compared to a control group of eight students who did not use the system. The evaluation period occurred over the course of three weeks, with 20 minute sessions with the system happening three times per week. Students solved math problems adapted from standardized tests and were given pre and post assessments to measure motivation and academic performance. 1.5 Contributions

The results of this dissertation relates to several areas including Computer Science, Math Education, and the area of Human Computer Interaction (HCI). This research resulted in a full body learning technology which is contribution to the area of Computer Science. The results from the evaluation of the tool suggests that it can contribute to Math Education by providing a new means of engaging the statistically underperforming African-American demographic and improving the academic gap between this demographic and their Caucasian counterparts in this subject area. With regards to the area of HCI, in particular in the area of designing and developing gestural user interfaces, the gesture recognition database that was developed for the system can be utilized in future development of similar systems that are math oriented and incorporate the use of Kinect software. 1.6 Organization of the Dissertation

This dissertation is organized into eight chapters. Chapter 1 provides an overview of the motivation, goals and questions for this research and Chapter 2 is a compilation of multiple bodies of related literature that is the foundation for this research. Chapter 3 will discuss the design and development of the technology, Makin’ Math Move, which was created to address the research problem. Chapter 4 will discuss the first study, a gesture elicitation study, which

20 was performed to garner gestural inputs for the system. Chapters 5 and 6 will discuss the studies conducted to evaluate the usability and cultural relevancy of the system. Chapter 7 will discuss the final evaluation of Makin’ Math Move and its motivational and academic impact on students. Chapter 8 will conclude the dissertation with a summary of the results and future work.

21 CHAPTER 2 LITERATURE REVIEW 2.1 Algebraic Misconceptions

Researcher R. M. Welder categorized nine areas of pre-algebraic content that students should be knowledgeable of prior to entering an algebra course, which align with Common Core State Standards (CCSS) for grades six through eight [196, 197]. Her work considered indicators drafted by the Southern Regional Educational Board (SREB), as well as findings from extensive literature. Numerous researchers have concluded that students head into Algebra I courses with a host of misconceptions and fallacies in each of these nine foundational pre-algebraic areas that carry over and lead to failure in learning algebraic content [171]. A misconception occurs when foundational knowledge is ”inconsistent with the concepts being taught”[116]. The following section addresses common misconceptions experienced when learning algebra under the classification proposed by Welder197 [ ]:

1. Number and Numerical Operations Various subsets of misconceptions are included in this area including those related to: comparing and ordering, fractions, decimals and percents, integers (mainly negative integers), exponents and scientific notation, and properties43 [ ]. The generalized error associated with comparing and ordering any type of value centers around the student’s failure to remember specific rules such as finding a common denominator when comparing fractions [43]. Knowing how to compare and order correctly conveys an understanding of inequality, which is important in Algebra. Common errors related to fractions include performing the four basic operations (addition, subtraction, multiplication and division), simplification and the understanding of equality in termsof fractional parts [15]. Ashlock found that students made errors when performing the basic operations with decimals as well. They also had issues when comparing different decimal values; often students assume the more the decimal places, the larger the number value [15]. Some of the misconceptions surrounding the use of negative integers include: confusion when subtracting two negative numbers and/or a positive and a negative number and not associating the negative sign with the coefficient of variables15 [ , 190]. Of the three properties utilized commonly in algebra (associative, commutative and distributive), the first two properties have been reported to give students trouble. While the two properties are only applicable in cases of addition or multiplication, students incorrectly extend their capabilities to expressions requiring subtraction and/or division [194].

2. Ratios and Proportions

22 Students struggle in this area due to the number of ways ratios and proportions can be written and the many different relationships they can possibly represent (part-whole, whole-part, or part-part) [53]. Because ratios can be displayed in fraction format, students often assume that all ratios are indicative of the part-whole relationship [107].

3. Order of Operations When executing operations in order (parentheses, exponents, multiplication or division from left to right, and addition or subtraction from left to right), students are often taught the acronym PEMDAS or ”Please excuse my dear Aunt Sally”. However, this acronym does not specifically state that the last two groupings of operations donot have to be executed in the stated order. For example, in a problem such as 30 / 6 * 4 , students will still multiply first, even though division occurs112 first[ ]. Another common error is that students will apply the operations from left to right regardless of the hierarchy [33].

4. Equality Students often struggle with what equality/the equal sign means in algebra. This is due to its alternate meaning in arithmetic operations before the switch to algebraic thinking [27, 101]. Prior to algebra, the symbol indicates the result or answer to an equation. Students have trouble associating the symbol with balance or the equivalence of two expressions after the switch.

5. Patterning In regards to patterning, students often struggle with being consistent in their methods of forming generalizations [10, 170]. Healy and Hoyles and Radford noticed that students were indeed capable of making correct generalizations; their problems occurred with giving an explanation [81, 148]. Few were able to justify connections and use algebraic symbolism when expressing the pattern; others used informal language or detailed written descriptions that lacked algebraic language.

6. Algebraic Symbolism and Letter Usage This area covers the concept of variables. Students often ignore variables, assume that they are labels or a form of an abbreviated word, or do not understand that they are not fixed unknowns103 [ , 171]. Students’ partial comprehension of variables also affects their ability to solve algebraic expressions correctly [171]. With regards to algebraic expressions, students also struggle to: comprehend that their answer can be something other than a number value (e.g. 4ab + c), simplify expressions, and write them correctly [46, 99].

7. Algebraic Equations Research suggests that students also hold misconceptions regarding equations. Some include reversal order error, miscalculations due to not checking their work, combining terms to solve or simplify equations, inverse operations, symbolic notation and writing equations [15, 146, 176, 177].

8. Functions

23 Understanding slope and proportionality are two common error areas for students when learning functions [189]. The form of functions, such as y=mx +b and their inclusion of multiple variables is also a cause of concern [188].

9. Graphing Students have difficulty with interpreting graphs. They often misconstrue the graphas an image related to the context of the math problem to be solved or mistake the slope for the height [51]. In a study reported in 2008, Scheuermann and van Garderen found that students also have issues when understanding graphing’s purpose, certain graphing notation and the relationships that graphs are used to represent [163]. 2.2 Culturally Relevant Pedagogy

Since the early 1980s, educators have attempted to utilize various pedagogical strategies to relate the mainstream schooling experience with the everyday lives of minority students [104].Researchers have found that the presence of the longstanding achievement gap between Black and White students has been attributed to multiple causes; one being cultural discontinuity or the disconnect between the culture and the schooling environment of students of color [86]. African-Americans have their own cultural traditions, mannerisms, behaviors that are not always reflected in the Eurocentric school system. Because aspects of their culture are not typically integrated in their educational setting, students can often encounter academic disengagement, lack of interest and school discontent, furthering the divide in the achievement gap [87]. In an effort to resolve or improve this cultural discontinuity, researchers over theyears have coined terms to link the two disjointed areas. ”Culturally appropriate” and ”culturally congruent” were used in the early 1980s to improve academic performance in Native Hawaiian and Native American students, respectively [18, 129]. Each team utilized language interaction styles and patterns common to the respective cultures; Mohatt and Erickson also combined the use of Anglo language patterns in their study [105]. ”Culturally compatible” and ”culturally responsive” have also been utilized to explain success in academic achievement in minority cultures [44, 191]. All of the terms with the exception of ”culturally responsive”, suggest that the students’ culture is accommodated into the culture of mainstream society; aspects of their culture are used to help them fit in with current culture without changing its’ structure105 [ ]. However, ”culturally responsiveness”, seems to invoke a more synergistic relationship [105].

24 With culturally responsive instruction, aspects of the mainstream schooling culture are adapted to be more inclusive and aware of minority cultures. This particular ideology helped to shape the pedagogy popularized by Dr. Gloria Ladson-Billings, culturally relevant pedagogy. This practice of teaching rests on the following premises: students must obtain success academically, develop and/or maintain cultural competence and become critically conscious whereas they are able to challenge the status quo socially [105]. Students achieve academically by being taught “through their own filters”; these filters include prior experiences, learning preferences and cultural frames of references69 [ ]. Like the success found in the early 1980’s with children of Hawaiian and Native American descent, culturally relevant/responsive teaching has found academic success with the African-American learner population [11]. One teacher has integrated the use of hip hop and rap music to successfully teach students about poetry, while another used sports, specifically basketball, to investigate math concepts [105, 133]. Gertrude Winston invited parents and relatives in her classroom to create an artist in residence program to address Ladson-Billings ideal of culture competence [105]. The students attended “seminars” given by the residents and did background research about their skill or art. For example, a parent came in to teach how to make sweet potato pies, a staple in the African-American community. This lesson led to research on notable African-American inventor and scientist George Washington Carver as well as the development of plans to market the pies. The use of familiar people and relatable topics was important to reinforce the idea that who the students were and where they came from were of worth and value. [105]. 2.2.1 Culturally Relevant Pedagogy in Math Education

Current standards used to guide instruction, such as those provided by the National Council of Teachers of Mathematics (NCTM), do not suggest how to incorporate cultural relevance into the curriculum; there is also a lack of proper professional development to assist in this incorporation [110]. Ladson-Billings suggests that helping individuals succeed using culturally relevant pedagogy, in particular African-Americans, includes embedding math

25 in everyday situated contexts [106]. She indicates that this may include having students engage in ”unmathlike” activities such as interviews and creating autobiographies. Leonard et al. performed a case study with two fourth grade classes that focused on problem solving during the era of the Underground Railroad. Students read books with multicultural context and crafted a quilt made of student-created patches. The quilts and patches, which held various meanings to the individual students were reflective of their personal lives and culture’s history, were used to learn math concepts such as perimeter and area [110]. Sandra Mason incorporated social change and awareness into her culturally relevant classroom [181]. Students chose to tackle a community problem that affected them, an influx of nearby alcohol stores, from a mathematical standpoint. Students studied local laws, codes, and zoning documentation and formulated a plan that was presented to local government agencies, complete with percentages and other mathematical representations. Their work was picked up by local and national press and resulted in noticeable changes; some stores were closed and many others received citations. Others have utilized cultural aspects, such as video games and music, to reach minority students in the area of math. The African-American Distributed Multiple Learning Styles System (AADMLSS) City Stroll is a 3D environment, depicted in Figure 2-1, which resembles an urban neighborhood, that incorporates rap/hip hop music into algebra lessons [72]. AADMLSS also features an African-American avatar and pedagogical agent that serves as a guide around the neighborhood and a tutor in the practice area, respectively. AADMLSS was evaluated for usability with 23 junior and senior high school minority students; there were 22 African-American students and 1 Native American student. Results showed that overall students enjoyed the City Stroll lesson, found that it captured their attention, ”moderately agreed” that it assisted them in learning algebra and would prefer for their instructors to utilize the application in the classroom. Culturally situated design tools (CSDTs) have also been created to incorporate cultural ties into education. Ron Eglash’s research has led to the creation of multiple CSDTs that utilize artifacts relevant to various minority cultures including African-American, Latino and

26 Figure 2-1. AADMLSS, Adopted From:[72] Figure 2-2. Cornrow Curves, Adopted From: [1]

Native American cultures [61]. Using the Cornrow Curves CSDT, students can practice translational geometry by creating their own braids or mimicking supplied images of the popular hairstyling technique which originated in Africa. Braiding requires geometrical skills such as rotation, scaling and reflection to perfect simple cornrows and are utilized when creating intricate designs. A screenshot of Cornrow Curves is seen in Figure 2-2. There are over 15 CSDTs copyrighted by Eglash and include a Virtual Beam Loom (Native American) to help with Cartesian coordinate plotting as well as one that uses Pre-Columbian Pyramids (Latino) to practice symmetry and pre-algebra. The Virtual Bead Loom CSDT was evaluated by seventh grade pre-algebraic middle school students [117]. Results suggests that the use of the CSDT, when used along with traditional classroom techniques, can increase student performance in the area of coordinate planes and related concepts. The CSDTs were also evaluated for their effect on students’ attitudes towards computers and information technology careers. Eglash and his research team ran two-week workshops in the summers of 2002, 2003, and 2004 for students in grades 8-12 where minority students utilized CSDTs for two hours each day [61]. Students completed a workforce survey and the Bath County Computer Attitudes scale at the end of the workshops. When compared to a baseline of 175 surveyed low income 8th grade students, the 24 participants in the workshops scored significantly higher.

27 Employing culturally relevant mathematics pedagogy correctly can potentially create students with an encouraging mathematical identity and high confidence in their ability to learn math while increasing their motivation to continue to gain knowledge on the subject [121]. 2.2.2 Nine Interrelated Dimensions of African-American Culture

Culture is related to a group’s ”deep structure of knowing, understanding, acting, and being in the world” and is defined by shared experiences, skills, behaviors and social institutions [106, 185]. Researcher and educator A. Wade Boykin defined nine interrelated dimensions essential to African-American culture which are described below [36, 37].

• Spirituality Spirituality is defined as ”an approach to life as being essentially vitalistic rather than mechanistic [37].” This aspect relates to the belief that there are non-tangible factors or powers that can impact one’s life. Spirituality is regarded highly in the African-American community and affects most if not all domains of daily life including health, music, community, and politics [135].

• Orality This element is in regards to the African-American culture’s inclination to value the art of communication. Oral communication is highly regarded and nurtured and is still observed through spoken word, poetry, and rap music.

• Harmony The idea of harmony is the belief that one’s life and destiny is connected to nature; this also ties into the ability to read people, non-verbal cues and the environment well [67].

• Expressive Individualism Expressive individualism is the notion that having one’s own distinctive personality is not only accepted, but encouraged. It is appropriate for individuals within the culture to ”march to the beat of their own drum”; this could be in the form of dress or hairstyle [67].

• Affect African-American culture stresses the importance of emotions and expressions. Individuals can sense emotional cues and tend to have strong emotional responses [67].

• Social Time Perspective The African-American culture views time in terms of sociality or events, rather than its physical construct. More value is placed on the actual event(s) than the time attached to it; thus it easy for time to slip by or run over when the engagement in the event is very high [67].

28 • Verve Verve is the preference for energetic and invigorating actions and stimulating environments. The Black home is typically full of lots of people with loud levels of noise, which is contradictory to traditional White middle-class educational settings. Researchers have noted that this may have an effect on the underperformance of African-Americans35 [ ].

• Communalism Communalism is the belief that the community is greater than the individual; the African-American culture places high value on social connections and responsibilities. The communalistic characteristic supports the notion of extended families and success of African-Americans in cooperative learning environments [39, 174].

• Movement Movement is described as the ”interweaving of movement, rhythm, percussiveness, music and dance, which are taken as central to psychological health.” African-Americans are recognized across the world for their gifts in the performing arts; this does not just include the varying forms of dance, but also the African tradition of stepping [67]. 2.3 Embodied Cognition

Embodied cognition is a theory based on the notion that the way our mind functions is tied to how our body interacts with the world around us [199]. Gallese and Lafkoff stated that ”the sensory-motor system not only provides structure to conceptual content, but also characterises the semantic content of concepts in terms of the way that we function with our bodies in the world [68].” Wilson evaluated six main claims that are the supporting statements for this theory. Those claims include that 1) cognition is situated or takes places in the real world, 2) cognition occurs under the pressure of real time interactions, 3) we use the environment in which we are interacting with to reduce cognitive load, 4) the same environment is a member of the cognitive system, 5) cognitive processes are needed to perform and guide actions and 6) cognitive processes are grounded in sensory and motor functions [199]. Embodied cognition has been believed to be essential in cognitively connecting mental or abstract thoughts to physical settings, which can aid in the thinking and learning processes [70, 74]. Different literature either combines all or utilizes some of the aforementioned claims, but tend to agree that cognitive and motor-sensory functions are related, which fuels the proponents of embodied and kinesthetic learning. These types of learning cover a range of

29 movement related learning activities which includes letter tracing, gestures, physical object manipulation and even full body interactive learning environments (FUBILEs). 2.3.1 Gestures in Education

Gestures are often cited as proof that ”the body is involved in thinking and speaking about the ideas expressed in those gestures [12].” It further strengthens the belief that cognition and knowledge is embodied. In particular with mathematical learning, teachers and students often use gestures when teaching and discussing, respectively, mathematical knowledge [12, 76]. Gestures have been utilized often in education and have been found to have increase memory and recall, enhance knowledge development and change, and support comprehension when applied to learning experiences [75, 77, 149, 153, 182]. While many researchers agree that gestures influences learning, some have investigated the “how.” Goldin Meadow et. al conducted research to investigate whether or not movement, in the form of gestures, was the cause for increased understanding or if the gestures had to be meaningful in order to produce a significant result75 [ ]. 128 third and fourth grade students were split into three different groups with varying levels of gesture usage with accompanying pre-lesson instructions. Each of the lessons showed students a problem similar to: 2 + 3 + 5 = + 5 . The phrase, ”I want to make one side equal to the other side” was recited in each condition and gestures taught to each group is shown in the table below.

Table 2-1. Gesture Conditions Gesture Condition Gesture No Gesture None Correct Gesture Point with V-hand to the “2 +3” and index point to blank Partially Correct Gesture Point with V-hand to the “3+5” and index point to blank

Students completed this two more times with similar problems. For the main lesson, after a researcher verbalized how to solve a similar problem, students were asked to do the same thing six different times. In the post-test, students were asked to solve six more problems and explain how to solve the problems; gestures and verbiage used was noted and coded. Results indicated that students who were in the correct gesture group solved more math problems

30 accurately than the partially correct gesture group; the latter performed more accurately than the no gesture group. This suggests that the use of gestures, whether correct or incorrect, assisted students beyond the mere fact of focusing their attention; the gesture based grouping strategy made the math lesson more effective. 2.3.2 Gesture Based Educational Technologies

Technology has allowed for embodied learning to go beyond traditionally gesturing with paper and objects to interacting with various digital virtual learning environments (VLE) . VLEs, which can be loosely defined as an interactive 3D computer generated environment that provides learners with an immersive experience, can be sub-categorized into various sectors which, in this context , all involve some level of body movement to engage in the learning process [144]. Mixed reality (MR) is one sector of technologies that allow the “merging of real and virtual worlds [128].“ Mixed reality technologies allow students to be immersed into a learning environment, which simulates the real world, via the use of some physical instrument. This type of technology has been applied in various educational purposes including improving storytelling abilities, chemistry, and biology [90, 144]. Virtual reality (VR) , while similar to MR, often does not require the use of any physical input, but instead uses ”various display and interface devices” to produce the immersive effect144 [ ]. A user, usually via the use of some type of VR headset, is fully immersed into the 3D environment and can interact with the constructed environment freely using their body parts. Environments that utilize full body interaction without the head mounted display specifically linked to VR and do not require any physical objects for manipulation can be classified as FUBILEs or full body interactive learning environments, for the context of this paper. While MR and VR environments can involve full body movements, each of those two sectors have the distinguishing qualities of additional physical input and the use of a headset, to separate them from this category. FUBILEs tend to rely on the use of sensors, including motion, visual and pressure sensors to allow the user to interact with the environment. Malinverni and Pares conducted a review on FUBILEs utilized in the educating of abstract concepts in 2014. This review found over 30 papers

31 dedicated to environments used across various educational contexts, including music, science, math and physics of both child and adult users [118]. Most papers (70%) were based on the embodied cognition theory; other approaches used in FUBILE design include constructivist and constructionist frameworks and the research that supports the positive impact of physical activity on various cognitive processes including memory and academic performance. As far as design methods are concerned, only three of the thirty papers utilized teachers and students in the design. The review reported on each technology’s mapping or relationship between ”the controls and their movements and results in the world [137].” The mapping approach definitions are similar in description to the gesture type taxonomies that will be described in more detailed in Section 2.4.1.2; the three approaches used to categorize the technologies were functional, identity and metaphorical. In the functional approach, user actions are not directly related to the context of the material, but are incorporated due to its common usability (i.e. drag and drop functionality). The identity approach features close knit mapping between gestural actions and the context. Embodied technologies in the math sector employ such an approach when using arms to depict angle measurements and graphing of variables such as time and distance [13, 114]. The metaphorical approach was used also because not all concepts can be expressed as literally as in the identity approach; these actions instead symbolize something else. For example, the SoundMaker interactive environment uses a motion tracking system to alter audio output based on metaphorical actions [14]. The following embodied metaphor mappings were employed by the system:

Table 2-2. Embodied Mapping Metaphors Movement Parameter Mappings Speed Tempo Fast is fast, Slow is slow Activity Volume More is loud, Less is quiet Proximity Pitch Near is high, Far is Low Note: Taken from [14]

32 2.4 Interface Design

2.4.1 Natural User Interface Design

A natural user interface (NUI) refers to a piece of technology designed to make the user’s interaction with the device seem as ”natural” as possible; they are designed to ”reduce barriers of computing” and to ”increase the power of the user” [198]. NUIs should feel like an extension of a user’s body and comfortable and natural for all levels of expertise. NUIs have the tendency to reduce interface learning costs due to their intuitive nature [198]. There are several different types of NUIs including those that are touch-based, gestural-based and those controlled via voice recognition [131]. Touch screen NUIs have transitioned from featuring resistive panels made of flexible plastic to now more likely containing capacitive panels made of rigid glass. The latter are commonly found in today’s smartphones and tablets. Touch based interfaces encompass single, multi-touch and surface interfaces [80]. Voice user interfaces use complex voice recognition algorithms to understand various languages and even accents [131]. These interfaces tend to require a lot of processing power to support the complicated algorithms. Gestural based interfaces have become more popular with the increasing use of sensors such as accelerometers and gyrometers [158]. These interfaces have been incorporated into several applications including video games, television control and assistive devices for the elderly [30]. There are several different types of gestures that can be used to control these interfaces such as full body, foot and mid-air gestures [80]. Because there are various types of NUIs, there has been a limited number of guidelines proposed to address design considerations for them all inclusively. Sean Murphy of Texas Instruments composed four design considerations for all NUIs [131].

1. Cost and Size Parameters: Cost and size constraints can often influence and dictate which type of NUI can be implemented for a given application. A large, stationary NUI can possibly have a completely different interface from a pocket-sized, mobile device that is cheaper to produce.

2. Connectivity and Power: Whether an NUI requires wired or wireless connectivity can also affect its design. The power required for the NUI to operate affects not only thecost,

33 but also the design choices. Large, powered stationary machines have more flexibility in implementation options.

3. Processing and Power Issues: Applications with higher resolutions and more complex graphical content (3D vs 2D) require more processing power. NUI designers must consider if the processor for the NUI selected will have the power to handle the graphic acceleration needed to display ”computationally intense” 3D graphics or robust visual and/or vocal recognition and tracking algorithms.

4. Operating Systems (OSs): In the context of Murphy’s document, there are two types of OSs: High Level (HLOSs) and Real Time (RTOSs). High level OSs, such as Windows or Linux, are intuitive and can provide tools to incorporate multimedia and complex functionality. The downfall to HLOSs is that they often require a lot of code to implement such functionality which can affect responsiveness of the NUI. RTOSs, like Nucleus and Neutrino, are created to ensure quick responsiveness and involve less code. However, development may be more time intensive when implementing multimedia features. Joshua Blake also details four guidelines for NUI design which is applicable for all NUIs [31].

1. Instant Expertise: Blake recommends reusing common human skills and reusing domain specific skills when designing NUI interactions. These two considerations allow forusers to become experts of the NUI application at a faster rate.

2. Cognitive Load: In an effort to minimize a user’s load, the previous guideline is needed. It is favorable to utilize basic human skills over specific skills as the former are more natural and easier to learn.

3. Progressive Learning: Blake suggests creating a NUI that starts with interactions that require basic skills that then progress to the use of more complex skills. In order to not hinder the experience of more expert users, interactions and tasks should be composed of basic tasks (for novices) that can be combined to be completed by more experienced users.

4. Direct Interaction: This guideline suggests that interactions should be direct, high frequency and contextualized. Directness refers to the proximity and action in which the user interacts with the NUI. There are two types of proximity: spatial (the user’s actions are physically close to the NUI’s element) and temporal (the NUI responds immediately). NUIs comprised of high frequency interaction are more engaging and easier to perform. The context of each interaction consists of the user’s action, the action’s nearness to NUI’s element and prior interactions. It is important to only display content pertinent to the context of the current interaction to prevent an overload of choices.

34 Wigdor and Wixon in Brave NUI World indicate guidelines necessary to adhere to when designing touch based and gestural NUIs. The following are guidelines reflecting the design ethos of those specific NUIs. The book also offer additional guidelines for understanding technological artifacts in regards to new technologies, creating an interaction language, solely touch-specific devices and the design process.

1. Ecological Niche/Less is More: The authors suggest that designers do not simply translate or transfer one application medium to another; Ex. Don’t copy and paste the code of a web application to an NUI. Instead, consider the context and possibilities that the NUI can offer to enhance the application. Also, begin with designing the most basic interactions first; polish those before designing the complex ones.

2. Contextual Environments: NUIs differ from traditional graphical user interfaces (GUIs) in what environment they are typically designed for. GUIs are made to be used more in isolation, whereas NUIs are found in more public and social domains. When designing NUIs, it is suggested to pick an interface that is most appropriate for the context it will be utilized in. Begin designing by considering what actions will be utilized in this context and then, shift the focus to the content. Minimize interface elements needed for interaction so that the NUI is also focused on the content. Finally, design interactions that require intuitive actions that can be easily learned.

3. The Spatial NUI: For 2D NUIs, it is suggested that an environment’s layout, feedback and behaviors be enhanced for touch and gestures. The authors suggest that all responsive elements be at least: 15 mm in size (all around) and have 5 mm of space between each element. For 3D NUIs, it is recommended to use the element of depth as visual feedback and to use the standard notion of backward and forward movement on the z axis.

4. The Social NUI: Social NUIs are those designed to support the interactions of multiple users, thus the designs should be tested by having multiple users operate the interface at the same time. This type of NUI can require the implementation of different levels of tasks and must be supportive of each kind. Highly coupled tasks require users to work together on the same task, at the same time. Lightly coupled tasks requires multiple users as well to reach an objective while working on distinct individual assignments. Uncoupled tasks engage users in separate tasks, but they need to use the same space.

5. Seamlessness: A NUI should be able to ”erase the line between the physical and virtual world” with a seamless experience. In order to do so, it must able to provide immediate and accurate responses for every interaction with a smooth transition. There should be no hard and abrupt animations.

6. Super Real: Super realism extends experiences beyond their natural possibilities. This can be implemented within NUIs by utilizing basic direct manipulations of content such as

35 a single finger drag and flick. The use of time based gestures hinders the naturalnessof NUIs by interrupting a smooth experience.

7. Scaffolding: Actions designed to manipulate and control the NUIs should be scaffolded in a way that the user should be able to predict how the system should respond. In the case where there is unseen information that is available to the user, affordances need to be incorporated to guide the user on how to access such information. For example, in a tv guide NUI that offers a list of TV shows that extends beyond the listing that that is visually shown, designers should partially display the next listing so that the user knows there are more options available. Visual feedback should be incorporated when foreshadowing the system’s state change so that users can change or reverse their actions if it causes an unanticipated result. For irreversible and harmful functions, additional confirmatory input from the user should be required.

8. User Differentiation: Unless necessary, leave the identity of the user as anonymous. If identification is necessary, assure the user that it is beneficial and that theywillnot endure any negative repercussions. 2.4.1.1 Designing of Gesture Sets

The set of gestures utilized within a system is determined via either gesture elicitation by a set of end users or are decided by the user on a case-by-case basis [80, 201]. Gesture elicitation, a form of participatory design, is conducted in a variety of ways. Wobbrock et al. used a referent/command based elicitation to gather gestures to be utilized on an interactive tabletop device [201]. 20 adults were presented 27 referents (i.e. Rotate) via a recorded voice describing the action as well as an animation showing an object with the action being performed on it. After the animation was complete, the object was returned to its original state so that the participants could generate a gesture that would cause the observed effect. Others have used the Wizard of Oz approach in which an unseen ”wizard” manipulates content and is behind the scenes controlling the prototyped version of the device. [52]. Connell et al. utilized this approach when conducting a guessability study with children and gestures for Kinect-based applications. Their team featured a ”wizard”, a researcher who captured video data, and one researcher who interacted with the children. The children were given 22 tasks, such as ”move object along path”, via a television screen, and asked to complete them using any part of their body. While elicitations are commonly conducted for a specific system, it is common for gesture sets defined for one type of gestural interface (e.g. interactive tabletop) to

36 be agreed upon by users of another device (e.g. a smartphone) [158]. Ruiz suggests performing guessability studies, such as one conducted by Connell et al. before selecting a gesture set for an interface; these studies simply gives tasks to users and asks for what gesture would execute the identified task. 2.4.1.2 Gesture Classification and Taxonomies

There has been no universal classification for gesture sets, but taxonomies have been developed for varying contextual applications including those for surface tabletops, human-robot interaction, pen/stylus input devices, and other purposes. Nielsen suggests that there are three types of labels that can be given to gestures: semantic, functional and descriptive [136]. These three labels are used to describe what the gesture communicates, what it does in regards to the interface and how it is performed, respectively. Wobbrock et al. developed a taxonomy for surface computing elicitation study mentioned in the previous subsection where researchers classified each gesture across four dimensions: form, nature, binding and flow [201]. Form describes the pose of the hand or finger and nature characterizes the gesture as symbolic, physical, metaphorical or abstract. Binding takes into account the gesture with respect to other objects, while flow defines the relationship between the gesture and the response. This taxonomy was developed specifically for surface interfaces, so it was limited to finger and hand input. Obaid et al. adapted their taxonomy from Wobbrock et al. and Salem et al. for full body gestures to control a humanoid robot [138, 160]. Their categories included form (static or dynamic), body parts (hand(s) or full body), view point (based on user or robot), and nature (deictic, iconic or miming). There currently exists no gestural taxonomies for full body educational technologies or any that take into account cultural gestures in regards to the nature of the gesture. The gesture elicitation conducted in this research resulted in a taxonomy that took into account both of these missing factors. 2.4.2 Designing for Culture

The term ”culture” has been given various definitions and has a broad range of meanings. Culture in this section will be defined in terms of a group’s shared knowledge, behaviors,

37 ways of thinking, values and customs [34, 45, 82, 173]. Culture can be inclusive of race and ethnicity, but can also be used on a more concentrated level such as the culture of an organization or business. Cultural researcher Geert Hofstede developed five cultural dimensions, power distance, individualism vs. collectivism, masculinity vs. femininity, uncertainty avoidance and long term time orientation, that he suggested world cultures differ in82 [ ]. Power distance (PD) refers to the extent of which a culture’s people accept uneven distributions of power, with low power corresponding to cultures where low level and high level positions are familiar and focus less on the differences in hierarchy. Individualist cultures praise accomplishments of the single person, while collectivist cultures emphasize and favor group efforts. Masculine societies champion separation of traditional gender roles, while feminine cultures encourage the merging of roles. Uncertainty avoidance (UA) is the extent in which a society is uncomfortable with uncertainty and aim to diminish it; societies with high UA often stress punctuality and formality, while low UA cultures are more risky. Long term time orientation cultures emphasize patience and other Confucian practices, while those with short term time orientation are focused on immediate results. Aaron Marcus mapped these cultural dimensions to the following five design components: metaphors, mental models (MM), navigation, interaction and appearance [120]. Table 2-3 shows a portion of the mapping of the design components to higher end of power distance, uncertainty avoidance, and long term time orientation. It also maps to the individualist and masculine positions of their respective dimensions. The lower ends, collectivist and feminist positions are opposite of the characteristics reflected in the table. It is suggested that the mapping of dimensions to interface elements can improve user experience, lead to the development of user interface design patterns and assist in creating adjustable interfaces for related cultures [26, 120, 184]. When designing an interface intended to be utilized in many cultures across various nations, the internationalization-localization process is used [45]. Internationalization strips an interface of any culturally relevant elements so that they can be adapted for any culture [19]. Localization is the process of tailoring the UI

38 Table 2-3. Section of Mapping between Cultural Dimensions and Design Components Metaphors MM Navigation Interaction Appearance Power Govt or organized and restrictions; Error images of Distance institutional categorized; authentications; messages, leaders or (Hi) buildings; charts; ref. passwords; wizards or govt themes; objects with data guides lead slogans hierarchies usage Power Informal Simple, Open access, Supportive Images of Distance or popular informal options, error people, (Lo) buildings; organization; shareable messages, groups, objects with data with paths cue cards popular size equality; relevancy things, colors, games/play; informality Note: Taken from [120] for the particular culture [45]. When creating culture-specific interfaces, designers need tobe cautious when selecting graphical elements, such as icons (and their metaphors) and colors, as well as the structure and format of their information [45]. Callahan suggests that designers keep in mind the following question when choosing icons: Will it be understood, appropriate and acceptable? Often, metaphors translate differently across cultures and carry various connotations that need to be taken into consideration [45]. Deciding on the use of colors warrant the same attention; different hues can have different meanings within certain cultures and context is often vital to interpretation. Likewise, formats, such as those used in dates and time, vary from society to society, so using the appropriate one is essential. Organizing the content relies on knowing which metaphors work well within the culture. For example, the use of a desktop may not be appropriate for all cultures who are not familiar with the concept. Including users of the specific culture is necessary in the design process to avoid the misuseor wrong connotation of elements related to interface design [45]. 2.4.3 Designing for Children

School-aged children are being introduced to and becoming accustomed to technology at early ages due to its prevalence in society. They are considered ”digital natives”, a term used to specify the generation that has some form of technology integrated into a majority of all

39 areas in their lives [147]. Children have different needs (cognitive and physical), interests and abilities that need to be addressed when using technology specifically designed for them84 [ ]. Hourcade provides generalized principles for all types of technology geared toward children in the areas of visual design, interaction styles and pointing devices [84]. In regards to visual design, icons should be created to recognizably represent specific actions or objects. They should be large enough to be clicked by the targeted age demographic and be distinguishable as interactive elements. Depending on the demographic, the text should vary; ideally, the amount of text would be minimized to accommodate a wide span of reading levels. Visually, children should only be able to access a few actions or interactions at a time. Interfaces should utilize a multi-layer system where users are exposed to more and more actions as they become proficient users. Hourcade supports Shneiderman’s idea of incorporating direct manipulation that is fast, reversible and supports incremental action [167]. Quick feedback and reversibility holds children’s attention and encourages them to explore the interface. Incremental actions, such as those mentioned previously in a multi-layered system, allows for simpler instructions to be understood by all ages. Limiting the amount of text based interactions can also be beneficial when designing interfaces for large age ranges; visual interactions can reduce the frustrations felt by those who are unable to or not well versed in reading. Immediately visible and simple menus should also be used when designing for kids; they should avoid hierarchies as all children may not be familiar with the structures. With regards to pointing, he suggests using age appropriate devices and creating large enough pointing targets for the specified age group. There are conflicting results as to whether dragging and dropping is a more effective method than clicking and moving for certain age groups. Large and Beheshti are among several researchers who have investigated the design of web portals specifically for children [60, 66, 109, 161]. Of the 13 guidelines proposed by the two researchers, 11 can be utilized beyond the narrow scope of web portals and applied to various child-specific interfaces. The two guidelines that are limited to web portals and

40 information retrieval systems are related to retrieval capabilities and the presentation of results. The 11 guidelines were grouped into the following categories: portal objectives, metaphors, visual design, icons, portal name, characterization, terminology, advertisements, online help, personalization and interactivity. The guidelines regarding the name and objectives of the portal can be applicable for any interface; the name should be fun, appropriate, meaningful and allude to the purpose of the application. The objectives should be targeted to its intended focus; do not include too many elements that do not align with the purpose. Available help features should also be very context focused with specific solutions. Visually, the design layout should be clear with brightly colored attention grabbing elements, which is typically disliked by older users. Graphics, animations and characters are acceptable, but should be used appropriately; the use of characters should be uniform throughout the application. Users should have the ability to personalize certain features of the interface, such as the layout, characters or icons, because one interface design cannot cater to all preferences. Children interpret icons for exactly what they are, so labels should be included to avoid confusion. Any metaphors and terminology implemented in the system should be relatable to the users’ age and culture. Large and Beheshti were not definitive on the use of social interactivity within portals; although features such as chatting can be useful in the swapping of information, they can also be an easy means of distraction. Designers should carefully consider using such features. The final guidelines address the demographics of the targeted users. If the application is designed to be open to a wide range of cultural users, designers should allow the entire system to be available in multiple languages. Researchers encourage utilizing children as they are designing various interfaces intended for them. There are several design theories including user centered design, contextual design/inquiry, participatory design, cooperative inquiry, informant design, and learner centered design, that can be used to include kids at varying phases of design [60, 109, 134, 161]. The first two approaches limit the role of children to providing feedback on designs already created by the adult designers [134]. They have more equal roles as designers in the latter four

41 approaches. In these methods, children are actively creating possible design solutions that are iterated upon until a final design choice has been reach. They also are involved in the testing and evaluation of the final design. 2.5 Digital Game Based Learning and Gamification

Digital game based learning (DGBL) products are broadly defined as a result of the intersection of pedagogical content and entertaining digital gaming [41, 147]. The vagueness in the definition has led to debates on its comprising subareas. The subareas included under this definition for this context include both serious learning games and edutainment games. Serious games are those designed specifically for educational intent; these games are inclusive of those used for military and workforce training and commercial off the shelf (COTS) games. [9, 175]. Abt states that the educational component does not have to be rooted in the game’s design, but can instead be found in the context in which the game is used [9]. For example, in the instance that Angry Birds is used to introduce physics concepts, this traditional COTS game can be deemed a serious game. This branch of gaming routinely necessitates higher levels of thinking and problem solving skills; conversely, edutainment games do not [58]. Edutainment games, such as the popular 1990s game Where in the World is Carmen Sandiego?, are purposely designed to have prominent entertaining features alongside their educational objectives [150]. This section of games is often criticized for numerous reasons; one being how they differ from serious games in terms of some of the elements of Marc Prensky’s widely regarded definition of a game. Prensky defines a game as a form of play that contains six elements: rules, goals or objectives, outcomes and feedback, challenge or competition, interaction, and a narrative/story line [147]. Edutainment games often feature fixed rules which limit the flexibility in terms of skill sets practiced as well as the amount of exploration and testing of hypotheses that can be done [50]. Challenges also often vary between the two subareas. In edutainment, the educational challenges are distinct from the fun; well-designed serious games can seamlessly blend the two [50]. Edutainment games are often software equipped with gamification features, such as scoring and appealing graphics, but lack ingame

42 elements such as a rich storyline [50, 141]. Another source of criticism derives from the fact that these games focus on the use of a smaller scope of skills and learning techniques, such as rote memorization [50, 100, 175]. Regardless of such critiques, the use of such games has continued to be used across multiple fields of education. Gamification is another area that has recently been placed on the DGBL spectrum. The term gamification was initially introduced in the area of digital media in 2008 and hasbeen applied to a host of different areas including business and education. It has been defined asthe ”use of game design elements in non-game contexts”= [55].” It has been used interchangeably with terms such as ”productivity games”, ”funware”, ”playful design” [65, 124, 178]. Despite the term not appearing frequently until 2008, it has its ties to the field of HCI under different terminology. During the early wave of computer gaming in the 1980s, Thomas Malone utilized video games aesthetics and developed ”heuristics for designing enjoyable user interfaces [119].” Similarly, early text-based video games, such as ”Adventure” led to suggestions on how to make work related activities more fun and usable via ”metaphoric cover stories [47, 48]. ” In the 2000s, HCI researchers used gaming to begin discover ways to measure game experience and the heuristic of playability [162]. While ”play” and ”gaming” are related, play is used to describe more expressive and free interaction, while gaming is more align with the structured rules of interaction as it relates to gaming [55]. The desire to ”gamify” contexts comes from popularity and success of video games as motivating forces. Video games can engage and motivate users with ”unparalleled intensity and duration”; it is the belief that gamification can have the same effect in other contexts, such as the workforce and education203 [ ]. As noted above, some believe that edutainment is in fact just gamification; many more define a stark difference between serious games and gamification. The main difference is that the resulting product is not a full fledged game165 [ ]. Another difference is the intent of the two different approaches when used in the education space. Serious games are specifically designed to teach or cause learning to take place without the presence of an instructor; Their main goals are not to purposefully influence motivation and engagement108 [ ]. However, [108] theorizes that

43 those are in fact the primary goals and intent of gamification; to ”alter a contextual learner behavior or attitude, and which is intended to improve pre-existing instruction as a consequence of that behavioral or attitudinal change.” Thus, gamification is meant to enhance or improve instruction, but not to become a substitution for it. There are various game design elements that have been incorporated into educational contexts in order to ”gamify” them. Table 2-4 features some frequently utilized educational gamification design principles as compiled56 by[ ]. The table features the principles, game mechanics (cells with blanks are self-explanatory), as well the intrinsic , social and extrinsic motivation theories related to the featured elements as modeled by [152]. Table 2-5 gives a brief synopsis of each of the featured motivation theories and which category of motivation they fall under.

Table 2-4. Educational Design Principles, Adopted from [56] Design Principles Game Mechanics Motivation Theories Goals Goal Setting, Need Achievement Challenges and Quests Missions Hierarchy of Needs, Goal Setting, Expectancy Value Customization Avatar, Difficulty Level Self-Efficacy, Personal Investment, Expectancy Value Progress Points, Progress bars, virtual Self Efficacy, Need Achievement goods Feedback Self-Efficacy, Personal Investment, Goal Setting Competition and Badges, Leaderboards Social Comparison, Personal Investment, Cooperation Self-Efficacy Visible Status Points, Badges, same as above Leaderboards Access/Unlocking Bonus Levels, Rewards Content Freedom of Choice Choose Your Own Adventure Expectancy Value, Personal Investment Freedom to Fail Multiple Lives Onboarding Walkthrough, Tutorials Hierarchy of Needs Time Restriction Countdown Clock

44 Table 2-5. Types of Motivation and Corresponding Theories, Adopted from [152] Type of Motivation Theory Synopsis Intrinsic Goal Setting Specific, immediate and context-appropriate goals increase motivation to achieve [113] Need Achievement Based on the desire to demonstrate high achievement ability to oneself and/or others [17] Hierarchy of Needs Users need to satisfy physiological needs before complex needs, such as belongingness. Users first need to understand game rules and eventually move to a need for aesthetic and self-actualization [168] Self-Efficacy Perceived self-efficacy affects the selection of activities, difficulty levels, energy used, persistence and performance. The level of perception is influenced by past performance, social influences, persuasion and observation of others [24]. Social Social Comparison Humans assess their capabilities, reactions and beliefs by comparing themselves with others in similar contexts [71]. This is the explanation for competitiveness in games and challenges Personal Investment The amount of effort, participation and time expended on an activity is dependent on cognitive elements such as beliefs, feelings and goals [79]. Incentives are employed in games to increase these factors. Extrinsic Expectancy Value Theory The belief that goal oriented behavior is a product of the idea that effort leads to performance that will ultimately result in reaching a goal. The amount of effort is determined by the value of said goal [166].

Gamification allows the incorporation of various elements at varying degrees of implementation levels and the vagueness of its definition has drawn different critiques. One staunch critique is that gamification is an easy cop out to creating a ”gaming experience32 [ ].” It allows

45 developers to take away from the entirety of the gaming experience by picking and pulling the ”least essential game element and making it the focus[155].” ”Pointisfication” or solely utilizing points or badges as a means of gamifying may be the rationale for such criticism. Critics have gone as far as to suggesting alternative terms for the use of gamification; one being explotationware, due to the ”villainous reign of abuse” that gamification results in [125]. Despite the criticism, the educational realm, as well as other contexts, have garnered mixed results after implementing gamification in the classroom. [54] explored how a gamified online learning system, via the use of badges, would improve participation. The use of badges increased the number of answers contributed by students (without impacting the quality of responses) and the length of engagement with the system. However, results showed that students who did not utilize the badge system contributed four times the amount of responses needed, which inferred that the system itself was motivating sans the gamification. [57] used gamification for the Blackboard online course management tool in the form of challenges, trophies, badges and leaderboards. 44% of those in the experimental group used the features aforementioned and received higher overall scores and better performance on the practical assignments. Alternatively, the class participation and performance on written assignments decreased. [111] created a gamified tutorial system for novice AutoCAD users which featured missions, scoring, time pressure and rewards, among other elements. The study saw an increase in engagement, performance, enjoyment and a significant improvement in the completion rate of four tasks. 2.5.1 Math DGBL

The National Council of Teachers of Mathematics (NCTM) encourages the use of technology in math education [139]. Technology has progressed from basic four function calculators to computer assisted instruction (CAI) to current math games. CAI, an immediate precursor to current educational math games, operated as a computerized version of paper-and-pencil drill and practice problems in the 1960s and 1970s [126]. The CAI systems, which featured immediate feedback, were solely dedicated to teaching the material; they

46 would later evolve to incorporate graphics and games. [95, 102, 122]. The games were initially utilized as a reward structure, but eventually emerged as separate standalone entities that lacked in most areas of Prensky’s current game definition195 [ ]. Although they did not have engaging narratives or intrinsic integration of content and entertainment, the early games did find success in improving low-level math proficiency62 [ , 102, 122, 195]. Modern math learning games have continued to produce academic success across most K-12 levels of math. There are numerous websites dedicated to a host of quick play/arcade style games, such as the Arcademics collection games [2]. These sites have individual games that focus on varied narrowed topics, such as the addition of two digit numbers or single multiplication tables. Math Blaster is a standalone, widely renowned math edutainment series that covers a wide range of math topics. It began as a spaced theme arcade style game that required the player to solve basic math problems in the drill and practice format while trying to save their companion, Spot, by shooting flying debris132 [ ]. Since its creation in the early 1980s, it has spun off into several editions for all levels of K-12 math, including algebra and geometry [151]. It has also morphed into an adventure gaming format with a rich, appealing storyline to captivate students [88]. The series has garnered the same critiques as the edutainment branch of games; perhaps even more due to its outstanding selling performance. Math Blaster, due to its drill and practice nature restricts the learner to the lower branches of learning, knowledge and comprehension, in Bloom’s Taxonomy [151]. Additionally, Math Blaster, like many other edutainment games, incorporates exogenous fantasy into their gameplay. The mathematical skills practiced within the individual games in the software are typically not linked to the game’s mission [29, 132]. Despite these critiques, the series has been credited by educators as being very successful in the math classroom. Research is limited on studies measuring the series’ effectiveness, with the exception of its positive effecton students with varying learning disabilities [20, 123]. Regardless of the lack of empirical data, teachers praise games like Math Blaster because of their ability to engage and the ease of implementation into the curriculum [151, 169].

47 Dimension M is another popular math based game series. While it is still categorized in the DGBL sector, it falls outside of the edutainment realm. The Dimension M games feature single and multiplayer strategy, problem solving games for both pre-algebra and algebra students. The games cover topics such as probability and data analysis and utilize a higher level of thinking as compared to games more akin to Math Blaster [4]. The games also are aligned with Common Core State Standards (CCSS), which makes it easy to align with classroom lesson plans. Like Math Blaster, these games also over emphasize gameplay and design and do not tie the math learning objectives to the game goals. This design choice can cause students to focus more heavily on winning than practicing the desired concepts. Several studies were conducted on the series and garnered mixed results. Rizhaupt et al. noticed positive effects on self efficacy and math attitudes, but found no significant effecton math achievement [154]. However, three studies conducted on the games did report positive effects in math achievement/performance and additional measures including, self efficacyand sustained motivation [21, 73, 96]. Despite the availability and advantages of so many math educational games, educators still face some barriers with classroom implementation, such as lack of time, knowledge and confidence [192]. Some teachers feel extreme pressure to focus heavily on the curriculum so that benchmark exams can be passed successfully; this limits the amount of time they feel they have to incorporate gaming and other technologies into their classrooms. Other barriers include traditional schooling mindsets, lack of resources and concerns about appropriate and focused use [59, 83, 92, 157]. 2.5.2 Gesture Based DGBL

While the aforementioned games and most other math learning games are played on computers or mobile devices with Internet access, there are other platforms that are capable of gameplay for educational purposes. Platforms that feature NUIs, in particular gestural interfaces, have begun to be utilized in the educational space. Games with such an interface are also called exergames and use physical movements as their input modality [145]. The

48 (a) (b)

(c)

Figure 2-3. Math DGBL: a) Mathmazing, b) MeMa Pads and c) Jumpido, Adopted from[94, 142, 159]

intertwining of movement with learning can affect the body physically, cognitively and behaviorally; these effects have been mentioned in previously in the section on embodied cognition. [42, 78]. Because of the additional benefits outside of the overt physical ones, exergames have expanded from just being used in physical education to other school subjects such as math. There are few math educational exergames that have been developed and/or deployed. MathMazing, shown in Figure 2-3a, is a game developed using the Kinect system that teaches basic math operations using 3D graphics [159]. Students are presented with problems and must navigate an in-game character to one of three answer choices marked by flags. The student’s gestures controls their character’s movements. MeMa Pads was designed to practice math facts using a structure akin to Dance Dance Revolution [94]. The game uses multicolored pads, as seen in Figure 2-3b, which are embedded with pressure sensors that correspond to various values on a connected display. The system offers a customized experience by allowing for the difficulty of the randomized facts shown to be adjusted. The aforementioned games have not been deployed at the time of this dissertation. Jumpido is

49 a set of math exergames that is currently available commercially [142]. Like MathMazing, it uses the Kinect to offer 60 engaging movement based games across 6 different activities for students ages 6 to 12. The games offer single player modes, such as the one the student is seen playing in Figure 2-3c, as well as opportunities for collaboration and competition. Jumpido covers 20 math concepts and is in alignment with CCSS, NCTM and state standards. 2.6 What’s Missing: Significance of Current Research

There is an identified lack of proficiency for students in the areas of algebra, whichhas been link to their weak foundation in pre-algebra. African-American students, in particular, are performing worse than their counterparts. Research has found effectiveness in using culture in education to help address areas of underachievement for African-Americans and other minority cultures. Movement based learning, grounded in the theory of embodied cognition, as well as digital game based learning/ gamification have also been effective in improving academic achievement.

50 CHAPTER 3 DESIGN AND DEVELOPMENT OF MAKIN’ MATH MOVE: A CULTURALLY RELEVANT FULL BODY INTERACTIVE LEARNING ENVIRONMENT FOR PRE-ALGERBRAIC PRACTICE Makin’ Math Move is a gesture based educational technology designed to help African-American students practice pre-algebraic concepts needed to be successful in a typical Algebra 1 course. In an effort to appeal and motivate the targeted demographic, it incorporates audio elementsof hip hop instrumental music , visual urban elements, gamification features and the opportunity to use kinesthetic movement as the input modality. The system also features a selection of avatars to choose from that intentionally represent and reflect African-American features. The avatar mirror the movement performed by the student users. This chapter discusses how the system was utilized by students, rationale behind the design of Makin’ Math Move, the content featured in the system, as well as the software and interface design. 3.1 Use of System and Rationale

The system offers three different modes and a demo/tutorial for the user to interact with. The demo/tutorial consists a short video that walks through how to solve problems in the system, as well as navigational functionality. The first mode, the ”Dress Rehearsal” features an area where users are allowed to solely test the accuracy of their gestures against the system’s recognition database. Users can select from the given functions (add, subtract, multiply, divide, home, help, and pause) to choose which gesture they want to practice. An image or animation, depicting the gesture, us displayed in the upper left hand corner of the screen. The user (and their avatar) can then perform the gesture. A text box will appear if the gesture was performed correctly as well as a percentage representative of the system’s confidence in recognizing the performed gesture. The user can practice any and all of the gestures as many times as they desire within this section. The other two sections or modes are where the actual mathematical practice will take place. Essentially, both areas of practice will feature the same content and design; the only difference is one is more of a guided practice (”Run Through”) that provides specific hints

51 Figure 3-1. Modes; Photo courtesy of the author when certain problem solving errors occur, while the other (”Showtime”) is more unrestricted in terms of scaffolding. Hints are still provided in Showtime, but they have to be askedfor and aren’t as specific as what is given in the Run Through area. The Run Through section was the area used in the final evaluation. The Run Through area allows students to practice from various types of equations based on the content selected for this system; the content employed in Makin’ Math Move will also be mentioned in Section 3.2. Students are shown a problem and instruction for the problem (e.g. ”Solve For x” or ”Complete the problem using Order of Operations”) as seen in Fig. 3-2. Students are shown the gestures for all four mathematical operations. All of the gestures utilized in the system were obtained during the gesture elicitation study, which is defined in detail in Chapter 4. Students can then beginthe problem solving process by selecting their first operand (which is either a single number or variable with a coefficient), which will become highlighted as depicted in Fig. 3-3. Next,they perform the gesture that corresponds to the math operation that will be shown. The screen is updated to show that the operation was performed. Figure 3-4 shows the immediate result of performing the multiplication gesture. Notice the inclusion of the multiplication symbol on the screen. Then, they will select the second operand for this step in the equation; Figure 3-5 shows the result after this selection. If at any point in this process, the selection is incorrect

52 (wrong operand or operator), the system will display a message bubble with a hint that informs them which action was incorrect and suggests how they can arrive at the right selection. Hints were generated at different times throughout the problem solving process: 1) if students asked for a hint, 2) if students performed the correct operation, but used the incorrect operands, 3) if students started with the correct operand, but performed the wrong operation, or 4) if students entered the incorrect answer for the current step. If a hint was generated, the problem was reset and the student could begin again; if it was a multi-step equation, it would only reset to the beginning of that step.

Figure 3-2. Step 1: Problem Shown to Student; Photo courtesy of the author

Figure 3-3. Step 2: Selecting the First Operand; Photo courtesy of the author

Once the correct operands and operator have been selected, a virtual numerical key pad (Fig. 3-5) will be displayed and the student will be able to select their answer. The system will inform the student if the answer for this step is correct or not. If the answer is correct and

53 Figure 3-4. Step 3: Performing the Appropriate Gesture; Photo courtesy of the author

Figure 3-5. Step 4: Selecting the Second Operand and Solving;Photo courtesy of the author for a multi-step equation, they will repeat the process until they successfully reach the final answer. If the answer is correct and the problem has been solved in its entirety, the student can move on to another problem by utilizing the on-screen button. If the answer is incorrect, the system will alert the student and erase the text box so that they can try again.

A summarized rationale for the targeted demographic, content and visual and input design for this system is shown below.

• African-American Students: These students are severely underperforming at every measured grade level when compared to other racial/ethnic groups

• Pre- Algebra Topic: Research suggests students are performing poorly in Algebra due to a weak foundation and transition from arithmetic to algebraic thinking

• 6th and 7th grade: Students are often introduced to Algebra 1 in the 8th grade at the earliest. This ensures students are still studying pre-algebraic content.

54 • Array of Avatar choices: Research suggests that utilizing agents that match a player’s race can have positive effects on motivation and engagement28 [ , 156]

• Gestural interface: Embodied cognition and related theories suggested that movement tied to cognitive tasks can have a positive effect on performance

• Instrumental Hip Hop Music and visual cultural content: Research suggests that the inclusion of culturally relevant content can have positive effects on both academic performance and motivation 3.2 Content

Makin’ Math Move was designed to assist African-American 6th and 7th grade students in practicing pre-algebraic topics. This section describes the mathematical content used for the system’s problems and how it was selected. As discussed in Chapter 2. there are nine areas, that coincide with CCSS and NCTM standards, that have been found to be necessary for success in a standard Algebra I course [197]. The area chosen for this system will be Algebraic Equations. This area was chosen after carefully reviewing the results from the Spring 2017 Florida Standards Assessments (FSA) for Lincoln Middle and Bishop Middle Schools [64], the initial evaluation locations for Makin’ Math Move. These middle schools have the largest African-American population in Alachua County, the county in which the primary researcher’s university resides. After analyzing the targeted population’s (6th and 7th graders) data for the five tested areas (Ratios, Expressions and Equations, Geometry, Statistics and Probability, and The Number System) , it was determined that they averaged the most poorly on the Geometry and Expressions and Equations sections. Since Geometry was not determined to be one of the nine essential areas, Expressions and Equations was selected to be the content area for the system.

Table 3-1. 2017 FSA Performance by Predominantly African American Schools in Gainesville School Name Grade Level Number of Students Ratios EE Geometry SP Number System Max Points Per Category 8 18 8 10 12 Lincoln 6 165 4 7 2 5 5 Bishop 6 206 4 8 2 5 6 Max Points Per Category 14 12 13 9 8 Lincoln 7 221 7 6 6 5 4 Bishop 7 220 6 6 6 5 5

55 Due to the inability to use the schools in the evaluation study, it was decided to move the evaluation sites to Hopewell, Virginia, an area close to the primary’s researcher’s hometown. The same process took place to determine what topic to utilize in the system. Unfortunately, the state of Virginia did not offer online resources that broke down the students’ performance by category performance on their end of the course assessments, the Standards of Learning (SOLs). At the end of the school year, students statewide are required to take the SOL tests for various subjects at different grade levels. For math, in particular, students are tested at the end of grades 3-8 and after specific math courses offered at the high school level, such as Algebra I and Geometry [140]. However, after looking at the distribution of the categories on the test, it was decided to keep same content area because it made up the largest portion of the test (48%). The majority of the problems in the problem set were algebraic one-step equations as well as inequalities. In addition to these type of problems, problems based on Computation and Estimation, the second largest portion of the SOL tests were used as well to diversify the problem set. These problems were encompassed in the Expressions and Equations portion on the FSA. The actual problems utilized in the system will be adapted from released SOL tests. The categorical breakdown for the strands tested by the FSA and SOLs for students in both grades six and seven can be seen in Tables 3-2 and 3-3. Note that in the breakdown found in the Virginia’s SOL documentation for grade 7, the first two strands were combined when listing the categorical breakdown. The specific strands from which problems were taken from correlated with standards defined on both the FSAs as well as the SOLs as shownin Table 3-4.

Table 3-2. Strands Tested on SOLs by Grade Level and Percentage Strand Name Percentage Tested in 6th Grade Percentage Tested in 7th Grade Number and Number Sense 18 28 Computation and Estimation 24 Measurement and Geometry 22 24 Probability and Statistics 36 48 Patterns, and Functions, and Algebra

56 Table 3-3. Strands Tested on FSA by Grade Level and Percentage Strand Name Percentage Tested in 6th Grade Percentage Tested in 7th Grade Ratio and Proportional Relationships 15 25 Expressions and Equations 30 21 Geometry 15 23 Statistics & Probability 19 16 The Number System 21 15

Table 3-4. Standards highlighted in Makin’ Math Move Category Test Strand Algebraic SOL The student will solve one-step linear equations in one Equations variable.. SOL The student will solve one-step linear inequalities in one variable.. FSA Apply and extend previous understandings of arithmetic to algebraic expressions FSA Write an inequality of the form of x >c or x

3.3 Interface Design

The main problem solving area’s interface design was influenced by several dance based video games such as Dance Central and Just Dance; figures 3-6 and 3-7 show images of both of the games’ interface. Both interfaces feature their scoring mechanisms at the top of the screen; Just Dance uses stars and encouraging words, while Dance Central incorporates points. This system’s scoring mechanism was displayed in stars, but was calculated by a point-based system. Students earned points for solving problems; greater points were earned by solving problems without triggering the integrated hints. Dance Central’s interface displays the upcoming gestures for users on the side of the screen which influenced the placement of the gestures for Makin’ Math Move. The images of the gestures were designed using one of the male avatars rigged for the system.

57 Figure 3-6. Screenshot of Dance Central Figure 3-7. Screenshot of Just Dance interface; Taken from [6] interface; Taken from [7]

The Modes menu (Fig. 3-1) was designed to emulate various menus frequently employed in video games. Once displayed, the graffiti-inspired background of Makin’ Math Move’s login page is dimmed out and the Modes menu is overlaid on the page. Each of the modes featured a triangle button , which commonly used to symbolize playing of music, games and other media. The colors are reflective of the colors incorporated on the login page of the system.

Figure 3-8. Login Page of Makin’ Math Move;Photo courtesy of the author

The interface design of Makin’ Math Move has gone through relatively few changes. Figure 3-9 represents the initial mock-up of the problem solving area of Makin’ Math Move. Figure 3-10 shows the final implemented interface of the system. Initially, there wasonly going to be one choice for the background design, but it was decided to add options to allow students to have a greater choice of customization. In the usability studies, discussed in

58 both Chapters 5 and 6, students were asked to navigate to a specified topic, but due to the low success rate in navigation, it was decided to remove this screen completely. In the final evaluation, it was decided that participants would be practicing questions across the various strands found on the Virginia standardized tests.

Figure 3-9. Mock-up of Problem Solving Figure 3-10. Implemented Problem Solving Screen; Photo courtesy of the Screen; Photo courtesy of the author author

Figure 3-11. Background and Music Settings; Photo courtesy of Figure 3-12. Avatar Settings; Photo the author courtesy of the author

Another change that occurred as a result of the usability study was the appearance of the Settings menu. Students often asked how to navigate to the selection of the avatars in the first iteration of the screen shown in Figures 3-11 and 3-12. Originally, students hadtoselect the up and down arrows to navigate the Settings page. In the final iteration, each of settings’ categories (Background, Music and Avatar) added individual horizontal scrolling functionality

59 as shown in Figures 3-13 and 3-14. This was added due to the desire of students to increase the number of options per category from three to five. The Avatar category was moved toa second screen completely and there was the inclusion of an arrow that overtly indicated the location of the Avatars. The avatars were created using two applications owned by Adobe, Fuse and Mixamo. Fuse allowed for the creation and customization of each of the avatars. Here, avatars could be altered based on various categories including body type, skin color, hair styling, clothing and shoes. The software allows for personalization of even the smallest details such as crafting the faces to express different levels of excitement or anger. Once the model was created inFuse,it was uploaded to Mixamo. Mixamo added the necessary rigging mechanisms.

Figure 3-13. Updated Background and Music Settings; Photo Figure 3-14. Updated Avatar Settings; courtesy of the author Photo courtesy of the author

3.4 System Architecture and Implementation

Makin’ Math Move is Unity-based system. It can be accessed via an executable application. Unity was selected as the development environment due to its support of the Kinect for Windows SDK. In addition to Unity and the Kinect, programming was completed in C. Makin’ Math Move is comprised of five main components that work together to gather and display information from and to the student user. The first component is the Microsoft’s

60 Figure 3-15. System Architecture of Makin’ Math Move

Kinect for Windows sensor. The Kinect for Windows sensor is a standalone version of the sensor utilized within the Xbox gaming systems. The sensor allows for skeletal tracking of up to 25 joints for six distinct people, voice recognition capabilities through its embedded microphone, shared use of a single sensor by multiple applications simultaneously and many other features. The sensor comes equipped with a software development kit (SDK) which was essential in the developing of the next component, the gesture recognition database. The gesture recognition database was created with the use of two applications provided by the SDK, Kinect Studio and Visual Gesture Builder (VGB), which allow users to manage and work with gestures to be used in Kinect-enabled applications. The Kinect Studio allows users to record and save clips of their gestures, view data collected by the sensor in different views (2D, 3D, infrared, or color), and playback recorded data. The data recorded in Kinect Studio is then utilized in the Visual Gesture Builder; a screenshot of the application can be found in Figure 3-13 [127]. The VGB transforms this information into data that is usable by gestural based applications. The developer tags the beginning and ending of clips using gesture detection techniques, AdaBoostTrigger (discrete) and RFRProgress (continuous) , to train a specific gesture. Training continuous gestures also requires the tagging of progress stages between any discrete gestures that make up the overall continuous gesture. For example, if training a singular sit and stand gesture, the developer must tag the period for which they are sitting (discrete), the moment they begin standing (progress), the period for which they are standing (discrete), and the moment they begin sitting (progress). The amount of training

61 (number of clips tagged) required varies based on the complexity of the gesture. A single clip can also be used to identify positive and false detection of multiple gestures. If two gestures are similar, such as one requires the use of one arm raised vertically (Gesture A) while the other requires both to be raised (Gesture B), it is important to denote the distinction between the two in training. So for each self-identified gesture in the clip, it is important to notonly tag it as ”True” or positively associated with Gesture A, but also as ”False” or negatively associated with Gesture B, so that system can mark the differences between the two. The database used in the evaluation of this system consisted of 149 tagged clips. Each clip varied in length and contained anywhere from 20 to 150 total tags per clip. The trained gestures are then compiled and built into the gesture recognition database. The gesture database was then able to be imported as an asset into the game engine component.

Figure 3-16. Screenshot of VGB software, Source: [127]

As mentioned previously, the system was developed using the Unity game engine. This component is the center of the entire architecture and works with the various other components to execute the problem solving functionality by receiving and sharing information with the three databases (gesture, content and player’s profile) and displaying information to the user. The content database houses all of the problems utilized within the system and

62 communicates with both the game engine in order to relay what information is to be used as well as the player’s profile database to update the status of the student’s success. The player’s profile database consists of not only the student’s settings customization, but also tracks their progress. It also keeps a record of their time spent on each question, hints used, as well as the status for each problem (correct or incorrect). The final component, the presentation module, refers to the displaying of the technology to students and is inclusive of all visual and audio components. The visual interface was created in the Unity Game Engine as well and was displayed to the students through the use of a Vankyo Leisure 3 Mini projector connected to the individual laptops.

63 CHAPTER 4 GESTURE ELICITATION 4.1 Purpose

The purpose of the gesture elicitation study was to understand what gestures or movements students would associate to various mathematical and navigational functions needed to operate Makin’ Math Move. The goal of the elicitation study was to garner gestural input in an effort to finalize a gesture to be trained for system recognition for each ofthenine functions (add, subtract, multiply, divide, input, home, help, pause and next) employed by the system. 4.2 Participants

There were 20 6th and 7th grade participants, aged between 11 and 13 years old, in this study. The participant group consisted of 13 females and 7 male students. Thirteen of the students were sixth graders, with the remaining seven were seventh grade students. With the exception of one student who failed to answer the race/ethnicity question, all participants self identified as partially or fully African-American. This demographic represents the targeted population of the users of Makin’ Math Move. The students were recruited from three local Gainesville area schools. 4.3 Instruments and Measures

Two types of data were collected from the students during this elicitation study. First, the gestures performed by students were recorded and collected to be analyzed. Then, students completed a survey that collected basic demographics, their thoughts and feelings towards math and technology in the classroom, their use of gestural interfaces, and feedback regarding the elicitation session. 4.4 Materials

The students did not require any physical materials to complete the elicitation phase of this study. A laptop was used, following the gestural portion, by each student to complete the survey. The gestures were recorded by the primary researcher using a Canon EOS 80D camera

64 and instrumental hip hop music was optionally played (at the request of the student pair) using the researcher’s personal laptop. 4.5 Procedure

Due to the prominence of gestural technologies, particularly as interfaces for gaming consoles, three techniques (priming, partnering and production) were implemented in this study to reduce potential legacy bias [130]. Each session began with the primary researcher explaining the purpose of the elicitation study and performing a sample gesture for students (priming). Students were then put into two-person groups of their own choosing and, as a group, asked to come up with a total of 18 gestures within an hour time frame. The 18 gestures comprised of 2 distinct gestures for the following 9 tasks/referents: add, subtract, multiply, divide, input (calculated value after a performed operation), home, help, pause, next. Instrumental hip hop music was played as students created their gestures and was optional when students completed their task and was recorded by a researcher. This style of music was played during the sessions because it would be included in the final design of the system, as music is encompassed in the movement cultural factor. After recording, students completed the aforementioned survey questions. 4.6 Results

4.6.1 Gesture Taxonomy

The gestures were classified, as shown in Table 4-1 along five dimensions: form, body parts, nature, flow and functional which aligns with the three labels defined by136 [ ].The taxonomy created for this culturally appropriate educational technology was adapted from previously described taxonomies [138, 160, 186, 201], with it being most similar to [138]. However the taxonomy in [138] was utilized for a different context (interaction with a humanoid robot) and also the resulting taxonomy goes into more detail with the body parts utilized in each gesture, adds more categories to the Nature dimension and is also inclusive of two new dimensions, Form and Functional. Since the technology will allow for full body gestures, the form dimension encompasses the shape and movement of fingers, hands, feet

65 Table 4-1. Taxonomy of Full Body Gestures for a Culturally Appropriate Educational Technology Dimension Category Description Form Shape Describes the physical attributes associated with the shape and movement of body parts Movement Body Parts One Hand Gesture is performed with one hand only Both Gesture is performed with both hands Hands One Foot Gesture is performed with only one foot Both Feet Gesture is performed with both feet Finger(s) Gesture is performed with one or more fingers Full Body Gesture is performed with multiple body parts outside of those listed above Nature Symbolic Gesture depicts an icon or symbol Cultural Gesture references aspect of African-American culture Deictic Gesture is indicative of a position or direction Arbitrary Gesture has no connection to the referent Flow Discrete Gesture is a static pose Continuous Gesture is dynamic and has movement Functional Command Gesture is mapped to system and navigational functions Operational Gesture is mapped to operate on math content and other specified body parts. The form dimension was primarily included and broken down in the given order to be specific when programming the Kinect for Windows sensor, which is the motion and gesture recognition device that is utilized in the final system. For example, the same hand position (i.e. fist) with movement to the left could convey an entirely different gesture when moved to the right. The breakdown of the Form dimension can be seen in Table 4-2 The body parts dimension refers to which and/or how many body parts were utilized in the gesture. The nature dimension gives the semantic description of each gesture. Symbolic gestures are those which depict a common symbol or icon, such as making fingers cross perpendicularly to form a plus to indicate addition. Cultural gestures are those made with some reference to African-American influenced culture; many of the gestures performed were representative of current popular hip hop dance moves, such as the Milly Rock or Hit the Folks.

66 Table 4-2. Taxonomy for Form Dimension Category Sub-Category Elements Shape (Hand) Palm Orientation In (facing body), Out (away from body), Left, Right, Up, Down Fingertip same as Palm Orientation Orientation Hand Pose Fist, Blade (fingers closed with thumb connected to index), Separated (fingers splayed), Neutral (pose not clearly defined), Fingers(s) Used Hand Position 1-8; see Fig. 4-1 (X/Y) Hand Position Near, Mid-Range, Extended (Z) Arm Pose Bent, Straight Finger(s) Used Pinky, Ring, Middle, Index, Thumb Shape (Other Orientation Forward, Backward, Left, Right, Inward (sole facing Body Part(s)) body), Outward (sole away from body) Movement(Each X, Y, and Z Left, Right, Up, Down, Towards, Away, Hand) direction Both-direction* Rotation Clockwise, Counterclockwise, Both-direction Movement(Other X, Y, and Z Left, Right, Up, Down, Forward, Backward, Body Part(s)) direction Both-direction* Rotation same as Hand Movement’s Rotation *Both indicates that the gestures went in the indicated direction and then moved in the opposite direction (i.e. both-up means that the body part went up and then went down

Arbitrary gestures appeared to not be connected to the Figure 4-1. Hand Position Guide; referent at all. Deictic gestures expressed a position or direction, Modified such as swiping to the right to reveal more information. Flow from: [3] refers to whether or not the gesture was discrete (a static pose) or continuous (a gesture with movement). The final dimension, functional, is divided into two subcategories: command and operational. Command gestures are used to control navigation aspects and system functions, such as pause. Operational gestures are those which are acting on the actual math content presented in the system, such as the gestures created for addition and subtraction.

67 Figure 4-2. Taxonomic Breakdown of the 180 Elicited Gestures

Figure 4-2 shows a taxonomic breakdown of all 180 elicited gestures in the study. The Form dimension was not included in the taxonomic breakdown due to the numerous subcategories; the resulting breakdown would not have returned meaningful percentage values. There is a taxonomic breakdown by individual referents found in Appendix A. As seen in Fig. 4-2, nearly two-third of all gestures performed were continuous, meaning they contained movement. The gestures differed more widely across the nature and body parts dimensions. There was a set number of gestures for the Functional dimension as five of the referents assigned to each of the 20 participants were relegated to the operational category and the remaining four were assigned to the command category. 4.6.2 Agreement

The primary researcher and a fellow colleague each classified all of the gestures using the taxonomy as featured in the Table 4-1 and Table 4-2 which lists the description for each of

68 the dimensions as well as the subcategories. After classifying each of the 180 gestures, the two researchers met to discuss any gestures that received conflicting categorization among them. As a result of the discussion, the description of each of the 180 gestures were solidified. Next, the entire set was divided by the 9 referents and analyzed to see what gesture(s) garnered the greatest agreement amongst the participants. The agreement equation (Eq.4-1) utilized by [200] was also used in this study.

( ) ∑ ∑ 2 Pi Pr rϵR P ⊆P A = i r (4–1) |R|

In Eq.4-1, r is a single referent (i.e. add or input) of all nine referents R.Pr is the subset of gestures for the individual referent r, while Pi represents all of the identical gestures from that subset. Due to the elicitation allowing for full body gestures and the various subcategories under the form dimensions, the gestures grouped together as ”identical” do not take into account the palm orientation for most of the referents. The equation was used more loosely to allow for a meaningful calculated agreement value. Also gestures that were the same with the exception of direction or hand used were counted as ”identical” for the purposes of this dissertation. Identical gestures included pointing in the same direction, but with opposite hands or performing the same gesture (i.e. open hand to indicate ”stop”), but with opposite hands. Eq. 4-2, seen below, gives an example of the agreement equation (Eq.4-1) for the next referent

( ) ( ) ( ) ( ) 5 2 2 2 2 2 1 2 A = + + + + ... (4–2) Next 20 20 20 20 Eq. 4-2 indicates, with the first four terms, that there were four distinct gestures thatwas agreed upon by five, two, and two participants, respectively. The remainder of the equation not shown adds eight additional agreement equation components with a Pi of 1. Although each of these gestures disagreed with the others performed by participants, they are still taken into account because each one agrees with itself. Figure 4-3 shows the agreement scores for

69 each of the nine referents with the highest agreement score calculated for the input referent, which had the highest number of identical gestures, 7, amongst any referent. The lowest value was calculated for help; this referent had four gestures that were performed with the same frequency in the study.

Figure 4-3. Agreement Scores for the Elicited Gestures

Due to the low reproduction of similar gestures for each referent, the agreement equation resulted in very low agreement scores. As a result of the trivial scores, it was decided to use the max consensus of the 20 gestures for each referent to determine which gestures would be included in the system’s gesture set. Table 4-3 shows each of the repeated gestures, as well as how many times they were repeated, for each of the nine referents. Only those gestures repeated more than once were included in the table. The gesture that was performed the most was chosen. For Divide and Help, multiple gestures were repeated for the same number of times. For these referents, students in the usability study determined which gesture would

70 represent the referent; the gesture that was performed in the study would represent the referent in the final system.

Table 4-3. Gesture Consensus for Each Referent Referent Repeated Repeated Repeated Repeated Repeated Gesture (#) Gesture (#) Gesture (#) Gesture (#) Gesture (#) Add Arms form a Fingers form a ”Cooking” cross (7) cross (2) dance (2) Subtract Arm held Sidestep (3) Point down horizontally in (2) front of body (4) Multiply Form X with Form X with hands (3) arms (2) Divide Arms held Diagonal arm ”Hit the ”Milly Rock” ”Cooking” horizontally (3) Folks” dance dance (2) dance(2) above one (2) another (3) Input Display fingers Hands moving ”Cooking” (8) from L to R dance(2) (2) Home Form triangle ”Dab” dance Point down Hands moving (4) (3) (2) L to R(2) Help Kick (2) ”Milly Rock” ”Whip” dance dance (2) (2) Pause Hand(s) Blade hands ”Hit the raised (6) (2) Folks” dance (2) Next Swipe (5) Slide L to R Running (2) (2)

4.6.3 User Defined Gesture Set

The purpose of this study was to identify the most agreed upon gestures for each referent to be utilized in the proposed gesture based educational technology. With the exception of two referents (help and divide), the analysis showed that there was a generalized gesture performed most frequently for each of the remaining seven referents. Figure 4-4 (a-g) depicts each of the gestures chosen after classifying the gestures. These gestures were repeated the most frequently in each of their individual categories. Figure 4-4 also includes a depiction of the multiple gestures with the highest frequency for the divide and help referents. Help had

71 a fourth gesture that received the same frequency (2), but due to it being the same as the selected gesture for the pause referent, it was removed as being a viable option. The options were narrowed down to one gesture for each referent in the usability phase of this research. 4.6.4 Survey Results

Study participants were asked to complete a semi-structured survey that pertained to the purpose of the study (designing a gestural technology for math), any feedback they had, their attitudes towards math in general, and their previous experience with movement based technology. Based on responses from a seven point Likert scale (strongly disagree to strongly agree), 65% of the participants agreed to some degree with the idea that technology made math easier to understand and 85% were comfortable with using technology for math. Many of the participants reported that they used computers, Smart boards and games in their math lessons. Computers and calculators were their primary source of technology used for math education outside of the classroom. With regards to experience with gestural devices, 60% have used the Kinect gaming console in some capacity; 40% had prior experience with the Nintendo Wii and 3 participants used the Playstation Move. Generally, there was positive feedback about the idea of the technology and elicitation process itself. 70% of participants said they would like to practice math using the technology and 90% thought that learning while moving was more enjoyable than traditional learning. They enjoyed ”being as creative as we wanted” and thought ”the idea was awesome, smart, [and] entertaining.” 4.7 Discussion and Next Steps

4.7.1 Predictions vs. Actual Gestures

Due to the inclusion of instrumental hip hop music in the study and the technology, the primary researcher envisioned that most of the gestures would have a heavy cultural influence, primarily in the form of popular dance moves due in part to the use ofmusicduring the sessions. Figure 4-2 indicates that this initial assumption was incorrect. Participants produced more arbitrary gestures than any other category in the Nature dimension, and also

72 73 Figure 4-4. User Defined Gestures and Optional Choices for Divide and Help Referents more symbolic gestures than cultural gestures. Participants did indeed repeat the same dance moves, however they were repeated across varying referents leading to no consensus in a single referent. Figure 4-4k. represents the only cultural gesture (the Milly Rock) that received a high enough frequency count to be considered as a possible option for a gesture in the set. The researcher will explore different ways to include some of the highly repeated (across all referents) cultural gestures in the system; perhaps users can swap out a defined gesture for a popular cultural gesture once they obtain a certain amount of progress. 4.7.2 Limitations

Varying Physical Spaces: The gesture collection process happened across four sessions in three school locations. Due to the varying locations, the space given for the study sessions differed. All students were given the same instructions and the same tasks, the locationto perform and create their gestures just may have been smaller or larger in area than others. Two out of the four sessions had multiple pairs working in close proximity of each other. Due to the enclosed spaces, participants were able to observe other groups and could have potentially influenced and affected their gesture creation process. 4.7.3 Next Steps

The user defined gesture set was utilized in the usability evaluation of the technology. Chapters 5 and 6 will describe how participants of the same demographic tested the system for gestural accuracy as well as overall usability and perceived cultural relevancy. This subsequent study also determined which gestures for the divide and home referents were used. Participants in the usability study selected their preference between the two equally performed gestures for divide and the four gestures for home. 4.8 Summary and Conclusion

This chapter has presented a study on full body gestures for a culturally appropriate math educational gesture-based technology. Based on the classification and consensus of 180 gestures across 9 referents, a partial user defined gesture set was established. The study design and summarized results from this study was published and presented at the 17th ACM

74 Conference on Interaction Design and Children in 2018. The inclusion of representative users at this stage in the design process was essential to ensure that the gestural interface to be created functioned how the targeted population envisioned it. Although this gesture set is defined for mathematical content, a similar elicitation study could be performed when designing for various context-specific educational technologies. The adapted full body taxonomy could also be used for classification purposes for other full body gestural interfaces outside of the educational space.

75 CHAPTER 5 INITIAL EVALUATION OF MAKIN’ MATH MOVE’S USABILITY 5.1 Purpose

The objective of any usability study is necessary to ensure that the system being evaluated is functioning adequately to accomplish its intended goal. The purpose of this study was threefold and tested not only the visual interface and gestural recognition capabilities of Makin’ Math Move, but also an underlying intentional design choice of including cultural components. The usability study sought to address the following questions:To what extent is the system usable, in terms of system usability and gesture recognition accuracy, by 6th and 7th grade African-American students? and Is the system found to be culturally relevant by 6th and 7th grade African-American students? 5.2 Participants

Participants for this usability study were recruited from a STEM-focused summer camp at a predominantly African-American area charter school. The camp was for students who had just departed from grades Kindergarten through sixth grade in the Spring. The population targeted for this study were African-American students who had just completed the sixth or seventh grade. There were approximately 20-25 students who met these requirements, but only 9 students returned parental consent forms and assented to the study. The results discussed in this chapter will only feature the sessions for eight of the nine students; the first session experienced multiple technical delays, which resulted in the student not being able to complete the study. Of the eight participants, six identified as Black/African-American, one identified as being multi-racial (which did not include African-American) and one identified as being Haitian-American. Six of the students were female and two were males. Five had immediately completed seventh grade, while the other three had finished sixth grade. In regards to previous Kinect experience, only two had interacted with the device before.

76 5.3 Materials and Setup

The usability study was conducted using a Razer laptop which ran the Makin’ Math Move application. Connected the laptop was the Kinect V2 sensor which captured and recognized the students’ gestures. Each participant session, including the interview, was recorded using a Canon 80D camera. 5.4 Measures

In order to quantify each of the three testing factors, three different measures were utilized. A standard System Usability Scale (SUS) was administered following each student’s usability session. The SUS consisted of ten statements accompanied by a 5 point Likert Scale; responses ranged from ’Strongly Agree’ to ’Strongly Disagree’. Along with SUS survey, the student completed a survey created by the primary researcher that asked questions regarding their demographics, experience with the Kinect and gathered specific feedback about the system. The second part of the survey consisted of questions used to assist in determining the cultural relevance of the system. In order to determine gestural accuracy, the researcher indicated in their notes during the observation if each student was able to successfully perform the gesture, as recognized by the system. The time was also recorded via the system to indicate task completion; time was stopped when the student’s gesture was successfully recognized and completed the task or when they failed to complete the task and decided to move onto the next task. 5.5 Procedure

Each study session began with each student reading and completing the assent form. Next, the purpose of the study was reiterated to the participant (it was briefly explained on the assent document), as well as a general overview of how the study would proceed. There were 13 tasks to be completed in the study. Students were informed that they could spend as much time as they wished on each task and could also choose to skip ahead to the next task if they felt that they could not complete it.

• Tasks 1-3 consisted of steps needed to get students to the practice page.

77 1. Settings: ”Select your background, music and avatar and go to the next screen.” 2. Mode: ”Navigate to the Guided Practice areas.” 3. Topic: ”Navigate to the Usability Study topic.”

• Tasks 4-7 asked the students to solve basic, single digit equations for each of the four basic math operations (addition, subtraction, multiplication and division). The interface displayed the corresponding gesture to perform for each operation.

• Tasks 8-11 asked the students to perform gestures for the navigational functions (home, help, pause and next problem). The image for each of these gestures was also displayed on screen. It is important to note that for the Help function, students were presented with images of three different gestures (Punch, Kick, Milly Rock, which isan African-American cultural dance move) and asked to select which one they wanted to associate with this specific function. This is due to the fact that these three gestures received the same agreement frequency in the previous gesture elicitation study. The results from this study were used in determining which of the three gestures were utilized in the final implementation of this system.

• Tasks 12 and 13 required students to solve two more complex math problems. The first was a one-step Algebra problem (A + 6 = 9) and students were asked to solve for the letter A. The second problem was an Order of Operations styled problem which consisted of three operations. Once all 13 tasks were attempted, participants completed one large survey, which consisted of the demographic survey, the SUS survey, cultural relevance measurement questions, and other questions related to their participation in the study. After the conclusion of the survey, participants engaged in a short interview with the primary researcher. 5.6 Results

5.6.1 Task Completion and Gestural Accuracy

Of the 13 tasks, only 5 were able to be successfully completed by all students and these included tasks 1 and 2 (selecting settings and navigating to the guided practice page) and three of the four navigational gestures (home, help, and pause). The more complex math problems (Tasks 12 and 13) were the hardest to complete; only one student was able to get both tasks completely correct. Students went on to report in the survey that several of the gestures were hard to perform with the most frequent response being for Add (31.25%) and Next (25%). The tables below present the means of the number of tasks completed by each

78 grouping (Gender, Grade and Kinect Use). T-tests were conducted for each grouping assuming equal variance; none of the differences proved to be statistically significant. This is likely dueto the small sample size. A table displaying the success rate of each task is also found at the end of this section, along with the average task completion time.

Table 5-1. Average Task Completion by Grade Grade Number(N) Mean Number of Tasks Completed Standard Deviation (SD) p 7 3 9.33 1.53 .46 8 5 8.4 1.67

Table 5-2. Average Task Completion by Gender Gender Number(N) Mean Number of Tasks Completed Standard Deviation (SD) p Male 2 9.5 2.12 .48 Female 6 8.5 1.52

Table 5-3. Average Task Completion by Kinect Use Previous Kinect Use? Number(N) Mean Number of Tasks Completed Standard Deviation (SD) p Yes 2 10.5 0.71 .06 No 6 8.17 1.33

Table 5-4. Task Completion Tasks Completed Task Successfully By Average Time Spent on Task (s) Participants 1 8 80.74 2 8 12.35 3 4 41.56 4 4 91.40 5 5 45.83 6 6 65.54 7 7 33.17 8 8 6.76 9 8 17.21 10 8 15.28 11 2 54.81 12 1 75.70 13 1 69.01

79 5.6.2 Usability

After calculating the SUS scores for each of the eight participants, the scores were averaged with a result of 52.5. Research suggests that a score above 68 would be considered average. Based on an widely cited article, which adds an adjective rating to the SUS scale, a score of 52.5 would fall between the range of ”Ok” and ”Good”[25]. The tables below present the average SUS scores for students by gender, grade and previous Kinect use. To determine if there were significant differences between the means from each group, t-tests were conducted. Due to the small sample size during this evaluation and results from F tests, the tests were ran assuming unequal variances. None of the differences proved to be statistically significant.

Table 5-5. Average SUS Score by Grade Grade Number(N) SUS Mean Standard Deviation (SD) p 7 3 47.5 2.5 .50 8 5 56 19.89

Table 5-6. Average SUS Score by Gender Gender Number(N) SUS Mean Standard Deviation (SD) p Male 2 47.50 3.54 .62 Female 6 54.58 18.13

Table 5-7. Average SUS Score by Kinect Use Previous Kinect Use? Number(N) SUS Mean Standard Deviation (SD) p Yes 2 62.5 17.68 .35 No 6 49.58 15.28

5.6.3 Cultural Relevance

A portion of the cultural relevance measure was adapted from two measures, the Child Activity Questionnaire and Questionnaire of Stimuli Preference, utilized by A. Wade Boykin and colleagues as a means of measuring movement and verve in third and fifth graders in a study analyzing some of the interrelated dimensions mentioned in Section 2.2.2 [38]. The questions used can be found in Appendix A. The Child Activity Questionnaire evaluates students’ awareness of their expressions of movement using three subscales with items measured using a 5-point Likert scale ranging from 1 (Almost never) to 5 (Almost always) [23]. Of the three

80 subscales, only the adapted questions from the subscale related to Movement/Music Mosaic was administered to students. The descriptive statistical information from this subscale can be seen in Table 5-8.

Table 5-8. Descriptive Statistics for Movement/Music Mosaic Subscale Min Max Mean Std. Dev. Variance 1 5 3.0 1.285 1.652

The Questionnaire of Stimuli Preference evaluates students’ proclivity for varying stimulating school activities. It also features three subscales for each of the three verve dimensions (Stimulus Intensity, Stimulus Variability and Stimulus Density) and scores items from 1 to 3, with 1 signifying the lowest stimulus response [22]. In the context of this system, stimulus intensity refers to liveliness of the interface and presented content. The variability refers to the amount of variety in activities and the density refers to the number of stimulating elements. For the survey administered to students, two questions from each subscale was adapted to be context specific to the system and usability study. The descriptive statistical information from this measure can be seen in Table 5-9.

Table 5-9. Descriptive Statistics for the Questionnaire of Stimuli Preference Stimulus Type Min Max Midpoint Mean Std. Dev. Variance Intensity 2 3 2 2.75 0.45 0.2 Variability 1 3 2 2 0.89 0.8 Density 1 3 2 1.813 0.54 0.3

5.6.4 Additional Results from Survey, Interview and Observations

The survey asked students which gesture did they prefer for the Help function and 6 (75%) of the participants reported the punch or ”Whip” motion although only 3 participants actually performed that gesture during the study. Half of the participants reported that they were not fully clear what each gestural image was trying to portray. Despite low SUS scores, only 2 students reported that they were not interested in using the system. 87.5% of the students enjoyed the music options provided.

81 5.7 Discussion and Next Steps

5.7.1 Gestural Accuracy and Task Completion

The low recognition rate/task completion of the core mathematical operational tasks suggested that the system was not trained enough to elicit proper results. For some students, the inability to gain gesture recognition in the basic four math functions prevented some of them from being able to succeed in the more complex math tasks (tasks 12 and 13). For the majority of both of those two tasks, students a) could not get the system to recognize the operational gesture and/or or b) just did not know how to solve the problem (with or without the system). Of the 7 students who were unable to complete both tasks 12 and 13, only 2 knew and vocally expressed how to solve the algebra problem (task 12), but experienced gestural recognition issues; one student knew and vocally expressed how to complete the order of operations problem, yet experienced similar technical issues. As a result of this observation, the usability study ended prematurely with the 8 participants in order to perform additional training of each of the gestures and conduct another usability evaluation with 20 students. Videos collected by the researcher were reviewed and considerations were taken into account on how to retrain the gestures based on how students performed them in the videos. The Multiply gesture, for example, required the crossing of arms in the formation of an ’X’ in front of the chest. It was noticed that students were often holding their hands in fists and also resting their arms against their chest, as opposed to performing their gesture off of their body. It was important for the researcher to include these variations in gesture poses in the retraining to ensure that the system will recognize all the deviations and attribute it to that specific function. 5.7.2 Usability

It is believed that the low gestural recognition rate of the system could have negatively affected the students’ perceptions of the system and perceived usability. The increased training has the potential to result in a more smoothly executed system and increase students’ ease of use. It was also noticed that a few students had issues understanding some of the vocabulary

82 used in the SUS scale and asked for, as well as received, clarification. In the next usability study, it will be stressed that they should feel free to ask any questions to make the survey more understandable. 5.7.3 Cultural Relevance

Based on the results from the Movement/Music Mosaic subscale, featured in Table 5-8, combined with results from additional survey and interview questions, this limited study suggests that students enjoyed the presence of music in the system and did not find it to affect their ability to execute the steps essential to the problem solving process. A majority of the students enjoyed the options presented in the system and some requested the option to have more of the same style of music to choose from; this was implemented for the final evaluation. Based on the means on each of the three verve dimensions found in Table 5-9, it can be concluded that students felt that the system was above average with regards to stimulus intensity, average in terms of stimulus variability, and below average for stimulus density. There will be no change to the system in terms of this measure before the next full round of usability testing. Any changes, primarily just in the area that relates to stimulus density, will be made based on the results of the next usability evaluation. While this measurement is important, it does not have a strong effect on how the students will be able to complete the tasks. 5.8 Conclusion

Makin’ Math Move is a culturally relevant, full body gestural educational technology designed and developed to provide supplemental pre-algebraic practice for African-American students. This usability study sought to determine the overall usability of the system, the accuracy of the gestural recognition database and also the amount of perceived cultural relevance in the system. Due to limited participation at the recruitment camp, combined with noticeable observations in the students’ inability to gain recognition by the system, it was decided to treat this study as an initial usability study. Data, such as video recordings, was used to alter the existing recognition database by studying students’ movements and retraining gestures accordingly to cover the variations in posing. A secondary study was executed with

83 the desired sample size of 20 students with the retrained database. Despite the inability to successfully complete some of the tasks, students in the initial study were excited to participate and liked the idea of such a system to help with practicing math skills.

84 CHAPTER 6 COMPLETE USABILITY STUDY 6.1 Purpose

Due to the noticeable and unfavorable gesture recognition results and low participation rate, it was decided to repeat the usability testing of Makin’ Math Move. This study tested the same gestural recognition capabilities as well as overall usability of the system and perceived cultural relevancy. The measures and procedure for this second usability study were identical to the initial study. Beyond the results, the differences between the studies include the number of participants and some of the recruiting locations. 6.2 Participants

Participants for this usability study were initially recruited from a local predominantly African American charter school. The population targeted for this study were African American students who were in the sixth and seventh grade. The charter school featured grades Kindergarten through sixth, which accounts for the large amount of sixth grade participants. There were approximately 23 students who met these requirements, but only 15 students from the charter school returned parental consent forms and assented to the study. The remaining five students were recruited via word of mouth in both the local Gainesville area, as well as in Richmond, Virginia (the hometown of the researcher). Of the 20 participants, 19 identified as Black/African American; one failed to answer the question. None of the students reported to be Hispanic or Latino. Twelve of the students were female and eight were males. Sixteen students were in the sixth grade and three were in the seventh grade. Only one student was not technically in the sixth or seventh grade; however, she was in a lower level math class that taught pre-algebraic material at an alternative education program, which qualified her to be a participant of the study. In regards to previous Kinect experience, only seven had interacted with the device before. 6.3 Hypotheses

The hypotheses for this study are as follows:

85 • H0: The usability of the system will be rated as below average and the system will recognize less than 94% of gestures performed by users.

• HA: The usability of the system will be rated as above average and the system will recognize more than 94% of gestures performed by users.

• H0: The system will not be perceived as culturally relevant by users.

• HA: The system will be perceived as culturally relevant by users. 6.4 Results

6.4.1 Usability

After calculating the SUS scores for each of the twenty participants, the scores were averaged with a result of 59.375. Research suggests that a score above 68 would be considered average. Based on an widely cited article, which adds an adjective rating to the SUS scale, a score of 59.375 would fall between the range of ”Ok” and ”Good”[25]. The tables below present the average SUS scores for students by gender, grade and previous Kinect use. The student that was technically in the eighth grade was excluded from the test examining the effect between grade level and SUS score, but was included in the other two tests; herSUS score was 90. To determine if there were significant differences between the means from each group, t-tests were conducted. Due to results from F tests, the tests were ran assuming equal variances. None of the differences proved to be statistically significant.

Table 6-1. Average SUS Score by Grade Grade Number(N) SUS Mean Standard Deviation (SD) p 6 16 61.41 20.35 0.34 7 3 49.17 16.65

Table 6-2. Average SUS Score by Gender Gender Number(N) SUS Mean Standard Deviation (SD) p Male 8 61.88 17.20 0.65 Female 12 57.08 21.31

6.4.2 Task Completion and Gestural Accuracy

Of the 13 tasks, only 3 were able to be successfully completed by all students and these included tasks 1 and 2 (selecting settings and navigating to the guided practice page) and one

86 Table 6-3. Average SUS Score by Kinect Use Previous Kinect Use? Number(N) SUS Mean Standard Deviation (SD) p Yes 7 60.71 25.52 0.83 No 13 58.65 16.38 of the four navigational gestures (home). The task requiring students to perform the gesture for Next was the hardest to complete; only one student was able to successfully trigger the function. The gesture required was a swiping motion of the hands from left to right. The more complex math problems (Tasks 12 and 13) were the next hardest to complete; only 3 students were able to successfully complete Task 12, while 9 were able to complete Task 13. The tables below present the means of the number of tasks completed by each grouping (Gender, Grade and Kinect Use). T-tests were conducted for each grouping assuming equal variance; none of the differences proved to be statistically significant. A table displaying the success rateofeach task is also found at the end of this section, along with the average task completion time.

Table 6-4. Average Task Completion by Grade Grade Number(N) Mean Number of Tasks Completed Standard Deviation (SD) p 6 16 9.19 1.28 0.29 7 3 8.33 1.15

Table 6-5. Average Task Completion by Gender Gender Number(N) Mean Number of Tasks Completed Standard Deviation (SD) p Male 8 9.25 1.16 0.67 Female 12 9 1.35

Table 6-6. Average Task Completion by Kinect Use Previous Kinect Use? Number(N) Mean Number of Tasks Completed Standard Deviation (SD) p Yes 7 9.71 0.95 0.11 No 13 8.77 1.30

Of the 13 tasks, 8 were designed to focus on the gestural accuracy to ensure that the recognition database was properly trained. Those were Tasks 4-11 which required students to perform the four basic math operations and four navigational functions. Tasks 1-3 used simple selection via the right hand acting as a cursor and Tasks 12 and 13 featured math

87 Table 6-7. Task Completion Tasks Completed Task Successfully By Average Time Spent on Task (s) Participants 1 20 77.34 2 20 10.86 3 10 44.26 4 10 70.36 5 19 46.79 6 15 53.10 7 17 45.82 8 20 5.51 9 19 74.34 10 18 19.36 11 1 84.72 12 3 93.96 13 9 49.88 problems that would be solved in the finalized system and required the gestural accuracy that was being tested in the 8 aforementioned tasks. The table below shows the gestural accuracy in percentages for Tasks 4-11; these percentages were derived from the amount of students that were able to complete each of these tasks. The average total accuracy based on these eight tasks was 74.38%.

Table 6-8. Gestural Recognition Accuracy Task Function Gestural Recognition Rate 4 Add 50% 5 Subtract 95% 6 Multiply 75% 7 Divide 85% 8 Home 100% 9 Help 95% 10 Pause 90% 11 Next 5%

6.4.3 Cultural Relevance

As discussed in further detail in Section 5.6.3, two measures, the Child Activity Questionnaire and Questionnaire of Stimuli Preference, were adapted and used to assess

88 perceived cultural relevance for this usability study. As in the initial usability testing, only the adapted questions from the subscale related to Movement/Music Mosaic from the Child Activity Questionnaire was administered to students. The descriptive statistical information from this subscale can be seen in Table 6-9.

Table 6-9. Descriptive Statistics for Movement/Music Mosaic Subscale Min Max Mean Std. Dev. Variance 1 5 3.0 1.402 1.965

The Questionnaire of Stimuli Preference, which is used to evaluate students’ proclivity for varying stimulating school activities, was used in the same manner as the prior usability test. Two questions from each subscale was adapted to be context specific to the system and usability study. The descriptive statistical information from this measure can be seen in Table 6-10.

Table 6-10. Descriptive Statistics for the Questionnaire of Stimuli Preference Stimulus Type Min Max Midpoint Mean Std. Dev. Variance Intensity 1 3 2 2.675 0.616 0.379 Variability 1 3 2 2.205 0.833 0.694 Density 1 3 2 2.025 0.66 0.435

6.4.4 Additional Results from Survey, Interview and Observations

The survey asked students which gesture did they prefer for the Help function and 9 (45%) of the participants selected the kick gesture, which corresponds to the number of students who actually performed that gesture during the study; this gesture will be utilized to trigger the Help function in the final evaluation. Only 5 students reported not being ableto fully comprehend what each gestural image was showing. Students went on to report in the survey that several of the gestures were hard to perform with the most frequent response being for Next (28.13%), Divide (25%) and Add (21.88%). For this question students were able to list multiple gestures that were difficult to perform and the question received 32 responses. The interview clarified that students did not think the actual gestures were difficult to perform,but hard for the Kinect sensor to recognize. Despite the less than stellar SUS scores, only 1 student

89 reported that they were not interested in using the system; 70% of the students reported they would like to use Makin’ Math Move for practice and 25% responded ”Maybe.” 95% of the students enjoyed the music options provided and many requested that they had more options to choose from during the follow up interview. 6.5 Revisiting the Hypotheses

Based on the below average SUS score (59.375) and low average percentage of gestural accuracy, I failed to reject the null hypothesis for the first research question examined in this study:

• H0: The usability of the system will be rated as below average and the system will recognize less than 94% of gestures performed by users. 94% was used as a gesture accuracy target based on the average recognition response by [93, 164] when testing full body gestural detection on Kinect-based devices. Based on the interview responses, observations and average measure of movement expressiveness and above average measures of stimuli preference, I rejected the null hypothesis for the question pertaining to cultural relevancy:

• H0: The system will not be perceived as culturally relevant by users. 6.6 Summary, Discussion and Next Steps

In summary, this usability study was performed to determine students’ capability of using Makin’ Math Move, along with testing the gestural recognition database as well as the perceived cultural relevance of the system. The following subsections will discuss conclusions drawn from the obtained results and any changes made to the system as the result of the study. 6.6.1 Gestural Accuracy and Task Completion

For nearly half of the tasks, Tasks 1-4, 6 and 8, there was no change in task completion between the initial usability study with 8 students and the larger study with 20 participants. All students, in both studies, were able to successfully select their settings (Task 1), navigate to the guided practice area (Task 2) and perform the Home gesture (Task 8). Half of the

90 participants were able to navigate to the usability topic (Task 3) and perform the Addition gesture (Task 4). Due to the performance results of Task 3 and a redesign of the system’s structure, the screen that required students to navigate to specific topics was removed. The content utilized in final evaluation of the system will be combined in one area. Improvements made in terms of gestural accuracy will be mentioned in the following section. There was improvement for 3 of the tasks. There was a large improvement (over 30%) in the completion rate for the multiplication gesture (Task 5) and smaller improvement for the more complex mathematical problems that needed to be solved in Tasks 12 (an improvement of 2.5%) 13 (an improvement of over 30%). The improvement in Task 5 can be attributed to additional training of the gesture in the VGB application. The improvement in the mathematical tasks can be likely attributed to a combination of a few reasons. It is worth noting that for Tasks 12 and 13, there was a percentage of students who knew how to solve the problem, but were affected by poor gestural recognition. The percentages displayed in Table 6-8 for both tasks refers to students who were able to solve the problem correctly, logically and gesturally. For Tasks 12 and 13 in the larger usability study, an additional 25% and 15% of students, respectively, knew how to solve the problem logically but were not able to be recognized successfully by the system. In the initial usability study, the percentages were similar; 25% and 12.5% of the students knew how to logically solve Tasks 12 and 13, respectively. Additional training was done for all gestures in between the two usability studies, so for students who actually knew how to solve the problem, the lack of gestural accuracy became less of a hindrance. This second study was performed in the fall, whereas the pilot study was performed in the summer. Students in the first group may not have been as focused or practicing their math skills since they were not enrolled in a traditional schooling atmosphere. There was a minute drop (between 1.5% and 10%) in task completion for Tasks 7 (Division), 9 (Help) and 10 (Pause) between the two studies. Only one task, Task 11 (Next), dropped in task completion by 20%. Because of the low accuracy rate in both studies, the

91 gesture for Next was removed from the system. In the final version of the system, students will have the ability to either wait for the next problem to appear (following a brief pause) after correctly solving the problem or they may utilizing an on screen button to move onto the next problem to skip the current problem. All of the gestures were additionally trained after the conclusion of this study to improve the accuracy all over. 10 students from the alternative school who returned consent and assent forms, but did not meet the grade requirement for the usability study, were utilized in testing the gesture accuracy one final time before the final evaluation. Students were able to perform the gestures for all of the functions, with the exception of Next, which further solidified the decision to transform this function intoan on-screen button. 6.6.2 Usability

The average SUS score improved slightly between the two usability studies. It is believed that the increased gestural training may have improved the students’ sense of usability. Similarly to the smaller study with the eight students, the SUS measurement was not completely clear to all participants. When completing the statements associated with the SUS, multiple students frequently struggled with the same statements:

• I found the system unnecessarily complex.

• I found the various functions in this system were well integrated.

• I found the system very cumbersome to use.

• I thought there was too much inconsistency in this system. For most of the occurrences, it was simply due to not knowing the definitions of the words; primarily this happened when reading ”complex” and ”cumbersome”. In regards to the other two statements, students could read the words in the sentences, but did not necessarily comprehend what the statement was asking. When determining how to measure the usability of the system, an alternative system was considered. The Wong-Baker smiley face system was one option that was examined by the researcher before conducting the usability[202]. However, this visual portion of the scale would have only changed the response options for the

92 SUS; it would not have assisted in the comprehension of the questions. A suggestion would be to simply rewrite the ten statements with language and words suitable for fifth through eighth grades in order to assess usability of students at the middle school level. Fifth grade comprehension should be taken into consideration to account for those middle school students who may not be where they should be ideally according school and state standards. In an effort to address some usability issues due to visual aspects mentioned in the interviewand survey, the images that depict that gestures will be made larger and more defined. Also, the usability study required the students to interact with the technology using a 19 inch monitor. Projectors were utilized in the final evaluation so that visual elements can be clearer and larger for participants. 6.6.3 Cultural Relevance

Based on the results from the Movement/Music Mosaic subscale, featured in Table 6-9, combined with results from additional survey and interview questions, this study reported similar results to the initial usability session; the students enjoyed the musical options and it did not impede on their ability to problem solve. Students requested more musical options; while most liked the hip hop/ rap instrumental options, some did ask for options with a slower tempo, so two more options were included for students to select from. Based on the means on each of the three verve dimensions found in Table 6-10, it can be concluded that students felt that the system was above average with regards to all 3 stimulus types. Since the system ranked as above average, no changes were made with regards to verve representation within the system. 6.7 Conclusion

The system, Makin’ Math Move, was reevaluated to determine its usability after an initial usability study was conducted that yielded in less than desirable results. Like the first usability study, it sought to determine the overall usability of the system, the accuracy of the gestural recognition database and also the amount of perceived cultural relevance in the system after additional training was conducted following the first study. Despite the low SUS score, 70%

93 of the students was interested in using the system with another 25% reporting that they may want to use it. Albeit there is no official cultural relevance measurement, results from two scales, which have been used to measure movement expressiveness and verve in K-12 education, indicated that the system invoked an average amount of movement expressiveness from the participants and that the system contained a slightly above average amount of verve in terms of stimulus intensity, variability and density.

94 CHAPTER 7 EVALUATION OF MAKIN’ MATH MOVE FOR IMPACT ON MOTIVATION AND ACADEMIC ACHIEVEMENT 7.1 Purpose

This study was conducted in order to determine the impact of Makin’ Math Move on the math academic performance and motivation of African-American sixth and seventh grade students. 7.2 Participants

Participants for this evaluation study were initially recruited from an already established and voluntary after school program at a public middle school in the Greater Richmond Area in Virginia. Although African-American students were the targeted population of the study, the school district felt that only asking these students to participate, when distributing the recruitment announcement, that it would be discriminatory against students of other races. Due to this, the study was opened up to all races and ethnicities to participate. While data was collected and analyzed for all participants, the data reported in this dissertation will only be for the African American students; there were 4 White students who completed this study as well. Since the after school program did not require mandatory participation, some students who began and assented to the study did not stay through until its completion. In total, 24 students, (20 identified as partially or fully African-American) returned consent forms, however 2 of the 20 students failed to participate in any of the experimental sessions. One chose not to leave their after school class to attend the sessions, while the other did not return to the after school program at all after agreeing to participate and completing the pre-test. Of the 18 remaining African-American students, 3 students failed to complete one of both parts of the post-test (tools measuring math performance and motivation) and one student did not fully complete the motivational assessment during the pre-test and was determined to be an outlier, when analyzing the results. In an effort to recruit more African American students in both experimental andcontrol groups, additional participants were elicited from an already established summer STEM

95 program being conducted at a public charter school in Gainesville, Florida that was utilized in the elicitation and usability studies of this research. The STEM camp was designed to target populations that are underrepresented in STEM by embracing cultural identities and interests. While five students consented to research, only two of the newly acquired participants fully completed the experiment. The students’ grade levels reflect what grade that they had just completed in the 2018-2019 school year. The 16 total students who participated in the study can be described as:

• Grade:

– 6th: 11 – 7th: 5

• Boys: 6

• Girls: 10

• Race:

– Black or African-American, Not Hispanic: 12 – Black or African-American, Hispanic: 1 – Mixed Races, including Black or African-American, Hispanic: 2 – Mixed Races, including Black or African-American, Ethnicity Not Selected: 1 The students who were initially recruited at the Greater Richmond Area middle school were apart of an after school program entitled 21st Century. The middle school, as well as an elementary school within the district, received a grant from a local foundation to provide academic enrichment in the form of after school tutoring throughout the school year. 21st Century was in session on Monday, Wednesday and Friday from the end of the normal school day (2:25 pm) until 4:30 pm. Students are provided an after school snack following the traditional dismissal bells and are then escorted to either a math or English tutoring sessions. Students switch to the other session halfway through the after school program. The middle school is the only one serving the local district. For the current 2018-2019 school year, there was an enrollment of 931 students across the three grade levels [8]. Based

96 on enrollment in the fall of the 2018-2019 academic year, 61.7% of the students were Black, 26.4 % were White, 10% were Hispanic and the remaining approximately 2% identified as Asian or two or more races. For the previous year (2017-2018), 74% of all students at the middle school passed state end-of-course math assessments; 4% showed growth while 15% showed no proficiency or growth. When broken down by race, 70% of Black students were considered to be passing with 5% showing growth; 18% were not showing any growth in the math subject area. For White students in the same year, 80% were passing, with 2% showing growth and 11% showing no growth. The percentages of those showing growth were not included with those who passed the assessment that year. 7.3 Materials and Study Set Up

The students in the control group used laptops provided by the school. These students practiced the same randomized set of questions that the experimental group were given via the Qualtrics Survey Software. The experimental group used materials Figure 7-1. Control Group’s Interface; Photo provided by the researcher. The experiment courtesy of the author was conducted in the multipurpose room, for those who completed the study in the middle school. There were four identical stations set up within the room. Each station featured a Lenovo Yoga laptop which executed Makin’ Math Move using the Unity Development platform. Each laptop was connected to a Vankyo Leisure 3 Mini projector, which was used to mirror the application on to the multipurpose room’s walls. Due to the brick-like finish on the walls, poster board was hung on the walls in front of the projector space for a clearer projected image. The laptop was also connected to a Kinect V2 sensor to capture the gesture recognition. Since there were four

97 students in the room at a time, each student was provided a set of Prime Audio Slide wireless earbuds which were connected to the laptop via Bluetooth connectivity. Each package of earbuds came with a set of three earbud covers, so each student was given one at the start of the study and placed in a container with their name on it. The headphone covers for each respective student was applied to the headphones before, and removed after, each session by the researcher. For students at the STEM camp, the setup was similar to that of the middle school with two exceptions. Since there were only two students, only two stations were set up. The set up was identical to those in the first location, with the exclusion of the poster board because the classrooms at the charter school were smoother and did not require it. Figure 7-2 shows a depiction of an individual station.

Figure 7-2. Experiment Setup

7.4 Measures

In this study, there was one instrument used to measure the motivation of the students. The Course Motivation Survey (CMS) utilized in this research was created by Hirumi; it was an adaptation of the Instructional Materials Motivation Survey (IMMS) developed by Keller in 1987 [98]. The IMMS survey utilized Keller’s Attention, Relevance, Confidence, and

98 Satisfaction (ARCS) model of motivation, which was created as a means of enhancing the motivational aspect of learning materials [97]. Each of the four segments are grounded in various theoretical foundations as seen in Table 7-1.

Table 7-1. ARCS Components of Motivation and Their Foundational Theories Motivation Subset Theoretical Foundations Attention Curiosity, Perceptual Arousal, Inquiry Arousal Relevance Drive Theories, Needs Hierarchy, Need for Achievement Confidence Self-Efficacy, Locus of Control, Learned Helplessness Satisfaction Conditioning Theory, Cognitive Evaluation Theory

The Attention component is in reference to getting and sustaining Attention of the learner and encompasses the areas of perceptual arousal, inquiry or curiosity arousal, stimulus variability. The Relevance facet can refer to the actual content being taught or how the material is taught and can be addressed by altering instructional materials in the areas of goal orientation, specific needs of students and relating to familiar experiences. The Confidence component, which can affect students’ ability to persist, is essential to maintain and/or increase student self-efficacy. Some suggested ways of bolstering this Confidence isclearly stated learning requirements, different opportunities to experience success as well linking student success/failure with effort given. The final component, Satisfaction, is related tohow well students feel about their successes and whether they felt it was worthy of their effort. Satisfaction can be incorporated via natural, positive and equitable consequences. The original IMMS had 36 questions, but the CMS had 20 questions across the 4 areas; 5 questions per motivational facet. The survey features a 5 point Likert scale, ranging from Not True (1) to Very True (5). Six of the questions were written using a negative connotation and the scoring was reversed for these questions (i.e. Not True (5)). The minimum and maximum score for the entire survey is 20 and 100, respectively. Each individual subset has a range of 5 to 25. In order to evaluate the academic performance of the students, a math benchmark, found in Appendix A, which featured questions adapted from Virginia’s end of year SOL assessments,

99 was given before and after the experimental period. Additionally, the Makin’ Math Move application tracked students time per question, which questions they skipped, and number of problems completed successfully during each students’ session. 7.5 Hypotheses

The hypotheses for this study were as follows:

• H0: There will be no significant difference between the motivation of the experimental group of students who utilize the proposed system versus the control group who practice math traditionally.

• HA: The experimental group of students who utilize the proposed system will experience higher levels of motivation as compared to the control group who practice math traditionally.

• H0: There will be no significant difference between the pre-algebra problem solving performance of the experimental group of students who utilize the proposed system versus the control group who practice math traditionally.

• HA: The experimental group of students who utilize the proposed system will experience significant improvement in pre-algebra problem solving as compared to the control group who practice math traditionally. 7.6 Procedure

7.6.1 Before the Study

Prior to the study’s commencement, there were several meetings between the school district’s superintendent, principal and after school program coordinator. Students in the program, that met the grade requirement, were sent home a parental consent form as well as a letter written by the principal explaining the study to the parents and guardians. Students who returned the form were then asked to indicate on an assent document that they would like to participate in the study. 7.6.2 Study Logistics

Study at Greater Richmond Area Middle School) The study occurred during the hour and a half long 21st Century after-school program each day for three weeks. During 21st Century, students, based on which group within

100 the experimental group they were placed into, students in groups of four travelled to the multipurpose room to participate in the study. Study at Gainesville STEM Camp The study occurred in a 30-minute block throughout the STEM camp’s daily activities. The two students were called out of their normal class (which grouped and educated students from grades six through eight) into a separate classroom that had the two stations set up. 7.6.3 Pre-Test and Post Test

Prior to any Kinect interaction, all 24 students were administered the math benchmark mentioned in the previous section. The questions, which were ranked by difficulty by a math teacher who has had previous experience teaching both Pre-Algebra and Algebra courses, were given different weights, 1-3 for difficulty levels of easy, medium and hard. The students’ scores were calculated using the weights and the students were ranked from highest performing to lowest performing, then every other student was placed into the experimental and control groups so that the groups were similarly representative of all performance levels. All students were also given the adapted Course Motivation Survey, which did not affect their placement into either the control or experimental group. The participants at the Greater Richmond area school completed these two instruments within their 21st Century classrooms on school sponsored laptops. The Gainesville participants completed the instruments on Lenovo laptops provided by the researcher. These additional two students, while their scores were calculated and weighted for analytic purposes, they were automatically placed in the experimental group as a means to achieve a sample size that would produce valid data results. At the time of their inclusion into the experiment, there were only 6 African American students in the experimental group with complete data. All students completed the same instruments measuring their math performance and motivation at the conclusion of the experiment as well. 7.6.4 The Experiment

Prior to conducting the evaluation, students in the experimental group had a 10 minute training session to ensure they are able to navigate the system and not hinder their problem

101 solving while using the system. They utilized the ’Dress Rehearsal’ area to familiarize themselves with the gestures. This part of the system allowed students to select a gesture to practice and perform the gesture until they gained system recognition. For the actual evaluation portion of the experiment, each student was given access to the system for 20 minutes per session for a maximum of 9 different sessions. Because the nature of both of the established programs in which the study was ran under, attendance was not mandatory. All 20 participants were not present for each of the nine sessions, but the participants in the experimental group whose data is analyzed in subsequent sections of this chapter, were present for all nine sessions. Each session was held on a different day and was reduced from the original 30 minutes to 20 minutes to allow travel time for students from their after school tutoring classrooms. Students in the experimental group arrived at the study area in three separate groups of four and proceeded to their individual station. Students used the ’Run Through’ area in the system during each session. This section produced randomized math questions for each student to solve using gestures. Each of the gesture pictures were provided for students at all times and hints were given based on input by the students. The control group completed the same randomized problems each day that the experimental group were given using the Qualtrics survey software for 20 minutes each session. Their sessions were completed within their 21st Century classrooms. 7.7 Results

IBM SPSS statistical software was used to conduct repeated measures analysis of variance (ANOVA) testing for each of the dependent variables (DVs): raw math Score, weighted math score, total motivation and the four motivational subsets (Attention, Relevance, Confidence, Satisfaction). The within-subjects factor was the time factor as the assessments were administered to measure the aforementioned variables twice; before and after the conclusion of the experiment. Between subject factors/independent variables (IVs) of Group (experiment vs. control), Gender (male vs. female), Grade (6th vs. 7th) and Performance Level (low, average or high) were included to determine if there were any significant effects based on

102 these groupings. The students were placed in their performance groups based on their weighted math score on the pretest.Low performers consisted of those who scored less than 8 based on the weighted score, while mid or average performers scored exactly 8. High performers score greater than 8 for their weighted scorer. This measure was used due to Virginia’s Department of Education incorporating the same technique when determining students’ scores on the SOLs. On the SOLs, the difficulty level of the questions are taken into account, along with howmany questions were answered correctly. They utilize an adaptive form of testing where questions are based on previous answered questions’ difficulty levels. The reported means are estimated marginal means, which are the means calculated by SPSS which are adjusted for all variables included in each model. For all of the ANOVAs performed, a test was first conducted featuring all of the IVs in the model. Non-significant IVs were removed individually until the simplest model was created for each DV. This chapter will only report on those effects that resulted in significant differences between the experiment and control groups. All other significant findings are reported in the Appendix. 7.7.1 Raw Score

The raw score was measured by totalling the amount of questions answered by each participant, with each question being valued at 1. Each participant for given the same assessment with 20 questions. For questions that were open ended or allowed for multiple responses, participants received some credit (0.5) if the answer was deemed partially correct by the researcher. The initial model when analyzing the effect of the IVs on the raw score from pre to posting time, revealed a non-significant effect (p=0.236) for the Performance Level. This factor was removed and a new model was produced. This resulted in a significant main effect for the three remaining IVs as well as several significant interactions. The main effectfor Group was significant, F(1,8)=12.229, p=0.008, such that the control group (M=9.906) scored higher than the experimental group (M=7.781) when considering performance at both pre and post testing times.

103 There was a significant interaction effect of Group and Grade on the raw math scoresof participants; F(1,8) = 7.151, p=0.28. Simple main effects analysis, plotted in Fig. 7-3, showed that for all 7th grade students, control students (M=12.250) garnered higher raw scores than experiment students (M=8.5)( p=0.05), but there was no significant effect between groups for 6th grade students (p=.514).

Figure 7-3. Group * Grade Interaction

There was another significant interaction effect of Gender, Group and Grade ontheraw math score; F(1,8)=6.612, p=.033. For 7th grade female students, the students in the Control group (M=15.750) scored higher than those in the experiment group (M=10)(p=0.004). There was no significant effect between the 7th grade male students (p=.205) nor 6th grade female (p=.419) or male students (p=.236) in regards to the group placement. The graphs of these interactions for the female and male students are shown in Figures 7-4 and 7-5, respectively. There was a significant interaction between Time, Gender and Group; F(1,8)=9.804, p=.014. Pairwise comparisons indicated that during the pre-testing time (1), female students in the control group (M= 12.437) scored higher than female students (M=9) in the experiment group (p=0.019). Fig. 7-6 has the plot of this interaction for the female students. There was

104 Figure 7-4. Gender * Group * Grade Interaction - Female

Figure 7-5. Gender * Group * Grade Interaction - Male also an indication that males in the control group (M = 12.75) scored significantly higher than males in the experiment group (M=7.750) at post-testing time, which is indicated by 2, (p=.007). The plot of this significant interaction is displayed in Fig. 7-7. There was also a significant interaction between all of the remaining IVs: Time, Gender, Group and Grade; F(1,8) = 7.603, p=.025. For 6th grade female students at post

105 Figure 7-6. Group * Gender * Time Interaction - Female

Figure 7-7. Group * Gender * Time Interaction - Male testing time, the experimental students (M=8) scored significantly higher than the control group(M=4.875)(p=.024). For 7th grade female students at pre testing time, control students (M= 17) scored significantly higher than experimental students (M=12)(p=.045). For 7th grade female students at post testing time, control students (M=14.5) scored significantly higher than experimental students(M=8)(p=.020). For 6th grade male students at post testing

106 time, control students (M=14.5) scored significantly higher than experimental students (M= 7.5)(p=.007). 7.7.2 Weighted Score

The weighted score was calculated based on question difficulty level (easy, medium or hard) as defined by a K-12 math teacher as described in Section 7.6.3. As mentioned previously, for questions that were open ended or allowed for multiple responses, participants received half credit. For example, if a student partially answered a hard question, they were awarded 1.5 points since a hard question was worth 3 points. For the initial model with all IVs, only Grade produced a significant effect (p=0.025), in regards to the weighted math scores, during pre and post testing times. The Gender variable was removed due to its high level of significance (p=.516). In the second model, there were no significant effects, soGroupwas removed (p=.924). The last model produced only a significant main effect of Grade; F(1,11) = 6.41, p=.028. 7.7.3 Motivation Score-Overall

The adapted Course Motivation Score encompasses four subsets of motivation including Attention, Relevance, Confidence and Satisfaction. These subsets are designed to measure whether or not the math class in which the technology was utilized in was able to capture and hold the participants’ Attention, appear to be relevant to their needs, bolster their Confidence in their math self-efficacy and their general Satisfaction. This research is interested inallfour facets of the ARCS motivation model and will feature the results of each as well as the total. There were no significant differences for any of the independent variables in regards to thetotal motivation scores of students. 7.7.4 Motivation: Attention

For the initial model with all IVs, the analysis revealed a main effect for the Group variable in regards to the Attention scores, when considering performance at both pre and post testing times; F(1,3)=13.988, p=.033. The experiment group scored higher (M=16.1167) than the control group (M=15.893).

107 7.7.5 Motivation: Relevance

For the initial model with all IVs, there were no significant main effects in regards to the Relevance scores, during pre and post testing times.; The Grade variable was removed from the model due to its high level of significance (p=.177). There were several significant interactions produced by this model. There was a significant interaction between the effects of Performance Level and Treatment group on the Relevance score; F(2,5)=10.311,p=.017. Pairwise comparisons revealed high performers in the experiment group (M=16.875) had higher relevance scores than the high performers in the control group (M=13.33) (p=0.15). There were no significant effects across groups for low performers (p=.088) or average performers (p=.622). This graph for this interaction is shown in Figure 7.-8.

Figure 7-8. AchieveWeighted (Performance Level) * Group Interaction

There was an additional significant interaction between the effects of Group and Gender on the Relevance score; F(1,5)=26.564, p=.004. Pairwise comparisons, as graphed in Fig. 7-9, indicated that males in the experiment group (M=17.875) scored significantly higher than males in the control group (M=14.5)(p=.018).

108 Figure 7-9. Group * Gender Interaction

7.7.6 Motivation: Confidence

There were no significant differences for any of the independent variables in regards tothe Confidence scores of students. 7.7.7 Motivation: Satisfaction

There were no significant differences for any of the independent variables in regards tothe Satisfaction scores of students. 7.8 Makin’ Math Move Performance Data

As mentioned previously, each of the eight students in the experimental group participated in nine, 20 minute sessions interacting with Makin’ Math Move. On average, the students spent 2.21 minutes per problem. Students spent approximately 2.55 minutes on problems that they were able to answer correctly. They spent less time on problems that they eventually skipped; 1.22 minutes per skipped problem. Per session, on average, students answered 4.69 problems correctly and skipped 6.67 problems. Of the 818 problems viewed by all 8 students across all 9 sessions, 338 or 41.3% were answered correctly, while 480 or 58.7% were skipped. There were 9 problem types/format presented to each student. The problems represented single step Algebraic equations for all four of the main math operations (add, subtract, multiply

109 and divide) as well as multi-step expressions that required the order of operations (parentheses, exponents, multiply, divide, add and subtract) to solve. Problem types A and E featured problems with addition, multiplication and division operations. Table 7-2 shows the nine types and an example of each problem.

Table 7-2. Problem Types Featured in Makin’ Math Move Problem Type Quantity Example Problem A 15 A + 4 = 8 B 2 4A = 12 C 3 2 + A <10 D 2 39 = 3A E 9 13 ≤ P + 8 F 4 5 + 32 ÷ 3 G 4 4A = 18 H 4 (23 − 1) ÷ 5 I 4 3 * 3 - 2 + 4

Overall, students had the greatest performance on problem type C. Across all nine sessions, this problem type was displayed 51 times and students correctly solved the problem type 33 times or 64.7% of the time. Students performed most poorly on problem type G. This type was displayed 62 times across the 9 sessions and students only solved it correctly 5 times or 8.06% of the time. Statistical tests were conducted to analyze if there was any significant difference due to gender in the experimental group across the different problem types.The values analyzed represented the success rate of the participants which was calculated by dividing the amount of times they answered the specific problem type correctly by how many times they were shown the problem type across the experimental period. All of the problem types indicated no significant difference between the genders, with the exception ofType C: Type A (t=-1.0361, p=0.3401), Type B t=-0.21818, p=0.8345, Type D (t=-0.54732, p=0.6039), Type E (t=-0.93877, p=0.3841), Type F (t=-1.1366, p=0.2991), Type G (t=-0.55944,p=0.5961), Type H (t=-1.66778, p=0.1524), and Type I (t=-0.2514, p=0.8099). The means and standard deviations for Type C are shown in Figure 7-10. The variances were determined to be equal (Fig. 7-11) and the two tailed t test (t= -3.24195, p=0.0176)

110 suggested that there was a significant difference between the success rate of male and female participants. As seen in Fig. 7-12, the lower t-tailed test (t=-3.24194, p=0.0008) suggested that the success rate of the female students was significantly higher than the males for Type C.

Figure 7-10. Means and Standard Deviations - Problem Type C

Figure 7-11. Variance Equality Test- Problem Type C

Figure 7-12. t-Test - Problem Type C

7.9 Observations and Post-Survey Responses

Math Related On Day 1 of the experimental sessions, observations indicated that students were struggling with interacting with Makin’ Math Move. After speaking with a few students,

111 it was clear that the students knew how to solve some of the problems, they just did not remember the problem solving steps that were featured in the how-to- video. Signs were made after the first day to remind students of the steps necessary to use the system. Thesigns had the following information: 1) Select a number 2) Perform an operation (add, subtract, multiply or divide) 3) Select another number and 4) Solve. The inclusion of the steps/rules was useful in eliminating one area of difficulty in this new math intervention for these students. Students may have already had difficulties with solving math problems traditionally; the signs eliminated the additional difficulty of learning how to use a new system. This was oneoftwo means of incorporating scaffolding into the experimental group’s experience. Scaffolding was also incorporated via visual feedback during their problem solving experience. As mentioned previously, the hints incorporated into the system were specific to the types of errors made. For example, if students performed the wrong gesture for a single-step algebraic equation, the system prompted them with a question : ”What is the opposite of [insert operation featured in the problem]?” This was done to remind students that solving single-step equations requires the application of the operation opposite to what is given in the problem. The hints guide students in the right direction without directly giving them the correct answer. It was also observed that some students had preferences for specific types of problems and would skip other problems because they were fully confident in their ability to solve those problems. On more than one occasion, students were observed counting on their fingers or struggling to add single digit numbers when trying to solve the problem. One student “drew” out numbers when counting using the table which the system was set up on. Two students mentioned the use of paper during the experiment. One male said that he did better on paper, while a female mentioned that she could counted on paper; at the time these comments were made, students were solving one digit operation problems/steps (e.g. 16-8, 12 * 2). Students also experienced difficulty which resulted in longer problem solving times or completely skipping the problems in particular when subtracting two two-digit numbers. Often times, students knew how to solve the problems, but were selecting operands out of order. The hints, such as “Is

112 this the number you should be operating on?” often guided them back on track. The errors exhibited in the evaluation when students encountered order of operations problems matched the misconceptions define in section 2.1 for that concept area. Outside of just attempting to solve the problem from left to right, students also had trouble when dealing with multiplication and division or addition and subtraction in the same problem. They tended to perform the operation in the order they appear in the acronym, PEMDAS, despite the rule being that one performs the operation within the pairs (multiplication and division being a pair and addition and subtraction being the other) in order of their occurrence from left to right. System Use and Features In the post-survey administered to the experimental group, three students indicated that they would like to use the system at home, while another three said they would use it during math class; two students preferred the use during an after school program. When asked what was their favorite part of the system, 37.5% of the responses included the mention of moving and/or dancing and another 37.5% included the word ”fun.” For their least favorite aspect, some responses were in regard to the system’s recognition, the math/fractions and ”moving my arm too much.” When asked what they would change, three students indicated nothing; the other responses varied and included : “the projector,” “more music,” “different device,” and “everything.” A majority of the students (62.5%) both enjoyed the music and indicated that they felt like they were dancing. Students seemed to enjoy the avatars in the system. One male student was often noticed just moving his character around as opposed to solving problems and had to be reminded to stay on track. There were mixed reviews about the music/use of headphones. Due to the style of headphones, some students did not like to keep them in consistently. When asked students said it was not due to the music, just the headphones. Physicality: System Recognition There were some complaints regarding the system’s recognition. Often times, the remedy was to remove loose clothing (baggy sleeves on jackets sometimes caused interference) or

113 just have the students repeat the operation until it was recognized. Students also became frustrated when entering at the answers at times. If students held their hand over the number pad for too long, it would add the same digit again to the answer box (e.g if a student wanted to enter ’2’, ’22’ would appear). The issue was simply fixed by deleting what was in the answer box and entering the number again. A few students realized that if they closed their hand after selecting the number once, the system would not use their hand as a selector any longer. The Kinect was set to recognized an open right hand as a cursor; therefore a balled fist would not be accepted by the Kinect as input. Post Motivation Related Questions Students were asked additional questions in terms related to motivation on how they felt specifically about the technology as opposed to their experience in the math classafter using the system. They were asked four additional questions across the four motivational aspects using the same 5 point Likert scale. They ranked the system the highest in terms of Satisfaction and Relevance with 70% of the possible 40 points. Students gave the system 24 out of 40 points or 60% in terms of capturing their Attention. In terms of Confidence, students felt as though the system was more difficult to use than necessary and that question resulted in 18 out of 40 points. 7.10 Discussion

Despite there being no significant main effect between the experimental and control groups over time on their math or motivational scores, the results indicate that there were certain groups of students who seemed to have benefited from the experiment. Many of the findings indicated that the control group performed better when averaging their pre andpost testing findings. However, it is important to note that these findings were not significant in regards to time. The control group, as a whole, did not significantly perform better than the experimental group in those measures from pre to post testing time. Results did indicate that the intervention seemed to have an effect on the 6th grade female population. There were an equal number (N=4) of female 6th grade students in both

114 the control and experimental groups. The female students who interacted with the system answered approximately 49% more questions at post testing time than those who practiced math problems on the computer. This is finding is very meaningful considering that overall female students in the control group had significantly higher scores than female students in the experiment group at pre-testing time. This indicates that although Makin’ Math Move was designed to enhance the academic ability of all African American students, it could potentially be more beneficial those females students who are just beginning coursework featuring pre-algebraic content. Although not significant, there were other findings worth reporting in regardsto motivational impacts of Makin’ Math Move on the experimental group as a whole. In terms of Attention, the scores increased 8.96% for experimental students, while control students saw a decrease of 3.15% from pre to post testing time. Experimental students in all three performance levels also saw an increase in Attention scores from pre to post testing time, with the largest increase occurring for high performers. These students in the experiment group saw a 13.05% increase in Attention score whereas the high performers in the control group’s Attention scores decreased by 14.93%. This suggested that the use of a different interactive interface could have played a role in captivating and sustaining the students attention. The administration stated that students at the Greater Richmond Area middle school frequently used technology in the learning process, but was usually in the form of computers and tablets. Despite this being an upgrade from the traditional pencil and paper method, the results from this research suggests that this era of digital natives are ready for and could benefit from further technological advances being incorporated in their learning experiences. With the exception of average performing students in the control group, all performance levels in both group either saw no change or decrease in change from pre to post testing times in regards to Relevance. This could be potentially linked to the timing of the experiment. Initially, the experiment was to be conducted early in the Spring semester of the school year to avoid end of year testing and spring holidays. Due to administrative delays with the

115 Greater Richmond Area middle school, the SOL testing window fell during the period that the experiment occurred. Students participation was uninterrupted, however the final week of the experiment (and the administration of the post-test) occurred after testing had been completed. Students made comments about why they still had to come since school was over and wondered why they were still practicing math. It is believed that they felt the system was no longer important or relevant to their needs due to the conclusion of the testing period and impending end of the school year. 7.11 Summary

To summarize, Makin’ Math Move was evaluated for its impact on math performance and motivation over the course of nine sessions by 6th and 7th grade African-American students. Students from two locations in Florida and Virginia participated to form the experimental and control group which consisted of 8 students each. Students in the control group utilized laptops to complete pre-algebra problems in their after-school program classrooms, while the experimental group utilized Makin’ Math Move for 20 minutes. Before and after the experimental period, both groups took a math assessment (based on Virginia’s end of the year assessments) and an adapted Course Motivation Survey to measure their thoughts towards their performance in their math classroom in the areas of attention, relevance, confidence and satisfaction. Results from statistical testing indicated that there were no significant differences between the groups at pre and post testing for any of the measures. Albeit not significant, the experimental group also experienced an increase of 8.96% on the Attention subscale during the experimental period. This indicates that the use of this new technology did have an captivating and attention-holding impact that was not experienced by students who simply completed math problems on a laptop. While students reported that they were not always happy with the responsiveness of the recognition capabilities, they also reported that they had fun using the system and would like to use it at home as well as in the classroom. Chapter 8 will discuss how Makin’ Math Move can be improved upon as well as other studies that can be conducted to examine other possible effects.

116 CHAPTER 8 SUMMARY AND FUTURE WORK In this research, a culturally relevant full body interactive learning environment was developed with the help of and assessed by its intended target audience of African-American 6th and 7th grade students. In this final chapter, a summary of the work completed, aswell as its results, is given. The research questions initially introduced in Chapter 1 will be restated along with contributions made in various research areas. It will conclude with the limitations of this work, any future plans for this research space and will finally end with conclusions. 8.1 Summary and Research Questions

Makin’ Math Move was designed with assistance of, developed for and evaluated by African-American 6th and 7th grade students. The research was designed to answer the following questions in regards to this learning tool :

• To what extent is the proposed system usable, in terms of system usability and gesture recognition accuracy, by 6th and 7th grade African-American students?

• Is the system found to be culturally relevant by 6th and 7th grade African-American students?

• To what extent are 6th and 7th grade African-American learners more motivated in pre-algebra problem solving when using Makin’ Math Move as compared to traditional practice?

• To what extent do 6th and 7th grade African-American learners improve pre-algebra problem solving performance after using Makin’ Math Move as compared to traditional practice? Throughout two of the studies described in Chapters 6 and 7, each of the stated questions were addressed. The complete usability study featured in detail in Chapter 6 concluded that according to the results of the SUS, the system was below average in terms of usability; despite this score, a second interpretation of SUS scores adjectively ranked the system between “Ok” and “Good.” The system was not able to recognize the targeted recognition rate of 94% of gestures at the time of the usability study, but the recognition system was sufficient for students in the final evaluation. They were able to solve the math problems provided

117 by Makin’ Math Move. While there is no standard measurement of cultural relevance, the results indicated that system was greater than or above average in terms of measures of stimulus preferences (verve) and awareness of musical/movement expressiveness. There was no statistically significant results when comparing the experimental and control groups, as awhole, after the experimental period, however 6th grade female students did significantly improve their raw score and the experiment group, as a whole, improved in the Attention facet of motivation measure. 8.2 Contributions

The contributions of this work are:

• A gestural database was created with 149 recorded clips using Microsoft’s Visual Gesture Builder. Each clip varied in length and contained anywhere from 20 to 150 total tags per clip. This gestural database consisted of both mathematical and functional/navigational gestures suggested by 20 African-American middle school students. This database can be used as a basis for future development of math based gestural games using the Kinect for Windows software.

• Makin’ Math Move, a culturally relevant full body interactive learning environment, was designed (via participatory design), developed and evaluated. The learning environment used African-American cultural ethos of movement and verve to aid African American middle schoolers in practicing pre-Algebraic skills. It incorporated embodied interaction to go beyond the traditional use of technology in the learning experience to capture and sustain the attention of the targeted demographic. Makin’ Math Move has the ability to be customized to include different cultural gestures and/or cultural audio. Results suggest that the use of the technology created a larger change in math performance indicators when assessing students before and after technology use as well as captured the attention of students.

• Qualitative data taken from interviews during the usability study suggested that the current System Usability Scale may not be appropriate for students at the middle school level. While students could read all of the survey questions, students were not able to comprehend the meaning of all of the words. The results of the SUS did not align with the students’ satisfaction and assessment of the system. This suggests that an adapted SUS survey should be developed for students at various reading comprehension levels. 8.3 Discussion, Limitations and Future Work

When attempting to design for specific cultures, whether it be by race or generation, it is important to realize that there may be design choices that will conflict with other design

118 methods. For this system, other design considerations were primarily taken from designing for children and gestural interfaces. With regards to designing and defining the gesture set, this system utilized a gesture elicitation study to garner gestures which were reflective of cultural, symbolic, deictic and arbitrary gestures. In order to define the gesture set, methods traditionally used in gesture interface design, agreement and max consensus, were utilized. However, the use of these appropriate and justifiable interface design methods resulted in alack of cultural relevance via the gestures used in the system. Students performed many cultural gestures (dances) throughout the elicitation, but none of the gestures had a consensus for any one referent/task. One way of mitigating this specific conflict, which could be incorporated in future versions of Makin’ Math Move, would be allowing the user to choose which gestures they want to represent each function/operation for the system. Students would have the opportunity to select from the gesture chosen by max consensus and one or two cultural gestures. One thing to note is that cultural dances frequently change; from the period when the gesture elicitation was conducted to the publication of this dissertation, many of the dances performed were ”out of style.” Future versions of this system and other systems that wish to include this aspect of culture will have to be mindful of the fast-paced changing nature of what is popular and enjoyed by the targeted demographic. In regards to student participation in the final evaluation of the system, initially there was a mix of apprehension and excitement to use the system. Some students saw the Kinect and immediately thought they were pulled out of class to play a game so they were eager to play. Other students appeared to not want to try anything new; they did not see the point of getting pulled out of class. As the experiment went on and students got the hang of how to operate the system, they appeared to be enthusiastic about using the system. While most did not mind using the system for all nine sessions, students sometimes asked if they were going to do the same type of thing every day. Although the system previously ranked slightly above average in terms of verve variability (the amount of variety in activities) , it may have been useful to assess the cultural measures investigated in this research summatively, as well as formatively.

119 When originally measured during the usability study, students only interacted with the system for a short period of time and were only asked to do specific tasks. They were not engaged as frequently as those students in the final evaluation who interacted with the problem solving mode for nine sessions. Although the exact terms in the questions were different for each session, the problems still followed the same format. While it did not seem to bother some students, those who posed the question could have wanted another activity that would still allow them to practice the same content. Additional problem solving activities could be added to future versions of Makin’ Math Move to add variability and increase the stimulating nature. This research utilized a small scale usability and evaluation study. Although participants from two different locations were used, this would not be considered a multi-classroom study. Also, it was the intent to have more experimental participants, but because this was an in the wild study and occurred after school hours, participation was not mandatory and attendance during the days of the experiment could not be enforced. The work is also limited because of the lack of support for the Kinect for Windows software. Microsoft ended production of the V2 sensors in 2015, but developers were able to continue development using the V2 sensor or the XBox One sensor with the purchase of an adapter. There has not been any recent updates to the software and its recognition capabilities since November 2018 and it is unlikely that Microsoft will offer much more developmental support5 [ ]. This research could also be limited due to the novelty of the Kinect. Some students expressed their eagerness vocally to not just be using the computers to do math. As the research in this area advances, several changes and additional steps could be employed. Multiple studies like the one conducted here could be performed, in addition to studies being conducted in different environments at different times during the school year. While the students at the first school were spread out in a multipurpose room with enough distance to not be in each others’ recognition space, they sometimes interfered if they left the room for water or a bathroom break. This study may also yield different results, particularly on the motivational aspects, if conducted completely prior to end of year assessments. This

120 study could be conducted on students of all ethnicities to determine the role of the cultural aspects on students’ performance and motivation. The study could also be adapted to different content; other areas essential to Algebra I success or other math areas, in general, could be formatted to utilize the same standard mathematical gestures. An additional study could also be conducted for a longer experimental period or as an in-class learning tool. Makin’ Math Move could serve as a reward or learning station within the classroom and evaluated in that aspect. While the Kinect SDK offers two applications that greatly assist in the gesture training and database building process, there were often times that the reflected accuracy for each individual gesture did not match the accuracy experienced by students. The VGB has a Live Preview function that allows developers to test the accuracy of individual or all gestures that apart of a database. The Live Preview includes charts that rise and fall as the confidence in the system in recognizing the gesture increases and decreases. It was important to use the feature for each gesture as well as for the entire database; when testing the entire database, the Live Preview shows one screen with one chart that corresponds to each gesture in the database. Testing each gesture against the whole database of gestures in the Live Preview led to the realization that some gestures were being recognized by one or more referents (i.e. the gesture for ‘Add’ was initially often mistaken for ‘Multiply’). Due to this, referents had to be retrained for false-positives. In the case of Add gestures being recognized as Multiply, clips of the Add gesture were added to the individual database for Multiply. Instead of tagging each occurrence of the Add gesture as True, it was marked as False; meaning if the gesture is performed, it would not trigger as a correct gesture for Multiply. The retraining for false positives, while necessary, added multiple hours to the training process. Despite the training for true and false positives and training for slight variations in gesture poses, students still did not always trigger the recognition of the operations the first time around. The Kinect is able to map each the skeletons of each individual user to a general skeleton that is mapped to the gestures trained in the database. This means that a small

121 child performing the Multiply gestures (arms forming an X across the chest) should trigger the same response from the system as a tall man performing the same pose. The gesture requires the same body parts which are recognized on each individual. Although the Live Preview consistently worked for the researcher who trained all of the gestures, it was less consistent for the participants. Some students had to attempt a gesture two to three times before the system recognized it. Although the trainer ideally should not have made a difference, it would be interesting to see if including the targeted demographic in the training would affect the consistency of the recognition. While this could improve recognition, this could be a very time consuming effort. The training of this system was a very lengthy process and conducted whenever the researcher saw fit; this included recording and tagging clips throughout various hours of the day. It would take much longer to conduct this process with students due to the restricted access, in regards to time, researchers would have with them. The means of gestural recognition could also be changed, such as utilizing Leap Motion software, instead of the Kinect to capture participants’ gestures. However, if a different technology such as Leap Motion was incorporated into the system, additional gesture elicitation studies may need to be performed based on the capabilities of the software; Leap appears to only track the hands which would limit the gestures using other body parts. Additional software would also be needed to render an avatar since the Kinect had skeletal tracking that was able to function with avatar rendering tools such as Fuse and Blender. 8.4 Conclusion

Although there were no significant findings between the control and experimental groups as determined by statistical testing, the experimental students, overall, saw a slight increase in the Attention facet of the motivational assessment. These finding indicate that there is potential for further research with this system and within this area. Observed conversation by the students as well as elicited post-survey responses indicated that students did enjoy using Makin’ Math Move and that they would like to use it again. Despite some frustration due to the system’s timeliness in recognizing certain movements performed by the users, results

122 suggested that Makin’ Math Move was able to capture the attention of students and they had fun with the embodied style of learning. With the conductance of future studies that incorporate improvement in the areas of gestural training and potentially the method of gesture recognition, Makin’ Math Move has the capacity to be an effective, energetic and embodied learning environment.

123 APPENDIX A TAXONOMIC BREAKDOWN BY INDIVIDUAL REFERENTS

Figure A-1. Breakdown of Add Gestures

Figure A-2. Breakdown of Subtract Gestures

Figure A-3. Breakdown of Multiply Gestures

124 Figure A-4. Breakdown of Divide Gestures

Figure A-5. Breakdown of Input Gestures

Figure A-6. Breakdown of Home Gestures

125 Figure A-7. Breakdown of Help Gestures

Figure A-8. Breakdown of Pause Gestures

Figure A-9. Breakdown of Next Gestures

126 APPENDIX B CULTURAL RELEVANCE B.1 Questionnaire for Stimuli Preference (Verve)

Verve Stimulus Intensity (liveliness):

1. When you used the system, did you think the interface was

(a) boring and dull (b) plain and ok to look at (c) exciting and energetic

2. Compared to normal pencil and paper practice was this system,

(a) more boring (b) about the same (c) more exciting

Verve/Stimulus Variability:

3. When practicing math with this system, would you prefer:

(a) to just solve the problems (b) see an example and then solve the problems (c) see an example, practice some problems and then get quizzed at the end

4. When using the system did you:

(a) think that the one kind of problem solving activity was ok (b) Wish there were two different problem solving activities (ex. fill in the blankand solve an equation) (c) Wish there many different problem solving activities

Verve/Stimulus Density:

5. When using the system, did you think

(a) there were not enough features on the screen (b) there were just enough features on the screen (c) there were too many features on the screen

127 6. How would you prefer to use this system?

(a) alone in a room (b) playing with another person on their own system (c) in a room with many people doing other things

B.2 Child Activity Questionnaire : Movement and Music Subscale

Students were asked to answer these questions using a Likert scale from 1 (Almost Never) to 5 (Almost Always):

1. How often did the music affect your problem solving moves?

2. How often do you think you would have been able to dance/ perform the gestures without music

3. How often would you have been able to be still (not move) with the music playing?

128 APPENDIX C MATH BENCHMARK

1. At the beginning of the school year, Tyrell bought 8 notebooks for a total price of $20.56. Janelle went to the same store and bought 5 notebooks. How much did Janelle Spend?

(a) $2.57 (b) $8.24 (c) $12.85 (d) $10.28

2. Each of Mrs. Johnson’s 16 students ate 3/8 of a pizza. How many pizzas did they eat, altogether?

(a) 48 (b) 6 (c) 3 (d) 12

3. Dave bought 3 1/ 2 gallons of gasoline to use in his lawnmower. If he uses 3 /4 gallon each time that he mows the yard, how many times can he mow the yard before he runs out of gasoline?

(a) 2 (b) 3 (c) 4 (d) 5

4. Jake bought 4 of the same candy bars for a total of $6.68. Jessie bought one of the same candy bars and paid with a $5. How much change did she receive?

(a) $2.34 (b) $1.68 (c) $3.33 (d) $4.58

5. Jared rode his bike for 3.2 miles on day 1 and 2.6 miles on day 2. How many miles would he have to ride on day 3 to reach 10 miles total.

129 (a) 5.8 miles (b) 4.2 miles (c) 3.7 miles (d) 1.5 miles

6. Evaluate 12/6 + 2(3+4)

(a) 16 (b) 28 (c) 2 (d) 18

7. Using the order of operations, what is the second operation that should be performed in the problem below : 2 2 + (10-6) * 3

(a) 2 2 (b) 10-6 (c) 4 * 3 (d) 4 + 4

8. Which of the following has a value of -2

(a) (5+3) /4 (b) 8 - 1 * 6 (c) 2 - 4 * 1 (d) 15/3 -7

9. What is the value of 32 - 4 +6?

(a) 10 (b) -1 (c) 8 (d) 11

10. What is the value of 2 + 5 * 2?

(a) 14

130 (b) 12 (c) 9 (d) 5

11. Explain how to solve the algebraic equation: p+4= 12

12. How would you solve the equation: 1/3x = -2

(a) Multiply both sides of the equation by 1/3 (b) Multiply both sides of the equation by 3 (c) Divide both sides by 3 (d) Divide both sides by 2

13. Which solution will make the linear equation statement true: 15.5 = -2z?

(a) z = 7.75 (b) z = 31 (c) z = 17.5 (d) z = -7.75

14. Which method can be used to solve the algebraic equation: z-7=13?

(a) Subtract 7 from both sides of the equation (b) Add 7 to both sides of the equation (c) Subtract 13 from both sides of the equation (d) Add 13 to both sides of the equation

15. Select the two methods that can be used to solve the algebraic equation: -2x=14

(a) Divide each side by 2 (b) Divide each side by 14 (c) Multiply each side by -1/2 (d) Add -2 to both sides (e) Subtract 14 from both sides

16. Solve the one-step linear inequality: 3+ x <7

(a) x >4

131 (b) x <10 (c) x <4 (d) x >10

17. What is the solution to 13 ≤ p - 8 ?

(a) 21 ≤ p (b) 5 ≤ p (c) p ≤ 5 (d) p ≤ 21

18. Select all of the numerical values that would make the inequality statement true : A - 8 <12

(a) 20 (b) 19 (c) 22 (d) 7

19. Solve the one step linear inequality: x- 6 ≥ 5

(a) x >11 (b) x <11 (c) x >1 (d) x <1

20. What is the solution to b + 3 ≤ 8

(a) b ≤ 24 (b) b ≤ 5 (c) b ≤ 11 (d) b ≥ 11

C.1 Course Motivation Survey

132

Course Motivation Survey (CMS)

Instructions

1. There are 20 statements in this questionnaire. Please think about each statement in relation to the mathematics class that you are about to participate in and indicate how true the statements are using the scale provided after each statement. Give the answer that truly applies to you and not what you would like to be true, or what you think others want to hear.

2. Think about each statement by itself and indicate how true it is. Do not be influenced by your answers to other statements.

3. Circle the number that best indicates your response, and follow any additional instructions that may be provided in regard to the answer sheet that is being used with this survey. Be sure to circle a number. DO NOT circle any space between the numbers.

Scale for Your Responses 1 (or A) = Not true 2 (or B) = Slightly true 3 (or C) = Moderately true 4 (or D) = Mostly true 5 (or E) = Very true

119

133 Course Motivation Survey (CMS)

Name: Teacher: Class:

Please remember to circle a number. DO NOT circle any space between numbers.

1. I think this mathematics class will be challenging, but neither too easy, nor too hard for me.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

2. There is something interesting about this mathematics class that will capture my attention.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

3. This mathematics class seems more difficult than I would like for it to be.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

4. I believe that completing this mathematics class will give me a feeling of satisfaction.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

5. It is clear to me how this mathematics class is related to things I already know.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

6. I believe this mathematics class will gain and sustain my interest.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

7. I believe that the information contained in this mathematics class will be important to me.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

8. As I learn more about this mathematics class, I am confident that I could learn the content.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

9. I believe that I will enjoy this mathematics class so much that I would like to know more about this topic.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

10. The mathematics class seems dry and unappealing.

1------2------3------4------5

120

134 Not true Slightly true Moderately true Mostly true Very true

11. The mathematics class is relevant to my interests.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

12. It is apparent to me how people use the information in this mathematics class.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

13. I will really enjoy completing assignments for this mathematics class.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

14. After working on this mathematics class for awhile, I believe that I will be confident in my ability to successfully complete all class assignments and requirements.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

15. I think that the variety of materials, exercises, illustration, etc., will help keep my attention on this mathematics class.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

16. The technology that will be used to deliver this mathematics class may be frustrating/irritating.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

17. It will feel good to successfully complete this mathematics class.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

18. The contents of this mathematics class does not include information that will useful to me.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

19. I do NOT think that I will be able to really understand the information in this mathematics class.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

20. I do not think that this course will be worth my time and effort.

1------2------3------4------5 Not true Slightly true Moderately true Mostly true Very true

121

135 Overview of the ARCS Model

Summary of the ARCS Model

The ARCS model, developed by Keller (1987a, 1987b), provides a systematic process for analyzing student motivation and designing motivationally effective instruction. It also helps to organize knowledge of human motivation. He argues that the plethora of constructs related to human motivation makes it difficult for practitioners to transfer theory into practice. To develop a comprehensive measure of learners’ motivation, educators would have to apply a battery of tests which is not practical in most instructional situations. By synthesizing the various theories of human motivation, Keller has constructed a model, with related instruments that allow researchers and practitioners to form a comprehensive profile of learners’ situational motivation.

Keller posits that theories of human motivation may be subsumed under four general categories: A--Attention, R--Relevance, C-Confidence, and S--Satisfaction. In order to motive students to learn, instruction must: (1) gain and sustain learners attention; (2) be relevant to their needs; (3) promote learners confidence in their ability to succeed; and (4) satisfy learners (e.g., results were worth time and effort). There are a number of concepts related to each major category. The following is a list of concepts related to each category, along with corresponding theories of human motivation.

Attention - To motivate students to learn, instruction must gain and sustain attention.

A1. Perceptual Arousal - Stimulate senses A2. Inquiry Arousal - Stimulate curiosity A3. Variability - Vary stimulus

Theoretical Foundations - Curiosity - Perceptual Arousal - Inquiry Arousal

Relevance - To motivate students to learn, instruction must be relevant to their needs.

R1. Goal Orientation - Help students create and achieve goals R2. Motive Matching - Address specific needs R3. Familiarity - Relate to learners' past experiences

Theoretical Foundations - Drive Theories - Needs Hierarchy - Need for Achievement

Confidence - To motivate students to learn, they must have confidence in their ability to succeed.

122

136

C1. Learning Requirements - Awareness of expectations and evaluation criteria. C2. Success Opportunities - Opportunities to experience success. C3 Personal Control - Link success or failure to student effort and abilities.

Theoretical Foundations - Self-efficacy - Locus of Control - Learned Helplessness Satisfaction - To motivate students to learn, learners must be satisfied that the results of instruction were worth their time and effort.

S1. Natural Consequences - Meaningful opportunities to apply learned skills S2. Positive Consequences - Positive reinforcement S3. Equity - Consequences perceived to be fair by all students

Theoretical Foundations - Conditioning Theory - Cognitive Evaluation Theory

Purpose of the CMS

The Course Motivation Survey is intended to be a situational measure of students’ perceived levels of motivation toward a course. It is based on Keller’s’ Instructional Materials Motivation Survey (IMMS) that assess learners’ motivation reaction to specific instructional materials.

The CMS and IMMS are designed in accordance with the theoretical foundation represented by the ARCS Model (Keller, 1987a, 1987b). This theory is derived from the current literature on human motivation, hence, many of the items in the CMS are similar in intent (but not in wording) to items established measures of psychological constructs such as need for achievement, locus of control, and self-efficacy, to mention three examples.

CMS Scoring Guide

The response scale ranges from 1 to 5. This means that the minimum score on the 20 item survey is 20, and the maximum score is 100 with a midpoint of 60. The minimums, maximums, and midpoints for each subscale are comparable because they have the same number of items.

An alternative scoring method is to find the average score for each subscale and the total scale instead of using sums. For each respondent, divide the total score on a given scale by 5 (the number of items in that scale). This converts the totals into a score ranging from 1 to 5 and makes it easier to compare performance on each of the subscales.

123

137 APPENDIX D ADDITIONAL SIGNIFICANT FINDINGS FROM EVALUATION

• Raw Score The main effect for the participants’ gender was significant, F(1,8) = 9.521, p=.015, such that females students (M=9.781) scored higher on the raw score than male students (M=7.906) when considering the performance on the pre and post tests. The main effect for Grade was also significant, F(1,8)=25.399, p=0.001, such thatthe 7th graders (M=10.375) scored higher than the 6th graders (M=7.312).The ANOVA revealed that there was a significant interaction effect of Gender and Grade on theraw math scores of participants; F(1,8)=26.446, p=.001. Simple main effects analysis showed that for all 7th grade students, female students (M=12.87) garnered higher averaged raw scores than male students(M=7.875)( p=0.01), but there was no significant effect between genders for 6th grade students (p=.126). The graph of this interaction can be seen in Fig. D-1.

Figure D-1. Gender * Grade Interaction

There were also significant findings with regards to students’ raw score measures from pre to post testing time. There was a significant interaction for the effect of Gender on time; F(1,8) = 24.079, p=.001. Pairwise comparisons indicate that female students (M=10.719), regardless of treatment group, scored higher than male students (M = 5.562) at pre-testing times (p=0.00).The graph of this interaction can be seen in Fig. D-2.

• Weighted Score There were no additional significant findings for the weighted score measure.

• Motivation-Overall There were no significant findings for the overall motivation measure.

138 Figure D-2. Time * Gender Interaction

• Motivation-Attention There was a significant main effect for the performance level (AchieveWeighted), F(1,2)=26.403, p=.012). Post hoc analysis using the Tukey test revealed that there was a significant difference between the Attention means of the low (M=17.5) and average (M=14) ranked students when considering performance during both testing times(p=.030)

• Motivation-Relevance The model used revealed a significant main effect for the Performance Level variable; F(2,5)=7.783, p=.029. Post hoc analysis using the Tukey test revealed that there was a significant difference between the Relevance means of the low (M=17.875) and average (M=14.5) ranked students (p=.033). There was also a significant difference between the Relevance means of the low (M=17.875) and high (M=15.104) ranked students (p=.033). There was a significant interaction between the effects of performance level and gender on average Relevance score F(2,5)=10.150 (p=.017). Pairwise comparisons revealed that females in the low performance group (M= 18.750) scored higher in Relevance than females in the average (mid) performance group (M=13.5) (p=.013). The graph of this interaction can been seen in Fig. D-3. There was also a significant effect of gender on Relevance score over time; F(1,5)=7.049, p = .045 .Pairwise comparisons indicated that at post-testing time, female students (M=17.11) had higher relevance scores than male students(M=14.2) (p=.024). These results are graphed in Figure D-4.

• Motivation-Confidence There were no significant findings for the Confidence measure.

• Motivation-Satisfaction

139 Figure D-3. Gender * AchieveWeighted (Performance Level) Interaction

Figure D-4. Gender * Time Interaction

There were no significant findings for the Satisfaction measure.

140 REFERENCES [1] [n. d.]. https://csdt.rpi.edu/culture/cornrowcurves/index.html [2] [n. d.]. Arcademic Skill Builders - Fun Educational Games for Kids. http://www. arcademics.com [3] 2016. . http://howtobecabincrew.com/wp-content/uploads/2016/04/5D__9412_ 10x15-2-350x600.jpg [4] 2017. DimensionU. http://www.dimensionu.com [5] 2018. https://www.microsoft.com/en-us/download/details.aspx?id=57578&WT.mc_ id=rss_alldownloads_devresources [6] 2019. https://learningworksforkids.com/playbooks/dance-central-spotlight/ [7] 2019. https://store.ubi.com/us/just-dance--2019/5b0bf6046b54a435c7d9139e.html? lang=en_US [8] 2019. http://schoolquality.virginia.gov/schools/carter-g-woodson-middle# [9] Clark C Abt. 1975. Serious games. Viking Press. [10] Janet Ainley, Liz Bills, and Kirsty Wilson. 2004. Constructing meanings and utilities within algebraic tasks. In Proceedings of the 28th conference of the International Group for the Psychology of Mathematics Education, Vol. 2. 1–8. [11] LR Albert. 2000. Lessons learned from the” Five Men Crew”: Teaching culturally relevant mathematics. Changing the faces of mathematics: Perspectives on African Americans (2000), 81–88. [12] Martha W Alibali and Mitchell J Nathan. 2012. Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. Journal of the learning sciences 21, 2 (2012), 247–286. [13] Stamatina Anastopoulou, Mike Sharples, and Chris Baber. 2011. An evaluation of multimodal interactions with technology while learning science concepts. British Journal of Educational Technology 42, 2 (2011), 266–290.

141 [14] Alissa N Antle, Milena Droumeva, and Greg Corness. 2008. Playing with the sound maker: do embodied metaphors help children learn?. In Proceedings of the 7th interna- tional conference on Interaction design and children. ACM, 178–185. [15] Robert B Ashlock. 2006. Error patterns in computation: Using error patterns to improve instruction (9 ed.). Pearson/Merrill Prentice Hall. [16] Robert Atanda. 1999. Gatekeeper Courses. National Center for Education Statistics 1, 1 (1999), 33. [17] John W Atkinson and George H Litwin. 1960. Achievement motive and test anxiety conceived as motive to approach success and motive to avoid failure. The journal of abnormal and social psychology 60, 1 (1960), 52. [18] Kathryn Au, Cathie Jordan, et al. 1981. Teaching reading to Hawaiian children: Finding a culturally appropriate solution. Culture and the bilingual classroom: Studies in classroom ethnography (1981), 139–152. [19] Nuray Aykin. 1999. Internationalization and localization of the Web sites. In Proceedings of HCI International (the 8th International Conference on Human-Computer Interaction) on Human-Computer Interaction: Ergonomics and User Interfaces-Volume I-Volume I. L. Erlbaum Associates Inc., 1218–1222. [20] Christine M Bahr and Herbert J Rieth. 1989. The effects of instructional computer games and drill and practice software on learning disabled students’ mathematics achievement. Computers in the Schools 6, 3-4 (1989), 87–102. [21] Haiyan Bai, Wei Pan, Astusi Hirumi, and Mansureh Kebritchi. 2012. Assessing the effectiveness of a 3-D instructional game on improving mathematics achievement and motivation of middle school students. British Journal of Educational Technology 43, 6 (2012), 993–1003.

142 [22] Caryn Bailey. 1997. Physical stimulation and cognitive performance of low-income African American and European American school children: Conceptual, motivational, and practical considerations for educational research and practice. (1997). Presented at the annual meeting of the American Educational Research Association, Chicago, IL. [23] Caryn Bailey. 1999. Psychometric soundness of home-culture socialization, cultural orientation and learning preference measures. (1999). Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada. [24] Albert Bandura. 1982. Self-efficacy mechanism in human agency. American psychologist 37, 2 (1982), 122. [25] Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies 4, 3 (2009), 114–123. [26] Wendy Barber. 1998. BA: 1998, Culturability: The merging of culture and usability. In 4th Conference on Human Factors and the Web. 2–10. [27] Arthur J Baroody and Herbert P Ginsburg. 1983. The effects of instruction on children’s understanding of the” equals” sign. The Elementary School Journal 84, 2 (1983), 199–212. [28] Amy Baylor and Yanghee Kim. 2003. The role of gender and ethnicity in pedagogical agent perception. In E-Learn: World Conference on E-Learning in Corporate, Govern- ment, Healthcare, and Higher Education. Association for the Advancement of Computing in Education (AACE), 1503–1506. [29] Katrin Becker. 2009. Video Game Pedagogy. In Games: Purpose and potential in education. Springer, 73–125. [30] Moniruzzaman Bhuiyan and Rich Picking. 2009. Gesture-controlled user interfaces, what have we done and what’s next. In Proceedings of the Fifth Collaborative Research Symposium on Security, E-Learning, Internet and Networking (SEIN 2009), Darmstadt, Germany. 25–29.

143 [31] J Blake. 2012. The natural user interface revolution. Natural User Interfaces in. Net (2012), 1–43. [32] Ian Bogost. [n. d.]. Persuasive Games: Exploitationware. https://www.gamasutra.com/ view/feature/134735/persuasive_games_exploitationware.php [33] Lesley R Booth. 1988. Children’s difficulties in beginning algebra. The ideas of algebra, K-12 19 (1988), 20–32. [34] Christine L Borgman. 1992. Cultural diversity in interface design. ACM SIGCHI Bulletin 24, 4 (1992), 31. [35] A Wade Boykin. 1978. Psychological/behavioral verve in academic/task performance: Pre-theoretical considerations. The Journal of Negro Education 47, 4 (1978), 343–354. [36] A Wade Boykin. 1983. The academic performance of Afro-American children. Achieve- ment and achievement motives (1983), 321–371. [37] A Wade Boykin. 1986. The triple quandary and the schooling of Afro-American children. The school achievement of minority children: New perspectives (1986), 57–92. [38] A Wade Boykin and Caryn T Bailey. 2000. The Role of Cultural Factors in School Relevant Cognitive Functioning: Description of Home Environmental Factors, Cultural Orientations, and Learning Preferences. Report No. 43. (2000). [39] A Wade Boykin, Robert J Jagers, Constance M Ellison, and Aretha Albury. 1997. Communalism: Conceptualization and measurement of an Afrocultural social orientation. Journal of Black Studies 27, 3 (1997), 409–418. [40] Brenda R Brand, George E Glasson, and Andre’M Green. 2006. Sociocultural factors influencing students’ learning in science and mathematics: An analysis ofthe perspectives of African American students. School Science and Mathematics 106, 5 (2006), 228–236. [41] Johannes Breuer and Gary Bente. 2010. Why so serious? On the relation of serious games and learning. Journal for Computer Game Culture 4 (2010), 7–24.

144 [42] B Brubaker. 2006. Teachers join the Dance Dance Revolution: Educators begin training to use the exercise video game. The Dominion Post (2006), B2. [43] Sarah B Bush. 2011. Analyzing common algebra-related misconceptions and errors of middle school students. Ph.D. Dissertation. University of Louisville. [44] Cazden C. and Leggett E. 1981. Culturally responsive education: Recommendations for achieving Lau remedies II. In Culture and the bilingual classroom: Studies in classroom ethnography, Trueba H., Guthrie G., and Au K. (Eds.). 69–86. [45] Ewa Callahan. 2005. Interface design and culture. Annual review of information science and technology 39, 1 (2005), 255–310. [46] Mary Margaret Capraro and Heather Joffrion. 2006. Algebraic equations: Can middle-school students meaningfully translate from words to mathematical symbols? Reading Psychology 27, 2-3 (2006), 147–164. [47] John M Carroll and John C Thomas. 1982. Metaphor and the cognitive representation of computing systems. IEEE Transactions on systems, man, and cybernetics 12, 2 (1982), 107–116. [48] John M Carroll and John C Thomas. 1988. Fun. ACM SIGCHI Bulletin 19, 3 (1988), 21–24. [49] Center for Public Education. 2013. Detail on mathematics graduation requirements from public high schools, by state. [50] Dennis Charsky. 2010. From edutainment to serious games: A change in the use of game characteristics. Games and culture 5, 2 (2010), 177–198. [51] John Clement. 1989. The concept of variation and misconceptions in Cartesian graphing. Focus on Learning Problems in Mathematics 11 (1989), 77–87. [52] Sabrina Connell, Pei-Yi Kuo, Liu Liu, and Anne Marie Piper. 2013. A Wizard-of-Oz elicitation study examining child-defined gestures with a whole-body interface. In Proceedings of the 12th International Conference on Interaction Design and Children. ACM, 277–280.

145 [53] National Research Council, Mathematics Learning Study Committee, et al. 2001. Adding it up: Helping children learn mathematics. National Academies Press. [54] Paul Denny. 2013. The effect of virtual achievements on student engagement. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 763–772. [55] Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From game design elements to gamefulness. Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments - MindTrek 11 (2011). https://doi.org/10.1145/2181037.2181040 [56] Darina Dicheva, Christo Dichev, Gennady Agre, Galia Angelova, et al. 2015. Gamification in education: A systematic mapping study. Educational Technology & Society 18, 3 (2015), 75–88. [57] AdriáN DomíNguez, Joseba Saenz-De-Navarrete, Luis De-Marcos, Luis FernáNdez-Sanz, Carmen PagéS, and José-Javier MartíNez-HerráIz. 2013. Gamifying learning experiences: Practical implications and outcomes. Computers & Education 63 (2013), 380–392. [58] Mary Jo Dondlinger. 2007. Educational video game design: A review of the literature. Journal of applied educational technology 4, 1 (2007), 21–31. [59] Paul Drijvers. 2015. Digital technology in mathematics education: why it works (or doesn’t). In Selected Regular Lectures from the 12th International Congress on Mathematical Education. Springer, 135–151. [60] Allison Druin, Benjamin B Bederson, Juan Pablo Hourcade, Lisa Sherman, Glenda Revelle, Michele Platner, and Stacy Weng. 2001. Designing a digital library for young children. In Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries. ACM, 398–405. [61] Ron Eglash, Audrey Bennett, Casey O’donnell, Sybillyn Jennings, and Margaret Cintorino. 2006. Culturally situated design tools: Ethnocomputing from field site to classroom. American anthropologist 108, 2 (2006), 347–362.

146 [62] Catherine Emihovich and Gloria E Miller. 1988. Effects of Logo and CAI on black first graders’ achievement, reflectivity, and self-esteem. The Elementary School Journal 88, 5 (1988), 473–487. [63] Valerie N Faulkner, Lee V Stiff, Patricia L Marshall, John Nietfeld, and Cathy L Crossland. 2014. Race and teacher evaluations as predictors of algebra placement. Journal for Research in Mathematics Education 45, 3 (2014), 288–311. [64] Florida Department of Education FDOE. 2017. Spring 2017 Florida Standards Assessments Mathematics District Reporting Category Results. [65] John Ferrara. 2012. Playful design: Creating game experiences in everyday interfaces. Rosenfeld Media. [66] Raya Fidel, Rachel K Davies, Mary H Douglass, Jenny K Holder, et al. 1999. A visit to the information mall: Web searching behavior of high school students. Journal of the Association for Information Science and Technology 50, 1 (1999), 24. [67] Donna Y Ford, Tyrone C Howard, J John Harris III, and Cynthia A Tyson. 2000. Creating culturally responsive classrooms for gifted African American students. Journal for the Education of the Gifted 23, 4 (2000), 397–427. [68] Vittorio Gallese and George Lakoff. 2005. The brain’s concepts: The role ofthe sensory-motor system in conceptual knowledge. Cognitive neuropsychology 22, 3-4 (2005), 455–479. [69] Geneva Gay. 2013. Teaching to and through cultural diversity. Curriculum Inquiry 43, 1 (2013), 48–70. [70] Raymond W Gibbs Jr. 2005. Embodiment and cognitive science. Cambridge University Press. [71] Daniel T Gilbert, R Brian Giesler, and Kathryn A Morris. 1995. When comparisons arise. Journal of personality and social psychology 69, 2 (1995), 227.

147 [72] Juan E Gilbert, Keena Arbuthnot, Stafford Hood, Michael M Grant, Melanie LWest, Yolanda McMillian, E Vincent Cross II, Philicity Williams, and Wanda Eugene. 2008. Teaching algebra using culturally relevant virtual instructors. IJVR 7, 1 (2008), 21–30. [73] Lucas B Gillispie. 2008. Effects of a 3-D video game on middle school student achieve- ment and attitude in mathematics. Ph.D. Dissertation. University of North Carolina Wilmington. [74] Arthur M Glenberg, Tiana Gutierrez, Joel R Levin, Sandra Japuntich, and Michael P Kaschak. 2004. Activity and imagined activity can enhance young children’s reading comprehension. Journal of educational psychology 96, 3 (2004), 424. [75] Susan Goldin-Meadow, Susan Wagner Cook, and Zachary A Mitchell. 2009. Gesturing gives children new ideas about math. Psychological Science 20, 3 (2009), 267–272. [76] Susan Goldin-Meadow, San Kim, and Melissa Singer. 1999. What the teacher’s hands tell the student’s mind about math. Journal of educational psychology 91, 4 (1999), 720. [77] Susan Goldin-Meadow, David McNeill, and Jenny Singleton. 1996. Silence is liberating: removing the handcuffs on grammatical expression in the manual modality. Psychological review 103, 1 (1996), 34. [78] Levent Gorgu, Gregory MP O’Hare, and Michael J O’Grady. 2009. Towards mobile collaborative exergaming. In Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services, 2009. CENTRIC’09. Second International Conference on. IEEE, 61–64. [79] Kent L Granzin and Marlys J Mason. 1999. Motivating participation in exercise: Using personal investment theory. ACR North American Advances (1999). [80] Daniela Grijincu, Miguel A Nacenta, and Per Ola Kristensson. 2014. User-defined interface gestures: Dataset and analysis. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces. ACM, 25–34.

148 [81] Lulu Healy and Celia Hoyles. 1999. Visual and symbolic reasoning in mathematics: making connections with computers? Mathematical Thinking and learning 1, 1 (1999), 59–84. [82] G.H. Hofstede. 1997. Cultures and Organizations: Software of the Mind. McGraw-Hill. https://books.google.com/books?id=wFW0AYqIM0AC [83] John P Holdren, ES Lander, Harold Varmus, et al. 2010. Prepare and inspire: K-12 education in science, technology, engineering, and math (STEM) for America’s future. Executive Report). Washington, DC: President’s Council of Advisors on Science and Technology (2010). [84] Juan Pablo Hourcade. 2008. Interaction design and children. Foundations and Trends in Human-Computer Interaction 1, 4 (2008), 277–392. [85] Steven J Ingels, Daniel J Pratt, Deborah R Herget, Laura J Burns, Jill A Dever, Randolph Ottem, James E Rogers, Ying Jin, and Steve Leinwand. 2011. High School Longitudinal Study of 2009 (HSLS: 09): Base-Year Data File Documentation. NCES 2011-328. National Center for Education Statistics (2011). [86] Jacqueline Jordan Irvine. 1990. Black students and school failure. Policies, practices, and prescriptions. ERIC. [87] Jacqueline Jorden Irvine and Beverly Jeanne Armento. 2001. Culturally responsive teaching: Lesson planning for elementary and middle grades. Education Review//Reseñas Educativas (2001). [88] Mizuko Ito. 2006. Engineering Play: Children’s software and the cultural politics of edutainment. Discourse: studies in the cultural politics of education 27, 2 (2006), 139–160. [89] Alexis Jetter. 1993. Mississippi learning. New York Times Magazine 21 (1993), 28–35. [90] Mina C Johnson-Glenberg, David A Birchfield, Lisa Tolentino, and Tatyana Koziupa. 2014. Collaborative embodied learning in mixed reality motion-capture environments: Two science studies. Journal of Educational Psychology 106, 1 (2014), 86.

149 [91] George Ghevarughese Joseph. 1987. Foundations of Eurocentrism in mathematics. Race & Class 28, 3 (1987), 13–28. [92] Marie Joubert. 2013. Using digital technologies in mathematics teaching: developing an understanding of the landscape using three “grand challenge” themes. Educational studies in mathematics 82, 3 (2013), 341–359. [93] Joon Young Kang, Ryunhyung Kim, Hyunsun Kim, Yeonjune Kang, Susan Hahn, Zhengrui Fu, Mamoon I Khalid, Enja Schenck, and Thomas Thesen. 2016. Automated Tracking and Quantification of Autistic Behavioral Symptoms Using Microsoft Kinect.. In MMVR. 167–170. [94] Ali Karime, Basim Hafidh, Abdulmajeed Khaldi, Jihad Mohamad Aljaam, and Abdulmotaleb El Saddik. 2012. MeMaPads: Enhancing children’s well-being through a physically interactive memory and math games. In Instrumentation and Measurement Technology Conference (I2MTC), 2012 IEEE International. IEEE, 2563–2566. [95] Mansureh Kebritchi. 2008. Effects of a computer game on mathematics achievement and class motivation: An experimental study. University of Central Florida. [96] Mansureh Kebritchi, Atsusi Hirumi, and Haiyan Bai. 2010. The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers & education 55, 2 (2010), 427–443. [97] John M Keller. 1987. Development and use of the ARCS model of instructional design. Journal of instructional development 10, 3 (1987), 2. [98] John M Keller. 1987. IMMS: Instructional materials motivation survey. Florida State University (1987). [99] Carolyn Kieran. 1992. The learning and teaching of algebra. Handbook of research on mathematics teaching and learning (1992), 390–419. [100] John Kirriemuir and Angela McFarlane. 2004. Literature review in games and learning. (2004).

150 [101] Eric J Knuth, Ana C Stephens, Nicole M McNeil, and Martha W Alibali. 2006. Does understanding the equal sign matter? Evidence from solving equations. Journal for research in Mathematics Education (2006), 297–312. [102] William H Kraus. 1981. Using a computer game to reinforce skills in addition basic facts in second grade. Journal for research in mathematics education 12, 2 (1981), 152–155. [103] Dietmar Küchemann. 1978. Children’s understanding of numerical variables. Mathemat- ics in school 7, 4 (1978), 23–26. [104] Gloria Ladson-Billings. 1992. Culturally relevant teaching: The key to making multicultural education work. Research and multicultural education: From the mar- gins to the mainstream (1992), 106–121. [105] Gloria Ladson-Billings. 1995. But that’s just good teaching! The case for culturally relevant pedagogy. Theory into practice 34, 3 (1995), 159–165. [106] Gloria Ladson-Billings. 1997. It doesn’t add up: African American students’ mathematics achievement. Journal for Research in Mathematics education 28, 6 (1997), 697–708. [107] Susan J Lamon. 2012. Teaching fractions and ratios for understanding: Essential content knowledge and instructional strategies for teachers. Routledge. [108] Richard N Landers. 2014. Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming 45, 6 (2014), 752–768. [109] JA Large and Jamshid Beheshti. 2005. Interface design, web portals, and children. Library Trends 54, 2 (2005), 318–342. [110] Jacqueline Leonard, Wanda Brooks, Joy Barnes-Johnson, and Robert Q Berry. 2010. The nuances and complexities of teaching mathematics for cultural relevance and social justice. Journal of Teacher Education 61, 3 (2010), 261–270. [111] Wei Li, Tovi Grossman, and George Fitzmaurice. 2012. GamiCAD: a gamified tutorial system for first time autocad users. In Proceedings of the 25th annual ACM symposium on User interface software and technology. ACM, 103–112.

151 [112] Liora Linchevski and Drora Livneh. 1999. Structure sense: The relationship between algebraic and numerical contexts. Educational studies in mathematics 40, 2 (1999), 173–196. [113] Kimberly Ling, Gerard Beenen, Pamela Ludford, Xiaoqing Wang, Klarissa Chang, Xin Li, Dan Cosley, Dan Frankowski, Loren Terveen, Al Mamunur Rashid, et al. 2005. Using social psychology to motivate contributions to online communities. Journal of Computer-Mediated Communication 10, 4 (2005), 00–00. [114] Chen-Chung Liu, Wen-Chi Chiou, Shu-Ju Tai, Chin-Chung Tsai, Gwo-Dong Chen, Chen-Wei Jong, and Baw-Jhiune Liu. 2006. Wristbands as interaction devices: A vision-based interaction space for facilitating full-body learning. In 2006 Fourth IEEE International Workshop on Wireless, Mobile and Ubiquitous Technology in Education (WMTE’06). IEEE, 171–173. [115] Tom Loveless. 2013. The algebra imperative: assessing algebra in a national and international context. (2013). [116] Joan Lucariello and David Naff. 2013. How do I get my students over their alternative conceptions (misconceptions) for learning. Removing barriers to aid in the development of the student (2013). [117] Adriana Moreno Magallanes. 2003. Comparison of Student Test Scores in a Coordinate Plane Unit Using Traditional Classroom Techniques Versus Traditional Techniques Coupled with an Ethnomathematics Software at Torch Middle School. (2003). [118] Laura Malinverni and Narcis Pares. 2014. Learning of abstract concepts through full-body interaction: A systematic review. Journal of Educational Technology & Society 17, 4 (2014), 100. [119] Thomas W Malone. 1981. Toward a theory of intrinsically motivating instruction. Cognitive science 5, 4 (1981), 333–369. [120] Aaron Marcus. 2005. User interface design and culture. Usability and internationalization of information technology 3 (2005), 51–78.

152 [121] Danny Bernard Martin. 2009. Liberating the production of knowledge about African American children and mathematics. Mathematics teaching, learning, and liberation in the lives of Black children (2009), 3–38. [122] James Frank Marty. 1985. Selected effects of a computer game on achievement, attitude, and graphing ability in secondary school algebra. Ph.D. Dissertation. [123] Jennifer A Mautone, George J DuPaul, and Asha K Jitendra. 2005. The effects of computer-assisted instruction on the mathematics performance and classroom behavior of children with ADHD. Journal of Attention Disorders 9, 1 (2005), 301–312. [124] Marc McDonald, Robert Musson, and Ross Smith. 2008. Using productivity games to prevent defects. The Practical Guide to Defect Prevention, Microsoft Press, Redmond (2008), 79–95. [125] Jane McGonigal. 2011. Reality is broken: Why games make us better and how they can change the world. Penguin. [126] Joanna McGrenere. 1996. Design: Educational electronic multi-player games a literature review. Technical Report. Department of Computer Science, University of British Columbia, Vancouver, British Columbia, V6T 1Z4. [127] Nicoleta Mihali. 2014. Gestenerkennung mit Kinect und Visual Gesture Builder. https://blogs.msdn.microsoft.com/the_language_of_technology/2014/10/22/ gestenerkennung-mit-kinect-und-visual-gesture-builder/ [128] Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77, 12 (1994), 1321–1329. [129] Gerald Mohatt, Frederick Erickson, et al. 1981. Cultural differences in teaching styles in an Odawa school: A sociolinguistic approach. Culture and the bilingual classroom: Studies in classroom ethnography 105 (1981). [130] Meredith Ringel Morris, Andreea Danielescu, Steven Drucker, Danyel Fisher, Bongshin Lee, Jacob O Wobbrock, et al. 2014. Reducing legacy bias in gesture elicitation studies. Interactions 21, 3 (2014), 40–45.

153 [131] Sean Murphy. 2014. Design considerations for a natural user interface (NUI). Texas Instruments Incorporated, Dallas, TX, Retrieved 11 (2014). [132] Megan Murray, Jan Mokros, and Andee Rubin. 1999. Mathematically rich, equitable game software. Mathematics Teaching in the Middle School 5, 3 (1999), 180. [133] Na’ilah Suad Nasir. 2002. Identity, goals, and learning: Mathematics in cultural practice. Mathematical thinking and learning 4, 2-3 (2002), 213–247. [134] Valerie Nesset and Andrew Large. 2004. Children in the information technology design process: A review of theories and their applications. Library & Information Science Research 26, 2 (2004), 140–161. [135] Kelley Newlin, Kathleen Knafl, and Gail D’Eramo Melkus. 2002. African-American spirituality: A concept analysis. Advances in Nursing Science 25, 2 (2002), 57–70. [136] Michael Nielsen, Thomas B. Moeslund, Moritz Storring, and Erik Granum. [n. d.]. Gesture Interfaces. Elsevier. [137] Donald A Norman et al. 1988. The psychology of everyday things. Vol. 5. Basic books New York. [138] Mohammad Obaid, Markus Häring, Felix Kistler, René Bühling, and Elisabeth André. 2012. User-defined body gestures for navigational control of a humanoid robot. In International Conference on Social Robotics. Springer, 367–377. [139] National Council of Teachers of Mathematics. 2000. Principles and standards for school mathematics. Vol. 1. National Council of Teachers of. [140] Commonwealth of Virginia Board of Education. 2016. Mathematics Standards of Learning for Virginia Public Schools. [141] Zühal Okan. 2003. Edutainment: is learning at risk? British Journal of Educational Technology 34, 3 (2003), 255–264. [142] Jumpido OOD. 2013. Jumpido. http://www.jumpido.com/en [143] Eleanor Wilson Orr. 1997. Twice as less: Black English and the performance of Black students in mathematics and science. WW Norton & Company.

154 [144] Zhigeng Pan, Adrian David Cheok, Hongwei Yang, Jiejie Zhu, and Jiaoying Shi. 2006. Virtual reality and mixed reality for virtual learning environments. Computers & graphics 30, 1 (2006), 20–28. [145] Marco Pasch, Nadia Bianchi-Berthouze, Betsy van Dijk, and Anton Nijholt. 2009. Movement-based sports video games: Investigating motivation and gaming experience. Entertainment Computing 1, 2 (2009), 49–61. [146] Jacob C Perrenet and Miriam A Wolters. 1994. The art of checking: A case study of students’ erroneous checking behavior in introductory algebra. The Journal of Mathematical Behavior 13, 3 (1994), 335–358. [147] Marc Prensky. 2001. Digital natives, digital immigrants part 1. On the horizon 9, 5 (2001), 1–6. [148] Luis Radford. 2000. Signs and meanings in students’ emergent algebraic thinking: A semiotic analysis. Educational studies in mathematics 42, 3 (2000), 237–268. [149] Luis Radford. 2009. Why do gestures matter? Sensuous cognition and the palpability of mathematical meanings. Educational Studies in Mathematics 70, 2 (2009), 111–126. [150] Rabindra Ratan and Ute Ritterfeld. 2009. Classifying serious games. Serious games: Mechanisms and effects (2009), 10–24. [151] John W Rice. 2007. Assessing higher order thinking in video games. Journal of Technology and Teacher Education 15, 1 (2007), 87. [152] Ganit Richter, Daphne R Raban, and Sheizaf Rafaeli. 2015. Studying gamification: the effect of rewards and incentives on motivation. In Gamification in education and business. Springer, 21–46. [153] Margaret Gwendoline Riseborough. 1981. Physiographic gestures as decoding facilitators: Three experiments exploring a neglected facet of communication. Journal of Nonverbal Behavior 5, 3 (1981), 172–183.

155 [154] Albert Ritzhaupt, Heidi Higgins, and Beth Allred. 2011. Effects of Modern Educational Game Play on Attitudes towards Mathematics, Mathematics Self-Efficacy, and Mathematics Achievement. Journal of Interactive Learning Research 22, 2 (2011), 277–297. [155] Margaret Robertson. [n. d.]. Can’t play, won’t play. http://hideandseek.net/2010/10/ 06/cant-play-wont-play/ [156] Rinat B Rosenberg-Kima, E Ashby Plant, Celeste E Doerr, and Amy L Baylor. 2010. The influence of computer-based model’s race and gender on female students’ attitudes and beliefs towards engineering. Journal of Engineering Education 99, 1 (2010), 35–44. [157] Andee Rubin. 1999. Technology Meets Math Education: Envisioning a Practical Future Forum on the Future of Technology in Education. (1999). [158] Jaime Ruiz, Yang Li, and Edward Lank. 2011. User-defined motion gestures for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 197–206. [159] Sameer Sahasrabudhe, SHAH Adeet, Mohini THAKKAR, Varun THAKKAR, and IYER Sridhar. 2012. MathMazing: 3D gesture recognition exergame for arithmetic skills. In Proceedings of the 20th International Conference on Computers in Education. [160] Maha Salem, Katharina Rohlfing, Stefan Kopp, and Frank Joublin. 2011. A friendly gesture: Investigating the effect of multimodal robot behavior in human-robot interaction. In RO-MAN, 2011 IEEE. IEEE, 247–252. [161] Mike Scaife and Yvonne Rogers. 1999. Kids as informants: Telling us what we didn’t know or confirming what we knew already. The design of children’s technology (1999), 27–50. [162] Noah Schaffer. 2008. Heuristic evaluation of games. K. Isbister and N. Shaffer, Game Usability. Morgan Kaufman, Amsterdam et al (2008), 79–89.

156 [163] Amy Scheuermann and Delinda van Garderen. 2008. Analyzing Students’ Use of Graphic Representations: Determining Misconceptions and Error Patterns for Instruction. Mathematics Teaching in the middle school 13, 8 (2008), 471–477. [164] Julia Schwarz, Charles Claudius Marais, Tommer Leyvand, Scott E Hudson, and Jennifer Mankoff. 2014. Combining body pose, gaze, and gesture to determine intention to interact in vision-based interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3443–3452. [165] Katie Seaborn and Deborah I Fels. 2015. Gamification in theory and action: A survey. International Journal of human-computer studies 74 (2015), 14–31. [166] James A Shepperd. 2001. Social loafing and expectancy-value theory. In Multiple perspectives on the effects of evaluation on performance. Springer, 1–24. [167] Ben Shneiderman and Catherine Plaisant. 2004. Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th Edition). Pearson Addison Wesley. [168] Ang Chee Siang and Radha Krishna Rao. 2003. Theories of learning: a computer game perspective. In Fifth International Symposium on Multimedia Software Engineering, 2003. Proceedings. IEEE, 239–245. [169] Kurt Squire. 2003. Video games in education. In International journal of intelligent simulations and gaming. Citeseer. [170] Kaye Stacey. 1989. Finding and using patterns in linear generalising problems. Educa- tional Studies in Mathematics 20, 2 (1989), 147–164. [171] Kaye Stacey and Mollie MacGregor. 1997. Ideas about symbolism that students bring to algebra. The Mathematics Teacher 90, 2 (1997), 110–113. [172] Kaye Stacey and Mollie MacGregor. 1999. Learning the algebraic method of solving problems. The Journal of Mathematical Behavior 18, 2 (1999), 149–167. [173] Detmar Straub, Karen Loch, Roberto Evaristo, Elena Karahanna, and Mark Srite. 2002. Toward a theory-based measurement of culture. Human factors in information systems 10, 1 (2002), 61–65.

157 [174] Niara Sudarkasa. 1998. Interpreting the African heritage in Afro-American family organization. Families in the US: Kinship and domestic politics (1998), 91–104. [175] Tarja Susi, Mikael Johannesson, and Per Backlund. 2007. Serious games: An overview. [176] Jane O Swafford and Cynthia W Langrall. 2000. Grade 6 students’ preinstructional use of equations to describe and represent problem situations. Journal for Research in Mathematics Education (2000), 89–112. [177] Malcolm Swan. 2000. Making Sense of Algebra. Mathematics teaching 171 (2000), 16–19. [178] Dean Takahashi. 2008. Funware’s threat to the traditional video game industry. https: //venturebeat.com/2008/05/09/funwares-threat-to-the-traditional-video-game-industry/ [179] WF Tate. 2002. African American children and algebra for all. Improving schools for African American students: A reader for educational leaders (2002), 147–157. [180] William F Tate. 1994. Race, retrenchment, and the reform of school mathematics. The Phi Delta Kappan 75, 6 (1994), 477–484. [181] William F Tate. 1995. Returning to the root: A culturally relevant approach to mathematics pedagogy. Theory into practice 34, 3 (1995), 166–173. [182] Marion Tellier. 2008. The effect of gestures on second language memorisation byyoung children. Gesture 8, 2 (2008), 219–235. [183] The Education Trsut. 2014. The State of Education for African American Students. [184] Jenifer Tidwell. 1999. Common ground: A pattern language for human-computer interface design. [185] Linda C Tillman. 2002. Culturally sensitive research approaches: An African-American perspective. Educational Researcher 31, 9 (2002), 3–12. [186] Jacqueline Urakami. 2012. Developing and testing a human-based gesture vocabulary for tabletop systems. Human factors 54, 4 (2012), 636–653.

158 [187] U.S. Department of Education, Institute of Education Sciences, National Center for Education, National Assessment of Educational Progress (NAEP). 2015. 2015 Mathematics Assessment. [188] Zalman Usiskin. 1988. Conceptions of school algebra and uses of variables. The ideas of algebra, K-12 8 (1988), 19. [189] John A Van de Walle, KS Karp, and JMB Williams. 2010. Elementary and middle school mathematics. Teaching development. New York: Pearson. [190] Joëlle Vlassis. 2008. The role of mathematical symbols in the development of number conceptualization: The case of the minus sign. Philosophical Psychology 21, 4 (2008), 555–570. [191] Lynn A Vogt, Cathie Jordan, and Roland G Tharp. 1987. Explaining school failure, producing school success: Two cases. Anthropology & Education Quarterly 18, 4 (1987), 276–286. [192] Patrick Wachira and Jared Keengwe. 2011. Technology integration barriers: Urban school mathematics teachers perspectives. Journal of Science Education and Technology 20, 1 (2011), 17–25. [193] Jill Walston and Jill Carlivati McCarroll. 2010. Eighth-Grade Algebra: Findings from the Eighth-Grade Round of the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K). Statistics in Brief. NCES 2010-016. National Center for Education Statistics (2010). [194] Elizabeth Warren. 2003. The role of arithmetic structure in the transition from arithmetic to algebra. Mathematics Education Research Journal 15, 2 (2003), 122–137. [195] Marley W Watkins. 1986. Microcomputer-based math instruction with first-grade students. Computers in human behavior 2, 1 (1986), 71–75. [196] Rachael M Welder. 2006. Prerequisite knowledge for the learning of algebra. In Conference on Statistics, Mathematics and Related Fields, Honolulu, Hawaii.

159 [197] Rachael Mae Welder. 2007. Preservice elementary teachers’mathematical content knowledge of prerequisite algebra concepts. Ph.D. Dissertation. Montana State University Bozeman. [198] Daniel Wigdor and Dennis Wixon. 2011. Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier. [199] Margaret Wilson. 2002. Six views of embodied cognition. Psychonomic bulletin & review 9, 4 (2002), 625–636. [200] Jacob O Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A Myers. 2005. Maximizing the guessability of symbolic input. In CHI’05 extended abstracts on Human Factors in Computing Systems. ACM, 1869–1872. [201] Jacob O Wobbrock, Meredith Ringel Morris, and Andrew D Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1083–1092. [202] Donna Lee Wong and Connie M Baker. 1988. Pain in children: comparison of assessment scales. Pediatr Nurs 14, 1 (1988), 9–17. [203] Gabe Zichermann and Christopher Cunningham. 2011. Gamification by design: Imple- menting game mechanics in web and mobile apps. ” O’Reilly Media, Inc.”.

160 BIOGRAPHICAL SKETCH Tiffanie Ra’Chawn Smith was born in Richmond, VA and is the third child ofYolanda Smith. She was the 2009 salutatorian of Varina High School in Henrico, VA. She completed her undergraduate degree in computer engineering with a 4.0 in 2013 at North Carolina Agricultural and Technical State University in Greensboro, NC. In 2019, she completed her Doctor of Philosophy degree in human centered computing, under the advisement of Dr. Juan E. Gilbert, in the Computer and Information Science and Engineering Department in the University of Florida’s Herbert Wertheim College of Engineering.

161