Knowledge Assessment in Statics: Concepts Versus Skills

Session 1168

Knowledge Assessment in Statics: Concepts versus skills

Scott Danielson

Arizona State University

Abstract

Following the lead of the physics community, engineering faculty have recognized the value of good assessment instruments for evaluating the learning of their students. These assessment instruments can be used to both measure student learning and to evaluate changes in teaching, i.e., did student-learning increase due different ways of teaching. As a result, there are significant efforts underway to develop engineering subject assessment tools. For instance, the Foundation Coalition is supporting assessment tool development efforts in a number of engineering subjects. These efforts have focused on developing “concept” inventories. These concept inventories focus on determining student understanding of the subject’s fundamental concepts.

Separately, a NSF-supported effort to develop an assessment tool for statics was begun in the last year by the authors. As a first step, the project team analyzed prior work aimed at delineating important knowledge areas in statics. They quickly recognized that these important knowledge areas contained both conceptual and “skill” components. Both knowledge areas are described and examples of each are provided. Also, a cognitive psychology-based taxonomy of declarative and procedural knowledge is discussed in relation to determining the difference between a concept and a skill.

Subsequently, the team decided to focus on development of a concept-based statics assessment tool. The ongoing Delphi process to refine the inventory of important statics concepts and validate the concepts with a broader group of subject matter experts is described. However, the value and need for a skills-based assessment tool is also recognized. Thus, initial efforts to delineate concepts and skills in statics are discussed, including an initial inventory of these concepts and skills.

Introduction

Statics is the first course in a series of courses within the broader subject area commonly referred to as engineering science for virtually all engineering and engineering technology students. It is a fundamental course prerequisite for other important courses like dynamics and strength of 1 materials. Success in these latter courses is directly correlated to success in statics. Page 9.834.1

Proceedings of the 2004 American Society of Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education Demonstrated proof of student learning and mastery of engineering concepts is now required in the Accreditation Board of Engineering and Technology’s (ABET) outcomes-based environment 2. Tools are needed for assessment of individual student learning. Faculty, departments, and institutional administrators all receive benefits from timely measurement of student learning—immediately upon completing the course. Second, engineering faculty need validated instruments for formative use in assessing implementation of new course design strategies and instructional practices intended to increase student learning. Following the lead of the physics community, the bulk of the effort has focused on concept inventories. Typically, these concept inventories focus on determining student understanding of the subject’s fundamental concepts. For instance, the Force Concept Inventory (FCI) was developed and fielded by physics educators. 3 The FCI probes student understanding and misunderstandings of Newtonian physics, primarily as applied to Newton’s Second Law, i.e., things in motion, and has been widely used by physics educators to assess student learning. In engineering education, the Foundation Coalition is currently supporting concept inventory development efforts in a number of engineering subjects—materials, dynamics, mechanics’ of materials—but not statics.

A National Science Foundation-supported effort to develop an assessment tool for statics is underway by the author and co-principal investigators. The specific goals of this project are to articulate the concepts and knowledge areas essential to statics, create a set of questions probing those concepts, accomplish field-testing and analysis to validate the questions and instrument, and to disseminate the instrument nationally. This paper reports the progress toward the first of these goals.

Initial Compilation of Statics Knowledge Areas

An initial list of statics’ knowledge areas had been previously developed by the project co- principal investigators (Danielson and Mehta) and is shown in Table 1. 4 This original list was broken into three categories: fundamental laws, their corollaries and related knowledge. Here, the word “concept” was used in an applied sense. While the most basic concepts of mechanics may be argued to be space, time, mass, and force, initial thoughts were focused on defining the concepts of statics as areas of knowledge critical to mastery of statics. It was, and remains, the intent to develop knowledge areas and tools independent of any particular textbook and applying to virtually any statics course. Table 1. Original Statics Concept Taxonomy Fundamental Concepts Newton’s First Law: equilibrium Newton’s Third Law: action and reaction Concepts that are corollaries to the fundamental concepts Equilibrium of the whole and parts (assuming no deflections) Free Body Diagrams Particle and rigid body analyses External reactions Friction

Moment Page 9.834.2 Center of Gravity

Proceedings of the 2004 American Society of Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education Moment of Inertia Cables, Springs, and Pulleys Related knowledge (includes both concepts and skills) Vector Mathematics: addition, unit vectors, dot product, cross product, reduction of vector expressions to a scalar form Integral Calculus (used for understanding distributed loads, centroids, etc.) Weight as a Force (using W = mg) Right-Handed Coordinate Systems Units and Calculations (significant digits, etc.)

For the purposes of statics, the First Law’s statement of static equilibrium and the Third Law’s statement about the forces of action and reaction between things in contact were viewed as fundamental. The second set of items was originally posited as corollaries of Newton’s First and Third Laws. For instance, Newton’s Third Law is the foundation for applying tools like free body diagrams and understanding external support reactions. It was felt that the concept of a free body diagram was critical to success in virtually all statics problems.

The Related Knowledge items in Table 1 consisted of both conceptual knowledge and skills, often intertwined together. For instance, implementing the concept of the moment of a force in a problem is often linked to the mathematical skill of performing a vector cross product. In addition, all of these knowledge areas are often portrayed as classic statics problems (e.g., two- dimensional equilibrium, or force resolution and manipulation).

Thus, as a first step in establishing the concepts and knowledge areas essential to statics, the original taxonomy of concept and knowledge areas shown in Table 1 was reviewed by eight other engineering mechanics’ educators. This team (listed in the acknowledgements) formed a diverse and experienced group of educators.

After email exchanges, the group met in person to discuss the project and this list of knowledge areas. During the discussion, the group decided that skills should not be a part of a concept test. Thus, each item on the original list was reviewed with the goal of eliminating skill-based items (analysis, problem solving or mathematics). However, the group also felt that since skills are also important, they agreed that fielding a concept inventory might not be sufficient for mechanics educators, as a separate skill-based test would also be useful.

Then the group used their judgement to remove from the list what seemed like skill-oriented items, leaving a shorter, more distilled concept list. In addition, the group made a conscious decision to focus on concepts as used or taught in the majority of statics classes. This meant that some aspects of our concept list were not universal to mechanics, merely to statics. For instance, while the assumption of equilibrium with no deflection is not valid for all of mechanics, it is almost universal within the realm of statics’ classes. The resulting concept list is shown in Table 2. However, the group decided that a Delphi process 5 would be a valuable way to obtain and validate consensus about both the concepts and skills of statics.

Page 9.834.3

Table 2. Statics Concepts after First Review (Pre-Delphi)

Newton’s First Law: equilibrium Newton’s Third Law: action and reaction “Nature” of force(s)—but only contact forces—to include parallelogram law (resultants) Equilibrium of the whole and parts (assuming no deflections) Free Body Diagrams Particles and rigid bodies External reactions (which may involve pins, cables, springs, pulleys, etc.) Friction Moment of a force Weight as a Force Distributed forces Center of Gravity (as applied to distributed loads) Statically indeterminate situations (recognition level, not solving)

Concept versus Skill

As mentioned above, most engineering educators are working on developing concept inventories based on the approach of the Force Concept Inventory (with the goal of probing student grasp of fundamental concepts and their misconceptions of those same concepts). However, the team involved here believes, as do other teams developing engineering concept inventories 6, that a skills-based instrument has value. Unfortunately, in many engineering subjects the distinction between a concept and a skill becomes difficult to draw, leading to difficulties when developing knowledge inventory questions. Thus, it seems important to develop an understanding of the differences between concepts and skills.

Cognitive psychologists have a rich history in the study of learning, and their literature serves as a starting point in defining these differences. In an edited volume titled “Cognitive Classroom Learning,” Phye and Andre 7 present several chapters which distinguish between aspects of concept and skill learning. Andre points out that the fundamental approach adopted to describe how knowledge is stored is the production system model 8. Simply put, a production system is a network of condition-activation sequences (much like traditional “If ( ), then ( )” statements used in programming). If a condition activates the production system, then a set of activities occur. Production systems can range from simple to complex. One can think of concepts and skills as merely different levels of production systems—one reason it is hard to draw clear distinctions between the two.

Classically, concepts are thought of as rules for classification. 9 Cognitive psychologists typically talk about concepts as schemata (scripts or productions) that serve as the organizational structure for knowledge about the concept. Andre argues that “school-learned” concepts are organized as a set of critical features often accompanied by a prototype. The prototype is used to identify typical instances of the concept, and the features are used when an instance is less familiar. Research 10 indicates that concepts are learned by children in four stages: discrimination of individual instances (features), classifying instances as members (recognizing features), verbally Page 9.834.4

Proceedings of the 2004 American Society of Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education and formally describing the concept, and relating it to other concepts. As indicated above, learning concepts, rules, skills, and problem solving can be thought of as the learners’ development of their mental production systems.

Production system models distinguish between declarative and procedural knowledge. Declarative knowledge is “knowing that” and procedural knowledge is “knowing how.” Or, “knowing that” is conceptual understanding and is represented by being able to describe or talk about the concept. “Knowing how” is the skill component and enables being able to do something. These types of knowledge are not stored differently—both are production systems— but represent different aspects of the knowledge. 8 Note that being able to describe a concept is incorporated into the four stages of children’s concept learning mentioned above.

Thus, using this declarative and procedural knowledge taxonomy, a distinction can be made between concepts and skills during construction of questions for student knowledge inventories. However, the author has not seen reference to, or use of, this taxonomy in relation to knowledge assessment in the engineering education literature. For instance, a problem requiring use of a formula (and subsequent calculation) is probably a skill-based question. Thus, probing a concept may require constructing a question that asks the student to describe the knowledge in some fashion. Perhaps without realizing the psychological basis for it, it is interesting to note that concept inventory developers have tended to use situational questions. The situation posed within the question and answer choices presented to students force them to make decisions based on their ability to describe what ought to be happening as opposed to calculating an answer based on “knowing how.”

Additional work remains to solidify guidelines for constructing questions that address the declarative and procedural knowledge differences within our students. Obviously, from an engineering education viewpoint, we want our students to have command of both concepts (declarative knowledge) and skills (procedural knowledge). Unfortunately, engineering instruction often focuses on procedural knowledge and learning how to calculate does not ensure the acquisition of declarative knowledge. Thus, we are often disappointed by our students’ lack of understanding of the concepts underlying our instructional materials. Thus, in addition to helping construct knowledge inventories, understanding the fundamentals of knowledge types and how they are learned by engineering students should impact on teaching strategies.

Delphi Process for Statics Knowledge Areas

Don Evans, Emeritus Professor at Arizona State University, provided insight into the Delphi process and how it had been conducted for other concept inventory development efforts. Thus, the implementation used for the statics Delphi process is following the general process recommended by Professor Evans. Participants were recruited from a mechanics list serve as well as by personal recommendation from the original team. Approximately 19 additional individuals were recruited. The Delphi process involves multiple rounds of interaction as it drives toward consensus. In this case, round one was completed via email and consisted of an instructional email that also asked participants to return a consent form. The first round instructions asked participants to email descriptions of the concepts in statics that their students

had difficulty understanding (as statics was taught at their institutions). In addition, the Page 9.834.5 participants were asked to separately list statics’ skills that were important or troublesome to

Proceedings of the 2004 American Society of Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education students. This step was completed in late December 2003. In some cases, participants used the recent experience of grading final exams to help them shape their responses.

The original responses from the first Delphi round included 101 items listed as concepts by the contributor and 24 separate items listed as skills. (A casual inspection of the these items indicated that some items listed as concepts by contributors were in fact more “skill based.”) This raw data was “reduced” by grouping similar responses, moving some from the concept category to the skill category, and stripping descriptive words from sentences, etc., to generate a list of phrases. This condensation process resulted in 32 concept items and 43 skill items (the complete list is shown in Appendix A). As expected, the differentiation between concepts and skills in these results is not always stark. This initial condensation process was not accomplished by a group process but will be validated or modified by the second round of the Delphi process. Thus, the concept list includes a number of items that, while suggested by one or more individuals, probably will not appear in the final list of critical statics’ concepts. Anther Delphi round will be used to determine an importance ranking of the concepts and skills generated thus far.

The second round of the Delphi process will ask participants to judge how important it is for their students to understand the various concepts and skills and, secondly, how many of their students understand or can perform the concept/skill. First, the reduced list of 32 concepts and 43 skill phrases in Appendix A were used to develop a technically sound statement (in sentence form) for each item. These statements also represent a compilation of the explanations, etc., that some participants included in their original submissions. As the concept statements were developed, some were ‘subdivided” to ensure one succinct concept per statement. Thus, there are now 47 concept statements being used in the round two of the Delphi process. This round is being accomplished via a dedicated web site.

It is of interest to note that all concepts in Table 2 are also included in the Delphi Round one data but that the Delphi data include a number of potential concepts not included in Table 2. This is one indicator of the value of a Delphi process and its enrichment of the solution space. The importance rankings being gathered via the Delphi process will provide the basis for identifying the key concepts and skills to be probed by the assessment tool questions. These key concepts will be many fewer than the current 47, probably on the order of 10 concepts. A side benefit of this process is that statics educators can potentially use the resulting Delphi importance rankings to guide their coverage of subjects when teaching statics on their own campuses.

Conclusions

One contribution of this paper is the discussion regarding the declarative and procedural knowledge taxonomy. To the author’s knowledge, this has not been discussed in the engineering literature, especially in regard to concept inventory development 11 . Since a number of people are working on concept inventories, it seems important to be able to distinguish a concept from a skill. Otherwise, question construction can easily stray into probing skills when concepts are the goal. While the “I know it when I see it” approach to distinguishing concepts from skills can be used, a more solid foundation is helpful and may guide question construction. However, the

author has not seen reference to, or use of, this cognitive psychology-based taxonomy in relation Page 9.834.6 to knowledge assessment in the engineering education literature.

Proceedings of the 2004 American Society of Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education Table 2 and Appendix A provide a current snapshot of the statics concept and skill identification results obtained to date. These results will continue to refined and validated as the second round of the Delphi process goes forward. The Delphi process has proven to be a process that provides both an enhancement of content and a solid verification of both the content and importance of the statics concepts and knowledge areas deemed important by the engineering educator community. Once completed, these results can also be compared to work done by others in statics. 12

Different from the other engineering subject concept inventories under development, the statics process is including skills as part of the Delphi process. This is in recognition of the importance of skills, both student obtainment and the subsequent measurement of that attainment, to engineering education. The skill results of the Delphi process will be used as the basis for eventual development of a statics skill assessment tool. Such a tool will complement a statics concept inventory.

Acknowledgements

Primary project team members are Sudhir Mehta (also a co-PI on the NSF grant, North Dakota State University), Nels Madsen (Auburn University), Jennifer Kadlowec (Rowan University), Christine Masters (Pennsylvania State University), Brian Self (Air Force Academy), Sally Steadman (University of South Alabama), Don Morris (Virginia Tech), Dave Myszka (University of Dayton), and Mike Magill (George Fox University). Their contributions and guidance are essential to the project’s work.

This work is partially supported by the National Science Foundation through the ASA grant DUE-0206990. This support is gratefully acknowledged.

Bibliography

1. Danielson, S.G., & Danielson, E.B. (1992). Problem solving: Improving a critical component of engineering education. In: Creativity: Educating world-class engineers; Proceedings, 1992 Annual Conference of the ASEE , Vol. 2, June 21-25, Toledo, OH (p. 1313-1317). New York: American Society for Engineering Education. 2. ABET (2000). See http://www.abet.org and follow the accreditation link to both the engineering and engineering technology criteria for the new accreditation criteria. 3. Hestenes, D., Wells, M. & Swackhamer, G. (1992). Force Concept Inventory, The Physics Teacher, 30 , p.141- 158. 4. Danielson, S. & Mehta, S. (2000b). Concept Questions for Improved Learning and Outcome Assessment in Statics. Proceedings of the Frontiers in Education 2000 conference, Kansas City, October 18-21. Available at http://fie.engrng.pitt.edu/fie2000/ . 5. Clayton, M. (1997). Delphi: A technique to harness expert opinion for critical decision-making tasks in education, Educational Psychology, 17 , pp. 373-386. 6. Gray, G., Evans, D., Cornwell, P., Costanzo, F., & Self, B. (2003). Toward a nationwide dynamics concept inventory assessment test. In the Proceedings of the 2003 Annual Conference of the ASEE , Nashville, TN. New York: American Society for Engineering Education. Page 9.834.7

Proceedings of the 2004 American Society of Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education 7. Phye, G. & Andre, T. (1986). Cognitive Classroom Learning: Understanding, thinking, and problem solving . Academic Press, Inc., Harcourt Brace Jovanovich Publishers. New York. 8. Andre, T. (1986). In Phye, G. & Andre, T. Cognitive Classroom Learning: Understanding, thinking, and problem solving , Chapter 7. Academic Press, Inc., Harcourt Brace Jovanovich Publishers. New York. 9. Gagne, R.M. (1977). The conditions of learning . New York: Holt. 10. Tennyson, R.D., & Park, O. (1980). The teaching of concepts: A review of instructional design literature. Review of Educational Research, 50 , p. 55-70. 11. Discussion at the Foundation Coalition’s Concept Inventory Developer’s meeting at the 2003 Frontiers in Education Conference, Boulder CO. 12. Steif, P. Initial data from a statics concept inventory. In press for the Proceedings of the 2004 Annual Conference of the ASEE , Salt Lake, UT. New York: American Society of Engineering Education

Biography SCOTT DANIELSON is the Department Chair of Mechanical and Manufacturing Engineering Technology at Arizona State University. He and Sudhir Mehta are authors of the Statics—The next generation , an instructor’s package published by Prentice-Hall Inc. accompanying the 10 th edition of Engineering Mechanics: Statics by R.C. Hibbeler.

Appendix A The following is a reduced set of the raw data from the first round of the Delphi process. Concepts and skills were reduced to simple phrases. Their order is random and reflects no assumed importance within the study of statics.

Statics Concepts: Equilibrium -- both without motion and with motion Equilibrium -- in two and three dimensions (particles and rigid bodies) Newton's 3rd Law Equivalent or resultant force systems (resultant of a force system) Moment and its direction Couples (2 and 3-Dimensional situations) Weight versus mass Free body diagram [concept] Centroids (first moment of area) Moment of inertia (second moment of area) Reaction/support constraints or forces Internal forces

Zero-force members Page 9.834.8 Two-force members

Proceedings of the 2004 American Society of Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education Vectors and unit vectors Friction and its regimes Indeterminate conditions (i.e., 4 unknowns) Water pressure Fundamental nature of contact forces Mechanical System: the system versus its surroundings Rotation and Angular Momentum Integration (Riemann sum and/or integration) Constraints and the restrictions associated with a particular constraint – the underlying concept for support reactions The three-dimensional world, it visualization and mathematical representation Inertial Reference Frames and Equilibrium Tipping and moving “location” of contact forces Potential Energy Degrees of Freedom Absolute versus Gauge Pressure Virtual Displacements and Virtual Work Sign convention for shear (context of shear and moment diagrams) Distributed Loads

Statics Skills : Visualize 3-D problems Grasping that a moment is a vector per the right hand rule Calculating moments—non-vector See a difference between a torque and a bending moment. Computation using the parallel axis theorem Free Body Diagram construction properly substitute for reaction forces where does a normal force act how are the forces represented on a section what do frames, machines, and wedges have in common Vectors and Vector operations (day not be an issue if vectors are not used or minimized in the statics class offering) Vector dot product Doing a cross product to find the moment about an axis Correct application of the concept of equilibrium: If a system is at equilibrium, then its individual' parts or sections are also at equilibrium and the same equilibrium equations apply. Misapplication or miss-substitution(s) of reaction forces when drawing a free body

diagram Page 9.834.9 Figuring the sense of the moment.

Proceedings of the 2004 American Society of Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education Resolution of vector quantities Finding or calculating equivalent force-couple systems Calculating the Moment of Inertia Calculating the Centroid - Center of Gravity Solving friction problems when the friction force is not equal to the normal force times the static coefficient of friction Understanding the relationship between the centroid and tipping over an object, instead of sliding Calculating the moment of a couple (simple 2-D case) Identifying support reactions based on type of support Determining support reaction using Equations of Equilibrium Identifying zero force members Using method of sections for truss analysis Identifying and using properties of a two-force member Friction around a round object Friction force calculations, e.g., direction and magnitude on a ramp Determining element for integration for Moment of Inertia calculation Trigonometry Integration Solution of simultaneous equations Finding and using unit vectors Resolve force into rectangular components using the unit vector or direction cosine Determine resultant by adding rectangular components of forces Determine type of reaction (force/moment) due to boundary conditions Apply equilibrium equations (sum of F = 0, sum of M= 0) to solve problems Disassemble a frame or truss to determine internal forces in those members. Write a vector from coordinate data (given magnitude) Finding moment based on M = r x F Basics of truss analysis Analyzing frames Computing the tension on the rope when pulleys are involved Generating and solving sets of independent equations Connecting engineering language of statics to everyday language Cutting a beam and using a new FBD for that segment of the beam to develop the shear and moment equations. Understanding forces applied to an object held between the jaws of a tool such as a pliers, wire cutter, crimping tool, etc. Analyze situation when external force applied to a pin connecting three members of a frame Calculate reaction force acting on a frame member when the member is supported by a smooth Page 9.834.10 surface