Paper ID #21494

Assessment of Scientific Literacy Skills and Attitudes of Undergraduate Con- struction Management Students

Dr. Andrea Nana Ofori-Boadu, North Carolina A&T State University

Andrea Ofori-Boadu, Ph.D. Assistant Professor of Construction Management Department of Built Envi- ronment, School of Technology North Carolina A & T State University Address: 112-A Price Hall Phone: 336-285-3128 Email: [email protected] Dr. Andrea Ofori-Boadu is an Assistant Professor of Construction Management (CM) with the Depart- ment of Built Environment. She has a Ph.D. in Technology Management (Specialization in Construction Management), an M.Sc. in Industrial Technology (Specialization in Construction Management), and a B.Sc. in Building Technology. Dr. Ofori-Boadu has over 20 years of relevant academic and industry experience. Her major areas of research are in quality management, sustainable cement replacement materials, and STEM education.

c American Society for Engineering Education, 2018 Assessment of Scientific Literacy Skills and Attitudes of Undergraduate Construction Students Abstract Scientific literacy (SL) skills are critical for technically-trained construction professionals who are capable of transforming built environments through strategic decisions based on evidence- based reasoning. While most undergraduate construction curricula are designed to improve SL, no recent study was found to have assessed undergraduate construction students’ SL skills. Therefore, the purpose of this research was to assess the SL attitudes, skills, and self-efficacy of 46 undergraduate construction students. Data was obtained through students’ completion of the standard 28-item Test of Scientific Literacy Skills (TOSLS) instrument; a student self-reporting SL self-efficacy survey; and focus group discussions. Results revealed that despite the overall mean SL self-efficacy score of 3.42 on the 5-point Likert scale, the mean TOSLS test score was 43%. The mean female TOSLS test score was approximately 7% higher than the mean male score; and the mean test score of senior construction students was 19% higher than that of the freshmen. This was statistically significant (p<0.05) and provided an indication that the construction program had somewhat contributed to the improvement of students’ SL skills; however, additional curriculum improvements are necessary to improve SL skills of construction students. The students’ best test performance was related to identifying valid science arguments and evaluating the misuse of scientific data; while their weakest performance was related to their somewhat weak quantitative aptitude. Their TOSLS problem solving quantitative scores were significantly (p<0.001) lower than TOSLS valid scientific argument scores. These results correlated well with departmental data associated with students’ struggles with courses such as ‘Surveying’ that need advanced quantitative skills. Overwhelmingly, construction students had positive attitudes towards science and mathematics, and agreed that these skills are needed for career success. This study demonstrates the need for curriculum improvements that will strengthen the SL skills of construction students. Furthermore, contributions add value to knowledge base necessary to advance construction education research on scientific literacy skill development. Insights provided may be used to guide construction curriculum improvement, with increased emphasis on quantitative skills for solving real-world problems. Strategies such as tutoring, mathematics laboratories, and math placement tests could increase students’ SL skills and better prepare them for careers in the construction industry. In the long-term, a more SL Built Environment workforce will have improved abilities and be better prepared to make evidence-based decisions that will transform global built environments. Introduction Undergraduate education should provide students with knowledge and experiences that will shape their capability and confidence in scientific reasoning and making evidence-based decisions that will advance progress made by organizations engaged in engineering, construction, infrastructure, and the built environment [1- 4]. The United States is one of the nations that believes that it is important for its leaders and citizens to be scientifically literate. A scientifically literate citizen needs to have: (1) a basic vocabulary of scientific terms and constructs; and (2) a general understanding of the nature of scientific inquiry [5, 6]. The six elements of science literacy are: (a) understanding basic science concepts, (b) understanding nature of science, (c) understanding ethics guiding scientists’ work, (d) understanding interrelationships between science and society, (e) understanding interrelationship between science and humanities, and (f) understanding the relationships and differences between science and technology [7]. The proportion of U.S. adults qualifying as being scientifically literate is now 28% and has doubled over the last two decades, but the current level is still problematic for a democratic society that values citizen understanding of major national policies and participation in the resolution of important policy disputes [8]. Scientific literacy and reasoning ability was found to be a strong predictor of self-efficacy, which is a can-do attitude that allows students to persist and be successful, even under extremely challenging circumstances [9]. Developing formal and informal reasoning ability is a primary factor influencing self-efficacy [9]. The engineering and construction industry will benefit from undergraduate education which prepares students who have high levels of scientific literacy and self-efficacy, and consequently, are able to reason, argue, and make decisions after their evaluation of evidence. The creation of learning environments that develop students’ abilities to reason from evidence and participate in scientific argumentation is considered a priority in education reform [10, 11]. The National Science Foundation and the National Academy of Science recognize the importance of developing flexible frameworks that fosters transformative changes to curriculums, pedagogical approaches and academic culture. The National Research Council emphasizes that science should be nonnegotiable a part of basic education, because some knowledge of science is essential for everyone [12]. Specifically, school science education should promote scientific proficiency because: 1. Science is a significant part of human culture and represents one of the pinnacles of human thinking capacity; 2. It provides a laboratory of common experience for development of language, logic, and problem-solving skills in the classroom; 3. A democracy demands that its citizens make personal and community decisions about issues in which scientific information plays a fundamental role, and they hence need a knowledge of science as well as an understanding of scientific methodology; 4. For some students, it will become a lifelong vocation or avocation; and 5. The nation is dependent on the technical and scientific abilities of its citizens for its economic competitiveness and national needs [12].

Upon graduation from undergraduate construction programs, graduates are engaged in the design, planning, engineering, construction, estimating/budgeting, management, quality assurance/control, operations, maintenance, and demolition of the nation’s buildings and infrastructure. Their position titles include but are not limited to Field Engineer, Superintendent, Supervisor, Transportation Technician, Project Engineer, Project Inspector, Project Manager, Estimator, and Safety Director. In these roles, graduates need the knowledge and confidence to make decisions related to construction materials, labor, equipment, subcontractors, schedules, costs, sustainability, safety, and quality. There is the need for facts and evidence-based decision when leaders of the engineering and construction industry are providing solutions to emerging problems associated with smart buildings, smart infrastructure, interconnected systems, energy consumption, efficient technologies, global partnerships, and other similar sweeping transformations in the built environment [2]. Despite the critical role to be played by construction graduates upon entry into their professional careers, it appears from anecdotal evidence that their scientific literacy and evidence-based reasoning skills may not be as high as needed to excel in this globally competitive environment. While most undergraduate construction curricula are designed to improve SL, no recent study was found to have assessed undergraduate CM students’ SL skills. Consequently, the primary goal of this research was to assess the scientific literacy skills, attitudes, and self-efficacy of construction students at a historically black university. Specifically, the study had the following objectives: 1. Assess construction students’ attitudes to mathematics and science; 2. Assess the scientific literacy skills of construction students; 3. Assess the scientific literacy self-efficacy of construction students; Methodology This research utilized a mixed methods approach to assess the SL skills, attitudes, and self- efficacy of 46 undergraduate construction students. Data was obtained through students’ completion of the standard 28-item Test of Scientific Literacy Skills (TOSLS) instrument; a student self-reporting SL self-efficacy survey; a student self-reporting mathematics self-efficacy survey, and focus group discussions. In order to facilitate the assessment of scientific literacy skills, the Test of Scientific Literacy Skills (TOSLS) was developed to measures skill related to major aspects of scientific literacy [13]. The TOSLS test had multiple-choice questions, and the students were required to circle the best responses to the test items. Also, the test items are classified into several SL categories and data analysis of these categories gave greater insights into specific SL skills. The surveys utilized a 5-point Likert scale that allowed the students to self-report and provide a rating on various SL and self-efficacy statements listed in the survey. The students also provided short statement responses to open ended questions. During focus group sessions, the students shared the opinions and suggestions to improve their SL skills. Description of the Construction Management (CM) program This CM program is offered at an HBCU committed to exploring transformative approaches in strengthening the much needed evidence-based reasoning of its undergraduate students. The uniqueness of this HBCU is its high social mobility and its maintenance of an educational environment that allows it to be one of the highest producers of Science, Technology, Engineering and Mathematics (STEM) graduates. The Bachelor of Science in Construction Management (CM) prepares its undergraduate students to be leaders and decision makers in the construction industry. This program is accredited by the American Council for Construction Education (ACCE) and the Association for Technology Management and Applied Engineering (ATMAE). The CM program is on track to pursue ABET accreditation, a requirement for all programs in the recently re-structured institution. The program requires students to complete 120 credits over a four-year period. Curriculum requires completion of courses in General Education, Mathematics and Science, Business and Management, Construction Science, and Construction Management.

Results and discussions Description of student population Forty-six students participated in this study. As expected in a male-dominated program, 21.74% of the population were female and 78.26% were male students. The underrepresentation of females in built environment professions is of great concern, especially considering the construction employers are concerned about the lack of skilled construction workers and professionals. Figure 1 shows the population of research participants based on their classification. The proportion of the various student classifications in this study allows the scientific literacy levels and perceptions of all the four major students classifications to be included in the research study.

Student Classification

Seniors, 28% Freshmen, 28%

Juniors, 21% Sophomores, 23%

Freshmen Sophomores Juniors Seniors

Figure 1. Student Classification Most (82.5%) of the students had a GPA greater than 2.5 on a 4.0 scale. Seventy percent (70%) had gained some form of occupational experience including internships and full-time positions in the construction industry. Objective 1. CM students’ attitudes to mathematics and science; Analysis of data from the 5-point Likert scale survey revealed that the mean rating (x̄ =3.31) for CM students’ attitudes to mathematics and science was satisfactory. Over 91% of the CM students agreed that they needed mathematics and science in their future roles in the construction industry. Seventy-five percent (75%) of the CM students who disagreed with their need for mathematics and science in their future roles were either freshmen or sophomores, suggesting that, perhaps, as students matured in the program, they increasingly became aware of their need for mathematics and science in the future career roles in the construction industry. It is expected that as students gain greater exposure to their future career roles and progress through the program, they will better understand the importance of science and mathematics in the construction management profession. The lowest rating (x̄ =2.85) was associated with CM students’ perception of their ability to do more advanced work in science. A higher rating (x̄ =3.28) was associated with their ability to do more advanced work in mathematics. Furthermore, students indicated through higher ratings that they performed better in mathematics (x̄ =3.48) than in science (x̄ =3.28). While it appeared that CM students preferred mathematics to science, none of these slight differences in ratings were statistically significant.

Objective 2. Scientific literacy skills of CM students The TOSLC test score was used to measure the scientific literacy skills of CM students. The mean TOSLS score for scientific literacy test was 43%. In agreement with existing literature on minorities, focus group discussions revealed that the reasons for low SL skills were associated with low socio-economic status; limited exposure to science and mathematics during high and middle-school education; lack of resources; gaps between physical sciences courses and CM courses; and low confidence in mathematics and science skills (Jordan and Sorby, 2014). The mean test score (x̄ =45.27%) for the female students was 6.91% higher than the mean test score (x̄ =38.36%) for the male students; however, this difference was not statistically significant. Nevertheless, it does suggest that female construction students may be more scientific literate than their male counterparts; although they are underrepresented in the construction field. Their underrepresentation may not be due to their lack of scientific literacy skills, but probably, due to their limited knowledge of construction careers and thus the low sense of belonging and interest in this field. Considering the growing labor crisis associated with a shortage of construction workers, nationwide gender-equitable strategies should be strategically implemented to identify, nurture, recruit, retain, and graduate female construction students who are well prepared for built environment professions. No significant correlations existed between test scores and GPA. As shown in figure 2, the mean test score (x̄ =49.16%) of the senior students was 18.52% higher than the mean test score (x̄ =30.64%) of the freshman, and this difference was statistically significant (p<0.05). The highest score was 81% for a male senior CM student, and the lowest score was 7% for a male CM freshman. This confirmed that the CM program curriculum, particularly the science and mathematics courses, had a significant effect on the scientific literacy skills of the students. However, the mean test score of the seniors was still low, strongly suggesting that additional strategies should be implemented to enhance the impact of the CM curriculum on the scientific literacy skills of the students.

TOSLS Test Scores by Classification 60.00

50.00

40.00

30.00

20.00

10.00 Mean Mean TOSLSScores (%) 0.00 Freshman Sophomore Junior Senior Classification

Figure 2. TOSLS test scores by classification A further breakdown of the overall test scores into TOSLC categories resulted in the mean scores for those categories as shown in figure 3. Students’ proficiency for each TOSLS skill was assessed by the percentage of students who provided correct answers to the TOSLS items that measured proficiency for each specific skill. The highest proficiencies were associated with the following TOSLS skills: (1) Understanding valid scientific arguments (56%); (2) Evaluating use of scientific information (52%); and (3) Reading and interpreting graphical representations of data (48%). The CM courses, particularly the science courses, and prior high school education contributed to students’ ability to identify scientific arguments. The students’ knowledge of misuse of scientific data was because the CM program places significant emphasis on ethical conduct in the construction industry.

Mean scores for scientific literacy categories

Valid Science Argument

Evaluate the use and misuse of scientific information

Read and interpret graphical representations of data

Elements of research design and impact on scientific findings Justify inferences, predictions, and conclusions based on quantitative data

Understand and interpret basic statistics

Effective Literature Search

Solve problems using quantitative skills

Make a graph

0 10 20 30 40 50 60 Mean score (%)

Figure 3. Mean scores for scientific literacy categories Figure 3 showed that some of the lowest scientific literacy scores were associated with TOSLS skills that were related to quantitative aptitude: (1) Making graphs (x̄ =22%); (2) Solving problems using quantitative skills (x̄ =24%); (3) Understanding and interpreting basic statistics (x̄ =34%); and (4) Justifying inferences, predictions, and conclusions based on quantitative data (x̄ =36%). CM students’ test scores for ‘solving problems using quantitative skills’ were significantly lower (p<0.001) than their scores for valid scientific argument scores. These results correlated well with CM program data associated with students’ struggles with courses such as ‘Surveying’ that need slightly more advanced quantitative skills. While students demonstrated relatively high abilities in reading and interpreting graphical representations of data, their abilities to make graphs were limited. This was attributed to the fact that perhaps, most of the CM assignments required students to read and interpret graphs that were already made; and few assignments require them to actually develop the graphs themselves. A follow up survey on the mathematical skills of CM students revealed mathematics topics that CM students perceived as easy, and those that they considered difficult. Topics rated as easy by CM students included equations (x̄ =4.24), decimals (x̄ =4.36), percentages (x̄ =4.21). The more difficult topics included logarithm (x̄ =2.85), descriptive statistics (x̄ =2.73), volume of irregular solids (x̄ =2.64), and indices (x̄ =1.73). While logarithm and indices are not regularly applied in typical CM roles, knowledge on calculations associated with the volumes of irregular solids are important for estimating quantities; and, descriptive statistics and data analysis are typically necessary to support decision making in the construction industry. Consequently, additional emphasis should be placed on giving CM students assignments that require volume calculations, statistical analysis, and development of graphs. These skills are critical for leaders who will rely on evidence-based reasoning when making important decisions in the construction industry. The mean mathematics self-efficacy (x̄ =3.35) of the CM students was satisfactory, with 94% agreeing that they did well in high school mathematics. However, of concern is the fact that 49% still felt that they needed assistance to enable them excel in mathematics. These self-efficacy results were somehow in agreement with the low TOSLS test results. Consequently, it appears that some CM students had a false sense of confidence associated with their mathematical abilities. While they did well in high school mathematics, the skills they had gained were not adequate to make them excel in the mathematics-based assignments in the CM program. During focus group discussions, CM students suggested that math tutoring should be provided to the students. They also emphasized that the tutoring should focus on math topics that are more applicable to their future roles in the construction industry. Also, CM students suggested math clubs and math labs; with, one student suggesting that a placement test be taken to identify new students who need additional math interventions. These early interventions would help CM students be more successful in the CM program. Objective 3. CM students’ scientific literacy self-efficacy Table 1 shows results from data obtained from the SL self-efficacy survey. It revealed that that the overall mean SL self-efficacy score was 3.42 on the 5-point Likert scale. Table 1. Scientific literacy self-efficacy Scientific self-efficacy statement Strongly Mostly Agree Disagree Mostly Agree Agree Disagree I can identify a valid science 15.22% 26.09% 41.30% 13.04% 4.35% argument I can develop a hypothesis 39.13% 17.39% 30.43% 10.87% 2.17% I can determine if a scientific source 17.39% 32.61% 23.91% 21.74% 4.35% is valid I can determine if there is a misuse 15.22% 28.26% 34.78% 15.22% 6.52% of scientific information I can design a research study 13.04% 23.91% 41.30% 19.57% 2.17% I can conduct literature review 10.87% 17.39% 41.30% 28.26% 2.17% I can develop graphs to represent 15.22% 32.61% 43.48% 8.70% 0.00% scientific data I can read and interpret graphical 17.39% 34.78% 34.78% 13.04% 0.00% representation of scientific data I can solve problems using scientific 6.52% 36.96% 43.48% 13.04% 0.00% data I can understand and interpret basic 8.70% 36.96% 41.30% 13.04% 0.00% statistics I can justify inferences, predictions, 13.04% 34.78% 32.61% 19.57% 0.00% and conclusions based on quantitative data

Table 1 showed that very few students strongly agreed with their ability to demonstrate the SL through the activities listed. Majority of the students just agreed that they could perform SL skills, and this gave an indication that students generally did not feel that they had developed high levels of mastery or expertise in activities such as solving problems using scientific data or understanding and interpreting basic statistics. The results are in agreement with the low TOSLS scores, and suggest that major curriculum improvements may be necessary for CM programs to produce graduates with high SL skills. Nevertheless, it must be noted that on the average, 15.5% of the students had very high levels of self-efficacy in all the listed SL skills. Although correlations tests revealed that no strong correlations existed between the TOSLS test scores and SL self-efficacy, the results showed that the CM students who had the highest TOSLS scores also had highest levels of SL self-efficacy, Conclusion Scientific literacy and reasoning skills are critical for the future leaders of the construction industry. However, the findings from this research study suggests that although undergraduate CM students recognize the need for science and mathematics in the future careers, their scientific literacy skills are limited. Nevertheless, the CM program had been somehow effective in improving the scientific literacy skills of the students as the seniors performed significantly better than freshmen on the scientific literacy skills tests. Additional investigations revealed that CM students did better on test items associated with valid science arguments and evaluating the misuse of scientific information. However, they struggled with basic statistics, solving problems using quantitative skills, and making graphs. The female students had higher levels of SL. This suggested that the underrepresentation of females in the CM program may not be primarily due to their limited SL skills, but perhaps due to their lack of knowledge and therefore low interest in construction careers. Considering that these skills are critical in evidence-based decision making, it is critical that CM curriculums are modified to improve quantitative skills that construction leaders need to make strategic decisions in the construction industry. CM students suggested additional tutoring that focused on mathematic topics that are relevant to their future careers, mathematics laboratory, and a placement test to identify CM students who may need early interventions to increase their mathematical skills. CM educators and researchers should seriously explore strategies that will improve the SL skills of CM students. In the long term, CM students will higher scientific literacy skills will be better prepared to make evidence-based decisions that will continue to transform global built environments. References 1. Association of American Colleges and Universities, “Recent Trends in General Education Design, Learning Outcomes, and Teaching Approaches”, 2016, Available: https://www.aacu.org/sites/default/files/files/LEAP/2015_Survey_Report2_GEtrends.pdf [Accessed December 16, 2017] 2. FMI (2017). “U.S. Markets Construction Overview 2017: Featuring FMI’s construction outlook”, Available: https://www.fminet.com/wp- content/uploads/2017/03/USOverview_FINAL-1.pdf [Accessed December 16, 2017] 3. Van Epps, A. S., “Educating for evidence based decisions in engineering: The view as librarian and instructor”, Conference Proceedings of the 2013 American Society for Engineering Education (ASEE) Annual Conference and Exposition, 2013. 4. UNC Taskforce 5 Evidence, Analysis, Interpretation and Critique, 2017, Available: http://curriculum2019.web.unc.edu/files/2017/09/TF5-Evidence-Reasoning.pdf [Accessed December 16, 2017] 5. J. D. Miller, “Scientific Literacy: a Conceptual and Empirical Review,” Daedalus, vol. 112, no. 2, pp. 29–48, 1983. 6. J. D. Miller, “The Scientifically Illiterate”, American Demographics, vol. 9, no. 6, pp. 26– 31, 1987. 7. B.S.P. Shen, “Science literacy and the public understanding of science”. In S. B. Day (Ed.), Communication of scientific information, pp. 44-52, Basel, Switzerland: S. Karger A.G., 1975. 8. J. D. Miller, “Public understanding of, and attitudes toward, scientific research: what we know and what we need to know”, Public Understand. Sci., Vol. 13, pp. 273–294, 2004. 9. A. E., Lawson, D. L., Banks, and M. Logvin, Self-efficacy, reasoning ability, and achievement in college biology. Journal of Research in Science Teaching, vol. 44, no. 5, pp. 706 – 724, 2006. 10. N.J.S., Brown, E. M., Furtak, M. J., Timms, S. O., Nagashima, and M. Wilson, The evidence-based reasoning framework: Assessing scientific reasoning. Education Assessment, vol. 15, no. 3, pp. 123 -141, 2010. 11. National Research Council (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press. 12. National Research Council, “Taking science to school: Learning and teaching science in Grades K-8. Washington, DC: The National Academic Press, (2007). 13. C. Gormally, P. Brickman, and M. Lutz., “Developing a Test of Scientific Literacy Skills (TOSLS): Measuring Undergraduates’ Evaluation of Scientific Information and Arguments”, CBE Life Sciences Education, Vol. 11, No. 4, pp. 364–377, 2012.