Workshop Result: Teaching Structured Reviews to Environmental Engineering Researchers
Total Page:16
File Type:pdf, Size:1020Kb
Paper ID #33111 Workshop Result: Teaching Structured Reviews to Environmental Engineering Researchers Dr. Daniel B. Oerther, Missouri University of Science and Technology Professor Daniel B. Oerther, PhD, PE joined the faculty of the Missouri University of Science and Tech- nology in 2010 as the John A. and Susan Mathes Chair of Civil Engineering after serving ten years on the faculty of the University of Cincinnati where he was Head of the Department of Civil and Environmental Engineering. Oerther earned his Ph.D. (2002) from the University of Illinois, Urbana-Champaign. Dan’s professional registrations include: PE, BCEE, BCES, CEng, CEnv, CEHS, and DAAS. Oerther’s schol- arship, teaching, service, and professional practice focus in the fields of environmental biotechnology and sustainable development where he specializes in promoting Water, Sanitation, and Hygiene (WaSH), food and nutrition security, energy efficiency, and poverty alleviation. Oerther’s awards for teaching in- clude the best paper award from the Environmental Engineering Division of ASEE and the society-wide Robert G. Quinn Award from ASEE, the Engineering Education Excellence Award from the NSPE, the Excellence in Environmental Engineering and Science Educator award from AAEES, and the Fair Dis- tinguished Engineering Educator Medal from WEF. Due to his collaborations with nurses and healthcare professionals, Professor Oerther has been inducted as a Lifetime Honorary Member of Sigma Theta Tau, the International Honor Society of Nursing (STTI), and he has been inducted as a Lifetime Honorary Fellow of the American Academy of Nursing (FAAN) and the Academy of Nursing Education Fellows (ANEF). Oerther has also been elevated as a Fellow of the Society of Environmental Engineers (FSEE), the Royal Society of Arts (FRSA), the Royal Society for Public Health (FRSPH), the Chartered Institute of Environmental Health (FCIEH), the Society of Operations Engineers (FSOE), and the Association of Environmental Engineering and Science Professors (FAEESP). c American Society for Engineering Education, 2021 Workshop Result: Teaching Structured Reviews to Environmental Engineering Researchers Daniel B. Oerther Missouri University of Science and Technology, 1401 North Pine Street, Rolla, MO 65409 Abstract As part of the 2019 biennial conference of the Association of Environmental Engineering and Science Professors, a pre-conference workshop on the topic of structured reviews was delivered to 22 participants. The workshop had three objectives, namely: 1) raising awareness about the process of structured reviews; 2) demonstrating the process of structured reviews; and 3) encouraging participants to adopt a structured review format for future research. To achieve these objectives, three activities were undertaken, including: 1) a pre-conference web site was constructed where participants completed mandatory exercises before the on-site portion of the workshop; 2) group-work to learn the structured review process during the on-site portion of the workshop; and 3) voluntary follow-up including mentoring/coaching by workshop conveners to support the use of structured reviews in future research after the completion of the on-site portion of the workshop. Assessment of the knowledge, skills, and attitudes of workshop participants was undertaken using an on-site, anonymous, voluntary readiness assessment test (RAT) as well as an on-site, anonymous, voluntary comprehension assessment test (CAT), administered immediately preceding and following the on-site portion of the workshop, respectively. To support the long-term adoption of structured reviews in research, the workshop moderators provided mentoring/coaching to workshop participants in the months following the completion of the on-site portion of the workshop. The value of this long-term mentoring/coaching was assessed by an additional CAT administered one year after the completion of the on-site portion of the workshop. The purpose of this paper is to share: 1) workshop content and format that could be used by other conveners of similar workshops; 2) results of the analysis of the RAT and CATs; and 3) the author’s experience with mentoring/coaching workshop participants on the use of structured reviews. Introduction Structured reviews – a formalized process to synthesize evidence-based practice and policy by extracting and analyzing research data from the scientific literature – are widely used in the fields of healthcare, social sciences, education, and economics, where humans are part of the system of study observed and manipulated by basic and applied researchers [1]. Structured reviews often are part of “translational medicine”, where discoveries at the “lab bench” are brought to the “bed side”. Structured reviews typically are not employed by engineers; although the value to engineering researchers is self- evident in collaborations with disciplines where structured reviews are regularly employed (i.e., [2], [3], and [4]), and prior efforts have been invested to educate engineers about systematic reviews in workshops (i.e., [5], and [6]). Disseminating the value of and demonstrating the basic approach to performing a structured review to a diverse audience of engineering graduate students, faculty, and administrators was the objective of a pre-conference workshop, which was part of the 2019 biennial conference of the Association of Environmental Engineering and Science Professors (AEESP) in Tempe, Arizona. The purpose of this paper is to share: 1) workshop content and format that could be used by other conveners of similar workshops; 2) on-site, anonymous, voluntary readiness assessment test (RAT) as well as an on-site, anonymous, voluntary comprehension assessment test (CAT), administered immediately preceding and following the workshop, respectively, as well as an additional CAT administered after a period of post-workshop mentoring/coaching; and 3) the author’s experience with mentoring/coaching workshop participants on the use of structured reviews. Methods Workshop content: background on structured reviews. Structured reviews often are attributed to the efforts of British physician, Archie Cochrane. In 1971, Cochrane authored “Effectiveness and Efficiency” [7] where he argued passionately for medical practice to be based on evidence. Cochrane identified the results of randomized controlled trials (RCTs), or experiments where test subjects are assigned either to an intervention or to a control condition using a method independent of human influence (i.e., random), as the “gold-standard” of scientific evidence. But not every RCT is of equal quality (i.e., the size of the sample population and the duration of the experiment vary among different trials), and the results from replicate RCTs often produce different results. Observing this fact, Dr. Cochrane concluded, “it is surely a great criticism of our profession [medicine] that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized control trials,” [7]. This concept, a critical summary of published research, organized, and updated regularly is what is meant by the term, “structured review”. Perhaps the most widely known example of an international collaborative effort to produce structured reviews is Cochrane (also known as, the Cochrane Library), a British charity founded in 1993 with the motto, “Trusted evidence. Informed decisions. Better health.” Cochrane is named after Dr. Archie Cochrane. Depending upon the approach employed in creating the summary and the types of published research studies analyzed, the various styles of structured reviews include scoping reviews, systematic reviews, and meta-analyses, among others [8]. Important aspects of performing a structured review include: 1) identifying a research question; 2) defining inclusion and exclusion criteria; 3) collecting all possible studies according to the inclusion criteria; 4) eliminating studies using the exclusion criteria; 5) extracting and thematically comparing relevant data; and 6) synthesizing the answer to the research question using the weight of evidence of the studies. The three learning objectives of the pre-conference workshop, included: 1. raising awareness about the process of structured reviews using a pre-conference website; 2. demonstrating the process of structured reviews using group-work during the on- site workshop; and 3. encouraging participants to adopt a structured review format for future research through voluntary mentoring/coaching in the months following the on-site workshop was completed. Workshop format and assessments. Founded in 1963, today’s Association of Environmental Engineering and Science Professors (AEESP) includes nearly 1,000- members who are faculty, postdocs, and students of environmental engineering and science. The mission of AEESP is to, “assist it’s members in the development and dissemination of knowledge in environmental engineering and science.” One way this occurs is through a biennial conference. In May, 2019, the membership of AEESP gathered in Tempe, Arizona. Conference organizers included Arizona State University, the University of Arizona, and Northern Arizona University. The conference theme was, “Environmental engineers and scientists see cities in 4-D,” which includes the built environment, the natural environment, cyberspace, and human health. As part of the three-day, on-site event, conference participants were invited to apply to attend one or more of 16 different pre-conference