C HAPTER 2 Review Criteria ABSTRACT Following the common IMRaD format for scienti®c re- when the data analysis was planned. Further, the author search reports, the authors present review criteria and dis- discusses the presentation of the body of evidence col- cuss background information and issues related to the re- lected within the study, offering information for reviewers view criteria for each section of a research report. on evaluating the selection and organization of data, the Introduction. The authors discuss the criteria reviewers balance between descriptive and inferential statistics, nar- should be aware of for establishing the context for the rative presentation, contextualization of qualitative data, research study: prior literature to introduce and describe and the use of tables and ®gures. the problem statement, the conceptual framework (the- Discussion. The authors provide information to enable ory) underlying the problem, the relevance of the re- reviewers to evaluate whether the interpretation of the search questions, and the justi®cation of their research evidence is adequately discussed and appears reliable, design and methods. valid, and trustworthy. Further, they discuss how review- Method. The authors discuss a variety of methods used ers can weigh interpretations, given the strengths and to advance knowledge and practice in the health profes- limitations of the study, and can judge the generalizability sions, including quantitative research on educational in- and practical signi®cance of conclusions drawn by inves- terventions, qualitative observational studies, test and tigators. measurement development projects, case reports, exposi- Title, authors, and abstract. The author discusses a re- tory essays, and quantitative and qualitative research syn- viewer's responsibility in judging the title, authors, and thesis. As background information for reviewers, the au- abstract of a manuscript submitted for publication. While thors discuss how investigators use these and other this triad orients the reader at the beginning of the review methods in concert with data-collection instruments, process, only after the manuscript is analyzed thoroughly samples of research participants, and data-analysis pro- can these elements be effectively evaluated. cedures to address educational, policy, and clinical ques- Other. The authors discuss the reviewer's role in eval- tions. The authors explain the key role that research uating the clarity and effectiveness of a study's written methods play in scholarship and the role of the reviewer presentation and issues of scienti®c conduct (plagiarism, in judging their quality, details, and richness. proper attribution of ideas and materials, prior publica- Results. The author describes issues related to reporting tion, con¯ict of interest, and institutional review board statistical analyses in the results, particularly data that do approval). not have many of the properties that were anticipated Acad. Med. 2001;76:922±951. 922 A CADEMIC M EDICINE,VOL.76,NO . 9/SEPTEMBER 2001 MANUSCRIPT INTRODUCTION Problem Statement, Conceptual Framework, and Research Question William C. McGaghie, Georges Bordage, and Judy A. Shea* REVIEW CRITERIA Ⅲ The introduction builds a logical case and context for the problem statement. Ⅲ The problem statement is clear and well articulated. Ⅲ The conceptual (theoretical) framework is explicit and justi®ed. Ⅲ The research question (research hypothesis where applicable) is clear, concise, and complete. Ⅲ The variables being investigated are clearly identi®ed and presented. ISSUES AND EXAMPLES RELATED TO THE CRITERIA statements are: ``With the national trend toward more pa- tient care in outpatient settings, the numbers of patients on Introduction inpatient wards have declined in many hospitals, contrib- uting to the inadequacy of inpatient wards as the primary A scholarly manuscript starts with an Introduction that tells setting for teaching students,'' 2 and ``The process of profes- a story. The Introduction orients the reader to the topic of sional socialization, regardless of the philosophical approach the report, moving from broad concepts to more speci®c of the educational program, can be stressful . few studies ideas.1 The Introduction should convince the reader, and all have explored the unique stressors associated with PBL in the more the reviewer, that the author has thought the topic professional education.'' 3 These statements help readers an- through and has developed a tight, ``researchable'' problem. ticipate the goals of each study. In the case of the second The Introduction should move logically from the known to example, the Introduction ended with the following state- the unknown. The actual components of an Introduction ment: ``The purpose of this qualitative study was to identify (including its length, complexity, and organization) will vary stressors perceived by physiotherapy students during their in- with the type of study being reported, the traditions of the itial unit of study in a problem-based program.'' In laying research community or discipline in which it is based, and out the issues and context, the Introduction should not con- the style and tradition of the journal receiving the manu- tain broad generalizations or sweeping claims that will not script. It is helpful for the reviewer to evaluate the Intro- be backed up in the paper's literature review. (See the next duction by thinking about its overall purpose and its indi- article.) vidual components: problem statement, conceptual framework, and research question. Two related articles, ``Ref- Conceptual Framework erence to the Literature'' and ``Relevance,'' follow the pres- ent article. Most research reports cast the problem statement within the 4 Problem Statement context of a conceptual or theoretical framework. A descrip- tion of this framework contributes to a research report in at The Introduction to a research manuscript articulates a prob- least two ways because it (1) identi®es research variables, lem statement. This essential element conveys the issues and and (2) clari®es relationships among the variables. Linked context that gave rise to the study. Two examples of problem to the problem statement, the conceptual framework ``sets the stage'' for presentation of the speci®c research question that drives the investigation being reported. For example, the conceptual framework and research question would be *Lloyd Lewis, PhD, emeritus professor of the Medical College of Georgia, participated in early meetings of the Task Force and contributed to the different for a formative evaluation study than for a sum- earliest draft of this section. mative study, even though their variables might be similar. A CADEMIC M EDICINE,VOL.76,NO . 9/SEPTEMBER 2001 923 Scholars argue that a conceptual or theoretical framework For some journals, the main study variables (e.g., medical always underlies a research study, even if the framework is competence) will be de®ned in the Introduction. Other jour- not articulated.5 This may seem incongruous, because many nals will place this in the Methods section. Whether speci®c research problems originate from practical educational or hypotheses or more general research questions are stated, the clinical activities. Questions often arise such as ``I wonder reviewer (reader) should be able to anticipate what will be why such an event did not [or did] happen?'' For example, revealed in the Methods. why didn't the residents' test-interpretation skills improve after they were given feedback? There are also occasions SUMMARY when a study is undertaken simply to report or describe an event, e.g., pass rates for women versus men on high-stakes The purpose of the Introduction is to construct a logical examinations such as the United States Medical Licensing ``story'' that will educate the reader about the study that Examination (USMLE) Step 1. Nevertheless, it is usually follows. The order of the components may vary, with the possible to construct at least a brief theoretical rationale for problem statement sometimes coming after the conceptual the study. The rationale in the USMLE example may be, for framework, while in other reports the problem statement instance, about gender equity and bias and why these are may appear in the ®rst paragraph to orient the reader about important issues. Frameworks are usually more elaborate and what to expect. However, in all cases the Introduction will detailed when the topics that are being studied have long engage, educate, and encourage the reader to ®nish the man- scholarly histories (e.g., cognition, psychometrics) where ac- uscript. tive researchers traditionally embed their empirical work in well-established theories. REFERENCES Research Question 1. Zeiger M. Essentials of Writing Biomedical Research Papers. 2nd Ed. London, U.K.: McGraw±Hill, 1999. A more precise and detailed expression of the problem state- 2. Fincher RME, Case SM, Ripkey DR, Swanson DB. Comparison of am- ment cast as a speci®c research question is usually stated at bulatory knowledge of third-year students who learned in ambulatory the end of the Introduction. To illustrate, a recent research settings with that of students who learned in inpatient settings. Acad Med. 1997;72(10 suppl):S130±S132. report states, ``The research addressed three questions. First, 3. Soloman P, Finch E. A qualitative study identifying stressors associated do students' pulmonary physiology concept structures change with adapting to problem-based learning. Teach