Flow Chart for Articles You Read

1. Author’s Objectives (State in your own words) A. Topic: What does the author want to add to what we know about the topic of the research?

B. Explanation: What does the author want to add to our ability to explain the phenomenon of interest?

C. Theory: What does the author want to add to the development of theory?

2. Research Questions State the author’s research question(s) in your own words.

3. Theoretical Constructs & Linkages Explored in the Research A. IF the author uses a theoretical framework, state the constructs (concepts) in the theory that are used in the study.

B. Variables Representing Theoretical Constructs. List the variables representing each theoretical construct identified above. These may not be stated in some studies using qualitative analysis. If this is the case, say so. However, many qualitative studies do identify specific variables (often called themes) representing constructs. List them by type of variable, grouping, independent or predictor, and dependent or outcome.

C. Does the author state research or what are sometimes called general or working hypotheses? These are NOT statistical hypotheses. Often they are something like “We hypothesized that resilience [a theoretical construct] would be greater in families with dense social networks [another theoretical construct]. That is, these statements are often statements of relationships between theoretical constructs that the authors expect to observe.

D. Interventions or Treatments. There may be none, especially in descriptive studies. There is always a treatment in a true or quasi-experiment. In experimental designs, the intervention or treatment typically is the basis for establishing the comparison groups. Comparison groups in longitudinal designs are often cohorts. 4. Sampling A. Comparison Groups. Some designs have only one group. There are no comparison groups and no comparisons between groups are made. This usually indicates that the research is descriptive rather than explanatory. If there are no comparison groups, say so.

B. Assignment to Comparison Groups. NOTE: This is almost always ex-ante for experiments and quasi-experiments. It is often ex-ante for cross-sectional and longitudinal designs as well, but may be post-hoc. Assignment to comparison groups may be either ex- ante or post-hoc for case studies.

C. Theoretical Population(s). If the author compares groups, each group “belongs to” a different theoretical population (e.g., youth who have been arrested at least once and youth who have never been arrested). Theoretical populations may be identified post-hoc.

D. What is the accessible population? This may be different for the populations. The accessible population might be youth between 16 and 18 serving a sentence in a juvenile facility while the accessible population for the other youth might be all youth in a geographic area between ages 16 and 18.

E. Is there a sampling frame? If so describe it. The sampling frame and accessible population CANNOT BE THE SAME. Some studies do not have a sampling frame. All studies have an accessible population.

F. What are the selection or screening criteria? Many (perhaps most) studies use some screening criteria. This does NOT make the sample a “purposive” or “judgmental” sample. It is simply a way to reduce non-experimental variance or establish comparison groups.

G. What is the size of the sample? How was sample size determined? Note that some studies (a weakness) fail to justify or explain how sample size was determined. If this is the case, say so. If there are comparison groups, give this information for each group.

H. What procedures were used to recruit participants? For example, in a volunteer sample a researcher might put ads in a local newspaper like the Alligator to get a student sample, put flyers up about the research in the neonatal unit to get a sample of parents whose infants require hospitalization, or get people to spread the word via e-mail. In a true random sample, the researcher has to have some kind of list – how did s/he get or create the list? In other words, how did the researcher identify and contact potential participants?

I. What was the response rate? Note that in most studies at least a few people who were contacted to participate in the study say no, cannot be contacted, etc. Some authors fail to explain the response rate. If that is the case, say so. J. Were replacement procedures used? Some researchers use replacement procedures to make sure that the calculated (required) sample size was met. This does not mean that the response rate was 100%. You might need 200 people for a study. Half of the people contacted say no—but through replacement you finally get 200 people who say yes. In that case, 400 were contacted, but 200 said no and the response rate is 50%.

5. Data Collection Procedures Explain how the data were collected. Was an internet-based questionnaire used? Did they do in-person interviews? Did they conduct a focus group?

6. Variables & Level of Measurement NOTE Give the level of measurement for EACH variable: There are four levels of measurement, nominal, ordinal, interval and ratio. For all practical purposes in our work, interval and ratio data are equivalent levels of measurement. Most narrative data (like answers to questions in a focus group) are nominal, but such data may be transformed into other levels of measurement (by counting how many people used the term “terrible” to describe an event, for example). A. Grouping variables (basis for comparison groups)

B. Independent or predictor variables (NOT the same as the grouping variables)

C. Dependent or outcome variables

7. Statistical Data Analysis Complete ONLY if the article uses statistical tests. Descriptive statistics of the sample are NOT statistical data analysis. A. Make a NUMBERED LIST of the statistical hypotheses. Remember that statistical tests always test some hypothesis. It may not be clearly stated, but there is an implicit hypothesis for statistical testing, even if it is not stated. For example, if the author uses a t- test to compare two groups, even if s/he does not say it, the underlying hypothesis is that the two groups differ with regard to the variable. State the statistical hypotheses in your own words, do not copy them from the article. If an article uses any statistical tests, you must state the underlying statistical hypothesis, even if the authors do not.

B. For each hypothesis, state the specific statistical test(s) used to test the hypothesis. You do not need to repeat the hypotheses, just use the numbers from the list you created above.

C. For each hypothesis, state the results of the statistical test used to test the hypothesis. Identify by the number for each hypothesis. D. Describe any additional (unplanned, post-hoc) statistical analyses performed. As researchers conduct analyses, they sometimes add statistical procedures because they see a trend or an artifact in the data that they want to explore. Typically, there are no stated hypotheses for these analyses, but in some cases, there are. Give the hypothesis (stated or if not you state it), test used, and result for each of these tests.

8. Qualitative Data Analysis Provide answers for all procedures used A. Describe the data archiving procedures (like transcription, initial purely descriptive coding, etc.)

B. Preliminary analysis (by case, such as memos, topical coding, or analytic coding)

C. Basic or level 1 cross-case analysis (such as identifying themes or creating categories)

D. Higher level analyses (such as finding relationships between categories or creating typologies)

E. Creating generalizable results (such as creating qualitative models)

F. Any other procedures not included above

9. Conclusions This is NOT the same thing as results. Results are specific to a study. Conclusions are the broader implications of the research. The conclusions typically directly address the author’s stated research objectives. As you examine the conclusions, organize them by the three types of objectives you identified at the beginning of this form. Do NOT repeat any results here. Topical Objective(s).

Explanatory Objective(s).

Theoretical Objective(s).