Preferences for Tues

Center on Disability and Community Inclusion University of Vermont Mann Hall – 3rd floor 208 Colchester Ave., Burlington VT 05405-1757

Evidence-based Journal Club December 1, 2009 (4:30 – 6:30 pm) Locations Expected Participants Waterbury Lini Alberghini Sandra Cameron Mary Ellen Seaver-Reid Pam Barnard Mary Chambers Louise Mir Williston Charlene Mongeon Bonnie Lachtrupp Linda Kogut Deb O’Rourke Stephen Contompasis Peggy Owen Karen Downey Ruth Dennis Paul Woodruff Marie MacLeod Marie-Christine Potvin Heather Preis Pat Crocker Christina Allard Liz Robitaille White River Junction Denise Mitchell Beth Pastor Jane Duffy Paula Rossvall Chris Knippenberg Rutland Susan Boles Katrina Abatiell Karen Gartner Sally Short Phoebe Chestna Margaret Duffy Patty Thomas Johnson Carol Lehmann Sarah Stutz Rachelle Peterson Stephanie Veit

Instructions to Participants: 1) Please sign-in AND sign-out on the TRIPSCY sign-in sheet. This is critical for OTs who want to report this activity for continuing education credits. 2) Handouts were sent to you by email. 3) Please assign one person at each site who will collect all the material at the end (i.e., sign-in sheets, completed surveys, etc.) and give it to the VIT technician.

Thank you and see you all on the air,

Marie-Christine Potvin Center of Disability and Community Inclusion/UVM Mann Hall – 3rd floor 208 Colchester Ave. Burlington VT 05405-1757

Evidence-based Journal Club Page 1 ROLES Facilitator: Marie-Christine Keeper of the rudder: Recorder: Peggy Sands Processor: Timekeeper: Wellness provider: Factsheet Editors: Pam Barnard and …

AGENDA 4:30 Welcome, check-in, reminder to sign-in, and survey

4:40 Evidence-based Journal Club Process

4:45 Choice of today’s topic and literature review

4:50 Reviewers summarize their critical appraisals (10 min. per article)

5:50 Discussion

6:20 Plan for next meeting

6:25 Processor and Wellness

Sign-out!!! REVIEWS

REFERENCE 1: Inder, J. M., and Sullivan, S. J. (2004) Does an Educational Kinesiology Intervention Alter Postural Control in Children with a Developmental Coordination Disorder? Clincial Kinesiology, 58(4), 9-26.

Reviewers: Marcy Pelkey and Peggy Sand

Summary: The Inder and Sullivan article examined the effect of Educational Kinesiology techniques on postural responses of 4 children with a Developmental Coordination Disorder. The authors used a single- subject, multiple baseline methodology with baseline, intervention, and follow-up phases. The Edu-K intervention consisted of PACE and individualized Laterality Repatterning processes. Weekly computerized Sensory Organization Tests (SOT) were performed through each phase of the study. There was a great deal of variability in the data from session to session and from phase to phase. In general, there was a decrease in the number of falls in all 4 children. There were some improvements in sensory system scores, with the vestibular system condition scores being most improved. The authors concluded that the Edu-K techniques may have been effective in improving postural response in these 4 children, however the possibility of learning with the weekly SOT testing cannot be discounted.

STUDY PURPOSE: Outline the purpose of the study: Was the purpose The purpose was to examine the effect of selected Edu-K techniques on postural stated clearly? Yes responses in 4 children with a developmental Coordination Disorder (DCD) LITERATURE: Describe the justification of the need for this study (3-4 key points)

Evidence-based Journal Club Page 2 Was relevant Children with DCD are seen more frequently on PT/OT caseloads and there are background sensory and motor characteristics with postural control being a complicating literature reviewed? factor for these children. Posturography is used to investigate the role of ___ Yes different sensory systems, especially the vestibular system, in postural control. _X_ No This sensory organization deficit, for these kids, may be due to developmental delay and some deficiency in the CNS. Edu-K is one treatment methology. No other discussion of postural control interventions and whether they are effective or not, thus leading to the need to try Edu-K as a treatment method. DESIGN: Describe the study design: __x_ single case Multiple baseline, single subject, 3 phases: baseline (A), intervention (B), design follow-up (C) Can the author answer the study question with the study design? Yes Were the design and/or method used introducing biases. If so describe: There was one primary investigator with no blinding of study described. No control group, but staggered phases. SAMPLE SIZE: Sample Description (e.g., age, gender, diagnosis, other characteristics) N = 4 4 children ages 9-12 with DCD Was sample size How was sample identified? Was it a representative sample? justified? The sample had been previously involved in the University based movement __X_ Yes developmental clinic, sample of convenience. ___ No If there were more than one group, was there similarity and differences between ___ N/A the groups? Describe: Similarities were they are all children between the ages of 9-12, all have DCD, Was Power all were former patients in the Movement Dev. Clinic, all parents had expressed Discussed? an interest in having their child participate in research groups. Differences: Some ___ Yes were males, some were females, varying cognitive and motor involvement, and ___ No different diagnoses. Two were sisters. __X_ N/A Was informed consent and assent obtained? Yes OUTCOMES: List measures used Reliable & Valid? BOTMP- Bruninks-Oseretsky Test of Motor Proficiency- Inclusion criteria only. NO SMP – Standard Progressive Measure- Inclusion criteria only. YES PPVT- Peabody Picture Vocabulary test- Inclusion criteria only. YES SOT Sensory Organization Tests (computerized)- used for baseline (A), YES intervention (B) and Follow-up (C) phases on a weekly basis. INTERVENTION: Provide a short description of the intervention including type of intervention, Intervention was who delivered it, how often and in what setting.

Evidence-based Journal Club Page 3 described in detail? Six weekly sessions for all children with 2 follow up sessions ( 30 Min. each) __X_ Yes Consisting of : ___ No  PACE: A sequence of 4 brain gym activities that represent a learning ___ Not addressed readiness technique to settle one into his own “best learning pace”.  Dennison Laterality Repatterning (DLR)- A 5 step “balance” process which Contamination was stimulates the key stages of laterality from infancy through walking. avoided?  The participants did the PACE in am/pm and before the SOT testing session ___ Yes on their own. __X_ No?? Contamination: For two subjects- they continued, voluntarily, to use PACE ___ Not addressed during no intervention and follow-up phase and two subjects were sisters, so staggered baseline may not have been true. RESULTS: What were the results? Results were reported Outcomes Results Analysis in terms of statistical Improved Collectively, 7 out of the 24 They used + one significance? SOT score conditions in SOT tests showed or two standard _X_ Yes means from improved scores of at least 2 SD deviations to _X_ No one phase to and another 8 conditions with determine __ NA another for improved scores of 1 SD. important change. ___ Not addressed different Showed more hip and ankle Celeration lines to sensory strategies for postural control show the trend in conditions rather than falls data for baseline The total The number of falls changed from and intervention number of phase A into phases B and C. phases. falls on each All subjects show increases in day of testing mean sensory ratio scores from Sensory ratio phase A to B in the use of each scores were sensory system to maintain used to balance. The Vestibular ratio compare the scores show the most overall means improvement. All the sensory between ratio means in phase C were phases in higher than phase A, but were not each of the all maintained as high at phase B. sensory systems. Comparing One subject’s composite phase A SOT equilibrium score was at peer mean scores range score from first intervention with mean phase score to the end of the scores for intervention phase, then dropped peers. at the follow-up phase. Two subjects’ composite equilibrium scores only reached peer range for one time and were well below peers the rest of the time. The 4th

Evidence-based Journal Club Page 4 subject reached peer scores at the fourth data point during intervention phase and his scores remained above this level into the follow-up phase. During phase A, mean SOT scores for all subjects wee 1 to 2 SD below their mean peer scores. All subjects shoed improvements into peer range: Subject 1 in 4 conditions, Subject 2 in one condition, Subject 3 in three conditions and subject 4 in four conditions. Was the analysis Explain: The authors compared the means of the SOT scores from phase A to appropriate for the phase B to see if the mean for phase B was more than mean A + one or two type of outcome standard deviation(s). This change in mean score was used to indicate that there measures and the was change in performance from the baseline (A) to intervention (B) phase. methodology? Means that changed by at least 2 standard deviations did indicate important __X Yes change in scores. ___ No ___ Not addressed The authors also used Celeration lines to measure if the changes seen during the baseline phase indicated learning using the SOT testing equipment. The slope of the line changed for 3 of the 4 subjects from baseline to intervention phase. The one subject with the largest number of baselines test points was the subject that did not show a change in Celeration line from phase A to phase B.

Peer scores from another research article were used for comparison to determine whether the change in SOT and Sensory ratio scores made the subject scores closer to peer scores. If not statistically significant (i.e., p < 0.05 or 0.01), was study big enough to show an important difference if it should occur (power and sample size)? The study seemed to be big enough to show changes in SOT scores from baseline to intervention phase. They used a variety of methods to review the scores to determine all the ways that scores changed. Clinical importance What is the clinical importance of the results (that is even if the results were was reported? statistically significant were the differences large enough to be clinically _X_ Yes meaningful? ___ No Subjects had more mature postural responses and fewer falls during SOT testing. ___ Not addressed They performed better on the SOT tests, but these scores were not related to clinical importance. There were anecdotal reports of improved ball skills. Drop-outs were If yes, why did they drop out? How were drop-out participants included in the reported? No statistical analysis? One subject missed three sessions due to being sick. CONCLUSIONS What did the authors conclude?

Evidence-based Journal Club Page 5 AND CLINICAL Despite confounding influences, significant improvement in the sensory systems IMPLECATIONS: and their integration and organization for postural stability were shown in this The conclusions made study, suggesting some effectiveness of Edu-K intervention. by the authors were What were the main limitations of the study as stated by the author(s) and from appropriate given your point of view? study methods and Without collecting baseline measures until having a stable baseline, it is difficult results. to tell if the subjects learned how to perform the SOT tests, thus showing ___ Yes improved scores. The one subject who had the most baseline data points showed _X_ No better celeration line in Phase A over Phase B. Sample of convenience- these children were part of other tests by the same group. Two sisters as participants- even with the staggered start to intervention, there was no tracking of whether each child adhered to the protocol. Continued to use PACE in the no intervention and the follow-up times. What are the implications of these results for your practice? There was no application of the results to functional skills, so it is difficult to determine how to use the information gained from this research in practice. Given the limited support for Edu-K, the information gained from this article would not change my practice.

REFERENCE 2: Hyatt, K. (2007, March/April) Brain gym: building stronger brains or wishful thinking? Remedial and Special Education, 28-2, 117-124.

Reviewers: Patty Thomas and Marie MacLeod

Summary: The article reviews research regarding the effectiveness of Brain Gym in improving/enhancing learning, as claimed by the program’s developers. The conclusion is that the research completed fails to support these claim and that we consider whether any benefit reported outweighs any harm and/or cost of time out of class. As such, we as related service providers should consider the change of practice in Vermont schools to use interventions that have sound objective research support.

Review: 1. Did the review ask a clearly-focused question? XYes Can’t tell No This article is a review of the available peer reviewed research studies regarding the effectiveness of Brain Gym Program. There are two clearly stated purposes for this article. 1. To review the theoretical basis and research findings on the theories which the developers of Brain Gym base their interventions to claim that the Brain Gym exercises will “enhance” learning 2. To determine whether these claims are based on valid scientific research

2. Did the review include the right type of study? XYes Can’t tell No Consider if the included studies: – address the review’s question- Yes this review looked at research basis for the theoretical foundations of the program (Neurological repatterning, cerebral dominance and perceptual-motor training), & provided a critique of the relevant research findings across the last 60 years of evidence

Evidence-based Journal Club Page 6 – have an appropriate study design- yes the authors addressed the flaws in each of the studies designs when appropriate.

3. Did the reviewers try to identify all relevant studies? Consider: Yes XCan’t tell No – which bibliographic databases were used- Multiple database mentioned but search strategy was not described.- Despite this it appears they did and included all articles those which met their standards of being published in a peer reviewed journal(vs ones where the author paid to have it published)and did not include one article in which the author was also a subject in the study.

They did not mention other strategies used such as contact with authors, but did mention 3-4 other articles that were not reviewed due to not meeting inclusion criteria (such as those that were available for purchase from the brain gym website and one was out of print or had insufficient citing to allow for review). There are many” case reports” or testimonials that were not reviewed.

4. Did the reviewers assess the quality of the included studies? Consider: XYes Can’t tell No – if a clear, pre-determined strategy was used to determine which studies were included. YES, This author REVIEWED STUDIES PUBLISHED IN PEER REVIEWED JOURNALS – The Quality of each article was discussed with a listing and discussion of the methodological problems in each study, including the lack of controls, threats to valiudity, lack of reliability evidence, fidelity of interventions, and inappropriate statistical procedures.

5. If the results of the studies have been combined, was it reasonable to do so? XYes Can’t tell No To Consider the research to support the theoretical basis of the theories that Brain gym is grounded in it was very logical to combine the results:

Neurological repatterning- based on Doman-Delacato Theory of Development- To date, the idea of neural organization or neurological repatterning has not met the rigors of scientific research, and programs that offered repatterning have not resulted in increased learning (American Academy of Pediatrics,1999; Cohen, 1969; Robbins, 1966; Stone & Pielstick,1969). Novella (1996) has gone so far as to state that the Institutes for Achievement of Human Potential are practicing fraud.

Cerebral dominance-(Based on work done by Orton, in 1937) Research has not substantiated the impact of cerebral dominance on learning (Mayringer and Wimmer, 2002; Mohan, Singh, & Mandal, 2001; Pipe, 1988), and after reviewing brain research studies, Hynd and Semrud-Clikeman (1989) determined that the evidence failed to support the contention that deviations in brain morphology were related to learning difficulties.

Perceptual-motor training A considerable amount of research has failed to demonstrate that perceptual–motor training activities are effective academic interventions (Arter & Jenkins, 1979; Bochner, 1978;Cohen, 1969; Hammill,

Evidence-based Journal Club Page 7 Goodman, & Wiederholt, 1974; Kavale& Forness, 1987; Kavale & Mattson, 1983; Sullivan,1972).

To consider the combined available evidence on the effectiveness of Brain Gym exercises the results: Five different research studies were reviewed for this article. One was dropped from the review immediately due to a conflict of interest on the part of the author (he counted himself as one of the subjects in his study.) Taken together, these studies clearly failed to support claims that Brain Gym® movements were effective interventions for academic learning. They were overcome by methodological difficulties, and two studies failed to address academic learning at all.

6. How are the results presented and what is the main result? XYes Can’t tell No -the comparison results of each study design are clearly displayed in a table format in the article and discussed in the written part of the summary.

“Research findings have strongly refuted the theoretical foundations on which Brain Gym® was developed. Neurological repatterning has been described as fraudulent, cerebral dominance has not been linked to learning, and perceptual– motor training has not withstood rigorous scientific investigation.” All studies on the efficacy of Brain Gym had significant flaws rendering them useless to support the continued use of Brain Gym exercises to improve academic learning in schools.

7. How precise are these results? Yes xCan’t tell No Consider: Due to flaws in the designs and statistical errors having these type of result were not possible. – NO confidence intervals were reported – No p-value is reported where confidence intervals are unavailable

8. Can the results be applied to the local population? xYes Can’t tell No Consider whether: – the population sample covered by the review could be different from your population in ways that would produce different results

Samples for the Four Studies reviewed: 1. 60 students identified as having LD age 7-11 2. 60 college men and women age 19-40 3. 25 students identified as having LD 4. 986 students pre-K –grade 5

Many of the same methodological errors that were in the studies done in schools would be similar to control for if preformed in VT schools (Validity threats including the number of different classroom teachers, student ages, teacher training and procedures designed and in place to ensure treatment fidelity across teachers and setting, & controls for natural maturation).

9. Were all important outcomes considered? xYes Can’t tell No Brain Gym is a program that is widely used and is gaining in popularity. Educators are required to use programs, teaching methods, interventions and curriculum that are research based. The author raises the question of benefit vs costs to students of time spend on non academic activities and discusses it below:

Evidence-based Journal Club Page 8 “The No Child Left Behind Act of 2001 and the Individuals with Disabilities Education Improvement Act of 2004 have required that whenever possible, schools must provide students with academic instruction using scientific, research-based methods. Although an exact definition of scientific, research-based methodology is not contained in either law, As providers of educational services we must keep in mind the importance of using and building upon instructional methodologies that have sound empirical support from high-quality research studies. It is time that educators and educational training institutes—whether university or other entity—ensure that practices such as Brain Gym® that have no substantive theoretical or research support are no longer used with children in the hope of ameliorating a learning problem. This responsibility requires that school personnel take the time to critically review instructional programs and select those that have sound, objective research support.”

10. Should policy or practice change as a result of the evidence contained in this review? xYes Can’t tell No

REFERENCE 3: Witcher, S. H. (2001). Effects of Educational Kinesiology, Previous Performance, Gender and Socioeconomic Status on Phonological Awareness Literacy Screening Scores of Kindergarten Students. Virginia Polytechnic Institute and State University, School of Education (Dissertation).

Reviewer: Deb O’rourke

SUMMARY: This study examined the effects of Educational Kinesiology, previous performance, gender and socioeconomic status on emergent literacy as measured by the Phonological Awareness Literacy Screening (PALS) test in kindergarten students. One hundred and twenty-six students from 8 kindergarten classes and 2 relatively matched rural southeastern Virginia public schools participated in this study. Classes were randomly assigned to intervention and control groups. In addition to regular instruction, children in the intervention group participated in a prescribed set of six Brain Gym® exercises lead by their teachers for 8-10 minutes twice each school day for an entire academic year. The control group received regular instruction and did not participate in any Brain Gym® exercises. The PALS, the outcome measure used in this before and after design study, was administered in October as part of the system- wide pre-assessment of kindergarten students and again in the spring. A four-way ANOVA was used to examine multiple main and interaction effects. The primary findings were a significant main effect for previous performance and non-significant effects for Brain Gym® participation, socioeconomic status, and gender. Study limitations include a small sample size and lack of discussion of power or sample size estimates.

STUDY PURPOSE: Outline the purpose of the study (i.e., study objective or aim): Was the purpose The primary purposes were 1) to examine differences in emergent literacy as stated clearly? measured by the Phonological Awareness Literacy Screening (PALS) test in _√__ Yes kindergarten students from southeastern Virginia who did and did not participate ___ No in Brain Gym® exercises and 2) to explore interaction effects between Brain Gym® intervention, previous performance, gender and SES on PALS test scores LITERATURE: Describe the justification of the need for this study (3-4 key points)

Evidence-based Journal Club Page 9 Was relevant The review of the literature was thorough and included relevant information background regarding the intervention (theoretical framework, expected outcomes and literature reviewed? studies), the outcome variable and factors known to have robust effects __√_ Yes (gender, socioeconomic status and previous performance) on the outcome variable DESIGN: Describe the study design: ___ randomized Quasi-experimental study with a pre and post design. ___ cohort (population -based) Can the author answer the study question with the study design? _√__ before and after Yes ___ case-control Were the design and/or method used introducing biases. If so describe: ___ cross-sectional Sample / selection bias – Classes randomly assigned to control /intervention (1+ group at 1 point in groups time) ___ single case design Measurement - Title I teachers administering PALS had no knowledge of study Intervention – Potential for contamination - teachers and students in intervention classrooms might have shared information with those in control classrooms SAMPLE SIZE: Sample Description (e.g., age, gender, diagnosis, other characteristics) N = 126  126 students from 8 kindergarten classes in 2 public schools Was sample size  80 students in intervention group and 46 in control group justified? How was sample identified? Was it a representative sample? ___ Yes The schools were selected because the composition of students was similar and _√__ No because the principals were willing to participate in the study. ___ N/A If there were more than one group, was there similarity and differences between the groups? Describe: Was Power Student demographics were similar between intervention and control groups. Discussed? Was informed consent and assent obtained? ___ Yes _√__ No Yes, parents of children assigned to intervention classrooms were informed of the study and an opt-out option. Consent was provided for all in the intervention group. OUTCOMES: Specify the frequency of outcome measurement (i.e., pre, post, follow-up): Outcome measure Frequency Reliable and Valid? Yes  Phonological  Pre-test – the PALS was  measures young children’s ability to Awareness administered in October as demonstrate fundamental components of the Literacy part of the system-wide pre- learning-to-read process Screening assessment of kindergarten  correlate with specific objectives of (PALS) test students. Standards of Learning for Virginia Public scores  post-test PALS scores were Schools, K - 3rd grade obtained in the spring  The PALS has been extensively evaluated INTERVENTION: Provide a short description of the intervention including type of intervention, Intervention was who delivered it, how often and in what setting.

Evidence-based Journal Club Page 10 described in detail? In addition to regular instruction, students in the intervention classrooms _√_ Yes participated in a prescribed set of six Brain Gym® activities for 8-10 minutes twice a day for an entire year. Brain Gym® activities were lead by classroom Contamination was teachers who received in-service training from the researcher. Students in the avoided? control classrooms received regular instruction and did not participate in Brain __√_ Yes Gym® activities. RESULTS: What were the results? Results were reported Outcomes Results Statistical in terms of statistical ANOVA Main effects: Significance significance? PALS Previous performance p= .001 ___√ Yes ___ No Brain Gym® participation p=.94 ___ NA Gender p=.10 ___ Not addressed SES p=.18 ANOVA Interaction Effect: Gender, SES, Brain Gym® p=.02 ANCOVA Effects not significant Was the analysis Explain: appropriate for the  A t-test was appropriately used to examine between-group differences in type of outcome PALS scores during pre-test phase of study before implementing the measures and the intervention. methodology?  A four-way analysis of variance (ANOVA) was used to examine the main __√_ Yes effects of and interactions among the 4 independent variables of interest __√_ No (Brain Gym® participation, previous performance, socioeconomic status and ___ Not addressed gender) on PALS.  A three-way analysis of covariance (ANCOVA) was also used to examine main effects and interactions among Brain Gym® participation, SES and gender after adjusting for previous performance. The sample is small for this # of analyses. If not statistically significant was study big enough to show an important difference if it should occur? Unknown- no sample size estimates or power calculations were provided. Clinical importance What is the clinical importance of the results? Unable to determine was reported? __√_ No Drop-outs were If yes, why did they drop out? How were drop-out participants included in the reported? statistical analysis? ___√ Yes Two parents in intervention group initially checked the opt-out clause but ___ No decided to participate after the study was explained in more detail. There were no drop-outs. CONCLUSIONS What did the author concluded? AND CLINICAL This study failed to show that Brain Gym® exercises were effective in enhancing IMPLECATIONS: learning-to-read skills in kindergarten students following one year of The conclusions made intervention. There were also no significant differences in pre-test and post-test by the authors were PALS scores based on gender or SES. Students who had higher PALS scores at

Evidence-based Journal Club Page 11 appropriate given pre-test had significantly higher scores at post-test (previous performance). The study methods and author states “it is possible that participation in Brain Gym® exercises may afford results. some academic benefits to low SES males and high SES females. However, __√_ Yes these students would probably benefit more from spending the same amount of ___ No time engaged in an instructionally sound activity.” What were the main limitations of the study as stated by the author(s) and from your point of view?  First, the sample size was small, particularly given the complexity of the analytic plan. For example, the number of students represented in certain ANOVA cells was inadequate to conduct a four-way analysis.  Secondly, the student demographics and teacher experiences in the two schools differed somewhat and may have had some effect on the study outcome.  Third, the author questioned whether a different set of Brain Gym® activities than the 6 selected for this study might have resulted in different effects. What are the implications of these results for your practice? Although this study appears to be underpowered for the number analyses conducted, it was generally well designed and carefully implemented. The failure to show any effect of daily Brain Gym® exercises on emergent literacy in kindergarten students is persuasive and contradictory to the claims made by proponents of this intervention. In the words of the author, “Until more empirical evidence substantiates the purported academic benefits of Brain Gym®, human and financial resources of schools should be focused on using something more clearly researched as producing desirable results.”

REFERENCE 4: Thompson, H. L. (2007). Impacts of Educational Kinesiology Activities on Fourth Graders’ Reading Comprehension Achievement. Walden University, School of Education (Dissertation)

Reviewer: Marie-Christine Potvin

Summary: The study intended to determine if educational kinesiology (i.e., Brain Gym) would improve reading comprehension if 4th graders. Two classrooms were selected (n=40), one was the researcher’s own classroom (treatment) and one was selected as students had characteristics that were most similar to the first one (comparison). Sample size estimation and power were not discussed. The treatment group had the standard reading instructions preceded by 7-10 minutes of Brain Gym activities for 10 weeks. The comparison group had the standard reading instruction preceded by silent reading. Two well developed standardized measured (i.e., Princeton Review and Star Reading test) were used to measure changes pre and post intervention. A third measure was used weekly, the Scott Foresman, the validity and reliability of this measure is unclear. The Princeton Review showed statistical difference between group (p=0.04) in favor of the treatment group. The STAR Reading Test did not show statistical difference between groups (p= 0.244). Scott Foresman Basal Reading showed statistical differences between groups. Clinical importance of the changes found was not discussed.

Evidence-based Journal Club Page 12 Limitations: The teachers were different for each group; no quantitative measure of fidelity was provided. The comparison group had a student with limited English proficiency who lowered the average group score and 2 students with ADHD who were not taking medication. It is unclear whether silent reading is a true neutral addition to the control group. Not sufficient information is provided about the subjects to know who this can be generalized too.

STUDY PURPOSE: Outline the purpose of the study (i.e., study objective or aim): Was the purpose The purpose was to examine the effects of educational kinesiology stated clearly? YES activities on reading comprehension among 4th grade students at Cranberry Elementary School in North Port, Florida. LITERATURE: Describe the justification of the need for this study Was relevant Both the general relevant literature and background to the choice of this background lit. intervention were reviewed. reviewed? YES DESIGN: Describe the study design: ___ randomized Quasi-experimental study pre-post group design. ___ cohort Can the author answer the study question with the study design? __x_ before and after Yes ___ case-control ___ cross-sectional Were the design and/or method used introducing biases. If so describe: (1+ group at 1 point in Selection bias: the 2 classrooms were not selected at random; instead the time) researcher chose them, one being her classroom the other one a classroom with ___ single case design similar demographics. SAMPLE SIZE: Sample Description (e.g., age, gender, diagnosis, other characteristics) N = 40  40, 4th grade students; 2 classrooms – 1 with intervention and 1 control. Was sample size  Equal numbers of boys and girls justified?  Attempts to choose classrooms that match demographics on the county. ___ Yes How was sample identified? Was it a representative sample? __X_ No Convenience sample: Researcher chose 2 classrooms to conduct the study in ___ N/A from 1 school with poor improvement in reading test scores. If there were more than one group, was there similarity and differences between Was Power the groups? Describe: Discussed? ___ Yes Care was taken to find the closest match with regard to race, gender, aptitude, __X_ No socioeconomic status, and behavior between the two classrooms ___ N/A Was informed consent and assent obtained? Yes OUTCOMES: Outcome measures Frequency Reliable and Valid? Sarasota County Planning Tool: The Princeton  Pre and Post at week Yes Review (computerized - Paper) 10 Scott Foresman selection tests ? STAR Reading Test (computerized)  Weekly Yes Pre and Post at week

Evidence-based Journal Club Page 13 10

INTERVENTION: Provide a short description of the intervention: Intervention was The treatment group participated in 7 to 10 minutes of kinesiology activities described in detail? before each reading lesson. The control group participated in silent reading __X_ Yes during this time. The researcher taught reading to the treatment group. Both ___ No groups were exposed to the same reading skills and concepts. Teachers ___ Not addressed participating in the study have been trained in consistent delivery of district curriculum and adhered to Florida’s Sunshine State Standards. This researcher, Contamination was the experimental teacher, and the control teacher planned together and executed avoided? lessons with the same concepts and skills on the same day and at the same time. __X_ Yes The principal, assistant principal, data coach and reading coach conducted ___ No random observations to ensure the teaching styles and data collection materials indicated were implemented throughout the study. RESULTS: What were the results? Results were reported Outcomes Results Statistical in terms of statistical Significance significance? The Princeton Tx group average gain 6%; p=0.04 ___ Yes Review: Pre and control group average loss of ___ No Posttest Results 4% ___ NA ___ Not addressed Scott Foresman p= 0.000 Basal Reading STAR Reading Tx group scored on average . p= 0.244 Test 24 points higher than the comparison group Was the analysis Explain: appropriate for the t test for two independent samples and the independent measures design for outcome measures and between subjects the methodology? If not statistically significant was study big enough to show an important __X Yes difference if it should occur? ___ No Unknown – no sample size estimation or power discussed. ___ Not addressed Clinical importance What is the clinical importance of the results? was reported? NO Unclear whether changes are educationally important Drop-outs were If yes, why did they drop out? How were drop-out participants included in the reported? YES statistical analysis? None CONCLUSIONS What did the author concluded? AND CLINICAL The conclusion drawn from the posttest is that educational kinesiology activities IMPLECATIONS: yield a significant benefit in terms of student performance in reading The conclusions made comprehension. by the authors were What were the main limitations of the study as stated by the author(s) and from appropriate given your point of view? study methods and - Students in the treatment group, collectively, had slightly better results. attendance.

Evidence-based Journal Club Page 14 ___ Yes - The comparison classroom had 1 student from Korea. His scores were __X No consistently the lowest and pulled down the comparison groups averages each time. - While both classes had four ADHD students, two of the four in the comparison group were not medicated while all four in the treatment group were on medication. - 2 teachers – 1 per group; fidelity mentioned but not measured quantitatively - 1 teacher was the evaluator and the researcher - Is silent reading a true neutral addition to the control group or does it risk decreasing attention for subsequent reading instruction. - Not sufficient information is provided about the subjects to know who this can be generalized too. What are the implications of these results for your practice? This study yielded some support to the use of educational kinesiology in the classroom but the study had important flaws so results should be considered cautiously. If educational kinesiology is used in a classroom, it would be essential to use a multi-point data collection system to determine whether this intervention is useful.

REFERENCE 5: Goswami, U. (2006). Neuroscience and Education: From Research to Practice? Nature Reviews Neuroscience, Advance online publication, April 12.

Reviewers: Christine Knippenberg and Pam Barnard

Article Summary: This article brings into question the use of un-vetted practices that claim to reflect new findings in neuroscience. She notes that teachers are very excited, curious, and eager to apply new research in brain-based learning, yet she believes that the research to make this possible is still in its early stages. Packaged programs such as Brain Gym that purport to capitalize on how brains function to enhance learning are not, in her opinion, either sound or appropriate given what we know, and don’t know, to date. Rather, they serve to perpetuate a myth that it is easy to apply brain-based teaching strategies and by doing so, students will become better learners. At the same time, she notes that researchers are unable to provide any credible alternatives to these teachers, leaving them discouraged and dissatisfied. She feels that ex-scientists and others from the research community could perhaps be the best qualified to help translate research into possible practice.

Key Points: The author believes that the current fads in “brain-based” approaches detract from the real research that will lead to real and validated changes in the future. She believes that “Good instructional practice can be undermined by brain-based factors such as learning anxiety, attention deficits and poor recognition of social cues. All of these factors disrupt an individual’s capacity to learn, and also have an effect on other learners in the same classroom.” She then goes on to describe some of the newer research (more than 50 studies) and its possible future promise Dr. Goswami highlights studies in brain research in the areas of reading, math, attention, and social cognition. For her, the gold standard and level of specificity required to examine change is at the brain-imaging level. She cites studies that suggest strong readers can be identified based on early phonological awareness (e.g., pre-readers being able to hear rhyming sounds, or indentify words starting

Evidence-based Journal Club Page 15 with the same sound) that is consistent with brain imaging patterns. Other studies show that when individuals with developmental dyslexia are offered effective programming, their brain studies show more normal processing patterns. Imaging is currently able to record changes in brain functions, but not yet to predict what intervention would be effective. She postulates that in the near future interventions will be based on the information from imaging studies. Interestingly, she discusses a study of the possible role of the cerebellum in reading and a research remediation program using balls and balance equipment. In her opinion, adding neuro-imaging would enable much more accurate assessment of this and many other types of potential interventions. In looking at the development of numeracy skills, she examines a number of studies of both children and adults. Some brain imaging research is showing that children acquire a sense of quantity and comparative amounts at a surprisingly early age, even if they lack some of the linguistics sophistication to express these concepts. Other studies are looking to identify where in the brain specific mathematical operations are processed. Explorations of attention point to research in priming for attending skills with mixed outcomes, and possible latent effects on intelligence over a few days. She looks at research on the role of the amygdala on anxiety/ recognition of facial gestures with individuals with autism and coming from homes with abuse and other social challenges. A brief introduction to newer research on “mirror neurons” and the postulated way they allow us to recognize and understand emotions and actions of others is nicely provided. She believes that if we can accurately identify brain abnormalities within different populations, then we can design effective interventions to alter brain function and improve learning. Finally, the author comments on the selling of brain-based packages based on “myths”—that their marketing is effective in promoting optimism but that any changes are most likely short-lived and due to the placebo effect. She notes that scientists, who should be in a position to know what is best, may not be able to effectively represent the research findings and possible next steps to the audience of educators who she finds want a more “big-brush” approach. Developing a “network of communicators” is suggested as a way to explain what the neuroscience breakthroughs really mean to the general public. If these individuals are working for a public agency rather than a not-for-profit, there is a better likelihood that their ideas are not driven by commercial motives.

Response to Key Points: A cursory look on the Internet shows that Dr. Goswami has spent her career examining the neurological details of issues such as phonology, numeracy, and other very discrete components of learning and function along with child development. Clearly she is an informed and engaged commenter on the brain-based research scene. Having such detailed research distilled into simplistic models for consumption by the general public must be maddening in a way. At the same time, she is blunt in stating that there really isn’t much yet to use via brain research that can translate directly into action in the classroom. Her solution is to increase the role of brain imaging across all aspects of learning and all populations, pairing potential interventions with images of their results. The research on attention and anxiety is of course promising, yet also so limited as to make us who need to deal with this on a daily basis frustrated in trying to extract possible tools from preliminary research. New research, though, continues to modify earlier, and often, more simplistic work, and so we constantly have to re-think and revise based on new information. The latest research on mirror neurons is really exciting in terms of identifying some discrete reasons why certain recognitions functions are not emerging in autism, as well as offering exactly what she proposes—a better way to measure if what we do for teaching will make a difference. The three factors she states that significantly impact ANY learning and which are often important concerns to related service providers are anxiety, attention deficits, and poor recognition of social cues. Many of us are very aware that if we can positively affect the overall “set” of a student—helping them

Evidence-based Journal Club Page 16 experience success, feel part of a larger community, and mobilize their attention, these students will learn more and more effectively than when they are frightened, confused, disorganized, or socially isolated. We can use simple low-tech empirical methods to determine if we have created positive learning environments for children resulting in improved learning—no need to wait for those pesky brain-imaging studies. As for the myths, my only complaint with this article is that many of the brain-based strategies purport to work at a systemic level while the imaging research she cited is at a discrete function level. Other articles show little efficacy for these methods, but in this article, it feels a little like comparing apples with futuristic (and not quite yet ready for general consumption) oranges.

REFERENCE 6: Jennifer Stephenson (2009). Best practice? Advice provided to teachers about the use of Brain Gym® in Australian Schools. Australian Journal of Education, 53(2), 109-124.

Reviewers: Linda Kogut and Pam Barnard

Main Points The intent of this article was to be informative, particularly to those in the education field regarding the perceptual motor program, Brain Gym® and to document its value or lack of as being evidenced based. Those responsible for education professional development state that teachers should use programs that are evidenced based. Perceptual and motor skills and programs often claim, that they will positively impact/improve academic learning.

 The author cited many articles from several authors to demonstrate that there is no scientific evidence to support the Brain Gym® program: Hyatt (2007), Maskell et. al (2004).  The Brain Gym® web site offers a document summarizing research, but the majority of studies were not published in the peer-reviewed literature where studies are open to scrutiny and criticism.  Neuroscientists and educators with expertise in neuroscience and cognition have stated that there is no evidence that specific parts of the brain can be stimulated by physical exercise. One report stated, “The scientific terms that are used to explain how this (Brain Gym®) works, let alone the concepts they express, are unrecognizable, within the domain of neuroscience” (Teaching and Learning Research Program, 2007).

The aim of Stephenson’s work was to investigate through the exploration of websites, sources that were presumed to be resources to teachers regarding the Brain Gym® program in Australian schools. Using internet sites, the author completed a search 42,900 hits. The results of the searches were pooled. A summary of the classifications, nature of the sites and documents and relevant claims about Brain Gym® were included in the article.

 Although education departments are responsible for providing advice/professional development, administrators continue to endorse and support Brain Gym®.  All state and territory educations departments provided some level of explicit support for the use of Bain Gym®.  No source offered empirical research on Brain Gym®.  There was active provision and recommendation of Brain Gym® workshops as a form of professional development.  Despite evidence, there is apparently little change in beliefs about the program’s efficacy.

Evidence-based Journal Club Page 17  The author acknowledged that using exercises such as Brian Gym® may increase a student’s level of alertness by providing a break from academics. The children appear refreshed, which reinforces the belief that Brain Gym® is beneficial.

Brain Gym’s® claims have not been substantiated in its 30 plus year history. Its theoretical basis is not accepted by neuroscientists. Brain Gym® has been linked with the rise of ‘neuro-myths and brain-based’ teaching. It would appear that Brain Gym’s® marketing department has done an excellent job, enticing teachers and others to buy into the program.

It would appear that educational entrepreneurs or marketing professionals have been successful in selling Brain Gym®. Their influence on the educational community has unfortunately been quiet successful. “The community, parents and teachers are being badly let down by the continued endorsement of Brain Gym® and the lack of critical information about it provided by education departments. Who guards the guards?”

Opinion of Reviewer

I knew of Brain Gym® only by the ads in OT publications. I volunteered to participate in this Journal Club in order to familiarize myself with this program and read additional articles to get a sense of this author’s objectivity and accuracy. She was consistent with those sources. Stephenson’s article was clear and concise. She supported her statements throughout the article. As did other authors, she was very kind to teachers- crediting them with the desire to offer credible programs to their students, but not researching a product prior to using it. It seems that that responsibility is left to program directors and administrators who profess evidenced-based practice.

The Stephenson article was useful in that-  The information was presented in a “no-frills, no-nonsense” manner.  She focused on her intent, which was to inform the educational and rehabilitation communities about the need for, but lack of research that supports the Brain Gym® program.  She consistently substantiated her initial statements.  An extensive reference list was appreciated and useful

TRIPSCY Evidence-based Journal Club Questions to Guide Clinical Implications Discussion

1. Are the results of the studies reliable and valid (quality of studies)?

2. What do these articles mean for your clinical practice?

3. Take-home message?

Evidence-based Journal Club Page 18 TRIPSCY Journal Club Evaluation December 2010

SELF ASSESSMENT: For each item, circle the number, which most appropriately represents your evaluation of your skills at this time. The highest rating is 5 (strongly agree) and the lowest is 1 (strongly disagree).

ACCESS 1. I can define evidence-based practice. 1 2 3 4 5 2. I have access to quality literature to inform my practice. 1 2 3 4 5 3. I know how to conduct an efficient literature search. 1 2 3 4 5 4. I know how to pose a question that will yield evidence that can be 1 2 3 4 5 applied to my practice. CRITIAL APPRAISAL SKILLS 1. I can ascertain whether the sample of a study was appropriate (e.g., 1 2 3 4 5 sample size, minimum bias & similarity between groups) 2. I can identify the study design and the level of evidence associated 1 2 3 4 5 with it. 3. I can ascertain whether the assessments or outcome measures used in 1 2 3 4 5 a study are valid and reliable. 4. I can ascertain whether the analysis method of a study was 1 2 3 4 5 appropriate. 5. I can ascertain the applicability of a study to the children and team I 1 2 3 4 5 work with. IMPLEMENTATION 1. I have strategies to implement newly acquired study findings into my 1 2 3 4 5 practice. 2. I have strategies to ascertain whether newly implemented practices 1 2 3 4 5 are resulting in better outcome for the students/children I work with. 3. I am comfortable sharing relevant study findings with parents, school 1 2 3 4 5 staff and other health professionals. 4. I can prepare quality evidence-based review fact sheet. 1 2 3 4 5

Evidence-based Journal Club Page 19