An Original Study D. P. Feghhi et al Critical Review of Patient Education Materials From the American Academy of Orthopaedic Surgeons Daniel P. Feghhi, MD, Nitin Agarwal, MD, David R. Hansberry, MD, PhD, Wayne S. Berberian, MD, and Sanjeev Sabharwal, MD the medical conditions and symptoms they are experiencing.3 Abstract About 80% of Americans report using an Internet resource to We performed an expanded readability analysis to seek answers to health questions.3,4 Studies have identified a determine if the American Academy of Orthopae- correlation between “health literacy” and clinical outcomes: dic Surgeons (AAOS) had sufficiently improved its lower health literacy is associated with adverse patient out- patient education materials since they were originally comes, such as more frequent hospitalizations and emergency studied in 2007. department visits, and higher health literacy is associated with 4-6 In March 2013, we downloaded patient education more favorable patient outcomes. Given these findings, it is materials from the AAOS patient information website, essential to ensure that the health care information being ac- cessed is easily comprehended. Your Orthopaedic Connection. Using 10 different Orthopedic conditions are among the most common condi- readability formulas, we found that the mean grade tions that cause patients to seek medical care.6-9 Many ortho- level of patient education materials on the website is pedic conditions call for surgical intervention. Controversy 8.84. Flesch-Kincaid analysis showed a mean grade arises regarding nonoperative and operative treatments for level of 9.98 (range, 6.6-12.6). Nine other readability certain diseases. Once a patient has been diagnosed with a analyses showed a mean reading level of 7.7 (range, particular injury or condition, it is almost instinctive to seek 6.5-13.7). Although this is an improvement over the an immediate online source describing it. Online patient infor- 2007 level, it is above the average national reading mation should be composed at an appropriate reading level to comprehension level. prevent placing undue stress on patients and to foster healthy The readability of patient education materials on dialogue at office visits. Badarudeen and Sabharwal10 found in the AAOS website still exceeds the average reading their study that the mean grade level of patient information ability of a US adult. Revisions made over the 5 years in the children sections of the AAOS and Pediatric Orthopae- leading up to this latest study resulted in better read- dic Society of North America (POSNA) websites was 8.9. The ability. The Prevention & Safety entries, written near estimated mean national reading level is about seventh grade. seventh-grade level, should serve as a model for the The American Medical Association (AMA) and the National remaining articles. Institutes of Health (NIH) recommended that the readabil- ity level for patient education information be no higher than seventh grade.11 In addition, the Centers for Disease Control and Prevention (CDC) recommended that the readability of ne of the goals of the American Academy of Ortho- patient education materials be between sixth grade and sev- paedic Surgeons (AAOS) is to provide patient educa- enth grade.12 O tion materials in line with the readability skills of the In April 2007, Sabharwal and colleagues2 performed a read- general patient population. The mean reading grade level in ability analysis of patient education materials available on the the United States is estimated at about seventh grade.1 A study AAOS website. The Flesch-Kincaid grade formula (FKGL), used completed in 2007 found that the mean grade level of patient to analyze all the listed articles, computed a mean grade level education materials on the AAOS website was 10.43, according of 10.43.2,10,13 to Flesch-Kincaid analysis.2 Using 10 different readability formulas, we performed an The Internet is fast becoming a primary source of health expanded readability analysis of all entries in the patient educa- care information. Every day, more than 8 million Americans tion library of the AAOS website to determine if any readability use Internet resources to try to answer their questions about improvements had been made since the 2007 study. Authors’ Disclosure Statement: The authors report no actual or potential conflict of interest in relation to this article. E168 The American Journal of Orthopedics® August 2014 www.amjorthopedics.com An Original Study D. P. Feghhi et al Materials and Methods for all 6 body part subsets in the category of Diseases & Conditions The “readability” of text is a metric that can be used to deter- was above the seventh-grade reading level (range, 8.03-9.36) mine the level of reading comprehension required to under- (Tables I, II). On analysis of Treatments & Surgeries, all 6 body stand online patient information. This metric can be measured part subsets contained information above the seventh-grade with numerous commonly used assessments, such as the FKGL, reading level (range, 8.55-9.65) (Tables I, II). The information the Flesch Reading Ease score (FRE), the Simple Measure of in Shoulder & Elbow, Treatments & Surgeries had the highest aggregate Gobbledygook (SMOG), the Gunning Fog Index (GFI), the New reading level, 9.65 (Table II). Dale-Chall formula (NDC), the Coleman-Liau Index (CLI), the The AAOS website provided Prevention & Safety information New Fog Count (NFC), the Raygor readability estimate (RRE), on Foot & Ankle, Hip & Thigh, and Neck & Back. Readability analyses the FORCAST formula, and the Fry readability graph (FRG).14-19 suggested that all this information is at the seventh- to eighth- The AAOS website’s patient education library features 6 grade reading level (range, 6.18-7.23) (Tables I, II). categories of body parts: Shoulder & Elbow, Hand & Wrist, Hip & NFC analysis produced a mean readability grade level of Thigh, Knee & Lower Leg, Foot & Ankle, and Neck & Back. Each has 8.18, and SMOG analysis produced a mean level of 12.37 (Ta- subcategories Broken Bones & Injuries, Diseases & Conditions, Treatments bles I, II). FRE analysis of patient information in Foot & Ankle and & Surgeries, Prevention & Safety, and Patient Stories.20 On the website, Neck & Back suggested that the materials are written in difficultto we found a total of 260 entries; 9 were deemed unsuitable plain English (Figures 1, 2). Difficult indicates easily understood for readability analysis. Patient Stories entries were omitted be- by college graduates, plain English indicates easily understood cause they contained many direct quotations from patients. by eighth graders, and fairly difficult indicates easily understood Therefore, the 9 entries were not provided with the average by 13- to 15-year-old students.21 With FRE analysis, entries US reading level in mind. All information was found at http:// pertaining to Knee & Lower Leg and Hip & Thigh were described as orthoinfo.aaos.org. difficultto fairly difficult( Figures 3, 4). Likewise, with FRE analysis, Each entry was copied into Microsoft Word (Microsoft, entries in Shoulder & Elbow and Hand & Wrist were difficult to fairly Redmond, Washington), and a readability analysis was per- difficult (Tables I, II). formed with Readability Studio Professional Edition Version Reading-level ranges for sections in Foot & Ankle, Hand & Wrist, 2012.1 for Windows (Oleander Software, Vandalia, Ohio). and Hip & Thigh were 7.14 to 8.55, 7.78 to 8.76, and 7.23 to 9.16, Scores from the 10 readability assessments previously men- respectively. Reading-level ranges for sections in Knee & Lower tioned were calculated. Leg, Neck & Back, and Shoulder & Elbow were 8.23 to 9.36, 6.18 to The process by which the readability scores for all patient 8.92, and 8.43 to 9.79, respectively. information were calculated is best understood by example. Ten methods were used to evaluate the patient information in Discussion the Broken Bones & Injuries part of the Knee & Lower Leg section, and Other studies have shown that health literacy and health out- a mean was calculated. This was done for each section. Values comes are related.5,22 Better health outcomes can be achieved were averaged to produce an overall readability level for the in part through improved health literacy. Patients with good specific body part. health literacy are, in general, healthier.5,6,22,23 According to The mean grade level generated by each readability method Health Literacy: A Manual for Clinicians, produced by the AMA,24 the was calculated as well. For example, RRE generated a grade best predictor of a person’s health status is that person’s health level for Broken Bones & Injuries, Diseases & Conditions, and Treatments literacy. It follows that the lower a person’s health literacy, the & Surgeries and calculated their mean. This was done for each more likely the person is in poor health. Poor health literacy readability scale, and a mean grade level of all the methods can lead to more frequent hospitalizations and office visits, was calculated. additional testing, increased overall morbidity, and higher The mean grade level of each specific section and the mean medical expenses.5,6,22,23 A more health-literate patient can be of all readability analyses were averaged together to generate more efficient during an office encounter. The patient can -en a final reading grade level for the body part. The final read- gage in an informed dialogue and present directed questions ing grade level for each body part was averaged to produce about his or her medical condition. This can improve the phy- an overall readability level of the patient information on the sician–patient relationship.10 To obtain these benefits, patients AAOS website. must be able to access comprehensive, readable information. The Internet can be a source of this information. Results Internet access and use of the Internet as a source of health Applying 10 different readability assessment tools produced a information are rapidly increasing throughout the world.10 In mean readability level of 8.84 for the patient information on one study, 75% of patients who attended an orthopedic clinic the AAOS website.