INTERPRETING KEY TRIALS

JOSEPH BERNSTEIN, MD TONY QUACH, BA Assistant professor of , University of Pennsylvania University of Pennsylvania, Philadelphia

A PERSPECTIVE ON THE STUDY OF MOSELEY ET AL Questioning the value of arthroscopic surgery for

■ ABSTRACT This popularity implies that the operation is well received by both doctors and patients. for degenerative conditions of Nevertheless, its effectiveness (like the effec- the knee is among the most commonly tiveness of many surgical operations) has employed orthopedic procedures, but its never been proven in prospective trials, and effectiveness (like the effectiveness of many the precise mechanism by which it may surgical operations) has never been proven improve the course of degenerative conditions in prospective trials. Moreover, the precise of the knee has not been established conclu- mechanism by which arthroscopy improves sively. the course of degenerative conditions of the knee has not been established conclusively. See related editorial, page 384 Moseley et al performed a double-blinded, randomized, placebo-controlled trial to com- To address these issues, a double-blind- pare the effectiveness of arthroscopic lavage ed, randomized, placebo-controlled trial and arthroscopic debridement vs a sham was undertaken by J. Bruce Moseley, MD, Exactly how procedure. Data regarding pain and function and colleagues.1 This study compared the arthroscopy were obtained at multiple time points over a effectiveness of arthroscopic lavage and relieves 2-year period. The authors found that all arthroscopic debridement to a sham proce- three treatment groups fared equally: each dure. Data regarding pain and function has never been reported subjective symptomatic relief, but were obtained at multiple time points over a 2-year period. proven no objective improvement in function was In this article we will outline the findings noted in any of the groups. These data sug- of the Moseley trial and some of the interest- gest that the benefits of arthroscopy for the ing questions it raises. treatment of osteoarthritis of the knee is to provide subjective pain relief, and that the ■ RATIONALE FOR ARTHROSCOPY: means by which arthroscopy provides this MODERATE OSTEOARTHRITIS IS VEXING benefit is via a placebo effect. Minor symptoms of osteoarthritis of the knee RTHROSCOPIC SURGERY is frequently can be managed with pain relievers such as A employed to ameliorate the symptoms acetaminophen or nonsteroidal anti-inflam- associated with degenerative conditions of matory drugs. Severe, end-stage osteoarthritis the knee. Moseley et al1 estimate that at can be treated with total . least 650,000 such procedures are performed Moderate —too severe for simple each year in the United States, making it medications but not bad enough to warrant the second most commonly employed —is more vexing. For years, orthopedic procedure (ranked behind only orthopedic surgeons have offered arthroscopic arthroscopy of the knee for nondegenera- lavage and debridement as treatments for tive conditions). moderate osteoarthritis of the knee.

CLEVELAND CLINIC JOURNAL OF MEDICINE VOLUME 70 • NUMBER 5 MAY 2003 401 Downloaded from www.ccjm.org on September 28, 2021. For personal use only. All other uses require permission. ARTHROSCOPIC SURGERY BERNSTEIN AND QUACH

Why arthroscopy might be effective efficacy of arthroscopic surgery of the knee in In theory, arthroscopy for arthritis should relieving pain and improving function in relieve symptoms and even alter the natural patients with osteoarthritis.” history of the disease by several mechanisms. The premises the authors assert are rea- Lavage can remove debris and inflammatory sonable. The questions they pose, therefore, cytokines that cause synovitis.2,3 Debridement are most worthy of study. can remove torn meniscal fragments and loose articular . These steps should mini- ■ STUDY DESIGN AND METHODS mize mechanical symptoms and improve the distribution of weight on the joint surfaces.4–7 Moseley et al compared the subjective and objective outcomes attributed to arthroscopy Why arthroscopy might not be effective for knee osteoarthritis after three types of There are equally powerful reasons to suggest interventions: arthroscopic debridement, that arthroscopy has limitations: arthroscopic lavage, and sham (placebo) • Arthroscopy addresses only surface phe- surgery. The primary hypothesis was that the nomena, and the pathology of osteoarthritis pain level as measured 2 years after the proce- that generates pain may reside far from the dure would be no different in the three groups, surface (for example, deep within the ) regardless of the type of intervention. • The damaged articular cartilage itself may The protocol was approved by the hospi- play only a contributory role in the clinical tal’s institutional review board. manifestations of the disease: for instance, car- tilage has a limited ability to sense pain. Inclusion and exclusion criteria • Because the protective lamina splendens Participants were recruited from the Houston of the articular cartilage is disrupted in Veterans Affairs (VA) Medical Center over a osteoarthritis, even if arthroscopic removal of 3-year period. Eligible subjects were all those cartilage debris helps the patient, arthroscopy age 75 or younger who presented to the ortho- cannot prevent more debris from accumulat- pedic clinic for consideration of arthroscopic ing. Thus, the benefit of arthroscopy on the management of osteoarthritis of the knee as There are basis of “cleaning out” the joint alone is apt to defined by the American College of good reasons be short-lived. Rheumatology, and who had at least moderate knee pain despite maximal medical treatment arthroscopy ■ RATIONALE FOR THE MOSELEY TRIAL for at least 6 months. may be Patients were excluded if they had serious Further clouding the picture is the evidence medical conditions, if they had undergone effective— from prior clinical studies. Many published arthroscopy of the knee within 2 years, or if or not studies have found that arthroscopy offers a they had radiographic evidence of severe positive effect. Nevertheless, these studies degeneration or malalignment. cannot serve as the final word, as one could Of the 324 consecutive patients who met reasonably deem them methodologically the criteria for inclusion, 144 (44%) declined flawed. to participate. The remaining 180 were strati- For instance, most studies lacked random- fied into three groups according to the severi- ization, control groups, and blinding of ty of osteoarthritis. Participants in each of the observers. Also, publication bias may skew the three severity classes were randomly assigned presentation, ie, “positive” studies are the ones to one of three treatment groups (see below). more apt to be published. In sum, prior studies are not definitive. The operations: Moseley et al1 state “the physiological Lavage, debridement, and sham surgery basis for the pain relief offered by arthroscopy Dr. Moseley performed all the operations. for osteoarthritis is unclear. There is no evi- Arthroscopic lavage was performed with dence that arthroscopy cures or arrests the general anesthesia and included diagnostic osteoarthritis. Therefore, we conducted a ran- arthroscopy and a washout of the joint with domized, placebo-controlled trial to assess the saline.

CLEVELAND CLINIC JOURNAL OF MEDICINE VOLUME 70 • NUMBER 5 MAY 2003 405 Downloaded from www.ccjm.org on September 28, 2021. For personal use only. All other uses require permission. ARTHROSCOPIC SURGERY BERNSTEIN AND QUACH

outcome assessment. Postoperatively, partici- Arthroscopic surgery vs placebo surgery pants in none of the three treatment groups for osteoarthritis of the knee were more likely to guess whether they received the placebo surgery. 100 Debridement 90 Lavage Placebo Follow-up measures 80 Data were collected at 2 weeks, 6 weeks, 3 70 months, 6 months, 12 months, 18 months, and 24 months after the procedure. 60 Pain was measured on a self-reported 50 knee-specific pain scale, with scores ranging 40 from 0 (no pain) to 100 (most severe pain). 30 Additional established scoring systems were used to ensure validity. 20 Function was assessed via the walking- 10 bending subscale of the Arthritis Impact Mean Knee-Specific Pain Scale score 0 Measurement Scale, and with a physical func- Before 2 wk 6 wk 3 mo 6 mo 1 yr 18 mo 2 yr tioning scale devised for this study. The physi- procedure After procedure cal functioning scale recorded the time in sec- onds that a patient required to walk 100 feet FIGURE 1. Mean values and 95% confidence intervals and to climb up and down a flight of stairs. on the Knee-Specific Pain Scale show similar outcomes in all three groups. ■ RESULTS: PAIN IMPROVED IN ALL FROM MOSELEY JB, O’MALLEY K, PETERSON NJ, ET AL. A CONTROLLED TRIAL OF ARTHROSCOPIC GROUPS, FUNCTION DID NOT SURGERY FOR OSTEOARTHRITIS OF THE KNEE. N ENGL J MED 2002; 347:87–88. Of the 180 patients who started the trial, 16 were lost to follow-up. Arthroscopic debridement was performed All three groups reported an initial in like manner, but in addition to the lavage, decrease in knee pain, an effect that faded areas of rough articular cartilage were shaved, somewhat over the 2 postoperative years loose debris was mechanically removed, and (FIGURE 1). At no point did arthroscopic torn menisci were trimmed. lavage or debridement provide significantly The placebo procedure was “simulated greater pain relief than the sham proce- surgery”: the patient was sedated, prepped, dure. Indeed, the placebo group reported and draped, and incisions were made in the the greatest net improvement in pain skin, but the joint was not entered. Saline was scores on the Arthritis Impact Measure- dripped on the floor and instruments were ment Scale. handed back in forth in case a patient would Similarly, all three groups reported equiv- be able to recall the experience. alent scores for function: on the walking- In all, 61 patients underwent lavage bending subscale of the Arthritis Impact alone, 59 underwent debridement, and 60 Measurement Scale, the groups that under- underwent sham surgery. The characteristics went the true arthroscopic interventions did of all three groups were comparable. no better than the group that underwent placebo surgery, and at a few points in time Postoperative care they did worse. Postoperative care was uniform across all In sum, the Moseley study found: groups. The use of analgesics after surgery was • All three groups were roughly the same similar in the three groups. The patients, the in terms of subjective and objective nurses involved with postoperative care, and findings; the study personnel involved with follow-up • Pain improved subjectively in all three data collection were blinded to the treatment groups; and groups. The surgeon did not participate in any • Function did not improve in any group.

406 CLEVELAND CLINIC JOURNAL OF MEDICINE VOLUME 70 • NUMBER 5 MAY 2003 Downloaded from www.ccjm.org on September 28, 2021. For personal use only. All other uses require permission. ARTHROSCOPIC SURGERY BERNSTEIN AND QUACH

■ STRENGTHS OF THE STUDY arthroscopic surgery that it may have skewed their responses in the subjective outcome The study by Moseley et al is noteworthy for measures.1 its inclusion of a sham treatment—a rarity in surgical studies. Using a placebo group is the Other limitations only way to allow inferences to be drawn Another limitation of the study is that a single about a possible placebo effect. surgeon performed all of the procedures. The The study’s sample size afforded 90% sta- authors defended this by noting the surgeon’s tistical power: that is, if differences were to impeccable credentials, as well as some exist between groups, it is highly likely that impressive but irrelevant experience he had the study would detect these differences. (he is the orthopedic surgeon for a National Follow-up was detailed, and fewer than 10% of Basketball Association team and was the patients were not included in the final assess- physician for the US Olympic basketball ment. The design of the study was robust and team—however, candidates for neither posi- well conceived. tion are chosen on the basis of technical pro- ficiency at arthroscopy). ■ LIMITATIONS OF THE STUDY But the question is not one of technical proficiency but of bias. It is possible that (per- The principal limitation of the study is not haps subconsciously) the operating surgeon a flaw of the study per se, but rather of its did not believe in the effectiveness of debride- context. That is, this was a study from a sin- ment and therefore did not try to do the best gle institution that included 180 patients, possible debridement operation. Since there but osteoarthritis affects millions of people. are no videos of the surgeries, this criticism Just as one swallow does not make a sum- cannot be refuted. mer, one study, however robust, cannot and Accordingly, a perhaps better way to com- probably should not turn practice on its pare surgical outcomes in a head-to-head trial head. is to have surgeons who are advocates of each Is improvement procedure perform the operation they favor. In after Patient selection bias that case, there will be no question then that Moseley et al admit that patients recruited effort was withheld. from the Houston VA hospital “may not be The study also did not report preopera- also a placebo representative of all candidates for arthroscop- tive range of motion, nor did it note mechan- ic treatment of osteoarthritis of the knee.”1 ical symptoms or effusions. At the time of effect? The population was mostly male, and the surgery, the pathologic findings were not cat- investigators state that they do not know alogued and photographed. It must be said in whether their findings would be the same in defense of the authors that any bias intro- women, although responses to arthroscopic duced by differences in these parameters surgery are not known to differ between the should have been washed out by the random- sexes. ization process. Other factors may be even more signifi- Along those lines, one may say that this cant, such as the prevalence of cases involving study may have indeed invalidated the use of disability ratings and secondary gains. arthroscopy for osteoarthritis of the knee in The patients’ mental bias could also affect patients who met the study’s inclusion crite- the results, at least the subjective outcome ria but that these inclusion criteria were too measurements. broad and arthroscopy based on these indi- As the authors point out, 44% of eligible cations should be invalidated. That is, the patients declined enrollment because they did reason this population of patients did not not want to take the chance of not undergoing show improvement following arthroscopic a real procedure. The 56% who did enroll— surgery was because of poor patient selec- and who thereby chose to take a one-in-three tion. Of course, that raises the question chance of not undergoing a real operation— whether all 650,000 operations that are per- may have had such high expectations of formed annually employ the rigorous indica-

408 CLEVELAND CLINIC JOURNAL OF MEDICINE VOLUME 70 • NUMBER 5 MAY 2003 Downloaded from www.ccjm.org on September 28, 2021. For personal use only. All other uses require permission. tions the critics suggest (ie, cases of Implication 3: osteoarthritis featuring mechanical symp- Other procedures called into question toms such as locking or effusions, without If the placebo effect plays a role in the out- simultaneous involvement of both the medi- come of knee arthroscopy for osteoarthritis, al and the lateral side). does it also play a role in other procedures offered for pain relief, such as for ■ IMPLICATIONS OF THE STUDY axial back pain and acromioplasty for shoul- der impingement? Since its publication, the Moseley study has caused a great deal of controversy. What are Implication 4: the implications of this study? Possible effects on reimbursement The authors conclude that “if the efficacy of Implication 1: arthroscopic lavage or debridement in Public awareness of placebo effect patients with osteoarthritis of the knee is no The study invites the possibility that the ben- greater than that of placebo surgery, the bil- efits of arthroscopy of the knee for osteoarthri- lions of dollars spent on such procedures tis are due to the placebo effect. Since annually might be put to better use.”1 arthroscopy is such a common procedure, this Well, maybe. When doctors treat arthri- is big news. Indeed, the New York Times ran an tis, they are attempting, foremost, to offer sub- article about the study on the front page,8 and jective improvement. After all, the reason all major media picked up the story. As a patients come to them is because of a subjec- result, many patients were made aware of a tive complaint: pain. If arthroscopy offers sub- placebo effect of surgery. What impact this jective improvement, regardless of mecha- will have on patient attitudes (given that in nism, it can be deemed successful. this country people spend tens of billions of Accordingly, one should state that funds dollars each year on alternative medicine) may be put to better use only if there is a remains to be seen. cheaper means to achieve that benefit, but the authors did not show this. After all, a Even a placebo Implication 2: placebo benefit is still a benefit. The obvious benefit has Fewer question is whether this benefit could be The New York Times front-page report of the attained at lower cost and with lower risk. value Moseley study findings appeared under the headline “Arthritis surgery in ailing is Implication 5: cited as sham.”8 Because of this and other sim- Personal observation is limited ilar articles, the Arthroscopy Association stat- Because a subjective benefit was reported ed that the “casual reader of these articles across all groups, a surgeon monitoring his or might think that arthroscopic surgery is use- her own practice (and observing such subjec- less for any type of knee surgery involving tive happiness) would be correct in conclud- arthritis.”9 ing that arthroscopy provides real benefit. This would be unfortunate. Still, many But that conclusion could be wrong. The physicians may become more reluctant to rec- perceived benefit could be a placebo effect or ommend arthroscopy (at least for the short simply a manifestation of the natural history term). More than that, the Department of (that is, the waxing and waning) of the dis- Veterans Affairs (which sponsored the ease. Moseley study and therefore can be confident The lesson from the Moseley study is that that the results apply to its population) has only with control groups and blinded issued an advisory to its doctors recommend- observers can one make definitive statements ing that they not perform arthroscopy for about the effectiveness of a treatment. osteoarthritis of the knee without clear clini- Personal observation of one’s own practice is cal evidence of significant derangement or obviously the first step toward evidence-based symptoms due to anatomic and mechanical medicine, but it is clearly not the only step abnormalities. either.

CLEVELAND CLINIC JOURNAL OF MEDICINE VOLUME 70 • NUMBER 5 MAY 2003 409 Downloaded from www.ccjm.org on September 28, 2021. For personal use only. All other uses require permission. BERNSTEIN AND QUACH

Implication 6: The value of arthroscopy must be proved All other commentary notwithstanding, the value of “cleaning out” the arthritic joint with arthroscopic surgery, in the absence of painful lesions defined preoperatively, has certainly been called into question. The challenge is now made for researchers to repeat the Moseley methodology, taking into account the various criticisms, and demonstrate the effec- tiveness of arthroscopy. In at least a small mea- sure, the burden of proof has shifted.

■ REFERENCES

1. Moseley JB, O’Malley K, Peterson NJ, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002; 347:87–88. 2. Evans CH, Mazzocchi RA, Nelson DD, Rubash HE. Experimental arthritis induced by intraarticular injection of allogenic cartilaginous particles into rabbit knees. Arthritis Rheum 1984; 27:200–207. 3. Smith MD. Triantafillou S, Parker A, Youssef PP, Coleman M. inflammation and cytokine pro- duction in patients with early osteoarthritis. J Rheumatol 1997; 24:365–371. 4. Baumgaertner MR, Cannon WD Jr, Vittore JM, Schmidt ES, Maurer RC. Arthroscopic debridement of the arthritic knee. Clin Orthop 1990; 253:197–202. 5. McLaren AC, Blokker CP, Fowler PJ, Roth JN, Rock MG. Arthroscopic debridement of the knee for osteoarthritis. Can J Surg 1991; 34:595–598. 6. Sprague HF III. Arthroscopic debridement for degenerative knee joint. Clin Orthop 1981; 160:118–123. 7. Jennings JE. Arthroscopic debridement as an alternative to total . Arthroscopy 1986; 2:123–124. 8. Wald ML. Arthritis surgery in ailing knees is cited as sham. New York Times 2002 Jul 11;Sect. A:1 (col. 6). 9. Burkhart SS, Ewing JW. Knee arthroscopy for osteoarthri- tis. Arthroscopy Association of America position paper. Available at www.aana.org/membership/OAposition.html. Accessed February 4, 2003.

ADDRESS: Joseph Bernstein, MD, Department of Orthopedic Surgery, University of Pennsylvania, 424 Stemmler Hall, Philadelphia, PA 19104; e-mail [email protected].

410 CLEVELAND CLINIC JOURNAL OF MEDICINE VOLUME 70 • NUMBER 5 MAY 2003 Downloaded from www.ccjm.org on September 28, 2021. For personal use only. All other uses require permission.