Quantification of the Hawthorne Effect in Hand Hygiene Compliance Monitoring Using an Electronic Monitoring System: a Retrospective Cohort Study
Total Page:16
File Type:pdf, Size:1020Kb
BMJ Quality & Safety Online First, published on 7 July 2014 as 10.1136/bmjqs-2014-003080ORIGINAL RESEARCH BMJ Qual Saf: first published as 10.1136/bmjqs-2014-003080 on 7 July 2014. Downloaded from Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study Jocelyn A Srigley,1,2 Colin D Furness,3,4 G Ross Baker,1 Michael Gardam5,6 1Institute of Health Policy, ABSTRACT healthcare-associated infections (HAIs), Management & Evaluation, Background The Hawthorne effect, or which are the most common adverse University of Toronto, Toronto, Ontario, Canada behaviour change due to awareness of being event experienced by patients during 1 2Department of Medicine, observed, is assumed to inflate hand hygiene medical care. HCW hand hygiene is McMaster University, Hamilton, compliance rates as measured by direct known to be suboptimal2 and multifa- Ontario, Canada observation but there are limited data to ceted improvement programmes are 3Infonaut Inc, Toronto, Ontario, Canada support this. recommended, including measurement 3 4Faculty of Information, Objective To determine whether the presence and feedback of compliance rates. Direct University of Toronto, Toronto, of hand hygiene auditors was associated with an observation, in which human auditors Ontario, Canada 5 increase in hand hygiene events as measured by monitor the hand hygiene compliance of Department of Infection Prevention & Control, University a real-time location system (RTLS). HCWs as they carry out clinical tasks, is Health Network, Toronto, Methods The RTLS recorded all uses of alcohol- the most commonly used method of Ontario, Canada based hand rub and soap for 8 months in two measurement. However, this method is 6Department of Medicine, University of Toronto, Toronto, units in an academic acute care hospital. The RTLS subject to biases, including observer bias, Ontario, Canada also tracked the movement of hospital hand selection bias and possibly the http://qualitysafety.bmj.com/ hygiene auditors. Rates of hand hygiene events Hawthorne effect.34If the resulting data Correspondence to per dispenser per hour as measured by the RTLS are publicly reported, as is the case in Dr Jocelyn A Srigley, Department of Medicine, McMaster were compared for dispensers within sight of many jurisdictions around the world, the 5 University, 711 Concession auditors and those not exposed to auditors. potential for bias may be even greater. Street, M1-Room 8, Hamilton, Results The hand hygiene event rate in dispensers The Hawthorne effect, also called ON, Canada, L8V 1C3; visible to auditors (3.75/dispenser/h) was observation bias, refers to the tendency [email protected] significantly higher than in dispensers not visible to of people to change their behaviour Received 31 March 2014 the auditors at the same time (1.48; p=0.001) and when they are aware of an observer.34It Revised 20 June 2014 in the same dispensers during the week prior (1.07; is widely assumed that the Hawthorne on September 28, 2021 by guest. Protected copyright. Accepted 21 June 2014 p<0.001). The rate increased significantly when effect transiently increases HCW hand auditors were present compared with 1–5minprior hygiene compliance rates when auditors to the auditors’ arrival (1.50; p=0.009). There were are present, despite the fact that the exist- no significant changes inside patient rooms. ence of the Hawthorne effect has been Conclusions Hand hygiene event rates were extensively debated in the psychological approximately threefold higher in hallways within literature6 and studies that examine the eyesight of an auditor compared with when no Hawthorne effect specifically in hand auditor was visible and the increase occurred after hygiene behaviour have significant meth- the auditors’ arrival. This is consistent with the odological limitations. Much of the evi- existence of a Hawthorne effect localised to areas dence in hospital hand hygiene where the auditor is visible and calls into question compliance monitoring comes from To cite: Srigley JA, the accuracy of publicly reported hospital hand studies demonstrating that compliance Furness CD, Baker GR, et al. hygiene compliance rates. rates are higher when audits are BMJ Qual Saf Published please include announced in advance or the auditors are Online First: [ 7–11 Day Month Year] BACKGROUND known to the units. However, the doi:10.1136/bmjqs-2014- Healthcare worker (HCW) hand hygiene potential for observer and selection bias 003080 is an important strategy to prevent is high, particularly since it was not Srigley JA, et al. BMJ Qual Saf 2014;0:1–7. doi:10.1136/bmjqs-2014-003080 1 Copyright Article author (or their employer) 2014. Produced by BMJ Publishing Group Ltd under licence. Original research BMJ Qual Saf: first published as 10.1136/bmjqs-2014-003080 on 7 July 2014. Downloaded from stated in any of these studies whether the observers obtained for the overall study, and administrative REB were blinded to the study hypothesis. approval was obtained from the University of Toronto The recent development of electronic systems for specifically for this component of the study on the hand hygiene monitoring offers an alternative Hawthorne effect. approach to direct observation.12 These systems monitor HCW hand hygiene on a constant, real-time Measures basis, making them a promising tool for determining The RTLS measured hand hygiene events by counting whether the Hawthorne effect exists and characteris- all times that ABHR and soap dispensers were used. ing the nature of the effect. Furthermore, electronic Tags installed on the dispensers transmitted a signal to systems apply consistent algorithms to measure hand a nearby receiver each time the levers were pushed, hygiene events and compliance rates, eliminating and a time-stamped hand hygiene event was recorded many of the biases inherent to direct observation. The in a centralised database. If multiple dispenses objective of this retrospective cohort study was to use occurred at a single dispenser within 4 s of each other, a real-time location system (RTLS) to determine only one dispense event was counted. Prior to the whether the presence of human auditors on inpatient start of this study, the RTLS was tested in a simulated units was associated with an increase in hand hygiene clinical environment and 100% of dispenses were events. detected. The hand hygiene event rate was defined as the total number of ABHR and soap dispenses mea- METHODS sured by the RTLS per dispenser per hour. Setting and participants Auditors also measured directly observed hand An RTLS was installed on two multi-organ transplant hygiene compliance rates on the study units as part of units at an academic acute care hospital as part of a the overall hospital hand hygiene programme. larger research study and was operational between 18 Auditors wore white lab coats as per usual hospital July 2012 and 11 March 2013. The RTLS used small practice and were not specifically identified as audi- battery-operated tags that could be worn by people or tors but may have been recognisable to some HCWs. attached to equipment. Tags emitted ultrasound Auditors were blinded to the study hypothesis and signals at regular intervals, which were picked up by a conducted audits in accordance with the Ontario Just network of 618 wireless receivers situated in patient Clean Your Hands programme,13 once or twice rooms, hallways, and above all 257 alcohol-based monthly on each unit from 29 November 2012 to 11 hand rub (ABHR) and 148 soap dispensers. Signals March 2013. No more than two audits were con- from the receivers were processed by a geographical ducted per unit per month so as to prevent HCWs information systems engine, which computed move- from noticing an increased frequency of auditing and http://qualitysafety.bmj.com/ ment, location and proximity of tags to each other. changing their behaviour as a result. The duration of All HCWs on the study units, including physicians, each audit was variable due to the requirement for nurses and allied health professionals, were invited to auditors to remain on the unit until they had observed participate in the overall study of the RTLS through 60 hand hygiene opportunities. Auditors were presentations by study personnel. Posters describing instructed not to enter patient rooms in order to the RTLS and the study were also displayed on the maintain privacy. participating units. HCWs who volunteered to wear tags contacted the research assistant to provide Study design written informed consent. There were 60 tags avail- The Hawthorne effect was conceptualised as an on September 28, 2021 by guest. Protected copyright. able for HCWs, which represented approximately a exposure–outcome relationship, in which the expos- quarter of the staff who routinely worked on these ure was the presence of a hand hygiene auditor and units. The tags were a component of the overall study the outcome was hand hygiene events. This study but did not contribute data to the current study. The used a retrospective cohort design, defining the objective of the overall study was to determine the exposed cohort as the ABHR and soap dispensers on effect of the RTLS in combination with a behaviour the two multi-organ transplant units that were within change approach called ‘positive deviance’ on HAIs. eyesight of an auditor. Data from the RTLS, including movement of equip- The auditors who rotated through the study units ment and trends in hand hygiene (but not specific wore staff RTLS tags to track the exact time and loca- compliance rates as measured by the RTLS), were pre- tion of auditing. After audits, a list of receivers that sented to front line HCWs, who were then empow- had detected the tag and the time period during ered to make changes on the units to address the which the tag was within that zone was obtained.