Accepted Manuscript

Diffusion of Innovations model helps interpret the comparative uptake of two methodological innovations: co-authorship network analysis and recommendations for the integration of novel methods in practice

Suzanne M. Cadarette, Joann K. Ban, Giulia P. Consiglio, Cody D. Black, Mina Tadrous, Alexandra Marin, David Dubins PII: S0895-4356(16)30835-6 DOI: 10.1016/j.jclinepi.2016.12.006 Reference: JCE 9292

To appear in: Journal of Clinical Epidemiology

Received Date: 1 April 2016 Revised Date: 29 September 2016 Accepted Date: 1 December 2016

Please cite this article as: Cadarette SM, Ban JK, Consiglio GP, Black CD, Tadrous M, Marin A, Dubins D, Diffusion of Innovations model helps interpret the comparative uptake of two methodological innovations: co-authorship network analysis and recommendations for the integration of novel methods in practice, Journal of Clinical Epidemiology (2017), doi: 10.1016/j.jclinepi.2016.12.006.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. ACCEPTED MANUSCRIPT 1

Diffusion of Innovations model helps interpret the comparative uptake of two

methodological innovations: co-authorship network analysis and recommendations for the

integration of novel methods in practice

Suzanne M. Cadarette 1* , Joann K. Ban 1, Giulia P. Consiglio 1, Cody D. Black 1, Mina Tadrous 1,

Alexandra Marin 2, David Dubins 1

1 Leslie Dan Faculty of Pharmacy, University of Toronto

2 Department of Sociology, University of Toronto

*Correspondence to: Suzanne M. Cadarette, Leslie L. Dan Pharmacy Building, University of

Toronto, 144 College Street, Toronto, Ontario, M5S 3M2 Canada. Tel: 416-978-2993, Fax: 416-

978-8511, E-mail: [email protected], @CadaretteSM #RxEpi MANUSCRIPT Conflict of Interest:

Suzanne M. Cadarette – none

Joann K. Ban – none

Cody D. Black – none

Giulia P. Consiglio – none

Mina Tadrous – none

Alexandra Marin – none

David Dubins – noneACCEPTED

ACCEPTED MANUSCRIPT 2

Financial support:

Project support: No specific funding was received to support this work. This research was

supported in part by unrestricted funds from an Ontario Ministry of Research

and Innovation Award (ER09-06-043) to Dr. Cadarette.

Cody D. Black: Life Sciences Committee Undergraduate Research Opportunity Program

Award, University of Toronto.

Mina Tadrous: Frederick Banting and Charles Best Canada Graduate Scholarship (GSD-

11342), Canadian Institutes of Health Research

Main Text: 3109 (max: 3000-5000)

MANUSCRIPT

ACCEPTED

ACCEPTED MANUSCRIPT 3

ABSTRACT

Objective: To characterize the diffusion of methodological innovation.

Study Design and Setting: Comparative case study analysis of the diffusion of two methods that summarize confounder information into a single score: disease risk score (DRS), and high- dimensional propensity score (hdPS). We completed systematic searches to identify DRS and hdPS papers in the field of pharmacoepidemiology through to the end of 2013, plotted the number of papers and unique authors over time, and created sociograms and animations to visualize co-authorship networks. First and last author affiliations were used to ascribe institutional contributions to each paper and network.

Results: We identified 43 DRS papers by 153 authors since 1981, reflecting slow uptake during initial periods of uncertainty and broader diffusion since 2001 linked to early adopters from

Vanderbilt. We identified 44 hdPS papers by 147 authors since 2009, reflecting rapid and integrated diffusion, likely facilitated by opinion leaders, MANUSCRIPT early presentation at conferences, easily accessible statistical code, and improvement in funding. Most contributions (87% DRS, 96% hdPS) were from North America.

Conclusion: When proposing new methods, authors are encouraged to consider innovation attributes and early evaluation to improve knowledge translation of their innovations for integration into practice, and we provide recommendations for consideration.

Word count: 200 (max: 200)

Key Words: Bibliometrics, Diffusion of Innovation, Disease Risk Score,

Pharmacoepidemiology,ACCEPTED Social Networks, Propensity Score, Methodological Innovation

Running Title: Diffusion of Methodological Innovation and Recommendations

ACCEPTED MANUSCRIPT 4

Box 1. What is new? • When proposing new methods, authors are encouraged to consider innovation

attributes and early evaluation to improve knowledge translation of their

innovations for integration into practice, and we provide recommendations for

consideration.

• Co-authorship network analysis can be used to examine the diffusion of

methodological innovation by visualizing the prominence of, and connections

between authors that publish using novel methods.

• We propose methods to ascribe institutional credit to publications, and encourage

researchers to consider and comment on our approach.

• Web of Science citation and possibly author searchers are important to help find the

application of innovative methods, as keyword searches are limited.

MANUSCRIPT

ACCEPTED

ACCEPTED MANUSCRIPT 5

INTRODUCTION

The field of post-marketing drug safety and effectiveness research (pharmacoepidemiology) has experienced rapid scientific progress and growth [1, 2], particularly in the last decade [3-6]. The rapid increase may partly relate to the emerging availability of healthcare utilization data [7, 8], and significant funding investment [9, 10]. The recent investment in pharmacoepidemiology is motivated by the recognition that drug safety and efficacy data from randomized controlled trials are limited [2], and thus more evidence is needed post-marketing to improve our understanding of drug benefits and harms [7]. Real-world drug safety and effectiveness data are important for patient and physician prescribing decisions, as well as for drug policy decision making.

Methodological challenges in pharmacoepidemiology have required innovative solutions. Prior research has identified slow knowledge translation of statistical innovations [11, 12]. We identified two statistical innovations that summarize confounder information into a single score: disease risk score (DRS, [13]) and high dimensional MANUSCRIPT propensity score (hdPS, [14]) to serve as comparative case studies in the diffusion of methodological innovation. Our aim was to examine the speed (number of publications over time) and spread (across institutions) of each innovation, and interpret uptake relative to innovation attributes, the social system, and communication channels described in Rogers’ Diffusion of Innovations model [15].

METHODS

We apply comparative case study methods with the Diffusion of Innovations model [15-18]. In brief, the DiffusionACCEPTED of Innovations model defines diffusion as a process by which an innovation

(something perceived as new) is communicated through channels (how messages are passed between individuals) over time among members of a social system, Box 2 . In particular, the rate

ACCEPTED MANUSCRIPT 6 of adoption of an innovation is proposed to be affected by five innovation attributes: 1) relative advantage over existing ideas or methods, 2) compatibility with the needs and values of potential adopters, 3) complexity (hereafter referred to as simplicity), 4) trialability (degree it can be tested), and 5) observability (degree its use and results are visible to others) [15].

We selected two methodological innovations in pharmacoepidemiology that cover a range of innovation attributes and time frame within a social system, according to the Diffusion of Innovations model, Table 1 .

Case Study 1: Disease Risk Score (DRS)

Stratification or matching by confounding variables was a common approach to control for confounding in the 1970s. However, stratification become inefficient as the number of strata or confounding variables to control for increases. The disease risk score (DRS), proposed by Miettinen in 1976 (‘multivariate confounder score’) MANUSCRIPT [13], summarizes all confounder information into a single summary score. Authors can then use DRS for stratification and thus reduce the number of strata. The innovation addressed an important limitation at the time and had a clear advantage over traditional stratification by individual confounding variables (relative advantage).

Because the DRS is based on the baseline probability of disease risk, it can also be used to provide a meaningful scale to examine effect modification [19-21]. Despite its advantages, a recent systematic review (from 1976 to May 2011) identified that DRS initially received little attention or application in the epidemiologic literature [6]. DRS application was characterized by a bimodal distributionACCEPTED with a peak in 1979/1980 and resurgence since 2000 [6]. DRS was first proposed in 1976 [13], yet a simulation paper published in 1979 introduced early uncertainty in the method by concluding that it overestimates confounding and thus induces bias [22]. A

ACCEPTED MANUSCRIPT 7 subsequent simulation published in 1989 concluded that overestimation of confounders was rare

[23], and more recent contributions corroborates DRS ability to control for confounding and highlight its potential advantages [19-21, 24, 25]. This case study thus provides an opportunity to consider the diffusion of an innovation introduced during the infancy of the field of pharmacoepidemiology and over a 40-year time span in the context of initial uncertainty.

Case Study 2: high-dimensional Propensity Score (hdPS)

Studies that rely on healthcare utilization (administrative claims) databases may be biased if important confounding information is missing. In theory, statistical adjustment for proxy variables or combinations of variables that indirectly capture information on unmeasured confounder(s) may yield better control for confounding. The hdPS is an adaptation of the commonly used propensity score [5], and uses a multistep algorithm to empirically identify candidate proxy variables based on their estimated MANUSCRIPTstrength of confounding. The proxy variables are then included into the hdPS [14]. The innovation paper included simple figures to help contextualize the theory around proxy variables (simplicity), compared statistical adjustment using a standard confounder model to that using the hdPS (compatibility), documented results closer to those from clinical trials when using the hdPS (advantage), and authors posted statistical code on their research website to facilitate application of the innovation by other researchers (trialability). In addition, preliminary results were presented at the International

Society for Pharmacoepidemiology meeting (observability and active communication channel), the first authorACCEPTED (Schneeweiss) served as the president of the International Society for

Pharmacoepidemiology in 2010, and several co-authors are eminent researchers in the field of pharmacoepidemiology (important communication sources as key opinion leaders). This case

ACCEPTED MANUSCRIPT 8 study thus provides an opportunity to consider the diffusion of an innovation by a number of opinion leaders, presented with several critical innovation attributes, in a short time period and during a period of growth in the field of pharmacoepidemiology.

Systematic Literature Search

We updated a recent systematic review of DRS applications [6], to identify English language papers in the field of pharmacoepidemiology through to December 2013. The original systematic review identified 97 papers (24 in the field of pharmacoepidemiology) using three search strategies: 1) MEDLINE® keyword search (terms listed in Appendix A), 2) Web of Science® citation search [13, 22-24], and 3) Web of Science® author search (PG Arbogast and WA Ray)

[6]. In our update, we restricted the original search strategy to January 2011 through to the end of

December 2013. We then performed an EMBASE® keyword search from 1976 to December 2013, and citation searches of more recent DRS meth MANUSCRIPTods papers [19-21]. A new systematic search was completed to identify all English language articles on the hdPS in pharmacoepidemiology from the time of innovation (July 2009) to the end of December

2013 using: 1) MEDLINE® and EMBASE® keyword searches (terms listed in Appendix A), and 2) Web of Science® citation search of the innovation paper [14].

Pharmacoepidemiologic empirical, method and review papers were eligible. Post-hoc analyses of clinical trial data were excluded to be consistent with our prior review [6]. Electronic searches were facilitated using EndNote X5 (Thomson Reuters, 2011). We created study flow diagrams to summarizeACCEPTED results of each systematic search and Venn diagrams to illustrate search strategy yield, with circle size proportional to the number of eligible articles identified by each search strategy. We plotted the number of publications by study type (empirical, methodological,

ACCEPTED MANUSCRIPT 9 review), and the cumulative number of unique authors by calendar year to characterize the rate of adoption of each innovation over time. As a crude way to get a sense of the general trend in pharmacoepidemiology publications, we completed a systematic literature search using

PubMed® only, as well as a combination of PubMed®, EMBASE®, and MEDLINE®, from their dates of inception to 2013 using the keyword “pharmacoepidemiology.” We limited our search to English language publications on humans from 1902 to 2013, and excluded duplicate publications, commentaries, editorials, letters, and conference abstracts.

Co-authorship Network Analysis

Author names and order were downloaded from EndNote X5 into Excel 2010. Names of authors presented inconsistently (e.g., Fireman BH and Fireman B) were collapsed into the most common presentation or in the event of a tie, the one with more initials. We generated sociograms in Cytoscape, version 3.2.0 (Harvard AnaMANUSCRIPTlytic Technologies, 2002) [26] to examine the co-authorship network structure of each innovation. Sociograms depict authors as “nodes,” with lines or “ties” between nodes denoting co-authorship. Directed sociograms were generated to clarify network structure by only sending ties as arrows from first authors to each co-author.

Node size was created proportional to the number of articles published by that author. The number of components was then identified. A component is a group of authors connected directly (co-author the same paper), or indirectly (connected through a mutual co-author on separate papers).

ACCEPTED Institutional Credit

Institutional affiliations of the first and last author were used to ascribe institutional credit to each paper, component and network. Institutional credit was given proportional to the number of

ACCEPTED MANUSCRIPT 10 institutional affiliations between the first and last author. Institutional departments and divisions were collapsed into the main institution. For example, affiliation with Harvard Medical School and Harvard School of Public Heath were ascribed to Harvard University. Each publication was given equal weight when determining contribution to the network. Affiliations of the first, second, and last author were considered in a secondary analysis.

RESULTS

We identified 43 DRS (Appendix B) and 44 hdPS eligible articles (Appendix C), Figure 1 . The citation search strategy identified the most papers, including 30 (75%) of the 40 DRS papers identified through electronic searches, and all but the hdPS seminal paper used as the citation in the search, Figure 2. Three DRS papers were identified outside the electronic search, as documented previously [6]. Web of Science citation (DRS and hdPS) and author (DRS) searches proved critical to the identification of papers; indeed MANUSCRIPT keyword searches alone would have missed half (n=22 of 44) of the hdPS papers, and three-quarters (n=30 of 40 identified in the electronic search) of DRS papers.

DRS uptake was initially slow, with few publications in the field of pharmacoepidemiology before 2001, Figure 3a . The number of unique authors increased from

23 in 2001 to a total of 153 by the end of 2013; with 34 (79%) of the 43 papers published between 2011/01 and 2013/12. The network comprised 13 components (Figure 4a), and as illustrated in the animated sociogram ( Figure 5a , available in the online supplemental material ), diffusionACCEPTED did not extend from the seminal paper. Instead, Vanderbilt University led

DRS diffusion starting in 2001, with initial spread facilitated by collaborations with WA Ray and

PG Arbogast from Vanderbilt. Institutional credit is summarized in Table D1, Appendix D . This

ACCEPTED MANUSCRIPT 11 largest component included 21 papers (49%) and 67 authors (44%) with primary institutions

Vanderbilt University (48%), Veteran Affairs - Tennessee Valley Healthcare System (29%) and

University of Alabama at Birmingham (10%) (Appendix B1-B21 ). The second largest component included six papers (13%), and 16 authors (11%) from Harvard University (39%),

Brigham and Women’s Hospital (39%), University of Toronto (19%) and University of North

Carolina (3%) (Appendix B22-B27). The third largest component included four papers (9%)

(Appendix B28-B31 ), and 12 authors (8%) with 100% of contributions from institutions in

Toronto, Canada. The remaining ten components included only one or two articles each

(Appendix B32-B43 ).

hdPS uptake was characterized by a steep slope reflecting rapid diffusion from publication in July 2009 to December 2013, Figure 3b. The steep hdPS slope is consistent with the dramatic general increase in pharmacoepidemiology publications since 2011, Appendix E. The 44 hdPS papers by 147 authors comprised seven MANUSCRIPT components (Figure 4b ), and as illustrated in the animation (Figure 5b, available in the online supplemental material ), innovators led its diffusion. Institutional credit is summarized in Table D2, Appendix D . The largest components included 34 papers (77%) and 96 authors (65%) with primary institutions Harvard University

(36%), Brigham and Women’s Hospital (34%), and University of British Columbia (7%)

(Appendix C1-C34 ). The second largest component contained five papers (11%), and 16 authors

(11%), with primary institutions National Institutes of Health (28%) and Johnson and Johnson

(19%) (Appendix C35-C39 ). The remaining five components included one paper each

(Appendix C40-C44ACCEPTED), with contributions largely from the US.

Overall, 87% of DRS contributions were from North America (14% Canada, 73% US;

24% Vanderbilt University), with some credit attributed to France, Greece, Netherlands, Spain,

ACCEPTED MANUSCRIPT 12

Taiwan and the UK; and 96% of hdPS contributions were from North America (17% Canada,

79% US; 29% Harvard University, 27% Brigham and Women’s Hospital) with some credit attributed to China, Germany, Saudi Arabia and the UK. Almost 90% of publications using each innovation were attributed to academic institutions (86% DRS, 90% hdPS), with some contribution by industry (10% DRS, 6% hdPS) and least by government (4% DRS, 5% hdPS),

Table 2. Including the second author in addition to the first and last author in the secondary analysis had little impact on institutional credit assigned to each innovation.

DISCUSSION

Our systematic search identified a similar number of DRS and hdPS articles published in pharmacoepidemiology through to the end of 2013, yet the rate of diffusion of each method differed considerably. The stark contrast in the rate of adoption of these two methodological innovations may be explained by concepts in theMANUSCRIPT Diffusion of Innovations model [15]. DRS adoption was slow. The widely accepted seminal paper was published in the broad field of epidemiology in 1976 [13], yet early controversy over its merits (advantage) stifled its adoption, and inconsistent use of language made it less observable [6]. Indeed, as indicated in our prior review, our systematic search may have missed some DRS applications. In fact, even the 1976 seminal methodological article by Miettinen uses an earlier application that considers coffee drinking and myocardial infarction [27], to explain and review the relative advantage of the DRS method. Another early application outside the field of pharmacoepidemiology [28] citing an even earlier source [29],ACCEPTED was brought to our attention after presentation of our work in August 2016 [30]. We thus acknowledge the challenge when studying methodological innovation to pin point the original publication. Nonetheless, we anticipate that few if any applications were in the field

ACCEPTED MANUSCRIPT 13 of pharmacoepidemiology prior to what we have identified as the methodological DRS contribution [13].

The DRS co-authorship network started to expand in 2001 with a publication by authors from Vanderbilt University [31]. Researchers from this group were “early adopters” of the DRS in pharmacoepidemiology and early DRS use then spread to other institutions by collaboration with authors from Vanderbilt University. The relatively recent uptake in usage of the DRS in pharmacoepidemiology may also be partially explained by the change in the social system, with significant investment in funding opportunities not only to complete drug safety and effectiveness studies, yet also to examine methods to improve the validity of pharmacoepidemiologic applications. Indeed, several recent papers underscore the comparative advantages of DRS over existing methods to adjust for confounding variables [19-21, 24, 25].

General increase in use of the propensity score method may have also improved perception of the benefits of confounder summary score methods MANUSCRIPT [5], and thus indirectly improved perceived compatibility of DRS in pharmacoepidemiology.

In contrast, hdPS experienced immediate and rapid diffusion over a short four and a half year period. Much of the strength of diffusion may relate to innovation attributes, how the method was presented, authors and timing. Indeed, all five key innovation attributes were maximized: 1) evidence provided of distinct advantages over the existing methods (advantage),

2) adaptation of the well-established propensity score method (compatibility), 3) presentation in preliminary form at conferences (observability), 4) paper included figures to simplify concepts with step-by-stepACCEPTED approach to execute the method (simplicity), and 5) free statistical code was posted on a website (trialability). In addition, authors included several key opinion leaders (well known and respected communication sources), and hdPS was launched during a period of

ACCEPTED MANUSCRIPT 14 government investment and incentives for inter-institutional research collaboration (congruent social system). We must also acknowledge a general increase in the publication of pharmacoepidemiology papers, reflecting social system advances.

In contrast to DRS, where the innovator Miettinen was not readily engaged in its application, Schneeweiss and co-authors helped to push hdPS adoption through collaboration.

For example, collaboration with hdPS seminal authors helped to bring the method to the

University of North Carolina (i.e., hdPS seminal author Brookhart moved institutions), and

Canada (e.g., Dormuth trained with hdPS seminal author Schneeweiss). Our results are also consistent with a recent survey of researchers and statisticians that identified lack of: awareness of the new method or its relative benefits over existing methods, expertise, software and time as key barriers to the implementation of new statistical methods [11]. Even among those with expertise to implement new statistical methods, lack of time to learn and implement new methods, as well as concerns about translating non-MANUSCRIPTstandard methods, were identified as key barriers [11]. Providing statistical code for implementing new methods clearly facilitates the adoption of methodological innovation.

Our review is complete through to the end of 2013, providing a comparative case analysis of the diffusion of DRS and hdPS. For the purposes of considering the relative uptake of these innovations in the field of pharmacoepidemiology, this timeline is sufficient by providing comparative evidence of the benefits of the Diffusion of Innovations model in launching statistical innovations. Nonetheless, it is important to consider the lifecycle of these innovations.

It is possible thatACCEPTED new information about the relative advantage of these methods, or the introduction of newer methods may impact future applications.

ACCEPTED MANUSCRIPT 15

In our primary analysis, we ascribed institutional credit to the first and last author.

Traditionally, the first ( principal ) author contributes most and receives the most credit. However, in pharmacoepidemiology as in biomedical sciences, the last (often senior ) author often gets as much, or even more credit than the first author. This is because the senior author is assumed to be the driving force, both intellectually and financially, behind the research [32]. However, this may not always be the case and indeed some other authors may drive use of statistical methods.

The last author may also not always be the senior author, as is the case in our paper; yet authors often strategically consider authorship order based on contributions, seniority, or relative benefit to career stage. We note little change in institutional credit when ascribing credit to first and last only versus first, second and last author. To our knowledge, we are the first to attempt to ascribe institutional credit to a methodological innovation, and future research in the area may help to develop methodological standards. Based on our findings and the Diffusion MANUSCRIPT of Innovations model, we summarize recommendations for authors of methodological innovations to consider ( Table 3 ). In brief, we encourage authors to remain mindful of the five innovation attributes, and propose five principles that collectively tap into the five innovation attributes: 1) clearly describe using foundational principles; 2) consider comparing results to established methods; 3) provide sample data, code or calculation examples and instructions; 4) early communication, support and testing; and 5) provide methodological and reporting guidance. We encourage authors to consider and comment on our recommendations to facilitate the knowledge translation of innovations for rapid integration into practice.ACCEPTED A key part of the integration of novel methods includes evaluation. It is through application and testing that the relative merits and shortcoming of novel methods are identified.

ACCEPTED MANUSCRIPT 16

ACKNOWLEDGEMENTS

The authors declare no conflict of interest.

No specific funding was received to support this work. This research was supported in part by unrestricted funds from an Ontario Ministry of Research and Innovation Award (ER09-06-

043) to Dr. Cadarette. Mina Tadrous was supported in part by a CIHR Doctoral Award -

Frederick Banting and Charles Best Canada Graduate Scholarship (GSD-11342), and Cody D.

Black was supported in part by a Life Sciences Committee Undergraduate Research Opportunity

Program Award. Authors thank Patrick R. Edwards, PharmD for initial work with the disease risk score co-authorship analysis and use of the Diffusion of Innovations model; Gina Matesic,

MLIS for help with search strategies; and Alison Thompson, PhD for an introduction to comparative case study analysis.

Dr. Cadarette acknowledges contributions through conversations with researchers (in alphabetical order) from McGill University (Michal MANUSCRIPT Abrahamowicz, Olli Miettinen, Robert Platt), University of British Columbia (Colin Dormuth, Malcolm Maclure) and University of

North Carolina (M. Alan Brookhart, Til Stürmer) – thanks for getting this research out of “death valley,” i.e., the floor of my office.

ACCEPTED

ACCEPTED MANUSCRIPT 17

REFERENCES

1. Hartzema AG, Tilson HH, Chan KA. Pharmacoepidemiology and Therapeutic Risk Management. Cincinnati: Harvey Whitney Books; 2008 2. Strom BL, Kimmel SE, Hennessy S. Pharmacoepidemiology. West Sussex: Wiley- Blackwell; 2011 3. Consiglio GP, Burden AM, Maclure M, et al. Case-crossover study design in pharmacoepidemiology: systematic review and recommendations. Pharmacoepidemiol Drug Saf 2013;22:1146-1153 4. Jandoc R, Burden AM, Mamdani M, et al. Interrupted time series analysis in drug utilization research is increasing: systematic review and recommendations. J Clin Epidemiol 2015;68:950-956 5. Stürmer T, Joshi M, Glynn RJ, et al. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol 2006;59:437-447 6. Tadrous M, Gagne JJ, Stürmer T, et al. Disease risk score as a confounder summary method: systematic review and recommendations. Pharmacoepidemiol Drug Saf 2013;22:122-129 7. Schneeweiss S. Developments in post-marketing comparative effectiveness research. Clin Pharmacol Ther 2007;82:143-156 8. Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol 2005;58:323-337 9. Suissa S, Henry D, Caetano P, et al. CNODES: the Canadian Network for Observational Drug Effect Studies. Open Med 2012;6:e134-140 10. Selby JV, Beal AC, Frank L. The patient-centereMANUSCRIPTd outcomes research institute (PCORI) national priorities for research and initial resear ch agenda. JAMA 2012;307:1583-1584 11. Pullenayegum EM, Platt RW, Barwick M, et al. Knowledge translation in biostatistics: a survey of current practices, preferences, and barriers to the dissemination and uptake of new statistical methods. Stat Med 2016;35:805-818 12. Altman DG, Goodman SN. Transfer of technology from statistical journals to the biomedical literature. Past trends and future predictions. JAMA 1994;272:129-132 13. Miettinen OS. Stratification by a multivariate confounder score. Am J Epidemiol 1976;104:609-620 14. Schneeweiss S, Rassen JA, Glynn RJ, et al. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology 2009;20:512-522 15. Rogers EM. Diffusion of Innovations. Toronto: Free Press; 2003 16. Stake RE. Qualitative Case Studies. SAGE Publications; 2005 17. Yin RK. Applications of Case Study Research. SAGE Publications; 2003 18. Haider M, Kreps GL. Forty years of diffusion of innovations: utility and value in public health. J Health Commun 2004;9 (Suppl 1):3-11 19. Cadarette SM,ACCEPTED Gagne JJ, Solomon DH, et al. Confounder summary scores when comparing the effects of multiple drug exposures. Pharmacoepidemiol Drug Saf 2010;19:2-9 20. Arbogast PG, Ray WA. Performance of disease risk scores, propensity scores, and traditional multivariable outcome regression in the presence of multiple confounders. Am J Epidemiol 2011;174:613-620

ACCEPTED MANUSCRIPT 18

21. Glynn RJ, Gagne JJ, Schneeweiss S. Role of disease risk scores in comparative effectiveness research with emerging therapies. Pharmacoepidemiol Drug Saf 2012;21:138-147 22. Pike MC, Anderson J, Day N. Some insights into Miettinen's multivariate confounder score approach to case-control study analysis. Epidemiol Community Health 1979;33:104-106 23. Cook EF, Goldman L. Performance of tests of significance based on stratification by a multivariate confounder score or by a propensity score. J Clin Epidemiol 1989;42:317-324 24. Stürmer T, Schneeweiss S, Brookhart MA, et al. Analytic strategies to adjust confounding using exposure propensity scores and disease risk scores: nonsteroidal antiinflammatory drugs and short-term mortality in the elderly. Am J Epidemiol 2005;161:891-898 25. Wyss R, Lunt M, Brookhart MA, et al. Reducing bias amplification in the presence of unmeasured confounding through out-of-sample estimation strategies for the disease risk score. J Causal Inference 2014;2:131-146 26. Arbogast PG, Kaltenbach L, Ding H, et al. Adjustment for multiple cardiovascular risk factors using a summary risk score. Epidemiology 2008;19:30-37 27. Saito R, Smoot ME, Ono K, et al. A travel guide to Cytoscape plugins. Nat Methods 2012;9:1069-1076 28. Jick H, Miettinen OS, Neff RK, et al. Coffee and myocardial infarction. N Engl J Med 1973;289:63-67 29. Rothman KJ, Monson RR. Epidemiology of trigeminal neuralgia. J Chronic Dis 1973;26:3- 12 30. Miettinen O, Neff, RK. Computer processing of epidemiologic data. Hart Bulletin 2 1971:98–103 31. Cadarette SM, Platt R, Ban JK, et al. Catch the wave: diffusion of methodological innovation in pharmacoepidemiology [abstract]. International Society for Pharmacoepidemiology and Therapeutic Risk ManagemenMANUSCRIPTt. Dublin, Ireland; 2016 32. Ray WA, Meredith S, Thapa PB, et al. Antipsycho tics and the risk of sudden cardiac death. Arch Gen Psychiatry 2001;58:1161-1167 33. Tscharntke T, Hochberg ME, Rand TA, et al. Author sequence and credit for contributions in multiauthored publications. PLoS Biol 2007;5:e18

ACCEPTED

ACCEPTED MANUSCRIPT 19

Box 2. Summary of Rogers’ Diffusion of Innovations Model [15]: The adoption of methodological innovation in pharmacoepidemiology can be described using Rogers’ Diffusion of Innovations model [15]. Over 5000 studies in a variety of disciplines have utilized the Diffusion of Innovations model since it was first published in 1962 [18]. The model includes four key elements that are proposed to impact the uptake of an innovation, 1. Innovation Attributes, 2. Communication Channels, 3. Time, and 4. Social System: i. Innovation Attributes : An innovation is an idea, practice, or object that is perceived as new. Rogers identifies five key innovation attributes that affect its rate of uptake: Attribute Description 1. Relative advantage Perceived relative advantage over existing ideas or methods 2. Compatibility Perceived consistency with existing needs and values of potential adopters 3. Simplicity* Degree perceived to be simple to understand or use 4. Trialability Degree can be tested (testing reduces uncertainty) 5. Observability Degree its use and results are visible to others *Rogers’ model labels this attribute complexity (degree perceived to be difficult to understand or use), yet we have used simplicity so that the label corresponds with a quicker rate of adoption ii. Communication Channels : How messages are passed between individuals. Communication channels have two important components; its source and the method of communication: 1. Source: An individual or an institution that originatesMANUSCRIPT a message The more similar individuals who interact are (e.g., education, beliefs, institution, region), the more likely the source will be effective. In addition, the more well known/respected (opinion leaders) the source, the more likely the recommendation will be considered for adoption. 2. Channel: Means by which a message gets from the source to the receiver The more interpersonal (vs. mass media) and active (vs. passive), the quicker it will diffuse. • Interpersonal vs. mass media: direct communication between individuals is advantageous over mass media (e.g., one or few to reach many) • Active vs. passive: degree more actively targeted to individual (e.g., small seminar or interpersonal email) will diffuse more quickly vs. passive communication (e.g., publication) iii. Time : Adopters of an innovation are classified into one of five categories: innovators, early adopters, early majority, late majority and laggards. Innovators are those that develop methodological innovations as well as the first members of a group to adopt the technology. Early adopters typically interact with the innovators, with early majority attending conferences and late majority and laggards learning after the method is well established. Early adopters’ leadership in adopting a new technology serves to reduce the uncertainty about the ACCEPTEDinnovation by other researchers. iv. Social System : A social system is a group of individuals who work together towards a common goal, e.g., researchers, decision makers and funding agencies. Units in the social system may be comprised of individuals, groups or organizations.

ACCEPTED MANUSCRIPT

20

B A

MANUSCRIPT

Figure 1. Flow diagram of systematic search results. A. Disease risk score (DRS), B. High-dimensional propensity score (hdPS). *3 eligible articles identified in conversation with local research group outside the review [6].

ACCEPTED

ACCEPTED MANUSCRIPT 21

A

MANUSCRIPT

B

Figure 2. Venn diagram of systematic search results depicting unique and overlap papers identified by each search engine. Circle size is proportional to the number of eligible papers identified by each search strategy. A. Disease risk score electronic search yielded 40 papers: Web of Science® citation (n=30), Web of Science® author (n=17), MEDLINE® keyword (n=10), EMBASE® keyword (n=8). An additional 3 articles were identified outside the electronic search in our original review [6]. B. High-dimensionalACCEPTED propensity score search yielded 44 papers: Web of Science® citation (n=43), MEDLINE® keyword (n=18), and EMBASE® keyword (n=22).

ACCEPTED MANUSCRIPT 22

18 160

16 140 14 120 12 100 10 80 8 60 6 40 4

2 20 Number of Publications, DRS N=43

0 0 Cumulative Number of Authors, N=153 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005 2007 2009 2011 2013 Publication Year

A 18 160

16 140

14 120 12 MANUSCRIPT100 10 80 8 60 6 40 4

Number of hdPS Publications, N=44 20 2 Cumulative Number ofAuthors, N=147

0 0 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005 2007 2009 2011 2013 Publication Year

B

Figure 3. NumberACCEPTED of publications and cumulative authors by year of publication. Cumulative number of authors represented by solid black line. Empirical application (solid), methodological contributions (striped) and review papers (checkered). A. Disease risk score, 153 authors publishing 43 (35 empirical application, 6 methodological contributions and 2 review) papers. B. High-dimensional propensity score, 147 authors publishing 44 (33 empirical applications, 10 methodological contributions and 1 review) papers.

ACCEPTED MANUSCRIPT

23

MANUSCRIPT

Figure 4A

ACCEPTED

ACCEPTED MANUSCRIPT

24

MANUSCRIPT

Figure 4B Figure 4. Directed sociograms. Authors of the seminal paper for each method are distinguished by a thicker node border. Arrows are directed from first author to co-authors of each paper. Node size is proportional to the number of published articles. Circles represent authors who have only published articles applying each innovation, diamonds represent authorship of innovation reviews or methods papers, triangles represent authorship of both application and review or methods papers. Red nodes represent authors with institutional affiliation with the early adopters (DRS, Vanderbilt University) and innovators (hdPS, Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women’s Hospital), that drove the diffusion of each innovation. Purple nodes denote authors that had a prior affiliation with the groups that droveACCEPTED innovation and since moved to other institutions. Please refer to on-line supplemental material for animated versions of each figure. A. Disease risk score, 13 components, 43 eligible papers plus the seminal DRS paper [13] B. High-dimensional propensity score, 7 components, 44 papers

ACCEPTED MANUSCRIPT 25 Table 1. Case study summary according to characteristics in the Diffusion of Innovations model Characteristic* Case Study 1 Case Study 2 Disease Risk Score (DRS)[13] High-dimensional Propensity Score (hdPS)[14] Innovation type new statistical (confounder summary score) adaptation of an established [5] statistical method (confounder summary score) method Innovation year 1976 2009

Innovation field Epidemiology Pharmacoepidemiology

Rate of uptake Initially slow Rapid

Attributes* • Relative advantage over conventional methods • Relative advantage over propensity • Controls for several confounders by scores/conventional regression “-” negative impact summarizing all into a single score ( +A) • Identifies proxy variables to help control of + “+” positive impact • Study effect modification by baseline disease unmeasured confounding ( A) risk (+A) • Adaption of the propensity score method (+C, +S) • Initial uncertainty in 1970s/80s (-A) • Sample macro coding available on-line for free • Based on conventional methods (regression and (+T) stratification, +C, +T) thus easily understood • Presented prior to publication at the Annual (+S) meeting of the International Society for • Little consistency in language used to describe Pharmacoepidemiology (+O) the method over time (-O) Social system when Field “born” in 1970s Significant investment/incentives in 2000s innovation • 1962: Kefauver-Harris Amendment to the US • 2005: AHRQ launches DEcIDE Network • introduced Food, Drug and Cosmetic Act in response to 2006: US Medicare Part D-drug data thalidomide disaster • 2007: Canada calls for improved drug safety

• 1966: Boston Collaborative Drug Safety • 2007: US FDA Amendments Act (FDAAA) Research established mandates “Sentinel Initiative” – drug surveillance • Mid-1970s: Drug Epidemiology Unit (now system of electronic data from healthcare Slone Epidemiology Centre) information holders (goal data on >100 million) • 1976: Joint Commission of Prescription Drug • 2008: Canada launches DSEN, commits $32 Use formed MANUSCRIPTmillion over 5 years + $10 million annually after • 1977: Computerized online Medicaid analysis • 2010: US Patient Protection and Affordable Care and surveillance system developed Act creates PCORI and mandates development of • 1985: First International Conference on Methodological Standards Pharmacoepidemiology (“ICPE”) Advantage of this New statistical method to control for confounding Adaptation of an established statistical method over case study over an almost 40 year period and in the context a short period of time (since 2009), during a period of methodological uncertainty. of rapid growth in the field and innovators facilitated the uptake of the innovation through readily available statistical code. AHRQ: Agency for Healthcare Research and Quality, US; DSEN: Drug Safety and Effectiveness Network, Canada; FDA: Food and Drug Administration, US; ICPE: International Conference in Pharmacoepidemiology; PCORI: Patient Centered Outcomes Research Institute, US; Developing Evidence to inform Decisions about Effectiveness (DEcIDE), US; US: United States *The rate of adoption of an innovation is proposed to be affected by its five attributes, A: advantage (relative advantage), C: compatibility (with user values), S: simplicity (in contrast to complexity), T: trialability (degree can be tested, testing reducesACCEPTED uncertainty), O: observability (degree results are visible to others), Box 2 [15].

ACCEPTED MANUSCRIPT 26 Table 2. Institutional affiliations by country and institution type for DRS and hdPS networks DRS Network Credit (%) hdPS Network Credit (%) First and First, Second and First and First, Second and Last Author Last Author Last Author Last Author Country Canada 14.2 14.1 17.4 17.7 China * 0.6 2.3 2.3 France 2.3 2.4 * * Germany * * 0.8 1.1 Greece 2.4 2.3 * * Netherlands 3.1 2.1 * * Saudi Arabia * * 0.4 0.3 Spain 0.8 0.5 * 0.5 Taiwan 3.8 3.5 * * United Kingdom 0.8 1.2 0.6 0.5 United States 72.6 74.0 79.0 77.9 Institution Type Academia 86.5 87.7 89.6 89.6 School 57.5 60.8 58.2 58.9 Hospital 29.0 26.9 31.4 30.7 Government 3.6 3.2 4.7 4.1 Industry 9.9 9.1 5.7 5.8 Pharmaceutical company 0.8 1.3 4.7 4.7 Health insurance plan 7.0 6.0 * * Clinical research organization** 2.1 MANUSCRIPT 1.8 1.0 1.6 DRS: disease risk score, hdPS: high-dimensional propensity score *No credit based on authors used to ascribe credit. ** One clinical research organization was a non-for-profit organization.

ACCEPTED

ACCEPTED MANUSCRIPT 27 Table 3. Recommendations to improve the diffusion of methodological innovation 5 Innovation Attributes Recommendations A C S T O


1. Clearly describe using foundational principles (simple language)

• Clearly articulate the methodological gap and thus need for the innovation

• Clearly articulate relative advantage over existing methods

• Use standard (foundational) biostatistics and epidemiologic language; avoid technical jargon; simplify according to information or selection bias
• Consider using simple figures to walk the reader through concepts

• Leverage language used in foundational principles and established methods when naming the innovation

2. Consider comparing results to established method(s)

• Consider comparing results achieved using the innovation methods to established method(s) in the context of known effects (e.g., known safety or effectiveness of drug), or suspected null findings (e.g., do not expect to see drug-outcome association)

3. Provide sample data, code or calculation examples, and instructions

• Create macros or provide example statistical code in the seminal publication (e.g., appendices) or on a research website


• Provide example dataset(s) and code or calculation examples for easy manipulation and trialability

• Publish a step by step guide for utilization (e.g., when is the method appropriate vs. inappropriate) and clear instructions for how to implement the methods and interpret results MANUSCRIPT




4. Early communication, support and testing



• Present at research conferences and obtain feedback on the method, language and approach prior to publication in full form




• Consider creating workshops to support innovation use and understanding


• Consider creating webinars or open source content to facilitate access and understanding

• Consider using social media (e.g., twitter) to showcase the innovation and highlight or comment on its use

• Continue to apply and test/challenge innovation merits and encourage others to do the same

5. Provide methodological and reporting guidance



• Make clear recommendations for when the method may be appropriate or inappropriately applied, i.e., strengths and limitations

• Identify future areas of methodological development and testing


ACCEPTED
• Make recommendations for reporting standards, e.g., language to describe, data display, supplemental information to present in appendices



• Encourage future applications to cite the seminal innovation paper A: advantage, C: compatibility, S: simplicity, O: observability, T: trialability

ACCEPTED MANUSCRIPT 28 Appendix A:

Keyword terms used for DRS search 1. “disease$ risk$ score$” 2. “summary$ risk$ score$” 3. “multivariate$ risk$ score$” 4. “confounder$ score$” 5. “Miettinen$ confounder$ score$”

Keyword terms used for hdPS search 1. “high-dimensional propensity score” 2. “high dimensional propensity score” 3. “highdimensional propensity score” 4. “high-dimensional” and ‘propensity score” 5. “high dimensional” and “propensity score” 6. “highdimensional” and “propensity score”

MANUSCRIPT

ACCEPTED

ACCEPTED MANUSCRIPT 29 Appendix B: List of Eligible DRS Publications, by Component Component 1: Appendix B1-B21 Component 2: Appendix B22-B27 Component 3: Appendix B28-B31 Component 4: Appendix B32-B33 Component 5: Appendix B34-B35 Component 6: Appendix B36 Component 7: Appendix B37 Component 8: Appendix B38 Component 9: Appendix B39 Component 10: Appendix B40 Component 11: Appendix B41 Component 12: Appendix B42 Component 13: Appendix B43

[B1] Arbogast PG, Kaltenbach L, Ding H, et al. Adjustment for multiple cardiovascular risk factors using a summary risk score. Epidemiology. 2008;19(1):30-7.

[B2] Arbogast PG, Ray WA. Use of disease risk scores in pharmacoepidemiologic studies. Stat Meth Med Res. 2009;18(1):67-80.

[B3] Arbogast PG, Ray WA. Performance of disease risk scores, propensity scores, and traditional multivariable outcome regression in the presence of multiple confounders. Am J Epidemiol. 2011;174(5):613-20. [B4] Bobo WV, Cooper WO, Stein CM, et al. AntipsychoticsMANUSCRIPT and the risk of type 2 diabetes mellitus in children and youth. JAMA Psychiatry. 2013;70(10):1067-75. [B5] Curtis JR, Xie FL, Chen L, et al. The comparative risk of serious infections among rheumatoid arthritis patients starting or switching biological agents. Ann Rheum Dis. 2011;70(8):1401-06.

[B6] Curtis JR, Xie FL, Chen L, et al. Use of a disease risk score to compare serious infections associated with anti-tumor necrosis factor therapy among high- versus lower-risk rheumatoid arthritis patients. Arthritis Care Res. 2012;64(10):1480-9.

[B7] Fireman B, Toh S, Butler MG, et al. A protocol for active surveillance of acute myocardial infarction in association with the use of a new antidiabetic pharmaceutical agent. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):282-90.

[B8] Graham DJ, Campen D, Hui R, et al. Risk of acute myocardial infarction and sudden cardiac death in patients treated with cyclo-oxygenase 2 selective and non-selective non-steroidal anti-inflammatoryACCEPTED drugs: nested case-control study. Lancet. 2005;365(9458):475-81.

[B9] Grijalva CG, Chung CP, Arbogast PG, et al. Assessment of adherence to and persistence on disease-modifying antirheumatic drugs (DMARDs) in patients with rheumatoid arthritis. Med Care. 2007;45(10 Suppl 2):S66-76.

ACCEPTED MANUSCRIPT 30 [B10] Habel LA, Cooper WO, Sox CM, et al. ADHD medications and risk of serious cardiovascular events in young and middle-aged adults. JAMA. 2011;306(24):2673-83.

[B11] Ray WA, Chung CP, Murray KT, et al. Atypical antipsychotic drugs and the risk of sudden cardiac death. N Engl J Med. 2009;360(3):225-35.

[B12] Ray WA, Chung CP, Stein CM, et al. Risk of peptic ulcer hospitalizations in users of NSAIDs with gastroprotective cotherapy versus coxibs. Gastroenterology. 2007;133(3):790-8.

[B13] Ray WA, Meredith S, Thapa PB, et al. Cyclic antidepressants and the risk of sudden cardiac death. Clin Pharmacol Ther. 2004;75(3):234-41.

[B14] Ray WA, Meredith S, Thapa PB, et al. Antipsychotics and the risk of sudden cardiac death. Arch Gen Psychiatry. 2001;58(12):1161-7.

[B15] Ray WA, Murray KT, Hall K, et al. Azithromycin and the risk of cardiovascular death. N Engl J Med. 2012;366(20):1881-90.

[B16] Ray WA, Murray KT, Meredith S, et al. Oral erythromycin and the risk of sudden death from cardiac causes. N Engl J Med. 2004;351(11):1089-96.

[B17] Ray WA, Stein CM, Daugherty JR, et al. COX-2 selective non-steroidal anti- inflammatory drugs and risk of serious coronary heart disease. Lancet. 2002;360(9339):1071-3.

[B18] Ray WA, Stein CM, Hall K, et al. Non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease: an observational cohortMANUSCRIPT study. Lancet. 2002;359(9301):118-23. [B19] Ray WA, Varas-Lorenzo C, Chung CP, et al. Cardiovascular risks of nonsteroidal antiinflammatory drugs in patients after hospitalization for serious coronary heart disease. Circ Cardiovasc Qual Outcomes. 2009;2(3):155-63.

[B20] Roumie CL, Choma NN, Kaltenbach L, et al. Non-aspirin NSAIDs, cyclooxygenase-2 inhibitors and risk for cardiovascular events-stroke, acute myocardial infarction, and death from coronary heart disease. Pharmacoepidemiol Drug Saf. 2009;18(11):1053-63.

[B21] Roumie CL, Mitchel EF, Kaltenbach L, et al. Nonaspirin NSAIDs, cyclooxygenase 2 inhibitors, and the risk for stroke. Stroke. 2008;39(7):2037-45.

[B22] Cadarette SM, Gagne JJ, Solomon DH, et al. Confounder summary scores when comparing the effects of multiple drug exposures. Pharmacoepidemiol Drug Saf. 2010;19(1):2-9.

[B23] Gagne JJ, Bykov K, Willke RJ, et al. Treatment dynamics of newly marketed drugs and implications for comparativeACCEPTED effectiveness research. Value Health. 2013;16(6):1054-62. [B24] Glynn RJ, Gagne JJ, Schneeweiss S. Role of disease risk scores in comparative effectiveness research with emerging therapies. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 2):138-47.

ACCEPTED MANUSCRIPT 31 [B25] Solomon DH, Avorn J, Stürmer T, et al. Cardiovascular outcomes in new users of coxibs and nonsteroidal antiinflammatory drugs: high-risk subgroups and time course of risk. Arthritis Rheum. 2006;54(5):1378-89.

[B26] Stürmer T, Schneeweiss S, Brookhart MA, et al. Analytic strategies to adjust confounding using exposure propensity scores and disease risk scores: nonsteroidal antiinflammatory drugs and short-term mortality in the elderly. Am J Epidemiol. 2005;161(9):891-8.

[B27] Tadrous M, Gagne JJ, Stürmer T, et al. Disease risk score as a confounder summary method: systematic review and recommendations. Pharmacoepidemiol Drug Saf. 2013;22(2):122-29.

[B28] Gomes T, Mamdani MM, Dhalla IA, et al. Opioid dose and drug-related mortality in patients with nonmalignant pain. Arch Intern Med. 2011;171(7):686-91.

[B29] Gomes T, Redelmeier DA, Juurlink DN, et al. Opioid dose and risk of road trauma in Canada: a population-based study. JAMA Intern Med. 2013;173(3):196-201.

[B30] Juurlink DN, Mamdani MM, Kopp A, et al. The risk of suicide with selective serotonin reuptake inhibitors in the elderly. Am J Psychiatry. 2006;163(5):813-21.

[B31] Park-Wyllie LY, Mamdani MM, Li P, et al. Cholinesterase inhibitors and hospitalization for bradycardia: a population-based study. PLoS Med. 2009;6(9).

[B32] van Staa TP, Abenhaim L, Cooper C, et al. Public health impact of adverse bone effects of oral corticosteroids. Br J Clin Pharmacol. 2001; 51(6):601-07.MANUSCRIPT [B33] van Staa TP, Rietbrock S, Setakis E, et al. Does the varied use of NSAIDs explain the differences in the risk of myocardial infarction? J Intern Med. 2008;264(5):481-92.

[B34] Wu CS, Chang CM, Chen CY, et al. Association between antidepressants and venous thromboembolism in Taiwan. J Clin Psychopharmacol. 2013;33(1):31-37.

[B35] Wu CS, Ting TT, Wang SC, et al. Effect of benzodiazepine discontinuation on dementia risk. Am J Geriatr Psychiatry. 2011;19(2):151-9.

[B36] Rosenberg L, Slone D, Shapiro S, et al. Aspirin and myocardial infarction in young women. Am J Public Health. 1982;72(4):389-91.

[B37] Journois D, Baufreton C, Mauriat P, et al. Effects of inhaled nitric oxide administration on early postoperative mortality in patients operated for correction of atrioventricular canal defects. Chest. 2005;128(5):3537-44.ACCEPTED [B38] Applebaum KM, Nelson HH, Zens MS, et al. Oral contraceptives: a risk factor for squamous cell carcinoma? J Invest Dermatol. 2009;129(12):2760-5.

[B39] Kritsotakis EI, Tsioutis C, Roumbelaki M, et al. Antibiotic use and the risk of carbapenem-resistant extended-spectrum-β-lactamase-producing Klebsiella pneumoniae infection in hospitalized patients: results of a double case-control study. J Antimicrob

ACCEPTED MANUSCRIPT 32 Chemother. 2011;66(6):1383-91.

[B40] Block GA, Kilpatrick RD, Lowe KA, et al. CKD-mineral and bone disorder and risk of death and cardiovascular hospitalization in patients on hemodialysis. Clin J Am Soc Nephrol. 2013;8(12):2132-40.

[B41] Joseph KS, Blais L, Ernst P, et al. Increased morbidity and mortality related to asthma among asthmatic patients who use major tranquillisers. BMJ. 1996;312(7023):79-82.

[B42] Stuart EA, Lee BK, Leacy FP. Prognostic score-based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research. J Clin Epidemiol. 2013;66(8):S84-S90.

[B43] Swan SH, Brown WL. Oral-contraceptive use, sexual activity and cervical carcinoma. Am J Obstet Gynecol. 1981;139(1):52-7.

MANUSCRIPT

ACCEPTED

ACCEPTED MANUSCRIPT 33 Appendix C: List of Eligible hdPS Publications, by Component Component 1: Appendix C1-C34 Component 2: Appendix C35-C39 Component 3: Appendix C40 Component 4: Appendix C41 Component 5: Appendix C42 Component 6: Appendix C43 Component 7: Appendix C44

[C1] Carney GA, Bassett K, Wright JM, et al. Is thiazolidinediones use a factor in delaying the need for insulin therapy in type 2 patients with diabetes? A population-based cohort study. BMJ Open. 2012;2(6).

[C2] Dhalla IA, Gomes T, Yao Z, et al. Chlorthalidone versus hydrochlorothiazide for the treatment of hypertension in older adults: a population-based cohort study. Ann Intern Med. 2013;158(6):447-55.

[C3] Dormuth CR, Hemmelgarn BR, Paterson JM, et al. Use of high potency statins and rates of admission for acute kidney injury: multicenter, retrospective observational analysis of administrative databases. BMJ. 2013;346(7900).

[C4] Gagne JJ, Bykov K, Willke RJ, et al. Treatment dynamics of newly marketed drugs and implications for comparative effectiveness research. Value Health. 2013;16(6):1054-62.

[C5] Gagne JJ, Glynn RJ, Rassen JA, et al. Active safety monitoring of newly marketed medications in a distributed data network: applicatMANUSCRIPTion of a semi-automated monitoring system. Clin Pharmacol Ther. 2012;92(1):80-86. [C6] Gagne JJ, Rassen JA, Walker AM, et al. Active safety monitoring of new medical products using electronic healthcare data selecting alerting rules. Epidemiology. 2012;23(2):238-46.

[C7] Garbe E, Kloss S, Suling M, et al. High-dimensional versus conventional propensity scores in a comparative effectiveness study of coxibs and reduced upper gastrointestinal complications. Eur J Clin Pharmacol. 2013;69(3):549-57.

[C8] Gomes T, Juurlink DN, Ho JMW, et al. Risk of serious falls associated with oxybutynin and tolterodine: a population based study. J Urol. 2011;186(4):1340-4.

[C9] Huybrechts KF, Brookhart MA, Rothman KJ, et al. Comparison of different approaches to confounding adjustment in a study on the association of antipsychotic medication with mortality in older nursing home patients. Am J Epidemiol. 2011;174(9):1089-99.

[C10] HuybrechtsACCEPTED KF, Gerhard T, Crystal S, et al. Differential risk of death in older residents in nursing homes prescribed specific antipsychotic drugs: population based cohort study. BMJ. 2012;344:e977.

[C11] Huybrechts KF, Rothman KJ, Silliman RA, et al. Risk of death and hospital admission for major medical events after initiation of psychotropic medications in older adults admitted to nursing homes. CMAJ. 2011;183(7):E411-9.

ACCEPTED MANUSCRIPT 34 [C12] Huybrechts KF, Schneeweiss S, Gerhard T, et al. Comparative Safety of antipsychotic medications in nursing home residents. J Am Geriatr Soc. 2012;60(3):420-9.

[C13] Huybrechts KF, Seeger JD, Rothman KJ, et al. Bias in comparative effectiveness studies due to regional variation in medical practice intensity a legitimate concern, or much ado about nothing? Circ Cardiovasc Qual Outcomes. 2012;5(5):E61-4.

[C14] Juurlink DN, Gomes T, Shah BR, et al. Adverse cardiovascular events during treatment with glyburide (glibenclamide) or gliclazide in a high-risk population. Diabet Med. 2012;29(12):1524-8.

[C15] Kawasumi Y, Paterson MJ, Morrow RL, et al. Comparative Effectiveness of Tiotropium and Ipratropium in Prevention of Hospital Readmission for COPD: a population-based cohort study. Clin Ther. 2013;35(4):523-31.

[C16] Kulik A, Rassen JA, Myers J, et al. Comparative effectiveness of preventative therapy for venous thromboembolism after coronary artery bypass graft surgery. Circ Cardiovasc Interv. 2012;5(4):590-6.

[C17] Lam J, Gomes T, Juurlink DN, et al. Hospitalization for hemorrhage among warfarin recipients prescribed amiodarone. Am J Cardiol. 2013;112(3):420-3.

[C18] Le HV, Poole C, Brookhart MA, et al. Effects of aggregation of drug and diagnostic codes on the performance of the high-dimensional propensity score algorithm: an empirical example. BMC Med Res Methodol. 2013;13:142. [C19] Palmsten K, Hernandez-Diaz S, HuybrechtsMANUSCRIPT KF, et al. Use of antidepressants near delivery and risk of postpartum hemorrhage: cohort study of low income women in the United States. BMJ. 2013;347:f4877.

[C20] Patorno E, Bohn RL, Wahl PM, et al. Anticonvulsant medications and the risk of suicide, attempted suicide, or violent death. JAMA. 2010;303(14):1401-9.

[C21] Patorno E, Glynn RJ, Hernandez-Diaz S, et al. Risk of ischemic cerebrovascular and coronary events in adult users of anticonvulsant medications in routine care settings. J Am Heart Assoc. 2013;2(4):e000208.

[C22] Patrick AR, Franklin JM, Weinstein MC, et al. Sequential value-of-information assessment for prospective drug safety monitoring using claims databases: the comparative safety of prasugrel v. clopidogrel. Med Decis Making. 2013;33(7):949-60.

[C23] Polinski JM, Schneeweiss S, Glynn RJ, et al. Confronting "confounding by health system use" in MedicareACCEPTED Part D: comparative effectiveness of propensity score approaches to confounding adjustment. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 2):90-8.

[C24] Polinski JM, Shrank WH, Glynn RJ, et al. Association between the Part D coverage gap and adverse health outcomes. J Am Geriatr Soc. 2012;60(8):1408-17.

[C25] Polinski JM, Shrank WH, Glynn RJ, et al. Beneficiaries with cardiovascular disease and the Part D coverage gap. Circ Cardiovasc Qual Outcomes. 2012;5(3):387-95.

ACCEPTED MANUSCRIPT 35 [C26] Rassen JA, Avorn J, Schneeweiss S. Multivariate-adjusted pharmacoepidemiologic analyses of confidential information pooled from multiple health care utilization databases. Pharmacoepidemiol Drug Saf. 2010;19(8):848-57.

[C27] Rassen JA, Choudhry NK, Avorn J, et al. Cardiovascular outcomes and mortality in patients using clopidogrel with proton pump inhibitors after percutaneous coronary intervention or acute coronary syndrome. Circulation. 2009;120(23):2322-9.

[C28] Rassen JA, Glynn RJ, Brookhart MA, et al. Covariate selection in high-dimensional propensity score analyses of treatment effects in small samples. Am J Epidemiol. 2011;173(12):1404-13.

[C29] Rassen JA, Schneeweiss S. Using high-dimensional propensity scores to automate confounding control in a distributed medical product safety surveillance system. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):41-9.

[C30] Schneeweiss S, Patrick AR, Solomon DH, et al. Comparative safety of antidepressant agents for children and adolescents regarding suicidal acts. Pediatrics. 2010;125(5):876-88.

[C31] Schneeweiss S, Patrick AR, Solomon DH, et al. Variation in the risk of suicide attempts and completed suicides by antidepressant agent in adults: a propensity score-adjusted analysis of 9 years' data. Arch Gen Psychiatry. 2010;67(5):497-506.

[C32] Schneeweiss S, Rassen JA, Glynn RJ, et al. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512-22. MANUSCRIPT [C33] Setoguchi S, Shrank WH, Liu J, et al. Angiotensin receptor blockers and angiotensin- converting enzyme inhibitors: challenges in comparative effectiveness using medicare data. Clin Pharmacol Ther. 2011;89(5):674-82.

[C34] Solomon DH, Rassen JA, Glynn RJ, et al. The comparative safety of analgesics in older adults with arthritis. Arch Intern Med. 2010;170(22):1968-76.

[C35] Li X, Hui S, Ryan P, et al. Statistical visualization for assessing performance of methods for safety surveillance using electronic databases. Pharmacoepidemiol Drug Saf. 2013;22(5):503-9.

[C36] Madigan D, Ryan PB, Schuemie M, et al. Evaluating the impact of database heterogeneity on observational study results. Am J Epidemiol. 2013;178(4):645-51.

[C37] Reich C, Ryan PB, Stang PE, et al. Evaluation of alternative standardized terminologies for medical conditionsACCEPTED within a network of observational healthcare databases. J Biomed Inform. 2012;45(4):689-96.

[C38] Ryan P, Suchard MA, Schuemie M, et al. Learning from epidemiology: interpreting observational database studies for the effects of medical products. Stat Biopharm Res. 2013;5(3):170-9.

ACCEPTED MANUSCRIPT 36 [C39] Ryan PB, Madigan D, Stang PE, et al. Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership. Stat Med. 2012;31(30):4401-15.

[C40] Wong MCS, Tam WWS, Wang HHX, et al. Predictors of the incidence of all-cause mortality and deaths due to diabetes and renal diseases among patients newly prescribed antihypertensive agents: a cohort study. Int J Cardiol. 2013;168(5):4705-10.

[C41] Stenger MR, Slaughter JL, Kelleher K, et al. Hospital variation in nitric oxide use for premature infants. Pediatrics. 2012;129(4):E945-51.

[C42] Zhou X, Murugesan S, Bhullar H, et al. An evaluation of the THIN database in the OMOP common data model for active drug safety surveillance. Pharmacoepidemiol Drug Saf. 2013;36(2):119-34.

[C43] Eurich DT, Simpson S, Senthilselvan A, et al. Comparative safety and effectiveness of sitagliptin in patients with type 2 diabetes: Retrospective population based cohort study. BMJ. 2013;346(7908).

[C44] Toh S, García Rodríguez LA, Hernán MA. Confounding adjustment via a semi- automated high-dimensional propensity score algorithm: an application to electronic medical records. Pharmacoepidemiol Drug Saf. 2011;20(8):849-57.

MANUSCRIPT

ACCEPTED

ACCEPTED MANUSCRIPT 37 Appendix D: Detailed Institutional Credit by Network and Component

Table D1. DRS co-authorship network institutional credit, 43 papers, 153 authors, 13 components Credit (%) Institution Country First and Last First, Second and Author Last Author Overall Network Amgen US * 0.8 Boston University US 1.9 2.1 Brigham and Women’s Hospital US 5.4 6.1 Centro Español de Investigacion Farmacoepidemiologica Spain 0.8 0.5 Dartmouth University US 0.8 0.6 Denver Nephrology US 1.2 0.8 Drexel University US * 0.8 Far East Memorial Hospital Taiwan 1.9 2.1 Food and Drug Administration US 0.8 0.6 Harvard University US 6.2 7.5 Hȏpital Européen Georges-Pompidou France 2.3 1.6 Hȏpital Laennec-Necker France * 0.8 Institute for Clinical Evaluative Sciences Canada 3.4 3.3 Johns Hopkins University US 2.3 1.6 Kaiser Permanente US 7.0 6.0 Lin-Kou & Chang Gung University China * 0.6 McGill University Canada 2.3 2.8 Medicines and Healthcare Products Regulatory Agency UK 0.8 1.2 Mortimer B. Davis Jewish General Hospital Canada * 0.5 National Institute of Health Research Taiwan 1.9 1.4 Northwestern University US 0.8 0.6 Outcomes Insight Inc. US 1.2 0.8 Proctor & Gamble US 0.8 0.5 RTI Health Solutions US * 0.5 St. Michael’s Hospital Canada 1.2 1.4 Sunnybrook Hospital Canada 1.7 1.6 Tennessee Valley Healthcare System US Veterans Affairs US 14.3 11.6 University Hospital Heraklion MANUSCRIPTGreece 1.2 1.4 University of Alabama at Birmingham US 4.7 4.7 University of Crete Greece 1.2 0.9 University of Minnesota US * 0.6 University of North Carolina US 0.4 0.3 University of Pennsylvania US 1.2 0.8 University of Toronto Canada 5.6 4.5 Utrecht University Netherlands 3.1 2.1 Vanderbilt University US 23.6 26.7 By Component Component 1 – 67 authors, 21 publications Centro Español de Investigacion Farmacoepidemiologica Spain 1.6 1.0 Food and Drug Administration US 1.6 1.2 Harvard University US * 1.6 Kaiser Permanente US 9.5 7.5 RTI Health Solutions US * 1.0 Tennessee Valley Healthcare System, US Veteran Affairs US 29.4 23.7 University of Alabama Birmingham US 9.5 9.5 Vanderbilt University US 48.4 54.6 Component 2 – 16 authors, 6 publications Brigham and Women’s Hospital US 38.9 43.8 Harvard University US 38.9 43.8 University of NorthACCEPTED Carolina US 2.8 2.1 University of Toronto Canada 19.4 10.4 Component 3 – 12 authors, 4 publications Institute for Clinical Evaluative Sciences Canada 36.7 35.4 St. Michael’s Hospital Canada 13.3 14.9 Sunnybrook Hospital Canada 18.3 17.4 University of Toronto Canada 31.7 32.3 Component 4 – 7 authors, 2 publications McGill University Canada * 10.0 Mortimer B. Davis Jewish General Hospital Canada * 10.0

ACCEPTED MANUSCRIPT 38

Proctor & Gamble US 16.7 10.0 Medicines and Healthcare Products Regulatory Agency UK 16.7 25.0 Utrecht University Netherlands 66.7 45.0 Component 5 – 13 authors, 2 publications Far Eastern Memorial Hospital Taiwan 41.7 45.8 Link-Kou & Chang Gung University China * 12.5 National Health Research Institutes Taiwan 41.7 29.2 Northwestern University US 16.7 12.5 Component 6 – 6 authors, 1 publication Boston University US 33.3 25.0 Dartmouth University US 33.3 25.0 Harvard University US * 25.0 University of Minnesota US 33.3 25.0 Component 7 – 6 authors, 1 publication Hȏpital Européen Georges-Pompidou France 100.0 66.7 Hȏpital Laennec -Necker France * 33.3 Component 8 – 6 authors, 1 publication Boston University US 50.0 66.7 University of Pennsylvania US 50.0 33.3 Component 9 – 5 authors, 1 publication University of Crete Greece 50.0 40.0 University Hospital of Heraklion Greece 50.0 60.0 Component 10 – 5 authors, 1 publication Amgen US 50.0 33.3 Denver Nephrologists PC US 50.0 33.3 Outcomes Insight Inc. US * 33.3 Component 11 – 4 authors, 1 publication McGill University Canada 100 100 Component 12 – 3 authors, 1 publication Drexel University US * 33.3 Johns Hopkins University US 100 66.7 Component 13 – 22 authors, 1 publication Kaiser Permanente US 100 100 *no institutional credit based solely on first and last author

MANUSCRIPT

ACCEPTED

ACCEPTED MANUSCRIPT 39 Table D2. hdPS co-authorship network institutional credit, 44 papers, 147 authors, 6 components Credit (%) Institution Country First and Last First, Second and Author Last Author Overall Network Boston University US 0.9 1.0 Brigham and Women’s Hospital US 26.7 26.1 Brunel University UK 0.6 0.5 Centro Espanol de Investigacion Farmacoepidemiologica Spain * 0.5 Chinese University of Hong-Kong China 2.3 2.3 Columbia University US 1.7 1.5 Duke University US 1.3 0.9 Eli Lilly Canada 0.6 0.4 Florida Atlantic University US 0.5 0.3 Food and Drug Administration US 1.1 0.6 GlaxoSmithKline US 0.8 0.6 Harvard University US 28.8 28.3 HealthCore Inc. US * 0.3 Indiana University US 1.1 1.1 Institute for Clinical Evaluative Sciences Canada 2.7 3.9 Johnson & Johnson US 2.2 2.4 King Saud University Saudi Arabia 0.4 0.3 Massachusetts Institute of Technology US 0.6 0.5 McGill University Canada 1.1 0.8 McMaster Univeristy Canada 0.6 0.4 National Institutes of Health US 3.2 3.2 National Institutes of Mental Health US 0.5 0.3 Nationwide Children’s Hospital US 0.8 0.9 New York University US 0.6 0.5 Ohio State University US 1.5 1.4 Outcomes Science Inc. US 0.6 0.4 Pfizer US 1.1 1.4 Regenstrief Institute US 1.1 1.1 RTI Health Solutions US * 0.3 Rutgers University US 0.8 0.9 St. Joseph’s Healthcare MANUSCRIPTUS 0.6 0.4 St. Michael’s Hospital Canada 1.9 1.7 Sunnybrook Hospital Canada 0.4 0.5 University of Alberta Canada 2.3 2.3 University of Bremen Germany 0.8 1.1 University of British Columbia Canada 5.7 4.5 University of Calgary Canada * 0.8 University of California Los Angeles US * 0.8 University of Florida US 0.5 0.3 University of North Carolina US 2.0 2.4 University of Toronto Canada 2.1 2.4 By Component Component 1 – 96 authors, 34 publication s Boston University US 1.2 1.3 Brigham and Women’s Hospital US 33.9 33.2 Duke University US 1.87 1.2 Ely Lilly Canada 1.5 0.7 Florida Atlantic University US 0.6 0.4 GlaxoSmithKline US 1.0 0.7 Harvard University US 35.8 35.4 HealthCore Inc. US * 0.4 Institute for Clinical Evaluative Sciences Canada 3.4 5.3 King Saud UniversityACCEPTED Saudi Arabia 0.5 0.3 McGill University Canada 1.5 1.0 McMaster University Canada 0.7 0.5 National Institutes of Mental Health US 0.6 0.4 Outcomes Science Inc US 0.9 0.5 RTI Health Solutions US * 0.4 Rutgers University US 1.0 1.2 St. Joseph’s Healthcare Canada 0.7 0.5 St. Michael’s Hospital Canada 2.5 2.2 Sunnybrook Hospital Canada 0.5 0.7 University of Bremen Germany 1.0 1.5

ACCEPTED MANUSCRIPT 40

University of British Columbia Canada 7.4 5.9 University of Calgary Canada * 1.0 University of North Carolina US 2.0 2.7 University of Toronto Canada 3.4 3.6 Component 2 – 16 authors, 5 publication s Columbia University US 15.0 12.9 Food and Drug Administration US 10.0 5.0 Indiana University US 10.0 10.0 Johnson & Johnson USA US 19.0 21.2 National Institutes of Health US 28.0 28.6 Regenstrief Institute US 10.0 10.0 University of California Los Angeles US * 6.7 University of Florida US 4.0 2.9 University of North Carolina US 4.0 2.9 Component 3 – 11 authors, 1 publication Chinese University of Hong -Kong China 100.0 100.0 Component 5 – 8 authors, 1 publication Nationwide Children’s Hospital US 33.3 40.0 Ohio State University US 66.7 60.0 Component 4 – 7 authors, 1 publication Brunel University UK 25.0 20.0 New York University US 25.0 20.0 Pfizer USA US 50.0 60.0 Component 6 – 6 authors, 1 publication University of Alberta Canada 100.0 100.0 Component 7 – 3 authors, 1 publication Brigham and Women’s Hospital US 25.0 20.0 Centro Español de Investigacion Farmacoepidemiologica Spain * 20.0 Harvard University US 50.0 40.0 Massachusetts Institute of Technology US 25.0 20.0 *no institutional credit based solely on first and last author

MANUSCRIPT

ACCEPTED

ACCEPTED MANUSCRIPT 41 Appendix E: Crude Estimate of the Number of Pharmacoepidemiology Publications, by Year

400

350

300

250

200

150

NumberPublications of 100

50

0 197319751977197919811983198519871989199119931995199719992001200320052007200920112013 Year of Publication

A

1400

1200 1000 MANUSCRIPT 800

600

NumberPublications of 400

200

0 197319751977197919811983198519871989199119931995199719992001200320052007200920112013 Year of Publication

B

Figure E1. Crude estimate of the number of publications with “pharmacoepidemiology” as a keyword from 1902 to 2013. A. Number ofACCEPTED publications from PubMed® only, first publication in 1987 (n=2,854) B. Number of publications from PubMed®, EMBASE®, and MEDLINE® combined, first publication in 1973 (n=6,429)