Additional file 1: Table S1 Characteristics of studies focusing on processes, theories or frameworks assessing research impact. Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e Aymerich et al Spanish network center for M Domains: Methods and Payback Model  The method (2012) [15] research in epidemiology and i i. knowledge translation; indicators: showed a good public health x Self-reported discriminating ability e ii. research capacity building; assessment that makes it d (Questionnaire to possible to Investigators). measure, reliably, potential healthcare benefit; m iii. the extent to which e Peer review a project’s t iv. degree of objective fulfilment; assessment objectives were met h (Questionnaire to as well as the o v. level of impact of research groups external experts). degree to which the d performance; and project contributed s to enhance the r vi. wider social payback. group’s scientific e performance and of s its social payback. e a r c h Banzi et al Health research S Domains: Methods and Banzi Research  Evaluation of the (2011) [13] y i. advancing knowledge; indicators: Impact Model research impact is s Bibliometric & a heterogeneous, t ii. capacity building; citation analysis. e and evolving m discipline. Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e a iii. informing decision-making; Desk analysis.  Multidimensional ti conceptual c iv. health benefits; and Database review. frameworks take r into account e Interviews. v v. broad socio- economic benefits. several aspects of i impact and use a Case studies. e mix of analytical w approaches to Audit. measure impacts. Assessment panels.  It remains to be Econometrics. clarified how different impact dimensions should be valued and balanced by assessors to fit them to their specific purposes and contexts..

Barber et al Public involvement M Public involvement in research Methods: -  It is feasible to (2012) [16] (community engagement) in i  Two-round Delphi evaluate the impact research x study with pre- of public e specified 80% involvement on d consensus some research Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e criterion. processes, m outcomes and on e  Follow-up key stakeholders. t interviews with UK h and international o panellists from d different settings, s including r universities, health e and social care s institutions and e charitable a organizations. r c h Barker (2007) Research in the United C UK academic institutions Indicators: Research Assessment  RAE represents [17] Kingdom (UK) o Three quality profiles are defined (panel  RAE1: staff Exercise (RAE) 2008 one of the most m decides the weight given to each profile): information institutionalized m i. research output (minimum 50%); (volume and type forms of research e of contracts, evaluation in the n ii. research context (minimum 5%); and external OECD economies. t collaborators), It has become a a analysis of funding primary means of other indicators (minimum 5%). r iii. for research concentrating y fellows; resources for research in a  RAE2: research relatively small output number of Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e (publications, universities. patents, reports, database, software);

 RAE3: research scholarships

 RAE4: attractiveness for external funding; and

 RAE5a: information on groups of research (networking and research culture.

Boyack & National Institutes of Health C Research impact using citation analysis Methods: -  The median time Jordan (2011) (NIH) grants it Articles linked to lag to publication for [18] a using the grant a new grant is 3 ti acknowledgment years. o strings in PubMed n using a parsing and  Each grant matching process, contributes to a with additional data approximately 1.7 n from PubMed and articles per year, Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e a citation counts from averaged over all l Scopus added to grant types. y the linkage data. s i Indicators: s  No of unique Articles.

 No of multi- institute articles.

 Percentage multi-institute articles.

 Mean duration (years).

 Percentage of grants with articles.

 No of Scopus Articles

Average cites.

Buxton et al Medical Research in the E i. Health gains net of the health care Methods: Return on investment  Public and private (2008) [19] United Kingdom (UK) c Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e (cardiovascular health and o costs of delivering them  Review of the research leads to mental health) n economic improved o ii. Gross domestic product (GDP) gains evaluation productivity and m performance in the i (national income that results directly literature. c and indirectly from the medical research economy generally e and the further activity stimulated by it).  Monetised the v total Quality  The use of a a Adjusted Life standardised (and l Years (QALYs) mapped) way of u gained by classifying a research funding ti multiplying these o estimates by by funding n published figures agencies would on the opportunity aide future cost of a QALY research impact within the current assessment. National Health Service budget  Further research needs to be  Estimates of the undertaken to incremental health care costs understand the associated with time lag between each intervention research (multiplied by the expenditure and numbers of users Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e to quantify the health gain. incremental health care costs of each  Deeper intervention). understanding of the international  Estimated flows of knowledge ‘spillovers’ from and influence public/charitable would be valuable. research between organisations and between sectors to estimate the impact of this research on the UK’s GDP.

Indicators:  Total value of the QALYs gained from the specific interventions.

 Proportion of UK health care benefit attributable to UK Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e research.

 Mean lag between research and impact.

 Internal rates of return (IRRs)

on research investment from the value of the QALYs gained.

 GDP gains that result from increased public/charitable medical research.

Buykx et al Health Services Research L Elements: Methods and Health Services  Demonstrating (2012) [20] it i. research-related; indicators: Research Impact accountability e Framework Peer-reviewed through recording r articles (journal ii. impact (advancing knowledge); research impact a impact factor). t should be balanced iii. policy impact (informing decision u All other with realistic Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e r making); publications, clinical expectations about e guidelines, the degree of iv. service impact (Improving health and recommendations, certainty with which r health systems); and impact can be e policy briefs and so on. ‘proven’. v v. societal impact (creating broad social i and economic benefit). e Presentations to  Framework yet to w academic be tested but conferences, authors propose a decision makers, testing the utility of n non-governmental the framework by d organizations, recording and consumer groups & monitoring the c public. impact of their own r research. it Media (e.g. press i releases, Tweets, c blogs). a l Data sharing. a p Subsequent grants p received (type and r quantity). a i s Follow on research a by self or others. l Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e o PhD and f postdoctoral e completions. x i Development, s delivery and ti evaluation of n capacity g development m program. o d Active participation e in policy networks l (e.g. advocacy s groups, think tanks, o government f advisors). r e s Validated measures e to improve health a care input and r effectiveness. c h

i m p a Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e c t.

Deloitte National Health and Medical E Economic benefits: Methods: Return on investment  It is feasible Access Research Council (NHMRC) c  Health system expenditure savings Health system to conduct an Economics Research in Australia o expenditure extrapolated cost (2012) [21] n  Return on investment modelling based on benefit analysis of o projected health medical research. m care costs and i research and  For every c development (R&D_ dollar spent on e expenditure to additional NHMRC v derive return on R&D, seven cents a investment. would be returned in l health expenditure u Sensitivity analysis savings in the a future. ti Indicators: o  Health system n expenditure saving

 Net present value (NPV) of extra NHMRC expenditure ($bn) Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e  Cents returned on the dollar

Derrick et al Australian Researchers M Research Impact and Influence Bibliometric -  For 4 of the (2011) [22] in Six Fields of Public Health i indicators: h-index, six fields the results x m-index, m-quotient showed a modest e and q2-index. positive correlation d between different Peer assessment of research metrics m researcher and peer e influence. assessments of t research influence. h o  Peer d understanding of s research influence ( within these fields c differed from it visibility in the a mainstream, peer- ti reviewed scientific o literature n

a n a l y Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e s i s &

p e e r a s s e s s m e n t ) Franks et al Prevention research C Research impact Methods: -  Conventional (2006)[23] it  Research Centres bibliometric analysis a asked to list up to assess the ti to15 articles scientific impact of o considered the public health n most important to prevention research have been is feasible, but of a published by its limited utility. n group in peer- Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e a reviewed journals  Assessment l between 1994 and of impact on public y 2004. health practice, s policy, or on the i Each was also health of s asked to name the populations, will journals it require more than a considered to be bibliometric the most approach. influential in its field(s) of interest.

Indicators:  Impact factor.

 Citations.

 Impact half-life.

Graham et al Health research: Alberta M  Advancing Methods: Canadian  The model CAHS (2012) [24] Heritage Foundation for i  Citation analysis Academy of Health model can be applied Medical Research x Sciences (CAHS)  Knowledge at multiple levels e impact framework[25]  Document review from the micro (e.g. d (quantitative and (based on Payback  Building Capacity individual grantee qualitative) Model) m projects, research  Informing Decision- mapped to the e CAHS impact programs, etc.), meso t categories (e.g. organization) to h the macro (e.g. Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e o  Making;  Case studies contribution to the d provincial innovation s  Health; and and health systems, d etc.) and across e Indicators:  Broad Socio-Economic. different time s The impact categories i are divided into a intervals. g number of n subcategories with 66  The framework is indicators mapped to appropriate for each category or evaluating impacts subcategory. across the full spectrum of health research.



Group of To measure the innovation M Reach: The spread or breadth of influence Methods: Excellence in  It is possible to Eight dividend of research i or effect on the relevant constituencies  Citation analysis Innovation assess research Australian generated by Australian x Significance: the intensity of the influence impact across a (universities) universities across areas of: e or effect  Case studies broad range of and Australian  Defence d disciplines. Technology  Panel assessment Network m  Economic Development (made up of  Case study (2012)[26] e research and end approaches can t  Society (including health) users) provide a h compelling o The Panels assessed narrative of the Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e  Environment d each case study impact of research. s against overall Reach a and Significance  Research impact n rather than assessing could be assessed d each separately. against an Following outcomes based c assessment, a rating system of o was assigned to each classification. m case study according m to the following scale:  Expert Panels e  A = comprising a n Outstanding majority of end- t impacts user stakeholders a are able to assess r research impact. y  B = Very considerable  Panels should impacts „. include an appropriate  C = discipline mix Considerable covering the impacts breadth of research impacts being considered.  D = Recognised but modest impacts

 E = Of Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e limited reach or significance.

 Not classified – the impact was not underpinned by research or the link between the research and the claimed impact has not been demonstrated to the Panel’s satisfaction.

Hanney et al NHS Health Technology M Research and policy and practice impacts Methods: -  NCCHTA Program (2007) [27] Assessment i  National has had Programme x Coordinating considerable e Centre for Health impact in terms of d Technology knowledge Assessment generation and m (NCCHTA) perceived impact e documentation. on policy and to t some extent on h  Surveys of practice. o principal d investigators.  This high impact s may have resulted d Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e e  Detailed case partly from the s studies. NCCHTA i Programme’s g objectives, in that n topics tend to be of relevance to the NHS and have policy customers.

Higher High education funding in M Reach and significance of the impact or Methods: Research Excellence  It is possible to Education England i benefit  Case study Framework Funding x submissions (REF) 2011 assess the impact Council for e (include citations, of research through England d documentary an approach based (2011) [28] evidence) on expert review of m case studies. e  Panel t assessments  UK funding bodies h have decided that o the REF will d assess the ‘impact’ s arising from research, alongside the ‘outputs’ and ‘environment’ elements Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e  The framework provides recognition of economic and social benefits of excellent research

Kalucy et al Primary Care Research M  Advancing Methods: Payback Model  It is feasible to use (2009)[29] i  Telephone the Payback x  Knowledge interviews with framework to e research teams determine the d  Building Capacity and nominated proximal impacts of m users of research, primary health care  Informing Decision- e research. t  bibliometric  Making; h methods  Though resource o intensive, d  Health; and  documentary s telephone .  Broad Socio-Economic. evidence of interviews of chief impact where investigators and possible nominated users provided rich information.

Kuruvilla et al Health research L Framework domains: Methods: Research Impact  The framework (2006) [30] it i. research-related impacts;  Literature review. Framework provides prompts e and descriptive Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e r ii. policy impacts;  Semi-structured categories that a interviews with can help t researchers to iii. service impacts; principal u investigators to systematically r develop identify a range of health and intersectoral; & e iv. narratives. specific and verifiable impacts r v. societal impacts.  Case studies of related to their e research impact work (compared to v which were used ad hoc i to develop the approaches they e framework. had previously w used). Indicators: a  Type of  The standardized n problem/knowledge. structure of the d framework facilitates i  Research methods used. n comparison of t research impacts e  Publications across projects r and papers. and time, which is v useful from i  Products, analytical, e patents and management and w translatability assessment s potential. perspectives. w it Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e h  Research networks. r e  Leadership s and awards. e a  Research r management. c h e  Communicati r on. s  Level of policy-making.

 Type of policy.

 Nature of policy impact.

 Policy networks.

 Political capital.

 Type of Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e services: health/ intersectoral.

 Evidence- based practice.

 Quality of care.

 Information systems.

 Services management.

 Cost- containment & cost- effectiveness.

 Knowledge, attitudes & behavior.

 Health literacy.

 Health Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e status.

 Equity and human rights.

 Macroecono mic/related to the economy.

 Social capital & empowerment.

 Culture & art.

 Sustainable outcomes.

Kuruvilla et al Health services research I As above Methods: Research Impact  Researchers (2006) [8] m  impact Framework were relatively p narrative of easily and a research projects methodically able to c involving: identify and present t impacts of their n  semi- work. a structured r interviews  Researchers’ r with principal narratives contained a verifiable evidence ti Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e v investigators and highlighted a e wide range of areas  documentary in which health ( analysis of services and policy m the projects. research has i impact. x e Indicators:  Factors d As above thought to positively influence the impact m of research included e researchers’ t involvement in h research and policy o networks, d established track s records in the field, ) and the ability to identify and use key influencing events, such as ‘policy windows’.

Kwan et al Hong Kong Health and Health M Payback domains: Methods: Payback Model  Multivariate (2007) [31] Services Research Fund i i. knowledge production;  Surveys with analysis found (HHSRF) x principal participation of e research utilization; investigators. d ii. investigators in policy committees  Bibliometric Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e m iii. informing policy and decision making; analysis. and liaison with e potential users t iv. application of the findings through were significantly h Indicators: changed behavior; and associated with o  Publications per reported health d project. s v. health and health service benefit. service benefit.  Peer reviewed publications per  Payback outcomes project were positively associated with the  Journal impact amount of funding factor. awarded.

 Journal ranking.  Further studies are needed to better  Citations per year. understand the factors and  Led to pathways participation in associated with the health-related translation of policy/advisory research findings committees post into practice. research.

 Completion.

 Pre- and during- Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e research process liaison with potential users.

 Generated subsequent research.

 Led to qualifications.

 Led to career advancement

 Findings used in policy making.

 Findings expected to be used in policy making.

 Led to changes in behavior.

 Expected to lead to changes in behavior. Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e  Reported health service benefit.

 Expected future health service benefit.

Landry et al Research C Ladder elements: - Research  The results (2001) [32] o i. transmission (of research results to utilization suggest that there m practitioners and policy makers); ladder are barriers to m climbing the e ii. cognition (reading and understanding); research utilisation n ladder and that t these barriers are reference (quoting of research results a iii. primarily located r in reports, studies, actions); between the stage y of no transmission iv. effort (to adopt research results); and the stage of transmission. v. influence (on choices and decisions); and

vi. Application.

Lavis et al Health research C Model elements: Methods and Decision making  Research (2003) [33] o i. policy makers are the ones seeking Indicators: Impact model knowledge may be m research (user-pull);  Process measures used in m instrumental, e Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e n ii. researchers actively disseminating (if limited conceptual or t results (producer pull); and symbolic ways. a resources are r available). researchers and policy-makers are  The proposed y iii. both involved actively (exchange assessment tool as  Intermediate measures). research can assist outcome in better impact measures (by assessment and performing accountability in surveys). the health sector.

 Outcome measures (by

performing cases studies).

Laws et al Population health surveys M Domains: Methods: Banzi Health  Factors influencing (2013) [34] i i. advancing knowledge;  Semi-structured Research Impact use of the findings: x Model interviews with the the perceived e capacity building; chief investigators d ii. credibility of survey and findings, m iii. informing decision-making; dissemination e  end-users strategies used; t iv. health and socioeconomic benefits and contextual h  Bibliometric factors. o analysis and d Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e s  Verification using  Highlighted the d documentary importance of e evidence. engaging end- s i users from the g inception of survey n programs and utilizing existing policy networks and structures, and using a range of strategies to disseminate the findings that go beyond peer review publications.

Liebow et al National Institute of M Logic models elements: Methods: Research Impact  This logic model (2009)[35] Environmental Health i i. Inputs;  A logic model Logic Model approach to Sciences (NIEHS) Extramural x tailored to inputs, research impact Asthma e ii. Activities; and outputs, and assessment Research Program d outcomes of the demonstrates that NIEHS asthma it is possible to Outputs and outcomes (Immediate, m iii. portfolio. conceptualize e intermediate, ultimate). program t  Data from existing components, mine h National Institutes existing Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e o of Health (NIH) databases, and d databases. begin to show s longer-term d  Bibliometric data. impacts of e program results. s  Structured i elicitation of  The next g expert judgment. challenges will be n to modify current Indicators: data structures,  Input indicators improve the (NIH budget data, linkages among NIH asthma- relevant related budget databases, data, Non-NIH incorporate as agency budget much data). electronically available data as  Activity indicators possible, and (Grant awards, determine how to institutions, PIs). improve the quality and health impact of the science that Output indicators  we support. (Publications, Curricula, interventions, and outreach materials). Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e  Outcome indicators (Citation database, Commissions, task forces, advisory panels, work groups, patents, drugs, legislation, Guidelines and care standards, Environmental changes related air quality, Asthma mortality, emergency department utilization, Hospitalization, Rescue medicine use, Quality of life indicators).

Milat et al Health Promotion Applied M Domains: Methods and Banzi Health  Intervention (2013)[36] Research i v. advancing knowledge; indicators: Research Impact research projects x Bibliometric & Model can achieve the e vi. capacity building; citation analysis. greatest policy and d practice impacts if Desk analysis. they address informing decision-making;  m vii. proximal needs of Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e e viii. health and socioeconomic Database review. the policy context t benefits by engaging end- h Interviews with users from the o Investigators and inception of d End users of projects and s research utilizing existing d policy networks e Case studies. and structures, and s using a range of i strategies to g Assessment panel disseminate n that engaged both findings that go research and end beyond traditional users.. peer review publications

National Biomedical Research E Research impact evaluated in monetary Indicators: Cost benefit analysis  Cost benefit Institutes of c terms:  QALY analysis is a Health (1993) o i. savings for health care systems (direct feasible method to [37] n costs)  Profits assess research o impact. m ii. savings for the community on the whole i c (indirect costs) e v a l u Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e a ti o n Ovseiko et al Academic clinical M Impact domains: Methods: Research Excellence  The pilot exercise (2012) [38] medicine i i. delivering highly skilled people;  Existing Framework has confirmed that x administrative the majority of the e creating new businesses, improving sources. proposed indicators d ii. the performance of existing have some validity,  Online there are significant m businesses, or commercializing new products or processes; administrative challenges in e survey carried out operationalizing and t measuring these iii. attracting R&D investment from global by the university’s h Medical Sciences indicators reliably, business; o Division among as well as in d n=289 clinical comparing evidence s iv. better informed public policy-making or medicine faculty of research impact d improved public services; members. across different e cases in a s v. improved patient care or health Indicators: standardized i outcomes;  Staff movement manner g between n vi. progress towards sustainable academia and development, including environmental industry. sustainability;  Employment of vii. cultural enrichment, including improved post-doctoral public engagement with science and researchers in research; Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e viii. improved social welfare, social industry. cohesion or national security; and  Research ix. other quality of life benefits. contracts and income from industry.

 Collaborative research with industry measured through

co-authored outputs.

 Income from intellectual property.

 Success measures for spin-out companies.

 Patents granted/licences awarded and brought to market.

 Research income Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e from overseas business.

 Changes to legislation/ regulations/ government policy.

 Participation on public policy advisory committees.

 Influence on public policy debate.

 Research income from the NHS and medical research charities.

 Measures of improved health services.

 Changes to clinical or healthcare training, practice or Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e guidelines.

 Development of new or improved drugs, treatments or other medical interventions; numbers of advanced phase clinical trials.

 Changes to public behavior.

 Measures of improved health outcomes.

 Increased levels of public engagement with science and research.

 Measures of improved social equity, inclusion or cohesion. Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e  Application of new security technologies or practices.

Schapper et Murdoch Childrens Research M i. Knowledge creation; Methods: Research  The framework al (2012) [39] Institute, Australia i  Bibliometric Performance provides a fair and x Evaluation ii. Inputs to research; and analysis. transparent means e of disbursing d  Document internal funding. It iii. Commercial, clinical and public health analysis. is also a powerful m outcomes. tool for evaluating e  Performance the Institute’s t evaluation progress towards h committee made achieving its o up of peers. strategic goals, d and is therefore a s Indicators: key driver for  Publication activity research – number of peer excellence. reviewed articles per evaluation year.

 Publication impact – highly cited papers from the preceding 5-year Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e period.

 Technical papers that assist the translation of research into practice, e.g. policy, guidelines, books, book chapters.

 Competitive peer- reviewed funding weighted by associated infrastructure received.

 Research students trained.

 Commercialization activity: contract funding gained through contracted research Commercialization activity: patents filed Research Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e outcomes, adoption, implementation and evaluation.

Spoth et al Family-Focused Prevention L Translation Impact factors: Indicators: PROSPER Model  The model for (2011) [40] Science it i. effectiveness;  Analysis of long- community– e term effects. university r ii. extensiveness; partnerships has a  Effects across the potential to t facilitate the iii. efficiency; and subgroups u targeted by dissemination and r particular public health e iv. engagement of public health impact. universal impact of universal interventions. interventions to r prevent underage e  Analysis of key drinking and other v mechanisms of problem behaviors. i effects. e  This model fits well w within a comprehensive a strategic n framework for d promoting effective prevention. c o m Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e m e n t a r y Sullivan et al United Kingdom cancer C Research impacts Methods: -  UK cancer centres (2011) [41] centres (UKCC) it  Publication and focused on either a citation analysis. basic cancer ti biology o Indicators: n  Research level or highly clinical (basic or clinical). trials received the a highest aggregate n  Potential and citations. a actual citation l impact. y s i  International s collaborations.

Taylor & Nursing research C Impact mapping across: Indicators: Research impact  The international Bradbury- o i. Inputs;  Inputs (people, mapping principles of social Jones (2011) m time and impact assessment [42] m ii. Activities; equipment). provide a e meaningful n  Activities (actions framework in which Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e t iii. Outputs; that constitute the to describe nursing a project). research impact. r iv. Outcomes; and y  Outputs are the direct results of v. Impacts. the research (publications or dissemination to stakeholders).

 Outcomes (behavior change or improved health outcomes).

 Impacts (the outcomes less what would have happened anyway).

Warner & Tobacco control research S Policy impact Methods: -  Policy research Tam (2012) u  Surveys of goals established [43] r experts. in 1992 have been v largely realised. e  Case studies. y  For select tobacco control policies, research has made Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e truly important contributions to saving lives.

Weiss (2007) Medical research C Analyzes the ratio between input Indicators:: Weiss Logic Model  Change from an [9] o (resources), process (activity) and results  Publications. output-based to an m of research (products) outcomes-based m Domains of impact include:  Awareness of perspective is a e i. initial benefits; medical research. critical step in n examining the role t intermediate benefits; and  Results in policy that clinical a ii. making. scientists can play r in decreasing the y iii. long-term benefits.  Changes in well documented practice. efficacy- effectiveness gap.  Changes in well- Without this shift, being and health.  there will be little incentive for researchers to look beyond the outputs of their work.

Wooding et al Arthritis research campaign M Payback domains: Methods: Payback model  There was a (2004) [44] (arc) funded research in the i i. Knowledge production  Document review considerable range UK x of research e  Publication and d ii. Research targeting and capacity paybacks and Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e building citation analysis these would not m have been e iii. Informing policy and product  Key informant identified without t interviews employing h development o (Investigators and structured case d iv. Health and health sector benefits end users) study approach. s v. Wider economic benefit  Case study  When translation of analysis research into developments of practical value to Indicators: patients occurs it is  Number of largely due to the publications conviction, effort and personal  Citations networks of a particular  Proportion of investigator, and is citations not associated with internationally the type or mode of the funding stream  Number of or the bibliometric partnerships impact of the investigator  Research funding  payback framework Reference Type of research assessed S Domains of impact Methods and Frameworks applied Key lessons learned t indicators or proposed u d y

t y p e  Research capacity could be building operationalized and embedded.  Generated subsequent  There was as no research. quantifiable evidence of  Led to qualifications. broader economic returns arising from  Led to career the arc-sponsored advancement research.

 Findings used policy and product development.

 Reported health service benefit

 Economic benefits