Workshop reports

Data science and AI in the age of COVID-19

Reflections on the response of the UK’s data science and AI community to the COVID-19 pandemic Below are the individual reports from the research. Another example of collaboration return to a context in which advances achieved workshops that were convened by The Alan between organisations was the development of during the pandemic would not be possible Turing Institute and the Centre for Facilitation adaptive clinical trials (e.g. RECOVERY8). once the COPI regulations come to an end, in November and December 2020, following despite evidence indicating public support for Challenges the Turing’s ‘AI and data science in the age of the use of health data in research.12 COVID-19’ conference. The workshops were The ability of the data science community to Workshop participants also reported that data summarised by the facilitators and theme respond more successfully to the challenge of sources could benefit from being better linked leads, and the editorial team of the primary the pandemic was hindered by several hurdles. to each other – despite more collaboration, report then applied light editing. These reports many UKRI-funded studies (ISARIC4C,13 are a reflection of the views expressed by Better standardisation and documentation PHOSP-COVID14) are still relatively standalone. workshop participants, and do not necessarily of clinical data (and other forms of data, e.g. Connecting expertise – the right people to reflect the views of The Alan Turing Institute. location) would have been beneficial. Some positive examples of standardisation could the right data to the right problem – was Several of the reports end with references have been replicated for far wider benefit. identified by workshop participants as a to extra case studies that were submitted For example, workshop participants noted primary bottleneck in working on COVID-19’s by participants using an online form. Further ISARIC as a leading example of effectively pathogenesis and virus evolution, and vaccines details of these case studies can be found at documenting existing data and providing clarity and clinical trials. This impacted the ability the end of this document. on the meaning of each data field, and the of the data science community to respond Phenomics9 project as a good example of how better, from sharing expertise and learning Part A: Public health, modelling and pharmaceutical standardised code lists could be expanded. on a specific problem to avoid duplication, to Standardisation and codification of metadata including data asset holders, subject matter interventions would have made data more discoverable and experts, researchers and clinicians in analysis interoperable. to explain missing data or bias. Theme 1: Pathogenesis and virus evolution, for genomic surveillance as new strains of Connected to the lack of sufficient Shortcomings in the availability and detail vaccines and clinical trials (leads: Marion COVID-19 emerge. standardisation, data pipelines could also of data inhibited the inclusion of patient Mafham, Jim Weatherall) usefully have been more readily accessible and subgroups in trials to ensure there was an The data science community repurposed and linked. A good example of such a ‘standardised’ appropriately diverse range of people involved. Scope built on existing systems and databases (e.g. pipeline achieved through collaboration is The lack of diversity amongst researchers ISARIC,3 ICNARC4). This adaptation lowered The role of data science in bolstering the OpenSAFELY,10 which provides controlled carrying out work in this area was also raised costs and enabled researchers-at-large to act efforts of clinical scientists to analyse data access to a vast database of patient records by some in the workshop as a constraint on more quickly. For example, ICNARC5 processed at increasing scale and speed produced to support COVID-19 data analytics across the engaging vulnerable and underserved groups and published extensive weekly summaries several potentially useful insights during UK healthcare population. However, this is only in research. drawing on an existing audit that covered over the COVID-19 pandemic, from tracing the suitable for research studies where identifiable 95% of intensive care units in the UK. Similarly, International context phylogeny of different strains of the virus to data are not required. linked data from GPs and Trusts were available map its evolution, to predicting the structure of The global nature of the pandemic led to to researchers through the Discovery East understudied SARS-CoV-2 viral proteins. Further, workshop participants reported that numerous examples of collaboration and London6 platform that existed before COVID-19. greater access to healthcare data would have openness in which the UK’s data science Dashboards from this detailed local-level data The UK response been beneficial. For example, much data community contributed and benefitted, fed into London and national daily situation sharing during the pandemic was undertaken and which has led to further innovation in The availability of a central data repository reports. using the COPI (Control of Patient Information)11 researching the pathogenesis and evolution of through the COG-UK1 consortium generated notices as the legal basis. However, these Many members of the data science community the virus. For example, the Protein Data Bank15 whole genome sequence data (>100k notices were only ever envisioned to be a stop- collaborated and were ready to work outside enabled the sharing of data around protein sequences, November 2020), and was gap measure. Consequently, consideration areas of individual expertise to address the structures and protein interaction. Moreover, achieved through the coordination of might be given to how the best aspects of COPI challenges of discovery and therapeutics, COG-UK data and GISAID16 data were also numerous organisations (NHS, public health might be retained, whilst ensuring that the which helped speed research across institutes, brought together in new genomic surveillance agencies, Wellcome Sanger2 and academic permissiveness does not undermine individual institutions). Having those sequences available disciplines, regions and demographics. One pipelines.17 1rights and protections. Otherwise, we may from the earliest stages, and the central example of collaboration, combined with coordination and prioritisation of platform innovation and an open approach to data trials, was crucial in pushing forward the sharing, was the RAMP7 initiative that allowed development of vaccines, and will be vital people to contribute to ongoing work and 8 https://www.recoverytrial.net 1 9 http://covid19-phenomics.org 1 COVID-19 Genomics UK (COG-UK) https://www.cogconsortium.uk 10 https://opensafely.org 2 https://www.sanger.ac.uk 11 https://digital.nhs.uk/coronavirus/coronavirus-covid-19-response-information-governance-hub/control-of-patient-in- 3 International Severe Acute Respiratory and emerging Infection Consortium (ISARIC) https://isaric.tghn.org formation-copi-notice 4 Intensive Care National Audit and Research Centre (ICNARC) https://www.icnarc.org 12 https://understandingpatientdata.org.uk 5 ICNARC COVID-19 report: https://www.icnarc.org/Our-Audit/Audits/Cmp/Reports 13 Coronavirus Clinical Characterisation Consortium (ISARIC4C): https://isaric4c.net 6 Discovery East London: https://www.eastlondonhcp.nhs.uk/ourplans/discovery-east-london-case-study.htm 14 Post-hospitalisation COVID-19 study: https://www.phosp.org 7 Rapid Assistance in Modelling the Pandemic (RAMP): https://royalsociety.org/topics-policy/Health%20and%20well- 15 http://www.wwpdb.org being/ramp 16 https://www.gisaid.org 17 SARS-CoV-2 lineages platform: https://cov-lineages.org 2 3

Suggestions and governance frameworks to enable Theme 2: Epidemiological modelling and initiative modelling the entire potential the appropriate and necessary collection, outbreak (peak, second wave, etc.) at the start. – Improve availability of, and linkages prediction (leads: Spiros Denaxas, Deepak subgrouping and stratifying of relevant data. Despite limited data, much early analysis has between, datasets: It is vital to increase Parashar) Better monitoring of demographics (age, been reasonably accurate in predicting the the availability of datasets in appropriate sex, ethnicity of enrolled patients to vaccine Scope burden of infection on clinically vulnerable research environments to make it more and clinical trials compared to local eligible populations, or estimating age-specific risks straightforward to obtain and analyse in a From describing transmission dynamics, to population) will highlight critical gaps. of COVID hospitalisation and death.27 Novel much more connected way. The provenance identifying risk factors for infection and adverse approaches in tackling existing problems have of those data, their purpose, reliability, – Organise cross-agency ‘war game’ outcomes, and modelling control strategies, been tried (e.g. triaging patients based on their bias and what was left out, are also key simulation to connect expertise and identify arguably epidemiology and data modelling condition rather than scoring has provided to helping the data science community critical data / ICT gaps in preparedness for have been at the core of the data science more granularity and discretion) and new ways work effectively to better understand the similar pandemics. community’s contribution to the COVID-19 of modelling and their application have been COVID-19 pathogenesis and virus evolution pandemic response. – Recommend broader and more rapid developed. and support the development of vaccines access to data. The UK response and clinical trials. However, given the limited Data (other than for health settings) were – Coordinate with universities and usability of anonymous data, as part of this The pandemic has accelerated academic collected from various sources, such as industrial partners to identify how to use initiative the community must also address research, with researchers focusing on community surveillance schemes28 and from data science to make a case report. the issues of secure controlled access. All of providing rapid, accurate and actionable the public providing information via apps these may be achieved by greater support – Encourage more cross-domain data insights into the COVID-19 public health (resulting in the COVID Symptom Study29). Data for work being undertaken by Health Data linkage and modelling, to explore what emergency. It has reshaped clinical academic have also been collated from e.g. Facebook Research UK (HDR UK)18 and the UK Health can be achieved at the intersection of these research with rapid restricting and prioritisation Data for Good30, Google mobility reports which Data Research Alliance,19 who have an different datasets. taking place to accommodate the needed were used in calibrating models in terms established role in connecting datasets and Useful resource capacity on the NHS frontlines,22 with different of a proxy for social contacts,31 and social researchers and enabling healthcare data to streams of work delivered in parallel and quick mixing surveys.32 All of these assets played an be discoverable. Software: Open Targets COVID-19 Target posting on public repositories of analysis important role in forecasting the course of the – Increase preparedness through local Prioritisation Tool.21 protocol and code.23 pandemic. resilience: The experience of working with Collaboration increased between different Challenges data at the local level across the UK was groups, as well as sharing of data and models. Problems in data access, availability, granularity inconsistent, with some areas having far Examples include the collaboration between and levels (micro vs. macro) posed important greater access to data and success working Bristol University and NHS planners to help challenges to workshop participants. Often, with it than others. At the local level, data project hospitalisation demand and bed work was initiated only for the team to be needs to be available, and expertise and occupancy,24 and the approach used by unable to progress due to a lack of data or a resources easier to find and deploy, so that SPI-M – drawing on the collective expertise of delay in accessing the necessary data, such local organisations have the autonomy to several infectious disease dynamics experts as with research concerning the long COVID work with their data (as opposed to it being across multiple groups25 – to deliver consensus phenotype. Moreover, the ISARIC study33 has centrally held and decisions being made guidance based on a comparison of outputs been published but the underlying data have centrally). from multiple models. not been made widely available to researchers – Create an open data and clinical problem requesting access. In other cases, delays were democratisation platform: Following the Mathematical models of formal logic were encountered with accessing datasets when example of Kaggle,20 create a data-sharing used to capture prioritisation policy based the purpose was data linkage across multiple and modelling platform that connects the on epidemiological and clinical choices with sources in order to create longitudinal patient right people to the right data for the right outcomes that are auditable and dependable. cohorts. problem, ensuring it is integrated with For example, the RAMP26 and INI initiatives links to existing data depositories while demonstrated the “step up” of the modelling preserving privacy and security of the data. community, with the Royal Society RAMP – Increase diversity and inclusion through 1 1 better data analysis and communication of outcomes: To monitor the efficacy of 22 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0237298 drugs and vaccines for subpopulations, 23 https://cmmid.github.io/topics/covid19 24 https://doi.org/10.1101/2020.06.10.20084715 the data science community needs access 25 https://www.gov.uk/government/groups/scientific-pandemic-influenza-subgroup-on-modelling 26 https://royalsociety.org/topics-policy/Health%20and%20wellbeing/ramp 27 https://eprint.ncl.ac.uk/270736 28 https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulle- tins/coronaviruscovid19infectionsurveypilot/4december2020 REACT: https://www.imperial.ac.uk/medicine/re- search-and-impact/groups/react-study/ 29 https://covid.joinzoe.com 18 https://www.hdruk.ac.uk 30 https://www.medrxiv.org/content/10.1101/2020.10.26.20219550v1 19 https://ukhealthdata.org 31 https://www.google.com/covid19/mobility 20 https://www.kaggle.com 32 https://cmmid.github.io/topics/covid19/comix-reports.html 21 https://covid19.opentargets.org 33 https://www.bmj.com/content/369/bmj.m1985 4 5 Another challenge was communication to Inclusion and equality: Prioritise the the public and policy makers: the media exploration and understanding of the impact of Part B: Non-pharmaceutical interventions misunderstood some model results (e.g. the COVID-19 on different ethnic and social groups Oxford study at the start of the pandemic) and make sure that these insights are used in Theme 3: Testing, contact tracing and other requirements: what data existed, the quality and some misleading statements received conjunction with other key risk factors (age, public safety interventions (lead: Mark of the data, and the help that was needed. much media attention.34 The advantages care home residency, healthcare professional Briers) and limitations of modelling could have been worker status and other health exposures). This – Communicating about how scientific data disseminated to the public alongside the would help promote inclusion of underserved Scope and models are informing policies, including results, in order to enhance trust and enable groups by researchers designing studies and success measures, evaluation frameworks, understanding by a wider audience. by funders, reviewers evaluating focus areas, “Test and trace” is a well-established mantra and the lessons learnt from pilots to support and teams delivering modelling projects. of pandemic management that involves ongoing research. Suggestions Further, increase the visibility of female and identifying, assessing, and managing people – Communicating about how data science BAME scientists in the media.36 who have been exposed to a disease, and These fell into four areas: preventing onward transmission. However, can promote positive health outcomes. This could have helped to achieve the objective Communication is key to ensure that the Methodology/collaboration: There were understanding how to use modern technology of trust in data science (and models that public, policy makers, health leaders and significant amounts of research waste and data science to facilitate this process has underpin the analysis), which would have politicians understand and appreciate the data identified in the COVID-19 literature, specifically not been attempted at this scale before (i.e. increased the acceptance of policies and science community’s information, benefits where it concerned application of prediction prior to the COVID-19 pandemic). led to a greater engagement. One example and recommendations. Workshop participants modelling.37 Clarify the capabilities of modelling The UK response relates to public adherence of isolation suggested that the Turing could better engage and ensure access to the most reliable and notifications through test and trace; a the community at different levels via its network accurate data by having a Turing panel for Multidisciplinary collaborations were quickly targeting of the communication may have of partner universities, and provide targeted consultation on research, to save effort and established to deliver impactful scientific helped improve compliance.45 “training” for leaders and policy makers. time. contributions in a relatively short timescale. – Encouraging the public to engage with Workshop participants suggested that the Exemplar publications were shared, helping to The data science community should consider the contact tracing app.46 Concerns Turing provides something similar to the shape and inform policy.39 the information recipients need as “customers” about contact tracing focused heavily on Research Design Service,38 which provides when communicating with them, and use clear, This collaborative process has improved technology and law and made it difficult to researchers with free advice on protocols for transparent and reproducible communications. modelling, data sharing, and interfacing gain the public’s commitment. If the public clinical trials (funded by NIHR), and that the Notably, there are pre-existing mechanisms to policy makers,40 and has helped raise had engaged with the app, it may have Turing establishes a “Turing Health Research to support this type of communication, for awareness about technology and data helped to provide a clearer understanding Unit” at its partner universities for a one-stop example via the Science Media Centre,35 science for contact tracing. The data science about which venues and businesses needed shop for advice on data access, methodology, which has made significant efforts to curate community has developed methods to respond to be closed to prevent the spread of the and communicating to the public. This final expert reactions to stories that are making the quickly to accelerate innovation, in an open virus, rather than utilising national or city- point of engaging the public on research headlines in an effort to provide the necessary and transparent manner, yet also allowing full wide lockdowns. nuance and insight into the results. methods can increase public understanding formal peer review (for example, the DELVE and research credibility, thereby increasing working group).41 While data sharing has improved, more can Data are the cornerstone of effective public trust. be done towards data availability and visibility. epidemiological modelling. It is important to New data science related initiatives have been Some hospitalisation data were not available in plan data sharing early as an essential part of Extra case studies: 1, 5 implemented, both as national programmes the early stages of the pandemic. The lack of a any prospective data collection study. Effective such as the COVID Symptom Study,42 and central repository for data made it challenging future planning could be supported by: localised programmes, for example the to establish what data exist, and this presented Southampton COVID-19 Testing Programme43 difficulties in rapidly connecting and using data. – Having a directory of data sources together and the Norwich Testing Initiative for with guidelines for obtaining data access. universities.44 Alongside data availability is the need to have – Developing guidelines for establishing and clean data that preserve privacy and are ready Challenges enabling access to generic ‘data lakes’, to be analysed. Higher quality, spatial-temporal data could have assisted the support of policy containing anonymised data. Workshop participants highlighted that at both a local and national level, specifically 1 1communication to the public could have been – Utilising expertise within the Turing to with the testing initiatives. Feedback loops improved along several dimensions: provide advice and guidance on the ethical around interventions could have been and responsible use of data. – Communicating statistical findings and data advanced with improved quality data. – Having a platform for data sharing and modelling for deeper modelling of COVID-19 39 https://www.medrxiv.org/content/10.1101/2020.07.13.20152439v1 ; https://www.medrxiv.org/content/10.1101/ data. 2020.05.14.20101808v2 ; https://www.ucl.ac.uk/health-informatics/groups/public-health-data-science/research/vi- rus-watch 40 https://royalsociety.org/topics-policy/Health%20and%20wellbeing/ramp ; https://www.thelancet.com/journals/ 34 https://www.medrxiv.org/content/10.1101/2020.03.24.20042291v1.full-text lanchi/article/PIIS2352-4642(20)30250-9/fulltext 35 https://www.sciencemediacentre.org 41 https://royalsociety.org/news/2020/04/royal-society-convenes-data-analytics-group-to-tackle-covid-19 36 https://www.timeshighereducation.com/blog/women-science-are-battling-both-covid-19-and-patriarchy 42 https://covid.joinzoe.com/ 37 https://www.bmj.com/content/369/bmj.m1328 43 https://www.southampton.gov.uk/coronavirus-covid19/covid-testing/testing-programme 38 https://www.nihr.ac.uk/explore-nihr/support/research-design-service.htm 44 https://www.earlham.ac.uk/norwich-testing-initiative-covid19-testing-resources-universities 45 https://www.medrxiv.org/content/10.1101/2020.09.15.20191957v1 6 7 46 https://github.com/carstenmaple/SpeakForYourself

Participants suggested that the focus was these collaborations have brought. Data Theme 4: Behavioural analysis and policy periods to contain the spread of COVID-19 too much on “today’s” problem instead of scientists could take a more proactive role interventions (leads: Tao Cheng, Ed Manley) and prevent healthcare demand exceeding proactively considering what the next likely in communication to counter misinformation availability, while also underlining that other problem on the horizon might be. Two areas about testing, contact tracing and other Scope non-pharmaceutical interventions (testing and were highlighted. First was the predictable risk public safety interventions. tracing, mask wearing, physical distancing, While ‘Report 9’48 may have brought the idea of of mass student migration to universities at shielding, and self-isolation) on their own are – Focus on forward thinking: There needs policy interventions for managing a pandemic the start of autumn term 2020, and the need to insufficient.52 to be more emphasis on foresight so to the masses, the importance of non- have a robust process for testing and contact that the next problem can be identified pharmacological/behavioural interventions in tracing in place to deal with the consequent Workshop participants argued that there and addressed in a timely manner (e.g. this setting is well established. This highlights outbreaks. Second was the issue of how was a public appetite for clear explanations considering vaccine administration the role of data science in evaluating the knowledge of immunity or vaccine status and visualisations of data. Some data while vaccines are being developed, and effectiveness of policy interventions, such as might be operationalised to support reduction journalists (in the UK and abroad) successfully determining how data can be collected and lockdowns, distancing, masks, and fines for in non-pharmaceutical interventions (e.g. the communicated complex analysis and made available as vaccines are rolled out violating quarantine rules. It is vital that we exploration of ethical and non-discriminatory simulations to build public understanding to ensure the research opportunities are understand what insights data science has immunity passports). of the pandemic, and the impact of specific maximised). This also includes evaluation of contributed to the effects of interventions interventions to ‘flatten the curve'. Harry Workshop participants argued for greater previous policies, so we can improve testing, on behaviour, and how research on human Stevens’s article in The Washington Post use of better, effective and robustly designed contact tracing and other public safety behaviour has been captured in pandemic featuring a ‘corona simulator’ is now the apps and wearable technology for pre- interventions. modelling, to identify what was done well, newspaper’s most-read online story.53 symptomatic detection that can be used – Faster framework: A robust research/ and to highlight opportunities for the near and Moreover, John Burn-Murdoch’s data by the public, such as fitness devices and analysis/review framework is needed that distant future. visualisations helped explain ‘excess fatality smartphones. For example, the existing app rates,’ and illustrated how COVID-19 could can be deployed in times of urgency so The UK response previously used for asthma patients to detect that delays in publishing ideas are reduced, not be reasonably likened to the flu. Further, worsening asthma conditions could have been and trust from the public can be increased. A major breakthrough for the data science regular and prolonged public engagement repurposed to detect COVID-19 by listening An example is an evaluation plan with community’s response was the opening by prominent members of the data science to the sound of coughing (even if app use objectives and assessment measures which up to data researchers of various mobility community, such as Professor Devi Sridhar may have been imperfect). Other technology can be used for public engagement with datasets by commercial providers (e.g. Cuebiq), and Professor Christina Pagel, helped improve improvements would also have supported the contact tracing. alongside Apple and Facebook releases and public understanding of the pandemic and data science community, such as development the Google Mobility Report data. This enabled policy interventions needed. – Local approaches: A localised approach of environmental (faecal/waste-based) tests the detailed spatial analysis and large-scale to initiatives, such as contact tracing, may Challenges across the sewage network, or air pollutant- simulation that was key to generating evidence have increased levels of public trust and based tests in public areas. for government and the public on the potential engagement. The capacity of those working in behavioural and actual impacts of lockdowns. Workshop International context analysis to contribute more to the response – Technology: Workshop participants stated participants cited public transport data from to the pandemic was impacted by numerous Workshop participants believed there was that testing and tracing technology needs Transport for London and the Department for factors. notable divergence of approaches in the early to be improved, with greater development Transport as central to arguments about the stages of the pandemic, whereas a more of detection at places where the public effectiveness of the first lockdown. The ability The most significant of these was a lack effective response might have sought to gather, rather than solely personalised to monitor data in real time through dashboards of transparency in terms of the use of data collaborate with the international community detection. Two examples were highlighted: (e.g. i-sense COVID RED,49 Evergreen50 and the and studies in policy decision-making, data who have had greater experience of dealing development of air quality testing systems at GOV.UK dashboard51) was also significant in the provision, and data collection methods. Many with epidemics. supermarkets and hospitals, and conversion data science community’s capacity to respond workshop participants observed that the lack of of wearable based pre-symptomatic and report trends to the public at speed. transparency in policy decision-making made Suggestions detection algorithms (e.g. through short it difficult to know which research studies ECG measurements with disposable pads or Behavioural modelling, especially in relation to had ‘cut through’ or even been considered Improved data sharing is necessary for a test self-cleaning optical devices) into terminals mobility, was the standout area for success (in by government and expert advisory groups and trace system to be effective. Collaboration placed at entrances to public areas, that are terms of practical application of behavioural when deciding on policy interventions. Policy across disciplines should be facilitated. available for all to use. Notably, there was an analysis) in impacting policy intervention, makers were also not transparent about 1 1enabling data scientists to advise and compare – Establish ‘data lakes’: Workshop appreciation that these tools have non-trivial which data were important: it was known that the potential consequences of different participants felt that cleaned, anonymised ethical implications which would need to be Google and Apple data were central to policy- interventions (e.g. school closures, hospitality data ready for analysis would have been addressed as well. making, but the detail and quality of those data restrictions) for decision makers and the desirable, particularly in secondary care were unknown, while other data science (e.g. public to consider. Such analysis was key in which currently is messy and difficult to Faculty54) was not independently scrutinised. Useful resource emphasising the importance of lockdown access. Leonelli, S. (2021). Data Science in Times of – Maintain and further develop Pan(dem)ic . Harvard Data Science Review.47 48 https://www.imperial.ac.uk/mrc-global-infectious-disease-analysis/covid-19/report-9-impact-of-npis-on-covid-19 collaborations: There are opportunities 49 https://covid.i-sense.org.uk Extra case studies: 2, 3, 5 50 https://www.evergreen-life.co.uk/covid-19-heat-map to further leverage the benefits that 51 https://coronavirus.data.gov.uk 52 https://doi.org/10.1016/S2468-2667(20)30133-X 53 https://www.washingtonpost.com/graphics/2020/world/corona-simulator 47 https://hdsr.mitpress.mit.edu/pub/fi1rol2i/release/2 54 https://faculty.ai 8 9

There was a lack of transparency of apps used are failing to gain an understanding of the Suggestions – Build strong connections with companies for monitoring – for example, the accuracy medium- and long-term consequences of non- to facilitate rapid data sharing in fine spatial and effectiveness of the Test and Trace app pharmaceutical interventions and wider social – Develop more transparent models for and temporal granularity to enable local remains unknown. This lack of transparency, behavioural changes. predictions: Provide mechanisms and analysis for local policy-making. combined with an array of different approaches techniques to give accessible and accurate – Support provision and training in more by the various organisations involved in data The centrality of mobility, mobile device and interpretations to model predictions, so that data-intensive communication: To collection and analysis, hindered the ability of other digital data to behavioural analysis has both the computational process and the gain the public trust needed for a major the data science community to understand and resulted in significant challenges around results of such process can be explained vaccination programme, it is vital to be able integrate data from different sources. inclusion and diversity in the reporting and trusted. Be more open about biases and modelling of behaviour. Traditionally in data to avoid erroneous or biased data to present data in a way that is not prone to Effective policy interventions often require underrepresented groups are also scarce in becoming the basis of intervention.58 misinterpretation. localised responses (e.g. to implement digital data, especially those on the ‘invisible’ – Engage with training for public health – Build sustainable links between the data lockdown at the fine spatial scale) but the data side of the digital divide and those that are not professionals to better communicate data science community and others – those for localised analysis and modelling was simply represented in any data.55 Insufficient links research findings: There is a major need to who are closer to policy-making but also not publicly available to do this. Supported by between datasets meant underrepresented communicate models effectively to decision behavioural scientists who could enhance greater data sharing, cases for intervention groups may remain underserved (e.g. the makers and the wider public, and to link our interpretation of behavioural data. could be strengthened by deeper modelling intersection of the spread of the pandemic models to actions as a core part of the data and combining local micro-level data with with mental health or precarious labour – Enhance collaboration by emphasising an science process; to debunk misuse of data; regional/national macro-level data. Currently, markets). This is often made difficult where interdisciplinary and data-driven research and to provide clarity on the aims of ‘what- however, significant gaps in data availability different geographical aggregations are used. agenda. if?’ modelling. means that modellers are forced to rely on Disadvantaged groups have been insufficiently – Encourage research into deeper and more assumptions, and the extent to which project engaged on what research would be useful for – Improve access to a far broader spectrum integrated modelling that goes beyond findings stemmed from assumptions or data them, and the data science community would of data: For more effective analysis and epidemic / transmission models. has not been made clear. The limitations of benefit from building on the work of the few modelling, including at the fine spatio-

modelling were not widely understood, and organisations talking to these groups, such as temporal level, it is vital to integrate other Useful resources it is vital to start communicating these as the Ada Lovelace Institute.56 areas of data. The COPI notice approach part of a wider drive to ensure confidence in that allowed short-term release of health Code: Stochastic age-structured model of International context modelling. Due to the lack of data for other data could be used for other data (e.g. sector SARS-CoV-2 transmission for UK scenario data of relevance to understanding societal non-pharmaceutical interventions (social The variety of global government interventions projections.59 responses to policy). distancing, mask wearing, self-isolation and in response to the pandemic has provided Software: Identifying how the spread of shielding), the impacts of these interventions a rich opportunity for data sharing and – Increase inclusion of underrepresented misinformation reacts to the announcement of have not been fully integrated and calibrated monitoring of the impact of those interventions, groups: Research needs to investigate the the UK national lockdown.60 in epidemiological modelling. There was also which can inform UK policy, and vice versa. medium- and long-term impacts of the virus. a lack of wider availability of certain data (e.g. A notable example of this is the Oxford This will require more effective reaching out Extra case studies: 2, 3 financial transactions) at the granularity needed COVID-19 Government Response Tracker, a to communities but can also draw on the to understand behaviours and social change. comprehensive global tracking initiative from strong base of knowledge we already have Despite some notable standouts, workshop the University of Oxford’s Blavatnik School of on existing health inequalities and broader participants felt that overall communication Government.57 socio-economic inequalities. between the data science community and There also appear to be areas where the – Develop protocols and guidance on data the public could have been better. There is an UK could learn from the experience of quality to ensure standardisation and issue with poor communication of evidence, behavioural analysis and policy intervention transparency in data access, sharing and especially modelling outcomes, and a need in other countries. For example, in Singapore linkage. to communicate uncertainty more clearly and China, test and trace has been shown – Lead public debate on the kind of data and effectively to a wider audience. Some to work effectively only when linked with related to behaviour that could and should workshop participants suggested that the lack significantly more data, for example on mobility be collected and monitored, and the trade- of transparency had its roots in the reliance at the granular level of tracking the actual off between privacy and public health, so of1 decision makers on consultants who were 1 movements of those who are self-isolating. that in the event of another pandemic the able to provide data quickly at the start of the These approaches have rights implications response can be mobilised rapidly, and the pandemic. Partly because of continuing to (e.g. respect for privacy) that might not be policy impact can be evaluated effectively. work with those consultants, decision makers’ acceptable in the UK, but merit detailed focus for behavioural analysis has remained exploration to determine this formally. stuck in ‘fast-reaction’ mode. As a result, we

58 https://rss.org.uk/news-publication/news-publications/2020/general-news/professional-standards-to-be-set-for-da- 55 https://data-feminism.mitpress.mit.edu/pub/h1w0nbqp/release/2 ta-science 56 https://www.adalovelaceinstitute.org/our-work/themes/covid-19-technologies 59 https://github.com/cmmid/covid-uk 57 https://covidtracker.bsg.ox.ac.uk 60 https://github.com/markagreen/misinformation_uk_lockdown_2020 10 11

immediate and direct COVID-19 health issues, Data has different meanings, and the meaning Part C: Impacts as these could be well-defined, the full extent of data has changed during the pandemic. of the non-COVID effects remains uncertain. As an example, if the number of heart attacks Some non-COVID-related health issues increases, is this due to patients’ reticence to Theme 5: Non-COVID-related health impacts direct consequence of the COVID crisis. The are already apparent. Wellbeing has been attend GP or hospital appointments face-to- (lead: Bilal Mateen) London School of Hygiene & Tropical Medicine affected by lockdown measures.74 Mental face, or a lack of exercise during lockdown? investigated the indirect acute effects of the Scope health deteriorated overall, but more for Simple questions will often mean different pandemic on physical and mental health in some groups,75 especially the young, where things. An example is that the General Health the UK.68 Notably, the largest reductions in GP The COVID-19 pandemic raises health issues one in five children with a possible mental Questionnaire, used by practitioners to support contacts for acute physical and mental health that go far beyond the immediate effects of health condition reported fear leaving home.76 mental health screening, asks patients about outcomes during the pandemic were diabetic the virus, and span an extensive range of Participants agreed that for many non-COVID- their enjoyment in activities. The pandemic has emergencies, depression and self-harm. physical and mental health conditions. As an related health issues it is still too soon to have changed the meaning of this question due to Moreover, there are already early estimates example, during the pandemic, emergency a complete picture of the impact, but thought the restriction on many activities. The Referral of the huge backlog of elective surgery admissions for cardiovascular disease it would be a significantly greater impact than to Treatment rules are another example of cancellations during the pandemic,69 as well plummeted, and several studies have been direct COVID. There is growing evidence that where current data may be irrelevant for as on cancer services70 and cardiovascular initiated to understand what happened to the medium-term impacts on diseases such as our new times. Data are mainly collected disease.71 these individuals and whether (for example) cardiovascular disease and cancer is extensive for performance reporting against the 18- we should expect a surge in admissions Workshop participants commented on the and that the longer-term impacts, including week target and now we are observing an from late complications of missed cardiac fact that many analytics communities came mental health, may be even greater still. As unprecedented number of elective treatments, events. Waiting lists for elective procedures 77 together to share problems and best practice the boundaries of ‘non-COVID health’ are so with waiting times in excess of 52 weeks. are rapidly growing, screening services have between healthcare systems to explore ways unclear, this makes it challenging to curate the been paused, and many oncological treatment Tensions remain between quality and of meeting healthcare demands, including necessary data sources. pathways have been disrupted due to the pragmatism, and between funding on urgent non-COVID health needs. Online forums fear of immunosuppressing patients whilst During the pandemic, many successes have work with a short timeline to impact and such as FutureNHS72 and various nationally there is a highly virulent disease so prevalent been made through bilateral collaboration. longer-term studies. Convenience surveys with run webinars have supported these practice in the community. Furthermore, reports have Workshop participants saw the next step as samples of unknown representativeness were sharing opportunities. More widely, new ways identified an increase in mental health issues more challenging, where collaboration involves noted to be potentially misleading,78 especially of working with data have been adopted, such and domestic violence. more than two disciplines crossing disease because, unlike many routine/big data as a two-way flow of data science research, e.g. boundaries. It was recognised that the lack sources, there are often constraints on sample working with police data to understand ethnic The UK response of holistic understanding is more difficult and sizes which can restrict looking at relevant disparities and COVID-19. The ‘COVID-19 priority’ has sped the passage can result in outlining problems rather than subgroups (e.g. ethnicity) using meaningful through ethical and data access procedures. Finally, a number of innovative solutions outlining solutions and making decisions. categories. were quickly launched to address the loss of This reduced bureaucracy has resulted in Whilst data access has improved as a positive International context traditional means of population sampling that existing and new datasets being interrogated consequence of the pandemic, workshop 61 were no longer available during the pandemic. by emerging questions at a rapid pace . It participants noted that there is more to be We could learn from the Scandinavian model For example, the Mental Health of Children and was recognised that the COPI notice has had done. The challenges were associated with for data linkage that adopts an identity card and Young People survey adapted existing methods a positive impact on the progress that has data that did not seem to exist, data that were population register system to help data linkage, to make more use of online methodology,73 been made. Examples relevant to non-COVID not in the correct format, a lack of record-level sampling, coherence, tracking and duplication. 62 and these are now providing new longitudinal health impacts include the BHF Flagship, linked data, being unable to access data, and Normally, patient consent is not necessary, 63 64 resources and supporting researchers to use UK Biobank, Health Data Research UK, data lags. A lack of standardised systems at and research on health data does not need 65 66 different ways of population sampling beyond 79 OpenSAFELY, RTT performance and healthcare institutions hinders the capture approval from a research ethics committee. 67 the pandemic. characterising the shielded population. of decision-making-related data (e.g. patient The result of these efforts has been several Suggestions Challenges triage). Having more live data dashboards for studies that have already highlighted some monitoring services, benchmarking, audit – Data sharing agreements: Development non-COVID-related health issues that are a Whilst the UK was quick to respond to the and feedback would have helped monitor the of more general data sharing agreements 1 indirect impacts of COVID-19. Additionally, as opposed to those just for very particular it1 was noted that a more streamlined and purposes/uses/users, to reduce barriers 61 https://www.imperial.ac.uk/media/imperial-college/medicine/sph/ide/gida-fellowships/Imperial-College-COV- ID19-NPI-modelling-16-03-2020.pdf expedited ethics approval process would be and delays. This is particularly important 62 https://www.bhf.org.uk/for-professionals/information-for-researchers/national-flagship-projects helpful. for non-COVID-related health issues as the 63 https://www.ukbiobank.ac.uk/learn-more-about-uk-biobank impact is evolving slowly and the effects 64 https://www.hdruk.ac.uk/research/better-care 65 https://opensafely.org 66 https://www.tandfonline.com/doi/full/10.1080/17477778.2020.1764876 67 https://bmjopen.bmj.com/content/10/9/e041370 74 https://www.ons.gov.uk/peoplepopulationandcommunity/wellbeing/bulletins/coronavirusandlonelinessgreatbrit- 68 https://www.medrxiv.org/content/10.1101/2020.10.29.20222174v1 ain/3aprilto3may2020 69 https://bjssjournals.onlinelibrary.wiley.com/doi/epdf/10.1002/bjs.11817 ; https://bjssjournals.onlinelibrary.wiley. 75 https://www.understandingsociety.ac.uk com/doi/full/10.1002/bjs.11746 76 https://www.health.org.uk/news-and-comment/news/survey-presents-a-worrying-picture-of-children-and-young- 70 https://bmjopen.bmj.com/content/10/11/e043828.info peoples-mental-health 71 https://heart.bmj.com/content/heartjnl/early/2020/10/05/heartjnl-2020-317870.full.pdf ; https://www.medrxiv.org/ 77 https://www.england.nhs.uk/statistics/statistical-work-areas/rtt-waiting-times content/10.1101/2020.06.10.20127175v1 78 https://www.cambridge.org/core/journals/psychological-medicine/article/covid-conspiracies-misleading-evi- 72 https://future.nhs.uk dence-can-be-more-damaging-than-no-evidence-at-all/4E2EB003C65CADB8C2C774B114D68BEF 73 https://files.digital.nhs.uk/D1/D411D3/mhcyp_2020_meth.pdf 79 https://ec.europa.eu/health/sites/health/files/ehealth/docs/laws_denmark_en.pdf 12 13

will be experienced in the longer term. The Theme 6: Economic and social impacts well as their analysis of the social impacts of mobility and socio-economic data, and 85 data science community could continue (lead: Karyn Morrissey) COVID-19 and how people spent their time this prevented workshop participants from to promote this idea more, e.g. through during lockdown.86 understanding the wider socio-economic public and policy maker engagement to help Scope factors in the early weeks of the pandemic. ensure long-term success of research in this The necessary data on these key socio- The COVID-19 pandemic has touched every area. demographic trends from the ONS and other The visualisation of data via maps was a part of our lives. Beyond the devastating health public data providers were made available to core means of communicating data on the – Record-level data: A call to enhance the effects, the pandemic and pandemic responses the data science community, and frequently pandemic to policy makers and the public. collection of routine, structured electronic have caused the single largest contraction in updated, which workshop participants Data was generally made available at a local health record (EHR) data. High-quality the UK economy in the last 40 years. Moreover, believed made it possible for the community authority district level. However, to really baseline data would have a more immediate COVID-19 has prompted procedural changes, to inform policy makers and the public of the understand the transmission of COVID-19 impact. In a similar emergency pandemic such as replacing school-leaver exams with disproportionate impacts of COVID-19 on pre- across communities, more granularity is scenario, it would provide a robust starting grade predictions. Many of these changes have existing disadvantaged groups. Furthermore, needed, specifically with local-level data. This position to be able to evaluate the effect of disproportionately affected individuals from participants noted that the increased focus lack of spatial disaggregation, whilst preserving the pandemic on health-related impacts and minority ethnicities and those from more socio- on generating social and socio-economic the privacy of economic and social impacts by would have helped identify more easily non- economically disadvantaged backgrounds. data was vital, rather than a sole focus on demographic groups, was limiting and initially COVID-related health impacts. The data science and AI community may have the headline economic impacts such as hid some social inequalities associated with the played a role in mitigating or furthering these 89 – Address deficiencies in data sources: unemployment and GDP. pandemic in its initial phase. inequalities, for example with respect to the This is the perfect opportunity to collect issue of data representativeness (for example, A further success highlighted was that The workshop participants noted that the data some routine, robust and structured non- are minority groups appropriately represented members of the data science community science community had better access to wider COVID and non-communicable disease in datasets used when modelling the effect of were involved in the various cross-sector socio-economic data than industrial data, such data. Participants agreed that capturing the pandemic response measures?). collaborations (academic, commercial, as the impact of COVID-19 on production and missing or incomplete aspects of the NHS non-governmental organisations, etc.) employment by different industrial sectors. and allied services structured data is key The UK response that were rapidly formed in response to Participants questioned whether access to to being able to understand non-COVID- the pandemic. This included, for example, up-to-date industrial economic data at the local related health impacts and inequalities, as Collection and timely release of data including a unique collaboration between the data level would have supported the data science missing groups are the worst affected. A mobility and socio-economic information was science community as part of the RAMP and AI community, particularly when using widely predicted consequence of COVID-19 identified by the workshop participants as Urban Analytics group and Improbable (a data to underpin lockdown scenarios. However, is a recession, and the physical and mental the biggest headline success. Mobility data UK technology company), to understand the this data was not available, and lockdown health issues are likely to be catastrophic, as released by private companies at the start of impact of daily mobility and time use patterns decisions may have been made without being is socio-economic deprivation. A proactive the pandemic was identified as particularly and inform a COVID-19 hazard at the individual informed by data to properly understand the wide lens needs to be on ‘missingness’ such useful80 and assisted with understanding and small-area level. This made it possible relevant economic and social context. This as missing people (homeless) and missing how social contacts impact transmission. to identify the differential impact of disease meant that the economic impact of proposed diagnosis (is there a fall in heart attacks or Local government and central government spread on vulnerable populations, such as lockdown measures such as the publicly have they been undiagnosed?). departments’ data sharing supported young those with severe mental illness,87 and on controversial 10pm curfew was not able to be people and those who are vulnerable81 and – Holistic working: Prioritise efforts that see ethnicity.88 measured via a robust data-driven cost-benefit assisted in recognising the need for complete data science as a route to bringing together analysis, weighing up both the health and and accurate coding of ethnicity.82 multiple clinical disciplines (and economics Challenges economic impacts of such measures, as well experts). This may encompass better Demand for disaggregated data by socio- The first overarching challenge highlighted as medium- to longer-term socio-economic working with those on the frontline and demographic factors meant better breakdowns in the workshop encompassed difficulties impacts associated with such policies (such planners who can highlight problems before by ethnicity, income, sex, age, etc. to determine that were encountered during the pandemic as loss of employment across the sector and they are noticeable in the secondary data. impact and identify where help is needed.83 resulting from inaccessibility, availability subsequent impact on household welfare). This in turn meant that messages to both Extra case studies: 1, 2 The contributions of the Office for National and consistency of some data. Specifically, policy makers and the public were less clear Statistics (ONS) were praised on a number of participants noted that there is a robust occasions in our workshops, including their and rigorous system in place for obtaining and not as informative as they could have 1 work on deaths stratified by ethnicity,84 as epidemiological data (e.g. number of cases been, and the lack of transparency on the links 1 and deaths) via institutions such as Public between official sources of data and how policy Health England’s daily updates. However, such decisions were taken resulted in public trust a system does not exist for wider economic, being eroded.

80 https://www.google.com/covid19/mobility 81 https://cdei.blog.gov.uk/2020/08/05/covid-19-repository-local-government-edition ; https://leedscc.maps.arcgis. 85 https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/healthandwellbeing/bulletins/coro- com/apps/opsdashboard/index.html#/7a5fa9c4b3d74443be52d35dc1c34265 navirusandthesocialimpactsongreatbritain/4december2020 82 https://www.hdruk.ac.uk/news/championing-diversity-and-inclusion-through-data 86 https://www.ons.gov.uk/economy/nationalaccounts/satelliteaccounts/bulletins/coronavirusandhowpeoplespent- 83 https://fletcher.tufts.edu/news-events/news/disaggregated-data-and-real-reflection-covid-19-risks ; https://gisand- theirtimeunderrestrictions/28marchto26april2020 data.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6 ; https://data.humdata. 87 https://journalofpsychiatryreform.com/2020/07/29/vulnerable-populations-differential-impact-of-covid-19-on-popu- org/visualization/covid19-humanitarian-operations lations-with-severe-mental-illness 84 https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/articles/coronavirus- 88 https://www.health.org.uk/publications/long-reads/how-to-interpret-research-on-ethnicity-and-covid-19-risk-and- covid19relateddeathsbyethnicgroupenglandandwales/2march2020to15may2020 outcomes-five; https://www.bmj.com/content/369/bmj.m2503 89 https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)32465-X/fulltext

14 15

A second challenge reported was the lack To support this suggestion, the following sector organisations, private organisations, of a joined-up programme of data science actions were proposed during the workshop. and industry. activity at a national level across all sectors. It was thought that these would help the – Public trust and privacy: To enable the This led to repetition within the data science community to better understand the socio- effective use of public data to address community. Transdisciplinary collaboration was economic and wider societal impacts of crisis situations, the data science and AI needed during this pandemic to bring together COVID-19, including issues around inequality, community must continue to build trust data and expertise from health, HM Treasury by gaining access to appropriate health data amongst the public. This can be achieved and the private commercial sector. Although and data on the complex determinants of through a greater degree of transparency, as there were some important developments in health and using this in an appropriate manner. well as addressing public concerns related data sharing, there were instances where the These actions would give the potential to to data security, privacy and confidentiality, culture of protecting data may have worked develop large datasets that are safe and trusted whilst simultaneously developing a better against the pandemic preparedness effort, by the public. perception of the risk-benefit trade-off, both within the data science community and for example in discussions of the benefits the private sector. – Standardisation: Forming a public stewardship body which would develop versus costs of lockdowns. A third significant challenge that participants and maintain a national data governance – Public engagement: Ideally, the data noted was that the data science and AI process with a standardised framework for science community will develop vehicles to community lacked the necessary proactive data access to facilitate timely analysis. This develop public trust by greater engagement approach to prevent and stop data misuse and body would be able to provide protocols and with the public, for example by working with misrepresentation. Effective communication set a consistent approach for sharing private representative groups, informing them and with the public, media and policy makers could economic and social data prior to public consulting with them as the science around have mitigated against the many counter- health emergencies, with the benefit of an incident such as the pandemic develops. messages and ‘fake news’ stories relating to addressing the need for more granular data the pandemic that emerged. identified in the challenges. – Rolling data collection: Related to developing public trust and understanding in International context The steps towards this ambition were data science, rolling surveys to understand identified as: the economic and socio-economic impact The Oxford policy tracker90 tracks and 1. The data science and AI community of COVID-19 and various COVID-related compares COVID-19 policy responses around contributing to testing new data stewardship policy measures (e.g. business closures) the world, and the Imperial behaviour tracker91 models.93 The GO FAIR Principles94 would ideally be undertaken to help fully tracks global behaviours. Both proved to be for scientific data management and understand the economic and social impact useful resources when evaluating the UK stewardship would ideally be adopted. of such decisions. response in an international context. 2. Encouraging companies that hold data to Useful resources There may be something to learn from the trust sign up for this. gap in Black and Latinx communities in USA Data and code: ‘Living through Pandemics regarding the COVID-19 vaccine rollout.92 3. Holding conversations about privacy and – Using Protective Cordons to Enhance putting agreements into place. Continuity.’95 Suggestions 4. Providing standardised indicators such as Extra case studies: 4, 5 There was a strongly held view from the local economic performance and inequality, respondents to the survey and in the workshop beyond the Index of Multiple Deprivation. that the COVID-19 crisis has exposed the Variables such as local economic value- vulnerabilities of individuals, societies and added figures, gross local output, local economies, calling for a rethink of how data unemployment rates and local housing on economic and social aspects of society affordability could be made public at an are collected, shared and analysed in the intermediary level of disclosure for the wider data science and AI community. During the scientific community. workshop, participants highlighted one all- 1 – Collaboration: The data science and 1 encompassing suggestion in particular: AI community should make greater Focus attention on the medium- and strides to work collaboratively and bring long-term rather than just the immediate transdisciplinary groups together, including COVID-19 response, and build a data e.g. local communities, health and social infrastructure that supports this. care providers, local partners, analysts, third

90 https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-government-response-tracker 91 https://www.imperial.ac.uk/centre-for-health-policy/our-work/our-response-to-covid-19/covid-19-behaviour-tracker/ 92 https://www.covidcollaborative.us/content/vaccine-treatments/coronavirus-vaccine-hesitancy-in-black-and-lat- inx-communities 93 https://theodi.org/article/applying-new-models-of-data-stewardship-to-health-and-care-data-report 95 https://figshare.dmu.ac.uk/articles/dataset/Data_Code_for_Living_through_Pandemics_-_Using_Protective_Cor- 94 https://www.go-fair.org/fair-principles dons_to_Enhance_Continuity_/12477515 16 17

responsive to the need from the public for data extraction and use. Part D: Enabling data science response regular project consultation and data updates, encouraging a more transparent approach to A lack of pandemic-specific ethical data sharing. oversight for many COVID-19 data projects is Theme 7: Ethics, law and governance (lead: have provided a solid starting point for thinking symptomatic of a wider lack of regulatory and David Leslie) about the development and codification Challenges statutory infrastructure that might effectively of principles-based standards and binding Scope control irresponsible research and innovation regulations in situations of a global health The scramble to use existing datasets, or to behaviour, even as the changing environment The pandemic has brought into sharp focus pandemic, and in a post-COVID-19 world. rapidly create new ones recording COVID-19 of scientific challenges and practices provoked several issues pertaining to the ethics and symptoms and capturing clinical treatments reflection.111 The widespread sharing of data was brought governance of data science, and AI research and outcomes, ran the risk of proceeding about by cross-sectoral and inter-organisational in general – from the impact of unrefereed without due regard for sampling biases. Over-reliance in the public health community collaboration. There has been a willingness to scientific preprints and how this information Workshop participants were concerned on procurement and third-party vendors collaborate in the context of the pandemic,103 is communicated to the public, to changes that biases may be “baked into” datasets for data science and AI solutions was also with new collaborative approaches, some of in research protocols in response to urgent (for reasons of systemic discrimination and highlighted. Many of the roles played by AI it stimulated by UKRI (and others) to generate demands for rapid response data analytics and structural inequalities), and other constraints and data science in COVID-19 are filled by the more flexible and responsive modes of funding AI innovation, to the permissive data sharing intrinsic to data collection methods. Another increasing involvement of the commercial and cooperative, multi-institutional research, 112 environment that was created by the COPI concern was that AI and data science sector. This creates issues around the such as the DECOVID initiative.104 notice to facilitate the national response to the professional standards had been overlooked or tension between the ‘public good’ orientation neglected, and that existing recent guidelines of health-supportive AI technologies and the public health crisis. Some aspects of public engagement and and standards regarding AI were either not value profiles of private sector companies. participation have been positive;105 work by the The UK response adhered to or not given full consideration, and It also creates issues around the systemic Ada Lovelace Institute106 on how to engage the that such derogation had been justified with dependencies on ‘Big Tech’ for the provision The view was expressed in our workshops that public rapidly and directly during lockdown has reference to public health interests. of data processing infrastructure, off-the-shelf the experience of the pandemic might spur the shown that, even amidst a public health crisis, AI solutions, and other goods/services across development of broad-reaching guidelines for the public’s voice can be heard. Workshop Urgency was often used as justification to digital supply chains. AI in healthcare. There has been considerable participants felt that the data science avoid public engagement when setting up interest from many data scientists and some community and wider groups of impacted data collection and analysis. This sense of International context sections of the wider community in transparent individuals did “step up”. For example, the expediency and the race for outputs enabled a and accountable data, scientific innovation, Royal Statistical Society contributed to flagging tendency to expand the claimed scope of the There was a strong suggestion from workshop and Open Science. There has been particular up hazards early in the contentious Ofqual results of data analyses. participants that the focus of biomedical and interest in data sharing practices96 and issues process.107 Public dialogue surrounding the public health related technologies should surrounding data accessibility and useability.97 A Level grading issue had references to While the WHO has stated that the release move from a national perspective to a more The government’s Data Ethics Framework98 algorithmic bias and fairness and to data of preprint results is a “moral obligation” in global viewpoint, with greater participation 109 and its public sector guidance for safe and ethics and human-centred innovation, thus the context of public health crises, the in transnational efforts of collaborative and 113 ethical AI,99 ONS’s ‘Five Safes’ protocol for data demonstrating that AI ethics has progressed group agreed that the “post-truth” context equitable research and standard setting. 100 some way to becoming a part of the moral of misinformation on social media and integrity, and the ICO and The Alan Turing International differences in classification and vocabulary of public debate.108 amplification of false news stories made best Institute’s guidance for explaining decisions legal restrictions on collecting and sharing on made with AI101 have all set standards for practices for publication during the pandemic Workshop participants highlighted the benefits matters such as ethnicity was highlighted, as it 1responsible data use and AI.102 These cross- complex. There is a lack of understanding of an Independent SAGE, which provided a makes some data sharing difficult; for instance, sector interventions in creating criteria for of, and low/inappropriate involvement model of how scientists can analyse, interpret US data includes ‘Latinx’, which is rarely used best practices for responsible data scientific of, many data science communities in and comment about the situation from a more 110 in Europe. research and innovation have helped to media processes. Preprint protocols and evidence-based, neutral and interdisciplinary establish public expectations about “what good accelerated peer review processes on papers Suggestions standpoint. The data community was also looks like” in socially impacting data use. They must be approached with this in mind. And although paper retractions due to data – Establish professional accreditation114 for inaccuracies, and commercial data sharing data scientists, including a code of conduct, 96 https://opendatasaveslives.org “by press release”, seemed to be justified by with specific expectations for working 1 97 https://digital.nhs.uk/about-nhs-digital/our-work/nhs-digital-data-and-technology-standards/framework/beta---da- “the pandemic”, these corrective behaviours rapidly/outside specialism in a humanitarian ta-security-standards#the-data-security-standards crisis, which would provide credibility and 98 https://www.gov.uk/government/publications/data-ethics-framework could also do damage due to the dynamics of 99 https://doi.org/10.5281/ZENODO.3240529 social media message amplification, and the increase trustworthiness. 100 https://www2.uwe.ac.uk/faculties/bbs/Documents/1601.pdf predominant culture of distrust surrounding 101 https://ico.org.uk/for-organisations/guide-to-data-protection/key-data-protection-themes/explaining-deci- – Increase public involvement in evaluating sions-made-with-ai 102 https://www.gov.uk/government/publications/data-ethics-framework/data-ethics-framework-2020 103 DELVE: https://royalsociety.org/news/2020/04/royal-society-convenes-data-analytics-group-to-tackle-covid-19 109 https://www.who.int/medicines/ebola-treatment/blueprint_phe_data-share-results/en 104 https://www.decovid.org 110 https://reutersinstitute.politics.ox.ac.uk/communications-coronavirus-crisis-lessons-second-wave ; https://reu- 105 https://covid.joinzoe.com/about tersinstitute.politics.ox.ac.uk/sites/default/files/2020-06/DNR_2020_FINAL.pdf 106 https://www.adalovelaceinstitute.org 111 https://doi.org/10.1111/1740-9713.01463 107 https://rss.org.uk/news-publication/news-publications/2020/general-news/rss-alerts-ofqual-to-stats-issues-relat- 112 https://reutersinstitute.politics.ox.ac.uk/industry-experts-or-industry-experts-academic-sourcing-news-coverage-ai ing-to-2020 113 Such as the Research Data Alliance and the European Open Science Cloud 108 https://www.turing.ac.uk/blog/secret-life-algorithms-time-covid-19 114 https://rss.org.uk/news-publication/news-publications/2020/general-news/professional-standards-to-be-set-for- data-science 18 19

outcomes, co-design in projects and some aspects were neglected, and others not shortfalls. Examples include the NHS COVID-19 schools and workplaces. The COVID Symptom negotiating overall priorities. This includes executed as precisely as necessary to be fully Data Store,120 the DECOVID dataset121 (which Study129 supported the understanding of both engaging with a wider audience, especially effective. aimed to provide a secure mechanism to disease spread and significant early symptoms. with minority and underrepresented collect data about patient care across all Another pressing need was care home data, groups, perhaps by setting up “citizen The UK response participating hospitals in the UK), and the and the VIVALDI130 study sought to investigate 122 groups” or resourcing the Turing to be the Workshop participants believed that the AI Health Data Research Innovation Gateway, levels of COVID-19 infections in homes. focus of public engagement. Developing and data science community had responded established to provide information about mechanisms for well-funded community the datasets held by members of the UK Valuable support was provided by NHS rapidly to the challenges of the pandemic, by 131 involvement in setting research priorities Health Data Research Alliance, facilitating England, the Open Data Institute and several supporting clinicians, local authorities and national disease societies who put in a huge and for the end-to-end participatory design other organisations to collect and make use the navigation of multiple datasets. The care of research and innovation projects was homes Capacity Tracker,123 established in amount of effort to facilitate access to existing of all available data in innovative, iterative data sources. These provided rapid answers to emphasised. and collaborative ways. Workshop attendees 2019, was rapidly adapted to provide valuable information about infection monitoring and disease-specific questions around COVID-19 – Develop better ways to communicate in an commented about how data science work had audit/compliance. risk, including for rheumatic musculoskeletal evidence-based and accessible way and come to “fruition” and that their contribution to diseases (see international context) and work with “today’s world” of social media, the pandemic response had been welcomed A major strength was that the UK already had diabetes.132 preprints and the press, including increasing and encouraged. The data legacy from the in place a protocol for national collection of the understanding and involvement pandemic will be extensive, and offers the epidemiological data in the event of a severe Innovative solutions to provide real-time of data scientists in media processes. potential to support data scientists to respond respiratory pandemic. This was immediately access, rapidly and securely, to detailed patient 133 Communicate to the public better by rapidly in a future pandemic situation. deployed through the ISARIC4C124 consortium, health records (notably the OpenSAFELY providing channels for the provision of platform led by the University of Oxford The pre-existing data infrastructure in the and meant researchers were rapidly able reliable information and inclusive means for and London School of Hygiene & Tropical UK includes traditional data sources such to provide vital information to characterise critical information consumption. and prognosticate patients hospitalised with Medicine), were identified as likely to be of as surveys and routinely collected patient benefit to the UK health research base beyond – Form a relevant body115 or use the Turing data recorded in electronic health record COVID-19, and to assess patterns of disease 125 the end of the pandemic. or a relevant national academy to bring systems. Open data provided by institutions in children. Respondents also identified together AI technologists, data science such as the Office for National Statistics116 the quality of data on critical care admissions Several UK longitudinal studies established researchers, experts from across the and Public Health England have given provided in weekly reports by the Intensive specific data collections in response to disciplines of the social sciences and researchers and the public the ability to access Care National Audit & Research Centre COVID-19, including the UK Biobank134 and 126 humanities, and members of civil society. near real-time national data on COVID-19, (ICNARC ), and the utility of aggregate data on Understanding Society.135 Notably, the ELSA136 The aim of this being to increase capacity including information on testing, deaths and a similar critical care population collected by study is collecting data on the effects of the across communities for understanding hospitalisations. CHESS (COVID-19 Hospitalisation in England COVID-19 crisis on the older UK population; 127 and operationalising responsible research Surveillance System ) which was adapted data from the first wave of collection is and innovation practices in AI and data There were regional data management from the UK Severe Influenza Surveillance available now from the UK Data Service.137 science in general, and with regard to the systems that proved highly valuable. The System by Public Health England. SAIL117 data linkage service was instrumental Local authorities and policy makers were able specificities that might arise in a global New collaborations were established to identify health pandemic. Utilise this body to in redeploying existing datasets to support to make use of existing simulations produced pandemic efforts in Wales. Public Health the prevalence of COVID-19 in populations. by the AI and data science community, e.g. provide the training and upskilling needed 128 118 These include a partnership between the to produce ethical and democratically Scotland provided a daily data dashboard the SPENSER project (Synthetic Population University of Southampton, Southampton City 138 governed technologies. to update policy makers and the public, Estimation and Scenario Projection Model ) including data collected from care homes. Council and the NHS to develop a non-invasive and the COVIDSurg Collaborative,139 which The ‘DataLoch’119 is a repository of all routine saliva test for large-scale settings such as used global predictive data to inform surgical Theme 8: Data readiness, collection and 1 health and social care data for the Edinburgh monitoring (leads: John Dennis, Sabina 120 https://www.england.nhs.uk/contact-us/privacy-notice/how-we-use-your-information/covid-19-response/nhs-cov- Leonelli) and South East Scotland Region (funded by id-19-data-store the Data-Driven Innovation Programme) and 121 https://www.decovid.org Scope provides data to a range of researchers to 122 https://www.healthdatagateway.org 123 https://www.necsu.nhs.uk/capacity-tracker address COVID-19-related questions. High-quality data are the cornerstone of an 124 https://isaric4c.net 125 BMJ 2020;370:m3339 / BMJ 2020;369 / BMJ 2020;370:m3249 1effective pandemic response. The UK was As part of the data response, it was recognised 126 https://www.icnarc.org able to rapidly create dedicated data assets that there were critical gaps in the data to 127 https://www.england.nhs.uk/coronavirus/wp-content/uploads/sites/52/2020/03/phe-letter-to-trusts-re-daily-covid- to address issues related to the COVID-19 support both research and policy-making. 19-hospital-surveillance-11-march-2020.pdf pandemic. The speed and complexity of 128 https://www.southampton.ac.uk/news/2020/09/saliva-phase-two.page Funding was provided by UKRI, NIHR and many 129 https://covid.joinzoe.com/data the response, and deficiencies in existing health charities to establish new datasets 130 https://www.ucl.ac.uk/health-informatics/research/vivaldi-study infrastructures in some domains, meant that and management systems to address these 131 https://theodi.org/topic/covid-19 132 https://www.thelancet.com/journals/landia/article/PIIS2213-8587(20)30272-2/fulltext ; https://www.thelancet. com/journals/landia/article/PIIS2213-8587(20)30271-0/fulltext 115 UK Statistics Authority 133 https://opensafely.org 116 https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/datalist?fil- 134 https://www.ukbiobank.ac.uk/explore-your-participation/contribute-further/serology-study ter=datasets 135 https://www.understandingsociety.ac.uk/topic/covid-19 117 https://saildatabank.com 136 https://www.elsa-project.ac.uk/covid-19 118 https://public.tableau.com/profile/phs.covid.19#!/vizhome/COVID-19DailyDashboard_15960160643010/Overview 137 https://www.ukdataservice.ac.uk 119 https://www.ed.ac.uk/usher/dataloch 138 https://lida.leeds.ac.uk/research-projects/spenser-synthetic-population-estimation-and-scenario-projection-model 139 https://bjssjournals.onlinelibrary.wiley.com/doi/epdf/10.1002/bjs.11746 20 21

recovery plans. Local authority representatives lacking, difficult to find, or else that there were International context effects on outcomes. The standardised from Richmond/Wandsworth and Haringey in inconsistencies in collection methods used across data specification should have a stress test the workshops reported how they now value different regions which made direct comparisons The AI and data science community made use by using regular expected events, e.g. the predictive analytics, as these methods enabled of data challenging. Examples provided include: of global data sources and other data sources annual influenza season. 146 them to identify residents likely to need additional (e.g. the COVID-19 Data Repository ). – Automated data collection systems: Data support.140 – Care home data: Although data collection improved, initially there was limited information European cooperation was helpful, collection is resource-intensive, and there Challenges available and this meant some early signs of particularly where they were in a later stage would therefore appear to be clear benefits the egress of COVID-19 within the sector were of the pandemic to the UK. An example of to automated data collection systems, Workshop participants reported some difficulties missed. a successful collaboration was between reducing the reliance on frontline staff to which they believe impacted on effective data Newcastle University and the University record data (often using manual systems). readiness, collection and monitoring. – Social care data: There were no systems or Hospital in Modena, Italy. The hospital team – Clear protocols for collecting data on incentives in place to support care agencies provided UK-based researchers with data protected characteristics: These would The lack of a centralised data hub for all health and personal carers to collect data about access and updates quickly, so that these 143 include, for example, ethnicity, sex, age data (not just NHS data) for the UK meant that it COVID-19 infections within home settings. could be used by the researchers to help with was difficult to avoid duplication of data. There and socio-economic status. We should – School attendance data: Respondents the predictions of likely respiratory failure in were ethical/data protection limitations on access 147 explore ways to develop clear protocols for observed that they had established regular patients with COVID-19 pneumonia. to data where they included personal data, and anonymisation/synthetic data generation reports to inform policy makers using school some respondents found it difficult to gain access The EULAR COVID-19 Database148 was to include important demographic attendance data. Some date input fields were to data because they were not health specialists. established to capture how rheumatology information in open datasets, such as removed with no warning or documentation 149 conditions and their treatment affected the the OpenPseudonymiser technology to track the changes. This meant that the Good-quality data requires systematic collection risk of and severity of COVID-19. It was part of developed by the University of Nottingham. requested monitoring could not be completed. and curation, and this is extremely labour- a global initiative that started from a post on intensive and expensive. Data collection – Investigate greater access to NHS data: – Large-scale collection of detailed biological Twitter. The team in the US shared a database Access to high-quality, professionally methods in routine use within many healthcare data vital to understanding COVID-19 such template with the UK which enabled the organisations seemed neither appropriate nor managed, national NHS data repositories as blood test results, proteomics, and genetic EULAR team to go live less than one week later. would clearly be of great value to many robust enough for data collection on the scales sequencing were reported by respondents to Clinicians were able to quickly submit data required.141 Respondents stated that data entry for research groups. Clearly, this raises be lacking. Limitations of funding to projects and give feedback on the database design. sensitive and complex issues around ethics most, if not all, COVID-19 epidemiological studies like the BRAINS Imagebank144 also meant that Data import pipelines were set up with several (e.g. ISARIC) were manual, and often could only and privacy, but given the very obvious there were limited healthy controls to compare EU countries. The UK database template has research value of such data, there would be done by senior staff. Furthermore, each study against brains that had possibly been damaged been shared with national societies in the EU had an entirely separate data specification and seem to be some benefit to investigating by COVID-19. and other disease-specific researchers to ways to provide greater access. collection mechanism. The result of this was a help them set up their own COVID-19 disease- There were reported gaps in many data sources, substantial time drain for frontline hospital staff specific databases. – Promote institutional and global data and a potentially detrimental impact on patient particularly around standardisation of capture sharing for health emergencies: Data care. of socio-economic status and ethnicity, at Suggestions sharing agreements between the UK and both national and sub-national levels. Data other countries take time to organise, and The DECOVID142 project had aspirational aims – Standardised data specifications: There standardisation would have enabled researchers many were supported by EU funding. Having to address some of these issues and engaged are clear benefits to creating stability in to more easily triangulate information from more flexible regulations for global data the involvement of many data scientists, some the data schema, and clarity in each field different data sources and thus be better sharing during future health emergencies of whom had limited experience of the health so that the perception of inconsistencies is equipped to answer policy-relevant questions and will help facilitate global learning. system. Workshop attendees reported that many to allow testing for any contextual effects on the reduced, and the data can offer improved found the realities of multi-dimensional healthcare outcomes. real-time information with minimised – Create a central repository of available data overwhelming, and this made it difficult to subjective data processing. Changes in shared datasets, signposting to open produce robust data analysis. An example is the Workshop participants also reported examples data collection and methodology should be datasets, and, for non-open data, the details simple question “Was the patient ventilated?”. of where routinely collected data was perhaps clearly communicated, highlighting updates of data sharing agreements and protocols This may seem straightforward and intuitive, but not harnessed effectively by policy makers. An and changes in each data release. This will for securely accessing datasets. The Turing given the complex treatment regime provided to example of this was daily situation reports from facilitate the comparisons between datasets could support the development of this many1 COVID-19 patients, in some cases it was individual hospitals, which provided information 1 and the sharing of data between different type of ‘data lake’, enabling the provision of impossible for clinicians to answer the question including daily bed occupancy rates for hospitals organisations, e.g. local authorities. Using a shared datasets and equitable data access, with a simple “yes” or “no”. across England. This information could potentially standard “baseline” data specification would and facilitating unique linkages between have been used more effectively to manage permit linkage of various sources which datasets. Workshop participants reported that data patient intake and ease pressure on hospitals at or would allow the testing for any contextual collection for several crucial areas was either approaching unsafe occupancy levels.145

140 https://policyinpractice.co.uk/councils-get-faster-data-insights-to-boost-their-covid-19-recovery 141 https://www.gov.uk/government/news/phe-statement-on-delayed-reporting-of-covid-19-cases 142 https://www.decovid.org 143 https://www.health.org.uk/news-and-comment/blogs/strengthening-social-care-analytics-in-the-wake-of-covid-19- initial-findings 146 https://github.com/CSSEGISandData/COVID-19 144 https://www.brainsimagebank.ac.uk 147 https://www.medrxiv.org/content/10.1101/2020.05.30.20107888v2 145 https://www.medrxiv.org/content/10.1101/2020.06.24.20139048v2 148 https://www.eular.org/eular_covid19_database.cfm 149 https://www.openpseudonymiser.org 22 23

– Initiate a strand of work on minimum data standards, developing the existing Extra case studies international work on minimum standards and defining what metadata would look Submitted by workshop participants using an future best practice guides? like, and explore the data requirements for online form. Continue to invest in using real-world NHS decision makers in the UK. data on an individual level, and use the vast – Establish a forum to set principles Case study 1: Nottingham monitoring/ skills and knowledge within NHS Trust analyst and standards for data collection, follow-up system communities to enable that to happen. and promote the adoption of FAIR150 Organisation: Nottingham University Hospitals (Findability, Accessibility, Interoperability NHS Trust and the University of Nottingham. Case study 2: Discovery Data Service and Reusability) principles, as well as responsible data collection and curation Overview Organisation: Barts Health NHS Trust practices. We have created a near real-time system Overview – Transnational work: Strengthen the global of monitoring and follow-up of all people The Discovery Data Service (DDS) was collaborations that evolved during the suspected of having or confirmed as having developed in north east London with the aim of pandemic, e.g. the work of EULAR151 and SARS-CoV-2 attending Nottingham University providing aggregated data from primary care the Research Data Alliance,152 to produce Hospitals NHS Trust. We have developed and and secondary care systems in a single system guidance for data documentation and implemented an individual risk prediction for point-of-care decision support, quality metadata. score using all available electronic information (clinical observations, blood results, clinical improvement initiatives, population health – Develop collaborative working decisions, oxygen and ventilation requirements development and research. In the first instance, relationships between data scientists and and ongoing follow-up for readmission and vital the data providers were GP practices in north clinicians and other domain experts: Data status (in and out of hospital)). The prediction east London and Barts Health NHS Trust. scientists can provide insights about the score is calculated on a daily basis using collection and storage of data that clinicians Why funded this initiative? longitudinal information from the entire patient may not be aware of. Clinicians and other journey within the hospital. This system and At the beginning, the project was funded by the domain experts can provide valuable score has been implemented in live hospital Endeavour Health Charity which is a charitable insights to data scientists about the multi- data and is being used to optimise resource health trust endowed by Dr David Stables, one dimensional nature of data collection within and patient management within the hospital. of the founders of the EMIS GP system. As the health and social care settings to improve project has developed, contributions have been collection methods and reliability across Who funded this initiative? made through the NHS by STPs and the One different settings. This was a collaboration of existing staff based London programme. Over the period of the Extra case studies: 1, 2, 3, 5 in Nottingham University Hospitals NHS Trust project to date, NHS & charitable sources are and the University of Nottingham. matched. Overall, did the initiative make the desired Description / intended outcomes impact? The Discovery Data Service is a cloud- Yes. based publisher-subscriber system aiming to make access to large-scale health datasets What worked well? from as many providers as possible. In this system, the data controllers remain the health Excellent collaboration between all parties organisations publishing the data to the DDS involved in the work. Rapid near real-time and data are only processed when appropriate automatically generated, dynamic tracking, data sharing agreements have been made scoring and prediction implemented within between publishers and the subscribers to hospital systems. 1 the system. Once in the DDS environment, the What were the difficulties and challenges? data are transformed to a common data model which in turn is defined by a semantic ontology. Getting the time and resource to support it The data model is configured for business within the NHS. purposes, while the semantic ontology is configured for interpreting meaning using Based on your learning from this initiative, subsumption and reasoners. what recommendations would you make for

150 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4792175 151 https://www.eular.org/eular_covid19_database.cfm 152 https://www.rd-alliance.org 24 25

The main intended outcome is to provide governance process that includes patients What worked well? What worked well? semantically useful and aggregated data both and citizens. to publishers and subscribers of the data to The development of data collection tools and Close collaboration with key stakeholders; 6. Funding the DDS along with sustainable support patients at the point of care, improve data analysis algorithms and the novelty of the development of a framework leveraging on new investment in health IT across sectors of population health and to support researchers approach. forms of data (Twitter) to measure immigration health. sentiment in a scenario in which traditional in developing new knowledge. What were the difficulties and challenges? If the initiative did not achieve the intended data sources were unavailable; provide novel While the system is intended for large-scale outcomes, what do you believe were the The City Council didn’t provide the fund: they insights in near real-time. data analytics, a specific use case has been reasons for this? had to prioritise and feed the fund somewhere What were the difficulties and challenges? developed to aid reporting of primary care else. The DDS is in continuous development. Many assessments of COVID-19 infections during Accessing and processing the data. the pandemic. GPs record COVID-related intended outcomes have been achieved while If the initiative did not achieve the intended structured data (SNOMED CT) in their clinical others are still being worked on. outcomes, what do you believe were the If the initiative did not achieve the intended systems and this data can be linked to reasons for this? outcomes, what do you believe were the Based on your learning from this initiative, demographic information, hospital activity data reasons for this? what recommendations would you make for Lack of funding and support from policy (Admissions, Discharges and Transfers) as future best practice guides? makers. The complexities of extracting meaningful well as laboratory and imaging data. The DDS information from tweet posts. developed interactive dashboards in April 2020 1. Ensure each step in the development is Based on your learning from this initiative, which continue to be used at NHS London defined by a project plan held in an overall what recommendations would you make for Based on your learning from this initiative, level to show COVID-related disease statistics programme. future best practice guides? what recommendations would you make for in north east London, including geospatial 2. Work early on the data access rules, future best practice guides? displays of infection cohorts. More support from policy makers, and better data governance and citizen involvement publicity plan for our solutions. Develop a comprehensive understanding of Overall, did the initiative make the desired systems. the use of Twitter APIs. They are expensive impact? 3. Organise a programme board at the start Case study 4: Monitoring attitudes towards and requests need to be carefully planned. A carefully designed list of search terms is Yes. of the project. immigrants key as well as knowing that Twitter retrieves 4. Explore different governance models for What worked well? Overview information from a point backwards. managing the project. There are many examples of data extraction This is an ongoing project which aims to 5. Invest and develop teams focused on data Case study 5: Predicting the extent of and analysis from the DDS. Three current measure and monitor changes in attitudes curation and data quality across systems as towards immigrants during the early stages COVID-19 impact examples illustrating point-of-care use, well as within publisher systems. population health and research are: of the current COVID-19 outbreak in five Organisation: De Montfort University. countries: Germany, Italy, Spain, the United 1. NHS 111 frailty flagging for call handlers. Case study 3: Nottingham overcrowding Kingdom and the United States, using Twitter Overview detection data and natural language processing. 2. Primary care COVID tracking and We used simple, publicly available data to Overview reporting to NHS London based on primary Who funded this initiative? predict the extent of COVID-19 impact in the care data during the pandemic (see above). Overcrowding detection for city centres and 632 parliamentary constituencies of Great The United Nations’s International Organization 3. Provision of structured clinical phenotypic shopping malls. Britain. We also proposed and modelled a for Migration. data for the East London Genes & Health scenario that showed that up to 25% of GB Who funded this initiative? programme, which aims to identify genomic Description / intended outcomes could have avoided a lockdown. health risk in the south Asian population of Supported by Nottingham City Council. Who funded this initiative? east London. The project seeks to determine the extent of Description / intended outcomes intensification in anti-immigration sentiment Self-funded. What were the difficulties and challenges? as the geographical spread and fatality rate of The aim was to collect footfall and people 1. Working with vendors to flow data into the COVID-19 increases; identify key discrimination Description / intended outcomes count data (using our people count devices) DDS. and racism topics associated with anti- to detect overcrowding in Nottingham City immigration sentiment; and assess how these The initiative was to use simple, publicly 2. Data curation and data modelling for Centre using spatial analysis and clustering topics and immigration sentiment change over available data to model the spread and impact data taken from enterprise secondary care techniques. Based on the output algorithms, an time and vary by country. of COVID-19. This could then be used to systems. alert is sent to venue managers. “protect” certain areas from spread whilst Overall, did the initiative make the desired treating infected areas. 3. Securing data sharing agreements. Overall, did the initiative make the desired impact? 4. Securing agency support for the initiative. impact? Not known at this stage. 5. Developing an agile but safe information No.

26 27 Overall, did the initiative make the desired impact? Not known at this stage. What worked well? 1. Gathering simple, disparate, publicly available data. 2. The AI modelling using self-organising maps. 3. The accuracy of the modelling as the pandemic progressed. What were the difficulties and challenges? There were no challenges as such, it is just a case of better (more accurate) data would result in more accurate modelling. Getting more accurate data requires government involvement. If the initiative did not achieve the intended outcomes, what do you believe were the reasons for this? The ideal outcome would have been for the government to pilot this in a few regions. In essence we argue that they did pilot it (e.g. preventing people from travelling from Tier 3 to Tier 2 etc.), but not to the extent that we would have liked. Based on your learning from this initiative, what recommendations would you make for future best practice guides? AI and data analytics can certainly make a rapid and significant positive impact to pandemic modelling and containment, and governments should focus more attention in this area.

turing.ac.uk @turinginst

28 29