En innbygger - én digital tvilling og fremtidens AI-baserte økosystem for økt egenhåndtering av helse og beslutningsstøtte i helsetjenesten
Hemit konferansen 2019, Trondheim, 19. september, 15:00-15:30
Frank Lindseth, IDI, IE, NTNU Agenda
• Norwegian Open AI Lab (NAIL)
• Alt. Platform / Økosystem for Helse
• Bærbare sensorer (Imaging)
• Digitale Tvillinger
• AI-basert Økosystem
• Egen-håntering (beslutnings-støtte)
• Piloter
• Viktige tema ikke adressert i dag:
• (personvern), data-sikkerhet og etikk
• Singulariteten Norwegian Open AI Lab (NAIL)
Norwegian University of Science and Technology What is Artificial Intelligence?
"The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages."
Oxford English Dictionary
AI > ML > DL Norwegian Open AI Lab Vision & Goals
• A national hub for AI education, research and innovation • Promote collaboration between academia, non-profit sector, industry and government. • To enable both basic and applied research • To support a wide variety of research areas • To perform research at the highest international level • To foster cross-disciplinary collaboration
Norwegian University of Science and Technology 6 Focus on education, research (basic / applied) and innovation
Theory & Method Implementation Value Creation
research collaboration innovation student projects domain knowledge commercialisation
Norwegian University of Science and Technology 7 Select Norwegian Open AI Lab Events and Activities
• Every other Friday - Open AI seminars – https://www.ntnu.edu/ailab/open-ai-seminars • Student organization (BRAIN) events and seminars • Biweekly meetings in NAIL Core Committee • Inspirational Days – March and October each year • March 13: Strategy Workshop – Result: Strategy-document to government • April 26: Nordic Five Tech (N5T) AI workshop – Nordic AI Network being established as a result • May 27-28: NAIS Symposium – https://www.aisociety.no/nais2019/ • June 3-7: Probabilistic AI Summer School – https://probabilistic.ai/ • June 18: AI CHALLENGE — the Future of AI – https://www.ntnu.edu/ai-challenge-2019 • Prominent guests from Norway and abroad AI and UN’s Sustainability Goals
Norwegian University of Science and Technology 9 AI Lab Projects
Smart diagnoses Sustainable Energy efficient and Optimized transport Fish welfare, safety personalized energy production smart IoT sensors planning and Smart and environment health care City monitoring
Norwegian University of Science and Technology 10 NAP Lab (NTNU Autonomous Perception Lab @ IDI, IE, NTNU) Education, research and innovation related to autonomous, connected, shared and electric vehicles in a nordic environment
«The Mother of all AI projects»
- Focus on converting sensor input (camera, LiDAR, radar etc.) to control output (steering, throttle, brake and shifting) - Modular approaches to autonomous vehicles (AVs) (i.e. mapping and localization, perception and prediction, planning and control) - End-to-end approaches to AVs like imitation and reinforcement learning - Simulated environments for AVs - Privacy and Security. - AVs as mobile sensor platforms (inc. ITS, V2I, V2V, Digital Road Twins, HD maps) - Mobility as a Service - MaaS (inc. NAPApp) Early diagnosis of CP in infants
Norwegian University of Science and Technology 12 En innbygger - én digital tvilling og fremtidens AI-baserte økosystem for økt egenhåndtering av helse og beslutningsstøtte i helsetjenesten
Frank Lindseth, IDI, IE, NTNU Did you know that, in principle, you are the owner of all your health-related data?
Do you know where all your health related data is stored?
Would you like to be more responsible of your own health?
Do you think that your health-data could help you being more responsible of your own health?
Do you think that you have the right tools to help you being more in charge of your own health?
What about wearables (implantables), to what degree would that be part of your future health?
What about the data from other people, would that be helpful, in what form?
Would you be willing to donate some of your health-related data to generate new knowledge
Would it help to know that this could benefit mankind, family and friend, your own children and even yourself?
Do you think that a personalized ecosystem for self-management and decision support will be realized within your time?
What does a program like «Hva feiler det deg?» tell you? Challenges: Citizens
owner where responsible Challenges: Challenges: tools Knowledge Generation Knowledge Use help other donate benefit heath registry time consuming realized subset paper patients and data company apply to use product silos put to use fragmented new knowledge complicated vs. time personal data ineffective Challenges: Society
Sustainability Challenges: Data not used Industry Self-management Decision support Precision medicine Relieve Imagine: that all the knowledge gained from all previous cases could be used in a personalized, preventive, diagnostic, predictive and prescriptive manner for the next unseen case (or just use the data from the subgroup in-line with your own profile) Imagine: that you had your own personal twin that looked after you from before you were born to after you passed away, somebody that know you inside out, that can provide you with a status report at any time, that can notify you (or others) if something is about to happen and that can give you great advices regarding decisions to make and what to do, (if you want, which is an important point) Imagine: that you were in complete control of this twin, that you could be confident that all your health related data was safely stored in one place, that your twin is always available, that you decide what data to «donate», and imagine that by doing that you can be certain that your anonymized data is handled confidentially and used to the best of humanity, your family (current and future) and yourself (i.e. while you are living). Imagine: that your own personal twin was part of a huge self-learning ecosystem, that continuously improves taking advantage of increased computing power, more data and better models. That the new knowledge would be immediately available for your digital twin and you, so that you are provided with the tools needed to be more in charge of your own health
Digital Twins (DTs):
the missing link that can bridge the gap between citizens / patients and the healthcare system (one view) DT
Enabling increased: self-management & decision-support
resulting in: increased personalization citizen empowerment & responsibility
increased efficiency & lower cost
better quality & reproducibility
Healthcare-system Citizen / using SotA GP / Radiologist / Patient technology DT-based interaction Surgeon e.g. IoT, BigData & AI DTs Model generation Registries
Models Services
Sensors
DT-based facilitated interaction
Conventional interaction Citizens Health care system Digital Twin (DT) technology DT: a digital copy or model of a physical asset (the PT, with sensors)
Essentials (PSM) MyMDT (DT) HealthEU (Avatar) SFI (PT+DD++)
Life-cycle: from (long before) cradle to (long after) grave (documentation) - planning, design, simulations, optimization, manufacturing, monitoring, prevention, diagnostic, predict, prescribe / decision support, automation, maintenance and destruction. Hierarchical: different scales / zoom in&out, from complex systems down to the smallest detail. How could such a DT-based system look like? What functionality would you like your DT to have? How would you like to interact with your DT? Would you be willing to «donate» some of your data to science and what would you like to get back? How «honest» would you like your DT to be and would you be willing to change your lifestyle to the better based on recommendations from your DT (vs. your PT)
Your smart Digital Twin (sDT) - the future personalized intelligent health (iHealth) ecosystem The ultimate sDT platform: BigData/IoT/DataCleaning - Always available AI/ML/DL/VC/CV/ - Always safe, secure and private Security/Privacy/Ethics - Always learning / discover new connections sDT_N - The best models always available Trusted and explainable AI (XAI) - Optimal view of your data Hybrid models (data + physical models) research challenges
1a) sDT_i 1b) Knowledge Generation - Training - Benchmarking Data from Input from - Challenges Individuals society
Ethics SmartAnalytics & AI/VC sDT_2 Impact & SmartData & Sensors/IoT Impact & benefit for benefit for Security & Privacy individuals: sDT_1 society: smart Digital Twin (sDT) Abbreviations: 1) Infrastructure (in the Norwegian Cloud) (cloud computing, datacenter, TPU/GPU/CPU) - sDT: smart Digital Twin - AI: Artificial Intelligence - ML: Machine Learning - DL: Deep Learning Access to your data 1c) Knowledge Bank - VC: Visual Computing - You are in charge of your data - CV: Computer Vision - You decide which data is used for what Best methods / models - VR / MR: Virtual / Mixed Reality - You (and your doctor) decide which models to subscribe to (can be applied to the sDT directly) DTs: Piloter
• Self-management of Health: Wearables
• Decision-support: Medical Imaging
• (DT-based Interaction: e.g. preop. planning, intraop. navigation and postop. control) DT-based interaction Watches Withings Health monitors DTs: Pilot 1: Wearables
• Many smartwatches and dedicated health monitors are commercially available Fitbit today and many more will enter the marked the coming years. Wireless Blood Pressure Monitor • Many of these offer an API (or use HealthKit or Google Fit that also offers an API) so that data can be streamed into your own DT living in the cloud somewhere (e.g. GCP first and maybe HUNT cloud later)
• Want to iteratively investigate how a DT (and the associated ecosystem) for Apple Watch Self-management of Health can be built on top of a SotA cloud platform. Sleep Smart Temporal Tracking Mat Thermometer • Many have such devices today (and many more will have in the future), they seem to motivate people to be more active and more in charge of their own
health, people are becoming more and more used to these devices and what Norwegian Health Industri they can provide.
BPro • Offer a more uniform approach to data handling (no data silos) and validated high-quality predictive models (combine data from various sensors, use donated data from many citizens, transparent benchmarked models that continuously improve etc.) Somnofy (VitalThings) EVO (Moon • Early prototype already up an running. Labs) HeartGuide DTs: Pilot 2: Medical Imaging
US • Medical Imaging is a very important part of modern healthcare systems and huge amounts of diagnostic images are acquired each day (various modalities like X-ray, CT, MR, US etc.).
• Looking trough all these images in search of abnormalities, deceases and lesions is a time consuming process requiring expert knowledge (i.e. a radiologist).
• Recent advances in AI (particularly DL) have shown that fully automatic segmentation MR of most structures (inc., vessels and tumors) from various organs, imaged by different modalities, are possible given enough and varied labeled data.
• Want to iteratively investigate how a DT (and the associated ecosystem) for Decision- support in Health can be built on top of a SotA cloud platform:
• An increasing amount of open labeled data exists (e.g. from various challenges) and can be used to find an initial model. No privacy issues, ideal for developing an initial prototype / proof-of-concept. CT • Used in a an decision-support setting more high-quality labeled data can be generated as part of the process further increase the accuracy of the predictive models.
• Main field of research (visual intelligence), already have a lot of methods / models that could be put in use, a typical segmentation scenario using DTs with image data could look like this.. X-ray Initial scenario: Example: Medical Image Computing (CT, MR, US, etc.) Update scenario: - Scanning - Donate scan - Stream to DTs - Manually check / update segmentation - Manually segment - More data available for training - Donate - Update model & - Extract donated - Put new and improved in the bank - Find optimal model - Put optimal model in bank sDT_N - Load new data - Apply optimal model
1a) sDT_i 1b) Knowledge Generation - Training - Benchmarking Data from Input from - Challenges Individuals society
Ethics SmartAnalytics & AI/VC sDT_2 Impact & SmartData & Sensors/IoT Impact & benefit for benefit for Security & Privacy individuals: sDT_1 society: smart Digital Twin (sDT) 1) Infrastructure (in the Norwegian Cloud) (cloud computing, datacenter, TPU/GPU/CPU)
1c) Knowledge Bank
Best methods / models (can be applied to the sDT directly) • One citizen - one DT («smart kjerne-journal++»)
• wise, good reasons, collect in stead of distribute
• Knowledge generation: continuously improving and benchmarked («virtual real-time registry»)
• Knowledge use: closing the loop, optimal model bank
• Self-managed citizens, sustainable health-care system, research for improved health (generic knowhow)
• Data sharing under citizen control: selling: don’t think so, mandatory: probably / not necessary, donate: over 90% / would, start here.
• Data use & value creation: data available for all: including the big tech giants..
• Norwegian academia & industry collaboration - think about the possibilities and how rewarding it would be to see it’s own method being put to use. Questions?
Makes sense? Possible? Improvements? Other approaches / strategies? Contribute?
Thank you for the attention