DOCSLIB.ORG

Sensors and Instrumentation Dr Peter M Allan Director, Hartree Centre, STFC Overview

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Sensors and Instrumentation Dr Peter M Allan Director, Hartree Centre, STFC Overview Computational and Data Challenges in Environmental Modelling Sensors and Instrumentation Dr Peter M Allan Director, Hartree Centre, STFC Overview • Hartree Centre – what are we • Sensors and instrumentation • The data challenge The Hartree Centre • Part of the STFC National Laboratories – Close working with the Scientific Computing Department is essential to delivering our mission • Our mission – Provide market led research and innovation and development services for competitive advantage and wider economic impact by accelerating the application of High Performance Computing, Data Science, and Cognitive techniques The Hartree Centre Establishment 2012 Creation of Hartree Centre as industrial driver in HPC • 2011 Tildesley Report recommendation • Investment from BIS • Economic impact through Software & Modelling • Focus on industrial engagement 2015 Major Investment in Collaborative Research • Data Centric & Cognitive Computing • Embedded IBM Research Centre • Extended industrial & scientific reach What is the Hartree Centre? Joint Centre for Collaborative R&D in the UK Research Innovation Development Deployment IBM Research Division STFC Daresbury Laboratory IBM UK Research Industry Public Sector Science Academia Government Story so far Mission Phase 1 – HPC Enablement ⁻ Industrial engagement leveraging STFC scientists ⁻ Leading Research infrastructure for software development and industrial support ⁻ Collaboration with IBM for global insights Mission Phase 2 ⁻ Research into Energy efficient computing ⁻ Expanding HPC into Big Data Analytics ⁻ Building global partnerships Results ⁻ Completed 100+ engagements ⁻ 33 Collaborative R&D projects completed ⁻ 27 published case studies ⁻ 6 joint IBM / Hartree research projects ⁻ Pioneering convergence of HPC & Big Data Analytics ⁻ Intel Parallel Computing Centre ⁻ IBM / Lenovo Benchmarking Centre ⁻ IBM / nVidia Development Centre ⁻ MOU and partnership with Lawrence Livermore ⁻ MSc with University of Liverpool Mission Phase 3 – R&D programme ⁻ Develop tools needed by industry in the future Hartree Centre Clients Sensors • Location – Land, air, sea, beneath the surface, space, everywhere! • In situ or remote sensing • Space – Instruments on satellites – Mostly remote sensing • Everywhere – Internet of Things – Mostly in situ measurements Sensors in Space • Earth Observation – Mostly from satellites in low Earth orbit (several hundred km altitude) – Some from geostationary orbit (weather satellites) • Space Weather – Solar storms • Both remote sensing and in situ Sierra Leone River Estuary © Copernicus Sentinel data (2015)/ESA Sensors in Space • Sentinel satellites from ESA/EU – S-1 : Synthetic Aperture Radar (5GHz) – S-2 : Multispectral imager (optical, IR) – S-3 : Sea and Land Surface Temperature Radiometer Ocean and Land Colour Altimetry (dual band SAR) Microwave radiometer – S-4&5 : Composition of atmosphere – S-6 : Radar altimeter for ocean topography Advantages of Sensors in Space • Continuous stream of data covering the whole globe • Can be very accurate – SLSTR will measure temperatures accurate to 0.1K • Cost-effective if you need global coverage • Data often provided free at the point of use for non- commercial applications • Data is well curated Disadvantages of Sensors in Space • Repetition rate of data from a given place typically several days (but see later) • If something breaks, you generally can’t fix it • Long time from conception to delivery of first data Faster delivery of data from space • Geostationary Orbit – High repetition rate, but lower spatial resolution • Constellations of satellites – Surrey Satellites’ DMC – Planet Labs, ~100 cubesats • Relay satellites – European Data Relay System • Laser inter-satellite communications Internet of Things • Sensors are everywhere – Low cost – Rapid deployment – May be owned by individuals or organisations • Data are of variable or unknown accuracy • Sensors may be active at variable times • When are such uncontrolled sensors of practical use? Data Intensive Solutions • What type of data best address your problem? • Do you need multiple types of data to even get you started? • Do other types of data improve your solution or are they a distraction? • Who are the experts in data that you have not used before? • Who is going to use your solution and what will they do with it? Big Data Challenge • Satellite data are large volume – Well structured, standard formats – How to access the large volume of data? • Provide computing next to the data (JASMIN/CEMS) • IoT sensors have high variety • The Challenge – How best to make use of the increasing number and variety of data sources? – There are many experts in the use of some data sources, but few in exploiting the full range Questions? .
Load more

Recommended publications
  • High Performance Computing and BD
    High-Performance Computing and Big Data Challenge Dr Violeta Holmes Matthew Newall The University of Huddersfield Outline • High-Performance Computing – E-Infrastructure – Top500 -Tianhe-II – UoH experience: HPC resources and applications • Big Data – Hadoop – Case study: Close Call • Summary High-Performance Computing • The HPC and Big Data are new technologies vital for the advancement in science, business and industry. • ”High Performance Computing (HPC) and e- infrastructure in general are drivers of economic growth and societal well-being. They are also vital for maintaining international competitiveness in the generation of knowledge and its application.”(Kenway, Coveney, McGuire, Parchment, & Parsons, 2012) Where we were 50s-early 80s • Centralised computing facilities • Dummy Terminals • Punch cards • Batch Processing μ Processors • Allowed users to have “intelligent” machines under their table – Seen as an expensive typewriter – Few organisations and educational institutions had them • These machines were quickly overtaking Mainframes in price-performance factors, and later overtaking them in performance. 90s – early 2000 • Information on Demand - For the first time sharing and communicating became easy and quick. Where we are now • Efficient Multitasking Desktops • Capable Workstations • Mobile Devices – less capable Through the use of middlewares: • Clusters • Grids The 5th Utility Utility Computing/Cloud Computing/On- Demand Computing/Service Computing • The concept of just plugging in your problem into a computer system and getting results. • Defined as: “A large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted virtualised, dynamically-scalable, managed computing power, storage, platforms and services are delivered on demand to external customers over the internet.” (Foster et al.
    [Show full text]
  • Intel Parallel Computing Centre – First Report the Hartree Centre
    Benefits of Leveraging Software Defined Visualization (OSPRay) Sergi Siso IPCC @ Hartree Centre, STFC Daresbury Laboratory United Kingdom Thursday, 23 March 2017 Software Defined Visualization 2 Index 1. Visualization Challenges in HPC 2. Ray tracing with OSPRay 3. Hands-On 1: Installing and using OSPRay 4. Remote Visualization with Paraview 5. Hand-One 2: Remote vis. of OpenCH and HVAC 6. Hartree Use Case 1: HVAC simulation 7. Hartree Use Case 2: IMAT Facility 8. Questions Thursday, 23 March 2017 Software Defined Visualization 3 Section 1 VISUALIZATION CHALLENGES IN HPC Why we need visualization in HPC “Yes," said Deep Thought, "I can do it.“ [Seven and a half million years later.... ] “The Answer to the Great Question... Of Life, the Universe and Everything... Is... Forty-two,' said Deep Thought, with infinite majesty and calm.” Douglas Adams, Hitchhiker’s Guide to the Galaxy Thursday, 23 March 2017 Software Defined Visualization 5 Why we need visualization in HPC Thursday, 23 March 2017 Software Defined Visualization 6 But not just rendering . In HPC visualization incudes: – Feature extraction – Isosurfaces / Isovolumes – Thresholding – Streamlines – Clip, Slice, gradient, … Some HPC visualization workloads are better suited to CPU . Visualization is a “big data” problem architectures. – Data movement problem (IO bottleneck) – Needs lots of memory – Need scalability to solve Thursday, 23 March 2017 Software Defined Visualization 7 Dataset Size Problem . Issues to simulate large datasets… May fit in System Memory but not in GPU memory PCI bus is slow Thursday, 23 March 2017 Software Defined Visualization 8 Rasterization Libraries OpenSWR vs MESA LLVMpipe Performance Comparison (drop in replacement for OpenGL) Thursday, 23 March 2017 Software Defined Visualization 9 Rasterization Libraries *Scientific Visualization in HPC presentation from Peter Messmer (NVIDIA) Thursday, 23 March 2017 Software Defined Visualization 10 In-Situ Visualization .
    [Show full text]
  • Hartree Centre Challenge Led Applied Research and Innovation Our Mission
    The Hartree Centre Challenge led applied research and innovation Our mission Transforming UK industry by accelerating the adoption of high performance computing, big data and cognitive technologies. What we do − Collaborative R&D Define a challenge in your business and we build a team to deliver a solution − Platform as a service Give your own experts pay-as-you-go access to our compute power − Creating digital assets License the new industry-led software applications we create with IBM Research − Training and skills Drop in on our comprehensive programme of specialist training courses and events or design a bespoke course for your team Our platforms Our platforms Intel platforms Accelerated & emerging tech Bull Sequana X1000 Maxeler FPGA system Lenovo NeX​tScale | 8,192 cores ARM​ 64-bit platform​ Lenovo System x​​ iDataPlex​ system​ | 2048 cores Clustervision novel cooling demonstrator​ Intel Xeon Phi | Knight's Corner IBM big data analytics cluster | 288TB​ IBM data centric platforms​ IBM Power8 + NVL​ink​ + Tesla P100 IBM Power8 ​​+ Nvidia K80​​​ How you can benefit − Access to domain specialists and a vast network of expertise − Technologies and systems that do it faster and cheaper − Full time scientists delivering on industrially relevant timescales − Collaboration with a trusted partner − Professional project management Government & public sector Network of expertise Local business networks Technology partners Universities Academia International research communities Our track record Case study | Computer aided formulation Collaborative
    [Show full text]
  • Science, Technology and Innovation Exploiting the Excellence
    sciencescience technologytechnology scienceandand innovationinnovation technology and innovation Science, Technology and Innovation Exploiting the excellence England’s North West: where world-class science underpins wealth-creating innovation www.nwblt.co.uk @nwblt A5 NWBLT ET:Layout 1 14/3/14 15:22 Page 1 ‘Exploiting The Excellence’ England’s North West: where world-class science underpins wealth-creating innovation In collaboration with local enterprise partnerships, local authorities, academic and research institutions and other North West partners March 2014 A5 NWBLT ET:Layout 1 14/3/14 15:22 Page 2 NWBLT The North West Business Leadership Team is an independent group of influential business leaders, who work together to promote the sustainable economic development and long-term well-being of North West England. It was launched by HRH The Prince of Wales in July 1989. Membership of the Team is made up of senior executives from the region’s major businesses (see Appendix E). It operates as an independent business voice for the whole North West, thus providing valuable strategic support for the region’s Local Enterprise Partnerships. Further information regarding the work of the North West Business Leadership Team can be obtained by writing to the Chief Executive, NWBLT, Daresbury Laboratory, Sci-Tech Daresbury, Keckwick Lane, Daresbury, Warrington WA4 4AD or by visiting www.nwblt.co.uk A5 NWBLT ET:Layout 1 14/3/14 15:22 Page 3 Contents Executive Summary 3 1. Introduction 4 2. Scientific and Technological Excellence in the North West 6 2.1 Advanced materials and molecular manipulation 2.2 Energy research, including nuclear energy and nuclear engineering 2.3 Cancer research and health innovation (including translation into patient care) 2.4 High performance computing (HPC), big data and digital manufacturing 3.
    [Show full text]
  • Scafell Pike - Public Hpcaas Offering to UK Academics and Industry Image Courtesy of STFC
    AI and HPC as Drivers for Industrial Competitiveness Alison Kennedy Director | STFC Hartree Centre Who are we? • The UK Science and Technology Facilities Council’s high performance computing, data analytics and cognitive technology centre • Primary purpose is to apply the latest capabilities in computing technologies to industrial challenges • Provides businesses (and applied researchers) with access to powerful technologies, facilities and scientific computing expertise • Based at Sci-Tech Daresbury alongside Daresbury Laboratory in North West England Our mission Transforming UK industry by accelerating the adoption of high performance computing, big data and cognitive technologies (AI, ML, DL) through challenge-led research and innovation Our purpose − To lead UK research into the (industrial) application of high performance computing, data & cognitive technologies (with IBM Research, ATOS Bull and other partners) − To drive the economic development of UK industry through the use of leading edge scientific knowledge and applications What we do − Collaborative R&D We build a team to deliver a solution to a particular challenge − Platform as a service Pay-as-you-go access to our compute power − Creating digital assets License the new industry-led software applications we create with IBM Research (IROR programme) − Training and skills Run specialist training courses and events Our track record Case study | Computer aided formulation Collaborative R&D Faster development process for products like shampoo, reducing physical testing “The Hartree Centre’s high performance computing capabilities help us achieve better design solutions for our consumers, delivered by more efficient, cost-effective and sustainable processes.” − Paul Howells, Unilever Consumable HPC Service provider User Getting HPC to “work smart, not hard” − Typically HPC development is focused on increased speed.
    [Show full text]
  • The Hartree Centre
    The Hartree Centre Andrew Gargett AI team lead | STFC Hartree Centre Our mission Transforming UK industry by accelerating the adoption of high performance computing, big data and cognitive technologies. Our platforms Intel platforms Accelerated & emerging tech Bull Sequana X1000 Maxeler FPGA system Lenovo NeX tScale | 8,192 cores ARM 64-bit platform Lenovo System x iDataPlex system | 2048 cores Clustervision novel cooling demonstrator Intel Xeon Phi | Knight's Corner IBM big data analytics cluster | 288TB IBM data centric platforms IBM Power8 + NVL ink + Tesla P100 IBM Power8 + Nvidia K80 Our people − Engineers − Chemists − Software − Life Scientists Developers − Mathematicians − Data Scientists Project examples • Engineering & Manufacturing – Vehicle Design & Testing – Consumer Electronics Design – Consumer Packaged Goods Products and Packaging • Environment – Weather modeling • Life Sciences – Genomics for better crop yields – Disease mapping • Energy – Advanced Battery Cell Design – Efficient Well Head Oil extraction • Financial Services – Risk Management – Service Modelling • Transport – Network simulation Case study | Building the cognitive hospital Collaborative R&D Transforming the patient experience using cognitive technology and data analytics “Helping our patients and their families prepare properly for coming into hospital will really reduce their anxiety and could mean they spend more meaningful time with doctors so we are able to get them better faster.” − Iain Hennessey, Alder Hey Children’s Hospital LCR 4.0: CiiVSOFT CiiVSOFT build software tools which automate recruiting tasks, save costs, time and improve the efficiency of recruitment processes. Problem : Wanted to explore the potential of using social media analytics to optimise matching candidates to roles. Solution : Hartree Centre developed a working prototype application that integrates social media data into the candidate selection pipeline.
    [Show full text]
  • STFC's Scientific Computing Department and the Hartree Centre
    STFC’s Scientific Computing Department and The Hartree Centre Paul Sherwood [email protected] Overview • Background to STFC and its Facilities • STFC’s Scientific Computing Department • STFC Hartree Centre: future developments to meet future challenges Organisation HM Government (& HM Treasury) RCUK Executive Group Daresbury Laboratory UK Astronomy Technology Daresbury Science and Innovation Campus Centre, Edinburgh, Scotland Warrington, Cheshire Polaris House Swindon, Rutherford Appleton Laboratory Wiltshire Harwell Science and Innovation Campus Didcot, Oxfordshire Chilbolton Observatory Stockbridge, Hampshire STFC’s Sites Isaac Newton Group of Telescopes Joint Astronomy La Palma Centre Hawaii STFC’s Facilities Neutron Sources ISIS - pulsed neutron and muon source/ and Institute Laue-Langevin (ILL), Grenoble Providing powerful insights into key areas of energy, biomedical research, climate, environment and security. High Power Lasers Central Laser Facility - providing applications on bioscience and nanotechnology HiPER Demonstrating laser driven fusion as a future source of sustainable, clean energy Light Sources Diamond Light Source Limited (86%) - providing new breakthroughs in medicine, environmental and materials science, engineering, electronics and cultural heritage European Synchrotron Radiation Facility (ESRF), Grenoble Understanding our Universe STFC’s Science Programme Particle Physics Large Hadron Collider (LHC), CERN - the structure and forces of nature Ground based Astronomy European Southern Observatory (ESO), Chile
    [Show full text]
  • Hartree Centre Phase 1&2 Baseline Evaluation
    Hartree Centre Phase 1 & 2 Baseline Evaluation Hartree Centre Phase 1&2 Baseline Evaluation Final Report technopolis |group| October 2018 Paul Simmonds Neil Brown Cristina Rosemberg Vivek Seth Disclaimer This report has been prepared for the UK Science and Technology Facilities Council (STFC) by Technopolis |Group| and represents the views of the authors only. To the best of our knowledge, the information presented is correct at the time of writing. Whilst finalising this report, updated guidance on appraisal and evaluation in central government was published by HM Treasury (The Green Book, March 2018). The evaluation itself (including all data collection and analysis) was undertaken before this date and so follows the previous edition of the Green Book (2003) Acknowledgements The authors would like to thank those who gave of their time to provide inputs to the study, as well as representatives of STFC and the Hartree Centre for the information, guidance and support that they provided throughout the evaluation. www.technopolis-group.com Executive Summary This document presents the findings of a baseline evaluation of the Hartree Centre; a high performance computing (HPC) and data analytics research centre that is working to transform the competitiveness of UK industry by accelerating the adoption of HPC, big data and cognitive technologies. It provides an early view of the benefits the centre is delivering to its industry partners and the wider economy through an assessment of its first years of operation (phase 1 and 2 investments). Key
    [Show full text]
  • The Northern Powerhouse in Health Health Science Alliance Ltd Research - a Science and Innovation Audit
    The Northern Submitted by the Northern Powerhouse in Health Health Science Alliance Ltd Research - A Science and Innovation Audit Main Report June 2018 Prepared by: Prepared for: SDG Economic Development Submitted by the Northern Health 61 Mosley Street Science Alliance Ltd Manchester M2 3HZ C/O Weightmans LLP, Hardman Square No 1 Spinningfields, Manchester, M3 3EB +44 161 261 9154 www.sdgED.com Steer Davies Gleave has prepared this material for Submitted by the Northern Health Science Alliance Ltd. This material may only be used within the context and scope for which Steer Davies Gleave has prepared it and may not be relied upon in part or whole by any third party or be used for any other purpose. Any person choosing to use any part of this material without the express and written permission of Steer Davies Gleave shall be deemed to confirm their agreement to indemnify Steer Davies Gleave for all loss or damage resulting therefrom. Steer Davies Gleave has prepared this material using professional practices and procedures using information available to it at the time and as such any new information could alter the validity of the results and conclusions made. The Northern Powerhouse in Health Research - A Science and Innovation Audit | Main Report Contents 1 Introduction to the North . and our SIA’s footprint .......................................................... 1 Headline Messages ....................................................................................................................... 1 Our Place in a Nutshell.................................................................................................................
    [Show full text]
  • STFC Corporate Powerpoint Template
    The Hartree Centre Neil Morgan Smarter Innovation at Scale 1 The Hartree Centre 2011 Tildesley Report Economic impact through software & modelling UK-wide investment from BIS 2012 Creation of the Hartree Centre Focus on industrial engagement Partnership with IBM 2014 Phase 2 investment Big data expansion & EEC R&D 2015 Major Investment in R&D with IBM Research Data Centric & Cognitive Computing Embedded IBM Research Centre Extended industrial & scientific reach 2 “To transform UK competitiveness by facilitating and accelerating the adoption of data-centric computing and cognitive technologies.” Lenovo / Arm development system Research & innovation Development & Better products & services Deployment through collaboration prototyping delivered faster & cheaper Our mission 3 • Collaborative Projects • Software & Algorithms • Training & Skills • Platform as a Service • R&D - Proof of Technology Our services 4 • Engineers • Chemists • Life Scientists • Mathematicians • Data Scientists • Software Engineers • Visualisation/VR/AR Engineers credit: istock Our people 5 Example Projects • Engineering and manufacturing – Vehicle design and testing – Consumer electronics design – Consumer goods products and packaging • Environment – Ocean and weather modelling • Life Sciences – Genomics for better crop yields – Disease mapping • Energy – Advanced battery cell design – Efficient well head oil extraction • Financial Services – Risk management – Service modelling • Transport – Network simulation 6 “Integrated Data Analytics and Modeling & Simulation Platforms” Ideas Proof of Proof of Roll Out • Creation of Technology Concept • Scale out as new ideas • Feasibility • Capability a robust • Minimal • Demonstrate service, viable to a user business or product base the licence value application Closing the loop between experiment andHartree comp Centreutati oPipelinen 7 Different Types of Workflows, Different Solutions Data Centric Conceptual View of Data Intensive-Integer Workflow Applications e.g.
    [Show full text]
  • The Hartree Centre: Experience in Addressing Industrial, Societal and Scientific Challenges
    The Hartree Centre: Experience in Addressing Industrial, Societal and Scientific Challenges Prof Vassil Alexandrov Chief Science Officer Our mission Transforming UK industry by accelerating the adoption of high performance computing, big data and AI. Hartree Centre Mission: To transform UK industry by accelerating the adoption of high performance computing, data-centric computing and AI. Lenovo / Arm development system Research & innovation Development & Better products & services Deployment through collaboration prototyping delivered faster & cheaper What we do − Collaborative R&D Address industrial, societal and scientific challenges. − Platform as a service Give your own experts pay-as-you-go access to our compute power − Creating digital assets License the new industry-led software applications we create with IBM Research − Training and skills Drop in on our comprehensive programme of specialist training courses and events or design a bespoke course for your team Government & public sector Network of expertise Local business networks Technology partners Universities Academia International research communities Hartree Research Themes Data Science HPC AI Emerging Technological Paradigms Our track record Case study | Code optimisation for aero engines Collaborative R&D Hartree Centre experts optimised codes for modelling component design, speeding up run times by approx 20% “Working with the Hartree Centre, we have quickly made significant improvements to our code, delivering faster turnaround and more capability to our engineers.” −
    [Show full text]
  • Introduction to HPC and Parallel Architectures: Software Mike Ashworth
    Introduction to HPC and parallel architectures: software Mike Ashworth Head of Technology Division Scientific Computing Department and STFC Hartree Centre STFC Daresbury Laboratory [email protected] Overview Hardware and software: from evolution to revolution STFC’s Scientific Computing Department STFC Hartree Centre: future developments to meet future challenges Opportunities for Discussion Overview Hardware and software: from evolution to revolution STFC’s Scientific Computing Department STFC Hartree Centre: future developments to meet future challenges Opportunities for Discussion TOP500 Global Trends from Jack Dongarra, IESP from Jack Dongarra, IESP Limits Reached? “It would appear that we have reached the limits of what is possible to achieve with computer technology, although one should be careful with such statements, as they tend to sound pretty silly in five years” John von Neumann, 1949 Technology Challenges: The Power Wall Transistor density is still increasing Clock frequency is not due to power density constraints Cores per chip is increasing, multi-core CPUs ( currently 8-16) and GPUs (500) Little further scope for instruction level parallelism Source: Intel, Microsoft (Sutter) and Stanford (Olukotun, Hammond) Increase in the number of cores Increase in the number of cores 1 system has >1,000,000 cores 21 systems have >100,000 cores Exascale Reports DARPA Exascale Computing Report Exascale systems achievable by 2017 Little increase in clock rate 10M-100M processor cores Heterogeneous (1:100 full instruction set to lightweight cores) 3D packaging with memory close to cores Mixed optical plus electrical interconnect 10PB-100PB memory Fault tolerance with support in hardware and software For total system power of 10-20 MW, need 50-100 GF/W cf.
    [Show full text]