Big data, big impact: how big data is shaping everyday life Introduction We live in an information age, where data is being produced at an ever-increasing rate. 90% of all the data in existence has been created in the last two years, and it’s no longer just businesses and governments who are amassing it. With the birth of the internet and the rise of social media, the ubiquity of smartphones and tablets, and the development of smart fridges and smart scales and devices that can record our physical movements, we are creating vast personal datasets as well. The challenge presented by big data isn’t management and harvesting of crops. just because of its sheer volume; it also Sensors included in shipping containers comes from the variety of the different could monitor the transport of refrigerated types of data involved and the velocity at goods, alerting operators to changes in which it is created. The value of all this conditions; keeping food in optimum data lies in the useful information and storage conditions could greatly reduce the meaning we can extract from it, and big amount of spoilage (estimated at 10-15% data is being used in all aspects of life. and worth $25 billion) and ensure more of In crime prevention it helps the police to our precious food ends up on plates, rather spot trouble before it starts, and facilitates than in the bin. cross-border collaborations that can put Big data is helping us to learn more about an end to international crime. Bankers and the Universe we live in, and to answer insurers can use big data techniques to some fundamental questions. Reaping spot anomalous behaviour, and bring about all of the benefits that big data offers us huge reductions in fraud. means constant innovation in computing The information our smart devices gather and communications. about us can be linked to public health There are various approaches to processing records and allow the development of big data, including using supercomputers, personalised apps that could help monitor distributed computing systems and and manage health conditions like volunteer computing networks that rely on diabetes. Algorithms can unused computer power being work their way through ‘donated’ by members of the a maze of data that’s public. The current generation impenetrable to the of supercomputers is limited by human eye and come up its power consumption; we’re with recommendations developing energy-efficient that can reduce the computing techniques to help amount of money usher in the next generation – the NHS spends on exascale computers that will be prescriptions, reduce able to perform a million trillion waiting lists and track calculations every second. STFC patient movements have a long history of high (without compromising performance computing, and healthcare). The Part of STFC Supercomputer. Credit: STFC we’re proud to be building on human genome project took 13 years and that and being part of the team pushing $3 billion to sequence the first human boundaries to develop the technologies genome. New DNA sequencing machines needed for ambitious science projects such can do the same thing in one day, bringing as the Square Kilometre Array (SKA) and the cost per genome below $1000 and the Gaia galactic survey. making it possible to tailor treatments The case studies included in this brochure to both the patients and the disease are just a small fraction of the ones we (choosing the most suitable cancer drugs, could tell, but they illustrate the UK’s for example). industrial and research strengths in this In agriculture big data can be used arena, and show why we are ideally placed to track bovine health, and to make to deal with the big data challenges of the recommendations for the planting, future. Science and research Artists’ impression of the SKA Dish arrays in operation at night. Credit: SKA Organisation 1 SKA Science and research The Square Kilometre Array (SKA) will be the world’s largest and most sensitive radio telescope. Thousands of radio wave receptors (antennae) will combine to allow the SKA to see back into the early universe, before the stars were formed and there was only gas. It will allow researchers to investigate a wide range of fundamental questions in physics, astronomy, cosmology and astrobiology, exploring distant parts of the Universe for the first time. The SKA presents unprecedented big the volume of data, and which will require data challenges. It digitally combines enough fibre optic cable to encircle the the signals from each antenna – using Earth twice. Clocks at each antenna will powerful supercomputers – to provide a have to be synchronised to a thousand- virtual telescope with a collecting area of a billionth of a second, which is no mean feat square kilometre. It will be 50 times more when you discover that they are spread powerful than any existing radio telescope. across two continents, with SKA sites in The data it will collect in just one day will Australia and South Africa. be enough to fill 15 million 64 Gb mp3 The SDP consortium has an equally players; it would take nearly two million challenging job, focusing on the technology years to play back on an iPod. The SKA that’s needed to turn the data collected central computer will need to have the into useable science products. The SKA processing power of over a 100 million project is driving technology development PCs. The dishes will produce ten times in antennae, data transport, software the current amount of internet traffic; the and computing and power – all of completed aperture arrays will multiply which will have enormous commercial that by ten again. potential. SKA technologies could lead to The Government has committed faster smartphones and higher internet £100 million to the SKA project, and the speeds, and the UK’s role in developing Universities of Manchester, Cambridge cutting-edge data analysis techniques and Oxford are playing an active role in will give us a competitive advantage in a several aspects of the design process. The global market that is expected to be worth UK is leading two consortiums – Signal and £31 billion by 2016. Data Transport (SaDT) and Science Data Processor (SDP). The SaDT consortium “The SKA will be the largest and most is responsible for the design of the data sensitive radio telescope in the world, transport networks that will have to handle stretching data analysis and science technologies to their limits.” Dr Benjamin Stappers, University of Manchester Background image credit: SKA Organisation/Swinburne Astronomy Productions Science and research 2 Artists’ impressions of Gaia. Credit: ESA Gaia The Gaia spacecraft was launched in December 2013, and made its way to an orbit at L2 (a Lagrange point) from where it is best placed to get a good view of the galaxy. Gaia’s mission, scheduled to last five years, is to perform the most detailed survey of the Milky Way. It will be charting the position, motion, luminosity, temperature and composition of a billion stars, creating an extraordinarily precise 3D map of our galaxy. The spacecraft will observe each target star 80 times over five years, making an average of 40 million observations every day. The UK helped to build the spacecraft itself, computers, essential to process the torrent and the very special billion-pixel camera of data, were produced by Astrium in that gives Gaia an unprecedented view Stevenage. of the galaxy; UK industry and science Why go to all this trouble? Gaia’s stellar institutes won around €80 million in census will provide the data needed to contracts. As well as the considerable answer a wide range of questions relating engineering challenge involved in building to the origin, history and structure of the Gaia, the UK is involved in the technical Milky Way. By examining the large-scale challenge of processing the vast amount motion of stars, it will also show us the of data into useable science products. distribution of dark matter, the otherwise The Data Processing and Analysis invisible substance that is thought to hold Consortium (DPAC) is the pan-European our galaxy together. Gaia’s mission will team of scientists and software developers improve our understanding not only of the responsible for processing Gaia’s data and solar system and our galaxy, but also of producing the Gaia catalogue. The data the fundamental physics underpinning the will be processed in six data centres, each entire Universe. of which is dedicated to a different aspect of the data. The Cambridge data centre is responsible for processing photometric data. Gaia’s data will be transmitted ‘raw’ to “We expect that the photometric data Earth for processing. Even after it has been processing software to which we have compressed, the amount of data produced contributed as part of the UK-led team during the five-year mission would fill will offer the first opportunity ever to more than 30,000 CD ROMS. Processing precisely measure the brightness of the the data is such a complex process that it billion objects that GAIA will see.” has to be automated. The UK will be at the Dr Peter Allan forefront of processing Gaia’s images, and Head of the Space Data Division STFC helped to set up the data applications at RAL Space centre. Gaia’s powerful on-board Science and research 3 Climateprediction.net Science and research The winter of 2013/14 proved to be an unusually wet one for the southern UK – the wettest in 250 years. Most of us would prefer not to relive it, but climate researchers from the University of Oxford are doing just that, several thousand times over.