Project acronym: BYTE Project title: Big data roadmap and cross-disciplinarY community for addressing socieTal Externalities Grant number: 285593 Programme: Seventh Framework Programme for ICT Objective: ICT-2013.4.2 Scalable data analytics Contract type: Co-ordination and Support Action Start date of project: 01 March 2014 Duration: 36 months Website: www.byte-project.eu Deliverable D1.1: Understanding and mapping big data Rajendra Akerkar and Guillermo Vega-Gorgojo, University of Oslo Author(s): Grunde Løvoll, DNV GL AS Stephane Grumbach and Aurelien Faravelon, INRIA Rachel Finn, Kush Wadhwa, and Anna Donovan, Trilateral Research & Consulting Lorenzo Bigagli, National Research Council of Italy Dissemination level: Public Deliverable type: FINAL Version: 3.0 Submission date: 31 March 2015 D1.1: Understanding and mapping of big data Table of contents 1 INTRODUCTION ............................................................................................................... 5 1.1 Abstract ......................................................................................................................... 5 1.2 Purpose and scope ........................................................................................................ 5 1.3 Target audience ............................................................................................................ 5 1.4 Methodology ................................................................................................................. 5 2 BIG DATA CONCEPT AND ITS ORIGIN ....................................................................... 7 3 DEFINING BIG DATA .................................................................................................... 10 3.1 Common Definitions .................................................................................................. 10 3.2 Working big data definition for BYTE ...................................................................... 13 4 OPPORTUNITIES & CHALLENGES ............................................................................. 15 4.1 Opportunities .............................................................................................................. 15 4.2 Challenges .................................................................................................................. 16 4.2.1 Data acquisition and storage ................................................................................ 18 4.2.2 Data communication ............................................................................................ 19 4.2.3 Data management and curation ........................................................................... 20 4.2.4 Data analysis ........................................................................................................ 21 4.2.5 Data visualization ................................................................................................ 21 4.2.6 Human role in life cycle of big data .................................................................... 23 5 BIG DATA APPLICATIONS ........................................................................................... 26 5.1 Big Data in Business .................................................................................................. 26 5.1.1 Changing industries with big data ....................................................................... 30 5.1.2 Internet of things and big data ............................................................................. 30 5.1.3 The industrial Iinternet of things ......................................................................... 31 5.1.4 Examples of typical big data use cases ................................................................ 32 5.2 Big Data in Science .................................................................................................... 35 5.2.1 Examples of typical big data use cases ................................................................ 35 5.3 Big Data in Public Administration ............................................................................. 37 5.3.1 Examples of typical big data use cases ................................................................ 38 6 BIG DATA DEFINITIONS IN SELECTED SECTORS ................................................. 41 6.1 Big Data Definition in Oil & Gas Sector .................................................................... 41 6.1.1 Big Data Applications in Oil & Gas Sector ......................................................... 42 6.1.2 Big Data Challenges in O&G Sector ................................................................... 45 6.2 Big Data Definition in Healthcare .............................................................................. 47 6.2.1 Big Data Applications in the Healthcare Sector .................................................. 48 2 D1.1: Understanding and mapping of big data 6.2.2 Big Data Challenges in the Healthcare Sector .................................................... 49 6.3 Big Data Definition in environment ........................................................................... 51 6.3.1 Big data applications in environment sector ....................................................... 51 6.3.2 Big data challenges in environment sector .......................................................... 53 6.4 Big Data Definition in Crisis Informatics .................................................................. 54 6.4.1 Big data definitions and typologies ..................................................................... 54 6.4.2 Big data applications in crisis informatics ........................................................... 58 6.4.3 Big data challenges in the crisis informatics sector ............................................. 63 6.5 Big Data Definition in Smart Cities ........................................................................... 67 6.5.1 Big Data Applications in the Smart Cities Sector ............................................... 69 6.6 Big Data Definition in Shipping ................................................................................. 74 6.6.1 Big Data Applications in the Shipping Sector ..................................................... 76 6.7 Big Data Definition in Culture ................................................................................... 79 6.7.1 Defining big cultural data .................................................................................... 80 6.7.2 Big data applications in the cultural sector .......................................................... 83 7 CARTOGRAPHY OF DATA FLOWS ............................................................................ 87 7.1 Country study ............................................................................................................. 87 7.1.1 USA ..................................................................................................................... 88 7.1.2 China .................................................................................................................... 88 7.1.3 France .................................................................................................................. 89 7.1.4 Korea ................................................................................................................... 90 7.1.5 Egypt .................................................................................................................... 90 7.1.6 Brazil ................................................................................................................... 91 7.1.7 Global perspective ............................................................................................... 91 8 SUMMARY ...................................................................................................................... 93 REFERENCES ........................................................................................................................ 96 3 D1.1: Understanding and mapping of big data List of Figures Figure 1 The graphics illustrating big data (Source: Expand Your Digital Horizon With Big data by Brian Hopkins and Boris Evelson) ...................................................................................................................................... 11 Figure 2 A knowledge-discovery lifecycle for big data ......................................................................................... 17 Figure 3 EY’s 2013 Global Information Security Survey ....................................................................................... 26 Figure 4 General Electric’s vision of industrial internet (source: http://www.ge.com/stories/industrial-internet) .................................................................................................................................................................... 32 Figure 5 Oil & Gas supply chain (Source: S. Oladunjoye, R. Price, N. Rahman, and J. Wells, Transfer pricing Transfer pricing in the oil & gas sector- A primer, International Tax Review, July 2012) ........................... 41 Figure 6 Type of trade and transaction flows in O&G industry ............................................................................ 42 Figure 7: Big data application areas in crisis informatics ..................................................................................... 59 Figure 8 Overview of generating meaning via analytics (Data Warehousing vs.