275G /grassdata/eu_laea/modis_lst_reconstructed_europe_monthly 1.1T /grassdata/eu_laea/modis_lst_reconstructed_europe_weekly 2.0T /grassdata/eu_laea/modis_lst_reconstructed_europe_GDD 3.6T /grassdata/eu_laea/modis_lst_reconstructed_europe_daily 4.8T /grassdata/eu_laea/modis_lst_reconstructed 55G /grassdata/eu_laea/modis_lst_validation_europe 15G /grassdata/eu_laea/modis_lst_reconstructed_europe_winkler 38G /grassdata/eu_laea/modis_lst_reconstructed_europe_monthly_averages 40G /grassdata/eu_laea/modis_lst_reconstructed_europe_bioclim 22G /grassdata/eu_laea/modis_lst_reconstructed_europe_annual 48G /grassdata/eu_laea/modis_lst_koeppen ... processing big geospatial data geospatial big processing Efficiently 7: GIS GRASS http://grass.osgeo.org http://gis.cri.fmach.it E. – Mach CRI, Italy Fondazione Development Team & Neteler GRASS Markus FOSDEM 2015, Brussels FOSDEM 31 Jan & 1 Feb 2015 1Feb & 31 Jan © 2015, Markus Neteler, Italy – CC-BY-SA license What you think it is...it think you What The new release: GRASS GIS 7 User User 7 GIS GRASS release: new The interface © 2015, Markus Neteler, Italy – CC-BY-SA license What it really is...really it What is...it think you What The new release: GRASS GIS 7 User User 7 GIS GRASS release: new The Volumes Vector, Raster, interface series time LiDAR © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: Map histogram tool histogram Map 7: GIS GRASS Histogram in Additionally: legend © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: New Geospatial Modeller Geospatial New 7: GIS GRASS Extra bonus: Extra Export it as a Python a as it script Export © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: Supervised image GIS classification GRASS regions of interest. of interest. regions for an image by allowing Generates spectral signatures userthe to outline for supervised classification of imagery data. Tool http://geo.fsv.cvut.cz/~landa/publications/2012/ogrs2012/poster/figures/ © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: Unsupervised image Unsupervised GIS 7: classification GRASS i.segment - Identifies segments (objects) from imagery data. data. imagery from (objects) segments Identifies - Pietro Zambelli Pietro © 2015, Markus Neteler, Italy – CC-BY-SA license Vector dataprocessing Vector © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: Topological Vector Digitizer Vector Topological 7: GIS GRASS as single boundaries (= shared). Vector topology in a nutshella in topology Vector Common boundaries between two adjacent areas are stored stored are two areas adjacent → Simplifies vector data maintenance (nomaintenance gaps and and slivers possible) © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: Topological Vector Digitizer in Digitizer Vector Topological 7: GIS GRASS PostGIS 2 PostGIS http://grass.osgeo.org/grass70/manuals/v.out.postgis.html http://grasswiki.osgeo.org/wiki/PostGIS_Topology (under development) Cofunded by Municipality of Trento, Italy Cofunded by Municipality of Trento, Programmer: MartinLanda © 2015, Markus Neteler, Italy – CC-BY-SA license Vector network analysis in GRASS GIS 7 GIS in GRASS analysis network Vector Programmer: Stepan Turek Programmer: © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7 Temporal Framework: GIS 7 Temporal GRASS Time-series support Time-series © 2015, Markus Neteler, Italy – CC-BY-SA license New Space-Time functionality in GRASS GIS 7 GIS inGRASS functionality Space-Time New environmental modeling E., 2014. S.,Gebbert, Pebesma, . Environmental Modelling & Software 53, 1-12. ( 1-12. 53, Modelling & Software . Environmental TGRASS: A temporal forGIS field based TGRASS: DOI ) © 2015, Markus Neteler, Italy – CC-BY-SA license New Space-Time functionality in GRASS GIS 7 GIS GRASS in functionality Space-Time New Time series plot (Chlorophyll vs Time) forseries certaina plot coordinate(Chlorophyll pairTime) vs Time Screenshot:S Gebbert/A.Petrasova (by Veronica Andreo) Veronica (by © 2015, Markus Neteler, Italy – CC-BY-SA license Visualization © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS 7: New animation tool for time series time for tool animation New 7: GRASS http://grass.osgeo.org/grass70/manuals/g.gui.animation.html over 9 years (NC, USA) series: moving dune time LiDAR Nagshead By AnnaBy Petrasova © 2015, Markus Neteler, Italy – CC-BY-SA license New Map swiping tool for multitemporal maps multitemporal for tool swiping Map New (MODIS images taken on February 26 and March 13, 2011) 13, and March 26 on February (MODIS taken images images disaster post Pre and of the tsunami in Japan in 2011 © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS 7: New visualization tool: wxNVIZ tool: visualization New 7: GRASS http://grasswiki.osgeo.org/wiki/WxNVIZ Programming/screenshot: AnnaPetrasova © 2015, Markus Neteler, Italy – CC-BY-SA license New vizualization methods(NC state university) state methods(NC vizualization New LiDAR derived DSM: 100k pixels derived xDSM: 50k 100k LiDAR theatre goes GIS GRASS © 2015, Markus Neteler, Italy – CC-BY-SA license Connecting to other software packages Connecting to other software GIS backbone: GIS as Open Source GRASS © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7 and QGIS Integration: Processing Integration: QGIS and 7 GIS GRASS (here: SHAPE file) SHAPE (here: back result the delivers which session virtual attributes: column string by geometry Dissolving Processing calls GRASS GIS in a in GIS GRASS calls © 2015, Markus Neteler, Italy – CC-BY-SA license > writeRAST(myrast, "elev_filt", zcol="elev") > writeRAST(myrast, > ... > title("Elevation versus geological classes") ~ myrast$geology) > boxplot(myrast$elevation > str(myvect) > <-myvect readVECT("roadsmajor") "elevation"), cat=c(TRUE, FALSE)) > myrast <- readRAST(c("geology", and location: nc_spm_08_grass7 7.0.0svn (2015) version: GRASS interface GIS loaded with GRASS GRASS XML package: required Loading sp package: required Loading > library(spgrass7) [...] (64-bit) x86_64-redhat-linux-gnu Platform: for Statistical Computing Copyright The (C)Foundation R 2014 Helmet" -- "Pumpkin R version 3.1.2 (2014-10-31) > (nc_spm_08_grass7):~ 7.0.0svn R GRASS raster=elevation -p > (nc_spm_08_grass7):~ 7.0.0svn g.region GRASS GRASS GIS 7 and R Integration R and 7 GIS GRASS http://grass.osgeo.org/wiki/R_statistics © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: Native OGC WPS Support WPS OGC Native 7: GIS GRASS http://grasswiki.osgeo.org/wiki/WPS qgis $HOME/gisoutput/warm.tif qgis elsewhere result the use # -r r.external.out files: raster GRASS write to back turn and connection output GDAL cease # )" null() modis_celsius, 20.0, > if(modis_celsius = "warm.tif r.mapcalc GeoTIFF: as directly output write # C) deg 20 > pixels extract (here: calculation GRASS perform # format="GTiff" directory=$HOME/gisoutput/ r.external.out calculation: GRASS from resulting files for directory output define # output=modis_celsius input=tempmap1.tif r.external database: GRASS in file GeoTIFF register # Service Processing Web © 2015, Markus Neteler, Italy – CC-BY-SA license Programming own applications Programming with GRASS GIS 7 with GRASS © 2015, Markus Neteler, Italy – CC-BY-SA license New GRASS 7 Python 7 API GRASS New Lifting thewaysfog onto the develop different for GRASS Lifting http://grass.osgeo.org/wiki/GRASS_and_Python https://fosdem.org/2015/schedule/event/grass_api/ GRASS Development APIs GRASS More in the next talk: ByMoritz Lennert © 2015, Markus Neteler, Italy – CC-BY-SA license in GRASS GIS in GRASS spatial datasets Support for massive © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: Support for massive datasets massive for Support 7: GIS GRASS Massive is relative to to Massiveisrelative What massive?is Limiting factors 4 4 ● ● ● Software capabilities Software resources Hardware Operating system capabilities Processing time Processing RAM Largest supportedfile size Largest Diskspace 4 expensive ... to solve issue cheap © 2015, Markus Neteler, Italy – CC-BY-SA license Speed improvements in the vector engine vector inthe improvements Speed Query of vector point maps Spatial query example CLI: GUI: onclick vector map, what is there? seconds 120 240 360 480 600 v.what east_north=east,north 0 0 2 million points 4 6 8 10 12 GRASS 7 GRASS 6.4.2 GRASS Programmer: Markus Metz Markus Programmer: © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7: Support for massive datasets massive for Support 7: GIS GRASS Example surfaces: cost seconds 120 240 360 480 600 0 0 1 2 Computational time 3 million points 4 5 r.cost 6 7 8 presentation laptop... presentation in 6 seconds PCA of 30 million pixels Other speedfigure: 9 10 effort linear computational non-linear to From GRASS GIS 7 GIS GRASS 6 GIS GRASS on smallthis © 2015, Markus Neteler, Italy – CC-BY-SA license GRASS GIS 7 goes supercomputer goes 7 GIS GRASS ● ● ● ● ● Hints: Various ways of parallelization Various (MS-Windows)... netBSD, on Linux, AIX, Solaris, freeBSD, Runs Bratislava, …) Aurel ENEA Frascati, Umeå, (AKKA Installed on supercomputers Grids and TOP500 Support on Windows 32bit File 7 alsoGIS offers Large GRASS Since http://grasswiki.osgeo.org/wiki/Compile_and_Install 2005 (10 years) GRASS GIS is running (10 years) GRASS natively on CPUs 64bit © 2015, Markus Neteler, Italy – CC-BY-SA license °C Metz, Rocchini, Neteler, 2014: Remote2014: Sens 6, Metz, Rocchini, Neteler, EuroLST: EuroLST: MODIS LST daily timeseries EuroLST: 1 4 http://gis.cri.fmach.it/eurolst/ Each pixel = virtual meteo station meteo= virtual