Maa-123.2441 GIS Software Development 4 op Data and databases 10/27/2013 Jari Reini GIS – in 4 categories •Categories based on competences – Data collection and management – Spatial data analysis – Software development – GIS Solutions • Today concentrating on data collection, management and analysis (and partly SW development) Jari Reini 10/27/2013 Architectural choices in GIS (simplified) Desktop Client-Server Internet Application Server Database Jari Reini Database 10/27/2013 Principles •Re-use, Keep it simple •Use COTS as much as possible –Less in-house development –Less maintenance –Already includes a lot of required functionality •For example –Storage: Oracle Spatial database –Editing: ESRI ArcGIS –Quality checking: Radius, JavaML, JTS –Product Creation: FME and own tools 10/27/2013 Architecture and flow Data Sourcing Editing Conversion Products Public Comm upload Delivery unity Database Server Other Rule engine Quality Pre-filter checks Specs Jari Reini 10/27/2013 Content types •Maps (2D) •Voice-Enabled Maps •Address Points •Points of Interest •City Guides •Landmarks •3D maps Jari Reini 10/27/2013 Content gategories •Business content Guide –Office locations, factories, stores … -Voice Maps –Environmental data -Speed Cameras –Etc -Traffic •Find -Logistics –POIs Display –Address Points -2D/3D City Maps –Locality Index -3D Landmarks -3D Terrain Model Jari Reini 10/27/2013 Tele Atlas:Mobile Mapping Vans •Faster data collection •Superior accuracy •Advanced image collection for richer, more accurate content –Renders texture to buildings –Provides back up system for reported issues 10/27/2013 Databases •PostgreSQL/PostGIS •Oracle Spatial •SQL Server / ArcSDE •Etc •Note: in exercises we're using only files (due to some IT issues), think OrbisGIS as a database Jari Reini 10/27/2013 Concepts •relational database management system (RDBMS) •a system for managing data stored in relations –relation is essentially a term for table –Table has a collection of rows –Row has a set of columns –Tables are grouped into databases •A more modern development is the object-oriented database Jari Reini 10/27/2013 Example •CREATE TABLE cities ( –name varchar(80), –Int temperature –location point –); •INSERT INTO cities VALUES ( –‘Helsinki', –-10, –'(60.16, 24.94)‘ –); •SELECT * from cities; •UPDATE cities set temperature=3 where name=‘Helsinki’; Jari Reini 10/27/2013 Spatial SQL • the need for spatial query formalism has been clearly identified by and answered in – Spatial SQL (Max J. Egenhofer, 1994), ● geoPOM (Nittel and al., 1997), ● GeoSQL (Wang and al., 2000)... • SQL spatial has been standardized : – SQLMM : ISO/IEC 13249l – Simple Features Specification for SQL, OGC • efficient implementations that spatially enabled RDBMS (PostgreSQL/PostGIS, Oracle Spatial, SQLite-spatial...). Jari Reini 10/27/2013 OrbisGIS SQL • Portable spatial processor. No RDMS needed • Deals with several formats • Generic datasource model • Multisources language : – able to deal with geometries, raster, TIN and Remote data (WFS, WCS). – Match SQL and OGC standards – Great extensibility and customization ability. Jari Reini 10/27/2013 Put screenshots from spatiallanguage pdf Jari Reini 10/27/2013 Jari Reini 10/27/2013 Geometry functions in databases • Relationship – True or false • Processing – Gives you a new geometry • Measurement – Calculates something Jari Reini 10/27/2013 Geometry relationship functions •Distance(geometry, geometry) •Equals(geometry, geometry) Covers(geometry A, geometry B) •Disjoint(geometry, geometry) CoveredBy(geometry A, geometry B) •Intersects(geometry, geometry) Intersects(geometry, geometry) •Touches(geometry, geometry) Relate(geometry, geometry) •Crosses(geometry, geometry) •Within(geometry A, geometry B) •Overlaps(geometry, geometry) •Contains(geometry A, geometry B) Jari Reini 10/27/2013 Geometry processing functions •Centroid(geometry) •Area(geometry) •PointOnSurface(geometry) •Boundary(geometry) •Buffer(geometry, double, [integer]) •ConvexHull(geometry) •Intersection(geometry, geometry) •SymDifference(B,A). •Union(geometry, geometry) •Union(geometry set) 10/27/2013 Measurement functions •ST_Area(geometry) •ST_distance_sphere(point, point) •ST_distance_spheroid(point, point, spheroid) •ST_length2d(geometry) •ST_length3d(geometry) •ST_length_spheroid(geometry,spheroid) •ST_length3d_spheroid(geometry,spheroid) •ST_distance(geometry, geometry) •ST_max_distance(linestring,linestring) •ST_perimeter(geometry) •ST_perimeter2d(geometry) •ST_perimeter3d(geometry) •ST_azimuth(geometry, geometry) 10/27/2013 Server programming •Extending SQL •Triggers •PL/SQL 10/27/2013 GIS Objects •"Simple Features" defined by the OpenGIS Consortium (OGC) –OGC "Simple Features for SQL" specification. •Expression of spatial objects –Well -Known Text (WKT) –Well-Known Binary (WKB) •include a spatial referencing system identifier (SRID) 10/27/2013 Examples: WKT •POINT(0 0) •LINESTRING(0 0,1 1,1 2) •POLYGON((0 0,4 0,4 4,0 4,0 0),(1 1, 2 1, 2 2, 1 2,1 1)) •MULTIPOINT(0 0,1 2) •MULTILINESTRING((0 0,1 1,1 2),(2 3,3 2,5 4)) •MULTIPOLYGON(((0 0,4 0,4 4,0 4,0 0),(1 1,2 1,2 2,1 2,1 1)), (( -1 -1, -1 -2, - 2 -2,-2 -1,-1 -1))) •GEOMETRYCOLLECTION(POINT(2 3),LINESTRING((2 3,3 4))) •INSERT INTO geotable ( the_geom, the_name ) • VALUES ( GeomFromText('POINT(63.34 23.39)', 312), 'A Place'); 10/27/2013 SQL-MM Part 3 •SQL Multimedia Applications Spatial specification •extends the simple features for SQL spec by defining a number of circularly interpolated curves •SQL -MM used by many databases (e.g PostGIS, Oracle Spatial etc.) 10/27/2013 SQL-MM3 ST_Area ST_AsBinary ST_ExteriorRing ST_MPointFromText ST_SRID ST_GeometryN ST_MPointFromWKB ST_StartPoint ST_AsText ST_GeometryType ST_MPolyFromText ST_SymDifferenc ST_Boundary ST_GeomFromText ST_MPolyFromWKB e ST_Buffer ST_GeomFromWKB ST_NumGeometries ST_Touches ST_Transform ST_Centroid ST_InteriorRingN ST_NumInteriorRing ST_Intersection ST_NumPoints ST_Union ST_Contains ST_Intersects ST_OrderingEquals ST_Within ST_ConvexHull ST_IsClosed ST_Overlaps ST_WKBToSQL ST_CoordDim ST_IsEmpty ST_Perimeter ST_WKTToSQL ST_Crosses ST_IsRing ST_Point ST_X ST_Y ST_Difference ST_IsSimple ST_PointFromText ST_IsValid ST_PointFromWKB ST_Dimension ST_Length ST_PointN ST_Disjoint ST_LineFromText ST_PointOnSurface ST_Distance ST_LineFromWKB ST_PolyFromText ST_EndPoint ST_MLineFromText ST_PolyFromWKB ST_MLineFromWKB ST_Polygon ST_Envelope ST_Relate ST_Equals 10/27/2013 Loading/retrieving GIS data •Loading –Using SQL –Using the loader •Retrieving –Using SQL –Using dumper –Using e.g ODBC/JDBC 10/27/2013 Data Cities and countries Shape files 10/27/2013 Databases: PostGIS 10/27/2013 PostGIS data upload and use Create user 10/27/2013 PostGIS data upload and use Convert and upload shape files using shp2pgsql C:\Tools\PostgreSQL\8.3\bin>shp2pgsql.exe -c c:\temp\data-v1_1\countries.shp tkk.countries > countries.sql 10/27/2013 PostGIS data upload and use C:\Tools\PostgreSQL\8.3\bin>psql -U tkk -d postgis -f countries.sql 10/27/2013 10/27/2013 10/27/2013 PostGIS usage Using SQL INSERT INTO gtest (ID, NAME, GEOM) VALUES ( 1, 'First Geometry', GeomFromText('LINESTRING(2 3,4 5,6 5,7 8)', -1)); SELECT * FROM geotable WHERE ST_DWithin(geocolumn, 'POINT(1000 1000)', 100.0); 10/27/2013 PostGIS from Java •Use JDBC •Class.forName("org.postgresql.Driver"); String url = "jdbc:postgresql://localhost:5432/database"; conn = DriverManager.getConnection(url, "postgres", ""); Statement s = conn.createStatement(); ResultSet r = s.executeQuery("select AsText(geom) as geom,id from geomtable") 10/27/2013 Oracle Spatial Spatial operators Network Management Geocoder Geo Raster Topology Spatial Data Mining 3D 10/27/2013 Geometric Types Polygon Point Line string Polygon with hole Self --crossing Self --crossing line strings polygons Valid Not valid 10/27/2013 Spatial Data Model Spatial layer Geometries Elements Point Line Polygon 10/27/2013 Elements •Basic building block of a geometry •Element types Element 6 –Point (1) Element 5 –Line (2) Element 4 –Polygon (3) Element 7 Element 2 •Constructed Element 1 using coordinates Element 3 •No limit on number of coordinates Hawaii , USA 10/27/2013 Geometry Geometry 1 Geometry 2 California Texas •Represents a spatial feature •Consists of an ordered set of primitive elements Geometry 3 Geometry 4 •Uniquely identified Florida Hawaii with a geometry identifier 10/27/2013 Layers •Consist of geometries that share a common set of attributes States layer 10/27/2013 Spatial Indexing •Provides an exclusive and exhaustive coverage of spatial objects •Indexes points, lines, and polygons •Fixed tiling or hybrid tiling 10/27/2013 Spatial Queries •Issued through SQL •Contains a locational constraint, for example: –Find all insurance holders in the projected path of a hurricane –Find all ATMs in my area 10/27/2013 Spatial Joins •A join of two layers based on the spatial component of the layers •Implemented using spatial index and spatial operators •Example: –Find all the wetlands within the national parks 10/27/2013 Java-based geometries 10/27/2013 Java Topology Suite (JTS) •The JTS Topology Suite is an API of 2D spatial predicates and functions •Based on Simple Features Specification for SQL (OGC) 10/27/2013 10/27/2013 Example •Geometry g1 = new WKTReader().read("LINESTRING (0 0, 10 10, 20 20)"); •Geometry buffer = g1.buffer(100.0); 10/27/2013 Javasacript in geometries 10/27/2013 Geoscript •http://www.geoscript.org •GeoScript adds spatial capabilities to dynamic scripting languages – JavaScript – Python – Scala – Groovy 10/27/2013.