Python Scripting for Spatial Data Processing. Pete Bunting and Daniel Clewley Teaching notes on the MSc's in Remote Sensing and GIS. May 4, 2013 Aberystwyth University Institute of Geography and Earth Sciences. Copyright c Pete Bunting and Daniel Clewley 2013. This work is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License. To view a copy of this license, visit http://creativecommons. org/licenses/by-sa/3.0/. i Acknowledgements The authors would like to acknowledge to the supports of others but specifically (and in no particular order) Prof. Richard Lucas, Sam Gillingham (developer of RIOS and the image viewer) and Neil Flood (developer of RIOS) for their support and time. ii Authors Peter Bunting Dr Pete Bunting joined the Institute of Geography and Earth Sciences (IGES), Aberystwyth University, in September 2004 for his Ph.D. where upon completion in the summer of 2007 he received a lectureship in remote sensing and GIS. Prior to joining the department, Peter received a BEng(Hons) in software engineering from the department of Computer Science at Aberystwyth University. Pete also spent a year working for Landcare Research in New Zealand before rejoining IGES in 2012 as a senior lecturer in remote sensing. Contact Details EMail: [email protected] Senior Lecturer in Remote Sensing Institute of Geography and Earth Sciences Aberystwyth University Aberystwyth Ceredigion SY23 3DB United Kingdom iii iv Daniel Clewley Dr Dan Clewley joined IGES in 2006 undertaking an MSc in Remote Sensing and GIS, following his MSc Dan undertook a Ph.D. entitled Retrieval of Forest Biomass and Structure from Radar Data using Backscatter Modelling and Inversion under the supervision of Prof. Lucas and Dr. Bunting. Prior to joining the department Dan completed his BSc(Hons) in Physics within Aberystwyth University. Dan is currently a post-doc researcher at the University of Southern California. Contact Details Email: [email protected] Postdoctoral Research Associate Microwave Systems, Sensors, and Imaging Lab (MiXIL) Ming Hsieh Department of Electrical Engineering The University of Southern California Los Angeles USA Table of Contents 1 Introduction 1 1.1 Background . .1 1.1.1 What is Python? . .1 1.1.2 What can it be used for? . .2 1.1.3 A word of warning . .2 1.2 Example of Python in use . .2 1.2.1 Software in Python . .2 1.3 Python Libraries . .3 1.4 Installing Python . .3 1.5 Text Editors . .4 1.5.1 Windows . .5 1.5.2 Linux . .5 1.5.3 Mac OSX . .5 1.5.4 Going between Windows and UNIX . .5 1.6 Starting Python . .6 1.6.1 Indentation . .6 1.6.2 Keywords . .7 1.6.3 File Naming . .7 v TABLE OF CONTENTS vi 1.6.4 Case Sensitivity . .8 1.6.5 File paths in examples . .8 1.6.6 Independent Development of Scripts . .8 1.6.7 Getting Help . .9 1.7 Further Reading . 10 2 The Basics 11 2.1 Hello World Script . 11 2.2 Comments . 12 2.3 Variables . 13 2.3.1 Numbers . 13 2.3.2 Boolean . 13 2.3.3 Text (Strings) . 14 2.3.4 Example using Variables . 15 2.4 Lists . 19 2.4.1 List Examples . 19 2.4.2 n-dimensional list . 20 2.5 IF-ELSE Statements . 21 2.5.1 Logic Statements . 22 2.6 Looping . 23 2.6.1 while Loop . 23 2.6.2 for Loop . 24 2.7 Exercises . 25 2.8 Further Reading . 26 3 Text Processing 27 TABLE OF CONTENTS vii 3.1 Read a Text File . 27 3.2 Write to a Text File . 30 3.3 Programming Styles . 30 3.3.1 Procedural Programming { File Outline . 31 3.3.2 Object Orientated Programming { File Outline . 32 3.4 Object Oriented Script . 33 3.4.1 Object Oriented Script for Text File Processing . 33 3.5 Exercise . 40 3.6 Further Reading . 41 4 File System { Finding files 42 4.1 Introduction . 42 4.2 Recursion . 44 4.3 Checking file Extension . 45 4.4 Exercises . 48 4.5 Further Reading . 48 5 Plotting - Matplotlib 49 5.1 Introduction . 49 5.2 Simple Script . 49 5.3 Bar Chart . 50 5.4 Pie Chart . 52 5.5 Scatter Plot . 53 5.6 Line Plot . 56 5.7 Exercise: . 59 5.8 Further Reading . 59 TABLE OF CONTENTS viii 6 Statistics (SciPy / NumPy) 60 6.1 Introduction . 60 6.2 Simple Statistics . 61 6.2.1 Exercises . 64 6.3 Calculate Biomass . 64 6.3.1 Exercise . 71 6.4 Linear Fitting . 72 6.4.1 Exercise . 79 6.5 Further Reading . 79 7 Batch Processing Command Line Tools 80 7.1 Introduction . 80 7.2 Merging ESRI Shapefiles . 81 7.3 Convert Images to GeoTIFF using GDAL. 90 7.3.1 Passing Inputs from the Command Line into your script . 91 7.4 Exercises . 92 7.5 Further Reading . 92 8 Image Processing using GDAL and RIOS 94 8.1 Reading and Updating Header Information . 94 8.1.1 Reading Image Headers . 94 8.1.2 Read image header example. 96 8.1.3 No Data Values . 98 8.1.4 Band Name . 100 8.1.5 GDAL Meta-Data . 103 8.2 Raster Input / Output Simplification (RIOS) Library . 108 TABLE OF CONTENTS ix 8.2.1 Getting Help { Reminder . 108 8.2.2 Band Maths . 109 8.2.3 Multiply by a constant . 109 8.2.4 Calculate NDVI . 111 8.2.5 Calculate NDVI Using Multiple Images . 114 8.3 Filtering Images . 117 8.4 Apply a rule based classification . 120 8.5 Exercises . 124 8.6 Further Reading . 124 9 Raster Attribute Tables (RAT) 125 9.1 Reading Columns . 125 9.2 Writing Columns . 127 9.2.1 Calculating New Columns . 127 9.2.2 Add Class Name . 129 9.3 Adding a colour table . 131 9.4 Using RATs for rule based classifications. 134 9.4.1 Developing a rule base . 134 9.5 Exercises . 145 9.6 Further Reading . 145 10 Golden Plover Population Model 146 10.1 Introduction . 146 10.2 Model Output . 146 10.3 Reading Parameters . 146 10.4 The Model . 149 TABLE OF CONTENTS x 10.5 Exporting Data . 153 10.6 Creating Plots . ..