The Value of Coding for GIS Jason Pardy GeoAlberta 2015, Edmonton, Alberta Jason Pardy Consulting October 27th, 2015 [email protected] About Me • 16 years in GIS • Own my own business, Jason Pardy Consulting • Software developer for Voyager Search (http://www.voyagersearch.com/) • Worked in software development at Esri for 14 years About You • How many of you have written a basic “Hello World” application? • How many of you consider yourself coders? • How many of you are analysts? • How many of you are managers? Contributors • David Howes - David Howes, LLC • Aaron Paul - First American Title • Jeff Berry - Erlandsen, Inc. • Spencer Gardner - Toole Design Group • Eldan Goldenberg - Freelance Developer • Bill Dollins - Zekiah Technologies, Inc. • Matt Stevenson - CORE GIS • Karsten Vennemann - Terra GIS • Mike McGuire - Ascent GIS, Inc. • Igor Talpalatski - Azimetry What is “Coding”? • See How Coding Works, codeconquest.com • Simple - instructions that tell a computer what to do • Complex - binary sequences of 1s and 0s that turn transistors on and off • Language types: • Low level languages - operate close to binary code (e.g., C++) • High level languages - operate far away from binary code (e.g., Python) • In between - e.g., C#, Visual Basic (.NET) Python import arcpy wa_bnd = arcpy.MakeFeatureLayer_management(“C:/Data/WA.gdb/bnd”, “WA”) arcpy.env.workpspace = “C:/Data/Washington.gdb” arcpy.env.outputCoordinateSystem = 2926 feature_classes = arcpy.ListFeatureClasses() for fc in feature_classes(): try: arcpy.analysis.Clip(fc, wa_bnd, “{0}_{1}”.format(fc, “wa”) print(arcpy.GetMessages(0)) except arcpy.ExecuteError: print(arcpy.GetMessages(2)) pass http://gismaps.kingcounty.gov/ForestryCPR/ ArcMap Add-In Levels of Coding • Scripting and tool development • GIS analyst • E.g., data manipulation, map-making, glue code • Application development • GIS developer • E.g., add-in, plugin • Software engineering • Computer scientists/IT professionals • E.g., ArcMap Why Code? “Coding is the hottest skill on the job market, the modern-day language of creativity, and a powerful force in the economy” “coding is … an innovative and artistic process” Madeline McSherry, New America Foundation - Why Everyone Should Learn to Code: An Event Recap http://www.slate.com/blogs/future_tense/2013/03/29/codecademy_ hacker_school_why_everyone_should_learn_to_code.html http://www.ibtimes.co.uk/coding-uk-classroom-python-overtakes- french-most-popular-language-primary-schools-1517491 Why Should Coding Be Important? • Personal development perspective Why should coding be important to you? • Professional growth • Streamline workflows • Expanded toolbox • Policy perspective Why should coding be important to your employer? • Return on investment • Standard operating procedures • Leveraging/freeing up resources Do You Need to Code? • Not necessarily, but it can really help • It depends on your job • Very valuable for analysts • Helpful for managers (increasing need to understand role of coding) Industry Needs See Yes, You Need to Code - Bill Dollins, geoMusings.com “I am very publicly on the record that I think some form of coding skill is essential for any GIS analyst entering the workforce today” http://blog.geomusings.com/2013/01/30/yes-you-need- to-code/ Technology Trend • Coding has always been valuable for GIS • Increasing emphasis on coding • Local example: Maptime Seattle (Meetup group) • Making a web page and a web map (HTML5, CSS, JavaScript, Leaflet.js) • Git & GeoJSON • Mapping with D3.js “software tools expose less than 10% of their full capability through their default interfaces” Bill Dollins, geoMusings Being able to take advantage of extensibility options can really set you apart as a GIS professional Importance of Coding “If you choose to get by with just using the GUI tools, you are doing yourself two disservices: 1. You are placing yourself at the mercy of others who can code to get around to building the customizations you need. 2. You are allowing your skills to erode by not using a significant amount of capability.” Bill Dollins, geoMusings - Yes, You Need to Code Coding Types • Configuration Setting values to adjust base functionality • Programming • Creating an executable formulation of a computing problem • Scripts, programs, add-ins/plugins Benefits of Coding 1. Satisfaction 2. Efficiency 3. Repeatability & validation 4. Freedom 5. Enablement 6. Creativity 7. Clarity & logic 8. Documentation & organization 9. Integration & interoperability 10. Employability Benefits of Coding - Efficiency • Tasks can be quantified • Level of effort • Cost • Value • Justifies creation of streamlining tools and processes Copy Map Point Tool (Python Add-In) ArcMap Field Update & Feature Navigation Tools (.NET Add-Ins) Extending ArcGIS for Desktop Using Python and .NET Add-Ins http://gispd.com/training Benefits of Coding - Repeatability & Validation • Allow for re-running of processes • Custom validation procedures • Increases confidence in methods and results • Crash-resistant processing Benefits of Coding – Freedom • “Roll your own…“ • Build sequences of instructions using libraries of functions, e.g., GDAL Benefits of Coding - Enablement • Learning to code • Opens new doors • Advances your professional capabilities • Increases your standing • Access (more) open source options • Open source experience makes you a better proprietary software user • Opening the Door to Open Source GIS • Explorations into Open Source GIS • Treading Into Open Source GIS http://dhowes.com/presentations, http://gispd.com/events Benefits of Coding - Creativity • Required functionality may not exist • Develop new processes, & procedures E.g., Quadrat Analysis – evaluate spatial patterns of point features by frequency with a set of quadrats. Benefits of Coding - Clarity & Logic • Coding helps you think clearly and logically about problems • Understand what’s really happening under the hood E.g., Whitebox http://www.uoguelph.ca/~hydrogeo/Whitebox/index.html Benefits of Coding - Documentation & Organization • Create a record of actions • Allow for reproduction of results at any time Benefits of Coding - Integration & Interoperability • Don’t come to a halt if a few core programs won’t do everything you need them to do • Bring together the best of all worlds E.g., using data from R in ArcMap Benefits of Coding - Employability • Look at most recent GIS analyst job openings • Ability to code demonstrates interest in • Your career • Advancing yourself • Investing in your capabilities • Demonstrates appreciation for GIS investment Coding Recommendations - 1 • Start small • Be realistic & don’t overdo it • Resist the temptation to start coding too soon Help others Coding Recommendations - 2 • Be persistent Coding Recommendations - 3 • Consider reuse and readability • By yourself • By others • Keep code neat, simple, clean • Use plenty of comments //This comment will save me a lot of grief later Coding Recommendations - 3 (cont.) Follow style standards • Python PEP 8 Style Guide https://www.python.org/dev/peps/pep-0008/ • C# Coding Conventions https://msdn.microsoft.com/en-us/library/ff926074.aspx • Conventions for your language of choice Coding Recommendations – 4 Use Functions • Great way to organize and re-use functionality • Gives your program structure def dd_to_dms(dd): """Convert decimal degrees to degrees, minutes, seconds.""" minutes, seconds = divmod(abs(dd)*3600, 60) degrees, minutes = divmod(minutes, 60) seconds = float('{0:.2f}'.format(seconds)) return int(degrees), int(minutes), seconds Coding Recommendations – 5 Think in terms of objects Object-oriented code • Professional • Reusable • Clean Coding Recommendations - 6 • Adopt a cookbook approach in your coding process • Get each piece working in turn Coding Recommendations - 7 • Think about how your tools could be misused • Handle errors cleanly • Develop testing methods Path for Growth - ArcGIS for Desktop • Python command line in ArcMap • Standalone Python scripts • Create a geoprocessing tool • Create a Python toolbox • Create a Python add-in • Create a .NET Windows form application • Create a .NET ArcGIS for Desktop add-in Path for Growth - Open Source GIS • Write a spatial SQL query http://postgis.net/ • Write a Python script that uses the GDAL library http://www.gdal.org/ • Create a Leaflet.js (JavaScript) webmap http://leafletjs.com/ • In QGIS • Write a Python script • Create a Python plugin • http://plugins.qgis.org/ • http://www.qgisworkshop.org/html/workshop/plugins_tutorial.h tm Resources • Integrated Development Environments (IDEs) • PyScripter • Microsoft Visual Studio 2012 Express • Eclipse • Esri ArcGIS Resource Centers • Open source GIS help pages • Blogs (e.g., geoMusings, GISPD.com) • FOSS4G • GISPD.com • Meetup group (e.g., Maptime) Education • Coursera https://www.coursera.org/ • edX https://www.edx.org/ • Penn State http://open.ems.psu.edu/courseware • GEOG485 GIS Programming and Automation • GEOG585 Open Web Mapping • GISPD.com Extending ArcGIS for Desktop Using Python and .NET Final Thoughts… • “Coding liberates” Bob Mesibov, opensource.com • GIS Practitioners: Code often…and don’t give up. • Project Managers: Allow your staff the time to learn how to code Thank-you! Questions? Image Credits • https://watergis.files.wordpress.com/2012/03/python.jpg • https://darcymullin.files.wordpress.com/2011/08/growth1.jpg • http://www.blairkaplan.ca/wp-content/uploads/2012/06/policy.jpg • http://www.udel.edu/johnmack/frec480/arcmap_intro2.png • https://me4bruno.files.wordpress.com/2012/02/qgis-osm-admin4-border.png