Automated Feature Extraction in Arcgis

Please Share Your Feedback in the App

Download the Esri Select the session Scroll down to Log in to access the Complete the survey Events app and find you attended “Survey” survey and select “Submit” your event Automated Feature Extraction In ArcGIS

James Sill, Senior Solution Engineer, Esri, Boulder, Co Jacob Czawlytko, Chesapeake Conservancy, Senior Geospatial Analyst Kumar Mainali, Chesapeake Conservancy, Geospatial Data Scientist Agenda

• What is feature extraction in ArcGIS

• Methods for solving the problem - Unsupervised vs. Supervised - Collecting and managing training samples in ArcGIS Pro, Enterprise and Online - Deep Learning in ArcGIS for Feature Extraction

• Real Life Examples - Supervised classification of Landcover with Raster Analytics - Integrating external deep learning frameworks into ArcGIS - Chesapeake Conservancy Landcover Classification Tools for Feature Extraction from Imagery In ArcGIS

Classification

Prediction

ArcGIS Clustering Deep Learning Machine Learning Tools in ArcGIS

Classification Deep Learning

• Pixel & Object Based • Generate training samples • Image Segmentation • Detect objects • Maximum Likelihood • Classify pixels • Random Trees • End to End support • Support Vector Machine • Apply at Scale over large collections

Clustering Prediction • Spatially Constrained Multivariate Clustering • Empirical Bayesian Kriging • Multivariate Clustering • Areal Interpolation • Density-based Clustering • EBK Regression Prediction • Hot Spot Analysis • Ordinary Least Squares Regression and • Cluster and Outlier Analysis Exploratory Regression • Space Time Pattern Mining • Geographically Weighted Regression Built – in tools for Feature Extraction in ArcGIS Pro Unsupervised

Pixel or Object Based

GOAL Clean Up

Merge Classes

Assign Classes

ISO Clusters Choose Image Unsupervised Method: • ISO Clustering: - Based on K-means nearest neighbor algorithm

- Quick, least cost classification method

- Good for instances where there is a low familiarity with composition of area of interest

- Generally low overhead

- Advantages where a lack of resources to create training samples

- Use cases where unsupervised methods excel… - Flood extent mapping – ISO/Kmeans clustering to delineate water from non-water

- Presence absence of vegetation in post wild fire burn zones

- Distinct differences in spectral composition within your AOI Supervised

Pixel and Object Based

GOAL Accuracy

Classify Support Vector Machine

Maximum Likelihood

Forest Based Classification

Training Samples Choose Image Supervised

Object Classification

GOAL Accuracy

Classify

Support Vector Machine Training Samples

Segmentation Choose Image Supervised Feature Extraction and Pixel Level Classification

• Pixel and Object Based Classification/Identification - Pixels are classified or objects identified through supervised algorithms - Support Vector Machine - Forest Based Classification - Maximum Likelihood - Requires that the user collects training samples - Instances where supervised classification excels: - Ability and time to create training samples - Example Use cases where Supervised Classification Excels - Multiclass landcover classification and feature identification

- Impervious/non-impervious surface mapping

- Static data products that produced in a non – time sensitive environment Demo: Supervised Machine Learning for Feature Extraction in ArcGIS Pro

• Support Vector Machine classification of Sage Grouse Habitat in Southwest Colorado (ArcGIS Pro, Image Server, Raster Analytics) Feature Extraction and Machine Learning with ArcGIS: End to End Cycle

Feedback Loop

Imagery Imagery Creating Training Inference Derive Take Access Prep Training Products Action data

Distributed Processing ArcGIS – Machine Learning Workflow Detailed Workflow

ArcGIS User ArcGIS Professional Image (Data) Scientist

Machine Learning Pixel & Segment Based Machine Imagery Learning Generate Training Inference results Samples Deep Learning Based Deep Deep Learning Learning Training sites Training Tools Training Engine Model Inferencing Tools Definition

Feedback Loop Input Images

Pixel & Segment Based: Machine Learning: - Maximum Likelihood - Support Vector Machine - Support Vector Machine - Random Forest - Random Forest Deep Learning: Deep Learning Based: - TensorFlow* - TensorFlow* - CNTK* - CNTK* - PyTorch* - PyTorch* - Custom* - Custom* - + External via Python *Run External to ArcGIS *Requires framework installed Feature Extraction Workflow in ArcGIS End-to-end from raw imagery to structured information products

Image Service/ Mosaic Labelling Data Train/Fit Prediction Field Analysis Mobility, Dataset Prep Model Monitoring

Imagery Management Deep Learning Key imagery tasks for deep learning

Impervious Surface Agricultural Crop Building Footprint Damaged House Classification Detection Extraction Classification

Pixel Classification Object Detection Instance Segmentation Image Classification Where Deep Learning Excels

• Resources to create and maintain robust training datasets • Access to GPU’s to train and apply model… ☺ • Well defined problem and general knowledge of an area • Imagery collected under consistent conditions with minimal variations in quality • Large scale monitoring problems - There is a need to repeatedly measure the activity, composition and change in a particular area over the course of time - Use cases - Monitoring and identifying changes in landcover over time - Object detection- i.e.. Counting specified objects - Classification of detected objects – i.e.. Damaged or undamaged houses From Change Detection to Monitoring… Deep Learning with Imagery in ArcGIS ArcGIS supports end-to-end deep learning workflows

• Tools for: • Labeling training samples • Preparing data to train models • Training Models • Running Inferencing

• Supports all 4 imagery deep learning categories

• Supports image space, leverage GPU

• Clients • ArcGIS Pro • Map Viewer Part of ArcGIS Image Analyst • Notebooks Run distributed on ArcGIS Image Server Demo: Deep Learning • Managing Training Data and Applying a Deep Learning Model in ArcGIS Pro for Object Detection and Monitoring Conservation Innovation Center

The CIC was created by Chesapeake Conservancy to help shape proactive responses for one of the world’s largest environmental efforts—restoring the Chesapeake Bay. Since then, the CIC has continued to pioneer high- resolution GIS mapping that provides new perspectives about the state of landscapes and waterways. This information is used to identify specific project-level priorities that can maximize conservation outcomes. The Problem

• Poultry houses are important to Chesapeake Bay TMDL - mapping and accounting agricultural BMPs

• USDA data is restricted - Unknown total number in Chesapeake Bay Watershed • USGS data is new but limited to Delmarva • Very little available geospatial data on poultry houses • 5,747 poultry houses in the Delmarva peninsula identified using 2016 and 2017 USDA NAIP by USGS • “Very little” available geospatial data on poultry houses What’s the plan?

• Utilize ArcGIS tools and existing datasets to training data(USGS poultry houses) • Export Training Data For Deep Learning in ArcGIS Pro • Detect Objects Using Computer Vision • Run Detect Objects Using Deep Learning (ArcGIS Pro) • Compare results

Data and Models

• Input layers: red, NIR, thermal, ndsm, NDVI • Main model: computer vision - Scale Invariant Feature Transform (SIFT) - Gray Level Co-Occurrence Matrices (GLCM) • Other considerations: - Traditional machine learning models with features extracted manually • Questions to test: - How much benefit is there of using computer vision - do traditional machine learning models perform comparable with extracted features?

Next steps

• Pass data to BMP team to integrate dataset into BMP analysis • Accuracy assessment • Use same methods to identify novel classes Links and Contact

chesapeakeconservancy.org/conservation-innovation-center sciencebase.gov/catalog/item/5e0a3fcde4b0b207aa0d794e

[email protected] [email protected] Questions? Thank You!!! You can find me at the Civilian – Sciences Area of the Expo hall

James Sill [email protected] Demo Title Presenter(s)