Lab 2: Raster GIS Overlay

Due By beginning of lab, next Thursday, Feb 21

Topics MIT GIS File Management Revisited Getting Data Local Setting Up ArcMap for Portable Use Between Machines Creating and Using Geodatabases Land Cover Change Analysis with Raster Overlays Using the Reclassify Command Numeric Overlays with Map Algebra and NoData Logical Overlays with Raster Calculator Merge Proximity Relations in Raster – Simple Euclidean Distance

1) MIT GIS File Management Revisited

As discussed in lecture, file management is sometimes a challenge with current GIS software. One the default in ArcGIS is to work across the network, and this causes many problems, particularly with large files and later in the term as networks get heavily loaded. Another ArcGIS default creates ArcMap project documents which do not transfer correctly from machine to machine.

For these reasons, we have developed a standard workflow for the class. This is not the only possible way to manage your files of course, but rather one which is documented and as reliable as possible. We will use a standard local temporary space for working files. When our working session is done, we will create a new local geodatabase and copy the GIS layer data which we created or revised into it. Then we’ll upload the geodatabase and our ArcMap project file back up to the network file server for safe keeping or future work.

A) Getting Data Local

1. In Windows explorer, create a new folder under C:/usertemp named “telluride”. Inside that, create another new folder called “lab2work.” Because of the way in which windows permissions are set up by default at MIT, this should guarantee that you have a unique writable folder isolated from any other user. 2. Open the course disk (Z:/course/11/11.951/Telluride) and navigate to the “Telluride_clipped” folder. Copy into your local C:/usertemp/telluride folder the base GIS data for the study area. This is called GISDATA_clip.mdb and is a geodatabase containing a few dozen raster and vector layers. It will not likely change much from week to week and is not lab-specific, so we’ll keep it one level above the lab-specific working data. 3. Also from the course disk, copy the lab2.zip file and uncompress it in your local telluride folder. This zip file contains files specific to this week’s lab which are not in the base GIS. In particular, it should contain an ArcMap project file called telluride_week2.mxd, and some layer files. The layer files contain standard cartography for the files which we will use this week. (Logically, it would be convenient to store the layer files in the geodatabase, but this is not currently supported.) 4. Now that we have all the data we need local and uncompressed, open this week’s base ArcMap document, which should be called telluride_week2. This map document contains several types of things. The first and visually obvious is a set of layers in the Table of Contents (leftmost window pane). The second, and not visually obvious, is that map documents carry with them a number of user preference settings. By starting from a document where these are set up sensibly, we can avoid starting from scratch each time. 5. Save a copy of the ArcMap document modified to reflect your name: week2_[username]

At this point, you should have all of the data and documents you are going to need local to your machine.

B) Setting Up ArcMap for Portable Use Between Machines

Now, let’s look at one important (but rather deeply buried) setting and the concept behind it. By default in ArcMap, layers are stored with “absolute path names.” This means that within the map document, when you add a layer, ArcMap records the location of the added files on disk starting with a Windows drive letter. For example, if you added a layer called “polygon.shp” which was in your local working directory, ArcMap would record something like “C:/usertemp/telluride/lab2work/polygon.shp”

This generally works fine as long as you are either (a) working only on a single isolated computer or (b) working on a very fast network. However, the problem with absolute paths is that Windows drive letters and paths vary from machine to machine. In a lab environment, you often do not have permission as an end user to create folders in exactly the same location as on another machine. For example, in the SAP GIS lab in Building 9, the student temporary directory is set by administrators to “D:/temp” rather than “C:/usertemp.” The consequence is that by default, ArcMap project files are not portable between machines.

We can fix this, but only by changing a relatively obscure preference setting.

1. In the main menubar, select Document Properties and in the resulting dialog box, click the button labeled Data Source Options. 2. Change (if needed) from the default “Store full path names …” to “Store Relative Path Names …” 3. Also select the checkbox next to “Make relative paths the default …” Save your map document again to store these changes.

From now on, each time you add data to this or a new ArcMap document, the location of that data will be recorded relative to the location of the current map document. For example, data from your base GIS will now be stored as “GISDATA_clip/polygon.shp” if in the GISDATA_clip geodatabase located in the same folder as the geodatabase. Data from your weekly lab will be stored as “lab2work/my_new_grid” etc. The two effects of this change are: (1) the local copy of the GISDATA geodatabase will be used by default, which should be much faster than attempting to constantly access network data, and (2) your ArcMap project document should now be portable between machines.

C) Setting the Spatial Analyst Working Directory

Last week, we saw how to set up some common “Options” for Spatial Analyst. This week, we’ll look at one additional setting, which controls where new raster grids are stored by default. This is particularly important when using the map calculator, or a few other raster tools which do not make it easy to select a full output location.

The Spatial Analyst creates raster files by default in the folder set as the “working directory.” ArcMap defaults this location to the Windows user temp folder, which is rather deeply buried in the file system. We will change this to a new folder in our MIT-lab-sanctioned user temporary space. For the 37-312 lab, everyone should have permission to write to a local folder called “C:/usertemp” so we will create a new folder within that for base GIS data, and each week a second one within that to contain working grids and modified vector GIS files.

1. In ArcMap’s Spatial Analyst Toolbar, select Options, and under the General tab, point the “Working Directory” to C:/usertemp/telluride/lab2work. 2. Test your new working directory by making a simple mask. In the Raster Calculator, simply enter the following formula: mask = 1 3. If all of your Spatial Analyst settings are correct, this should create a new raster grid in your working directory, named “mask.” This mask should have the extent and cell size specified in your Spatial Analyst options. 4. Save you project one more time, ensuring that your options are preserved.

With all of this (admittedly tedious) setup work done, we are ready to start doing some actual raster analysis work!

D) Creating a Geodatabase

As mentioned in lecture, geodatabases have fewer format limitations than shape files, and avoid file propagation. However, they are not an open exchange format (i.e. can only be used with the most recent versions of ArcGIS), and they must first be created before being populated.

1. In ArcMap, open the Toolbox (red toolbox icon) 2. Select the Index Tab and in the text field at top of that tab, type “Create File” 3. Double click on the “Create File GDB” tool. Our license version of ArcMap (called Arc/Info) can create both “File” and “Personal” geodatabases. The difference is basically that “File” geodatabases can be accessed by multiple users at once. Less expensive licensed versions of ArcMap (called ArcView) can only create “personal” geodatabases, although they can read data from the file geodatabases. 4. Click the Folder Locator browse button to the right of the “Output Location” text field and navigate through the local file system until you find the folder in which you want the new geodatabase file to be created. Highlight that folder’s name by single clicking on it, and then hit the “Add” button. If you double click, you open the folder, and must back up one level in order to be able to “Add” it. In this case, we want to create a new geodatabase for this week’s lab, so select the working folder “C:\usertemp\telluride\lab2work” and hit the Add button. This interface is poorly designed and frequently causes confusion. Do not try typing a geodatabase name directly into the name field of the dialog box (where the cursor is flashing and which appears to be the default). If you try that, you will get an error message saying that “the object named [name you typed] does not exist’ 5. Once you have a valid output location (which should show as a folder path in the original dialog box’s output location text field), then type the desired name for the geodatabase directly into the “Output Personal GDB” text box. For example, let’s create a new geodatabase to contain the output of this lab all in one file. Name your output personal GDB “lab2” 6. At this point, you should have an empty geodatabase in the C:\usertemp\telluride\lab2work directory. The file will be named lab2.gdb, and will be about 500k in size, even though “empty” of geographic content. 7. From this point forward, geodatabases act somewhat like a folder. They are a container into which you can save raster and vector data. For vector data, you can save directly into a geodatabase. Unfortunately, there remain some significant limitations with directly saving raster files into a geodatabase, which we for now must document and work around.

E) Using Geodatabases

First, let’s save a subset of a vector data layer into the lab2 geodatabase. For example, let’s save out the fraction of San Miguel County which is within a broader regional study area.

1. In the “Standard Overlay” layer group, find the “County Boundaries” layer. Right mouse on the layer and select Data->Export data. 2. Choose the “Select Features” tool from the toolbar (white arrow) 3. Click anywhere within San Miguel County (surrounding Telluride) You should get an aqua highlight indicating that the appropriate polygon is selected. 4. In the “Export Data” dialog box, note that the default is to export only the selected records. It is this feature which we will use to subset the original vector data. 5. Browse to inside of your newly create lab2 geodatabase (the Saving Data dialog Look in: field should read lab2.mdb). Name the output feature class “Eastern_San_Miguel_County” (no extension needed because we are inside a geodatabase). 6. In the resulting dialog box, select the default, adding the new subset into the current map document. 7. Exporting raster data works rather similarly. For example, select the Land Cover 2001 layer, right mouse on it, and select Data->Data Export. You get a dialog box which looks a bit different from the vector case. Significantly, there is no option to limit the export only to selected records. This means that it is not possible to subset raster data on export (you will have to use the Reclassify command as described in class). In the dialog box, note that you can select a geodatabase as the output workspace, and that when you do so, the Format option grays out and becomes “Personal Geodatabase.” In other words, you can only store rasters in this format within a geodatabase (although you can always re-export them later to other raster or graphic image file formats). Save the land cover grid into the geodatabase.

2) Land Cover Change Analysis

Which factors do you think might have influenced land use change in the 90s? In this lab, we will generate and test a few hypotheses, first visually and qualitatively using exploratory spatial analysis, and then quantitatively.

Hypothesis 1: Most land use change was due to urban development in previously agricultural, grassland or forested areas.

Research Strategy: Determine how much land changed at all, determine share of change was conversion to urban from the three given categories.

Technique: Use Raster Reclassify to create a new grid containing only grid cells which changed. Add a new column to the raster attribute table and calculate Hectares of change in each category. Use tabular summary to generate a new table describing changes.

Method 1) Examine the attribute table of the “Land Cover Change 1992 to 2001” layer. 2) Click on the Options button in the table view and choose the “Select by Attributes” menu item. 3) You want the land which was urban in 2001 but not in 1992. Specify the following query: a. "lu2001" = 'Urban' and "Lu92" <> 'Urban' 4) Back in the table view, use the Show Selected button to limit the displayed rows. You should have four rows. Note the values in the values column. Since there is no interface in raster ArcMap which uses the current selection, we will need to manually re-enter these four values into the reclassify command. 5) Leaving the table visible, from the Spatial Analyst menu, select “Reclassify…” 6) In the Reclassify dialog box, you want to create a new raster representing only changed to urbanized land. In technical terms, you want to retain only the records with the same values indicated in the selected attributes of your land use change attribute table, namely 12, 42, 52, 62. Change the reclass field near the top of the dialog box to “Value.” In the first row of the table, enter the value 12 into both the “old value” and “new value” columns. In the second row, enter the value 42 into both “old value” and “new value” columns. Similarly, enter 52 and 62 into the next two rows (see figure below). 7) Select the other rows (you can shift-click), and delete them using the “delete entries’ button. 8) Check the “Change Missing Values to No Data” checkbox below the list. In this case, we want to pass through the original change values from the input grid unaltered, but to replace everything else with the value of “NoData”. Hit OK.

Hit “Yes” to continue if you get a warning dialog saying “The input raster minimum (1) is out of range …Do you want to proceed…” This warning comes up because you are purposefully excluding values out of range…

9) Save the result as a raster called something like “dev_change” 10) Create a new column in your grid attribute table of type float, named “Hectares.” Calculate the value of the Hectares column from the grid cells, based on their size in meters and a conversion factor from square meters to hectares. Each cell is 30 x 30 meters, or 900 square meters. There are 10,000 square meters in a hectare. Therefore, Hectares = [count] * 30 * 30 / 10000

Report Results. Create a world processing document to report your findings as narrative text with figures as appropriate. Was hypothesis 1 correct? How many hectares of land were converted to urban in the period covered by the data? How many hectares of land experienced other kinds of land use change. How would you generally characterize those changes? (Hint, you may need to zoom in to show them!) In your maps, feel free to zoom in to show areas of interesting change.

Create a map layout illustrating the changes you found.

1. Go to View->Layout View 2. Use the Insert menu to include a legend, scale bar, North arrow and title. 3. When you have a layout you like, select File->Export and save the resulting map image as a PNG graphics file. In your Word Processor, import the picture (in Microsoft Word, Insert->Picture->From File).

Hypothesis 2: New development occurred only on private lands

Research Strategy: Create a raster mask map of private lands. From the land use change raster, isolate out those categories involving development. Multiply your mask map by the “Newly Developed” (dev_change) data map, to create a map of “New Development on Private Lands.” Alternatively, you could use the “extract by mask” tool which has the same effect. Following the same strategy, create a map of “New Development on Public Lands.” For each grid, create a column showing land area in hectares and summarize.

Method: 1. Select the Ownership layer from the Standard Overlay layer group, and open its attribute table. 2. Add a new column, of type text named “OwnType” (default other params). 3. Calculate the value for this field to be the string “Public.” Note that in the vector data as given, there is no “private category” – this is implicit in not being public. In our raster, we will change that, but since we have no geometry to work with in the vector polygon file we’ll have to wait. 4. Select the vector ownership layer and convert it to raster, based on the value of the field “OwnType” (since it is a string which contains only one value, your resulting grid will have only the value 1). Call this raster layer public_mask. 5. Create a new raster, using reclassify. In your reclassification, take the NoData background, and recode it to the value “2” Name the result pub_priv. 6. Reclassify again, set the private lands value to 1, and setting the public lands to “NoData” Name this “priv_mask” 7. Multiply your public mask by the change layer to create a new layer of pub_change. 8. Multiply four private mask by the change layer to create a new layer of priv_change. 9. In each resulting change grid, create a new column, of type float, called Hectares and calculate the value of Hectares as above. (= [count] * 30 * 30 / 10000 )

Reporting: In the lab write up document above, describe and quantify the number of changed hectares on public and private land. Is the original hypothesis supported? What kinds of changes were going on where? Create a map layout showing development in two different colors depending if on public or private land. Again, export a graphic of your map layout, and import into your lab report.

Turn in the lab report (as Word document or PDF) to Stellar.

Saving / backing up

To save your work, export rasters from your map document into your lab2.gdb file. Be sure not to put any spaces in the raster file name requested (lest ArcMap refuse with the unhelpful error message “Failed to save raster dataset”). I recommend at this point that you add the data back into the map, adjust the cartography if needed, and remove the layer pointing to the original grid.

Once you have saved into your geodatabase anything you feel you need to save, you should save your final working map document, then upload the geodatabase to your personal space.