41
Python & ModelBuilder

Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

  • View
    273

  • Download
    4

Embed Size (px)

Citation preview

Page 1: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Python & ModelBuilder

Page 2: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Overview

• Python/ModelBuilder Concepts– The Geoprocessor– Checking some environment variables– Providing feedback from your model/script– Starting a script with a model– Bringing a completed script into ArcToolbox

• Exercises

Page 3: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Why model or write scripts?

• Automate workflows• Test hypotheses• Reproducibility• Extend Functionality• Simplification• Quality Control

Page 4: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

The Geoprocessing Object

• Also known as the "geoprocessor"• Has many methods• Manages layers and selections • Provides access to all of ArcGIS's tools• Provides status of ArcGIS licensing• Handles errors and warning messages

Page 5: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

The Geoprocessing Objectgp.Clip_analysis("Roads_2007", "Counties", "Mercer_Roads.shp")

Page 6: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Check Data Existence

• gp.exists() returns a boolean depending on whether or not the feature class exists

• Check most user-submitted data for existence– if gp.exists(data):

# continue workingelse:

# warn user about missing data

• ArcToolbox will check, but scripts run from the command line will not

Page 7: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Defining a Workspace

• When the geoprocessor has a workspace defined, all processing results will be stored in that folder, geodatabase, or SDE gdb

• gp.workspace = "C:\\Student\\"• Once defined, we do not need to specify

absolute paths to datasets or outputs located in that workspace

Page 8: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Describe Method

• gp.describe returns an object with numerous properties based on the type of data passed– Feature Classes (shapefiles, geodatabase layers)– Feature Datasets– Workspaces– Raster Data

• gp.describe provides you with GIS-specific information about your data

Page 9: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Using the Describe Method

• Use Describe to ensure that data passed to your model fits certain criteria– Prevent line or point features being used as Clip

Features– Add an area and perimeter field to shapefiles if

not present– Convert rasters to 8-bit unsigned if not already– Use Python to get a list of .shp files, then process

only the polygon shapefiles

Page 10: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Check License Availability

• Models and scripts might be dependent on the level of ArcGIS available

• The gp.CheckProduct("level") returns "Available" or "NotLicensed" depending on the availability of the product level passed

• The gp.SetProduct("level") tells the geoprocessor to use that specific level

Page 11: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Handling Input

• ArcGIS passes information to your Python script through sys.argv

• For example, a clip script would need to listen for three inputs– Input Feature / sys.argv[1]– Clip Feature / sys.argv[2]– Output Feature / sys.argv[3]

• Gets tricky when we ask for multiple inputs

Page 12: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Providing Feedback

• Geoprocessor methods to provide feedback– gp.AddMessage(message string)– gp.AddWarning(message string)– gp.AddError(message string)

• Adds messages to the message list• Message list is simply that – stores all the

errors, warnings and messages generated by the script

Page 13: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Providing Feedback

• gp.GetMessage() and gp.MessageCount• GetMessage(x) returns message number x in the

list• print gp.GetMessage(gp.MessageCount - 1)– prints the last message generated

• Print messages at the end of script or when an error is caught

• Print before and after a lengthy processing step

Page 14: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Defining a Function

• A Python Function allows you to write a block of code that accepts arguments and returns results

• Functions allow you to write something once and reuse it as needed

• Helps debugging• Keeps code clear and concise• Easier to program

Page 15: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Using Functions

• We use the def statement to start a block of code that will only be run when our function is called

• We want our functions to return values we can use in our geoprocessing

• The last statement in our function should be return.

Page 16: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Basic Function

• A simple function that adds 1 to value passed• def addone(x):

return x + 1• print addone(2)

>>> 3• var = addone(3)

print var>>> 4

Page 17: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Multiple Clips

• Like Exercise #2 from last week, we will provide two lists to a script– List of Input Features– List of Clip Features

• We also need to tell the script where to place the output, using gp.workspace()

Page 18: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Start in ModelBuilder

Page 19: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Export to Python

Page 20: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Our First Function

• We need a custom function that will take a string of text, split it into pieces and return those pieces as a list

• SplitMultiInputs will clean up a line of code and return it as a list

Page 21: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Adding SplitMultiFeatures

• Add the function and split the input string and the clippers string passed by the Toolbox to our script

Page 22: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Two Loops

• We will need another loop for each of the inputs

• The "inputs" loop contains the "clippers" loop • The loops are structured like so:

for input in inputs: (runs m times)for clip in clippers: (runs n times)

• The script will then perform mn clips

Page 23: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Completed Script

Page 24: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Back to the Toolbox

• Once our script is complete, we can test it in IDLE and on the command line

• What we'd really want is to run our script like any other tool in the Toolbox

• We can add the script back to the Toolbox• We need to specify how to talk to the script

Page 25: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Add Your Script

• Open ArcToolbox• Right click on the

Toolbox you want your script to be located

• Add > Script…

Page 26: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Add Your Script

• Our model takes two input parameters– Input Feature Classes– Clipping Feature Classes– Output Workspace

• Note that parameters can be set for multiple values

• MultiValue = Yes

Page 27: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

County Clipper

• We’ll take a small ModelBuilder model that clips one input using the statewide Counties layer

• Using Python, we’ll enhance the model to clip the input 21 times for each of the separate counties

• Add functionality to clip multiple inputs by each county

Page 28: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Setting Up Our Model

• We need a County layer as our clip feature, and a user-defined layer as our input feature

• Make Feature Layer• Select By Attributes• Clip (of course)• User-defined Output Workspace

Page 29: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Basic Model to Python Script

loop just a portion of the model

Page 30: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Exported Python Script

Page 31: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Adding County Tuple

• We need a static list of County names to pass as part of the Select by Attributes “where” clause

• Put all the County names into a tuple• Use a for loop to loop over the Select by

Attributes and Clip steps with different Counties each time– for county in counties

Page 32: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Looping Our Model

loop just a portion of the model

Page 33: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Looping Our Model

• For each county in the model– prepare a SQL statement to select County– run the selection to create our clipping features– use selected features to clip– create 21 output feature classes– each output layer named input_county

Page 34: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

SQL Statement

• SQL statements use single and double quotes to delimit field and value names

• Python also uses quotes to delimit strings• Keep track of which quotes are being used• Escape quotes using backslash (\') or (\")• SQL: "COUNTY" = 'MERCER'

Python: "\"COUNTY\" = 'MERCER'"

Page 35: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Looping Over SQL Expressions

• Each time the loop is run, we want to change the SQL statement to select the current county in the loop

• We can concatenate the parts of the SQL statement with the county name

• Example:

Page 36: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Changing the Output Feature

• Each output feature name needs to be unique• Append the current County name• Wetlands.shp becomes– Wetlands_Atlantic.shp– Wetlands_Bergen.shp– …

• Some string methods are needed

Page 37: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Changing the Output Feature

Page 38: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Finalizing Our LoopInput and OutputInput and Output

County LayerCounty Layer

Our For LoopOur For Loop

Modified SQL StatementModified SQL Statement

Create Output PathCreate Output Path

Geoprocessing TasksGeoprocessing Tasks

Page 39: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

USGS Quad Boundaries Clipped

Page 40: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Remember Multiple Inputs

• How will we modify the script to accept multiple inputs?

• We'll need another for loop in the script

• Set the Input parameter to accept multiple values

• MultiValue = Yes

Page 41: Python & ModelBuilder. Overview Python/ModelBuilder Concepts – The Geoprocessor – Checking some environment variables – Providing feedback from your model/script

Clip To Counties Tool

• If the parameters are set correctly, the dialog box will look similar to the one at right

• Our CountyClipper now ready to handle multiple layers, clipping each by county