ArcPy Overview – Extending and Automating ArcGIS with Python
Clinton Dow – Geoprocessing Product Engineer @ Esri
http://www.slideshare.net/ClintonDow/arcpy-overview-slides
Before we begin…Free stuff!
• Free ArcGIS Pro- http://bit.ly/ArcPyProTrial
• Free ArcGIS Online Account- http://bit.ly/PythonAGOLTrial
• ArcPy Documentation- http://bit.ly/ArcPyDocs
• ArcPy Resources- http://bit.ly/ArcPyResources
About MeClinton Dow – Calgary, AB > Los Angeles, CA
• Product Engineer @ Esri- ArcPy- Conda- Charts and Graphs
• GIS Developer @ Matrix Solutions Inc.- GeoProcessing Tool Development
- Oil & Gas Applications- Environmental Applications- Civil Engineering Projects
esriEnvironmental Systems Research Institute, Inc.
• Founded in 1969• HQ in Redlands California
- 41 Offices Globally- Employees from 67 Countries
• Supports Government, Military and Civilian organizations• Currently hiring Experienced Python Developers/Product Engineers - http://bit.ly/ArcPyJobs
ArcGISProduct Information
• Industry-leading Commercial GIS Software Suite• ArcGIS for Desktop
- Initial release in 1999- Current version of ArcMap 10.4- ArcGIS Pro included since 10.3 – January 2015
- Current version 1.2
• ArcGIS for Server- Current version 10.4- Portal for ArcGIS included since 10.2
• ArcGIS Online
ArcPyArcGIS Python site-package
• Introduced in ArcGIS 10.0- June 29, 2010
• Successor to arcgisscripting module• Create tools leveraging ArcGIS
- In-Application Geoprocessing Tools and Scripting- REST Web Services- Standalone Python Programs
• 1,000+ ArcPy-based Tools Included in ArcGIS
Python at EsriOther Uses of Python
• Test Framework- Proprietary test-runner and reporting
• Queries in ArcGIS- Tools not based on ArcPY
- Calculate Field Tool- Map Label Expressions
• CityEngine - Python Scripting- Built-in script editor
Agenda
• Python in ArcGIS: Moving Forward- Python 2 vs. 3- ArcGIS 10.x vs ArcGIS Pro
• Tip #1 –ArcPy in an IDE• Automating ArcGIS with Python
- Scripting Complex Workflows- Service Endpoint Automation
Agenda• Tip #2 – There must be a better way!• Extending ArcGIS with Python
- Python Standard Library- Tour of Complementary Python Packages
• Tip #3 – Comprehensions with Cursors• The future of Python in ArcGIS
- Managing Packages and Environments with Conda- Jupyter Notebook
ArcPy: Moving Forward‘Python is the language of GIS.’ – Bill Moreland, Esri Dev Summit 2016
• Python 2 vs. 3 in ArcGIS- Python 2 End of Support Countdown Clock!- ArcGIS 10.4 Python 2.7.10- ArcGIS for Server Python 2.7.10
- ArcGIS Pro 1.2 Python 3.4.3- ArcGIS Pro 1.3+ Decoupled Python!
ArcPy: Moving Forward‘Python is the language of GIS.’ – Bill Moreland, Esri Dev Summit 2016
•ArcGIS 10.x to ArcGIS Pro- ArcObjects SDK will not move to Pro- Great time to be a Python GIS Developer!- ArcObjects high cost of maintenance and very tightly
coupled to COM- Rewrite code in Python 3 in Pro
- Platform agnostic vs. .NET code- Web Services- Vast library of Packages- Improved features in Pro- … it’s Python and not .NET
Tip #1 – ArcPy in an IDEGIS from the comfort of your development environment
• Integrated Development Environments- Debuggers- Test Frameworks- Coverage- PyLint/pycodestyle- Version Control Integration- Refactoring Tools- Virtual Environment Support- AutoComplete
Tip #1 – ArcPy in an IDEGIS from the comfort of your development environment
• Recommended IDEs- PyCharm- Python Tools for Visual
Studio- Spyder- Eclipse with PyDev- Wingware
Automating ArcGIS with PythonToolboxes, Python Toolboxes, Interpreters, Command Line, Services…
• Multiple ways to create tools- ArcGIS Toolbox (.tbx)- Python Toolbox (.pyt)- Stand-alone Scripts (.py)- Interactive Python Window
• Decouple logic code from Toolbox- Testing!!
• Distutils to create packaged codeSharingInstall into tools as needed
Extending ArcGIS with Python ModulesPython is a ‘Batteries Included’ Programming Language
• Python Standard Library- Distutils- Collections- Itertools- XML- JSON- UnitTest- -asyncio
Tip #2 – There Must Be A Better Way!Never Underestimate The Python Standard Library!
• Raymond Hettinger- Transforming Code Into Beautiful, Idiomatic Python
- https://www.youtube.com/watch?v=OSGv2VnC0go- Beyond PEP 8 – Best Practices for Beautiful, Intelligible Code
- https://www.youtube.com/watch?v=wf-BqAjZb8M
Extending ArcGIS with Python ModulesUtilize the Extended Python Ecosystem in your ArcGIS Tools
• Scientific- Jupyter- IPython- NumPy- SciPy- Pandas- PySAL- SciKit Learn
• Utility- Requests- Arrow- Xlrd/xlwt- Chardet
• Visualization- Bokeh- Plotly
• Web-Scraping- BeautifulSoup- Scrapy- Tweepy
Tip #3 – Filter & Map with ComprehensionsUsing Pure Python Data Types to Work with Feature Classes
import arcpy
data_path = r"C:\Path\to\feature\class"
def cursor_comprehension_map(path): # Map function # m = map(lambda x: x[0] + 1.0, [row for row in arcpy.da.SearchCursor(path, ['OID@'])])
m = [row[0] + 1 for row in arcpy.da.SearchCursor(path, ['OID@'])] return m
def cursor_comprehension_filter(path): # Filter function # f = filter(lambda x: x ==1, [row for row in arcpy.da.SearchCursor(path, ['OID@'])])
f = [row for row in arcpy.da.SearchCursor(path, ['OID@']) if row[0] == 1] return f
Tip #3 – Comprehensions with CursorsUsing Pure Python Data Types to Work with Feature Classes
import arcpy
data_path = r"C:\Path\to\feature\class"
def cursor_list_comprehension(path): l = [row for row in arcpy.da.SearchCursor(path, ["SHAPE@", "Language"])] return l
def cursor_dictionary_comprehension(path): d = {row[0]: row[1:] for row in arcpy.da.SearchCursor(path, ["OID@", "MsgText", "Language"])} return d
def cursor_generator_comprehension(path): g = (row for row in arcpy.da.SearchCursor(path, ["SHAPE@", "Language"])) return g
def cursor_generator_statement(path): with arcpy.da.SearchCursor(path, ["OID@", "MsgText"]) as cursor: for row in cursor: yield str(row[0])
Tip #3 – Collections ExampleUsing Pure Python Data Types to Work with Feature Classes
from collections import Counterimport arcpy
data_path = r"C:\Users\clin8331\Desktop\Temp\tw.gdb\tw_lang_europe"
def count_field_occurances(data_path): c = Counter() with arcpy.da.SearchCursor(data_path, ["OID@", "Language"]) as cursor: for row in cursor: c[row[1]] += 1 return c
The Future of Python in ArcGISContinuum Analytics’ conda VirtualEnv & Package Manager
• Conda integration into ArcGIS- Advanced Python Package Management
- Curated channels of scientific Python packages- Includes C and Fortran compiler - Create and host your own conda packages- Easily share your code with the GIS community
- Built-In Virtual Environment Support- Manage multiple environments with access to ArcPy- Easily share environments- Trivializes dependency issues
The Future of Python in ArcGISJupyter Notebook/IPython Integration
• Interactive Scientific Computing with ArcGIS Support- Interact with ArcGIS for Server and ArcGIS Online via Jupyter Notebooks- ArcPy Integration coming- Embed maps with simple, pythonic API- Access scalable, web-based GeoProcessing Services
- GeoEnrichment Services- GeoAnalytics Servers
- All the regular functionality of notebooks- Pandas/matplotlib magic methods- Github integration- Easily Sharable
Help us make ArcPy better!
• GeoNet- http://geonet.esri.com- https://geonet.esri.com/community/developers/gis-developers/python
• ArcGIS Ideas- http://ideas.arcgis.com
• Conda Support Email- [email protected]
• ArcPy Café- https://arcpy.wordpress.com/
Conclusion
• ArcGIS- Industry-leading GIS Platform- Big push to move towards Python- Really big customers (NASA, ESA, Governments, Fortune 500 companies…)- Lots of opportunity for Python programmers
• ArcPy- Moving from Python 2 to 3- Conda support- Start in ArcGIS Pro if you’re learning
• We are hiring! Talk to me if you are interested in learning more.
Thank You!Clinton Dow - Contact Info
[email protected]://lnked.in/cdow
http://www.slideshare.net/ClintonDow/arcpy-overview-slides