Python Workshop

Beau Williamson

Python Workshop-rev4 1

Housekeeping• Please silence cellphones

– If you must take a call, please step outside

• If we hear your cellphone go off, . . . . . . you have to buy beer for everyone after the meeting.

Python Workshop 2

Opening Statement• I am not an Expert on Python Programming

– You may ask me questions at any time but . . .• There are no bonus points for “stumping” the presenter

– Other people here my have some/more experience with Python• If so, please speak up• We want to benefit from your knowledge too

Python Workshop 3

Equipment Prerequisites• Windows 7 or 8.1 or 10 Laptop• Intel 64-bit processors are preferred.• 8 GB or more of RAM• Browser Requirements: Chrome• Administrator Rights on Laptop

– Or at least ability to install new software• Software Installers

– Links listed in March 2018 Newsletter• If you didn’t bother to pre-stage those files, look over someone’s shoulder that

did

Python Workshop 4

Objectives• Setup a quality Python IDE programming environment

on your Laptop • Understand how to leverage features of PyCharm to

streamline and improve Python scripting• Accessing Python Library packages • Accessing GitHub Python packages

Python Workshop 5

Non–Objectives• Teach you the fundamentals of programming

– This is not intended to be your first class in or exposure to programming• (Although for some of you it might just be)

– Ideally you should already know some fundamentals of programming

• Teach you programming in Python– That’s a topic that just can’t be covered in two hours– There are a LOT of good resources to learn programming in Python

Python Workshop 6

Learning Python – Online Courses• Some Basic Python Programming Courses

– Coursera Python Programming Course(s)– Udacity Python Programming Course– Code Academy Python Programming Course

Python Workshop 7

Learning Python – Books

Learn Python the Hard WayAuthor: Zed A. Shaw

– Online Html version• https://learnpythonthehardway.org/book/

– Highly recommended– Fun to ready. Easy to follow.

DFWCUG Python Study Group– Any volunteers to start one?

Python Workshop 8

Workshop Agenda• Lesson 1: Basic Python Tools• Lesson 2: PyCharm IDE• Lesson 3: Python Libraries & Resources• Lesson 4: Accessing Network Devices• Lesson 5: YDK-py

Python Workshop 9

LESSON 1Basic Python Tools

Python Workshop 10

Lesson 1: Basic Python Tools• Agenda

– Installing Notepad++ for Windows– Installing Python 2.7 for Windows– Introduction to IDLE– Using pip to install/manage Python Packages from PyPI– Installing Python 3.6 for Windows– Using py to install/manage Python Packages for 2.7 & 3.6– Installing GIT for Windows– Cloning Packages from GIT repositories to your laptop– Installing cloned Python packages– Other Recommended Tools

Python Workshop 11

Installing NotePad++• Notepad++

– A source code editor and Notepad replacement– Supports several languages including Python– https://notepad-plus-plus.org/download/

• Download the 64-bit x64 version

Python Workshop 12

Installing NotePad++

Python Workshop 13

• Take all the Defaults

Installing NotePad++• Install the Plugin Manager

https://github.com/bruderstein/nppPluginManager/releases– Copy PluginManager.dll file to Notepad++ plugin folder

• C:\Program Files\Notepad++\plugins – Requires Admin Privs

– Restart Notepad++ (if running)

Python Workshop 14

Installing NotePad++

Python Workshop 15

Launch Plugin Manager• Plugins-->Plugin Manager-->Show Plugin

Manager• Check the “Compare” plugin• Click “Install”

Installing Python 2.7 for Windows• https://www.python.org/downloads/release/python-2713/

Python Workshop 16

Installing Python 2.7 for Windows

Python Workshop 17

Recommendation:– Install in “C:\Python27\” (default)

Take all other defaults Add Environment Variable

– From Windows Powershell type [Environment]::SetEnvironmentVariable("Path", "$env:Path;C:\Python27", "User")

– May not be necessary– Type “python” at command prompt

• If it comes up, you are good to go.

Introduction to IDLE

Python Workshop 18

From Start Menu Find “IDLE (Python GUI)”

Introduction to IDLE

Type <Tab> for Auto-Completion

Python Workshop 19

Introduction to IDLE• Type <Tab> for Auto-Completion

Python Workshop 20

Introduction to IDLE

Python Workshop 21

Using pip to install/manage Python Packageshttps://docs.python.org/2.7/installing/index.html

• List installed Packagespython -m pip list

• Search for a Packagepython -m pip search SomePackage

• Install a Packagepython -m pip install SomePackage

• Install a specific version of a Packagepython -m pip install –upgrade SomePackage==1.0.4

Python Workshop 22

Using pip to install/manage Python Packages• Install a minimum version of a Package

python -m pip uninstall SomePackage>=1.0.4

• Show information about an installed Packagepython -m pip show SomePackage

• Upgrade a Packagepython -m pip install –upgrade SomePackage

• Remove a Packagepython -m pip uninstall SomePackage

Python Workshop 23

Using pip to install/manage Python Packages• Package location

– Packages are installed in Lib\site-packages directory of each Python version

• Examples:– C:\Python27\Lib\site-packages– C:\Python36\Lib\site-packages

• PIP Documentation– https://pip.pypa.io/en/stable/

Python Workshop 24

Python Package Index

https://pypi.python.org/pypi

Python Workshop 25

Installing Python 3.6 for Windows

https://www.python.org/downloads

Python Workshop 26

Installing Python 3.6 for Windows

Recommendations:• Choose Customize Installation• Add Python 3.6 to your PATH

Python Workshop 27

Installing Python 3.6 for Windows

Take the default options• Click “Next”

Python Workshop 28

Installing Python 3.6 for Windows

Recommendations:• Install for all users on your Laptop• Change location to “C:\Python36”

Click “Install”

Python Workshop 29

Installing Python Packages in 2.7 & 3.6• Problem:

– pip will default to first version (2.7 or 3.6) in your %PATH% variable– Trying to install packages in both versions can be challenging

• Installing Packages the Hard Way– cd to C:\PythonXX\Scripts

– Where XX is Python version– Execute pipn.m.exe to install into version n.m of Python– Examples:

cd C:\Python36\Scriptspip3.6 install SomePackage

cd C:\Python27\Scriptspip2.7 install SomePackage

Python Workshop 30

Using py to install/manage Python Packages • Installing the Easy Way

– Use the Windows py Python launcher with the -m switch• Specify the Python version using the version switch (e.g. -3.6, -2.7)

– Example – Install SomePackage in Python3.6:py -3.6 –m pip install SomePackage

– Example – Install SomePackage in Python2.7:py -2.7 –m pip install SomePackage

Python Workshop 31

Git, Git Repositories and GitHub• Git – A Distributed Version Control System

– Allows groups of people to work on the same documents (often code) at the same time, and without stepping on each other's toes.

– Often used to publish code

• Git Repositories– Contains a directory structure of the files/documents.

• GitHub– A public set of Git Repositories– Contains many Python packages not necessarily on PyPi

Python Workshop 32

Git, Git Repositories and GitHub

Python Workshop 33

Installing Git for Windows

https://git-scm.com/download/win

Python Workshop 34

Installing Git for Windows

Launch Installer Click “Next”

Python Workshop 35

Installing Git for Windows

Recommendation:• Accept the defaults Click “Next”

Python Workshop 36

Installing Git for Windows

Recommendations:• Accept the Defaults

Click “Next”

Python Workshop 37

Installing Git for Windows

Recommendation:• Accept Defaults

Click “Next”

Python Workshop 38

Installing Git for Windows

Recommendation:• Accept Defaults

Click “Next” Add slide for OpenSSH

Python Workshop 39

Installing Git for Windows

Recommendation:• Accept Defaults

Click “Install”

Python Workshop 40

Git Basics• Download a git project and version history to local

directorygit clone url

• Download a git project to different local directory namegit clone url local-dir-path

• Examples:git clone https://github.com/adlhancock/morse.git

git clone https://github.com/adlhancock/morse.git MyMorse

Python Workshop 41

Git Basics• Check to see if a git project has updated files

cd <local git directory>git fetch

– It will say “Your branch is behind . . . ” if not up-to-date

– Otherwise it will return to command promptPython Workshop 42

PS C:\Users\bwilliam\Documents\git\kicad-library> git fetchremote: Counting objects: 273, done.remote: Total 273 (delta 152), reused 152 (delta 152), pack-reused 121Receiving objects: 100% (273/273), 1.78 MiB | 2.50 MiB/s, done.Resolving deltas: 100% (172/172), completed with 55 local objects.From https://github.com/KiCad/kicad-library

40685cc6..bb856328 master -> origin/master* [new branch] revert-1875-MCP1825S -> origin/revert-1875-MCP1825SPS C:\Users\bwilliam\Documents\git\kicad-library> git statusOn branch masterYour branch is behind 'origin/master' by 81 commits, and can be fast-forwarded.(use "git pull" to update your local branch)

Git Basics• Update your copy of a git project with the latest updates

cd MyMorsegit pull

– Will first perform a “fetch” to update status and then download changed files

Python Workshop 43

Git – Learning more

Try Git 15 minute walk-through of Git

https://try.github.io/levels/1/challenges/1

Python Workshop 44

Git – Reference

https://progit2.s3.amazonaws.com/en/2016-03-22-f3531/progit-en.1084.pdf

Excellent Book on Git Keep a copy on your bed stand

• Makes a great children’s bedtime story“Once upon a time, Fuzzy-wuzzy cloned a repository”

Python Workshop 45

Cloning your first package using GIT

Create a “git” folder under “Libraries->Documents” Right-Click on the “git” folder

and select “Git-Bash Here”

Python Workshop 46

Cloning your first package using GIT• At the bash prompt, type:

git clone https://github.com/CiscoDevNet/coding-skills-sample-code.git

Python Workshop 47

Cloning your first package using GIT• Display cloned directory structure, by typing:

cd coding-skills-sample-code

ls -la

Python Workshop 48

Installing cloned Python packages• Using Python “wheel” (.whl) file

python –m pip install somepackage.whl– Wheel files often found under a “dist” directory in the repository

• Using setup.py– Find directory in repository with “setup.py”

• Typically top-level directory– From within that directory type:python –m pip install .orpy -2.7 –m pip install .

Python Workshop 49

Other Highly Recommended Tools• Cygwin

– Large collection of GNU and Open Source tools (ls, cat, vi, ...) which provide functionality similar to a Linux distribution on Windows.

– Home Page: https://cygwin.com/– 64-bit Installer: https://cygwin.com/setup-x86_64.exe

Python Workshop 50

LESSON 2PyCharm

Python Workshop 51

Lesson 2: PyCharm• Installation, Overview and Setup of PyCharm (Videos: QT, 1, 2)

https://www.jetbrains.com/pycharm/download/#section=windows

• Running Code (Video: 3)– Interpreter Selection– Install/Manage Python Packages with PyCharm

• Coding and Code Navigation (Videos: 4, 5)– Quick Documentation

• Debugging (Video: 6)– Breakpoints, Single-Step, Watch-points

Python Workshop 52

LESSON 3Python Language Libraries & Resources

Python Workshop 53

Libraries (Bookmark these!)

• Python Tutorialhttps://docs.python.org/2.7/tutorial/index.html

– Good basic “go-to” Python Language reference– You should already know most of this but may need to refresh your memory

• Python Language Referencehttps://docs.python.org/2.7/reference/index.html#reference-index

– Guide to just about everything related to the Python Language• Syntax, Grammar, Expressions, etc.

• Python Library Referencehttps://docs.python.org/2.7/library/index.html

– Think of this as your “Python Power Tools”– Keep it under your virtual pillow

Python Workshop 54

Libraries (Bookmark these!)

• Python HOWTO’shttps://docs.python.org/2.7/howto/index.html

– Be sure to at least check out:• Logging HOWTO• Regular Expressions HOWTO• HOWTO Fetch Internet Resources Using urllib2

– Especially useful when formatting REST API calls• Argparse Tutorial

– Used to parse command line arguments

• Python FAQ’s– https://docs.python.org/2.7/faq/index.html– You may wish to check out “Python on Windows FAQ”

Python Workshop 55

Python Library Reference – A Closer Look• Keep these “Python Power Tools” handy while coding

– Built-in Types• str, set, dict and File Objects

– String Services• string – Common string operations• re – Regular Expressions operations

– Numeric and Mathematical Modules• math – Mathematical functions• random – Generate pseudo-random numbers

– Internet Data Handling• json – JSON encoder and decoder• random – Generate pseudo-random numbers

• Many others– Explore and develop your list of favorite “Python Power Tools”

Python Workshop 56

Resources• PEP8

– https://www.python.org/dev/peps/pep-0008/– Style Guide for Python Code

• PyCharm will flag your code that doesn’t follow PEP8• Get in the habit of following PEP8 coding style early in your coding

• Google– Enter “Python <some programming task> example”

• Don’t get me started on how we had to write code in the old days before Google

Python Workshop 57

LESSON 4Accessing Network Devices

Python Workshop 58

Accessing Network Devices• Paramiko• Netmiko

– (Coming DFWCUG Presentation by K. Byers)

• TextFSM• CiscoConfigParse

Python Workshop 59

ParaMiko• Overview

– A Python implementation of SSH with a whole range of supported features.• SSH• SFTP

• Pros– Extensive set of API’s covers a broad range of SSH/SFTP features & functions

• http://docs.paramiko.org/en/2.1/• Cons

– No built-in “Expect” functionality– No built-in Device prompt/response knowledge– Typically requires use of time.sleep() calls or complex prompt recognition code

Python Workshop 60

Paramiko – Simple Cisco Router Example# Use invoke_shell to establish an 'interactive

session'remote_conn = remote_conn_pre.invoke_shell()print "Interactive SSH session established"

# Strip the initial router promptoutput = remote_conn.recv(1000)

# See what we have for initial router promptprint output

# Turn off pagingdisable_paging(remote_conn)

# Now let's try to send the router a commandremote_conn.send("\n")remote_conn.send("show ip int brief\n")

# Wait for the command to completetime.sleep(2)

output = remote_conn.recv(5000)print output

import paramikoimport time

def disable_paging(remote_conn):'''Disable paging on a Cisco router'''remote_conn.send("terminal length 0\n")time.sleep(1)

# Clear the buffer on the screenoutput = remote_conn.recv(1000)return output

if __name__ == '__main__':# VARIABLES THAT NEED CHANGEDip = '1.1.1.16'username = 'testuser'password = 'password'

# Create instance of SSHClient objectremote_conn_pre = paramiko.SSHClient()# Automatically add untrusted hosts (make sure okay for security policy in your environment)remote_conn_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy())

# initiate SSH connectionremote_conn_pre.connect(ip, username=username, password=password, look_for_keys=False,

allow_agent=False)

print "SSH connection established to %s" % ip

Python Workshop 61

NetMiko• Features

– Successfully establish an SSH connection to the device– Simplify the execution of show commands and the retrieval of output data– Simplify execution of configuration commands including possibly commit

actions– Do the above across a broad set of networking vendors and platforms

Python Workshop 62

Netmiko• Platform support (partial list)

Platform Device_Type

Cisco IOS cisco_ios

Cisco IOS-XE cisco_xe

Cisco ASA cisco_asa

Cisco NX-OS cisco_nxos

Cisco IOS-XR cisco_xr

Cisco WLC cisco_wlc

Arista EOS arista_eos

HP ProCurve hp_procurve

Platform Device_Type

HP Comware hp_comware

Juniper juniper

Juniper JUNOS juniper_junos

Brocade VDX brocade_vdx

Brocade brocade_nos

F5 LTM f5_ltm

A10 a10

Alcatel SROS alcatel_sros

Platform Device_Type

Linux linux

OVS_Linux ovs_linux

Extreme extreme

Fortinet fortinet

Dell Force10 dell_force10

Aruba aruba_os

Huawei Huawei

generic termserver generic_termserver

Python Workshop 63

TextFSM – Text Finite State Machine• Description (courtesy of Jason Edelman’s Blog)

TextFSM is a state machine (basically like a templating engine) that is purpose built to simplify working with regular expressions and getting structured data out of traditional network devices.

• Using TextFSM– Obtain network device output (NetMiko, Paramiko, etc.)– Process using a TextFSM template– Produces structured data.

Python Workshop 64

TextFSM Information• TextFSM Overview

– Overview of TextFSM and its Finite State Machine Template files– https://github.com/google/textfsm/wiki/TextFSM

• Code Lab– Tutorial on writing TextFSM Templates

• Covers important Value Options (Filldown, Required, List, . . .) – https://github.com/google/textfsm/wiki/Code-Lab

• CLI Tables– https://github.com/google/textfsm/wiki/Cli-Table

• Jason Edelmann’s Blog– http://jedelman.com/home/programmatic-access-to-cli-devices-with-textfsm/

Python Workshop 65

CiscoConfParse• Parses Config Files

– Uses “Parent/Child” relationships of configuration

• Documentationhttp://ciscoconfparse.readthedocs.io/en/latest/

Python Workshop 66

CiscoConfParse• Example

#!/usr/bin/env pythonfrom ciscoconfparse import CiscoConfParse

parse = CiscoConfParse("../configs/sample_01.ios")

# Return a list of all ATM interfaces and subinterfacesatm_intfs = parse.find_lines("^interface\sATM")

# Return a list of all interfaces with a certain QOS policyqos_intfs = parse.find_parents_w_child( "^interf", "service-policy" )

# Return a list of all active interfaces (i.e. not shutdown)active_intfs = parse.find_parents_wo_child( "^interf", "shutdown" )

Python Workshop 67

LESSON 5YDK-py

Python Workshop 68

YDK-py• Ok, We Got YANG Data Models, Now What?

Santiago Alvarez, NANOG 68; Dallas, TX– Slides

• https://www.nanog.org/sites/default/files/20161017_Alvarez_Ok_We_Got_v1.pdf– Video

• https://www.youtube.com/watch?v=2oqkiZ83vAA&list=PLO8DR5ZGla8hcpeEDSBNPE5OrZf70iXZg&index=36

Python Workshop 69

YDK-py• Getting Started

– Git-Hub• https://github.com/CiscoDevNet/ydk-py.git

– Documentation• http://ydk.cisco.com/py/docs/

– Cisco dCloud Lab• https://dcloud2-

rtp.cisco.com/content/demo/308588?returnPathTitleKey=content-view

Python Workshop 70