TRAINING MATERIALS FOR IT PROFESSIONALS · Accompanying Class Files This manual comes with accompanying class files, which your instructor or sales representative will point out to

•

TRAINING MATERIALS FOR IT PROFESSIONALS EVALUATION COPY

Unauthorized Reproduction or Distribution Prohibited

Python Data Analysis with NumPy and pandas

(PYT252 version 1.1.0)

Copyright Information

© Copyright 2019 Webucator. All rights reserved.

The Authors

Nat Dunn

Nat Dunn founded Webucator in 2003 to combine his passion for technical trainingwith his business expertise and to help companies benefit from both. His previousexperience was in sales, business and technical training, and management. Nathas an MBA from Harvard Business School and a BA in International Relationsfrom Pomona College.

Roger Sakowski (Editor)

Roger has over 35 years of experience in technical training, programming, datamanagement, network administration, and technical writing for companies such asNASA, Sun Microsystems, Bell Labs, GTE, GE, and Lucent among other Fortune100 companies.

Stephen Withrow (Editor)

Stephen has over 30 years' experience in training, development, and consulting ina variety of technology areas including Java, C, C++, XML, JavaScript, AJAX,Tomcat, JBoss, Oracle, and DB2. His background includes design andimplementation of business solutions on client/server, Web, and enterprise platforms.Stephen is a published writer in both technical and non-technical endeavors.Stephen received an undergraduate degree in Computer Science and Physics fromFlorida State University.

EVALUATION COPY


Accompanying Class Files

This manual comes with accompanying class files, which your instructor or salesrepresentative will point out to you. Most code samples and exercise and solutionfiles found in the manual can also be found in the class files at the locations indicatedat the top of the code listings.

Due to space limitations, the code listings sometimes have line wrapping, whereno line wrapping occurs in the actual code sample. This is indicated in the manualusing three greater than signs: >>> at the beginning of each wrapped line.

In other cases, the space limitations are such that we have inserted a forced linebreak in the middle of a word. When this occurs, we append the following symbolat the end of the line before the actual break: »»

EVALUATION COPY


Table of Contents1. Jupyter Notebook..................................................................................1

Getting Started with Jupyter Notebook.............................................................1Exercise 1: Creating Your First Jupyter notebook..............................................2Exercise 2: More Experimenting with Jupyter Notebook...................................7Getting the Class Files....................................................................................10Markdown.......................................................................................................13Magic Commands...........................................................................................13

Automagic...............................................................................................13Autosave.................................................................................................14Directory Commands .............................................................................14Bookmarking...........................................................................................14Command History...................................................................................15Last Three Inputs and Outputs................................................................15Environment Variables............................................................................15Loading and Running Code from Files...................................................16Shell Execution.......................................................................................16More Magic Commands..........................................................................17

Getting Help....................................................................................................17

2. NumPy..................................................................................................19

Efficiency........................................................................................................19NumPy Arrays.................................................................................................21

Getting Basic Information about an Array...............................................21np.arange()..............................................................................................21Similar to Lists.........................................................................................23Different from Lists..................................................................................24Universal Functions.................................................................................25

Exercise 3: Multiplying Array Elements...........................................................31Multi-dimensional Arrays.................................................................................35Exercise 4: Retrieving Data from an Array......................................................39

Modifying Parts of an Array.....................................................................42Adding a Row Vector to All Rows............................................................42More Ways to Create Arrays...................................................................42Getting the Number of Rows and Columns in an Array..........................43

Random Sampling..........................................................................................44Exercise 5: Rolling Doubles............................................................................47Using Boolean Arrays to Get New Arrays.......................................................49More with NumPy Arrays................................................................................50

iVersion: 1.1.0. Printed: 2019-04-02.

Table of Contents

EVALUATION COPY


3. pandas..................................................................................................51

Series..............................................................................................................52Other Ways of Creating Series................................................................53np.nan.....................................................................................................54Accessing Elements from a Series.........................................................56

Exercise 6: Retrieving Data from a Series......................................................57Series Alignment.....................................................................................59

Exercise 7: Using Boolean Series to Get New Series.....................................62Comparing One Series with Another......................................................65Element-wise Operations and the apply() Method..................................65Series: A More Practical Example...........................................................67

DataFrame......................................................................................................72Creating a DataFrame from a NumPy Array...........................................73Creating a DataFrame using Existing Series as Rows...........................75Creating a DataFrame using Existing Series as Columns......................76Creating a DataFrame from a CSV.........................................................77Exploring a DataFrame...........................................................................78Getting Columns ....................................................................................80

Exercise 8: Exploring a DataFrame................................................................82Cleaning Data.........................................................................................87Getting Rows...........................................................................................87Combining Row and Column Selection...................................................91Scalar Data: at[] and iat[].........................................................................93Boolean Selection...................................................................................93Using a Boolean Series to Filter a DataFrame........................................95

Exercise 9: Series and DataFrames...............................................................97Plotting with matplotlib..................................................................................100

Inline Plots in Jupyter Notebook............................................................101Line Plot................................................................................................102Bar Plot.................................................................................................106Annotation.............................................................................................109

Exercise 10: Plotting a DataFrame................................................................111Other Kinds of Plots..............................................................................115

© Copyright 2019 Webucator. All rights reserved.ii

Table of Contents

EVALUATION COPY


Jupyter Notebook1.In this lesson, you will learn...1. How to use Jupyter Notebook for Python development.

IPython stands for Interactive Python. It provides an enhanced command shell, theIPython console, and a browser-based programming environment called JupyterNotebook. This lesson will focus on the Jupyter Notebook, which we will usethroughout the rest of the course.

Getting Started with Jupyter Notebook1.1

To get started with Jupyter Notebook, run jupyter notebook in a commandshell. You will get output similar to the following:

C:\Users\Nat> jupyter notebook[I 16:52:17.418 NotebookApp] Using MathJax from CDN:https://cdn.mathjax.org/mathjax/latest/MathJax.js[I 16:52:17.434 NotebookApp] Serving notebooks from local directory:C:\Users\Nat[I 16:52:17.450 NotebookApp] 0 active kernels[I 16:52:17.450 NotebookApp] The Jupyter Notebook is running at:http://localhost:8888/ [I 16:52:17.450 NotebookApp] Use Control-Cto stop this server and shut down all kernels (twice to skip con »» firmation).

A browser window or tab should automatically open showing the IPython dashboard.If, for some reason, a browser window does not open, you should be able to enterthe URL of the Jupyter Notebook in your browser's location bar to load the dashboard.That URL is in bold in the output above, but it could be slightly different on yourcomputer.

Page 1 of 115Version: 1.1.0. Printed: 2019-04-02.

Jupyter Notebook

EVALUATION COPY


Creating Your First Jupyter notebookExercise 15 to 10 minutes

In this exercise, which we will do together, you will create your first Jupyternotebook.

© Copyright 2019 Webucator. All rights reserved.Page 2 of 115

Jupyter Notebook

EVALUATION COPY


1. Click New > Python 3 to start a new notebook:

2. A new browser tab should open with a new untitled notebook:

3. Click on Untitled to pop up a Rename Notebook prompt. Name your newnotebook "My First Notebook":

4. A notebook is made up of cells, in which you write your code. The notebookstarts with a single cell. Click on the cell to enter text.


Jupyter Notebook

EVALUATION COPY


5. Type print('Hello, world!') and press Ctrl+Enter. Your notebookshould now look like this:

6. Save your file by clicking on the Save icon (below the Filemenu) or by pressingCtrl+S.

7. Return to your Home tab:

8. You should see a new file listing for My First Notebook.ipynb9. Return to My First Notebook.ipynb and leave it open. We'll do some more

work on it shortly.


Jupyter Notebook

EVALUATION COPY


Jupyter Notebook Modes

Jupyter notebooks have two modes:

1. Edit mode - for editing text in cells. Keyboard shortcut: Enter2. Command mode - for navigating around the notebook. Keyboard shortcut:

Esc

Useful Shortcut Keys

• Enter - Enter edit mode.• Esc - Enter command mode.• Ctrl+Enter - Run code in current cell.• Shift+Enter - Run code in current cell and select next cell (inserting new one

if current cell is the last cell in the notebook). Enter command mode.• Alt+Enter - Run code in current cell and insert new cell below. Enter edit

mode.• A (in command mode) - Insert cell above current cell.• B (in command mode) - Insert cell below current cell.• D,D (in command mode) - Delete cell.• L (in command mode) - Show/Hide line numbers.

In command mode, you can use the Up and Down arrow keys to navigate betweencells.


Jupyter Notebook

EVALUATION COPY


Code Completion

Use Tab for code completion:

A full list of shortcut keys is available from the Help menu.

Enclosing Code Snippets

In Jupyter Notebook, you can enclose text with quotation marks, parentheses, andbrackets by highlighting the text and then pressing the opening enclosing symbol.For example, if you have the following text:

print(Hello, World!)

You can wrap Hello, World! in quotes by highlighting it and pressing thequotation mark key.

You can do the same with any of the following characters:

1. " - Wrap in double quotes.2. ' - Wrap in single quotes.3. ( - Wrap in parentheses.4. [ - Wrap in square brackets.5. { - Wrap in curly brackets.


Jupyter Notebook

EVALUATION COPY


More Experimenting with Jupyter NotebookExercise 215 to 20 minutes

In this exercise, we will continue to experiment with our first Jupyter notebook.

1. In your My First Notebook notebook, insert a cell below the current one usingeither of these two methods:A. Use a shortcut key (B, Alt+Enter, or Shift+Enter).B. Select Insert Cell Below from the Insert menu:

2. In the second cell, enter 1+1 and press Ctrl+Enter:

3. The Out[#] line shows you the return value of the last operation in the cell.Notice that the printed value ("Hello, world!") in the upper cell has no Out[#]label. That is because the print() function does not return anything; it justprints.

4. To illustrate, navigate to the first cell using the mouse or the arrow keys andchange the code to read:

result = print('Hello, world!')print(result)

5. PressCtrl+Enter to run the cell. The print() function will still run, printing"Hello, world!", but because print() returns no value, resultwill containNone:

Hello, world!None

6. Let's take this a little further. Insert a new cell below the current one.


Jupyter Notebook

EVALUATION COPY


7. Type the following function:

def my_print(text):print(text)return 'I printed ' + text

8. Press Alt+Enter to run the cell and insert a new cell below it.9. In the new cell, type:

result = my_print('Hello, world!')print(result)

10. Press Ctrl+Enter to run the cell. The my_print() function prints "Hello,world!", but it also returns a message. When we call print(result) itprints that message:

Hello, world!I printed Hello, world!

11. Your notebook should look something like this now:

Don't worry if your input and output numbers are different.12. Enter edit mode in the cell that contains the call to my_print() (In [5] in

the screenshot above).


Jupyter Notebook

EVALUATION COPY


13. At the end of the cell, press Enter and type result. Then press Ctrl+Enterto run the cell. The result should look like this:

14. Notice the difference between printing and returning. Calls to print() willwrite text below the code, but it will not be labeled withOut[#]. Also note thatevery call to print() will be printed, but only the final output returned bythe cell code will be output. To illustrate this, enter edit mode in the final cell,which currently should contain 1+1. Add a new line that reads 2+2 and pressCtrl+Enter:

Notice that only the last result is output.


Jupyter Notebook

EVALUATION COPY


Getting the Class Files1.2

You may have done this already as part of setting up your computer for class.

We will use Jupyter Notebook to download and unzip the class files used in thiscourse.


Jupyter Notebook

EVALUATION COPY


1. Click New > Python 3 to start a new notebook:

2. A new browser tab should open with a new untitled notebook:

3. Click on Untitled to pop up a Rename Notebook prompt. Name your newnotebook "Temp":

4. Enter %load http://bit.ly/pyt211-py in the first cell:


Jupyter Notebook

EVALUATION COPY


5. Press Ctrl+Enter to load a script:

6. Press Ctrl+Enter again to run the script. You will be prompted to enter aCourse ID, which is most likely PYT211. If it's different, your instructor willlet you know. Enter the ID and press Enter.

7. Once the files are downloaded and unzipped, which may take a minute, thenumber 1 should appear in the square brackets to the left of cell 1:

8. Close this tab and return to your Jupyter NotebookHome tab. It should includethe ClassFiles folder:

If it doesn't, you may need to refresh the listing.


Jupyter Notebook

EVALUATION COPY


Markdown1.3

Python notebook supports Markdown, a simple formatting language that getsconverted to HTML. To convert the selected Jupyter notebook cell to Markdown,pressM or selectMarkdown from the drop-down menu on the toolbar:

This course does not cover Markdown, but the language is very easy to learn. Ifyou're new to it, you may want to check out ipython-notebook/Demos/Markdown.ipynb during a break. You may find the Markdown language useful for takingnotes directly in the Jupyter notebook files used in this class.

Magic Commands1.4

IPython includes a list of more than 100 "magic" commands that can be used in theIPython shell and in Jupyter Notebook. These are not Python commands, but extracommands that help make development easier.

Magic commands are generally prefixed with a percent sign (%); however, that canbe configured. Let's start with that.

AutomagicThe %automagic command is used to toggle whether magic functions are callablewithout having to type the initial %.

1. Create a new Jupyter notebook and name it "Magic".


Jupyter Notebook

EVALUATION COPY


2. In the first cell, type %automagic and press Ctrl+Enter. This will togglethe ON/OFF value of automagic:

3. If the output says "Automagic is ON" then you do not need the % prefix formagic commands. If it says "Automagic is OFF" then you do need the % prefixfor magic commands.

Whether automagic is on or off, using the % prefix is always allowed. In notebooksyou intend to reuse or share, it's a good idea to include the % prefix so the scriptsare not dependent upon the state of automagic.

AutosaveThe %autosave command is used set the autosave interval in the notebook (inseconds):

• %autosave 60 - sets the autosave interval to 60 seconds• %autosave 0 - disables autosave.

Directory Commands• %cd <dir> - changes current directory to <dir>.• %pwd - return current working directory

BookmarkingThe %bookmark command is used to create bookmarks to specific directories tomake it easy to navigate within IPython.

• %bookmark <name> - sets <name> bookmark to current dir.• %bookmark <name> <dir> - sets <name> bookmark to <dir>.• %bookmark -l - lists all bookmarks.


Jupyter Notebook

EVALUATION COPY


• %bookmark -d <name>- removes <name> bookmark.• %bookmark -r - removes all bookmarks.

Let's create a bookmark for the current directory:

1. In the Jupyter notebook named "Magic", add a new cell, type %ls, and pressCtrl+Enter to list the contents of the current directory.

2. If you are not already in the ClassFiles directory, use the %cd command tonavigate to it.

3. Once you are in the ClassFiles directory, type %bookmark Webucator andpress Shift+Enter.

4. In the new cell that comes up, type %cd Webucator and press Ctrl+Enterto test your bookmark:

Command HistoryThe %history command is used to get a history of your input commands. Type%history in a cell and press Ctrl+Enter to see the results.

Last Three Inputs and OutputsThe letter "i" preceded by a single underscore (_i) gets the previous input, precededby a double underscore (__i) gets the one before that, and preceded by a tripleunderscore (___i) gets the one before that. A quadruple underscore + i (____i)returns an error.

A single underscore (_) gets the previous output, a double underscore (__) gets theone before that, and a triple underscore (___) gets the one before that. A quadrupleunderscore (____) returns an error.

Environment VariablesThe %env command is used to list all environment variables and their values.

You can also get specific values using %env var. For example, the code belowwould get the value of PATH.


Jupyter Notebook

EVALUATION COPY


%env PATH

And you can assign the result to a variable. For example, the following code (fromDemos/List Path Entries.ipynb) lists all the values in the path list:

env_vars = %env# path = env_vars['PATH'].split(':') # for Mac / Unixpath = env_vars['PATH'].split(';') # for Windows

for i, item in enumerate(path,1):print(i, '. ', item, sep='')

You can also assign environment values using %env var=val or %env varval.

Loading and Running Code from Files• %load <source> - Loads code from <source> into cell.• %run <source> - Runs code from <source> in IPython.

Try it out:

1. In the Jupyter notebook named "Magic", add a new cell.2. To navigate to the right directory, type %cd Webucator press Ctrl+Enter

and then overwrite that with %cd ipython-notebook/Demos, and pressShift+Enter.

3. In the new cell, type %load hello_world.py and press Ctrl+Enter. Itshould load the contents of hello_world.py and comment out the %loadcommand:

4. Press Alt+Enter to run the code and add a new cell. It should print "Hello,world!".

5. In the new cell, type %run hello_world.py and press Ctrl+Enter. Thiswill run the file without loading its contents into the cell.

Shell Execution• ! - Runs shell command and captures output as a string.

• Try !ping google.com


Jupyter Notebook

EVALUATION COPY


• %sx (!! is shortcut) - Runs shell command and captures output as a formattedlist split on new line (\n) characters.• Try !!ping google.com

• %system - Alias for %sx.

More Magic CommandsWe have gone through some of the most useful magic commands, but there are manymore.

• Use the %lsmagic command to get a full listing of the magic commands.• Use %command_name? to get help on any specific magic command.• For general documentation on magic commands, run %magic and

%quickref.

Getting Help1.5

There are several ways to get help in Jupyter Notebook:

1. object? - Get short documentation on an object. Try

import datetimedatetime.datetime?

2. object?? - Get extended documentation on an object. Try

datetime.datetime??


Jupyter Notebook

EVALUATION COPY


3. help() - Python interactive help.

4. help(object) - Get standard Python documentation on an object. Try:

help(datetime.datetime)

5. Press Shift+Tab while inside the parentheses of a function to bring up help onthat function.

Conclusion1.6In this lesson, you have learned to get around Jupyter Notebook.


Jupyter Notebook

EVALUATION COPY


NumPy2.In this lesson, you will learn...1. The purpose of NumPy.2. How to create, modify and extract data from one-dimensional and

two-dimensional NumPy arrays.3. How to use boolean arrays to create new arrays.

NumPy, which is included with Anaconda, is a Python package for working witharrays andmatrices (two-dimensional arrays) of numbers. Although you can domosteverything NumPy can do with Python's core features, NumPy makes working withlarge quantities of numeric data easier andmore efficient.We'll start by demonstratinghow fast NumPy runs and then we'll give an overview of its most useful features.

Jupyter Notebook Examples

All examples from this section are available in numpy/Demos/NumPy Basics.ipynb.

NumPy is almost always imported as np:

import numpy as np

Efficiency2.1

Let's compare the time it takes to create a NumPy array of the first 10 squares (i.e.,0, 1, 4, 9, 16, 25, 36, 49, 64, 81) to the time it takes to create a standard Python list


NumPy

EVALUATION COPY


of those same squares. We'll start with the Python list and use a list comprehensionto create it:

And here is how we create a NumPy one-dimensional array, which is similar to aPython list, of the first 10 squares:

Now let's use timeit to compare the two:

As you can see, the NumPy array is created more than three times as fast. And thedifference is even more noticeable with large objects. Check out the difference whenwe create a list/array with 1,000 squares:

In this case, NumPy is about 140 times more efficient!


NumPy

EVALUATION COPY


NumPy Arrays2.2

NumPy arrays have a type of numpy.ndarray, which stands for n-dimensionalarray. The most straightforward way to create an ndarray is with the array()method, which can take any sequence. Here's a simple example:

Getting Basic Information about an ArrayThe following code shows how to get basic information about an array:

1. np.size() - Returns the number of elements in the array.2. np.ndim - The number of dimensions in the array.3. np.dtype - The data type of the elements within the array.

np.arange()As we saw earlier, an ndarray can also be created with the np.arange()method. Like Python's built-in range() method, np.arange() takes start,stop, and step parameters:


NumPy

EVALUATION COPY


Syntax

np.arange(stop)np.arange(start, stop)np.arange(start, stop, step)

The examples below illustrate:

In addition, the arange() method allows for fractional steps. For example:

np.arange(1,5,.5)

will produce the following array:

[ 1. 1.5 2. 2.5 3. 3.5 4. 4.5]

All elements in ndarrays must be of the same data type. In the array above, noticethat there are decimal points at the end of the whole numbers. That indicates thatthese are floats. Take a look at the following:

Notice that ar5 is an array of integers and ar6 is an array of floats. For those twoarrays, Python implicitly determined the data type:


NumPy

EVALUATION COPY


• If any of the arguments is a float, then it creates an array of floats.• lf all of the arguments are integers, then it creates an array of integers.

If you want to explicitly set the data type, you can pass in a dtype as we do forar7 above.

If you try to set an element in an array of integers to a float, the value will be coercedinto an integer as the following example illustrates:

Notice that we tried to set ar[5] to 1.5, but it was coerced into the integer 1.


NumPy

EVALUATION COPY


Similar to ListsIn some ways, ndarrays are like lists. For example, they are iterators:

Different from ListsIn other ways, ndarrays behave differently from lists. For example, considerthe following operations on a list:

1. When two lists are "added together" with the plus (+) operator, one list isappended to the other.

2. When a list is "multiplied" by a number with the asterisk (*) operator, the listis repeated that number of times.

Compare this to how the same operations work with ndarrays:

1. When two ndarrays are "added together" with the plus (+) operator, eachelement of the first array is added to the corresponding element of the secondarray to create a new array of the same length.


NumPy

EVALUATION COPY


2. When an ndarray is "multiplied" by a number with the asterisk (*) operator,that scalar is broadcast across all element of the array, meaning that eachelement of the array is multiplied by that number to return a new array of thesame length.

Mathematical operations can also be performed on arrays and lists together. Considerthe following three operations:

In each case, each element of the original array (ar1) is multiplied by 2 to create anew array of the same length.

Other mathematical operations work in the same way:

Universal FunctionsImagine you have two ranges and you want to multiply all the elements in one rangeby the corresponding elements in the other range. One way to do this is to use thebuilt-in zip() function, which takes any number of iterables and creates an iteratorof tuples from the elements at the same index in each of the iterables. For example,zipping the following two lists:


NumPy

EVALUATION COPY


[1,2,3][4,5,6]

will create an iterator with the following tuples:

(1,4), (2,5), (3,6)

The code below shows how to use zip() to create a list of products of elementsin two ranges:

Python 2 Difference

In Python 2, the zip() function returns a list object.

In Python 3, the zip() function returns a special zip object, which is an iterator.

There are many functions in NumPy that behave like this by default. Functions thatoperate on ndarrays one element at a time (i.e., element-wise) are called universal


NumPy

EVALUATION COPY


functions. NumPy includes many such built-in methods. For example, the followingcode multiplies one array by another element-wise:

NumPy's universal functions can take ndarrays or array-like objects. For example,np.add() could be used to add the elements of a list to the elements of a range,like this:

The tables below show some (but not all1) of these methods. Some of these methodsare demonstrated in the NumPy Jupyter notebook under "Some Examples ofUniversal Functions":

1. For additional documentation on NumPy's universal functions, see http://docs.scipy.org/doc/numpy/reference/ufuncs.html.


NumPy

EVALUATION COPY


http://docs.scipy.org/doc/numpy/reference/ufuncs.html

Element-wise Mathematical MethodsDescriptionMethod

Same as ar1 + ar2np.add(ar1, ar2)

Same as ar1 - ar2np.subtract(ar1, ar2)

Same as ar1 * ar2np.multiply(ar1, ar2)

Same as ar1 / ar2np.divide(ar1, ar2)

Same as ar1 // ar2np.floor_divide(ar1, ar2)

Same as ar * -1np.negative(ar)

Same as ar1 ** ar2np.power(ar1, ar2)

Same as ar1 % ar2np.remainder(ar1, ar2),np.mod(ar1,ar2)

Same as abs(ar)np.absolute(ar)

Rounds each element of ar to the nearest integer.np.rint(ar)

Returns 1 for positive numbers, -1 for negative numbers,and 0 for 0.

np.sign(ar)

Calculates the exponential of each element in ar.np.exp(ar)

Returns the natural logarithm of each element in ar.np.log(ar)

Returns the base-2 logarithm of each element in ar.np.log2(ar)

Returns the base 10 logarithm of each element in ar.np.log10(ar)

Return the positive square-root of each element in ar.np.sqrt(ar)

Return the square of each element in ar.np.square(ar)

Return the reciprocal of each element in ar.np.reciprocal(ar)

Returns the floor of each element.np.floor(ar)

Returns the ceiling of each element.np.ceil(ar)

Returns the truncated value of each element.np.trunc(ar)


NumPy

EVALUATION COPY


Element-wise Trigonometric MethodsDescriptionMethod

Returns the sine of each element.np.sin(ar)

Returns the cosine of each element.np.cos(ar)

Returns the tangent of each element.np.tan(ar)

Returns the inverse sine of each element.np.arcsin(ar)

Returns the inverse cosine of each element.np.arccos(ar)

Returns the inverse tangent of each element.np.arctan(ar)

Given "legs" of a right triangle, returns hypotenuse.np.hypot(ar1, ar2)

Converts degrees to radians for each element.np.deg2rad(ar)

Converts radians to degrees for each element.np.rad2deg(ar)

Element-wise Boolean MethodsDescriptionMethod

Returns a boolean ndarray: True if element is real.np.isreal(ar)

Returns a boolean ndarray: True if element is complex.np.iscomplex(ar)

Returns a boolean ndarray: True if element is NaN.np.isnan(ar)

NumPy: all() and any(). Not element-wise.DescriptionMethod

Returns True if all elements in array are True.np.all(ar) or ar.all()Returns True if any elements in array are True.np.any(ar) or ar.any()

The methods in the following table return boolean ndarrays.


NumPy

EVALUATION COPY


Element-wise Comparison MethodsDescriptionMethod

Same as ar1 > ar2.np.greater(ar1, ar2)

Same as ar1 >= ar2.np.greater_equal(ar1, ar2)

Same as ar1 < ar2.np.less(ar1, ar2)

Same as ar1 <= ar2.np.less_equal(ar1, ar2)

Same as ar1 != ar2.np.not_equal(ar1, ar2)

Same as ar1 == ar2.np.equal(ar1, ar2)

Compares each element of ar1 to the correspondingelement in ar2 and returns an ndarray of booleans:True if they are both true, False otherwise.

np.logical_and(ar1, ar2)

Compares each element of ar1 to the correspondingelement in ar2 and returns an ndarray of booleans:True if at least one is true, False otherwise.

np.logical_or(ar1, ar2)

Compares each element of ar1 to the correspondingelement in ar2 and returns an ndarray of booleans:True if one and only one is true, False otherwise.

np.logical_xor(ar1, ar2)

As noted in the table above, some of the methods could be replaced with logicaloperators. Some examples:


NumPy

EVALUATION COPY


Multiplying Array ElementsExercise 315 to 25 minutes

In this exercise, you will calculate the hypotenuses shown in the diagram below firstwithout NumPy and then with NumPy, and then you will compare the efficiency ofeach method:

The formula to calculate the hypotenuse of a right triangle is:

If leg1 and leg2 are equal, as in the diagram above, the formula can be simplifiedto:


NumPy

EVALUATION COPY


In Python, this can be written like this:

import mathmath.sqrt(2 * leg**2))

1. In Jupyter Notebook, navigate to numpy/Exercises and open the Hypotenusesnotebook.

2. The first cell contains the necessary import statements.3. In the second cell, write code to create a list of 100 hypotenuses for right

triangles with legs of lengths between 1 to 100. Do not use NumPy.4. In the third cell, using NumPy, write code to create an array of 100 hypotenuses

for right triangles with legs of lengths between 1 to 100.5. Once you have tested your code above, use timeit to compare the efficiency

of the two solutions. The NumPy solution should run much faster.


NumPy

EVALUATION COPY



NumPy

EVALUATION COPY


Exercise Solution

numpy/Solutions/Hypotenuses.ipynb


NumPy

EVALUATION COPY


Multi-dimensional Arrays2.3


All examples from this section are available in numpy/Demos/Multi-dimensionalArrays.ipynb.

NumPy arrays are n-dimensional, meaning they can be one, two, three, or moredimensions. However, you will mostly work with one-dimensional or


NumPy

EVALUATION COPY


two-dimensional arrays (also calledmatrices). The code below shows the most basicway of creating a two-dimensional array:

The diagram below provides a visual of the two-dimensional array created above:

1. The rows are the first dimension. To get row r of two-dimensional array mda,use mda[r]. A row of a two-dimensional array is a one-dimensional array:

2. A "cell" within a row is just an element within a one-dimensional array. To getthe second element in the third row, get the third row and then get the secondelement in that row:


NumPy

EVALUATION COPY


3. To get the "cell" in one step, use mda[r,c] (e.g., mda[2,1])

4. So, the cells within the rows (or the "columns") are the second dimension. Toget the nth "column" is the equivalent of getting the nth element of every row.A. To do this, we use slicing syntax. First, let's get all the rows:

B. In the code above, we specify 0 as the start index and len(mda) asthe stop index. But we can leave both of these out as they are the defaultvalues. So, the following will also get all the rows:


NumPy

EVALUATION COPY


C. To get only the second column, wewant to get the second element (element1) of every row. That means we are limiting the second dimension.

That can look a little confusing, so we'll make it a little bigger:

5. If we want to get the last two columns of the matrix, we can do it like this:

Or we could use negative indexing, like this:


NumPy

EVALUATION COPY


Retrieving Data from an ArrayExercise 415 to 30 minutes

The purpose of this exercise is to give you practice retrieving data from a NumPyarray.

1. In Jupyter Notebook, navigate to numpy/Exercises and open the Quiz notebook.2. Run the first cell to set up the notebook. This will import NumPy and create a

NumPy array called ar. It will also add answer boxes to be used to check youranswers to the questions in the notebook.


NumPy

EVALUATION COPY


Exercise Solution

numpy/Solutions/Quiz.ipynb

The answers are:

1. 1202. 23. 154. 85. 41616. 3596007. -40838. 69620

Here are some possible ways of arriving at the answers. There may be others:


NumPy

EVALUATION COPY



NumPy

EVALUATION COPY


Modifying Parts of an ArrayWe saw earlier that we can operate on arrays on an element-wise basis. We can dothe same with subsets of an array. For example, the code below shows how to add2 to the last two elements of the last two rows. We use negative indexing to do this:

Adding a Row Vector to All RowsWhen you add (or multiply or subtract, etc.) a vector (or an array-like object) to amulti-dimensional array, the operation is broadcast through the whole array. Thefollowing code illustrates:

More Ways to Create ArraysThe following methods provide additional ways to create NumPy arrays:


NumPy

EVALUATION COPY


1. np.zeros(shape, dtype=float) - Creates an array of zeros of specifiedshape, which must be a tuple.

2. np.reshape(ar, newshape) - Returns a reshaped array (e.g., creates atwo-dimensional array from a one-dimensional array). Use -1 for rows orcols to have the number be inferred based on the length of the array and thevalue of the other dimension.

A. The reshape() method can also be called on an ndarray object:ar.reshape(rows, cols). It does not modify the original array. Itreturns a new array.

3. np.linspace(start, stop, num) - Creates a one-dimensional arrayof evenly distributed numbers between (and including) start and stop.

This gives us five numbers equally distributed between 4 and 6.


NumPy

EVALUATION COPY


Getting the Number of Rows and Columns in an Array1. np.size(mda) - Returns the total number of elements in an array.2. np.size(mda, 0) - Returns the number of rows in an array.3. np.size(mda, 1) - Returns the number of columns in an array.4. mda.shape - A tuple of the format (rows, cols). np.shape(mda)

will return the same tuple.

Random Sampling2.4


All examples from this section are available in numpy/Demos/Random Sampling.ipynb.

You are likely familiar with Python's random module. NumPy includes acorresponding random module that performs many of the same tasks, but muchmore efficiently.


NumPy

EVALUATION COPY


NumPy Random MethodsDescriptionMethod

Returns an array of r rows and c columnscontaining random floats between 0 and 1.

np.random.rand(r, c)*

Returns an array of r rows and c columnscontaining random floats from a normaldistribution of mean 0 and variance 1.

np.random.randn(r, c)*

Returns an array of random integers fromlow(inclusive) to high(exclusive) in thespecified shape (tuple).

np.random.randint(low[, high,shape])**

Same as randint() except that the high isinclusive.

np.random.random_integers(low[,high, shape])**

Returns an array of random floats from 0.0(inclusive) to 1.0 (exclusive).

np.random.random([shape])**

Same as random().np.random.random_sample([shape])**

Same as random().np.random.sample([shape])**

Returns a random sample from aone-dimensional array. replace defaultsto True. If set to False, once an element ischosen, it will be removed from the sample.p takes a list of probabilities.

np.random.choice(ar[, shape,replace, p])**

Shuffles an array in place.np.random.shuffle(ar)

Initializes the random number generator.np.random.seed(seed)

*Can take additional dimensions for n-dimensional arrays

**If shape is not passed in, a single value is returned.

Notice that random sampling with NumPy is very very fast (1.37 ms vs. 119 ms perloop):


NumPy

EVALUATION COPY



NumPy

EVALUATION COPY


Rolling DoublesExercise 515 to 25 minutes

The following script simulates the rolling of two 6-sided dice 100,000 times andthen calculates the fraction of times doubles are rolled:

Note that when you sum a boolean array, the value of True is converted to 1 andthe value of False is converted to 0. So, sum(np.array([True, True,False, True, False])) would return 3.

In this exercise, you will rewrite this script to use a two-dimensional array.

1. In Jupyter Notebook, navigate to numpy/Exercises and open the RollingDoublesnotebook.

2. The first cell imports NumPy and seeds the random generator. This shouldmake it so that the results are exactly reproducible.

3. In a new cell, create a dice variable that holds a two-dimensional array with10,000 rows and 2 columns that holds random integers between 1 and 6.

4. Using the dice variable, create a doubles variable that holds aone-dimensional 10,000 element boolean array indicating whether each rowof dice contains two equal values.

5. Output the fraction of times doubles are rolled.


NumPy

EVALUATION COPY


Exercise Solution

numpy/Solutions/Rolling Doubles.ipynb


NumPy

EVALUATION COPY


Using Boolean Arrays to Get New Arrays2.5


All examples from this section are available in numpy/Demos/Using Boolean Arrays.ipynb.

A common trick with NumPy arrays is to use a boolean array to filter an array. Forexample, suppose you have an ndarray of integers from which you want to createa new ndarray of its odd integers. Here's how you can do that:


NumPy

EVALUATION COPY


1. Create an ndarray of random integers:

2. Create a boolean ndarrray showing which values of the integer array areodd:

3. Here's the magic: Use the boolean array to get just the odd integers from theinteger array:

Notice that the new array only includes the numbers at indexes that containedTrue values.

And here a shortcut for doing the same thing:

More with NumPy Arrays2.6

Though it is beyond the scope of this course, there is much more you can do withNumPy arrays, including:

1. Combine arrays with hstack(), vstack(), dstack(),concatenate(), column_stack(), and row_stack().

2. Split arrays with split(), hsplit(), vsplit(), and dsplit().3. Flatten arrays with ravel().

Conclusion2.7In this lesson, you have learned to work with one-dimensional and two-dimensionalNumPy arrays to work efficiently with large amounts of data.


NumPy

EVALUATION COPY


7400 E. Orchard Road, Suite 1450 NGreenwood Village, Colorado 80111

Ph: 303-302-5280www.ITCourseware.com

9-38-00261-000-05-14-19

EVALUATION COPY


Documents

TRAINING MATERIALS FOR IT PROFESSIONALS · Accompanying Class Files This manual comes with accompanying class files, which your instructor or sales representative will point out to