Python Programming and Visualization for Scientists

Python Programmingand Visualizationfor Scientists2nd Edition

Alex J. DeCaria

Grant W. Petty

Sundog Publishing LLCMadison, Wisconsin

© 2016 by Alex J. DeCaria, © 2020 by Alex J. DeCaria and Grant W. Petty

Manufactured in the United States of America.

All rights reserved.

No part of this book may be reproduced or transmitted in any form or by any means,electronic or mechanical, including photocopying, recording, or by an information storage andretrieval system — with the exception of a reviewer who may quote brief passages in a reviewto be printed in a newspaper or magazine — without written permission. For permission toreuse copyrighted content from this work, please contact the publisher via the contact page atwww.sundogpublishing.com.

Ordering information: Instructors and students should visit www.sundogpublishing.com totake advantage of discounts available to students who order directly from the publisher. Freeexamination copies may be requested by qualifying instructors.

Online resources: Electronic files, errata, and other materials relevant to this book can befound at www.sundogpublishing.com.

Sundog Publishing LLC, Madison, WisconsinPublisher’s Cataloging-in-Publication data Names: DeCaria, Alex J., author. | Petty, Grant W., author. Title: Python programming and visualization for scientists , 2nd edition / Alex J. DeCaria ; Grant W. Petty. Description: Madison, WI: Sundog Publishing, 2020. Identifiers: LCCN: 2020950960 | ISBN: 978-0-9729033-5-6 Subjects: LCSH Python (Computer program language) | Object-oriented programming (Computer science) | Python (Computer language). | BISAC: COMPUTERS / Programming Languages / Python | GSAFD: Instructional. Classification: LCC QA76.73.P98 .D43 2020 | DDC 005.133--dc23

10 9 8 7 6 5 4 3 2 1

Anaconda is a registered trademark of Continuum Analytics, Inc.Canopy is a registered trademark of Enthought, Inc.IDL is a registered trademark of Exelis Inc., a subsidiary of Harris Corporation.Macintosh and macOS are registered trademarks of Apple Inc.Mathematica is a registered trademark of Wolfram Research, Inc.MATLAB is a registered trademark of The MathWorks.Windows is a registered trademark of Microsoft CorporationOther trademarks and registered trademarks are the property of the respective trademarkholders.

Cover design by Linda J. Weidemann.

CONTENTS

Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

Preface to the Second Edition . . . . . . . . . . . . . . . . xxi

Preface to the First Edition . . . . . . . . . . . . . . . . . . xxiii

I Basic Python Programming . . . . . . . . . . 1

1 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1 Purpose of This Book . . . . . . . . . . . . . . . . . . . . . 31.2 Why Python? . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Python 3 versus Python 2 . . . . . . . . . . . . . . . . . . . 41.4 A Few More Things About Python . . . . . . . . . . . . . 51.5 Notation and Syntax Highlighting . . . . . . . . . . . . . . 51.6 Installing Python . . . . . . . . . . . . . . . . . . . . . . . . 6

1.6.1 Pre-built packages . . . . . . . . . . . . . . . . . . . . . . . 61.6.2 Installing individual libraries . . . . . . . . . . . . . . . . . . 61.6.3 Package managers . . . . . . . . . . . . . . . . . . . . . . . 7

1.7 Alternate Python Implementations . . . . . . . . . . . . . . 91.8 Set-up and Testing . . . . . . . . . . . . . . . . . . . . . . . 9

1.8.1 Path settings . . . . . . . . . . . . . . . . . . . . . . . . . 91.8.2 Testing the Python installation . . . . . . . . . . . . . . . . 101.8.3 Testing libraries . . . . . . . . . . . . . . . . . . . . . . . . 10

1.9 Executing Python Code . . . . . . . . . . . . . . . . . . . . 101.10 Additional Resources . . . . . . . . . . . . . . . . . . . . . . 12

v

vi CONTENTS

2 Syntax and Data Types . . . . . . . . . . . . . . . . . . . 152.1 General Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1.1 Python is case-sensitive . . . . . . . . . . . . . . . . . . . . 152.1.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.1.3 Writing output to terminal . . . . . . . . . . . . . . . . . . . 152.1.4 Code blocks and indentation . . . . . . . . . . . . . . . . . . 162.1.5 Continuation of lines . . . . . . . . . . . . . . . . . . . . . . 162.1.6 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.2 Variable Names and Assignment . . . . . . . . . . . . . . . 172.2.1 Valid names . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2.2 Reserved words . . . . . . . . . . . . . . . . . . . . . . . . 172.2.3 Variable assignment . . . . . . . . . . . . . . . . . . . . . . 17

2.3 Numeric Data Types . . . . . . . . . . . . . . . . . . . . . . 182.3.1 Boolean . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3.2 Integer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3.3 Floating-point . . . . . . . . . . . . . . . . . . . . . . . . . 192.3.4 Complex . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.3.5 Conversion between numeric types . . . . . . . . . . . . . . . 20

2.4 Objects, Attributes, and Methods . . . . . . . . . . . . . . 202.4.1 Data types as classes . . . . . . . . . . . . . . . . . . . . . 212.4.2 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.4 Inspecting objects . . . . . . . . . . . . . . . . . . . . . . . 22

2.5 The None Data Type . . . . . . . . . . . . . . . . . . . . . . 222.6 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.7 Lists and Tuples . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.7.1 Accessing elements of a collection . . . . . . . . . . . . . . . 232.7.2 Nested elements . . . . . . . . . . . . . . . . . . . . . . . . 232.7.3 Using ranges of indexes . . . . . . . . . . . . . . . . . . . . 242.7.4 Striding and reversing . . . . . . . . . . . . . . . . . . . . . 252.7.5 Reassigning values . . . . . . . . . . . . . . . . . . . . . . . 252.7.6 The range() function . . . . . . . . . . . . . . . . . . . . . 252.7.7 Functions and methods for lists . . . . . . . . . . . . . . . . 262.7.8 Zippering and unzippering lists . . . . . . . . . . . . . . . . 272.7.9 List comprehension . . . . . . . . . . . . . . . . . . . . . . 28

2.8 Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.8.1 Creating and accessing . . . . . . . . . . . . . . . . . . . . . 282.8.2 Adding elements . . . . . . . . . . . . . . . . . . . . . . . . 292.8.3 Functions and methods for dictionaries . . . . . . . . . . . . 29

2.9 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.10 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.11 Finding an Object’s Type . . . . . . . . . . . . . . . . . . . 302.12 Copying Objects . . . . . . . . . . . . . . . . . . . . . . . . . 312.13 Unpacking Lists and Tuples . . . . . . . . . . . . . . . . . . 32

CONTENTS vii

3 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.1 Defining Strings . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.1.1 Special characters . . . . . . . . . . . . . . . . . . . . . . . 333.1.2 Triple-quoted strings . . . . . . . . . . . . . . . . . . . . . . 343.1.3 Raw strings . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.2 Formatting Strings . . . . . . . . . . . . . . . . . . . . . . . 353.3 Indexing and Iterating . . . . . . . . . . . . . . . . . . . . . 363.4 Concatenation . . . . . . . . . . . . . . . . . . . . . . . . . . 373.5 Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . 383.6 More String Functions and Methods . . . . . . . . . . . . . 38

3.6.1 Retrieving information about a string . . . . . . . . . . . . . 383.6.2 Searching a string . . . . . . . . . . . . . . . . . . . . . . . 393.6.3 Editing, stripping, and justifying strings . . . . . . . . . . . . 393.6.4 Converting cases . . . . . . . . . . . . . . . . . . . . . . . . 403.6.5 Splitting strings . . . . . . . . . . . . . . . . . . . . . . . . 40

3.7 Byte Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

4 Mathematical Operators and Functions . . . . . . 434.1 Numerical Operators . . . . . . . . . . . . . . . . . . . . . . 43

4.1.1 Division . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.1.2 Truncating division . . . . . . . . . . . . . . . . . . . . . . 434.1.3 Modulo operator . . . . . . . . . . . . . . . . . . . . . . . . 44

4.2 Augmented Assignment . . . . . . . . . . . . . . . . . . . . 444.3 Boolean Operators . . . . . . . . . . . . . . . . . . . . . . . 444.4 Comparison Operators . . . . . . . . . . . . . . . . . . . . . 45

4.4.1 Simple comparison . . . . . . . . . . . . . . . . . . . . . . . 454.4.2 Chained comparison . . . . . . . . . . . . . . . . . . . . . . 46

4.5 Built-in Math Functions . . . . . . . . . . . . . . . . . . . . 464.6 The math Module . . . . . . . . . . . . . . . . . . . . . . . . 484.7 The numpy Module . . . . . . . . . . . . . . . . . . . . . . . 48

4.7.1 Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.7.2 Powers and roots . . . . . . . . . . . . . . . . . . . . . . . 484.7.3 Trigonometry functions . . . . . . . . . . . . . . . . . . . . 504.7.4 Hyperbolic functions . . . . . . . . . . . . . . . . . . . . . . 504.7.5 Inverse trig and hyperbolic functions . . . . . . . . . . . . . . 504.7.6 Exponentials and logarithms . . . . . . . . . . . . . . . . . . 514.7.7 Numeric functions . . . . . . . . . . . . . . . . . . . . . . . 514.7.8 Modulo arithmetic . . . . . . . . . . . . . . . . . . . . . . . 514.7.9 Special mathematical functions . . . . . . . . . . . . . . . . 524.7.10 Functions for infinity or NaN . . . . . . . . . . . . . . . . . . 52

4.8 Aliasing Modules and Functions . . . . . . . . . . . . . . . 53

5 Structure and Control . . . . . . . . . . . . . . . . . . . . 55

viii CONTENTS

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 555.2 Interactive Input from Terminal . . . . . . . . . . . . . . . . 555.3 Conditional Statements . . . . . . . . . . . . . . . . . . . . 56

5.3.1 The if-elif-else construct . . . . . . . . . . . . . . . . . . . . 565.3.2 The single-line conditional statement . . . . . . . . . . . . . 575.3.3 Using single objects or values as conditions . . . . . . . . . . 58

5.4 Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.5 Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5.5.1 For loops . . . . . . . . . . . . . . . . . . . . . . . . . . . 595.5.2 While loops . . . . . . . . . . . . . . . . . . . . . . . . . . 605.5.3 Skipping to top of a loop . . . . . . . . . . . . . . . . . . . 615.5.4 Breaking out of a loop . . . . . . . . . . . . . . . . . . . . . 61

5.6 Verifying User Input . . . . . . . . . . . . . . . . . . . . . . . 615.7 The pass Statement . . . . . . . . . . . . . . . . . . . . . . 625.8 List Comprehension . . . . . . . . . . . . . . . . . . . . . . . 625.9 The print() Function Revisited . . . . . . . . . . . . . . . . 63

5.9.1 Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635.9.2 Flushing the buffer . . . . . . . . . . . . . . . . . . . . . . . 64

5.10 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . 64

6 File I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 676.1 Opening and Closing Files . . . . . . . . . . . . . . . . . . . 67

6.1.1 Opening files . . . . . . . . . . . . . . . . . . . . . . . . . . 676.1.2 Closing files . . . . . . . . . . . . . . . . . . . . . . . . . . 686.1.3 Automatically closing files . . . . . . . . . . . . . . . . . . . 68

6.2 Interactive File and Directory Selection . . . . . . . . . . . 686.2.1 File selection . . . . . . . . . . . . . . . . . . . . . . . . . . 686.2.2 Directory selection . . . . . . . . . . . . . . . . . . . . . . . 70

6.3 Reading Text from Files . . . . . . . . . . . . . . . . . . . . 706.3.1 Reading individual lines . . . . . . . . . . . . . . . . . . . . 706.3.2 Looping over file . . . . . . . . . . . . . . . . . . . . . . . . 716.3.3 Reading an entire file into a list . . . . . . . . . . . . . . . . 71

6.4 Writing Text to Files . . . . . . . . . . . . . . . . . . . . . . 716.4.1 Writing individual lines of text . . . . . . . . . . . . . . . . . 716.4.2 Writing a list of strings . . . . . . . . . . . . . . . . . . . . 726.4.3 Buffered output . . . . . . . . . . . . . . . . . . . . . . . . 73

6.5 Reading and Writing Delimited Files Usingthe csv Module . . . . . . . . . . . . . . . . . . . . . . . . . 73

6.5.1 Reading delimited files . . . . . . . . . . . . . . . . . . . . . 736.5.2 Writing delimited files . . . . . . . . . . . . . . . . . . . . . 746.5.3 Additional notes regarding the csv module . . . . . . . . . . . 75

6.6 Reading/Writing Numpy Arrays from/to Files . . . . . . . 75

CONTENTS ix

6.7 The os.path Module: Working with Pathand File Names . . . . . . . . . . . . . . . . . . . . . . . . . 76

6.8 Internet File Access . . . . . . . . . . . . . . . . . . . . . . . 76

7 Numpy Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . 797.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 797.2 Creating an Array . . . . . . . . . . . . . . . . . . . . . . . . 797.3 Numpy Data Types . . . . . . . . . . . . . . . . . . . . . . . 807.4 Printing numpy Arrays . . . . . . . . . . . . . . . . . . . . . 817.5 Other Array-creation Functions . . . . . . . . . . . . . . . . 82

7.5.1 Arrays filled with a single value . . . . . . . . . . . . . . . . 827.5.2 Arrays patterned from an existing array . . . . . . . . . . . . 827.5.3 Uninitialized arrays . . . . . . . . . . . . . . . . . . . . . . . 83

7.6 Sequential Arrays . . . . . . . . . . . . . . . . . . . . . . . . 837.6.1 The arange() function . . . . . . . . . . . . . . . . . . . . . 837.6.2 The linspace() function . . . . . . . . . . . . . . . . . . . . 847.6.3 The logspace() function . . . . . . . . . . . . . . . . . . . . 84

7.7 Array Indexing and Slicing . . . . . . . . . . . . . . . . . . . 857.7.1 One-dimensional arrays . . . . . . . . . . . . . . . . . . . . 857.7.2 Multidimensional arrays . . . . . . . . . . . . . . . . . . . . 867.7.3 Indexing with lists . . . . . . . . . . . . . . . . . . . . . . . 86

7.8 Broadcasting and Implicit Loops . . . . . . . . . . . . . . . 877.8.1 Broadcasting of arrays . . . . . . . . . . . . . . . . . . . . . 877.8.2 Implicit loops . . . . . . . . . . . . . . . . . . . . . . . . . 88

7.9 Array Statistics . . . . . . . . . . . . . . . . . . . . . . . . . 897.10 Finding Shape and Size . . . . . . . . . . . . . . . . . . . . 907.11 Reshaping, Transposing, and Shifting . . . . . . . . . . . . 907.12 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

7.12.1 Sorting 2-D arrays . . . . . . . . . . . . . . . . . . . . . . . 917.12.2 Reverse sorting . . . . . . . . . . . . . . . . . . . . . . . . 92

7.13 Singular-dimension Arrays . . . . . . . . . . . . . . . . . . . 927.14 Adding or Removing Elements . . . . . . . . . . . . . . . . 93

7.14.1 Appending elements . . . . . . . . . . . . . . . . . . . . . . 937.14.2 Inserting elements . . . . . . . . . . . . . . . . . . . . . . . 947.14.3 Deleting elements . . . . . . . . . . . . . . . . . . . . . . . 94

7.15 Assigning and Reassigning Elements . . . . . . . . . . . . . 947.16 Stacking and Splitting Arrays . . . . . . . . . . . . . . . . . 94

7.16.1 Stacking . . . . . . . . . . . . . . . . . . . . . . . . . . . . 947.16.2 Splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

7.17 Merging Two 1-D Arrays into 2-D Arrays . . . . . . . . . . 97

x CONTENTS

7.18 Logical Operations with Arrays . . . . . . . . . . . . . . . . 977.18.1 The where() function . . . . . . . . . . . . . . . . . . . . . 977.18.2 More complex conditions . . . . . . . . . . . . . . . . . . . 98

7.19 Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . 997.19.1 Random integers . . . . . . . . . . . . . . . . . . . . . . . . 997.19.2 Uniformly distributed random floating-point values . . . . . . 997.19.3 Normally distributed random floating-point values . . . . . . . 100

7.20 Masked Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . 1007.21 Reading/Writing Arrays from/to Files . . . . . . . . . . . . 101

7.21.1 Writing arrays to a file . . . . . . . . . . . . . . . . . . . . . 1017.21.2 Reading data from a text file . . . . . . . . . . . . . . . . . 102

7.22 Saving/Loading Arrays as .npy and .npz Files . . . . . . . 1037.22.1 Numpy .npy files . . . . . . . . . . . . . . . . . . . . . . . . 1037.22.2 Numpy .npz files . . . . . . . . . . . . . . . . . . . . . . . . 104

7.23 Memory Mapping Arrays . . . . . . . . . . . . . . . . . . . . 104

8 Functions and Modules . . . . . . . . . . . . . . . . . . . 1078.1 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

8.1.1 Defining functions . . . . . . . . . . . . . . . . . . . . . . . 1078.1.2 Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . 1088.1.3 The return statement . . . . . . . . . . . . . . . . . . . . . 1088.1.4 Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . 1098.1.5 Scoping rules . . . . . . . . . . . . . . . . . . . . . . . . . 1098.1.6 Passing arguments by value versus by reference . . . . . . . . 1118.1.7 Bound functions . . . . . . . . . . . . . . . . . . . . . . . . 1128.1.8 The lambda operator . . . . . . . . . . . . . . . . . . . . . 1138.1.9 Querying and inspecting functions . . . . . . . . . . . . . . . 1138.1.10 Unpacking arguments and keywords . . . . . . . . . . . . . . 1138.1.11 Variable number of arguments and keywords . . . . . . . . . . 114

8.2 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1158.2.1 Creating modules . . . . . . . . . . . . . . . . . . . . . . . 1158.2.2 Importing and using modules . . . . . . . . . . . . . . . . . 1158.2.3 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1168.2.4 Specifying the path to module location . . . . . . . . . . . . 116

8.3 Documentation Strings . . . . . . . . . . . . . . . . . . . . . 1178.4 Decorators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

8.4.1 Interlude . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1178.4.2 Back to decorators . . . . . . . . . . . . . . . . . . . . . . . 119

CONTENTS xi

9 Defining Classes and Methods . . . . . . . . . . . . . 1219.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1219.2 Defining Classes . . . . . . . . . . . . . . . . . . . . . . . . . 121

9.2.1 The __init__() method . . . . . . . . . . . . . . . . . . . 1229.2.2 Instance methods . . . . . . . . . . . . . . . . . . . . . . . 122

9.3 Instantiating Objects . . . . . . . . . . . . . . . . . . . . . . 1229.4 Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1249.5 Types of Methods . . . . . . . . . . . . . . . . . . . . . . . . 125

9.5.1 Instance methods . . . . . . . . . . . . . . . . . . . . . . . 1259.5.2 Static methods . . . . . . . . . . . . . . . . . . . . . . . . 1269.5.3 Class methods . . . . . . . . . . . . . . . . . . . . . . . . . 127

9.6 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

II Plotting and Visualization . . . . . . . . . . . 131

10 1-D Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13310.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 13310.2 A Simple 1-D Plot . . . . . . . . . . . . . . . . . . . . . . . 13310.3 Figure and Axes Objects . . . . . . . . . . . . . . . . . . . . 135

10.3.1 Figure objects . . . . . . . . . . . . . . . . . . . . . . . . . 13510.3.2 Axes objects . . . . . . . . . . . . . . . . . . . . . . . . . . 13510.3.3 Plotting data onto axes . . . . . . . . . . . . . . . . . . . . 136

10.4 Discussion of pyplot.plot() and Axes.plot() . . . . . . . . . 13710.4.1 Which one to use? . . . . . . . . . . . . . . . . . . . . . . . 13710.4.2 Methods vs. functions—revisited . . . . . . . . . . . . . . . 13710.4.3 Getting references to the current figure and axes . . . . . . . 13810.4.4 Differences in naming conventions for functions

and methods . . . . . . . . . . . . . . . . . . . . . . . . . . 13810.5 Clearing Figures and Axes . . . . . . . . . . . . . . . . . . . 13810.6 Elements of a Plot . . . . . . . . . . . . . . . . . . . . . . . 13910.7 Controlling Line and Marker Properties . . . . . . . . . . . 139

10.7.1 Line styles . . . . . . . . . . . . . . . . . . . . . . . . . . . 14010.7.2 Line color . . . . . . . . . . . . . . . . . . . . . . . . . . . 14010.7.3 Line thickness . . . . . . . . . . . . . . . . . . . . . . . . . 14110.7.4 Markers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14110.7.5 Shortcut for line and marker properties . . . . . . . . . . . . 14210.7.6 Background color . . . . . . . . . . . . . . . . . . . . . . . 142

10.8 Logarithmic Plots . . . . . . . . . . . . . . . . . . . . . . . . 142

xii CONTENTS

10.9 Placing Text on Plots . . . . . . . . . . . . . . . . . . . . . . 14310.9.1 Plot titles . . . . . . . . . . . . . . . . . . . . . . . . . . . 14310.9.2 Axis labels . . . . . . . . . . . . . . . . . . . . . . . . . . . 14310.9.3 Data coordinates versus axes coordinates . . . . . . . . . . . 14310.9.4 Placing text in the body of a plot . . . . . . . . . . . . . . . 14410.9.5 Fonts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14410.9.6 Text properties . . . . . . . . . . . . . . . . . . . . . . . . . 14510.9.7 Mathematical symbols and Greek characters . . . . . . . . . . 146

10.10 Controlling Axes Properties . . . . . . . . . . . . . . . . . . 14710.10.1 Axis limits . . . . . . . . . . . . . . . . . . . . . . . . . . . 14710.10.2 Tick mark location and labels . . . . . . . . . . . . . . . . . 14810.10.3 Retrieving tick mark locations and labels . . . . . . . . . . . 149

10.11 Drawing Lines on Plots . . . . . . . . . . . . . . . . . . . . . 15010.11.1 Grid lines . . . . . . . . . . . . . . . . . . . . . . . . . . . 15010.11.2 Horizontal and vertical lines . . . . . . . . . . . . . . . . . . 15010.11.3 Arbitrary lines and connected line segments . . . . . . . . . . 151

10.12 Duplicate Axes . . . . . . . . . . . . . . . . . . . . . . . . . . 15210.12.1 Duplicate identical axes . . . . . . . . . . . . . . . . . . . . 15210.12.2 Duplicate nonidentical axes . . . . . . . . . . . . . . . . . . 153

10.13 Working with Multiple Axes and Figures . . . . . . . . . . 15310.14 Legends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

10.14.1 Simple legend . . . . . . . . . . . . . . . . . . . . . . . . . 15510.14.2 Keywords for controlling legend appearance . . . . . . . . . . 155

10.15 Saving Plots as Image Files . . . . . . . . . . . . . . . . . . 15710.16 Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15710.17 Other 1-D Plot Types . . . . . . . . . . . . . . . . . . . . . 158

10.17.1 Scatter plots . . . . . . . . . . . . . . . . . . . . . . . . . . 15810.17.2 Polar coordinate plots . . . . . . . . . . . . . . . . . . . . . 15910.17.3 Bar charts . . . . . . . . . . . . . . . . . . . . . . . . . . . 16110.17.4 Pie charts . . . . . . . . . . . . . . . . . . . . . . . . . . . 16310.17.5 Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . 164

10.18 Plotting Outside of Boundaries . . . . . . . . . . . . . . . . 16510.19 What is Pylab? . . . . . . . . . . . . . . . . . . . . . . . . . 166

11 Multi-Panel Plots . . . . . . . . . . . . . . . . . . . . . . . . 16711.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 16711.2 Multiple Calls to the figure.add_axes() Method . . . . . . 16711.3 The pyplot.subplot() Function . . . . . . . . . . . . . . . . 16711.4 The pyplot.subplots() Function . . . . . . . . . . . . . . . . 17011.5 Adjusting Subplot Position and Spacing . . . . . . . . . . . 17011.6 Sharing Axes among Subplots . . . . . . . . . . . . . . . . . 17111.7 Titles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17211.8 Looping Over an Array of Axes . . . . . . . . . . . . . . . . 172

CONTENTS xiii

12 2-D Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17312.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

12.1.1 Discussion of column vs. row ordering . . . . . . . . . . . . . 17312.1.2 Data sets used in the examples . . . . . . . . . . . . . . . . 173

12.2 Contour Plots . . . . . . . . . . . . . . . . . . . . . . . . . . 17312.2.1 Basic contour plot . . . . . . . . . . . . . . . . . . . . . . . 17412.2.2 Controlling the aspect ratio . . . . . . . . . . . . . . . . . . 17412.2.3 Controlling contour values and spacing . . . . . . . . . . . . 17612.2.4 Controlling contour properties . . . . . . . . . . . . . . . . . 17612.2.5 Labeling contours . . . . . . . . . . . . . . . . . . . . . . . 17712.2.6 Other keywords for clabels() . . . . . . . . . . . . . . . . . . 17812.2.7 Filled contour plots . . . . . . . . . . . . . . . . . . . . . . 178

12.3 Pseudocolor and Image Plots . . . . . . . . . . . . . . . . . 17912.3.1 pcolormesh() . . . . . . . . . . . . . . . . . . . . . . . . . . 17912.3.2 imshow() . . . . . . . . . . . . . . . . . . . . . . . . . . . 18012.3.3 Using imshow() with RGB and RGBA values . . . . . . . . . 182

12.4 Color Bars . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18312.4.1 Adding a color bar . . . . . . . . . . . . . . . . . . . . . . . 18312.4.2 Color bars and multi-panel plots . . . . . . . . . . . . . . . . 184

12.5 Colormaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18612.6 Vectors, Wind Barbs, and Streamlines . . . . . . . . . . . . 187

12.6.1 Vector plots . . . . . . . . . . . . . . . . . . . . . . . . . . 18812.6.2 Wind barbs . . . . . . . . . . . . . . . . . . . . . . . . . . 18912.6.3 Streamlines . . . . . . . . . . . . . . . . . . . . . . . . . . 190

13 Cartopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19113.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19113.2 Creating a Map . . . . . . . . . . . . . . . . . . . . . . . . . 191

13.2.1 An example . . . . . . . . . . . . . . . . . . . . . . . . . . 19213.2.2 Adding features to the map . . . . . . . . . . . . . . . . . . 195

13.3 Plotting Data . . . . . . . . . . . . . . . . . . . . . . . . . . 19513.4 Cyclic Boundaries . . . . . . . . . . . . . . . . . . . . . . . . 197

13.4.1 The add_cyclic_point() function . . . . . . . . . . . . . . . 19913.4.2 Fixing the data gap . . . . . . . . . . . . . . . . . . . . . . 199

14 3-D Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20114.1 Creating a Simple 3-D Plot . . . . . . . . . . . . . . . . . . 201

14.1.1 Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . 20214.1.2 Axis labels, limits, and tick marks . . . . . . . . . . . . . . . 20214.1.3 Saving figure . . . . . . . . . . . . . . . . . . . . . . . . . . 203

14.2 3-D Plots of 2-D Array Data . . . . . . . . . . . . . . . . . 20314.2.1 Surface plots . . . . . . . . . . . . . . . . . . . . . . . . . . 20314.2.2 Adding contours to 3-D plots . . . . . . . . . . . . . . . . . 204

14.3 Lines and Text . . . . . . . . . . . . . . . . . . . . . . . . . . 20514.3.1 Drawing lines . . . . . . . . . . . . . . . . . . . . . . . . . 20514.3.2 Adding text . . . . . . . . . . . . . . . . . . . . . . . . . . 206

xiv CONTENTS

III Additional Topics . . . . . . . . . . . . . . . . . . . 207

15 Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20915.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 20915.2 The datetime Module . . . . . . . . . . . . . . . . . . . . . 210

15.2.1 Basic operations . . . . . . . . . . . . . . . . . . . . . . . . 21015.2.2 datetime object methods and attributes . . . . . . . . . . . . 21315.2.3 Formatting datetime output . . . . . . . . . . . . . . . . . . 21415.2.4 Parsing strings representing times . . . . . . . . . . . . . . . 21615.2.5 timedelta objects . . . . . . . . . . . . . . . . . . . . . . . 21715.2.6 Arithmetic with times and dates . . . . . . . . . . . . . . . . 218

15.3 Putting a Program to Sleep . . . . . . . . . . . . . . . . . . 218

16 Pandas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21916.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 21916.2 Pandas Series . . . . . . . . . . . . . . . . . . . . . . . . . . 220

16.2.1 The index . . . . . . . . . . . . . . . . . . . . . . . . . . . 22016.2.2 Accessing data . . . . . . . . . . . . . . . . . . . . . . . . . 22116.2.3 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22316.2.4 Mathematical operations with series . . . . . . . . . . . . . . 22316.2.5 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22416.2.6 Plotting data . . . . . . . . . . . . . . . . . . . . . . . . . 22416.2.7 Rolling windows . . . . . . . . . . . . . . . . . . . . . . . . 22616.2.8 Shifting of data . . . . . . . . . . . . . . . . . . . . . . . . 227

16.3 Pandas DataFrames . . . . . . . . . . . . . . . . . . . . . . . 22816.3.1 Accessing data . . . . . . . . . . . . . . . . . . . . . . . . . 22916.3.2 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23116.3.3 Math and statistics . . . . . . . . . . . . . . . . . . . . . . 23116.3.4 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23216.3.5 Adding and removing rows and columns . . . . . . . . . . . . 233

16.4 Reading and Writing Files . . . . . . . . . . . . . . . . . . . 23416.4.1 Comma-separated value (CSV) files . . . . . . . . . . . . . . 23416.4.2 Excel files . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

16.5 Temporal Resampling . . . . . . . . . . . . . . . . . . . . . . 23816.6 A Few Additional Useful Methods . . . . . . . . . . . . . . 240

17 Scientific Datasets . . . . . . . . . . . . . . . . . . . . . . . 24117.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 24117.2 Working with NetCDF . . . . . . . . . . . . . . . . . . . . . 241

17.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24117.2.2 Opening and reading NetCDF files . . . . . . . . . . . . . . . 24217.2.3 Writing NetCDF files . . . . . . . . . . . . . . . . . . . . . 247

17.3 Working with HDF . . . . . . . . . . . . . . . . . . . . . . . 250

CONTENTS xv

17.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25017.3.2 Opening and reading HDF5 files . . . . . . . . . . . . . . . . 251

18 Regular Expressions . . . . . . . . . . . . . . . . . . . . . . 25718.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 25718.2 The re.findall() Function . . . . . . . . . . . . . . . . . . . . 25718.3 Regular Expression Syntax . . . . . . . . . . . . . . . . . . . 258

18.3.1 Matching the beginning or ending of a string . . . . . . . . . 25818.3.2 Matching the beginning or ending of a word . . . . . . . . . . 26018.3.3 Matching classes of characters . . . . . . . . . . . . . . . . . 26018.3.4 Matching consecutive occurrences . . . . . . . . . . . . . . . 26118.3.5 Matching sets of characters . . . . . . . . . . . . . . . . . . 26318.3.6 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26318.3.7 Matching a choice of two patterns . . . . . . . . . . . . . . . 26418.3.8 Matching text immediately following or not

immediately following a specific expression . . . . . . . . . . . 26418.3.9 Matching text immediately preceding or not

immediately preceding a specific expression . . . . . . . . . . 26518.4 More Functions in the re Module . . . . . . . . . . . . . . . 265

18.4.1 The re.sub() and re.subn() functions . . . . . . . . . . . . . . 26518.4.2 The re.split() function . . . . . . . . . . . . . . . . . . . . . 266

18.5 Regular Expression Pattern Objects . . . . . . . . . . . . . 26618.5.1 The re.compile() function . . . . . . . . . . . . . . . . . . . 26618.5.2 The r.findall() method . . . . . . . . . . . . . . . . . . . . . 26718.5.3 The r.sub() and r.subn() methods . . . . . . . . . . . . . . . 26718.5.4 The r.split() method . . . . . . . . . . . . . . . . . . . . . . 267

18.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

19 Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . 26919.1 Matrix Products . . . . . . . . . . . . . . . . . . . . . . . . . 26919.2 Systems of Linear Equations . . . . . . . . . . . . . . . . . 271

19.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . 27119.2.2 Solving a system of equations using np.linalg.solve() . . . . . . 27219.2.3 LU decomposition . . . . . . . . . . . . . . . . . . . . . . . 27319.2.4 Banded matrices . . . . . . . . . . . . . . . . . . . . . . . . 274

20 Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 27720.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 27720.2 The Discrete Fourier Transform . . . . . . . . . . . . . . . 278

20.2.1 Common form of the discrete Fourier transform . . . . . . . . 27820.2.2 Alternate form of the discrete Fourier transform . . . . . . . . 27820.2.3 A DFT example . . . . . . . . . . . . . . . . . . . . . . . . 27920.2.4 Real versus complex DFTs . . . . . . . . . . . . . . . . . . . 28020.2.5 The power spectrum . . . . . . . . . . . . . . . . . . . . . . 281

xvi CONTENTS

20.2.6 Leakage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28120.2.7 Noisy data . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

20.3 The numpy.fft Module . . . . . . . . . . . . . . . . . . . . . 28320.3.1 Forward and inverse DFT functions . . . . . . . . . . . . . . 28320.3.2 Frequencies/wavenumbers for Fourier coefficients . . . . . . . 28420.3.3 The fftshift() function . . . . . . . . . . . . . . . . . . . . . 28520.3.4 Real DFTs . . . . . . . . . . . . . . . . . . . . . . . . . . . 28520.3.5 Multi-dimensional DFTs . . . . . . . . . . . . . . . . . . . . 286

20.4 Example Using Numpy’s Complex DFT functions . . . . . 28620.5 DFT Example with Noisy Data . . . . . . . . . . . . . . . . 287

21 Miscellany . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29121.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 29121.2 Interpolation and Resampling . . . . . . . . . . . . . . . . . 291

21.2.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 29121.2.2 Resampling . . . . . . . . . . . . . . . . . . . . . . . . . . 292

21.3 Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . 29321.4 Numerical Differentiation and Integration . . . . . . . . . . 294

21.4.1 Discretely sampled functions . . . . . . . . . . . . . . . . . . 29421.4.2 Differentiation of a continuous function . . . . . . . . . . . . 29621.4.3 Integration of a continuous function . . . . . . . . . . . . . . 298

21.5 Saving and Loading Arbitrary Python Objects . . . . . . . 29921.6 Physical Constants . . . . . . . . . . . . . . . . . . . . . . . 300

21.6.1 Constants directly defined in scipy.constants . . . . . . . . . . 30021.6.2 Constants in the CODATA database . . . . . . . . . . . . . . 300

21.7 Physical Dimensions and Units . . . . . . . . . . . . . . . . 30121.8 Special Mathematical Functions . . . . . . . . . . . . . . . 30521.9 Speed and Optimization of Code . . . . . . . . . . . . . . . 305

21.9.1 Numpy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30621.9.2 Numba . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30621.9.3 Cython . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30721.9.4 F2PY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30821.9.5 Parallel processing . . . . . . . . . . . . . . . . . . . . . . . 30821.9.6 Timing and benchmarking code . . . . . . . . . . . . . . . . 31021.9.7 General comments on optimization . . . . . . . . . . . . . . 313

A Jupyter Notebooks - A Primer . . . . . . . . . . . . . 315A.1 About Jupyter Notebooks . . . . . . . . . . . . . . . . . . . 315A.2 Jupyter Lab or Juptyer Notebook? . . . . . . . . . . . . . . 315A.3 Opening and Running a Notebook . . . . . . . . . . . . . . 316

A.3.1 The Jupyter Lab Launcher tab . . . . . . . . . . . . . . . . . 316A.3.2 Moving, renaming, and deleting files . . . . . . . . . . . . . . 317

CONTENTS xvii

A.4 Changing The Default Directory . . . . . . . . . . . . . . . 318A.5 Working in a Notebook File . . . . . . . . . . . . . . . . . . 318A.6 Cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

A.6.1 Code cells . . . . . . . . . . . . . . . . . . . . . . . . . . . 319A.6.2 Markdown cells . . . . . . . . . . . . . . . . . . . . . . . . 320

A.7 Working with Matplotlib . . . . . . . . . . . . . . . . . . . . 321A.8 Widgets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322

A.8.1 Sliders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323A.8.2 Shorthand using a decorator . . . . . . . . . . . . . . . . . . 324A.8.3 Check boxes . . . . . . . . . . . . . . . . . . . . . . . . . . 324A.8.4 Text boxes . . . . . . . . . . . . . . . . . . . . . . . . . . . 325A.8.5 Dropdown menus . . . . . . . . . . . . . . . . . . . . . . . 325A.8.6 Interactive plots . . . . . . . . . . . . . . . . . . . . . . . . 325A.8.7 If widgets do not work in Jupyter Lab . . . . . . . . . . . . . 326

B LATEX Quick Reference . . . . . . . . . . . . . . . . . . . . 327B.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327B.2 Greek Letters . . . . . . . . . . . . . . . . . . . . . . . . . . . 327B.3 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328B.4 Fractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328B.5 Superscripts and Subscripts . . . . . . . . . . . . . . . . . . 328B.6 Escaping Special Characters . . . . . . . . . . . . . . . . . . 328B.7 Integrals and Summations . . . . . . . . . . . . . . . . . . . 328B.8 The Degree Symbol . . . . . . . . . . . . . . . . . . . . . . . 329B.9 Other Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . 329B.10 Mathematical Functions . . . . . . . . . . . . . . . . . . . . 329B.11 Scalable Symbols . . . . . . . . . . . . . . . . . . . . . . . . 330B.12 Forcing Whitespace . . . . . . . . . . . . . . . . . . . . . . . 330B.13 Text Appearance . . . . . . . . . . . . . . . . . . . . . . . . 330B.14 Final Note . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

LIST OF TABLES

1.1 Syntax highlighting conventions used in this book. . . . . . . . 5

3.1 Python string format specifiers . . . . . . . . . . . . . . . . . . 37

10.1 Some pyplot functions and corresponding Axesmethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

10.2 Values for the legend() keyword loc . . . . . . . . . . . . . . 156

13.1 Some projections available in Cartopy . . . . . . . . . . . . . . 194

15.1 Format specifiers for strftime() and strptime() . . . . . . . 215

18.1 Regular expression syntax . . . . . . . . . . . . . . . . . . . . . 259

19.1 Common matrix functions provided by NumPy. . . . . . . . . 271

21.1 Constants defined in scipy.constants . . . . . . . . . . . . . 301

21.2 Selected constants from the CODATA database . . . . . . . . . 302

21.3 SI Base Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

A.1 Jupyter notebook keyboard shortcuts . . . . . . . . . . . . . . . 319

xix

PREFACE TO THE SECOND EDITION

Writing a second edition was not anticipated so soon after the first was published.However, in the past few years the Python ecosystem has evolved considerably, withthe not-fully-backwards-compatible Python 3 becoming more robust and extensivelyused in the scientific community, and Python 2 no longer being developed. Newmeans of creating, communicating, and executing code have been widely adopted,most notably Jupyter notebooks. Writing a second edition also presented the oppor-tunity to add a co-author, allowing us to augment each other’s strengths and expandthe topics covered.

Major differences between this edition and the first include the appearance oftwo completely new chapters. One is on Cartopy, the replacement for the Basemaplibrary for plotting georeferenced data onto map backgrounds. The original chapteron Basemap has been omitted, since it is no longer being developed or supported.The second new chapter is a brief introduction to the Pandas library for analysis ofdata sets.

In addition, a section on linear algebra operations from the first edition has beengiven its own chapter, and the chapter on Time has been extensively revised. Someindividual sections have been expanded or reorganized, and a new section has beenadded on using physical dimensions and units in Python programs. Another differ-ence between the two editions is more subtle—we have included more functions andmethods throughout the various chapters and better explanations of some items. Thisnew edition also adds devotes appendices to explain the use of Jupyter notebooksand to introduce the LaTeX markup language which is used for rendering mathemat-ical formulas and symbols in plots and Jupyter notebooks. Finally, we have greatlyexpanded our use of color throughout the text. In particular, we now use syntaxhighlighting in all code and program output samples so that less effort is needed tovisually identify the functional elements.

In short, this new edition is not simply a ruse to get you purchase another book.The first edition remains a solid and useful transition to Python from other languages,and with few exceptions, works in both Python 2 and Python 3. However, the secondedition contains significantly more information than the original, and, even for sometopics that are not new, we have taken the opportunity to improve the exposition. Itis our hope that this edition will remain relevant and useful at least until the nextmajor revision of the Python language.

Alex DeCariaLancaster, Pennsylvania2020

Grant PettyMadison, Wisconsin2020

xxi

Part I

Basic Python Programming

CHAPTER

ONE

GETTING STARTED

1.1 Purpose of This BookThis book is written for those who want to use Python to perform scientificdata analysis and create visually pleasing, publication-quality plots of theirdata. It assumes familiarity with computer programming, though not neces-sarily with Python. The introductory chapters provide a quick start of Pythonsyntax, data types, and control structures, while the later chapters focus ondata analysis and plotting, primarily with the Matplotlib, Numerical Python(Numpy), and Scientific Python (Scipy) libraries.

1.2 Why Python?There are many computer languages that can be used for scientific data analy-sis and visualization. These include proprietary languages such as MATLAB,Mathematica, and IDL, for which a license must be purchased, as well asopen-source languages such as Perl, Fortran, and C++, which are free for use.With enough ingenuity and patience, virtually any language can be used toperform a specific data analysis and visualization task. However, Python hascertain advantages over other languages (in the authors’ opinion). These are:

- It is open source and does not require the purchase of a license for use.Anyone can download Python and the libraries that are used in thisbook. You can develop your own applications and code, without fearthat if you move to a different job or school you will no longer be ableto use your programs.

- Python syntax is intuitive and easy to learn. There are no cumbersomebraces or semicolons cluttering up the code. Python also forces the userto indent code blocks, making for clean-looking, easy-to-read programs.

3

4 1. Getting Started

- Python is an interpreted language, rather than a compiled language. Ina compiled language the entire source code text file is read and thenconverted into executable code (in machine language) that can be di-rectly executed by the operating system. This executable code is oftenoptimized for the operating system. In an interpreted language thesource code text file is read line-by-line and each statement convertedinto machine language and executed before the next line is read, or thetext file is read completely and converted into an intermediate code thatis then further converted into machine language for execution by theoperating system.

Compiled languages are generally more efficient and faster than inter-preted languages, since the conversion to machine language only occursonce creating a machine language code that can be executed wheneverthe program needs to be run. However, interpreted languages such asPython are often more flexible and changes can be easily made to theprogram at runtime, without going through the extra step of compilationafter every minor change to the code.

- Python is widely used and supported, and its use is growing rapidly. Itis neither a fad nor a flash in the pan.

1.3 Python 3 versus Python 2Python,1 developed by Guido van Rossum, began in 1991 with version 0.9and has transformed via dozens of subversions since. Python 2.0 appeared in2001, with the latest version, 2.7, appearing in 2010. Python 3 first appeared in2008, and is not backward-compatible with older versions of Python. Notabledifferences are: 1) the use of the print() function rather than the print state-ment; 2) how formatted strings are created; 3) an important new distinctionbetween str and bytes objects; 4) handling of integer division; and 5) objectsreturned from the range() function.

Because so many scientific libraries have been developed using the Python2 paradigm, transition and acceptance of Python 3 in the scientific commu-nity has been slower than it might have been; however, Python 2 ceased tobe maintained after January 1, 2020 making transition to Python 3 impera-tive. All code examples in this book are Python 3 compatible. Significantdifferences in syntax between Python 3 and Python 2 are pointed out as theyare encountered, so that those working with legacy code written in Python 2should still find this book useful. Sections where differences between Python 3and Python 2 are discussed may be located by referring to the index entry"Python 3 vs Python 2". Important differences are also boxed where theyappear in the text.

1Python is named after the British comedy troupe Monty Python.

A Few More Things About Python 5

Table 1.1: Syntax highlighting conventions used in this book.

Function Examples

Built-in keywords for, import, Class, passStrings 'This is a string'General identifiers x, radius, numpy, plt.plotComments # This is a comment

Shell command pip install numpyExternal text and program output This is some text output

1.4 A Few More Things About PythonPython is dynamically-typed. Most variables do not need to be declared as

a specific data type (real, string, integer, etc.) before assigning valuesto them. This feature is known as dynamic typing, because the type ofvariable is determined completely by the type of data assigned. Thetype of a variable can be changed at any time simply by assigning adifferent value of a different type to it.

Python is case sensitive. Python syntax, commands, and variable names areall case sensitive. Thus, three variables named Pressure, pressure,and PRESSURE are three completely different variables. Likewise, theprint() function is a valid Python statement, while Print() wouldthrow an error.

Python is object-oriented. Python was built from the ground up as an object-oriented language, though it can also be readily used for both proceduralprogramming and functional programming. Much of what we discussin this book can be learned without dwelling on the fact that Python isobject oriented. Relevant object-oriented concepts will be explained asthey appear.2

1.5 Notation and Syntax HighlightingIn general, we will identify Python statements, object names, code, stringliterals, user-typed input, and output written by a program using a type-writer typeface; for example, print(a, b). When placeholder names areused for function arguments or in assignment statements, these will beitalicized: fig, ax = plt.subplots(rows, cols). Optional argumentsfor functions and methods are indicated with italicized square brackets:contour([x, y,] z). The greater-than symbol ‘>’ is used to indicate in-put from an interactive terminal. Finally, for code samples and commands

2Those interested in quickly and effectively learning what object-oriented programmingis all about are encouraged to read the excellent book, The Object Oriented Thought Process, byMatt Weisfeld, published by Addison-Wesley.


appearing in separate blocks, we further use color-coded syntax highlightingto facilitate visual identification of the major language elements, as shown inTable 1.1

1.6 Installing PythonPython is open source and can be downloaded for and installed on any of thethree main operating systems—Windows, Linux, and macOS.3 The individualcomponents can be downloaded separately. There are also several pre-builtpackages assembled by various organizations and groups that install Pythonand a myriad of associated libraries. Some of these pre-built packages areopen source, while others charge a fee. Keep in mind that you can alwaysdownload and use Python and most libraries free of charge. Also, many Linuxand macOS operating systems come with Python preinstalled.

1.6.1 Pre-built packages

The easiest way to obtain and install Python and many of the analysis andplotting libraries useful for scientists is to download a pre-built package suchas Anaconda (www.anaconda.com) or Enthought’s Canopy (www.enthought.com).The big advantage of pre-built packages is that when you install libraries withthese packages the dependencies are automatically reconciled. Anaconda andCanopy may also be installed concurrently on the same computer withoutinterfering with one another (although you should always be aware of whichone is the default Python installation).

1.6.2 Installing individual libraries

As an alternative to installing one of the above distributions, one can down-load basic Python via www.python.org. Additional libraries can be downloadedindividually as needed. For the examples used in this book, Scipy, Numpy,Matplotlib, Pandas, and Cartopy may be downloaded from the followingsites:

- Numpy: www.numpy.org- Scipy: www.scipy.org- Matplotlib: www.matplotlib.org- Pandas: pandas.pydata.org- Cartopy: scitools.org.uk/cartopy

3The operating system for Apple’s Mac computers was previously known as Mac OS X, orOS X.

Installing Python 7

When downloading and installing individual components, it is very importantto ensure that the version of the library is compatible with the version ofPython being used. Also, there are certain dependencies that may need tobe met. For example, installing Matplotlib requires that Numpy already beinstalled. Careful reading of the documentation is imperative.

A website of great usefulness for Windows users is maintained by ChrisGohlke at http://www.lfd.uci.edu/~gohlke/pythonlibs/ . This site is a treasuretrove of Python extension installers tailored for Windows.

1.6.3 Package managers

Conda

Conda is an open-source package manager that is in common use for installingand updating Python libraries. Its reference page can be found athttps://docs.conda.io/en/latest. Conda has many commands that are executedin a terminal window, with relatively easy syntax. Conda comes installed aspart of the Anaconda Python installation.

If Conda is installed, then Python libraries may be easily installed with thecommand

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda -install newlibrary

where newlibrary is the name of the library to be installed. If you wishto install a community-supplied package available from the conda-forgerepository, then you would use

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda -install newlibrary -c conda -forge

Some users wish to permanently configure Conda so that it always automati-cally includes the conda-forge repository when searching for a library to beinstalled. This is accomplished as follows:

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda config --add channels conda -forge

Also, when working with multiple channels, it is now recommended that thefollowing command be executed once to reduce the likelihood of packageincompatibility problems. Beginning with Conda 5.0, this will be the defaultsetting. Note, however, that this command can cause problems with updatesto existing environments.

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda config --set channel_priority strict

Existing libraries may be updated using the command

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda -update library

What is particularly nice about Conda is that it will find and attempt to resolveany dependencies among the various libraries.


Conda may be used to update Python to the latest version4 using

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda update python

And Conda itself may be updated using the command

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda update conda

Multiple, independent Python environments may be created with Conda.Each virtual environment can have a different set of libraries. Environmentscan also be created that use different versions of Python itself. It is highly rec-ommended that you use virtual environments as a matter of habit rather thanthe default (root) environment, especially when installing packages outside ofConda. See additional comments under “Mixing package managers,” below.

To create a new Python virtual environment, type

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda create -n newenv library1 library2

where newenv is the name of the new environment, and libary1, library2,etc. are the names of any libraries to be installed in the new environment.If a different version of Python is desired, then include python=x.x in thecommand, where x.x is the version number desired (e.g., 2.7, 3.5).

Switching between environments is accomplished in Linux or macOS by

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginconda activate environment

and in Windows by

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginactivate environment

where environment is the name of the environment to be activated.

pip

A generic package manager that comes with most Python installations is pip,which is a recursive play on words standing for ’pip installs Python’. Thereference page for pip is at https://pip.pypa.io/en/stable.

To install a library from the command line using pip, type

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginpip install library

and to uninstall a library use

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginpip uninstall library

Mixing package managers

Occasionally it is necessary to use pip to install a specialized package thatisn’t available from the Conda or Enthought package managers. While this is

4This cannot be used to upgrade from Python 2.x to Python 3.x.

Alternate Python Implementations 9

possible, it should be done with care, because the default package managerwill not be aware of the changes, and it can break dependencies and lead to abroken environment. Before using pip to add a new package, use your regularpackage manager first to satisfy as many dependencies as possible for thatpackage. Also, it is important that you do all of this in a virtual environment,rather than your root environment, so that if anything does break, the damageis limited to that virtual environment, and you can easily recover by deletingthe affected environment and starting fresh with a new one.5

1.7 Alternate Python ImplementationsBecause Python is open source, there is no monopoly on interpreters forPython code. Besides the standard implementation of CPython6 (availablefrom https://www.python.org), several other interpreters and implementationare available:

- IPython (http:// ipython.org) is a version of Python for interactive com-puting from a command shell.

- Jython (http://www.jython.org) is a version of Python using a Java-basedinterpreter and running on the Java Virtual Machine.

- PyPy (http://pypy.org) is an implementation optimized for speed andefficiency of computational resources.

- StacklessPython (http://www.stackless.com) is based on CPython but isbuilt for thread-based programming.

1.8 Set-up and TestingIf you are using Anaconda or Canopy, you can skip this section. If youinstalled Python yourself, you may want to read on.

1.8.1 Path settings

A standard Python installation creates a directory structure where the pythoninterpreter and libraries are stored. On Windows, this is usually \python36.7For Linux and macOS installations, the directory is usually /usr/bin/python,/usr/local/bin/python, or /usr/local/python. Other Python installations

5For more about safely using pip with Conda, see https://www.anaconda.com/blog/using-pip-in-a-conda-environment.

6The name CPython comes from the fact that the Python interpreter itself is written in the Cprogramming language.

7The “36” refers to the version number, in this case Python 3.6.


may have a different directory name and structure. It is important to havethe PATH environment variable set to include the path to the desired Pythoninstallation.

1.8.2 Testing the Python installation

Testing from the command prompt

To test a standard Python installation, open a terminal window and typepython at the prompt. This should bring up an interactive shell. If nothinghappens, or an error appears, it is likely that the PATH environment variableis not set to include the proper directory (see previous section). If Python isinstalled properly and the correct path is set, you will see text similar to

xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginPython 3.7.3 (default, Mar 27 2�19, 16:54:48)xleftmarginxleftmarginxleftmargin xleftmargin xleftmargin[Clang 4.�.1 (tags/RELEASE_4�1/final)] ::xleftmarginxleftmarginxleftmargin xleftmargin xleftmarginAnaconda, Inc. on darwin

If IPython is installed then it may be opened by typing ipython from theterminal window.

1.8.3 Testing libraries

Most of the examples in this book use the Scipy, Numpy, and matplotliblibraries. To ensure that these libraries are properly installed, from an openterminal window running Python or IPython type import numpy and hit theenter or return key.8 If nothing happens other than a new prompt appearing,

it means that Numpy is installed and available. If error messages appear,it means that either Numpy is not installed or that the version of Numpyis not compatible with the version of Python installed. Likewise, typingimport scipy and import matplotlib will show whether these libraries areproperly installed.

1.9 Executing Python CodePython code can be executed interactively either by typing a statement directlyinto the Python shell or via one of the interactive development environments.Python code can also be saved to a file for later execution. The file canthen be executed by running it within one of the interactive developmentenvironments. The file can also be executed directly from the terminal windowcommand line.

8Python is case sensitive, so this command must be all lower case.

Executing Python Code 11

Under Windows, Python files are saved using either the .py or .pyw extension.9The difference between the two extensions is solely whether or not a newterminal window is opened during execution of the code.

- When a file having a .py extension is selected to run, the Python inter-preter is opened using the executable file python. This opens a newterminal window, and all standard input/output is then accomplishedvia this terminal window. Any error messages will also be displayed tothe open terminal window.

- When a file having a .pyw extension is selected, the Python interpreteris opened using the executable file pythonw. In this case a terminalwindow is not opened. This is appropriate if the program does not needto display or print information to the computer screen, and does notrequire user input.

Once a Python program file has been saved with either the .py or .pyw exten-sion, it can be executed via several different means:

Execution within a terminal window: Stored Python programs can be ex-ecuted from a terminal window prompt by typing python prognamewhere progname is the name of the stored program file. As long as thePATH environment variable includes the directory containing the Pythoninterpreter executable files, the system will locate these files, run theinterpreter, and execute your program.

Typically the user need not worry about the location of the pythonexecutable files, as the appropriate directory is normally added to theuser’s path variable when Python is installed. These files normallyreside in a directory named PythonXX, where XX is the version numberof the Python installation. However, they may be located in otherdirectories for nonstandard installations. To find the location of thePython executable files, first open the Python interpreter window andimport the sysmodule by typing import sys. Then type sys.prefix,which will return the path to the Python installation. The executablefiles will either be in this directory, or one of its subdirectories, likely thescripts or bin subdirectory. The specific location will depend on Pythoninstallation and the operating system.

Execution within an interactive development environment: You can openand edit programs within an interactive development environment suchas Canopy, Spyder, Scite, IPython, etc. The development environmentwill either have a play button or a ‘Run’ menu heading that can beselected to run the program.

9Under macOS and Linux, only the .py suffix is normally used.


Execution from a desktop icon: It is also possible to execute stored Pythoncode by directly clicking on the desktop icon for the program file. Inthis case, the program icon must be associated with either the python orpythonw files. Clicking on the file will then load the Python interpreterand execute the commands within the program file.

Execution as a Jupyter notebook: Another means of executing Python codeis as a Jupyter notebook, which allows text, mathematical symbols, andPython code to be combined in a single document. This makes forease of sharing computations and results, and is also very useful forclassroom instruction. Jupyter notebook files have a .ipynb file extension.Many Python installations, including Anaconda and Canopy, supportJupyter notebooks. Jupyter Lab is the latest, browser-based interface forrunning Jupyter notebooks. Appendix A contains a very basic primer onJupyter notebooks. More detailed information about using notebookscan be found at https:// jupyter.org.

1.10 Additional ResourcesThere is considerable online documentation for Python and of the librariesdiscussed in this book, if one is patient enough to track it down. Much ofthe information is spread across a variety of unrelated sites, and explanationsare often spotty. One purpose of this book is to bring the more essentialinformation together into a coherent tutorial and reference. However, thisbook certainly is not a complete reference manual for Python or its libraries,and other resources can be indispensable, including those listed below.

Books. An exhaustive bibliography is not presented here; however, a fewparticularly notable books relevant to new scientific users of Python are worthmentioning.

- An outstanding overall reference book for the Python language itself isthe Python Essential Reference by David Beazley.

- A beginning book for those working mainly in the atmospheric andoceanic sciences is A Hands-On Introduction to Using Python in the At-mospheric and Oceanic Sciences by Johnny Wei-Bing Lin. Scientific usersoutside of these fields may also find this text useful.

- High Performance Python by Micha Gorelick and Ian Ozsvald emphasizestechniques for using Python for numerically intensive computations.

- Effective Computation in Physics by Anthony Scopatz and Kathryn D. Huffdiscusses Python as one of several software tools for general purposecomputing in the physical sciences.

- Though not a Python-specific book, the aforementioned The Object-oriented Thought Process by Matt Weisfeld is an excellent resource forthose wanting to understand the essence of object-oriented program-ming in any language.

Additional Resources 13

Documentation web sites. Python and most libraries each have their owndocumentation web sites, some of which are listed here.

- Python: https://docs.python.org/3- NumPy: http://docs.scipy.org/doc/numpy/reference- SciPy: http://docs.scipy.org/doc/scipy/reference- matplotlib: http://matplotlib.org- Jupyter Notebook: https:// jupyter.org

Documentation web sites for other libraries can easily be found using a websearch.

User forums. There are several user forums for Python and its libraries. Thesecan be found using a web search.

CHAPTER

TWO

SYNTAX AND DATA TYPES

2.1 General SyntaxPython syntax is clean and fairly intuitive. This section discusses the majorfeatures.

2.1.1 Python is case-sensitiveCare must be exercised to ensure proper use of capitalization in Python syntax,in both Python statements and in names for variables, modules, and functions.A variable named Pressure is not the same as a variable named pressure.Likewise, the proper call to the print() function is all lower case. Use ofPrint() would result in an error, unless a user-defined function of this nameis created.

2.1.2 FunctionsFunctions are called by giving the name of the function, followed by thearguments list contained within parentheses and separated by commas. If afunction does not require any arguments, the parentheses must still be present.Functions are discussed in more detail in Chapter 8.

2.1.3 Writing output to terminalOne of the simplest Python functions is print(args), which prints the argu-ments args to the standard output, which is usually the terminal. The print()function is discussed in more detail in Sec. 5.9.

In Python 2 print is a statement, not a function, and hence is called not using parenthesesbut with a space before the first item to be printed.

15

16 2. Syntax and Data Types

2.1.4 Code blocks and indentationAlthough Python is a free-form language in the sense that there is no require-ment to begin a statement at any particular column of a record, it is structuredin the sense that indentation plays a key role in delineating blocks of code. Asan example, a simple for-loop in Python has the form

for i in range (�,5):x = i**2print(x)

The colon : indicates that a code block follows. Any statements after this thatare indented by the exact same amount are part of the code block. The codeblock ends when a statement is no longer indented. The amount of initialindentation is arbitrary, but most Python programmers prefer an indentationof between two and four spaces.

In other languages, code blocks are not required to be indented, though properprogramming style suggest they should be. One of Python’s many strengthsis that it requires code blocks to be indented.

2.1.5 Continuation of linesThere is no limit to how long a line in a Python program can be. However, itis best to break long lines for easier reading. The \ symbol is used at the endof a line of code to signify that the next line is a continuation. When using acontinuation line symbol the normal rules of indentation are suspended, sothe code can be formatted for readability using as many spaces as are desired.For example, the three lines of code below all produce the same result whenprinting the string variable y.

y = 'The quick brown fox jumped over the lazy dog.'y = 'The quick brown fox jumped \over the lazy dog.'y = 'The quick brown fox jumped ' \

' over the lazy dog.'

In certain circumstances a line can be continued without the use of the \character. Comma-separated items contained within parentheses (), squarebrackets [], or curly braces {} can be continued on multiple lines without theuse of \.

2.1.6 CommentsComments in Python are indicated by use of the # symbol. Any code betweenthis symbol and a new line is ignored. Comments can begin at the beginningor in the middle of a line of code.

Python syntax does not officially support commenting out entire blocks ofcode; however some programmers use the docstringmechanism of Python

Variable Names and Assignment 17

to do this by enclosing the block in triple single-quotes ('''). It should bekept in mind, however, that the intended purpose of docstrings is specificallyto document the programming interface to functions. Thus, docstrings are notactually ignored by the compiler, unlike true comments.

Many code editors do allow a block of code to be selected and then commentedor uncommented out using context menus. In Jupyter, for example, blockcommenting is accomplished by highlighting the code to be commented andthen pressing control + / at the same time, which will prepend # to all linesin the block. This same key sequence will also uncomment a highlighted blockof code.

2.2 Variable Names and Assignment2.2.1 Valid names

Any letter, number, or the underscore (_) character may be used when naminga variable, function, module, or object. However, there are a few rules to keepin mind.

- A name cannot begin with a number as the first character.- Although an underscore may be used as the leading or trailing character,

avoid doing so unless you know what you are doing since there are spe-cial rules for how the interpreter handles leading and trailing underscorecharacters. Details may be found in the Python documentation.

- Remember that Python is case sensitive, so that a variable namedPressure and one named pressure are two distinct variables!

2.2.2 Reserved words

There are 32 reserved words in Python that should never be used as variableor function names. These are: and, as, assert, break, class, continue, def,del, elif, else, except, exec, finally, for, from, global, if, import, in, is,lambda, nonlocal, not, or, pass, print, raise, return, try, while, with, andyield.

2.2.3 Variable assignment

Assigning a value or reference to a variable is done using the assignmentoperator =. For example,

a = 6.7

assigns the floating-point value 6.7 to a variable named a. Python also sup-ports parallel assignment, whereby multiple variables can receive assignedvalues on a single line of code. For example, instead of using three lines ofcode,


a = 6.7b = 'hello'c = 8

we could use a single line of code:

a, b, c = 6.7, 'hello', 8

2.3 Numeric Data Types

2.3.1 Boolean

The Boolean data type has two possible values, True and False (note thecapitalization.) The value of True is also numerically equivalent to the integervalue 1 or the floating-point value 1.0. The value of False is numericallyequivalent to the integer value 0 or floating-point value 0.0.

Integers, floating-point values, and some other data types can be converted toBoolean values using the bool() function. There are important rules to keepin mind when converting to Boolean.

- Any zero value or empty object will convert to False.- All other values or objects will convert to True.

For example:1

> bool (3)True> bool (�)False> bool(-3)True> bool(None)False> bool ([]) # An empty listFalse> bool([3, 5, 'hi'])True

2.3.2 Integer

The integer data type is used to hold integers (no decimals). The number ofdigits is limited only by memory constraints.

1The notation > indicates that this line is being input using an interactive Python terminal.

Numeric Data Types 19

In Python 2 there were two integer data types: integer, and long-integer. Long integershad an ‘L’ or ‘l’ after the number. Python 3 has consolidated these into a single integerdata type.

2.3.3 Floating-point

All floating-point numbers are stored as 64 bits, which is the equivalentof double-precision in other programming languages. Scientific notation isachieved by using either an upper or lower case E or e between the mantissaand the exponent, such as 4.6e9, -7.3e3, .��65E-64, etc.

2.3.4 Complex

Python has a built-in data type for storing and manipulating complex numbers.A complex number is represented in the form of 3.5+9.7j, where the firstnumber is the real part and the second number, followed by the j, is theimaginary part. Complex numbers can be created by either directly typing thenumbers in complex-number format, or by using the built-in complex(real,imag) function, where real and imag are the real and imaginary components.Examples are shown here.

> a = 4.9 - 8.3j> a(4.9-8.3j)> b = complex (2.3, 4.6)> b(2.3+4.6j)

The direct-typing means of creating a complex number only accepts actualnumbers (integer or floating-point), not variables representing numbers. Thecomplex() function accepts either. This feature is illustrated below.

> c = 3> d = -4.5> e = c+djTraceback (most recent call last):File "<stdin>", line 1, in <module>NameError: name 'dj' is not defined> e = complex(c, d)> e(3-4.5j)

The complex data type contains two attributes for returning the real and imag-inary components of the number, and one method for returning the conjugateof the complex number. These attributes and methods are accessed as shownhere.


> a.real4.9> a.imag-8.3> a.conjugate ()(4.9+8.3j)

Note that the attributes real and imag are accessed without parentheses,while the method conjugate has parentheses. Attributes and methods aremore fully explained in Sec. 2.4.

2.3.5 Conversion between numeric typesIntegers are converted to floating-point values by using the float() function.A floating-point number is converted to an integer via the int() function.2

> float (4)4.�> int (4.6)4

Note that the int() function results in truncation of the floating-point number.If rounding is desired, use the round() function. For numbers that are equallydistant between two integers, such as 4.5, the round() function rounds to theeven multiple, 4. The round() function is described further in Sec. 4.5.

> round (4.4)4> round (4.5)4> round (4.6)5> round (5.5))6

Rounding of numbers that are equally distant between two integers is very differentin Python 3 vs. Python 2. In Python 3 such numbers are always rounded to the evenmultiple, while in Python 2 they are always rounded away from zero.

2.4 Objects, Attributes, and MethodsThe complex data type provides us an opportunity to discuss the object-oriented nature of Python. In Python, nearly all entities are objects. An object

2Python 2 has a long() function for converting to the long-integer data type.

Objects, Attributes, and Methods 21

is a member of a class which can be thought of as a template or blueprint thatdefines the characteristics of the object. When an object is created from theclass it is said to be instantiated. For example, a complex variable defined as c= 4.5-3.2j is a complex object, meaning it is an instantiation of the complexclass.

2.4.1 Data types as classes

In previous sections we have been referring to data types such as integer,floating-point, Boolean, complex, etc. Actual variables that are of these data typesare really instantiations of the objects defined by the respective classes int,float, bool, and complex. We will continue to refer to data types, but reallythey are classes. So when we say something is of the integer data type wereally mean that it is of the int class. Likewise, when we talk about a complexnumber, we really mean an object of the complex class.

2.4.2 Attributes

Objects may have attributes associated with them, which are really nothingmore than data that belongs to or is associated with the particular object.Complex numbers (complex objects) have two attributes. One attribute isnamed real and contains the real part of the complex number. The otherattribute is named imag and contains the imaginary part of the number.

Attributes are accessed by giving the object’s name, a period, and then thename of the attribute.

> c = 4.5 - 3.2j> c.imag-3.2

Note that there are no parentheses associated with attributes.

2.4.3 Methods

Objects may also have methods associated with them. A method can simplybe thought of as a function that belongs to an object, and operates on theobject’s attributes. Complex numbers have one method called conjugate().This method operates on the two attributes, real and imag, and returns thecomplex conjugate of the number.

Methods are invoked by giving the object’s name, a dot, and then the name ofthe method with arguments contained in parentheses. Many methods requireno arguments, such as the conjugate() method of complex numbers, butthese still require the parentheses.

> c = 4.5 - 3.2j> c.conjugate ()(4.5+3.2j)


If the parentheses are not provided, Python will not give an error. Instead, itwill simply return a bound method, which is then primed to return values oncearguments are provided.

> d = c.conjugate> d<function complex.conjugate>> d()(4.5+3.2j)

Though using bound methods is sometimes desired, neglecting to provideparentheses to a no-argument method such as c.conjugate() can cause manydebugging headaches, so care should be taken to provide parentheses tomethod calls.

2.4.4 Inspecting objects

Objects can be inspected using the built-in dir(obj) function, where obj isthe object to be inspected. This returns a list of an object’s attributes andmethods.

Many objects also contain a documentation string, which contains informationabout the object. The documentation string can be accessed by calling the__doc__() method of the object (note the two leading and trailing under-scores.)

The inspectmodule is another powerful tool for inspecting objects. Thoughonly covered very briefly in Sec. 8.1.9, the interested reader may consult theonline Python documentation for more information.

2.5 The None Data TypeIf a variable is created but not explicitly assigned a value or an object, thenit has the None data type. The None data type is essentially the null value orempty value. It is not the same as the value 0. The None data type can beassigned to a variable by simply typing a = None.

2.6 StringsStrings are defined by including text within single, double, or triple quotes.Chapter 3 contains a thorough discussion on defining and manipulatingstrings.

2.7 Lists and TuplesLists and tuples are both collections of values or objects. The major differencebetween them is that lists are mutable, meaning that their contents can be

Lists and Tuples 23

changed after creation, while tuples are immutable. Other than this differ-ence, most operations on lists and tuples are identical. The values or objectscontained within a list or tuple do not have to all be of the same type. Thecollection can be a mixture of any data types or objects, and can even containother lists or tuples.

Lists are denoted by enclosing a comma-separated collection of objects withinsquare brackets, while tuples are enclosed within parentheses.

> l = [4.5, -7.8, 'pickle ', True , None , 5] # List> t = (4.5, -7.8, 'pickle ', True , None , 5) # Tuple

2.7.1 Accessing elements of a collection

Lists and tuples are both examples of collections, which are objects that containmultiple elements. In this section we describe how elements of collections,including lists and tuples, are accessed. We will use lists and tuples in the fol-lowing examples, but the indexing scheme can be applied to other collectionsas well.

The elements of a collection are accessed by specifying the integer indexof the element’s position enclosed within square brackets after the name ofthe collection. The first element always has index 0. Negative indexes arereferenced backwards from the end of the collection, with �1 representingthe last element. Some examples:

> mylist = [�, -5, 8, 'hi', (1,3,5), False]> mylist [�]�> mylist [1]-5> mylist [4](1, 3, 5)> mylist [5]False> mylist [-1]False> mylist [-3]'hi'

Notice that mylist[5] and mylist[-1] both return the last element.

2.7.2 Nested elements

Since lists and tuples may contain other lists and tuples, we need to know howto access nested elements. This is accomplished by using a series of indexes,each contained within its own brackets. This is best shown with an exampleof a nested list with two levels of nesting.


> mylist = [[3,4,8],[6,2], [8,-8,1], [-3, 5, 2, 7]]> mylist [�][3, 4, 8]> mylist [�][1]4> mylist [2][ -1]1> mylist [�][�]3> mylist [-1][-1]7

2.7.3 Using ranges of indexes

Multiple, consecutive elements can be accessed at once using ranges of in-dexes. The syntax for this is b:e, where b is the beginning index and e isone integer greater than the ending index to be accessed. The range specifiedin this manner will access indexes beginning with b and ending with e-1.For example, mylist[2:8]would only return elements with indexes from 2through 7. For those not familiar with this indexing system, this is unintuitiveand can be a headache when trying to debug Python programs. It is perhapsless confusing if you think of the numerical indices as referring to boundariesbetween elements of a collection, as illustrated here:

A B C Y Z0 1 2 3 24 25 26

-1-2-26 -25 -24 -23

In this example, the ranges �:3, -26:3, �:-23 and -26:-23 all refer to thesame elements A, B, and C. The range 24:26 is the same as the range -2: andrefers to elements Y and Z.

If the leading integer b is left out then all elements from 0 to e-1 will bereturned, while if the trailing e is omitted then all elements from b to the endof the collection are returned. If : is used without either b or e, then everyelement is returned.

> mylist = [�, 1, 2, 3, 4, 5, 6, 7, 8]> mylist [3:7][3, 4, 5, 6]> mylist [:][�, 1, 2, 3, 4, 5, 6, 7, 8]

Lists and Tuples 25

> mylist [:4][�, 1, 2, 3]> mylist [3:][3, 4, 5, 6, 7, 8]> mylist [-5:][4, 5, 6, 7, 8]

2.7.4 Striding and reversing

Using ranges of indexes provides a powerful means of skipping over indexesor even reversing the contents of a list or tuple. Striding is accomplished byincluding a third integer s in the range construct, b:e:s, which defines thestriding interval. For example:

> mylist = [�, 1, 2, 3, 4, 5, 6, 7, 8]> mylist [�: -1:2][�, 2, 4, 6]

Striding can even be performed backwards by using a negative interval.

> mylist [:: -1][8, 7, 6, 5, 4, 3, 2, 1, �]> mylist [-1:�: -2][8, 6, 4, 2]> mylist [:: -2][8, 6, 4, 2, �]

2.7.5 Reassigning values

Once a tuple is created it may not be altered, since it is immutable. Lists,however, may have their values reassigned, as shown in these examples.

> p = ['cat', 'dog', 'ferret ', 'llama']> p[2] = 'squirrel '> p['cat', 'dog', 'squirrel', 'llama']> p[�:2] = ['zebra', 'monkey ']> p['zebra', 'monkey', 'squirrel', 'llama']

2.7.6 The range() function

The built-in range() function generates a range object. A range object be-haves somewhat like a list containing a sequence of integers. However, it isnot an actual list, but is instead an iterator that generates the next integer inthe sequence on demand as needed (see Sec. 5.4). The syntax is range(b,e,s)where b is the beginning integer, e is one greater than the last integer to begenerated, and s is the stride.


Two important things to note about the range object are:

- It is not an actual list!- It does not generate the integer representing the ending integer in the

arguments list, e.

A range object can be converted to a list by using the list() function, or to atuple by using the tuple() function. Some examples:

> range (�,1�)range(�, 1�)> list(range (�,1�))[�, 1, 2, 3, 4, 5, 6, 7, 8, 9]> list(range (�,1�,2))[�, 2, 4, 6, 8]> list(range (1�,�,-1))[1�, 9, 8, 7, 6, 5, 4, 3, 2, 1]> tuple(range(1�,-1,-3))(1�, 7, 4, 1)

The behavior of the range() function differs significantly between Python 3 and Python 2.In Python 2 the function returns a list object, not a range object. To return a range objectin Python 2 you would use the now-deprecated xrange() function.

2.7.7 Functions and methods for lists

There are several built-in function and methods for manipulating lists, andfor returning information about lists and tuples. Some of these are explainedhere using ls to indicate the name of a list.

- len(ls) returns the number of items in the list.- del ls[i:j]3 deletes items at indexes i through j-1.- ls.append(elem) adds element elem to the end of the list.- ls.extend(elems) adds multiple elements given by another list or tu-

ple, elems, to the end of the list.- ls.count(target) returns the number of instances of target contained

in the list.- ls.index(target) returns the first index of the list that contains target.

If optional i and j are given, it returns first index of occurrence in therange i through j-1.

- ls.insert(i, elem) inserts elem at index i.

3del is actually a statement, not a function.

Lists and Tuples 27

- ls.remove(target) removes first occurrence of target from the list.- ls.reverse() reverses the list in place.- ls.sort() sorts the list in place. If the keyword reverse is set to True,

then it also reverses the results of the sort.

Note that the reverse() and sort()methods both change (mutate) the origi-nal list. They do not merely return a copy.

2.7.8 Zippering and unzippering lists

The zip() function allows you to combine two or more separate lists or tupleslike a zipper. The result is a zip object, which is an iterator (see Sec. 5.4). Thezip object can be turned into a list using the list() function.

> a = [1, 2, 3]> b = [4, 5, 6]> c = list(zip(a, b))> print(c)[(1, 4), (2, 5), (3, 6)]

If the original lists each have N elements, the result is a list of N tuples. If onelist is longer than the other the extra elements are simply lost.

The zip() function does especially interesting things when combined withthe asterisk * operator (see Sec. 2.13), as in the following example.

> c = [(1, 4), (2, 5), (3, 6)]> d = zip(*c)> print(d)[(1, 2, 3), (4, 5, 6)]> e = zip(*d)> print(e)[(1, 4), (2, 5), (3, 6)]

Note that the list e is the same as the original list c. In other words, zip() isthe inverse of itself when applied to an asterisked list! In essence, it becomesan unzippering function.

Like the range() function, the behavior of the zip() function differs significantly betweenPython 3 and Python 2. In Python 2 the function returns a list object, while in Python 3it returns a zip object.


2.7.9 List comprehension

A shorthand means of looping through the elements in a list or tuple and cre-ating a new list is known as list comprehension. Although a full understandingof list comprehensions requires knowledge of loops, which are not covereduntil Sec. 5.5, a brief example is provided here.

Suppose we have a list comprised of numbers represented as strings,ls = ['2.3', '-4.5', '7.8', '12.3'], and we want to convert it into alist of floating-point values. We cannot simply use fls = float(ls), sincethe float() function cannot take a list as the argument. A list comprehensionwould look like fls = [float(s) for s in ls], which would create thenew list named fls holding the floating-point values. See Sec. 5.8 for details.

2.8 DictionariesDictionaries are similar to lists, except that instead of being indexed usingintegers, the values are each tied to a key that may be a number, a string, oranother object. In other languages dictionaries are referred to as hashes orassociated arrays.

2.8.1 Creating and accessing

There are several ways to create dictionaries. One is to use a series ofkey/value pairs within curly braces, with the keys separated from the valuesby colons.

d = {'first':'John', 'last':'Doe', 'age':34}

In this example, the keys are 'first', 'last', and 'age'while the values are'John', 'Doe', and 34.

Dictionaries can also be created using the dict() function and providing thekey/value pairs as arguments, either as a series with the keys and valuesseparated by an equals sign or as a nested list of key value pairs.

d = dict(first='John', last='Doe', age =34)d = dict ([['first','John'], ['last', 'Doe'],

['age', 34]])

The values in a dictionary are accessed by giving the name of the dictionaryfollowed by a key in square brackets.

> d['first']'John'> d['age']34

Dictionaries are mutable, so values can be reassigned.

Sets 29

2.8.2 Adding elementsEntries can be added to an existing dictionary by using the assignment opera-tor and providing a new key and value. For example, the code

> d'age': 34, 'last': 'Doe', 'first': 'John'> d['weight '] = 187> d'age': 34, 'last': 'Doe', 'weight': 187, 'first': 'John'

shows how to add a new entry with a key named 'weight' to an existingdictionary.

2.8.3 Functions and methods for dictionariesSome useful built-in functions and methods for dictionaries are listed here.

- len(d) returns the number of key/value pairs in d .- del d[k] removes the item from d whose key is k .- k in d checks to see whether d contains an item with key given by k .

It returns either True or False.- d.clear() deletes all items in the dictionary.- d.copy() returns a copy of the dictionary.- d.keys() returns a list of all the keys in the dictionary.- d.items() returns a list containing tuples of all the key/value pairs in

the dictionary.- d.values() returns a list of all the values in the dictionary.

2.9 SetsSets are a collection of objects that are unordered. They are created by includ-ing the members within curly braces, such as s = {4, 9, 2, 'cat'}, or byusing the set() function with a list or tuple as the argument. Sets cannotcontain duplicate members, so if a tuple or list is converted to a set, dupli-cates are eliminated. This property of sets is handy for eliminating duplicateentries from categorical data. For example, suppose we had a list containingautomobile makes, and we wanted to know how many unique makes the listcontains. The following shows how this is done.

> make = ['Ford', 'Chrysler ', 'Chevrolet ', 'Toyota ','Ford', 'Toyota ','Honda', 'Subaru ']

> mset = set(make)> mset'Chevrolet', 'Chrysler', 'Ford', 'Honda',

'Subaru', 'Toyota'> len(mset)6


Sets have operators that can return the union, intersection, difference, andsymmetric difference between two sets. These are illustrated for the two setss and t.

> s = {-5, -4, -3, -2, -1, �, 1, 2, 3, 4}> t = {�, 1, 2, 3, 4, 5, 6, 7, 8, 9}

Union: The operator | returns all the elements that appear in either set.

> s | t-5, -4, -3, -2, -1, �, 1, 2, 3, 4, 5, 6, 7, 8, 9

Intersection: The operator & returns all elements that are common to bothsets.

> s & t�, 1, 2, 3, 4

Difference: The operator - returns the items that are unique to the set on theleft side.

> s - t-5, -4, -3, -2, -1> t - s5, 6, 7, 8, 9

Symmetric difference: The operator ^ returns items that are unique to oneor the other of the sets.

> s^t-5, -4, -3, -2, -1, 5, 6, 7, 8, 9

A single item can be added to a set using the add(item) method. Multipleitems can be added using the update(items)method, where items is a listor tuple containing the items to be added. An item can be removed from a setusing the remove(item)method.

2.10 ArraysPython does have a built-in module for defining and manipulating arrays.These are very similar to lists, with the exception that they can only hold asingle type of data. For scientific computing it is much more efficient to usethe arrays defined in the Numerical Python (Numpy) module, which are thesubject of Chapter 7. Therefore, we will not discuss the built-in arrays further.

2.11 Finding an Object’s TypeThe built in type() function allows an object’s type to be determined.

Copying Objects 31

> a = 4> type(a)int> b = 4.5> type(b)float> c = 'Hello'> type(c)str> d = 4+7j> type(d)complex

2.12 Copying Objects

Copying objects in Python must be done with care, as there are different levelsof copying. Shallow copies are not actual separate copies of an object, butare merely pointers or references to the original object. Deep copies are trulyseparate but identical clones of an object. When the equals sign (=) is used tocopy an object in Python, the copy may be either deep or shallow dependingon the original type of the object. For simple objects such as integers orfloating-point numbers, the equals sign will return a completely new, or deep,copy. But for compound objects such as lists, tuples, or dictionaries, Pythoncreates only a shallow copy. The following code demonstrates this behavior.

> a = ['cat', 'dog', 'mouse']> b = a> a, b(['cat', 'dog', 'mouse'], ['cat', 'dog', 'mouse'])> a[1] = 'frog'> a, b(['cat', 'frog', 'mouse'], ['cat', 'frog', 'mouse'])

Note that in this example, a and b are the exact same object! When an elementof a is changed, the corresponding element of b also changes.

To create a deep copy of an object you must first import the copymodule, andthen use the deepcopy() function, as illustrated here:

> import copy> a = ['cat', 'dog', 'mouse']> b = copy.deepcopy(a)> a, b(['cat', 'dog', 'mouse'], ['cat', 'dog', 'mouse'])> a[1] = 'frog'> a, b(['cat', 'frog', 'mouse'], ['cat', 'dog', 'mouse'])


2.13 Unpacking Lists and TuplesThe unpacking operator, *, can be used to unpack lists, tuples, or othersequence-like objects. One use for this is illustrated here,

> my_list = [�, -5, 8, 'hi', 'brown', False]> a, b, *c = my_list

In this syntax the first element of my_list is assigned to variable a, the secondelement is assigned to variable b, and all of the rest of the elements, regardlessof how many are left, are assigned to variable c as a list, yielding

> a, b, c(�, -5, [8, 'hi', 'brown', False])

The variable preceded by the * does not have to be the last variable in the list.For instance,

a, *b, c, d = my_list

distributes the contents of my_list among the four variables as follows

> a, b, c, d(�, [-5, 8, 'hi'], 'brown', False)

Two important points to note:

- In any such unpacking there may only be one variable that is precededby the * operator.

- If there are not enough elements left for the asterisked variable then itwill contain an empty list.

The unpacking operator is particularly useful is when passing a tuple or alist containing arguments to a function. For example, say we had the list a= [-2, 11, 2] and we wanted to send it to the range() function to be usedas the beginning, ending, and striding values. However, the range functiondoes not accept a list as its argument, but needs the three values separated. Byusing the unpacking operator we can send our list to the function as follows

> range(*a)range(-2, 11, 2)

The previous code is much simpler than the following alternative:

> range(a[�], a[1], a[2])

INDEX

!= , see Mathematical operators, compari-son operators

** , see Functions, open-ended argumentlists, see Mathematical operators,power

* , see Functions, open-ended argumentlists, see Mathematical operators,multiplication, see Unpacking,lists and tuples

+ , see Mathematical operators, addition- , see Mathematical operators, subtraction,

see Sets, difference operator// , see Mathematical operators, truncat-

ing division/ , see Mathematical operators, division: , see Code blocks<= , see Mathematical operators, compari-

son operators< , see Mathematical operators, compari-

son operators== , see Mathematical operators, compari-

son operators= , see Assignment operator>= , see Mathematical operators, compari-

son operators> , see Mathematical operators, compari-

son operators@, see Decorators, see Matrices, multipli-

cation% , see Mathematical operators, modulo& , see Sets, intersection operator^ , see Sets, symmetric difference operator\ , see Continuation line| , see Sets, union operator

abs(); Python function, 46Absolute value

complex input, 51real input, 46, 51

absolute(); Numpy function, 51abspath(); os.path function, 76add(); set method, 30add_axes(); Figure method, 135, 167add_cyclic_point(); cartopy.util function,

199add_feature(); GeoAxes method, 195add_line()

Axes method, 152Axes3D method, 205

add_subplot(); Figure method, 168Animation, see Plotting, animationappend()

list method, 26Numpy function, 93

arange(); Numpy function, 83arccos(); Numpy function, 50arccosh(); Numpy function, 50arcsin(); Numpy function, 50arcsinh(); Numpy function, 51arctan(); Numpy function, 51arctan2(); Numpy function, 51arctanh(); Numpy function, 51array()

Numpy function, 79numpy.ma function, 100

Arrays, 79–104adding elements to, 93–94appending, 93assigning elements, 94Boolean arrays, 98broadcasting, 89converting data types, 81copying, 85creating, 79–80, 82–85data types, 80–81

converting, 81deleting elements, 94empty, 83

333

334 INDEX

fancy indexing, 98finding shape/size, 90implicit loops, 88indexing, 85–87

1-D arrays, 85lists, 86multidimensional arrays, 86striding, 86

inserting elements, 94masked arrays, 100memory mapping, 104–106Python arrays vs. Numpy arrays, 30reading from .npy and .npz files, 103–

104reading from text file, 102–103reassigning elements, 94removing

elements, 94empty dimensions, 92–93

reshaping, 90resizing, 90reverse sorting, 92reversing, 86searching, 97–99sequential, 83–85singular dimension, 92–93slicing, 85–86sorting, 91–92

reverse, 92splitting, 96squeezing, 92–93stacking, 94statistics, 89–90transposing, 90uninitialized, 83view vs. copy, 85, 87where() function, 97writing to .npy and .npz files, 103–

104writing to text file, 101–102

ASCII, 41askdirectory(); tkinter.filedialog function,

70askopenfilename(); tkinter.filedialog func-

tion, 69Assignment operator = , 17astropy module, 304astype(); array method, 81Attributes

of classes, 122of objects, 21, 122

average(); Numpy function, 89Axes, see Plotting, axes objectsAxes3D(); mpl_toolkits.mplot3d.axes3d

function, 201axhline()

Axes method, 150pyplot function, 150

axvline()Axes method, 150pyplot function, 150

Bar charts, see Plotting, bar chartsbarbs()


Basemap; replaced by Cartopy, 191basename(); os.path function, 76Benchmarking code, see Timing codeblackman(); Numpy function, 282bool(); Python function, 18bool_(); Numpy function, 80Boolean data type, see Data types, BooleanBound function, see Functions, boundbreak; Python statement, 61Buffer, 64, 73Byte strings, see Strings, byte strings

C programming language and Python, seeCython

.c file extension, see File extensions, .cCartopy, 191–199

Basemap replacement, 191creating map, 191–195cyclic boundaries, 197–199plotting data, 195–197

ceil(); Numpy function, 51center(); string method, 39clabels()


Class attributes, 123–124Class methods, see Methods, classclass; Python statement, 122Classes, 121–125

attributes, 122–124child class, 124defining, 121documentation string, 122inheritance, 124__init__ method, 122, 125instantiation, 21, see Objects, instanti-

ation

INDEX 335

methods, 122parent class, 124self statement, 122

@classmethod; decorator, 127clear()

Axes method, 138dictionary method, 29

Clearing figures and axes, 138clf()

Figure method, 138pyplot function, 138

clip_on keyword, 165, 189close()

Dataset method, 246file method, 68HTTPResponse method, 77

cls; reference to class, 125Code blocks, 16Collections, 23colorbar()

Figure method, 183pyplot function, 183

Colorscolormaps, 186–187names, 140–141plot lines, 140–142plot markers, 141–142text, 145

Comments, 16Comparison of objects, see Mathemati-

cal operators, comparison oper-ators

compile(); re function, 266Complementary error function, 52Complex data type, see Data types, com-

plexcomplex(); Python function, 19Compound objects, 31Conda; package manager, 7–8Conditional statements, 56–58conjugate(); complex method, 20Constants, see Physical constantsContinuation line, \, 16continue; Python statement, 61Contour plots, see Plotting, contour plotscontour()


contourf()Axes method, 178pyplot function, 178

copy()dictionary method, 29Numpy function, 85

Copyingcompound objects, 31deep copy vs. shallow copy, 31, 85Numpy arrays, 85

copysign(); Numpy function, 51corr()

Pandas DataFrame method, 232Pandas Series method, 224

Correlation, 224, 232, 293cos(); Numpy function, 50cosh(); Numpy function, 50count()

list method, 26Pandas Rolling method, 227string method, 38

cov()Pandas DataFrame method, 232Pandas Series method, 224

Covariance, 224, 232csv module, see Files, delimited files, csv

moduleCython, 307–308

Data typesas classes, 21Boolean, 18complex, 19–20conversion of numeric types, 20floating-point, 19integer, 18long-integer, 19None, 22Numpy, see numpy module, data

typesstrings, see Strings

Dataset(); netCDF4 class, 242date; datetime class, 210dateparser module, 216–217Dates and times, 209–218

arithmetic, 218current time, 210dateparser module, 216–217datetime module, 210–218

timedelta objects, 217formatting strings, 214–215naive vs. aware, 209–213parsing strings, 216–217pytz module, 212

336 INDEX

server time, 210the epoch, 213time module, 218timestamp, 213timezone, 209–212

datetime module, 210–216, see Dates andtimes, datetime module

timestamp(), 213timezone.utc, 211utcfromtimestamp(), 213

datetime objectsattributes, 213useful methods, 214

datetime; datetime class, 210Decorators, 117–119

@classmethod, 127@interact, 324@jit, 306@property, 129@staticmethod, 126

deepcopy(); copy function, 31def; Python statement, 107degrees(); Numpy function, 50del; Python statement, 26, 29delete(); Numpy function, 94derivative(); scipy.misc function, 296Derivatives, see Differentiationdescribe()


destroy(); tkinter window method , 69dict(); Python function, 28Dictionaries, 28–29Differentiation

continuous functions, 296discretely sampled functions, 294higher-order methods, 296

dir(); Python function, 22Directories

finding location of python executablefile, 11

interactive selection, 68–70dirname(); os.path function, 76divmod(); Python function, 46__doc__(); object method, 22Documentation string, 22, 117, 122drop(); Pandas DataFrame method, 233dtype

attribute of Numpy array, 81keyword, 80

Dynamic typing, 5

e; Numpy constant, 48elif; Python statement, 56else; Python statement, 57empty(); Numpy function, 83empty_like(); Numpy function, 83encode(); string method, 41endswith(); string method, 38enumerate(); Python function, 58, 60erf(); numpy.math function, 52erfc(); numpy.math function, 52Error function, 52Error handling, 64–66Escaping special characters, see Strings,

special characters, escapingExcel files, see Files, Excelexcept; Python statement, 64exists(); os.path function, 76exp(); Numpy function, 51expm1(); Numpy function, 51Exponential functions, 51extend(); list method, 26

F2PY, 308fabs(); Numpy function, 51Factorial, 52factorial(); numpy.math function, 52fft(); numpy.fft function, 283fftfreq(); numpy.fft function, 285fftshift(); numpy.fft function, 285Figure, see Plotting, figure objectsfigure(); pyplot function, 135, 154File extensions

.c, 308

.ipynb, 12, 316

.npy, 103

.npz, 104

.py, 10

.pyw, 11

.pyx, 308

.so, 308filedialog; tkinter submodule, 69Files

closing, 68delimited files, 73–75

csv module, 73–75csv.reader object, 74csv.writer object, 74reading, 73–74, 235–236writing, 74, 234–235

Excelreading, 237

INDEX 337

writing, 236–237input/output, 67–75interactive selection, 68–70internet access, 76–77Numpy array files, 103–104opening, 67reading text, 70–71reading to list, 71reading to Numpy arrays, 102–103reading to Pandas DataFrame, 234–

237writing text, 71–72

finally; Python statement, 66find(); string method, 39findall()

regular expression method, 267re function, 257

Finite value, 52flatten(); Numpy array method, 90, 172float(); Python function, 20float_(); Numpy function, 80float_power()

Numpy function, 49Floating-point data type, see Data types,

floating-pointfloor(); Numpy function, 51flush()

file method, 73file-like object method, 73sys.stdout method, 64

Flushing of buffer, 64, 73fmod(); Numpy function, 51Fonts, 144–146for; Python statement, 59format(); string method, 35–36, 215Fortran, see F2PYFourier transforms, 277–287

discrete (DFT), 278–287frequencies, 278, 284, 285leakage, 282noise, 282numpy.fft module, 283–287Nyquist frequency, 284, 285power spectrum, 281, 287–289scipy.fftpack module, 283wavenumbers, 278, 284, 285

fromfile(); Numpy function, 102Functions, 15, 107–115

arguments, 109passing by reference, 111–112passing by value, 111–112

bound, 109, 112defining, 107documentation string, 117inspecting, 113keywords, 108lambda operator, 113open-ended argument lists, 114–115Querying, 113return statement, 107, 108variables and scoping, 109vs. methods, 21, 137

Gamma function, 52gamma(); numpy.math function, 52gca(); pyplot function, 138gcf(); pyplot function, 138genfromtxt(); Numpy function, 103get_text(); axis label method, 149get_xticklabels(); Axes method, 150get_xticks(); Axes method, 150get_yticklabels(); Axes method, 150get_yticks(); Axes method, 150getsource(); inspect module function, 113Global variable, see Variables, global vs.

localglobal; Python statement, 110grid()


gridlines(); GeoAxes method, 195

h5py module, 251hamming(); Numpy function, 282hanning(); Numpy function, 282Hashes, see DictionariesHDF files, 250–255

exploring contents, 251–254h5py module, 251opening and reading, 251–255

head()Pandas DataFrame method, 240Pandas Series method, 240

hist()Axes method, 164pyplot function, 164

Histogram plots, see Plotting, histogramshlines()


hsplit(); Numpy function, 96hstack(); Numpy function, 94

338 INDEX

Hyperbolic functions, 50–51

if; Python statement, 56ifft(); numpy.fft function, 284ifftshift(); numpy.fft function, 285iloc; Pandas Series attribute, 222imag; complex attribute, 20ImageMagick software, 157import; Python statement, see Modules,

importingimshow()

Axes method, 180–183pyplot function, 180–183

in; Python statement, 29index()

list method, 26string method, 39

index; Pandas Series attribute, 222Infinite value, 52info()


Inheritance, see Classes, inheritance__init__ method, 122, 125Input from terminal, 55–56

verifying user input with while loop,61–62

input(); Python function, 55insert()

list method, 26Numpy function, 94Pandas DataFrame method, 233

inspect; Python module, 22, 113Inspecting

functions, see Functions, inspectinglambdas, see Functions, inspectingobjects, see Objects, inspecting

Instance methods, see Methods, instanceInstantiation, see Objects, instantiationint(); Python function, 20int_(); Numpy function, 80Integer data type, see Data types, integerIntegration

continuous functions, 298discretely sampled functions, 295

@interact; decorator, 324interact(); ipywidgets function, 322–326Internet file access, 76–77interp1d(); scipy.interpolate function, 291Interpolation, 291–292inv(); numpy.linalg function, 270

.ipynb file extension, see File extensions,.ipynb

IPython, 9, 10IPython notebooks, see Jupyter notebooksipywidgets module, 322irfft(); numpy.fft function, 286isalnum(); string method, 38isalpha(); string method, 38isdigit(); string method, 38isdir(); os.path function, 76isfile(); os.path function, 76isfinite(); Numpy function, 52isinf(); Numpy function, 52islower(); string method, 38isnan(); Numpy function, 52isneginf(); Numpy function, 52isposinf(); Numpy function, 52isspace(); string method, 38istitle(); string method, 38isupper(); string method, 38items(); dictionary method, 29__iter__(); iterable object method, 58iter(); Python function, 59Iterable object, 58Iteration, 58–59Iterator, 58

@jit; decorator, 306join(); string method, 37Jupyter Lab; application for running Jupyter

notebooks, 12, 315Jupyter Notebook; application for run-

ning Jupyter notebooks, 315Jupyter notebooks, 12, 315–326

change default directory, 318equations, 321image embedding, 321keyboard shortcuts, 318, 319LATEX, 321Markdown language, 320–321math symbols, 321plots, 321–322, 325–326URL embedding, 321widgets, 322–326

check box, 324dropdown menu, 325Jupyter Lab issue, 326slider, 323text box, 325

Jython, 9

keys(); dictionary method, 29

INDEX 339

Keywords, see Functions, keywords

lambda; Python statement, see Functions,lambda operator

LATEX, 146, 321, 327–331degree symbol, 329escape special characters, 328force whitespace, 330Greek letters, 327subscripts, 328superscripts, 328

legend()Axes method, 155pyplot function, 155

len(); Python function, 26, 29, 38, 90lgamma(); Numpy function, 52linalg

numpy module, 269scipy module, 269

Line2D(); matplotlib.lines function, 151Line3D(); mpl_toolkits.mplot3d.art3d func-

tion, 205Linear regression, 293–294lines; matplotlib submodule, 151linregress(); scipy.stats function, 293linspace(); Numpy function, 84List comprehension, 28, 62–63list(); Python function, 26, 27Lists and tuples, 23–28

accessing elements, 23–25adding elements to lists, 26functions and methods, 26–27list comprehension, 28, 62–63mutable vs. immutable, 23removing elements from lists, 27reversing lists, 27sorting lists, 27zippering, 27

ljust(); string method, 40load(); Numpy function, 103loadtxt(); Numpy function, 102loc; Pandas DataFrame attribute, 229, 233Local variable, see Variables, global vs. lo-

callog(); Numpy function, 51log10(); Numpy function, 51log1p(); Numpy function, 51Logarithmic functions, 51loglog()


logspace(); Numpy function, 84long(); Python function, 20Long-integer data type, see Data types,

long-integerLoops, 59–62

break statement, 61continue statement, 61enumerate, 60for loop, 59–60implicit loops with arrays, 88iterable objects and, 59iterators and, 59list comprehension, 28, 62–63pass statement, 62skipping to top of, 61verifying user input, 61–62while loop, 60–61

lower(); string method, 40lstrip(); string method, 39lu_factor(); scipy.linalg function, 273lu_solve(); scipy.linalg function, 273

Magic commands, 322Maps, see CartopyMarkdown language, 320–321Masked arrays, see Arrays, masked arraysmath module, 48

compared to numpy module, 48Mathematical operators, 43–53

addition +, 43augmented assignment, 44Boolean operators, 44comparison operators, 45–46division /, 43modulo %, 44multiplication *, 43power **, 43subtraction �, 43truncating division //, 43

%matplotlib; magic command, 322Matplotlib module, see PlottingMatrices, 269–270

inverse, 270multiplication, 269–270

max()Numpy array method, 89Numpy function, 89Pandas Resampler method, 239Pandas Rolling method, 227Pandas Series method, 224Python function, 46

340 INDEX

mean()Numpy array method, 89Numpy function, 89Pandas Resampler method, 239Pandas Rolling method, 226, 227Pandas Series method, 224

median()Numpy function, 89Pandas Resampler method, 239Pandas Rolling method, 227Pandas Series method, 224

memmap(); Numpy function, 105meshgrid(); Numpy function, 97Methods, see also Functions

bound, 22class, 127–129defining, see Functions, definingdocumentation string, 117__init__ method, 122, 125instance, 122, 125introduced, 21–22static, 126–127types of, 125–129vs. functions, 21, 137

min()Numpy array method, 89Numpy function, 89Pandas Resampler method, 239Pandas Rolling method, 227Pandas Series method, 224Python function, 46

modf(); Numpy function, 51Modules, 115–117

aliasing, 116documentation string, 117importing, 115–117

aliasing, 53avoiding *, 53

path to, 116Modulo arithmetic, 44, 51Moving average, see Smoothing of dataMoving window, see Pandas, rolling win-

dowMPI, see Parallel processing, MPImpi4py module, 310multiprocessing module, see Parallel pro-

cessing, multiprocessing mod-ule

\n, see Strings, special characters, newline

__name__; attribute, 113Namespace, 53NaN, see Not a Numbernan; Numpy constant, 52nanmean(); Numpy function, 90nanstd(); Numpy function, 90nanvar(); Numpy function, 90NetCDF files, 241–250

exploring, 242–244netCDF4 module, 242opening, 242writing, 247–250

New-line character, \n, see Strings, spe-cial characters, new line

__next__()csv.reader method, 74file method, 70iterator method, 58

next(); iterator method (Python 2), 59None data type, see Data types, Nonenormal(); numpy.random function, 100Not a Number (NaN), 52

assigning value, 52ignoring, 90

now(); datetime function, 210.npy file extension, see File extensions, .npy.npz file extension, see File extensions, .npzNull, seeNumba, 306–307Numerical Python, see numpy moduleNumpy

vectorizing operations, 306numpy module, 48

arrays, see Arrayscompared to math module, 48data types, 80–81numpy.fft module, 283–287numpy.ma module, see Arrays, masked

arraysnumpy.random module, see Random

numbers

Objects, 20–22, 121–129attributes, 21defining, 121documentation string, 22, 122inspecting, 22instantiation, 122methods, 21properties, 129

ones(); Numpy function, 82

INDEX 341

ones_like(); Numpy function, 82open(); Python function, 67Optimization of code, 305–313os.path module, 76

Package managerConda, 7–8pip, 8

Pandas, 219–240DataFrame, 228–233

creating, 228data access, 229–230sorting, 231

index name, 220–221moving window, see Pandas, rolling

windowplotting data, 224–225, 232read from file, 234–237

CSV, 235–236Excel, 237

resampling, 238rolling window, 226–227, 232Series, 220–227

creating, 220data access, 221–222sorting, 223

statistics, 224, 231–232write to file, 234–237

CSV, 234–235Excel, 236–237

Parallel processing, 308–310MPI, 310multiprocessing module, 308–310

parse(); dateparser function, 216pass; Python statement, 62path, see sys.pathpcolor()


pcolormesh()Axes method, 179pyplot function, 179

perf_counter(); time module function, 310Physical constants, 300–301

CODATA database, 300scipy.constants module, 300

Physical dimensions and units, 301–305pi; Numpy constant, 48pickle module, 299Pie charts, see Plotting, pie chartspie()


pint module, 304pip; package manager, 8Plate Carree; map projection, 192plot()

Axes method, 136, 137Axes method vs. pyplot function,

137Axes3D method, 201Pandas DataFrame method, 232Pandas Series method, 225pyplot function, 133, 137

plot_surface(); Axes3D method, 203plot_wireframe(); Axes3D method, 204Plotting

1-D plots, 133–1662-D plots, 173–190

column vs. row ordering, 173contour plots, see Plotting, contour

plotsstreamlines, 190vector plots, 188–189wind barbs, 189

3-D plots, 201–206animation, 157–158aspect ratio, 174–176axes objects, 135

duplicate, see Plotting, duplicateaxes

obtaining reference, 138setting current axes, 153

axislabels, 143, 202limits, 147, 202reversing, 147tickmarks, see Plotting, tick marks

background color, 142bar charts, 161–163clearing figures and axes, 138color bars, 183–185

labeling, 183multipanel plots, 184–185orientation, 184

colormaps, 186–187reversing, 186

contour plots, 173–178colors, 176contour set, 177, 183filled contours, 178labels, 177–178

342 INDEX

line style, 177line width, 177spacing, 176values, 176

coordinatesaxes, 143data, 143

drawing lines, 150–152, 205–206arbitrary lines, 151grid lines, 150horizontal lines, 150–151vertical lines, 150–151

duplicate axes, 152–153figure objects, 135

obtaining reference, 138setting current figure, 154

fonts, 144–146grid lines, 150histograms, 164–165images, 179–183legends, 155–156line properties, 139–142

color, 140shortcuts, 142style, 140thickness, 141

logarithmic plots, 142markers, 141–142multiple axes, 153–155multiple figures, 153–155multiple panels, 167–172

adjusting spacing and position, 170–171

looping over, 172sharing axes, 171titles, 172using add_axes() method, 167using subplot() function, 167using subplots() function, 170

outside of boundaries, 165pie charts, 163–164polar plots, 159–161pseudocolor plots, 179–180pyplot; matplotlib submodule, 133saving plots as images, 157, 203scatter plots, 158–159size of figure, 157streamlines, 190text, 143–146, 206

alignment, 146color, 145

fonts, 144–146Greek letters, 146, 327location, 144math symbols, 146, 328–331orientation, 146spacing, 146style, 146

tick marks, 148–150, 202titles, 143, 172vector plots, 188–189wind barbs, 189

Pointer, see Functions, arguments, pass-ing by reference

polar(); pyplot function, 159Polar-coordinate plots, see Plotting, polar

plotspow(); Python function, 46, 49power()

Numpy function, 49numpy.lib.scimath function, 50

Powers, 48–50prefix, see sys.prefixprint(); Python function, 15

options for, 63–64use in writing to files, 72

process_time(); time module function, 310@property; decorator, 129Pseudocolor plots, see Plotting, pseudo-

color plots.py file extension, see File extensions, .py.pyw file extension, see File extensions, .pyw.pyx file extension, see File extensions, .pyxpylab module, 166pyplot, see Plotting, pyplotPyPy, 9Python 3 vs Python 2, 4, 15, 19, 20, 26, 27,

36, 42, 43, 47, 55, 59, 68, 73python; executable file, 10pythonw; executable file, 11pytz module, 212

quad(); scipy.integrate function, 298quiver()


\r, see Strings, special characters, returnradians(); Numpy function, 50randint(); numpy.random function, 99random

Numpy module, 99–100

INDEX 343

Python module, 99Random numbers, 99–100random_sample(); numpy.random func-

tion, 99range(); Python function, 25

use in for loops, 60raw_input(); Python 2 function, 55rcParams; matplotlib dictionary object,

145, 146re module, see Regular expressions, re

modulere.I; regular expression option flag, 266,

268read_csv(); Pandas function, 235read_excel(); Pandas function, 237readline()

file method, 70, 71HTTPResponse method, 77

readlines()file method, 70, 71HTTPResponse method, 77

real; complex attribute, 20Recursion, see RecursionRegular expressions, 257–268

making case-insensitive, 266matching sets of characters, 263Pattern objects, 266–268re module, 257

Relational operators, see Mathematical op-erators, comparison operators

remove()list method, 27set method, 30

replace(); string method, 39resample(); Pandas DataFrame method,

238Resampling, 292–293

temporal (with Pandas), 238reshape(); Numpy function, 90Return character, \r, see Strings, special

characters, returnreturn; Python statement, 107, 108reverse(); list method, 27Reversing axis, see Plotting, axis, revers-

ingrfft(); numpy.fft function, 286rfftfreq(); numpy.fft function, 286rfind(); string method, 39rindex(); string method, 39rjust(); string method, 40roll(); Numpy function, 90

Roots, 48–50rot90(); Numpy function, 91round(); Python function, 20, 46–47Rounding functions, 20, 46, 51rstrip(); string method, 39

save(); Numpy function, 103savefig(); Figure method, 157, 203savetxt(); Numpy function, 101savez(); Numpy function, 104Saving and loading arbitrary Python ob-

jects, 299sca(); pyplot function, 153Scatter plots, see Plotting, scatter plotsscatter()


Scientific Python, see scipy modulescipy module

scipy.constants, 300scipy.fftpack module, 283scipy.integrate module, 298scipy.interpolate module, 291scipy.io.netcdf module, 242scipy.misc module, 296scipy.special module, 305scipy.stats module, 293

scipy.integrate function, 296self; reference to object, 122, 125, 126semilogx()


semilogy()Axes method, 142pyplot function, 142

set_aspect(); Axes method, 159, 174set_extent(); GeoAxes method, 193set_facecolor(); Axes method, 142set_global(); GeoAxes method, 195set_label(); color bar method, 183set_size_inches(); Figure method, 157set_theta_direction(); Axes method, 160set_theta_offset(); Axes method, 160set_title(); Axes method, 143set_xlabel()


set_xlim()Axes method, 147Axes3D method, 203

set_xticklabels()

344 INDEX


set_xticks()Axes method, 148Axes3D method, 203

set_ylabel()Axes method, 143Axes3D method, 202

set_ylim()Axes method, 147Axes3D method, 203

set_yticklabels()Axes method, 148Axes3D method, 203

set_yticks()Axes method, 148Axes3D method, 203

set_zlabel(); Axes3D method, 202set_zlim(); Axes3D method, 203set_zticklabels(); Axes3D method, 203set_zticks(); Axes3D method, 203Sets, 29–30shape(); Numpy function, 90shift(); Pandas Series method, 227show(); pyplot function, 134sin(); Numpy function, 50sinh(); Numpy function, 50size(); Numpy function, 90sleep(); datetime function, 218Smoothing of data, 226–227.so file extension, see File extensions, .sosolve(); np.linalg function, 272solve_banded(); scipy.linalg function, 275sort()

list method, 27Numpy array method, 91Numpy function, 91

sort_index()Pandas DataFrame method, 231Pandas Series method, 223

sort_values()Pandas DataFrame method, 231Pandas Series method, 223

SortingNumpy arrays, 91–92Pandas DataFrame, 231Pandas Series, 223

Special mathematical functions, 305Spectral analysis, see Fourier transformssplit()

Numpy function, 96

os.path function, 76re function, 266regular expression method, 267string method, 40

splitdrive(); os.path function, 76splitext(); os.path function, 76sqrt()

Numpy function, 48numpy.lib.scimath function, 49

Square root, 48–49squeeze()

Numpy array method, 93Numpy function, 93

Standard deviation, 89, 224Standard output, 15startswith(); string method, 39Static methods, see Methods, static@staticmethod; decorator, 126Statistics, see Arrays, statistics or Pandas,

statisticsstd()

Numpy array method, 89Numpy function, 89Pandas Resampler method, 239Pandas Rolling method, 227Pandas Series method, 224

stdout, see sys.stdoutStopIteration error, 58streamplot()


strftime(); datetime method, 214Strings, 22, 33–42

byte strings, 41–42case conversion, 40concatenation, 37editing, 39–41formatting, 35–36

format() method, 35indexing, 36iterating, 36justifying, 39mixed quotes, 33multiplication, 38pattern matching, see Regular expres-

sionsraw, 34regular expressions, see Regular ex-

pressionsreplacing text, 39searching, 39

INDEX 345

special charactersescaping, 34new line, \n, 33, 63return, \r, 33tab, \t, 33

splitting, 40, 266, 267stripping special characters, 39stripping whitespace, 39triple-quoted, 33, 34

strip(); string method, 39strptime(); datetime function, 216sub()

re function, 265regular expression method, 267

subn()re function, 265regular expression method, 267

subplot(); pyplot function, 167subplots(); pyplot function, 170subplots_adjust(); pyplot function, 170sum()

Numpy array method, 89Pandas Resampler method, 239Pandas Rolling method, 227Pandas Series method, 224

suptitle()Figure method, 172pyplot function, 172

swapcase(); string method, 40sys module, 11, 64, 116sys.path, 116sys.prefix, 11sys.stdout, 64Systems of equations, 271–276

\t, see Strings, special characters, tabTab character, \t, see Strings, special char-

acters, tabtail()


tan(); Numpy function, 50tanh(); Numpy function, 50text()

Axes method, 144Axes3D method, 206pyplot function, 144

Time, see Dates and timestime module, see Dates and times, time

moduletime; datetime class, 210

timedelta; datetime class, 210, 217timeit module, 311–312timeit(); timeit function, 311Timing code, 310–312title()

pyplot function, 143string method, 40

Tk(); tkinter class, 69tkinter; Python 3 module, 68to_csv()


to_excel()Pandas DataFrame method, 236Pandas Series method, 236

tofile(); Numpy array method, 101transpose()

Numpy array method, 91, 173Numpy function, 91, 173

Trigonometry functions, 50–51trunc(); Numpy function, 51Truncation functions, 51try; Python statement, 64tuple(); Python function, 26Tuples, see Lists and tuplestwinx()


twiny()Axes method, 152pyplot function, 152

type(); Python function, 30

Undefined value, 52Unpacking

arguments and keywords, 113–114lists and tuples, 32

unyt module, 304update(); set method, 30upper(); string method, 40urllib.request module, 76–77urlopen(); urllib.request function, 77urlretrieve(); urllib.request function, 76UTF-16 encoding, 41UTF-8 encoding, 41

value_counts(); Pandas Series method,240

valuesPandas DataFrame attribute, 230Pandas Series attribute, 222

346 INDEX

values(); dictionary method, 29var()

Numpy array method, 89Numpy function, 89Pandas Resampler method, 239Pandas Rolling method, 227Pandas Series method, 224

Variablesdata types of, see Data typesglobal vs. local, 109naming, 17

reserved words, 17parallel assignment, 17scoping rules, 109

Variance, 89, 224view_init(); Axes3D method, 202vlines()


vsplit(); Numpy function, 96vstack(); Numpy function, 94

where(); Numpy function, 97while; Python statement, 60Widgets, see Jupyter notebook, widgetswith; Python statement, 68, 77withdraw(); tkinter window method, 68,

69write(); file method, 72writelines(); file method, 72writerow(); csv.writer method, 74writerows(); csv.writer method, 74Writing output to terminal, 15

xkcd(); pyplot function, 313xlabel(); pyplot function, 143xlim(); pyplot function, 147, 161xrange(); Python 2 function (deprecated),

26xticks(); pyplot function, 148, 161

ylabel(); pyplot function, 143ylim(); pyplot function, 147, 161yticks(); pyplot function, 148, 161

zeros(); Numpy function, 82zeros_like(); Numpy function, 82zip(); Python function, 27

Documents

Python Programming and Visualization for Scientists