32
Python Extending/Integrating A Real World Example Tips Summary Python where we can, C ++ where we must Source: http://xkcd.com/353/ Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 1/25

Thinking Hybrid - Python/C++ Integration

Embed Size (px)

DESCRIPTION

Talk given at the January 2008 meeting of the New Zealand Python User Group in Auckland. Outline: Talk on integrating native C++ sensibly into Python for ease of use of the code base. Inheriting from C++ classes, overriding functionality, automatically generating the bindings using Py++ and SCons. Code demonstrated in the presentation can be found here: http://www.kloss-familie.de/moin/TalksPresentations

Citation preview

Page 1: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Python where we can,C++ where we must

Source: http://xkcd.com/353/

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 1/25

Page 2: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Thinking Hybrid –

Python/C++ Integration(Python where we can, C++ where we must∗)

Guy K. Kloss

New Zealand Python User Group MeetingAuckland, 30 January 2008

∗ Quote: Alex Martelli, Senior Google Developer

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 2/25

Page 3: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Outline

1 Python

2 Extending/Integrating

3 A Real World Example

4 Tips

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 3/25

Page 4: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Outline

1 Python

2 Extending/Integrating

3 A Real World Example

4 Tips

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 4/25

Page 5: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Why Python?

We all know why . . .Some reasons that are important for us:

Dynamic

High-level data typesEmbeddable/Mixable

Extend Python with componentswritten in C++, Java, CEmbed Python into your applicationand call it from C/C++

Platform independent

50 % less code, 300 % more productive

Automatic memory management

All those utilities, modules, . . .

Many, many more reasons . . .

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 5/25

Page 6: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Why Python?

Source: http://xkcd.com/371/

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 6/25

Page 7: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Outline

1 Python

2 Extending/Integrating

3 A Real World Example

4 Tips

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 7/25

Page 8: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

What if I could . . .

Use this code more effectively . . . ?

[NaSt2D demonstration (native executable)]

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 8/25

Page 9: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Extending/Integration

The Contestants:

The “classic way” . . .Extending and Embedding the Python Interpreter

The “new way” . . .Python ctypes

SWIG

Boost.Python

Others:

SIPPythonizerSILOON(Pyrex)

Extensions also for Java, C#, Fortran, . . . available

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 9/25

Page 10: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Extending/Integration

The Contestants:

The “classic way” . . .Extending and Embedding the Python Interpreter

The “new way” . . .Python ctypes

SWIG

Boost.Python

Others:

SIPPythonizerSILOON(Pyrex)

Extensions also for Java, C#, Fortran, . . . available

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 9/25

Page 11: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Extending/Integration

The Contestants:

The “classic way” . . .Extending and Embedding the Python Interpreter

The “new way” . . .Python ctypes

SWIG

Boost.Python

Others:

SIPPythonizerSILOON(Pyrex)

Extensions also for Java, C#, Fortran, . . . available

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 9/25

Page 12: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Extending/Integration

The Contestants:

The “classic way” . . .Extending and Embedding the Python Interpreter

The “new way” . . .Python ctypes

SWIG

Boost.Python

Others:

SIPPythonizerSILOON(Pyrex)

Extensions also for Java, C#, Fortran, . . . available

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 9/25

Page 13: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Extending/Integration

The Contestants:

The “classic way” . . .Extending and Embedding the Python Interpreter

The “new way” . . .Python ctypes

SWIG

Boost.Python

Others:

SIPPythonizerSILOON(Pyrex)

Extensions also for Java, C#, Fortran, . . . available

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 9/25

Page 14: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Boost.Python

Thinking Hybrid with Boost.Python

Bottom up and . . .

Top down possible

Develop quickly

Resolve bottle necks

Donald Knuth’s:“Premature optimisation is the root of all evil.”or: Don’t Optimise Now!

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 10/25

Page 15: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Boost.Python

Thinking Hybrid with Boost.Python

Bottom up and . . .

Top down possible

Develop quickly

Resolve bottle necks

Donald Knuth’s:“Premature optimisation is the root of all evil.”or: Don’t Optimise Now!

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 10/25

Page 16: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Boost.Python

Thinking Hybrid with Boost.Python

Bottom up and . . .

Top down possible

Develop quickly

Resolve bottle necks

Donald Knuth’s:“Premature optimisation is the root of all evil.”or: Don’t Optimise Now!

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 10/25

Page 17: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Hello World

char const* greet(unsigned x) {static char const* const msgs[] = {"hello", "Boost.Python",

"world!"};if (x > 2) {

throw std::range error("greet: Index out of range.");}return msgs[x];

}

#include <boost/python.hpp>using namespace boost::python;BOOST PYTHON MODULE(hello){

.def("greet", greet, "return one of 3 parts of a greeting");}

And here it is in action:

>>> import hello>>> for x in range (3):... print hello.greet(x)...helloBoost.Pythonworld!

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 11/25

Page 18: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Boost.Python

One of a few libraries that make it easyto integrate C++ and Python code

How does it pull off this trick?

Template meta–programming(i. e. Don’t ask!)

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 12/25

Page 19: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Boost.Python

One of a few libraries that make it easyto integrate C++ and Python code

How does it pull off this trick?

Template meta–programming(i. e. Don’t ask!)

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 12/25

Page 20: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

See it Happen

I’m making it work for you now . . .

[“MyClass” demonstration (MyClass.cpp, MyClass.h, mymodule.cpp)]

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 13/25

Page 21: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Outline

1 Python

2 Extending/Integrating

3 A Real World Example

4 Tips

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 14/25

Page 22: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

A Real World ExampleRe-visiting NaSt2D

Wrapping NaSt2D

Control the code

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 15/25

Page 23: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

NaSt2D in Python

This is what I’m going to show you:

Code to be wrapped

Generated wrapper code

Generator script

SCons (build system)

How it works with Python

[Wrapped NaSt2D demonstration (Wrapper.h, nast2dmodule.cpp,

generate bindings.py, SConstruct, demo0.py)]

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 16/25

Page 24: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Extend Wrapper Class

Inheriting from C++ classes

Interfacing numerical values

Change functionality

Follow the Computation

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 17/25

Page 25: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Overriding in Python

This is what I’m going to show you:

Overriding a native method in Python

Native method needs to be “virtual”

Live data plotting with GNUplot

[NaSt2D with plotting demonstration (demo1.py, demo2.py)]

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 18/25

Page 26: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Do more Computations

Parameter Study

Change input file

Compute several cases

Plot results automatically

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 19/25

Page 27: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Automating in Python

This is what I’m going to show you:

Using a template input file

Batch–calculating several runs

Plotting results with GNUplot

[NaSt2D with parameter variation demonstration (demo3.py, demo4.py)]

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 20/25

Page 28: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Outline

1 Python

2 Extending/Integrating

3 A Real World Example

4 Tips

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 21/25

Page 29: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Tips

To override C++ methods: make them virtual

C/C++ pit fall

Call by reference/valueSolution: calling policies

Map to “other/sane” languages

Java: JythonC#: IronPythonFortran: PyFort, Py2F(Native to other: SWIG)

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 22/25

Page 30: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Summary

Why is Python good for you?

How can performance bottle necks be resolved?

Advantages of “Thinking Hybrid”

Python–native wrapping using Boost.Python

Automated wrapper generation

SCons build system

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 23/25

Page 31: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Python and the “Need for Speed”

Cuong Do – Software Architect YouTube.com

“Python is fast enough for our siteand allows us to produce maintainable features

in record times,with a minimum of developers.”

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 24/25

Page 32: Thinking Hybrid - Python/C++ Integration

Python Extending/Integrating A Real World Example Tips Summary

Questions?

[email protected] and code available here:

http://www.kloss-familie.de/moin/TalksPresentations

Guy K. Kloss — Thinking Hybrid – Python/C++ Integration 25/25