User guide

NumPy User Guide, Release 1.9.0
agrees.
— Michel de Montaigne
Duct tape is like the force. It has a light side, and a dark side, and
it holds the universe together.
— Carl Zwanzig
Many people like to say that Python is a fantastic glue language. Hopefully, this Chapter will convince you that this is
true. The first adopters of Python for science were typically people who used it to glue together large application codes
running on super-computers. Not only was it much nicer to code in Python than in a shell script or Perl, in addition,
the ability to easily extend Python made it relatively easy to create new classes and types specifically adapted to the
problems being solved. From the interactions of these early contributors, Numeric emerged as an array-like object that
could be used to pass data between these applications.
As Numeric has matured and developed into NumPy, people have been able to write more code directly in NumPy.
Often this code is fast-enough for production use, but there are still times that there is a need to access compiled code.
Either to get that last bit of efficiency out of the algorithm or to make it easier to access widely-available codes written
in C/C++ or Fortran.
This chapter will review many of the tools that are available for the purpose of accessing code written in other compiled
languages. There are many resources available for learning to call other compiled libraries from Python and the
purpose of this Chapter is not to make you an expert. The main goal is to make you aware of some of the possibilities
so that you will know what to “Google” in order to learn more.
The http://www.scipy.org website also contains a great deal of useful information about many of these
tools. For example, there is a nice description of using several of the tools explained in this chapter at
http://www.scipy.org/PerformancePython. This link provides several ways to solve the same problem showing how to
use and connect with compiled code to get the best performance. In the process you can get a taste for several of the
approaches that will be discussed in this chapter.
5.2.1 Calling other compiled libraries from Python
While Python is a great language and a pleasure to code in, its dynamic nature results in overhead that can cause
some code ( i.e. raw computations inside of for loops) to be up 10-100 times slower than equivalent code written in
a static compiled language. In addition, it can cause memory usage to be larger than necessary as temporary arrays
are created and destroyed during computation. For many types of computing needs the extra slow-down and memory
consumption can often not be spared (at least for time- or memory- critical portions of your code). Therefore one of
the most common needs is to call out from Python code to a fast, machine-code routine (e.g. compiled using C/C++ or
Fortran). The fact that this is relatively easy to do is a big reason why Python is such an excellent high-level language
for scientific and engineering programming.
Their are two basic approaches to calling compiled code: writing an extension module that is then imported to Python
using the import command, or calling a shared-library subroutine directly from Python using the ctypes module (in-
cluded in the standard distribution with Python 2.5). The first method is the most common (but with the inclusion of
ctypes into Python 2.5 this status may change).
Warning: Calling C-code from Python can result in Python crashes if you are not careful. None of the approaches
in this chapter are immune. You have to know something about the way data is handled by both NumPy and by the
third-party library being used.
5.2. Using Python as glue 61