User guide

NumPy User Guide, Release 1.9.0
Functions with keyword arguments
These functions are very similar to functions without keyword arguments. The only difference is that the function
signature is:
static PyObject
*
keyword_cfunc (PyObject
*
dummy, PyObject
*
args, PyObject
*
kwds)
{
...
}
The kwds argument holds a Python dictionary whose keys are the names of the keyword arguments and whose values
are the corresponding keyword-argument values. This dictionary can be processed however you see fit. The easiest
way to handle it, however, is to replace the PyArg_ParseTuple (args, format_string, addresses...) function with
a call to PyArg_ParseTupleAndKeywords (args, kwds, format_string, char *kwlist[], addresses...). The kwlist
parameter to this function is a NULL -terminated array of strings providing the expected keyword arguments. There
should be one string for each entry in the format_string. Using this function will raise a TypeError if invalid keyword
arguments are passed in.
For more help on this function please see section 1.8 (Keyword Paramters for Extension Functions) of the Extending
and Embedding tutorial in the Python documentation.
Reference counting
The biggest difficulty when writing extension modules is reference counting. It is an important reason for the popular-
ity of f2py, weave, Cython, ctypes, etc.... If you mis-handle reference counts you can get problems from memory-leaks
to segmentation faults. The only strategy I know of to handle reference counts correctly is blood, sweat, and tears.
First, you force it into your head that every Python variable has a reference count. Then, you understand exactly
what each function does to the reference count of your objects, so that you can properly use DECREF and INCREF
when you need them. Reference counting can really test the amount of patience and diligence you have towards your
programming craft. Despite the grim depiction, most cases of reference counting are quite straightforward with the
most common difficulty being not using DECREF on objects before exiting early from a routine due to some error. In
second place, is the common error of not owning the reference on an object that is passed to a function or macro that is
going to steal the reference ( e.g. PyTuple_SET_ITEM, and most functions that take PyArray_Descr objects).
Typically you get a new reference to a variable when it is created or is the return value of some function (there are
some prominent exceptions, however — such as getting an item out of a tuple or a dictionary). When you own the
reference, you are responsible to make sure that Py_DECREF (var) is called when the variable is no longer necessary
(and no other function has “stolen” its reference). Also, if you are passing a Python object to a function that will “steal”
the reference, then you need to make sure you own it (or use Py_INCREF to get your own reference). You will also
encounter the notion of borrowing a reference. A function that borrows a reference does not alter the reference count
of the object and does not expect to “hold on “to the reference. It’s just going to use the object temporarily. When you
use PyArg_ParseTuple or PyArg_UnpackTuple you receive a borrowed reference to the objects in the tuple
and should not alter their reference count inside your function. With practice, you can learn to get reference counting
right, but it can be frustrating at first.
One common source of reference-count errors is the Py_BuildValue function. Pay careful attention to the differ-
ence between the ‘N’ format character and the ‘O’ format character. If you create a new object in your subroutine
(such as an output array), and you are passing it back in a tuple of return values, then you should most- likely use
the ‘N’ format character in Py_BuildValue. The ‘O’ character will increase the reference count by one. This will
leave the caller with two reference counts for a brand-new array. When the variable is deleted and the reference count
decremented by one, there will still be that extra reference count, and the array will never be deallocated. You will
have a reference-counting induced memory leak. Using the ‘N’ character will avoid this situation as it will return to
the caller an object (inside the tuple) with a single reference count.
56 Chapter 5. Using Numpy C-API