Passing struct data to and from C using Python

Using the ctypes module in Python it is possible to pass simple Python objects to C very easily. This circumvents writing your own wrappers to map data types. Particularly useful is the numpy.ctypes module for passing numpy arrays. As an example:

test.c

void square_array(int n, double* ina, double* outa)
{
    int i;
    for (i=0; i<n; i++){
        outa[i] = ina[i]*ina[i];
    }
}

compiled into a shared library:

gcc -c test.c
gcc -shared -o test.so test.o

imported and used in Python:

test.py

import ctypes
import numpy

testlib = ctypes.cdll.LoadLibrary('test.so')

n=5
outa = numpy.zeros(n,numpy.float)
ina = numpy.linspace(1.0,200.0,n)

print "initial array",ina
testlib.square_array.restype = None
testlib.square_array(ctypes.c_int(n),
                     numpy.ctypeslib.as_ctypes(ina),
                     numpy.ctypeslib.as_ctypes(outa))
print "final array",outa

will produce the following:

initial array [   1.     50.75  100.5   150.25  200.  ]
final array   [  1.00000000e+00   2.57556250e+03
                 1.01002500e+04   2.25750625e+04 4.00000000e+04]

It is important to note that we are not copying data but simply passing the pointers to the data from Python to C. In C, the data pointed to (in this case numpy arrays) is operated on which means we do not need to pass any data back. This in itself is a powerful tool and can allow for dramatic speed up of pure Python code. It is almost a necessity when using Python to create computationally intensive applications.

But what about more complex structures?

In some cases, you may require to pass more complex data to C such as data associated with a class or a collection of arrays. A simple way of doing this is using C structs. struct objects can be thought of as classes without methods, allowing you to group various types of data. Let's take the above example and use a struct instead of passing pointers individually.

Firstly we will define a C header file type mapping our new struct:

test_struct.h

struct DATA;

typedef struct DATA{
    int n;
    double *ina;
    double *outa;
} DATA;

Although not necessary, creating a new type makes passing the DATA struct easier and the code cleaner. Our C library now uses this header:

test_struct.c

#include "test_struct.h"

void square_array(DATA* data)
{
    int i;
    for (i=0; i<data->n; i++){
        data->outa[i] = data->ina[i]*data->ina[i];
        }
}

As you can see, we now receive in C a pointer to a new struct. We act on the data in the struct using the -> syntax as we are dealing with pointers. The shared library test_struct.so is compiled in the same manner as before, without the need to explicitly point to the local header file.

To declare the struct in Python requires the creation of new class based on the ctypes.Structure object :

test_struct.py

import ctypes
import numpy

testlib = ctypes.cdll.LoadLibrary('test_struct.so')

class Data(ctypes.Structure):
     _fields_ = [("n", ctypes.c_int),
                ("ina", ctypes.POINTER(ctypes.c_double)),
                ("outa", ctypes.POINTER(ctypes.c_double))]

n=5
outa = numpy.zeros(n,numpy.float)
ina = numpy.linspace(1.0,200.0,n)

data = Data(n,
            numpy.ctypeslib.as_ctypes(ina),
            numpy.ctypeslib.as_ctypes(outa))

print "initial array",ina
testlib.square_array.restype = None
testlib.square_array(ctypes.byref(data))
print "final array",outa

The data in the new Data class is declared in the _fields_ list of tuples which contain the field name and data type. The order of the fields when declaring the new Data class must correspond to the field order in the C header file. It is safer to maintain a consistent order in every file.

There you have it, a simple example to create and pass a more complex data structure between Python and C. The above example could be easily developed into a more sophisticated class where the fields and data types are managed more efficiently. Hopefully this helps those looking to create simple and clean interfaces to C.