Python's C API: Working with Sequences

Assorted notes on working with lists, tuples, iterators, and other sequences.

Passing Sequences to C Extensions #

November 21, 2003 | Fredrik Lundh

Q. Is there a way to pass a variable-length list with flexible length to a C extension?

Summary: Use a PyObject* variable to hold the list object (use “O” for the type code), and use the sequence API (dead link) to loop over the list contents.

Unless your sequences are really large (too large to hold in memory), the PySequence_Fast API is the most efficient way to extract the items. This API applies tuple() on the source, unless it’s already a tuple or a list, and uses direct access into the object structure to pick out individual items.

Here’s a sample snippet:

...

    PyObject* obj;
    PyObject* seq;
    int i, len;

    if (!PyArg_ParseTuple(args, "O", &obj))
        return NULL;

    seq = PySequence_Fast(obj, "expected a sequence");
    len = PySequence_Size(obj);
    for (i = 0; i < len; i++) {
        item = PySequence_Fast_GET_ITEM(seq, i);
        ...
        /* DON'T DECREF item here */
    }
    Py_DECREF(seq);

...

You can find plenty of examples in the source code for the standard library (grep for PySequence and look for GetItem, GET_ITEM, and Fast_GET_ITEM calls…)

You can also use the PyList API (dead link), but that only works for lists, and is only marginally faster than the PySequence_Fast API.

To squeeze out the last cycles from PySequence_Fast, expand the loop body yourself:

    seq = PySequence_Fast(obj, "expected a sequence");
    len = PySequence_Size(obj);
    if (PyList_Check(seq))
        for (i = 0; i < len; i++) {
            item = PyList_GET_ITEM(seq, i);
            ...
        }
    else
        for (i = 0; i < len; i++) {
            item = PyTuple_GET_ITEM(seq, i);
            ...
        }
    Py_DECREF(seq);

This design is as fast as a list-only implementation, but still supports tuples (fast) and other sequence objects, including arbitrary iterable objects (slower).

Here is a complete example: Help creating extension for C function