Adding GC Support to an Existing Python Type

October 14, 2002 | Fredrik Lundh

Note: The code in this example doesn’t work properly with the new GC scheme in Python 2.2 and later. For more information, see Neil Schemenauer’s Adding GC Support to a Python Type page. I’ll update this page when I find the time.

This note shows how to add support for garbage collection to Python types written in C or C++.

The approach used in this note allows you to add GC support to an existing type implementation, and still use the same source code with Python versions before 2.0.

The simplified MyObject type used in this note contains two fields that may point to other Python objects, and indirectly back to the object itself. The member1 member is always set to a valid Python object, while the member2 member may be NULL.

The first snippet sets the USE_GC variable if garbage collection is supported by the Python interpreter.

#if PY_VERSION_HEX >= 0x02000000
/* use garbage collection (requires Python 2.0 or later) */
#define USE_GC
#endif

Next, you have to add PyObject_GC_Init and PyObject_GC_Fini calls to the object allocation and deallocation functions. Call Init after you’ve initialized the object, and Fini just before you start releasing the internal pointers.

Also note that if you’re using the NEW/DEL interface, you must use PyObject_AS_GC to get a pointer that you can pass to PyObject_DEL. If you’re using the New/Del interface instead, leave out the AS_GC call.

static PyObject*
my_alloc(PyObject* self, PyObject* args)
{
    MyObject* self;

    ... parse arguments ...

    self = PyObject_NEW(MyObject, &My_Type);
    if (self == NULL)
        return NULL;

    Py_INCREF(Py_None);
    self->member1 = Py_None;

    self->member2 = NULL;

    ... initialize more members ...

#if defined(USE_GC)
    PyObject_GC_Init(self);
#endif

    return (PyObject*) self;
}

static void
my_dealloc(MyObject* self)
{
#if defined(USE_GC)
    PyObject_GC_Fini(self);
#endif

    Py_XDECDEF(self->member1);
    Py_XDECDEF(self->member2);
    ... release more members ...

#if defined(USE_GC)    
    PyObject_DEL(PyObject_AS_GC(self));
#else
    PyObject_DEL(self);
#endif

Next, you have to define two GC helper functions. The traverse function is used by the garbage collector to find all reachable objects. It should call the visit callback on all object members that may point to other Python containers. Python will assume that it’s safe to call the traverse function for an object after you’ve called PyObject_GC_Init on it, and until you call PyObject_GC_Fini.

#if defined(USE_GC)
static int
my_traverse(MyObject *self, visitproc visit, void *arg)
{
    int err;

    err = visit(self->member1, arg);
    if (err)
        return err;

    if (self->member2) {
        /* don't pass NULL to the visit function */
        err = visit(self->member2, arg);
        if (err)
            return err;
    }

    return 0;
}
#endif

The second helper is used to release all object references. The dealloc function will be called at a later time, so you must make sure to mark objects as released, and avoid releasing them again when the object space is reclaimed. Here, we set the members to NULL, and use Py_XDECDEF in the dealloc function:

#if defined(USE_GC)
static int
my_clear(MyObject* self)
{
    Py_DECREF(self->member1);
    self->member1 = NULL;

    Py_XDECREF(self->member2);
    self->member2 = NULL;

    return 0;
}
#endif

Finally, the type descriptor must be modified a bit. The following code patches the type descriptor in place, in the module’s init method:

initmymodule(void)
{
    /* patch object type */
    My_Type.ob_type = &PyType_Type;

#if defined(USE_GC)
    /* enable garbage collection for this type */
    My_Type.tp_basicsize += PyGC_HEAD_SIZE;
    My_Type.tp_flags |= Py_TPFLAGS_GC;
    My_Type.tp_traverse = (traverseproc) my_traverse;
    My_Type.tp_clear = (inquiry) my_clear;
#endif

    Py_InitModule("mymodule", my_functions);
}
 

A Django site. rendered by a django application. hosted by webfaction.