Idea In Progress: Dynamic Creation of Type Description

In today’s Python, type records are implemented as ordinary struct‘s, using a flag field to indicate what version of the struct a given extension implements.

You can see an example on this page (dead link) (Scroll down to the “static PyTypeObject NoddyType” definition).

I propose to replace this with a dynamic registration model for Python 3000. In this model, a single entry point is used to register all methods and hooks your C extension has implemented, and code in the Python library builds a properly initialized type descriptor based on the registered information, using fallback functions and error stubs as needed.

This approach has several advantages over today’s solution, including:

• Python can fill in all method slots, so that we can eliminate all NULL checks from the code that accesses type members.
• The type object layout can be changed at any time, without breaking binary extensions.
• It’s easier to write extensions: you don’t have to count initializers to add a new method, and you can add a new type without having to copy the method table from somewhere else.

For example, the impossible-to-write-from-scratch NoddyType struct initialization on the page mentioned above would collapse to:

    static PyTypeObject *NoddyType;

    ...

    NoddyType = PyType_Setup("noddy.Noddy", sizeof(Noddy));
    PyType_Register(NoddyType, PY_TP_DEALLOC, Noddy_dealloc);
    PyType_Register(NoddyType, PY_TP_DOC, "Noddy objects");
    PyType_Register(NoddyType, PY_TP_TRAVERSE, Noddy_traverse);
    PyType_Register(NoddyType, PY_TP_CLEAR, Noddy_clear);
    PyType_Register(NoddyType, PY_TP_METHODS, Noddy_methods);
    PyType_Register(NoddyType, PY_TP_MEMBERS, Noddy_members);
    PyType_Register(NoddyType, PY_TP_INIT, Noddy_init);
    PyType_Register(NoddyType, PY_TP_NEW, Noddy_new);
    if (PyType_Ready(&NoddyType) < 0)
        return;

Here, Setup prepares the type object for registration, and Ready is used to complete the setup, and flag the type object as completed. If an error occurs inside Setup, it simply returns NULL, and leaves it to the PyType_Ready function to report the actual error.

Note that in the above example, Ready still takes a pointer to the type variable. This allows the implementation to use a different object to collect type information, if that would simplify the implementation.

Methods and Members

The same approach can be used for method and member descriptions:

    static PyMethodDef Noddy_methods[] = {
        {"name", (PyCFunction)Noddy_name, METH_NOARGS,
         "Return the name, combining the first and last name"
        },
        {NULL}  /* Sentinel */
    };

    PyType_Register(NoddyType, PY_TP_METHODS, Noddy_methods);

becomes

    PyType_RegisterMethod(NoddyType, "name", Noddy_name, METH_NOARGS,
        "Return the name, combining the first and last name");

This allows the implementation to use different ways to store the methods, depending on argument type, list size, etc.