PythonDoc

Updated March 25, 2005 | April 10, 2002 | Fredrik Lundh

Update 2006-04-06: PythonDoc 2.1 beta 6 adds experimental support for putting documentation markup in docstrings. Use the -s option to tell pythondoc to check the docstrings too.

Update 2005-11-01: Current PythonDoc releases don’t work well under Python 2.4.1, due to a bug in Python’s parser. To work around this, upgrade to 2.4.2, or remove the coding directive from the top of the pythondoc.py source file.

Introduction #

The PythonDoc tool generates API documentation in HTML and other formats, based on descriptions in Python source files. PythonDoc is a Python implementation of the JavaDoc (tm) tool.

PythonDoc 2.0 parses Python source code, looking for special documentation comments and the variables, functions, classes, and methods they’re describing.

By default, PythonDoc generates a set of HTML pages describing all documented components in the source files. PythonDoc can also generate XML files, which you can process in various ways using your favourite XML toolbox.

What’s new in 2.0?

PythonDoc 2.0 is rewritten from scratch (mostly without even looking at the original code), so the obvious answer is “Everything!”. Here’s a more useful list:

• uses source code scanning instead of import/introspection; the new approach is safer and a lot more reliable.
• supports variables and class and instance attributes
• improved XML format, including proper nesting of classes and methods.
• XML format also includes undocumented functions, classes, and methods; this can be used by custom output formatters to list undocumented items, and to generate default documentation also for undocumented items.
• supports internationalized source code (the coding directive)
• includes experimental support for module-level documentation
• supports inline @link and @linkplain tags (in 2.1)
• supports pluggable and configurable output generators (-O, -D)
• nicer standard HTML output format (but you can write your own plugin if the default format doesn’t suit your needs)
• internal regression test suite includes a lot more tests, and many megabytes of marked-up real-life Python code (not all of it written by me).
• designed to run on Python 1.5.2 and newer, and parse code written for Python 1.0 and newer
• the old version was one big hack. this one isn’t.

The pythondoc Command-Line Utility #

The pythondoc utility is a simple command-line tool that you can use to generate HTML documentation for individual modules or an entire package.

The current version writes the output to the current directory, to files named pythondoc-module.html.

You can find sample output here (generated with PythonDoc 2.0 beta 1).

The -x option tells PythonDoc to generate XML descriptions as well as HTML:

$ pythondoc.py -x module.py

The -O option tells PythonDoc to use a custom output formatter. You can use the -D option to pass configuration variables to the formatter:

The default renderer accepts -Dstyle=url to add a style sheet link to the HTML header, or -Dzone to write the document body to a text file (this format still uses HTML markup for the body contents, but other HTML document elements are not included).

The -s option tells PythonDoc to look for documentation markup in docstrings too.

$ pythondoc.py -s module.py

This option was introduced in 2.1b6, and should be considered as experimental.

Documentation Comments #

To generate documentation for a function, class, or method, you must add a special documentation comment before the function or class declaration. The comment and the declaration must have the same indentation.

PythonDoc 2 also supports documentation for module variables and class attributes. Just put the documentation comment before you first assign to the variable.

##
# This comment provides documentation for the following
# function.

def function(argument):
    pass

##
# This comment provides documentation for the following
# variable.

variable = value

The first line must use double comment markers (##), and all following lines must begin with a single comment marker (#). The tool strips off the comment markers, and the space following directly after the comment marker (if any), but all remaining whitespace is preserved.

The documentation comment consists of two parts; the first part is a description of the following class, function, or method. The second part consists of one or more tags. Both parts can be empty.

The function or class declaration must follow immediately after the documentation comment. Empty lines, or lines containing only whitespace, are ignored, but any other line is treated as a syntax error, and forces the tool to ignore the comment.

In PythonDoc 2.0, you can also use documentation comments to describe the the module itself. A module comment looks like an ordinary documentation comment, but it ends with an extra comment marker (##):

##
# This is a module comment.
##

You can have only one module comment in each file, and it must be placed before any other PythonDoc comment.

Descriptions

Descriptions are HTML fragments, and you can use most standard HTML freely. The PythonDoc parser is currently a bit stricter than most browsers:

• P, LI, TR, and TD tags are automatically closed, if necessary. These elements cannot be nested; if you put two P start tags after each other, the parser will automatically insert a P end tag between them.
• Text at the beginning of the comment is automatically wrapped in a P element.
• End tags for IMG and HR are ignored.
• All other tags must be properly nested. Each start tag must be followed by a corresponding end tag.
• Attributes can be quoted or unquoted. The parser automatically adds quotes where necessary.
• Reserved characters must be properly escaped. This includes less-than (<), greater-than (>), and ampersand (&), and also the at-sign (@) at the beginning of a line, if that line does not contain a tag (see below).

Tags

The documentation comment can contain one or more tags. An at-sign (@) at the beginning of the line is used to indicate that a line contains a tag. The text before the first tag is called the description.

##
# This is a simple function.  This is the description.
# @param arg An argument to the function.

The PythonDoc tool currently supports a small subset of the JavaDoc tags, plus a few PythonDoc-specific extensions.

Tags can contain HTML markup, just like descriptions.

The @param tag

The @param tag is used to describe a function argument. It’s always followed by the argument name, and one or more lines of descriptive text.

The argument description can contain HTML markup.

The @keyparam tag

The @keyparam tag (a PythonDoc extension) is similar to @param, but should be used for parameters that should always be given as keyword parameters (e.g. options). Like @param, this tag always followed by the argument name, and one or more lines of descriptive text.

The output generator may place the keyword parameters under a separate heading, or add other kinds of annotations indicating that this is not an ordinary positional argument.

# @keyparam color An optional color value.

The @return tag

The @return tag is used to describe what a function returns. It can include one or more lines of descriptive text.

# @return An integer value.

The @exception tag

The @exception tag is used to describe why the function may raise a given exception. It’s always followed by the exception name, and one or more lines of descriptive text.

For compatibility with JavaDoc, this tag can also be spelled @throws.

# @exception IOError Failed to open or read the file.

The @def tag

The @def tag (a PythonDoc extension) can be used to control how PythonDoc describes function and methods. By default, PythonDoc reads the method name and the argument list from the source code. If you don’t want the documentation to contain what’s in the source code, you can use this tag to override the default behaviour:

# @def decode(text)
# @param text The string to decode.

def decode(s, translate=string.translate, repr=repr):
    ...

Note that the @def tag contains both the function name and the argument list.

The @defreturn tag (experimental)

The @defreturn tag (an experimental PythonDoc extension) can be used to specify a very brief description of a return value (e.g. “string”, “4-tuple”, etc). Some output formatters display this on the same line as the function definition:

# @def decode(text)
# @defreturn string

This may be rendered as “def decode(text) ⇒ string” by an output formatter.

The @see tag

The @see tag is used to add references to an item. This tag is currently not supported by any standard output generator, and the semantics remain a bit unclear.

The @link and @linkplain inline tags

(New in 2.1) The @link and @linkplain tags are used to add inline links to a description.

In the current release, the links must have the following syntax:

    {@link #tag text}
    {@linkplain #tag text}

Here, the tag must be the name of a subject in the current module (use class.method to refer to a method). For @link tags, the text part should contain the name of the subject (if the text is omitted, it defaults to the last part of the tag). For @linkplain, the text part can contain arbitrary HTML text.

The PythonDoc Infoset #

The PythonDoc parser generates an XML infoset based on the contents of each source file. For each function, class, or method, it create a corresponding descriptor element. The element contains ordinary HTML block elements (with can contain any number of valid HTML subelements), plus a number of special elements. Here’s an example:

Internally, this is represented as an Element structure. Output formatters can either operate directly on the structure, convert it to a standard Python DOM object, or write it out as XML and run an external filter (e.g. an XSLT processor) on it.

For more information on the infoset, see The PythonDoc Infoset.

The PythonDoc Module #

PythonDoc 2 can be used both as a command-line utility, and as a Python module. The latter can come in handy in situations where there’s not a direct match between the Python source files, and the programming interface exposed to the user. In such cases, you can use PythonDoc’s parse function to read in the documentation for the actual source files, and manipulate the resulting module descriptions in Python before generating the output files.

For more on this (including examples), see Generating PythonDoc Pages for Virtual Modules.

Copyright © 2002-2006 by Fredrik Lundh