The __builtin__ module

This module contains built-in functions which are automatically available in all Python modules. You usually don’t have to import this module; Python does that for you when necessary.

Calling a function with arguments from a tuple or dictionary #

Python allows you to build function argument lists on the fly. Just put all the arguments in a tuple, and call the built-in apply function:

# File: builtin-apply-example-1.py

def function(a, b):
    print a, b

apply(function, ("whither", "canada?"))
apply(function, (1, 2 + 3))

whither canada?
1 5

To pass keyword arguments to a function, you can use a dictionary as the third argument to apply:

# File: builtin-apply-example-2.py

def function(a, b):
    print a, b

apply(function, ("crunchy", "frog"))
apply(function, ("crunchy",), {"b": "frog"})
apply(function, (), {"a": "crunchy", "b": "frog"})

crunchy frog
crunchy frog
crunchy frog

One common use for apply is to pass constructor arguments from a subclass on to the base class, especially if the constructor takes a lot of arguments.

Python 2.0 provides an alternate syntax. Instead of apply, you can use an ordinary function call, and use * to mark the tuple, and ** to mark the dictionary.

The following two statements are equivalent:

result = function(*args, **kwargs)
result = apply(function, args, kwargs)

Loading and reloading modules #

If you’ve written a Python program larger than just a few lines, you know that the import statement is used to import external modules (you can also use the from-import version). What you might not know already is that import delegates the actual work to a built-in function called __import__.

The trick is that you can actually call this function directly. This can be handy if you have the module name in a string variable, like in the following example, which imports all modules whose names end with “-plugin”:

Note that the plugin modules have hyphens in the name. This means that you cannot import such a module using the ordinary import command, since you cannot have hyphens in Python identifiers.

Here’s the plugin used in this example:

# File: example-plugin.py

def hello():
    print "example-plugin says hello"

The following example shows how to get a function object, given that you have the module and function name as strings:

# File: builtin-import-example-2.py

def getfunctionbyname(module_name, function_name):
    module = __import__(module_name)
    return getattr(module, function_name)

print repr(getfunctionbyname("dumbdbm", "open"))

<function open at 794fa0>

You can also use this function to implement lazy loading of modules. In the following example, the string module is imported when it is first used:

# File: builtin-import-example-3.py

class LazyImport:
    def __init__(self, module_name):
        self.module_name = module_name
        self.module = None
    def __getattr__(self, name):
        if self.module is None:
            self.module = __import__(self.module_name)
        return getattr(self.module, name)

string = LazyImport("string")

print string.lowercase

abcdefghijklmnopqrstuvwxyz

Python provides some basic support for reloading modules that you’ve already imported. The following example loads the hello.py file three times:

# File: builtin-reload-example-1.py

import hello
reload(hello)
reload(hello)

hello again, and welcome to the show
hello again, and welcome to the show
hello again, and welcome to the show

reload uses the module name associated with the module object, not the variable name. This means that even if you’ve renamed the module, reload will still be able to find the original module.

Note that when you reload a module, it is recompiled, and the new module replaces the old one in the module dictionary. However, if you have created instances of classes defined in that module, those instances will still use the old implementation.

Likewise, if you’ve used from-import to create references to module members in other modules, those references will not be updated.

Looking in namespaces #

The dir function returns a list of all members of a given module, class, instance, or other type. It’s probably most useful when you’re working with an interactive Python interpreter, but can also come in handy in other situations.

In the following example, the getmember function returns all class-level attributes and methods defined by a given class:

Note that the getmembers function returns an ordered list. The earlier a name appears in the list, the higher up in the class hierarchy it’s defined. If order doesn’t matter, you can use a dictionary to collect the names instead of a list.

The vars function is similar, but it returns a dictionary containing the current value for each member. If you use it without an argument, it returns a dictionary containing what’s visible in the current local namespace:

Checking an object’s type #

Python is a dynamically typed language, which means that a given variable can be bound to values of different types at different occasions. In the following example, the same function is called with an integer, a floating point value, and a string:

def function(value):
    print value

function(1)
function(1.0)
function("one")

The type function allows you to check what type a variable has. This function returns a type descriptor, which is a unique object for each type provided by the Python interpreter.

# File: builtin-type-example-1.py

def dump(value):
    print type(value), value

dump(1)
dump(1.0)
dump("one")

<type 'int'> 1
<type 'float'> 1.0
<type 'string'> one

Each type has a single corresponding type object, which means that you can use the is operator (object identity) to do type testing:

# File: builtin-type-example-2.py

def load(file):
    if isinstance(file, type("")):
        file = open(file, "rb")
    return file.read()

print len(load("samples/sample.jpg")), "bytes"
print len(load(open("samples/sample.jpg", "rb"))), "bytes"

4672 bytes
4672 bytes

The callable function checks if an object can be called (either directly or via apply). It returns true for functions, methods, lambda expressions, classes, and class instances which define the __call__ method.

# File: builtin-callable-example-1.py

def dump(function):
    if callable(function):
        print function, "is callable"
    else:
        print function, "is *not* callable"

class A:
    def method(self, value):
        return value

class B(A):
    def __call__(self, value):
        return value

a = A()
b = B()

dump(0) # simple objects
dump("string")
dump(callable)
dump(dump) # function

dump(A) # classes
dump(B)
dump(B.method)

dump(a) # instances
dump(b)
dump(b.method)

0 is *not* callable
string is *not* callable
<built-in function callable> is callable
<function dump at 8ca320> is callable
A is callable
B is callable
<unbound method A.method> is callable
<A instance at 8caa10> is *not* callable
<B instance at 8cab00> is callable
<method A.method of B instance at 8cab00> is callable

Note that the class objects (A and B) are both callable; if you call them, they create new objects. However, instances of class A are not callable, since that class doesn’t have a __call__ method.

You’ll find functions to check if an object is of any of the built-in number, sequence, or dictionary types in the operator module. However, since it’s easy to create a class that implements e.g. the basic sequence methods, it’s usually a bad idea to use explicit type testing on such objects.

Things get even more complicated when it comes to classes and instances. Python doesn’t treat classes as types per se. Instead, all classes belong to a special class type, and all class instances belong to a special instance type.

This means that you cannot use type to test if an instance belongs to a given class; all instances have the same type! To solve this, you can use the isinstance function, which checks if an object is an instance of a given class (or of a subclass to it).

# File: builtin-isinstance-example-1.py

class A:
    pass

class B:
    pass

class C(A):
    pass

class D(A, B):
    pass

def dump(object):
    print object, "=>",
    if isinstance(object, A):
        print "A",
    if isinstance(object, B):
        print "B",
    if isinstance(object, C):
        print "C",
    if isinstance(object, D):
        print "D",
    print

a = A()
b = B()
c = C()
d = D()

dump(a)
dump(b)
dump(c)
dump(d)
dump(0)
dump("string")

<A instance at 8ca6d0> => A
<B instance at 8ca750> => B
<C instance at 8ca780> => A C
<D instance at 8ca7b0> => A B D
0 =>
string =>

The issubclass is similar, but checks whether a class object is the same as a given class, or is a subclass of it.

Note that while isinstance accepts any kind of object, the issubclass function raises a TypeError exception if you use it on something that is not a class object.

# File: builtin-issubclass-example-1.py

class A:
    pass

class B:
    pass

class C(A):
    pass

class D(A, B):
    pass

def dump(object):
    print object, "=>",
    if issubclass(object, A):
        print "A",
    if issubclass(object, B):
        print "B",
    if issubclass(object, C):
        print "C",
    if issubclass(object, D):
        print "D",
    print

dump(A)
dump(B)
dump(C)
dump(D)
dump(0)
dump("string")

A => A
B => B
C => A C
D => A B D
0 =>
Traceback (innermost last):
  File "builtin-issubclass-example-1.py", line 29, in ?
  File "builtin-issubclass-example-1.py", line 15, in dump
TypeError: arguments must be classes

Evaluating Python expressions #

Python provides several ways to interact with the interpreter from within a program. For example, the eval function evaluates a string as if it were a Python expression. You can pass it a literal, simple expressions, or even use built-in functions:

# File: builtin-eval-example-1.py

def dump(expression):
    result = eval(expression)
    print expression, "=>", result, type(result)

dump("1")
dump("1.0")
dump("'string'")
dump("1.0 + 2.0")
dump("'*' * 10")
dump("len('world')")

1 => 1 <type 'int'>
1.0 => 1.0 <type 'float'>
'string' => string <type 'string'>
1.0 + 2.0 => 3.0 <type 'float'>
'*' * 10 => ********** <type 'string'>
len('world') => 5 <type 'int'>

A problem with eval is that if you cannot trust the source from which you got the string, you may get into trouble. For example, someone might use the built-in __import__ function to load the os module, and then remove files on your disk:

# File: builtin-eval-example-2.py

print eval("__import__('os').getcwd()")
print eval("__import__('os').remove('file')")

/home/fredrik/librarybook
Traceback (innermost last):
 File "builtin-eval-example-2", line 2, in ?
 File "<string>", line 0, in ?
os.error: (2, 'No such file or directory')

Note that you get an os.error exception, which means that Python actually tried to remove the file!

Luckily, there’s a way around this problem. You can pass a second argument to eval, which should contain a dictionary defining the namespace in which the expression is evaluated. Let’s pass in an empty namespace:

>>> print eval("__import__('os').remove('file')", {})
Traceback (innermost last):
  File "<stdin>", line 1, in ?
  File "<string>", line 0, in ?
os.error: (2, 'No such file or directory')

Hmm. We still end up with an os.error exception.

The reason for this is that Python looks in the dictionary before it evaluates the code, and if it doesn’t find a variable named __builtins__ in there (note the plural form), it adds one:

If you print the contents of the namespace variable, you’ll find that it contains the full set of built-in functions.

The solution to this little dilemma isn’t far away: since Python doesn’t add this item if it is already there, you just have to add a dummy item called __builtins__ to the namespace before calling eval:

Note that this doesn’t protect you from CPU or memory resource attacks (for example, something like eval(“’*’*1000000*2*2*2*2*2*2*2*2*2”) will most likely cause your program to run out of memory after a while). In other words, eval may be useful for trusted sources, but you shouldn’t use it on arbitrary data, and especially not if you get data from potentially malevolent sources.

Compiling and executing code #

The eval function only works for simple expressions. To handle larger blocks of code, use the compile and exec functions:

# File: builtin-compile-example-1.py

NAME = "script.py"

BODY = """
prnt 'owl-stretching time'
"""

try:
    compile(BODY, NAME, "exec")
except SyntaxError, v:
    print "syntax error:", v, "in", NAME

syntax error: invalid syntax in script.py

When successful, the compile function returns a code object, which you can execute with the exec statement:

# File: builtin-compile-example-2.py

BODY = """
print 'the ant, an introduction'
"""

code = compile(BODY, "<script>", "exec")

print code

exec code

<code object ? at 8c6be0, file "<script>", line 0>
the ant, an introduction

To generate code on the fly, you can use the class shown in the following example. Use the write method to add statements, and indent and dedent to add structure, and this class takes care of the rest.

Python also provides a function called execfile. It’s simply a shortcut for loading code from a file, compiling it, and executing it. The following example shows how to use and emulate this function.

The hello.py file used in this example has the following contents:

# File: hello.py

print "hello again, and welcome to the show"

Overloading functions from the __builtin__ module

Since Python looks among the built-in functions after it has checked the local and module namespace, there may be situations when you need to explicitly refer to the __builtin__ module. For example, the following script overloads the open function with a version that opens an ordinary file and checks that it starts with a “magic” string. To be able to use the original open function, it explicitly refers to it using the module name.

# File: builtin-open-example-1.py

def open(filename, mode="rb"):
    import __builtin__
    file = __builtin__.open(filename, mode)
    if file.read(5) not in("GIF87", "GIF89"):
        raise IOError, "not a GIF file"
    file.seek(0)
    return file

fp = open("samples/sample.gif")
print len(fp.read()), "bytes"

fp = open("samples/sample.jpg")
print len(fp.read()), "bytes"

3565 bytes
Traceback (innermost last):
  File "builtin-open-example-1.py", line 12, in ?
  File "builtin-open-example-1.py", line 5, in open
IOError: not a GIF file