How does Python manage memory?
The details of Python memory management depend on the implementation. The standard C implementation of Python uses reference counting to detect inaccessible objects, and a separate mechanism to collect reference cycles, periodically executing a cycle detection algorithm which looks for inaccessible cycles and deletes the objects involved. The gc module provides functions to force garbage collection, obtain debugging statistics, and tune the collector’s parameters.
Jython relies on the Java runtime so the JVM’s garbage collector is used. The same applies to IronPython, which uses the CLR garbage collector. This difference can cause some subtle porting problems if your Python code depends on the behavior of the reference counting implementation.
Sometimes objects get stuck in tracebacks temporarily and hence are not deallocated when you might expect. Clear the tracebacks with:
import sys sys.exc_clear() sys.exc_traceback = sys.last_traceback = None
Tracebacks are used for reporting errors, implementing debuggers and related things. They contain a portion of the program state extracted during the handling of an exception (usually the most recent exception).
In the absence of circularities and tracebacks, Python programs need not explicitly manage memory.
Why doesn’t Python use a more traditional garbage collection scheme? For one thing, this is not a C standard feature and hence it’s not portable. (Yes, we know about the Boehm GC library. It has bits of assembler code for most common platforms, not for all of them, and although it is mostly transparent, it isn’t completely transparent; patches are required to get Python to work with it.)
Traditional GC also becomes a problem when Python is embedded into other applications. While in a standalone Python it’s fine to replace the standard malloc() and free() with versions provided by the GC library, an application embedding Python may want to have its own substitute for malloc() and free(), and may not want Python’s. Right now, Python works with anything that implements malloc() and free() properly.
Note that on systems using traditional GC, code that uses external resources without explicitly releasing them may run out of resources before the GC kicks in. Consider this example:
class Resource: def __init__(self, name): self.handle = allocate_resource(name) def __del__(self): if self.handle: self.close() def close(self): release_resource(self.handle) self.handle = None ... for name in big_list: x = Resource(name) do something with x
In current releases of CPython, each new assignment to x inside the loop will release the previously allocated resource. Using GC, this is not guaranteed. If you want to write code that will work with any Python implementation, you should explicitly close the resource; this will work regardless of GC:
for name in big_list: x = Resource() do something with x x.close()