CWE-401: Missing Release of Memory after Effective Lifetime - Python

Overview

Memory leaks in Python are usually resource leaks rather than memory-management ones: unclosed files, connections and sockets, and unbounded caches. Reference cycles belong here too, but as a delay rather than a leak - CPython reference-counts and runs a cycle detector, so a cycle is collected, just on a later periodic pass instead of at the moment the last reference drops. Anything the cycle holds stays held until then, which is why file handle exhaustion and connection pool depletion are the failures that actually show up.

Primary Defence: Use context managers (with statement) for all resources, implement custom context managers with @contextmanager decorator or __enter__/__exit__ methods, avoid circular references or use weakref to break them, implement bounded caches with TTL or LRU eviction, and explicitly close resources in finally blocks when context managers aren't available.

Common Vulnerable Patterns

Unclosed Files

# VULNERABLE - a file handle that stays reachable after the call that opened it
#
# This first shape is the one scanners flag, and on CPython it is usually not
# the leak: `f` is the only reference to the file object, so its count reaches
# zero at `return` and the handle is closed there.
def read_file(path):
    f = open(path, 'r')
    content = f.read()
    return content

# This is the shape that does leak. Each handle stays reachable from the
# instance, so no count ever reaches zero and nothing closes them.
class LogIndexer:
    def __init__(self):
        self._open_files = []

    def index(self, path):
        f = open(path, 'r')
        self._open_files.append(f)   # kept "to re-read later", never closed
        return sum(1 for _ in f)

indexer = LogIndexer()
for i in range(10000):
    indexer.index(f'data_{i}.txt')
    # Each call adds a descriptor that nothing releases
    # Eventually: OSError: [Errno 24] Too many open files

Why this is vulnerable: CPython reference-counts, so read_file closes its handle at return: measured on CPython 3.13.12, none of its file objects is still alive by the time the caller resumes. A scanner finding on that shape is usually worth checking rather than fixing. LogIndexer is the one that leaks - self._open_files keeps every handle reachable, so no count reaches zero and nothing closes them. Each call takes a descriptor from the process's limited pool (commonly 1024 on Linux, 256 on macOS by default) until open() raises OSError: [Errno 24] Too many open files, and that failure usually surfaces in whatever code next asks for a descriptor rather than here. On Windows an open handle also blocks deleting the file, so cleanup fails with PermissionError while the indexer is alive.

Database Connection Leaks

# VULNERABLE - Database Connection Leaks
import psycopg2

def get_users():
    conn = psycopg2.connect(dbname="mydb", user="user", password="pass")
    cursor = conn.cursor()
    cursor.execute("SELECT * FROM users")
    users = cursor.fetchall()
    return users
    # No cursor.close() or conn.close() - both leaked!

# After 10-20 calls, connection pool exhausted
# New requests block waiting for available connection

Why this is vulnerable: Database connections are backed by network sockets, server-side resources, and connection pool entries. When the connection and cursor aren't closed, they remain allocated in the connection pool (if using one) or hold server resources (if not), preventing reuse. Connection pools typically limit concurrent connections to 10-50. After enough calls without closing, the pool is exhausted and new requests block indefinitely, causing application-wide denial of service. Unlike file handles, database connections hold significant server-side state (transactions, locks, temp tables), so leaked connections waste resources on both client and database server. The garbage collector provides no help here - connection objects may not even have finalizers, and even if they do, finalization is too slow for high-throughput applications.

Unbounded Cache Without Eviction

# VULNERABLE - Unbounded Cache Without Eviction
class DataCache:
    def __init__(self):
        self._cache = {}  # Unbounded dictionary

    def get_data(self, key):
        if key not in self._cache:
            # Expensive operation: fetch from database or API
            data = fetch_from_database(key)
            self._cache[key] = data  # Cache forever
        return self._cache[key]

# Web application caching user data by user ID
cache = DataCache()

# After millions of users access the system
for user_id in range(1, 10_000_000):
    cache.get_data(user_id)
    # Cache grows to 10 million entries - gigabytes of RAM

Why this is vulnerable: The cache dictionary grows without bounds, storing every unique key ever requested; unlike a least-recently-used (LRU) cache, it never evicts anything. In a long-running application, this accumulates potentially millions of entries, consuming gigabytes of memory. After enough time, the application exhausts available RAM and crashes or is killed by the OS (OOM killer on Linux). Even before crashing, the large dictionary degrades performance - Python dictionaries resize when they grow, and each resize rehashes the whole table. The cache also holds outdated data indefinitely: stale user records, expired API responses. This pattern is particularly dangerous in microservices or serverless functions that handle diverse requests - each unique request parameter becomes a cache key, and unique keys grow linearly with traffic volume.

Circular References Delaying Resource Release

# VULNERABLE - the cycle defers cleanup from "now" to "whenever gc next runs"
class Parent:
    def __init__(self, path):
        self.children = []
        self.handle = open(path)        # a real resource, released on __del__

    def add_child(self, child):
        self.children.append(child)

class Child:
    def __init__(self, parent):
        self.parent = parent            # strong back-reference completes the cycle
        parent.add_child(self)

parent = Parent('data.txt')
child = Child(parent)

del parent
del child
# Without the cycle, dropping the last reference would close the handle here.
# With it, both objects survive: parent.children holds child, child.parent
# holds parent, so neither refcount reaches zero and the handle stays open.
#
# They are NOT leaked - CPython's cycle detector collects them, and the handle
# is closed at that point. What is lost is the timing: collection happens on a
# periodic generational pass, not at the del above.

Why this is vulnerable: Reference counting alone cannot free these objects, because each keeps the other's count above zero. CPython's cycle detector does collect them, including - since PEP 442 in Python 3.4 - cycles whose members define __del__, so the older advice that a finalizer makes a cycle permanently uncollectable no longer applies on any supported version. What the detector does not do is run at the moment the last reference is dropped: it runs on a periodic generational pass, so the memory is held until then and the timing is not something the code controls.

That delay is the part that matters, and it is a resource problem rather than a memory one. If the objects in the cycle hold a file handle, a socket or a database connection, those stay open for as long as the cycle does - so a descriptor quota can be exhausted while memory looks healthy, and the failure appears in unrelated code that next asks for a descriptor. Use weakref for the back-reference in tree and parent-child structures so the cycle never forms, and hold any real resource under a context manager rather than relying on the object's own lifetime.

Secure Patterns

Context Managers

# Using built-in context manager
def read_file(path):
    with open(path, 'r') as f:
        return f.read()
    # f.close() called automatically

# Multiple resources
def copy_file(src, dst):
    with open(src, 'r') as source:
        with open(dst, 'w') as dest:
            dest.write(source.read())
    # Both files closed automatically

# Python 3.1+ supports multiple context managers
def copy_file_v2(src, dst):
    with open(src, 'r') as source, open(dst, 'w') as dest:
        dest.write(source.read())

# Database connections - note that `with` on a psycopg2 connection does NOT
# close it. Its __exit__ ends the transaction and leaves the socket open.
import psycopg2
from contextlib import closing

def get_users():
    with closing(psycopg2.connect(dbname="mydb", user="user", password="pass")) as conn:
        with conn:                       # commits, or rolls back on exception
            with conn.cursor() as cursor:
                cursor.execute('SELECT * FROM users')
                return cursor.fetchall()
    # contextlib.closing calls conn.close() here; the inner `with conn`
    # committed the transaction on the way out

Why this works: Python's with statement ensures the context manager's __exit__ method is called when the block exits, regardless of whether it completes normally or via exception. For files, this closes the file handle at the end of the block rather than whenever the file object is finally released. If an exception occurs in the block, __exit__ still runs - with the exception info - before the exception propagates, which covers the error paths where an explicit close() is easiest to forget.

with on a database connection is the exception, not the rule. A psycopg2 connection's __exit__ commits or rolls back the transaction and returns; it never closes the connection. Measured on psycopg2 2.9.12: after with psycopg2.connect(...) as conn: exits, conn.closed is still 0, and the socket stays open until something calls conn.close(). That is deliberate - the DB-API expects a connection to outlive a single unit of work, so the context manager was given to the transaction rather than to the socket - but it reads exactly like a file, and a with block that looks like it releases the resource is worse than no with block at all, because it stops anyone looking. Wrap the connection in contextlib.closing() when the connection is genuinely short-lived, or call close() in a finally. The cursor is not affected: with conn.cursor() as cursor does close the cursor.

The same question is worth asking of any third-party object before trusting its with: sqlite3.Connection behaves the same way (transaction only), while socket.socket, open() and requests.Session all close on exit. The check is one line - enter the block, leave it, and read whatever closed-style attribute the object exposes.

Custom Context Managers

from contextlib import contextmanager
import socket

@contextmanager
def managed_socket(host, port):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    try:
        sock.connect((host, port))
        yield sock
    finally:
        sock.close()

# Usage
with managed_socket('example.com', 80) as sock:
    sock.sendall(b'GET / HTTP/1.1\r\nHost: example.com\r\n\r\n')
    response = sock.recv(4096)
# Socket closed automatically

# Class-based context manager
class DatabaseConnection:
    def __init__(self, db_config):
        self.config = db_config
        self.conn = None

    def __enter__(self):
        self.conn = psycopg2.connect(**self.config)
        return self.conn

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type is None:
            self.conn.commit()
        else:
            self.conn.rollback()
        self.conn.close()
        return False  # Don't suppress exceptions

with DatabaseConnection(db_config) as conn:
    cursor = conn.cursor()
    cursor.execute("INSERT INTO users VALUES (%s)", (username,))
# Transaction committed and connection closed

Why this works: The @contextmanager decorator turns a generator function into a context manager - the code before yield runs on entry, the yielded value is returned to the with block, and the code after yield (in the finally block) runs on exit, guaranteeing cleanup. Class-based context managers provide more control by implementing __enter__ and __exit__ methods. The __exit__ method receives exception information if an exception occurred, allowing custom error handling (commit on success, rollback on failure for databases). Context managers can return True from __exit__ to suppress exceptions, or False (default) to propagate them. This pattern extends automatic resource management to any custom resource (network connections, locks, temporary files, API clients).

LRU Cache with functools

from functools import lru_cache

@lru_cache(maxsize=1000)
def fetch_user(user_id):
    # Expensive database query
    return database.get_user(user_id)

# Cache automatically evicts least recently used entries
# when size exceeds 1000
for i in range(10_000):
    user = fetch_user(i)
    # Only 1000 most recent entries kept in memory

# Manual cache control
fetch_user.cache_info()  # hits, misses, size, maxsize
fetch_user.cache_clear()  # Clear entire cache

Why this works: functools.lru_cache provides a decorator that caches function results with automatic eviction of least recently used entries when the cache reaches maxsize. This prevents unbounded growth while maintaining high hit rates for frequently accessed data. The cache is a hash table with a doubly-linked list tracking access order. Setting maxsize=None creates an unbounded cache (dangerous for long-running apps), while maxsize=128 (or any positive integer) creates a bounded cache that won't exhaust memory. The cache is thread-safe (uses locks internally). For a pure function, adding the decorator is the whole change. For more complex scenarios (TTL-based expiration, size-based eviction, cache invalidation), use libraries like cachetools or implement custom caching with collections.OrderedDict.

Breaking Circular References with weakref

import weakref

class Parent:
    def __init__(self):
        self.children = []

    def add_child(self, child):
        self.children.append(child)

class Child:
    def __init__(self, parent):
        # Use weak reference to parent - doesn't prevent GC
        self.parent = weakref.ref(parent)
        parent.add_child(self)

    def get_parent(self):
        # Dereference weak reference
        parent = self.parent()
        if parent is None:
            raise ValueError("Parent has been garbage collected")
        return parent

# Create objects
parent = Parent()
child = Child(parent)

# Delete parent reference
del parent

# parent can now be garbage collected
# child.parent() will return None
# No circular reference leak

Why this works: weakref.ref creates a weak reference that doesn't prevent garbage collection. In a tree structure, parents hold strong references to children (preventing child GC), while children hold weak references to parents (not preventing parent GC). When the parent is no longer strongly referenced elsewhere, it can be garbage collected even though children still hold weak references to it. Those weak references become dead (dereferencing returns None), detectable via the callable interface. The cycle never forms, so the parent is released at the last strong reference instead of waiting for the cycle detector's next pass. The same applies to callback systems, caches and observer registries - anywhere you want navigation in both directions without either side pinning the other.

Connection Pooling with Context Managers

import psycopg2.pool
from contextlib import contextmanager

# Create connection pool (once at app startup). ThreadedConnectionPool, not
# SimpleConnectionPool: the "Simple" one is documented as "a connection pool
# that can't be shared across different threads" and takes no internal lock.
db_pool = psycopg2.pool.ThreadedConnectionPool(
    minconn=1,
    maxconn=20,
    dbname="mydb",
    user="user",
    password="pass"
)

@contextmanager
def get_db_connection():
    conn = db_pool.getconn()
    try:
        yield conn
        conn.commit()
    except Exception:
        conn.rollback()
        raise
    finally:
        db_pool.putconn(conn)  # Return to pool

def get_users():
    with get_db_connection() as conn:
        with conn.cursor() as cursor:
            cursor.execute("SELECT * FROM users")
            return cursor.fetchall()
    # Connection returned to pool automatically

# At app shutdown
db_pool.closeall()

Why this works: Connection pooling reuses a fixed number of database connections instead of creating new ones for each request, preventing connection exhaustion. The context manager pattern ensures connections are always returned to the pool after use, even if exceptions occur. getconn() borrows a connection from the pool and putconn() returns it, making it available for other requests. Without the context manager, forgetting to call putconn() would permanently remove a connection from the pool, eventually exhausting it.

Note what happens at the cap, because it decides how the leak surfaces: psycopg2's pool does not queue. The maxconn + 1-th caller gets psycopg2.pool.PoolError: connection pool exhausted immediately - verified on psycopg2 2.9.12 - so a leak here presents as a burst of 500s rather than as slow requests, and the traceback names getconn() rather than the code that forgot to return the connection. Size maxconn against the concurrency the process actually serves, and treat a PoolError in the logs as a leak report rather than a capacity signal until the leaked path is ruled out.

Detecting Leaks

tracemalloc is in the standard library and needs no extra dependency. Take a snapshot before and after the operation under suspicion and compare, which attributes the growth to the line that allocated it:

import tracemalloc

tracemalloc.start()
before = tracemalloc.take_snapshot()

for _ in range(1000):
    handle_request(sample_request)

after = tracemalloc.take_snapshot()
for stat in after.compare_to(before, "lineno")[:10]:
    print(stat)

Growth that survives a gc.collect() is the interesting kind: it means something still holds a reference, not that collection has yet to run. When the culprit is not obvious from the allocation site, objgraph shows what refers to a leaked object:

import gc, objgraph

gc.collect()
objgraph.show_growth(limit=10)                 # types growing between calls
objgraph.show_backrefs(objgraph.by_type("MyCache")[0], max_depth=5)

memory_profiler and pympler cover the same ground with more reporting if you need it, but both are third-party where tracemalloc is not.

Additional Resources

• CWE-401: Missing Release of Memory after Effective Lifetime
• Python Context Managers
• Python weakref Module
• functools.lru_cache