Beyond constant folding, CPython's abstract syntax tree (AST) optimizer implements several other transformations that reduce runtime work. These optimizations are applied during compilation, converting high-level constructs into simpler constant forms or more efficient bytecode sequences. Below we examine the key internal functions that drive this process.

The create_constant Utility

Many optimization routines rely on a helper that promotes an expression node to a constant. When a new constant value is computed (e.g., from folding 1+2 into 3), create_constant stores that value in the AST node and changes its type to Constant_kind. If the computed value is NULL (e.g., after a division‑by‑zero error), it clears the exception and returns success without modification.

static int
create_constant(expr_ty node, PyObject *value, PyArena *arena)
{
    if (value == NULL) {
        if (PyErr_ExceptionMatches(PyExc_KeyboardInterrupt))
            return 0;
        PyErr_Clear();
        return 1;
    }
    if (_PyArena_AddPyObject(arena, value) < 0) {
        Py_DECREF(value);
        return 0;
    }
    node->kind = Constant_kind;
    node->v.Constant.kind = NULL;
    node->v.Constant.value = value;
    return 1;
}

Unary Operator Folding

When a unary operator (like not, ~, +, -) is applied to a constant operand, the optimizer computes the result immediately. If the operand is not a constant, the function attempts to fold a not operator on a comparison into its inverse (e.g., not (x is y) becomes x is not y). The implementation uses a lookup table of unary operations.

static int
optimize_unary_operation(expr_ty node, PyArena *arena, _PyASTOptimizeState *state)
{
    expr_ty operand = node->v.UnaryOp.operand;
    if (operand->kind != Constant_kind) {
        /* Try to invert comparison operators under 'not'. */
        if (node->v.UnaryOp.op == Not && operand->kind == Compare_kind
            && asdl_seq_LEN(operand->v.Compare.ops) == 1) {
            cmpop_ty op = asdl_seq_GET(operand->v.Compare.ops, 0);
            cmpop_ty newop = 0;
            switch (op) {
                case Is:    newop = IsNot; break;
                case IsNot: newop = Is;    break;
                case In:    newop = NotIn; break;
                case NotIn: newop = In;    break;
                default: break;  // Eq, NotEq, Lt, etc. cannot be inverted safely
            }
            if (newop) {
                asdl_seq_SET(operand->v.Compare.ops, 0, newop);
                COPY_NODE(node, operand);
                return 1;
            }
        }
        return 1;
    }
    typedef PyObject *(*unary_op_fn)(PyObject*);
    static const unary_op_fn ops[] = {
        [Invert] = PyNumber_Invert,
        [Not]    = unary_not,
        [UAdd]   = PyNumber_Positive,
        [USub]   = PyNumber_Negative,
    };
    PyObject *newval = ops[node->v.UnaryOp.op](operand->v.Constant.value);
    return create_constant(node, newval, arena);
}

Iterable Folding

List and set literals that do not contain starred expressions can be turned into tuples (or frozensets) at compile time. This allows the constant tuple/frozenset to be reused and reduces allocation at runtime. The function fold_iterable first converts a list node into a tuple node and then attempts to produce a constant tuple from its elements. For sets, it builds a frozenset from the same tuple.

static int
fold_iterable(expr_ty arg, PyArena *arena, _PyASTOptimizeState *state)
{
    PyObject *newval;
    if (arg->kind == List_kind) {
        asdl_expr_seq *elts = arg->v.List.elts;
        Py_ssize_t n = asdl_seq_LEN(elts);
        for (Py_ssize_t i = 0; i < n; i++) {
            expr_ty e = (expr_ty)asdl_seq_GET(elts, i);
            if (e->kind == Starred_kind)
                return 1;
        }
        expr_context_ty ctx = arg->v.List.ctx;
        arg->kind = Tuple_kind;
        arg->v.Tuple.elts = elts;
        arg->v.Tuple.ctx = ctx;
        newval = make_const_tuple(elts);
    }
    else if (arg->kind == Set_kind) {
        newval = make_const_tuple(arg->v.Set.elts);
        if (newval)
            Py_SETREF(newval, PyFrozenSet_New(newval));
    }
    else
        return 1;
    return create_constant(arg, newval, arena);
}

Comparison Operator Optimization

When a in or not in comparison has a list or set literal on the right side, the optimizer converts that literal into a tuple or frozenset via fold_iterable. This transformation is safe because the containment test does not depend on mutability.

static int
optimize_comparison(expr_ty node, PyArena *arena, _PyASTOptimizeState *state)
{
    asdl_int_seq *ops = node->v.Compare.ops;
    asdl_expr_seq *comps = node->v.Compare.comparators;
    int op = asdl_seq_GET(ops, asdl_seq_LEN(ops) - 1);
    if (op == In || op == NotIn) {
        if (!fold_iterable((expr_ty)asdl_seq_GET(comps, asdl_seq_LEN(comps)-1),
                           arena, state))
            return 0;
    }
    return 1;
}

Subscript Folding

If both the subscripted object and the index are constants, the optimizer evaluates the indexing operation at compile time. For instance, "hello"[2] becomes the constant 'l'. The function uses PyObject_GetItem to perform the lookup and then stores the result as a constant.

static int
fold_subscript(expr_ty node, PyArena *arena, _PyASTOptimizeState *state)
{
    expr_ty obj = node->v.Subscript.value;
    expr_ty idx = node->v.Subscript.slice;
    if (node->v.Subscript.ctx != Load ||
        obj->kind != Constant_kind ||
        idx->kind != Constant_kind)
        return 1;
    PyObject *newval = PyObject_GetItem(obj->v.Constant.value,
                                         idx->v.Constant.value);
    return create_constant(node, newval, arena);
}

Tuple Folding

When a tuple literal is used in a load context, the optimizer converts it into a constant tuple. This is a straightforward call to make_const_tuple.

static int
fold_tuple(expr_ty node, PyArena *arena, _PyASTOptimizeState *state)
{
    if (node->v.Tuple.ctx != Load)
        return 1;
    PyObject *newval = make_const_tuple(node->v.Tuple.elts);
    return create_constant(node, newval, arena);
}

Practical Demonstrations

The following Python snippets show how these optimizations appear in the generated bytecode. In the first example, a list inside a containment test becomes a tuple, and a set literal used as an iterable becomes a frozan tuple.

import ast, dis

code = """
5 in [5, 6, 7]
8 not in [9]
for x in {10}:
    pass
"""
print(ast.dump(ast.parse(code, mode='exec'), indent=4))
dis.dis(code)

The second example demonstrates subscript folding: a constant string index is resolved during compilation.

code = """"abcdef"[3]"""
print(ast.dump(ast.parse(code, mode='eval'), indent=4))
dis.dis(code)

What Python Does Not Optimize

Despite these transformations, CPython leaves many classic loop optimizations unimplemented. For instance, a loop that iterates over a range that is statically known to be empty still produces bytecode for the full iteration structure. The optimizer does not eliminate such loops.

for i in range(0):
    print(i)

The corresponding bytecode still includes GET_ITER, FOR_ITER, and all related instructions, wasting both space and time.