CPython Build Configuration and Setup Guide

Building CPython: Prerequisites

Compiling CPython from source requires the following environment specifications:

  • A C11-compatible compiler. Optional C11 features are not mandatory.
  • IEEE 754 floating-point support, including Not-a-Number (NaN) handling.
  • Threading capabilities.
  • OpenSSL 1.1.1 or newer for the ssl and hashlib modules.
  • Microsoft Visual Studio 2017 or later on Windows systems.

Notable historical requirement shifts: C11, IEEE 754, NaN, and VS2017+ became mandatory in 3.11. OpenSSL 1.1.1 was required starting in 3.10. Thread support and OpenSSL 1.0.2 were introduced in 3.7. Selected C99 features like <stdint.h> and static inline were mandated in 3.6. VS2015+ became the Windows baseline in 3.5. Refer to PEP 7 (C Code Style Guide) and PEP 11 (Platform Support) for further details.

Source Tree Regeneration

The CPython repository includes pre-generated artifacts to minimize build dependencies. To recreate these files from scratch, execute:

make regen-all
make regen-stdlib-module-names
make regen-limited-abi
make regen-configure

The Makefile.pre.in file catalogs these generated outputs, their source dependencies, and the tooling used. Search for regen-* targets within it.

Configure Script Regeneration

Running make regen-configure triggers Tools/build/regen-configure.sh, which produces aclocal.m4 and the configure script. By default, it utilizes an Ubuntu container to guarantee consistent tool versions and reproducible outputs. If you prefer local execution without containerization, run:

autoreconf -ivf -Werror

Beware that local generation might yield differing results based on the installed versions of autoconf-archive, aclocal, and pkg-config.

Configure Script Parameters

Display all available configuration options via:

./configure --help

Additional context is available in Misc/SpecialBuilds.txt within the source tree.

General Configuration

  • --enable-loadable-sqlite-extensions: Permits dynamic extension loading within the _sqlite module (disabled by default). Relates to sqlite3.Connection.enable_load_extension().
  • --disable-ipv6: Turns off IPv6 networking support (enabled by default).
  • --enable-big-digits=[15|30]: Configures the bit-width for Python integer digits (defaults to 30). Sets PYLONG_BITS_IN_DIGIT accordingly. See sys.int_info.bits_per_digit.
  • --with-suffix=SUFFIX: Appends a custom suffix to the Python executable. Defaults to .exe on Windows/macOS, .js/.html/.wasm on WASM targets, and an empty string on standard Unix.
  • --with-tzpath=<pathlist>: Defines the default timezone search path for zoneinfo.TZPATH. Defaults to /usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo. Paths are separated by os.pathsep.
  • --without-decimal-contextvar: Builds _decimal using thread-local contexts instead of coroutine-local contexts (default). See decimal.HAVE_CONTEXTVAR.
  • --with-dbmliborder=<backendlist>: Overrides the database backend detection order for dbm. Accepts a colon-separated string of backends: ndbm, gdbm, bdb.
  • --without-c-locale-coercion: Disables automatic C locale coercion to UTF-8 (enabled by default). Unsets PY_COERCE_C_LOCALE. See PEP 538.
  • --without-freelists: Disables all freelists except the empty tuple singleton.
  • --with-platlibdir=DIRNAME: Sets the platform-specific library directory name (defaults to lib, frequently lib64 on Fedora/SuSE). See sys.platlibdir.
  • --with-wheel-pkg-dir=PATH: Specifies the wheel package directory for ensurepip. Useful for distributions that avoid bundled dependencies (e.g., Fedora uses /usr/share/python-wheels/).
  • --with-pkg-config=[check|yes|no]: Controls dependency detection via pkg-config. check (default) makes it optional, yes mandates it, no explicitly skips it.
  • --enable-pystats: Activates internal statistics gathering. Data is dumped to /tmp/py_stats/ (or C:\temp\py_stats\ on Windows). Parse results using Tools/scripts/summarize_stats.py.

WebAssembly Targets

  • --with-emscripten-target=[browser|node]: Sets the build profile for wasm32-emscripten. browser (default) minimizes stdlib and relies on MEMFS. node enables NODERAWFS and pthreads.
  • --enable-wasm-dynamic-linking: Activates dynamic linking for WASM, enabling dlopen. Increases binary size due to limited dead code elimination.
  • --enable-wasm-pthreads: Enables pthreads support for WASM builds.

Installation Directories

  • --prefix=PREFIX: Installs architecture-independent files into PREFIX (defaults to /usr/local). Accessible via sys.prefix.
  • --exec-prefix=EPREFIX: Installs architecture-dependent files into EPREFIX (defaults to PREFIX). Accessible via sys.exec_prefix.
  • --disable-test-modules: Skips compilation and installation of test packages and extensions like _testcapi.
  • --with-ensurepip=[upgrade|install|no]: Configures the ensurepip execution during installation. upgrade (default) runs --altinstall --upgrade, install runs --altinstall, no skips it entirely.

Performance Tuning

For maximum runtime efficiency, configure with --enable-optimizations --with-lto (PGO + LTO). The experimental --enable-bolt flag offers additional post-link optimizations.

  • --enable-optimizations: Enables Profile-Guided Optimization (PGO) using PROFILE_TASK. Clang requires llvm-profdata. When paired with --enable-shared and GCC, -fno-semantic-interposition is appended to compiler/linker flags.
  • PROFILE_TASK: Environment variable defining the Python command executed for PGO data collection. Defaults to -m test --pgo --timeout=$(TESTTIMEOUT).
  • --with-lto=[full|thin|no|yes]: Activates Link-Time Optimization. Clang requires llvm-ar and an LTO-capable linker (ld.gold or lld). ThinLTO can be explicitly requested via thin on Clang. Since 3.12, ThinLTO is the default Clang strategy if supported.
  • --enable-bolt: Activates the BOLT binary optimizer (requires llvm-bolt and merge-fdata). Considered experimental; success relies heavily on the specific environment and CPU architecture. LLVM 16+ is strongly recommended. BOLT_INSTRUMENT_FLAGS and BOLT_APPLY_FLAGS can override default BOLT parameters.
  • --with-computed-gotos: Enables computed gotos in the evaluation loop (enabled by default on supported compilers).
  • --without-pymalloc: Disables the specialized Python memory allocator (enabled by default). See PYTHONMALLOC.
  • --without-doc-strings: Omits static documentation strings to reduce memory footprint. Does not affect Python-defined docstrings. Relates to the PyDoc_STRVAR() macro.
  • --enable-profiling: Enables C-level profiling via gprof.
  • --with-strict-overflow: Appends -fstrict-overflow to C compiler flags (replacing the default -fno-strict-overflow).

Debug Builds

A Python debug build is activated via --with-pydebug. It introduces the following behaviors:

  • Default warning filters are emptied, displaying all warnings.
  • d is appended to sys.abiflags.
  • sys.gettotalrefcount() becomes available.
  • The -X showrefcount CLI argument is supported.
  • -d argument and PYTHONDEBUG environment variable enable parser debugging.
  • Defining __lltrace__ enables low-level bytecode tracing.
  • Memory allocation debug hooks are installed to detect overflows and memory errors.
  • Macros Py_DEBUG and Py_REF_DEBUG are defined.
  • Runtime checks inside #ifdef Py_DEBUG blocks are activated, including assert() and _PyObject_ASSERT(). Checks cover: function argument sanity, pattern-filled memory for Unicode/int objects to detect uninitialized use, exception state consistency during replacements, memory deallocator exception preservation, garbage collector object consistency, and integer overflow checks via Py_SAFE_DOWNCAST().

Since 3.8, debug and release builds are ABI compatible; defining Py_DEBUG no longer forces Py_TRACE_REFS.

Debugging and Sanitizers

  • --with-pydebug: Compiles in debug mode, defining Py_DEBUG.
  • --with-trace-refs: Enables reference tracing for debugging. Defines Py_TRACE_REFS, adds sys.getobjects(), and introduces PYTHONDUMPREFS. ABI incompatible with release and standard debug builds.
  • --with-assertions: Enables C assertions (unsets NDEBUG in OPT). Similar to debug mode assertions but without the full debug build overhead.
  • --with-valgrind: Enables Valgrind support.
  • --with-dtrace: Enables DTrace profiling.
  • --with-address-sanitizer: Enables ASan (AddressSanitizer).
  • --with-memory-sanitizer: Enables MSan (MemorySanitizer).
  • --with-undefined-behavior-sanitizer: Enables UBSan (UndefinedBehaviorSanitizer).

Linker Configuration

  • --enable-shared: Compiles the shared libpython library.
  • --without-static-libpython: Skips building libpythonMAJOR.MINOR.a and installing python.o.

External Libraries

  • --with-libs='lib1 ...': Links additional external libraries.
  • --with-system-expat: Builds pyexpat using the system's installed expat library.
  • --with-system-libmpdec: Builds _decimal using the system's installed mpdec library.
  • --with-readline=editline: Uses editline as the backend for the readline module. Defines WITH_EDITLINE.
  • --without-readline: Skips building the readline module entirely. Unsets HAVE_LIBREADLINE.
  • --with-libm=STRING: Overrides the default math library (libm).
  • --with-libc=STRING: Overrides the default C library (libc).
  • --with-openssl=DIR: Specifies the root directory for OpenSSL.
  • --with-openssl-rpath=[no|auto|DIR]: Configures OpenSSL runtime library paths (rpath). no (default) sets no rpath, auto detects based on --with-openssl and pkg-config, DIR sets an explicit path.

Security Algorithms

  • --with-hash-algorithm=[fnv|siphash13|siphash24]: Selects the hashing algorithm for Python/pyhash.c. Defaults to siphash13.
  • --with-builtin-hashlib-hashes=md5,sha1,sha256,sha512,sha3,blake2: Specifies which hash modules are built-in.
  • --with-ssl-default-suites=[python|openssl|STRING]: Overrides OpenSSL default cipher suites. python (default) uses Python recommendations, openssl preserves upstream defaults, STRING applies a custom string. Choosing python or a custom string also sets TLS 1.2 as the minimum protocol version.

macOS Specifics

  • --enable-universalsdk[=SDKDIR]: Creates a universal binary. Optionally specifies the macOS SDK directory.
  • --enable-framework[=INSTALLDIR]: Creates a Python.framework instead of a standard Unix install. Optionally specifies the installation path.
  • --with-universal-archs=ARCH: Defines the universal binary architecture (e.g., universal2, 32-bit, 64-bit, 3-way, intel, intel-32, intel-64, all). Requires --enable-universalsdk.
  • --with-framework-name=FRAMEWORK: Specifies the framework name (defaults to Python). Requires --enable-framework.

Cross-Compilation

Cross-compiling builds Python for a target architecture different from the build host. It requires a build-platform Python interpreter matching the target version.

  • --build=BUILD: Specifies the build platform (auto-detected by config.guess).
  • --host=HOST: Specifies the target platform.
  • --with-build-python=path/to/python: Points to the build-platform Python binary.
  • CONFIG_SITE=file: Environment variable pointing to a configuration overrides file.

Example config.site for an ARM64 target:

# config.site-arm64
ac_cv_func_getaddrinfo=yes
ac_cv_file__dev_ptmx=yes
ac_cv_have_decl_printf=yes

Example cross-compilation invocation:

CONFIG_SITE=config.site-arm64 ../configure \
    --build=x86_64-linux-gnu \
    --host=aarch64-linux-gnu \
    --with-build-python=../x86_64/python

Build System Architecture

Key Source Files

  • configure.ac generates configure.
  • Makefile.pre.in generates Makefile (via configure).
  • pyconfig.h is generated by configure.
  • Modules/Setup dictates C extension builds, processed by Module/makesetup.

Build Pipeline

  • C source files (.c) compile into object files (.o).
  • Object files are archived into the static libpython (.a).
  • python.o and libpython are linked into the final python executable.
  • C extensions are built by the Makefile according to Modules/Setup.

Makefile Targets

  • make: Builds Python alongside the standard library.
  • make platform: Builds the python executable without standard library extensions.
  • make profile-opt: Constructs Python using Profile-Guided Optimization. Automatically invoked by default if --enable-optimizations was configured.
  • make buildbottest: Compiles Python and runs the test suite mimicking buildbot behavior. Timeout defaults to 1200 seconds, adjustable via TESTTIMEOUT.
  • make install: Compiles and installs Python.
  • make regen-all: Regenerates most derived files. make regen-stdlib-module-names and autoconf must run separately for remaining artifacts.
  • make clean: Removes build artifacts.
  • make distclean: Removes build artifacts and files generated by configure.

C Extension Variants

Built-in modules (e.g., sys) are compiled with the Py_BUILD_CORE_BUILTIN macro. They lack a __file__ attribute:

>> import sys
>>> print(sys)
<module 'sys' (built-in)>
>>> print(sys.__file__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'sys' has no attribute '__file__'

Dynamic modules (e.g., _json) are compiled with the Py_BUILD_CORE_MODULE macro. They possess a __spec__.origin (and historically __file__) pointing to their shared object:

>> import _json
>>> print(_json)
<module '_json' from '/usr/lib64/python3.11/lib-dynload/_json.cpython-311-x86_64-linux-gnu.so'>
>>> print(_json.__spec__.origin)
'/usr/lib64/python3.11/lib-dynload/_json.cpython-311-x86_64-linux-gnu.so'

Within Modules/Setup, extensions defined before the *shared* marker become built-ins, while those after it become dynamic libraries.

The macros PyAPI_FUNC(), PyAPI_DATA(), and PyMODINIT_FUNC (defined in Include/exports.h) resolve to Py_EXPORTED_SYMBOL if Py_BUILD_CORE_MODULE is set, otherwise Py_IMPORTED_SYMBOL. Misapplying Py_BUILD_CORE_BUILTIN on a shared extension will hide its PyInit_xxx() function, resulting in an ImportError.

Compiler and Linker Flags

Flags are initially set by ./configure and environment variables, then consumed by the Makefile.

Preprocessor Flags

  • CONFIGURE_CPPFLAGS: Passed directly to ./configure.
  • CPPFLAGS: C/C++ preprocessor flags (e.g., -Iinclude_dir). Must contain shell values so extension modules can locate directories specified in environment variables.
  • BASECPPFLAGS: Base preprocessor flags.
  • PY_CPPFLAGS: Additional preprocessor flags for interpreter object files. Defaults to $(BASECPPFLAGS) -I. -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) $(CPPFLAGS).

Compiler Flags

  • CC: C compiler command (e.g., gcc -pthread).
  • CXX: C++ compiler command (e.g., g++ -pthread).
  • CFLAGS: General C compiler flags.
  • CFLAGS_NODIST: Used strictly for interpreter and stdlib C extension builds. Prevents flags from leaking into the installed CFLAGS, which would otherwise override user/package -I paths or impose unmanageable strictness like -Werror on third-party modules.
  • COMPILEALL_OPTS: Arguments for compileall during make install PYC generation. Defaults to -j0.
  • EXTRA_CFLAGS: Additional C compiler directives.
  • CONFIGURE_CFLAGS: Passed to ./configure.
  • CONFIGURE_CFLAGS_NODIST: Passed to ./configure.
  • BASECFLAGS: Base compiler flags.
  • OPT: Optimization flags.
  • CFLAGS_ALIASING: Aliasing strictness flags for compiling Python/dtoa.c.
  • CCSHARED: Flags for shared library compilation (e.g., -fPIC on Linux/BSD).
  • CFLAGSFORSHARED: Extra C flags for interpreter object files. Defaults to $(CCSHARED), or empty if --enable-shared is active.
  • PY_CFLAGS: Defaults to $(BASECFLAGS) $(OPT) $(CONFIGURE_CFLAGS) $(CFLAGS) $(EXTRA_CFLAGS).
  • PY_CFLAGS_NODIST: Defaults to $(CONFIGURE_CFLAGS_NODIST) $(CFLAGS_NODIST) -I$(srcdir)/Include/internal.
  • PY_STDMODULE_CFLAGS: Flags for interpreter object files. Defaults to $(PY_CFLAGS) $(PY_CFLAGS_NODIST) $(PY_CPPFLAGS) $(CFLAGSFORSHARED).
  • PY_CORE_CFLAGS: Defaults to $(PY_STDMODULE_CFLAGS) -DPy_BUILD_CORE.
  • PY_BUILTIN_MODULE_CFLAGS: Flags for stdlib built-ins. Defaults to $(PY_STDMODULE_CFLAGS) -DPy_BUILD_CORE_BUILTIN.
  • PURIFY: Purify memory debugger command. Defaults to empty.

Linker Flags

  • LINKCC: Linker command for executables like python and _testembed. Defaults to $(PURIFY) $(CC).
  • CONFIGURE_LDFLAGS: Passed to ./configure. Avoid setting directly so users can append values via command line without overriding presets.
  • LDFLAGS_NODIST: Used similar to CFLAGS_NODIST. Prevents installed LDFLAGS from overriding user/package -L library paths.
  • CONFIGURE_LDFLAGS_NODIST: Passed to ./configure.
  • LDFLAGS: Linker flags (e.g., -Llib_dir). Must contain shell values to support extension module builds using environment-specified directories.
  • LIBS: Linker flags passing libraries to the linker (e.g., -lrt).
  • LDSHARED: Command to build a shared library. Defaults to @LDSHARED@ $(PY_LDFLAGS).
  • BLDSHARED: Command to build the shared libpython. Defaults to @BLDSHARED@ $(PY_CORE_LDFLAGS).
  • PY_LDFLAGS: Defaults to $(CONFIGURE_LDFLAGS) $(LDFLAGS).
  • PY_LDFLAGS_NODIST: Defaults to $(CONFIGURE_LDFLAGS_NODIST) $(LDFLAGS_NODIST).
  • PY_CORE_LDFLAGS: Linker flags specifically for interpreter object files.

Tags: python CPython build-configuration compilation system-setup

Posted on Mon, 13 Jul 2026 17:02:10 +0000 by ee12csvt

Hot Tags

©2026 Freaks City