When working with machine learning frameworks that depend on specific CUDA releases, maintaining multiple toolkit versions on a single Linux host becomes essential. Rather than removing existing installations, you can deploy additional releases alongside the primary version and toggle between them dynamically.

Installing an Additional CUDA Release

Download the desired runfile installer from the NVIDIA archive:

wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
chmod +x cuda_11.8.0_520.61.05_linux.run
sudo sh cuda_11.8.0_520.61.05_linux.run

During the text-based installation flow:

1. Accept the EULA by typing accept
2. In the component selection screen, deselect Driver (assuming your existing driver supports the target CUDA version)
3. Navigate to Options → Toolkit Options:
   • Disable "Create symbolic link from /usr/local/cuda"
   • Disable "Create desktop menu shortcuts"
   • Disable "Install manpage documents"
   • Select "Change Toolkit Install Path" and verify it points to /usr/local/cuda-11.8/
4. Navigate to Library install path and set it to /usr/local/cuda-11.8/
5. Navigate to Driver Options and ensure all driver components remain unchecked
6. Return to the main screen and select Install

Upon completion, the installer displays:

===========
= Summary =
===========

Driver:   Not Selected
Toolkit:  Installed in /usr/local/cuda-11.8/

Please make sure that
 -   PATH includes /usr/local/cuda-11.8/bin
 -   LD_LIBRARY_PATH includes /usr/local/cuda-11.8/lib64

Integrating cuDNN Libraries

For each CUDA installation requiring cuDNN:

1. Download the "Local Installer for Linux x86_64 (Tar)" matching your CUDA version from the NVIDIA cuDNN archive
2. Extract the archive, yielding include/ and lib/ directories
3. Deploy the binaries to the CUDA installation:

CUDA_VER="11.8"
sudo cp cuda/include/cudnn*.h /usr/local/cuda-${CUDA_VER}/include/
sudo cp cuda/lib64/libcudnn* /usr/local/cuda-${CUDA_VER}/lib64/
sudo chmod a+r /usr/local/cuda-${CUDA_VER}/include/cudnn*.h /usr/local/cuda-${CUDA_VER}/lib64/libcudnn*

Verify the enstallation:

grep CUDNN_MAJOR /usr/local/cuda-11.8/include/cudnn_version.h -A 2

Expected output:

#define CUDNN_MAJOR 8
#define CUDNN_MINOR 9
#define CUDNN_PATCHLEVEL 7

Dynamic Version Switching

Create a shell function to manage environment variables. Place the following in ~/.bashrc or a dedicated script file (e.g., /usr/local/bin/cuda-switch):

#!/bin/bash
# CUDA Version Switcher Utility
# Usage: source cuda-switch <version_number>

switch_cuda() {
    local target_ver="$1"
    local base_location="/usr/local"
    
    if [[ -z "$target_ver" ]]; then
        echo "Installed CUDA versions:"
        ls -d ${base_location}/cuda-* 2>/dev/null | grep -oE 'cuda-[0-9]+\.[0-9]+$' | sed 's/cuda-/* /'
        return 1
    fi
    
    local cuda_home="${base_location}/cuda-${target_ver}"
    
    if [[ ! -d "$cuda_home" ]]; then
        echo "Error: CUDA ${target_ver} not found in ${base_location}"
        return 1
    fi
    
    # Strip existing CUDA entries from PATH
    local new_path=""
    local IFS_OLD="$IFS"
    IFS=':'
    for dir in $PATH; do
        if [[ ! "$dir" =~ ^${base_location}/cuda ]]; then
            if [[ -z "$new_path" ]]; then
                new_path="$dir"
            else
                new_path="${new_path}:${dir}"
            fi
        fi
    done
    IFS="$IFS_OLD"
    
    # Strip existing CUDA entries from LD_LIBRARY_PATH
    local new_ld_path=""
    IFS=':'
    for dir in $LD_LIBRARY_PATH; do
        if [[ ! "$dir" =~ ^${base_location}/cuda ]]; then
            if [[ -z "$new_ld_path" ]]; then
                new_ld_path="$dir"
            else
                new_ld_path="${new_ld_path}:${dir}"
            fi
        fi
    done
    IFS="$IFS_OLD"
    
    # Prepend target CUDA paths
    export PATH="${cuda_home}/bin${new_path:+:$new_path}"
    export LD_LIBRARY_PATH="${cuda_home}/lib64${new_ld_path:+:$new_ld_path}"
    export CUDA_HOME="$cuda_home"
    export CUDA_ROOT="$cuda_home"
    
    echo "Active CUDA: $(nvcc --version | grep 'release' | awk '{print $5, $6}')"
}

# Execute if sourced with argument
if [[ "${BASH_SOURCE[0]}" != "${0}" ]] && [[ -n "$1" ]]; then
    switch_cuda "$1"
fi

Activate a specific version:

source cuda-switch 11.8

Output:

Active CUDA: release 11.8, V11.8.89

To persist this configuraton across new terminal sessions, add the export statements to your shell profile or invoke the sourcing command in your initialization files.