Understanding Core Dumps

When a process terminates abnormally due to specific signals, the operating system can generate a core dump. This file contains a snapshot of the process's memory at the moment of termination. Using debuggers like GDB or LLDB, developers can inspect this state to diagnose the root cause of crashes, such as segmentation faults or assertion failures.

For comprehensive details regarding the behavior and configuration of core dumps, refer to the official documentation via the man 5 core command.

Configuring Core Dump Generation

By default, many systems disable the generation of core files or limit their size to zero to save disk space. To enable them, you must adjust the resource limits associated with the process.

The ulimit shell utility is the primary method for controlling these limits. The -c flag specifically targets the maximum size of core files. The distinction between soft limits (current enforcement) and hard limits (the maximum ceiling) is important, though typically users focus on setting the soft limit.

You can enable core dumps temporarily within a shell session using:

ulimit -c unlimited

This setting resets when the session ends. For permanent changes across the system, you can:

• Modify the global shell configuration file /etc/profile by adding the ulimit directive.
• Edit the system-wide limits configuration at /etc/security/limits.conf.
• Programmatically adjust the limit within the application code using the setrlimit() system call.

Programmatic Configuration

The following C code demonstrates how to programmatically set the core file size limit to unlimited within an application. This is useful for daemon processes or when you cannot control the shell environment.

#include <stdio.h>
#include <stdlib.h>
#include <sys/resource.h>

void enable_core_dumps() {
    struct rlimit resource_limits;

    // Retrieve current limits
    if (getrlimit(RLIMIT_CORE, &resource_limits) != 0) {
        perror("Error getting resource limit");
        exit(EXIT_FAILURE);
    }

    // Set both current and max limits to unlimited
    resource_limits.rlim_cur = RLIM_INFINITY;
    resource_limits.rlim_max = RLIM_INFINITY;

    if (setrlimit(RLIMIT_CORE, &resource_limits) != 0) {
        perror("Error setting resource limit");
        exit(EXIT_FAILURE);
    }

    printf("Core dump limits updated successfully.\n");
}

int main(void) {
    enable_core_dumps();
    
    // Application logic continues here...
    return 0;
}

Debugging a Crashed Process

To illustrate the debugging workflow, consider a program that intentionally triggers a segmentation fault after a specific number of iterations. In a complex real-world application, spotting the exact line of code causing the crash visually is difficult; a core file simplifies this significantly.

#include <stdio.h>
#include <stdlib.h>
#include <sys/resource.h>

int main() {
    // Ensure we can generate a core file large enough for our memory footprint
    struct rlimit cfg = { .rlim_cur = RLIM_INFINITY, .rlim_max = RLIM_INFINITY };
    setrlimit(RLIMIT_CORE, &cfg);

    printf("Starting simulation...\n");

    int *invalid_ref = NULL;
    
    // Loop to simulate state changes before the crash
    for (int counter = 0; counter < 10; counter++) {
        printf("Iteration %d\n", counter);
        
        // Trigger a segmentation fault at a specific point
        if (counter == 5) {
            *invalid_ref = 99; // Dereferencing NULL pointer
        }
    }

    return EXIT_SUCCESS;
}

Compile this code with debugging symbols (-g):

gcc -g -o crash_test crash_test.c

Runing the executable will terminate the process and generate a core file (e.g., core.1234 or a customized name depending on /proc/sys/kernel/core_pattern).

Load the core file into LLDB to perform post-mortem analysis:

lldb crash_test --core core.1234

Within the debugger, the execution stops exactly at the faulting line. The backtrace and local variables are preserved, allowing you to inspect the state:

(lldb) target create "crash_test"
Core file '/home/user/core.1234' (x86_64) was loaded.
(lldb) bt
* thread #1, stop reason = signal SIGSEGV
  * frame #0: 0x0000000000401136 crash_test`main + 86 at crash_test.c:19
    frame #1: 0x00007ffff7a2eb97 libc.so.6`__libc_start_main + 231
    frame #2: 0x0000000000401021 crash_test`_start + 33

(lldb) frame select 0
(lldb) p counter
(int) $0 = 5
(lldb) p invalid_ref
(int *) $1 = 0x0000000000000000

Here, we can immediately see that the crash occurred at crash_test.c:19, the loop index counter was 5, and the pointer invalid_ref was NULL.

Controlling Core File Location and Naming

The kernel parameter /proc/sys/kernel/core_pattern dcitates the filename and location of generated core dumps. A common template is core_p%p_s%s_t%t, which expands the filename to include the process ID (%p), signal number (%s), and timestamp (%t).

For example, a file named core_p20828_s11_t1536568024 indicates that the process with PID 20828 crashed due to signal 11 (SIGSEGV) at the specified Unix timestamp. If the pattern specifies an absolute path, the core file will be written to that location; otherwise, it defaults to the current working directory of the crashing process. Ensure the target directory is writable by the process user, or the core file will fail to generate.