Stack Frames and Function Calls
Every function allocates a dedicated region of memory on the call stack referred to as a stack frame. Understanding function call mechanics fundamentally involves understanding how the caller's frame interacts with the callee's frame.
A stack frame serves several critical purposes: it preserves parameters passed to the callee function, stores the return address indicating where execution should resume after the callee completes, holds the return value, and maintains the function's local variables.
The AMD64 architecture provides two essential registers for stack management:
- The rsp register always points to the current stack top
- The rbp register typically marks the starting address of the current function's stack frame
Consider a nested call sequence: A() calls B(), which then calls C(). This pattern reveals important characteristics:
- Return addresses and values are preserved in the caller's frame, while the callee primarily stores its temporary variables
- Stack memory grows downward (toward lower addresses), but data fills upward
- The rsp and rbp registers always reference the current function's stack top and bottom respectively
Assembly Instructions for Function Calls
call and ret
call target_address
ret
The call instruction performs two operations: it pushes the current instruction pointer value onto the stack (this becomes the return address for the callee), then transfers control to the target address by updating the instruction pointer. The ret instruction reverses this process by popping the saved instruction pointer from the stack, restoring execution to the instruction following the original call.
leave
leave
The leave instruction appears at the end of functions, immediately before ret. It adjusts both rsp and rbp without requiring explicit operands, effectively performing:
mov rsp, rbp
pop rbp
Before executing ret, rbp must point to the callee's frame base and rsp must reference the location where the caller's frame begins. The leave instruction accomplishes exactly this restoration.
push and pop
push source
pop destination
When a function calls another, the caller hasn't completed execution, so the callee cannot overwrite the caller's frame. The push and pop instructions create new frame space for the callee's data.
The push instruction first decrements rsp by 8 (on 64-bit systems), allocates space, then copies the source operand to the memory location rsp now references. This is equivalent to:
sub $8, rsp
mov source, (%rsp)
The pop instruction reverses this process: it copies the data at the rsp location to the destination operand, then increments rsp by 8:
mov (%rsp), destination
add $8, rsp
Practical Example: Tracing a Function Call
The following C program demonstrates a simple function call:
#include <stdio.h>
int add_values(int x, int y)
{
int result = x + y;
return result;
}
int main(void)
{
int value = add_values(3, 4);
printf("Result: %d\n", value);
return 0;
}
Debugging with GDB
Key GDB commands for tracing execution:
| Command | Purpose |
|---|---|
| run / r | Execute program until breakpoint |
| continue / c | Continue to next breakpoint |
| next / n | Step over function calls |
| step / s | Step into function calls |
| finish | Run until current function returns |
| quit / q | Exit debugger |
Breakpoint management commands:
| Command | Purpose |
|---|---|
| break n | Set breakpoint at line n |
| break func | Set breakpoint at function entry |
| delete n | Remove breakpoint n |
| disable n / enable n | Toggle breakpoint n |
| info b | List all breakpoints |
Useful inspection commands:
| Command | Purpose |
|---|---|
| print / p expr | Evaluate expression |
| display expr | Show expression after each step |
| info registers | Display register values |
| info frame | Show current stack frame details |
Disassembly and Register Analysis
Compiling with debug symbols and examining main's assembly:
0x0000555555554664 <main+0>: push rbp
0x0000555555554665 <main+1>: mov rbp, rsp
0x0000555555554668 <main+4>: sub rsp, 0x20
0x000055555555466c <main+8>: mov DWORD PTR [rbp-0x14], edi
0x000055555555466f <main+11>: mov QWORD PTR [rbp-0x20], rsi
0x0000555555554673 <main+15>: mov esi, 4
0x0000555555554678 <main+20>: mov edi, 3
0x000055555555467d <main+25>: call 0x55555555464a <add_values>
Initial register state:
rsp = 0x7fffffffe028
rbp = 0x5555555546b0 <__libc_csu_init>
rip = 0x555555554664 <main>
Step 1: push rbp
The push instruction decrements rsp by 8, creating space for the caller's frame base pointer:
rsp = 0x7fffffffe020
rbp = 0x5555555546b0 <__libc_csu_init>
rip = 0x555555554665 <main+1>
Step 2: mov rbp, rsp
This establishes the new frame base, so both registers now reference the same location:
rsp = 0x7fffffffe020
rbp = 0x7fffffffe020
rip = 0x555555554668 <main+4>
Step 3: sub rsp, 0x20
Allocating 32 bytes for local variables expands the stack frame:
rsp = 0x7fffffffe000
rbp = 0x7fffffffe020
rip = 0x55555555466c <main+8>
Step 4: Prepare function arguments
The calling convention uses rdi, rsi, rdx for integer arguments (in order):
0x55555555466c <main+8>: mov DWORD PTR [rbp-0x14], edi
0x55555555466f <main+11>: mov QWORD PTR [rbp-0x20], rsi
0x555555554673 <main+15>: mov esi, 4
0x555555554678 <main+20>: mov edi, 3
Step 5: call add_values
The call instruction pushes the return address (main+30) onto the stack and transfers control to add_values:
rip = 0x555555554682 <main+30>
rsp = 0x7fffffffe000
rbp = 0x7fffffffe020
Inside the Called Functon
The add_values function's disassembly:
0x000055555555464a <add_values+0>: push rbp
0x000055555555464b <add_values+1>: mov rbp, rsp
0x000055555555464e <add_values+4>: mov DWORD PTR [rbp-0x14], edi
0x0000555555554651 <add_values+7>: mov DWORD PTR [rbp-0x18], esi
0x0000555555554654 <add_values+10>: mov edx, DWORD PTR [rbp-0x14]
0x0000555555554657 <add_values+13>: mov eax, DWORD PTR [rbp-0x18]
0x000055555555465a <add_values+16>: add eax, edx
0x000055555555465c <add_values+18>: mov DWORD PTR [rbp-0x4], eax
0x000055555555465f <add_values+21>: mov eax, DWORD PTR [rbp-0x4]
0x0000555555554662 <add_values+24>: pop rbp
0x0000555555554663 <add_values+25>: ret
Notice that add_values doesn't adjust rsp like main does. The reason is subtle but important: main must reserve stack space because the call instruction automatically decrements rsp by 8 to store the return address. If main didn't adjust rsp beforehand, this automatic push would overwrite local variable space. The add_values function contains no instructions that automatically modify rsp, so no adjustment is needed.
The local variable result at [rbp-0x4] still resides on the stack, even though no explicit space allocation occurred. Stack memory is pre-allocated by the system and available for use without explicit management.
Returning to Caller
The ret instruction pops the saved instruction pointer, restoring control to main:
rip = 0x555555554682 <main+30>
The return value travels through the eax register. Main stores this result and prepares for the printf call:
0x555555554682 <main+30>: mov DWORD PTR [rbp-0x4], eax
0x555555554685 <main+33>: mov eax, DWORD PTR [rbp-0x4]
0x555555554688 <main+36>: mov esi, eax
0x55555555468a <main+38>: lea rdi, [rip+0xa3]
0x555555554691 <main+45>: mov eax, 0
0x555555554696 <main+50>: call 0x555555554520 <printf@plt>
Function Epilogue
The leave instruction restores the caller's stack frame state:
leave
This single instruction performs the equivalent of:
mov rsp, rbp
pop rbp
After leave executes:
rip = 0x5555555546a1 <main+61>
rsp = 0x7fffffffe028
rbp = 0x5555555546b0 <__libc_csu_init>
The final ret instruction pops the saved instruction pointer, transferring control back to the function that invoked main.