Flink TaskManager Memory Allocation Practical Examples

Configuration Requirements Overview

Flink memory configuration requires using one of three methods (except for local execution), otherwise startup will fail. You must explicitly configure one of these subsets which have no default values:

Component TaskManager Option JobManager Option
Method 1 taskmanager.memory.flink.size jobmanager.memory.flink.size
Method 2 taskmanager.memory.process.size jobmanager.memory.process.size
Method 3 taskmanager.memory.task.heap.size and taskmanager.memory.managed.size jobmanager.memory.heap.size

1. Configuring Total Process Size Only

taskmanager.memory.process.size: 2000m

Memory calculation steps:

  • Total Process Size = 2000 MB
  • JVM Overhead = 2000 MB × 0.1 = 200 MB (automatic when not specified)
  • JVM Metaspace = 256 MB (default)
  • Native Memory = JVM Overhead + JVM Metaspace = 456 MB
  • Total Flink Size = 2000 MB - 200 MB - 256 MB = 1544 MB
  • Network Memory = 1544 × 0.1 = 154.4 MB
  • Task Off-Heap = 0 MB (default)
  • Framework Off-Heap = 128 MB (default)
  • Total Direct Memory = 154.4 MB + 0 + 128 MB = 282.4 MB
  • Managed Memory = 1544 MB × 0.4 = 617.6 MB
  • Total JVM Heap Memory = 1544 MB - 282.4 MB - 617.6 MB = 644 MB
  • Framework Heap = 128 MB
  • Task Heap = 644 MB - 128 MB = 516 MB

2. Configuring Total Flink Size Only

taskmanager.memory.flink.size: 2000m

Calculation when only specifying taskmanager.memory.flink.size: 2000m:

  • Total Flink Size = 2000 MB
  • Managed Memory = 2000 MB × 0.4 = 800 MB
  • Network Memory = 2000 MB × 0.1 = 200 MB
  • Framework Off-Heap = 128 MB
  • Task Off-Heap = 0 Byte = 0 MB
  • Total Direct Memory = 200 MB + 128 MB + 0 MB = 328 MB
  • Total Off-Heap Memory = 800 MB + 328 MB = 1128 MB
  • Total JVM Heap = 2000 MB - 800 MB - 328 MB = 872 MB
  • Framework Heap = 128 MB
  • Task Heap = 872 MB - 128 MB = 744 MB
  • JVM MetaSpace = 256 MB (default)
  • JVM Overhead calculation: (JVM Overhead + 256 MB + 2000 MB) × 0.1, resulting in JVM Overhead = 250.667 MB (within 192MB ~ 1GB range)
  • Total Process Size = 2000 MB + 256 MB + 250.667 MB = 2506.667 MB

3. Configuring Heap Size and Managed Memory

taskmanager.memory.task.heap.size: 1000m
taskmanager.memory.managed.size: 1000m
  • Framework Heap = 128 MB (default)
  • Task Heap = 1000 MB (configured)
  • Total JVM Heap = 1000 MB + 128 MB = 1128 MB
  • Managed Memory = 1000 MB (configured)
  • Framework Off-Heap = 128 MB
  • Task Off-Heap = 0 MB
  • Network calculation: (Network + 1128 MB + 1000 MB + 128 MB + 0 MB) × 0.1, resulting in Network = 250.667 MB (valid within 64MB ~ 1GB range)
  • Total Direct Memory = 128 MB + 250.667 MB = 378.667 MB
  • Total Flink Size = 1128 MB + 1378.667 MB = 2506.667 MB
  • JVM Metaspace = 256 MB (default)
  • JVM Overhead calculation: (JVM Overhead + 1128 MB + 1000 MB + 378.667 MB + 256 MB) × 0.1 = 306.963 MB (valid within 192M ~ 1GB range)
  • Total Process Size = 2506.667 MB + 256 MB + 306.963 MB = 3069.63 MB

4. Process Size with Specific Heap and Managed Memory

Configuring Total Process Size while explicitly setting JVM Heap and Managed Memory components:

taskmanager.memory.process.size: 3000m
taskmanager.memory.task.heap.size: 1000m
taskmanager.memory.managed.size: 1000m
  • Total Process Size = 3000 MB
  • Framework Heap = 128 MB (default)
  • Task Heap = 1000 MB (configured)
  • Total JVM Heap = Framework Heap + Task Heap = 128 MB + 1000 MB = 1128 MB
  • Managed Memory = 1000 MB (configured)
  • Framework Off-Heap = 128 MB (default)
  • Task Off-Heap = 0 MB (default)
  • Network Memory calculation: (Network Memory + 1128 MB + 1128 MB) × 0.1 = 250.667 MB (within 64M ~ 1GB requirement)
  • Total Off-Heap = 1000 MB + 128 MB + 250.667 MB + 0 MB = 1378.667 MB
  • Total Flink Size = 1128 MB + 1378.667 MB = 2506.667 MB
  • JVM Metaspace = 256 MB
  • JVM Overhead = 3000 MB - 2506.667 MB - 256 MB = 237.333 MB (within 192M ~ 1GB range)

5. Flink Size with Specific Heap and Managed Memory

Configuring Total Flink Size while explicitly setting JVM Heap and Managed Memory components:

taskmanager.memory.flink.size: 3000m
taskmanager.memory.task.heap.size: 1000m
taskmanager.memory.managed.size: 1000m
  • Total Flink Size = 3000 MB (configured)
  • Managed Memory = 1000 MB (configured)
  • Task Heap = 1000 MB (configured)
  • Framework Heap = 128 MB (default)
  • Total JVM Heap = Framework Heap + Task Heap = 128 MB + 1000 MB = 1128 MB
  • Total Off-Heap Memory = 3000 MB - 1128 MB = 1872 MB
  • Direct Memory = Total Off-Heap Memory - Managed Memory = 1872 MB - 1000 MB = 872 MB
  • Task Off-Heap = 0 MB (default)
  • Framework Off-Heap = 128 MB (default)
  • Network Memory = Direct Memory - Task Off-Heap - Framework Off-Heap = 872 MB - 0 MB - 128 MB = 744 MB
  • JVM Metaspace = 256 MB (default)
  • JVM Overhead calculation: (JVM Overhead + 3000 MB + 256 MB) × 0.1, resulting in JVM Overhead = 361.778 MB (within 192M~1G range)
  • Total Process Size = 3000 MB + 256 MB + 361.778 MB = 3617.778 MB

6. Memory Allocation Summary

Flink cluster memory allocation can be approached through 3 main strategies:

  • Specify Total Process Size or Total Flink Size, depending on deployment method
  • Configure individual components like Task-Heap size, allowing others to be derived automatically
  • Set Total Process / Flink Size && Heap or Off-Heap with remaining components filled via defaults or derivation:
    • Total Flink Size = Total Heap Size + Total Off-Heap Size
    • Total Heap Size = Task Heap + Framework Heap
    • Total Off-Heap = Task Off-Heap + Framework Off-Heap + Network Memory + Managed Memory
    • Network = Total Flink Size × 0.1 (when no other components specifide)
    • JVM Overhead = Total Process Size × 0.1 (when no other components specified)

Tags: Flink memory-management TaskManager configuration performance-tuning

Posted on Wed, 04 Nov 2026 16:17:50 +0000 by Dujo

Hot Tags

©2026 Freaks City