Understanding Execution Modes in AI Compilation Frameworks

Deep learning frameworks support two primary execution modes: dynamic graph and static graph. MindSpore defaults to dynamic graph mode but provides mechanisms to utilize static graph compilation for performance optimization.

Dynamic Graph Mode (PyNative)

Dynamic graph mode executes operations immediately as they are defined (Define by Run), aligning with Python's interpreted nature. Tensors are computed and values determined upon creation, facilitating debugging through immediate intermediate results. However, this approach limits computational graph optimization since all nodes must be preserved.

In MindSpore, dynamic graph mode is called PyNative mode and is recommended for development and debugging phases.

import numpy as np
import mindspore as ms
from mindspore import nn, Tensor

ms.context.set_context(mode=ms.context.PYNATIVE_MODE)

class NeuralNet(nn.Cell):
    def __init__(self):
        super().__init__()
        self.flat = nn.Flatten()
        self.layers = nn.SequentialCell(
            nn.Dense(784, 512),
            nn.ReLU(),
            nn.Dense(512, 512),
            nn.ReLU(),
            nn.Dense(512, 10)
        )
    
    def construct(self, data):
        data = self.flat(data)
        return self.layers(data)

net = NeuralNet()
sample_input = Tensor(np.ones([64, 1, 28, 28], dtype=np.float32))
result = net(sample_input)
print(result.shape)

Static Graph Mode (Graph)

Static graph mode separates graph construction from execution. MindSpore's Graph mode enables global optimizations through graph transformations and computational graph sinking, delivering superior performance for fixed network structures.

import numpy as np
import mindspore as ms
from mindspore import nn, Tensor

ms.context.set_context(mode=ms.context.GRAPH_MODE)

class NeuralNet(nn.Cell):
    def __init__(self):
        super().__init__()
        self.flat = nn.Flatten()
        self.layers = nn.SequentialCell(
            nn.Dense(784, 512),
            nn.ReLU(),
            nn.Dense(512, 512),
            nn.ReLU(),
            nn.Dense(512, 10)
        )
    
    def construct(self, data):
        data = self.flat(data)
        return self.layers(data)

net = NeuralNet()
sample_input = Tensor(np.ones([64, 1, 28, 28], dtype=np.float32))
result = net(sample_input)
print(result.shape)

Static Graph Application Scenarios

MindSpore's compiler optimizes Tansor operations and their gradients effectively in static graph mode. Non-Tensor operations compile but receive limited optimization. The compilation overhead makes static graph beneficial primarily for frequently executed functions.

Enabling Static Graph Execution

MindSpore provides two approaches for static graph activation: decorator-based and context-based configuration.

Decorator-Based Activasion

The @ms.jit decorator compiles specific functions into static graphs, enabling targeted acceleration while maintaining dynamic execution elsewhere.

import numpy as np
import mindspore as ms
from mindspore import nn, Tensor

class NeuralNet(nn.Cell):
    def __init__(self):
        super().__init__()
        self.flat = nn.Flatten()
        self.layers = nn.SequentialCell(
            nn.Dense(784, 512),
            nn.ReLU(),
            nn.Dense(512, 512),
            nn.ReLU(),
            nn.Dense(512, 10)
        )
    
    def construct(self, data):
        data = self.flat(data)
        return self.layers(data)

sample_input = Tensor(np.ones([64, 1, 28, 28], dtype=np.float32))

@ms.jit
def execute_network(x):
    model = NeuralNet()
    return model(x)

output = execute_network(sample_input)
print(output.shape)

Alternatively, use function transformation:

def execute_network(x):
    model = NeuralNet()
    return model(x)

jit_execute = ms.jit(execute_network)
output = jit_execute(sample_input)

For accelerating specific network components:

class NeuralNet(nn.Cell):
    def __init__(self):
        super().__init__()
        self.flat = nn.Flatten()
        self.layers = nn.SequentialCell(
            nn.Dense(784, 512),
            nn.ReLU(),
            nn.Dense(512, 512),
            nn.ReLU(),
            nn.Dense(512, 10)
        )
    
    @ms.jit
    def construct(self, data):
        data = self.flat(data)
        return self.layers(data)

net = NeuralNet()
output = net(sample_input)

Context-Based Activation

Global static graph configuration affects all operations within the context:

ms.context.set_context(mode=ms.context.GRAPH_MODE)
net = NeuralNet()
output = net(sample_input)

Static graph mode operates under Python syntax constraints maintained by MindSpore's compiler. Advanced programming techniques and JitConfig options provide additional control over compilation behavior.