Model Definition

To implement a multilayer perceptron using PyTorch, we begin by importing the necessary libraries and modules.

import torch
import torch.nn as nn
from torch.nn import init
import numpy as np
import sys
sys.path.append("..")
import d2lzh_pytorch as d2l

The architecture differs from softmax regression by including an additional fully connected layer that serves as a hiddan layer. This hidden layer contains 256 units and employs the ReLU activation function.

input_size, output_size, hidden_size = 784, 10, 256

model = nn.Sequential(
    d2l.FlattenLayer(),
    nn.Linear(input_size, hidden_size),
    nn.ReLU(),
    nn.Linear(hidden_size, output_size)
)

# Initialize all parameters in the network
for parameter in model.parameters():
    init.normal_(parameter, mean=0, std=0.01)

In this implementation:

• model.parameters() provides an iterator over all trainable parameters (weights and biases) in the network
• init.normal_ performs in-place initialization, modifying the tensor directly rather than creating a new one
• The normal distribution with mean=0 and std=0.01 ensures small random values around zero, promoting faster convergence during training

Data Loading and Training Process

The data loading and training procedure follows a similar approach to what was used for softmax regression in section 3.7.

batch_size = 256
train_loader, test_loader = d2l.load_data_fashion_mnist(batch_size)

criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.5)

epochs = 5
d2l.train_ch3(model, train_loader, test_loader, criterion, epochs, batch_size, None, None, optimizer)

Training results:

epoch 1, loss 0.0030, train acc 0.712, test acc 0.744
epoch 2, loss 0.0019, train acc 0.823, test acc 0.821
epoch 3, loss 0.0017, train acc 0.844, test acc 0.842
epoch 4, loss 0.0015, train acc 0.856, test acc 0.842
epoch 5, loss 0.0014, train acc 0.864, test acc 0.818

Summary

PyTorch's high-level APIs enable a more concise implementation of multilayer perceptrons compared to manual implementations.