Prerequisites

Before starting, ensure you're machines meet these requirements:

• One or more machines running CentOS 7.x (x86_64)
• Hardware: minimum 2GB RAM, 2 CPUs, 30GB disk space
• Network connectivity between all machines
• Internet access for pulling container images
• Swap disabled

Objectives

1. Install Docker and kubeadm on all nodes
2. Initialize the Kubernetes control plane
3. Deploy a container networking solution
4. Join worker nodes to the cluster

System Configuration

Set hostnames for each node:

hostnamectl set-hostname k8s-master
hostnamectl set-hostname k8s-node1

Disable the firewall:

systemctl stop firewalld
systemctl disable firewalld

Disable SELinux:

sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0

Disable swap (temporary and permanent):

swapoff -a
vim /etc/fstab

Add hostname-to-IP mappings to /etc/hosts:

cat <<EOF > /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.31.62 k8s-master
192.168.31.63 k8s-node1
EOF

Enable bridge-nf-call-iptables:

cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

Installing Docker, kubeadm, and kubelet

Docker Installation

Kubernetes uses Docker as its default container runtime. Install Docker:

wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-18.06.1.ce-3.el7
systemctl enable docker && systemctl start docker
docker --version

Configure Docker daemon with registry mirror and logging settings:

mkdir -p /etc/docker/
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=cgroupfs"],
  "registry-mirrors": ["http://f1361db2.m.daocloud.io"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m", "max-file": "3"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF

systemctl daemon-reload
systemctl restart docker

Note: When modifying kubelet's cgroup driver in /lib/systemd/system/kubelet.service.d/10-kubeadm.conf, it gets overwritten during kubeadm initialization, causing failures. It's better to adjust Docker's cgroup driver instead.

Adding the Kubernetes YUM Repository

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

Installing kubeadm, kubelet, and kubectl

yum install -y kubelet kubeadm kubectl
systemctl enable kubelet

Initializing the Kubernetes Control Plane

kubeadm init \
  --apiserver-advertise-address=192.168.124.195 \
  --image-repository registry.aliyuncs.com/google_containers \
  --kubernetes-version v1.14.0 \
  --service-cidr=10.1.0.0/16 \
  --pod-network-cidr=10.244.0.0/16

This initialization may take several minute depending on network conditions.

Since k8s.gcr.io is not accessible from within China, the --image-repository flag points to an Alibaba Cloud mirror.

After successful initialization, follow the on-screen instructions:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

For root users, alternatively:

export KUBECONFIG=/etc/kubernetes/admin.conf

Check node status:

kubectl get nodes

Output:

NAME     STATUS     ROLES    AGE     VERSION
master   NotReady   master   1h      v1.14.0

The node shows "NotReady" because no networking plugin is deployed yet.

Deploying a Container Network Interface (CNI) Plugin

Apply the Flannel CNI plugin:

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

If pulling images from quay.io is slow, replace the registry with mirrors:

quay.io/coreos/flannel:v0.10.0-s390x → quay-mirror.qiniu.com/coreos/flannel:v0.10.0-s390x
gcr.io/google_containers/kube-proxy → registry.aliyuncs.com/google_containers/kube-proxy

Verify cluster components:

kubectl get pods --all-namespaces

Output:

NAMESPACE     NAME                              READY     STATUS    RESTARTS   AGE
kube-system   coredns-78fcdf6894-4744c          1/1       Running   0          5m
kube-system   coredns-78fcdf6894-jbvhd          1/1       Running   0          5m
kube-system   kube-apiserver-master1            1/1       Running   0          5m
kube-system   kube-controller-manager-master1   1/1       Running   0          5m
kube-system   kube-flannel-ds-amd64-kp7cr       1/1       Running   0          11s
kube-system   kube-proxy-6778v                  1/1       Running   0          5m
kube-system   kube-scheduler-master1            1/1       Running   0          5m

kubectl get nodes

Output:

NAME      STATUS    ROLES     AGE       VERSION
master1   Ready     master    1h        v1.14.0

kubectl get cs

Output:

NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-0               Healthy   {"health": "true"}

If componnet health checks show "Unhealthy" with connection refused errors on ports 10251/10252, check the scheduler and controller-manager manifests in /etc/kubernetes/manifests/. Remove any --port=0 flags and restart kubelet:

sudo systemctl restart kubelet

In Kubernetes v1.16.3 and later, kubectl get cs returns <unknown> status, which is expected behavior.

Allowing Pod Scheduling on the Control Plane

By default, the master node has taints that prevent pod scheduling. To allow pods on the master:

kubectl taint nodes --all node-role.kubernetes.io/master-

Without this step, pod deployments fail with taint tolerance errors.

Joining Worker Nodes

On worker nodes, execute the join command provided after kubeadm init. If the join command is no longer available, regenerate it:

kubeadm token create --print-join-command

Copy the admin configuration from master to worker:

scp /etc/kubernetes/admin.conf root@node1:/etc/kubernetes/admin.conf

On the worker node:

sudo kubeadm join 192.168.124.195:6443 --token qn1tr1.3xpu3qhu7ettn4gv \
    --discovery-token-ca-cert-hash sha256:932f99ce7b294b63dd0f511da047c6ea9cf56fa9d8b4b1df9be70013b0c049c9

Verify on master:

kubectl get nodes

Output:

NAME      STATUS    ROLES     AGE       VERSION
master1   Ready     master    7m        v1.14.0
node1     Ready     <none>    18s       v1.14.0

Testing with a Simple Pod

kubectl run -i --tty test-box --image=busybox --restart=Never -- sh

Check pod status:

kubectl get pod --show-all -o wide
NAME      READY     STATUS      RESTARTS   AGE       IP           NODE
busybox   0/1       Completed   0          48s       10.244.1.2   node1

Token Management

Tokens expire after 24 hours. To create a never-expires token:

kubeadm token create --ttl 0

To retrieve the discovery token CA certificate hash:

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | \
    openssl rsa -pubin -outform der 2>/dev/null | \
    openssl dgst -sha256 -hex | sed 's/^.* //'

Additional Commands

View kubeadm configuration:

kubeadm config view

Reset the cluster:

kubeadm reset