Kubernetes Environment Planning

Pod Network (podSubnet): 10.244.0.0/16
Service Network (serviceSubnet): 10.96.0.0/12

Lab Environment Specifications:
Operating System: CentOS 7.5
Configuration: 2GB RAM / 4 vCPU / 50GB HDD

Initializing the Kubernetes Cluster Environment

First, set up the production environment server. Later, node environments can be cloned from a created image.

1. Modify Hostname

hostnamectl set-hostname k8smaster1 && bash

2. Configure YUM Repository

wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
yum makecache

3. Disable Firewall

systemctl stop firewalld.service
systemctl disable firewalld.service
firewall-cmd --state
yum install -y iptables-services
service iptables stop && systemctl disable iptables
service iptables status
iptables -F

4. Disable SELinux

setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
getenforce
swapoff -a

5. Disable Swap Permanently

swapoff -a

To make this permanent, comment out the swap line in /etc/fstab by adding a # at the beginning of the line.

6. Configure Time Synchronization

yum install -y ntpdate ntp
ntpdate cn.pool.ntp.org
echo "* */1 * * * /usr/sbin/ntpdate cn.pool.ntp.org" >> /var/spool/cron/root
systemctl restart crond

7. Configure Hosts File

cat > /etc/hosts <<eof ::1="" eof="" k8smaster1="" k8snode1="" localhost="" localhost.localdomain="" localhost4="" localhost4.localdomain4="" localhost6="" localhost6.localdomain6=""></eof>

8. Modify Kernel Parameters

modprobe br_netfilter
echo "modprobe br_netfilter" >> /etc/profile
cat > /etc/sysctl.d/k8s.conf <<eof eof="" net.bridge.bridge-nf-call-ip6tables="1" net.bridge.bridge-nf-call-iptables="1" net.ipv4.ip_forward="1" sysctl=""></eof>

9. Configure Auto-loading Kernel Modules on Boot

cat> /etc/sysconfig/modules/br_netfilter.modules <<eof br_netfilter="" chmod="" eof="" modprobe=""></eof>

10. Configure Alibaba Cloud Repository for Kubernetes Components

cat > /etc/yum.repos.d/kubernetes.repo <<eof baseurl="https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/" enabled="1" eof="" gpgcheck="0" name="Kubernetes"></eof>

11. Install Base Software Packages

yum install -y device-mapper-persistent-data lvm2 wget net-tools nfs-utils gcc gcc-c++ make cmake libxml2-devel openssl-devel curl curl-devel unzip sudo ntp libaio-devel vim ncurses-devel autoconf automake zlib-devel python-devel epel-release openssh-server socat ipvsadm conntrack telnet

12. Install Docker (for building images via Dockerfile)

yum install yum-utils -y
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install docker-ce -y
systemctl enable docker --now

Configure Docker image accelerator and driver:

cat> /etc/docker/daemon.json <<eof docker="" eof="" restart="" systemctl=""></eof>

13. Install and Configure containerd

yum install containerd.io-1.6.6 -y
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml

Edit /etc/containerd/config.toml:

• Change SystemdCgroup = false to SystemdCgroup = true.
• Change sandbox_image = "k8s.gcr.io/pause:3.6" to sandbox_image="registry.aliyyun.com/google_containers/pause:3.7".

systemctl enable containerd --now

Configure containerd image accelerator (all k8s nodes):

mkdir -p /etc/containerd/certs.d/docker.io
cat>/etc/containerd/certs.d/docker.io/hosts.toml<<eof capabilities="["pull"]" containerd="" eof="" restart="" systemctl=""></eof>

14. Install Kubernetes 1.25 Packages

yum install -y kubelet-1.25.0 kubeadm-1.25.0 kubectl-1.25.0
systemctl enable kubelet

15. Set Container Runtime

crictl config runtime-endpoint /run/containerd/containerd.sock

16. Load Kubernetes and Calico Images

On both master and node, import the pre-downloaded Kubernetes v1.25.0 image package (e.g., k8s_1.25.0.tar.gz) and the Calico image package (calico.tar.gz).

ctr -n=k8s.io images import k8s_1.25.0.tar.gz
ctr -n=k8s.io images import calico.tar.gz
crictl images

Initialize the Kubernetes Cluster with Kubeadm

Power off the server, create a snapshot/image, then clone it for node setup.

Ensure services are running on the master:

systemctl restart docker
systemctl restart containerd

Generate a default kubeadm configuration:

kubeadm config print init-defaults > kubeadm-config.yaml

Customize the configuration file. Modify fields like imageRepository, set mode to ipvs for kube-proxy, and specify cgroupDriver as systemd.

cat> kubeadm-config.yaml <<eof advertiseaddress:="" apiversion:="" bindport:="" cgroupdriver:="" clusterconfiguration="" crisocket:="" eof="" imagerepository:="" initconfiguration="" ipvs="" k8smaster1="" kind:="" kubeadm.k8s.io="" kubelet.config.k8s.io="" kubeletconfiguration="" kubeproxy.config.k8s.io="" kubeproxyconfiguration="" kubernetesversion:="" localapiendpoint:="" mode:="" name:="" networking:="" noderegistration:="" podsubnet:="" registry.aliyuncs.com="" servicesubnet:="" systemd="" unix:=""></eof>

Initialize the cluster:

kubeadm init --config=kubeadm-config.yaml --ignore-preflight-errors=SystemVerification

Upon successful initialization, configure kubectl:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get nodes

To reset and re-initialize if needed:

kubeadm reset
rm -rf /root/.kube
rm -rf /etc/cni/net.d

Adding the First Worker Node

On the node server (k8snode1), set the hostname:

hostnamectl set-hostname k8snode1 && bash

Run the join command obtained from the master initialization output on the node:

kubeadm join 172.27.0.10:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:5d85ecacdc1520befa8fc33dfac4f16d03893888ba65949a02c293b26336efc4 \
        --ignore-preflight-errors=SystemVerification

Installing the Calico Network Plugin

Apply the Calico manifest on the master:

kubectl apply -f calico.yaml

Check the cluster status after a couple of minutes:

kubectl get pods -n kube-system
kubectl get nodes

Label the worker node:

kubectl label node k8snode1 node-role.kubernetes.io/worker=worker

Testing Pod Network Connectivity

Load a BusyBox image (v1.28) on all nodes:

ctr -n k8s.io images import busybox-1-28.tar.gz

On the master, run a test pod:

kubectl run test-busybox --image=busybox:1.28 --image-pull-policy=IfNotPresent --restart=Never --rm -it -- sh

Inside the pod, test network connectivity:

ping -c 4 www.baidu.com