Setting Up a Kubernetes Cluster with kubeadm

Infrastructure Overview

We will create a Kubernetes cluster using Vagrant. The environment consists of three CentOS-based virtual machines:

Vagrantfile Configuration

Below is the Vagrantfile used to define the virtual machines:

Vagrant.configure("2") do |config|
  config.vm.define "k8s-01" do |master|
    master.vm.box = "centos-base"
    master.vm.provider "virtualbox" do |vb|
      vb.memory = "2048"
      vb.cpus = 2
    end
    master.vm.network "private_network", ip: "192.168.33.11"
    master.vm.hostname = "master1"
    master.vm.provision "shell", path: "./init.sh"
  end

  config.vm.define "k8s-02" do |node1|
    node1.vm.box = "centos-base"
    node1.vm.network "private_network", ip: "192.168.33.12"
    node1.vm.hostname = "node1"
    node1.vm.provision "shell", path: "./init.sh"
  end

  config.vm.define "k8s-03" do |node2|
    node2.vm.box = "centos-base"
    node2.vm.network "private_network", ip: "192.168.33.13"
    node2.vm.hostname = "node2"
    node2.vm.provision "shell", path: "./init.sh"
  end
end

Initialization Script

The init.sh script performs the following tasks:

• Configures YUM repositories
• Installs system tools
• Configures time synchronization
• Adjusts shell history and system limits
• Disables firewalls and SELinux
• Disables swap
• Configures iptables and kernel modules
• Installs Docker and configures its daemon
• Installs Kubernetes components (kubelet, kubeadm, kubectl)

#!/bin/bash

# Configure YUM repository
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
yum makecache fast

# Install utilities
yum install -y gcc make autoconf vim sysstat net-tools iostat iftop iotp wget lrzsz lsof unzip openssh-clients ntpdate

# Configure timezone and sync time
timedatectl set-timezone Asia/Shanghai
ntpdate time.windows.com
if ! crontab -l | grep ntpdate &>/dev/null; then
    (echo "* 1 * * * ntpdate time.windows.com >/dev/null 2>&1"; crontab -l) | crontab
fi

# Shell history format
if ! grep HISTTIMEFORMAT /etc/bashrc; then
    echo 'export HISTTIMEFORMAT="%Y-%m-%d %H:%M:%S `whoami`"' >> /etc/bashrc
fi

# System limits
if ! grep "* soft nofile 65535" /etc/security/limits.conf &>/dev/null; then
cat >> /etc/security/limits.conf << EOF
* soft nofile 65535
* hard nofile 65535
* soft nproc 4096
* hard nproc 4096
EOF
fi

# Disable firewall
systemctl stop firewalld && systemctl disable firewalld

# Disable SELinux
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0

# Disable swap
swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab

# Enable iptables for bridged traffic
modprobe br_netfilter

# Load kernel modules
cat <<eof br_netfilter="" cat="" configuration="" daemon="" device-mapper-persistent-data="" docker="" docker-ce="" eof="" fast="" http:="" install="" lvm2="" makecache="" net.bridge.bridge-nf-call-ip6tables="1" net.bridge.bridge-nf-call-iptables="1" sysctl="" tee="" yum="" yum-config-manager="" yum-utils=""> /etc/docker/daemon.json
{
  "registry-mirrors": ["https://f2q9sv8j.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

systemctl enable docker
systemctl daemon-reload
systemctl restart docker

# Add Kubernetes repository
cat <<eof> /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

# Install Kubernetes components
setenforce 0
yum install -y kubelet kubeadm kubectl
systemctl enable kubelet && systemctl start kubelet
</eof></eof>

Initializing the Cluster

On the master node, run the following command to initialize the cluster:

kubeadm init --apiserver-advertise-address=192.168.33.11 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.23.3 \
--pod-network-cidr=10.244.0.0/16

After initialization, a kubeadm join command will be printed. Run this command on the worker nodes to join the cluster.

Deploying the Network

After joining all nodes, deploy the Flannel network:

kubectl apply -f kube-flannel.yml

Deploying Ingress Controller

Label a node to run the ingress controller:

kubectl label node node1 app=ingress

Modify the ingress-nginx.yaml file to enable host networking and node selector:

hostNetwork: true
nodeSelector:
    app: ingress
    kubernetes.io/os: linux

Apply the configuration:

kubectl apply -f ingress-nginx.yaml

Adding New Nodes

To add a new node, first ensure its hostname is resolvable in all nodes' /etc/hosts files. Then generate a new token and discovery hash on the master:

kubeadm token create
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

Run the generated kubeadm join command on the new node.

Allowing Pods on Master Node

By default, the master node does not run application pods. To allow scheduling on the master:

kubectl taint node master1 node-role.kubernetes.io/control-plane:NoSchedule-

Testing the Cluster

Create a sample deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

Expose the deployment as a service:

kubectl expose deployment nginx-deployment --port=80 --type=NodePort

Common kubectl Commands

Some frequently used kubectl commands include:

• kubectl get pods - List all pods
• kubectl describe pod <pod-name> - Show detailed info about a pod
• kubectl exec -it <pod-name> -- bash - Execute a command in a pod
• kubectl logs <pod-name> - View logs of a pod
• kubectl scale deploy/<deployment-name> --replicas=<count> - Scale replicas
• kubectl delete pod --all - Delete all pods