- Overview of Kubernetes Architecture
Kubernetes clsuters consist of two main types of nodes:
- Control Plane (Master):
- etcd: A distributed key-value store used for cluster state managemant.
- kube-apiserver: The entry point for all REST commands and cluster interactions.
- kube-controller-manager: Handles routine tasks such as replication and node monitoring.
- kube-scheduler: Assigns workloads to specific nodes based on resource availability.
- Worker Nodes:
- kubelet: The agent responsible for esnuring containers are running as expected.
- kube-proxy: Manages network rules and facilitates service communication.
- Cluster Bootstrapping and Setup
Container Runtime
Install Docker on all nodes and prepare the necessary pause image:
# yum install docker-1.12.rpm
# docker pull gcr.azk8s.cn/google_containers/pause-amd64:3.0
# systemctl start docker
State Store (etcd)
Configure the key-value database:
# yum install -y etcd
# vim /etc/etcd/etcd.conf
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.11:2379"
# systemctl start etcd.service
# etcdctl set name test
# etcdctl get name
# netstat -lntp | grep etcd
Master Node Configuration
Install and configure the control plane components:
# yum install kubernetes-master -y
# vim /etc/kubernetes/apiserver
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
KUBE_API_PORT="--port=8080"
KUBE_ETCD_SERVERS="--etcd-servers=http://10.0.0.11:2379"
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# vim /etc/kubernetes/config
KUBE_MASTER="--master=http://10.0.0.11:8080"
# systemctl start kube-apiserver.service
# systemctl start kube-controller-manager.service
# systemctl start kube-scheduler.service
# kubectl get cs
Worker Node Configuration
Join nodes to the cluster:
# yum install kubernetes-node -y
# vim /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=10.0.0.12"
KUBELET_HOSTNAME="--hostname-override=10.0.0.12"
KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"
# systemctl start kubelet.service
# systemctl start kube-proxy.service
Overlay Networking (Flannel)
Enable cross-host communication:
# yum install flannel -y
# vim /etc/sysconfig/flanneld
FLANNEL_ETCD_ENDPOINTS="http://10.0.1.11:2379"
FLANNEL_ETCD_PREFIX="/atomic.io/network"
# etcdctl mk /atomic.io/network/config '{ "Network": "172.16.0.0/16" }'
# systemctl start flanneld.service
# ifconfig flannel0
Private Registry
Set up a local Docker registry for images:
# docker run -d -p 5000:5000 --restart=always --name registry -v /opt/registry:/var/lib/registry registry
# vim /etc/sysconfig/docker
OPTIONS='... --insecure-registry=10.0.0.11:5000'
# docker tag nginx 10.0.0.11:5000/oldguo/nginx:v1
# docker push 10.0.0.11:5000/oldguo/nginx:v1
- Managing Pod Resources
Pods are defined using YAML manifests:
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: web
spec:
containers:
- name: nginx
image: 10.0.0.11:5000/oldguonginx:v1
ports:
- containerPort: 80
Common operations:
# kubectl create -f k8s_pod.yml
# kubectl get pod -o wide
# kubectl describe pod nginx
# kubectl delete pod nginx
# kubectl replace --force -f k8s_pod.yml
- ReplicationController (RC)
RC ensures a specific number of pod replicas are running:
apiVersion: v1
kind: ReplicationController
metadata:
name: nginx
spec:
replicas: 3
selector:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: 10.0.0.11:5000/oldguo/nginx:1.13
Scaling and updates:
# kubectl scale rc nginx --replicas=2
# kubectl rolling-update nginx -f nginx-1.15-rc.yaml --update-period=20s
- Deployment Management
Deployments offer advanced features like versioning and rollbacks:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: 10.0.0.11:5000/oldguo/nginx:1.13
ports:
- containerPort: 80
Rolling updates and history:
# kubectl set image deployment/nginx nginx=10.0.0.11:5000/oldguo/nginx:1.15
# kubectl rollout status deployment nginx
# kubectl rollout undo deployment/nginx --to-revision=1
- Service Discovery
Services expose applications via a stable Virtual IP (Cluster IP):
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
type: NodePort
ports:
- port: 80
nodePort: 30001
selector:
app: nginx
# kubectl get svc
# kubectl delete svc nginx
- Persistent Storage (PV and PVC)
Define storage volumes using NFS:
apiVersion: v1
kind: PersistentVolume
metadata:
name: test01-pv
labels:
pv: test01
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
nfs:
path: "/data"
server: 10.0.0.11
Claim the storage:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc_nfs
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
selector:
matchLabels:
pv: test01
# kubectl create -f pv-test01.yaml
# kubectl create -f pvc_nfs.yaml
# kubectl get pv
# kubectl get pvc
- Case Study: WordPress on K8s
Storage Preparation
Configure NFS shares on the master and mount them on nodes.
MySQL Setup
Create the RC and Service for the database, mounting the NFS volume:
apiVersion: v1
kind: ReplicationController
metadata:
name: mysql
spec:
replicas: 1
selector:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: 10.0.0.11:5000/mysql:5.7
ports:
- containerPort: 3306
volumeMounts:
- name: nfs-vol
mountPath: /var/lib/mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: "123456"
volumes:
- name: nfs-vol
persistentVolumeClaim:
claimName: pvc-mysql
Expose MySQL internally:
apiVersion: v1
kind: Service
metadata:
name: mysql
spec:
ports:
- port: 3306
selector:
app: mysql
WordPress Setup
Deploy the frontend application connecting to the MySQL Cluster IP:
apiVersion: v1
kind: ReplicationController
metadata:
name: myweb
spec:
replicas: 2
selector:
app: myweb
template:
metadata:
labels:
app: myweb
spec:
containers:
- name: myweb
image: 10.0.0.11:5000/wordpress:latest
ports:
- containerPort: 80
env:
- name: WORDPRESS_DB_HOST
value: '10.254.69.58'
volumeMounts:
- name: nfs-vol
mountPath: /var/www/html
volumes:
- name: nfs-vol
persistentVolumeClaim:
claimName: pvc-wp
Expose WordPress to the outside world:
apiVersion: v1
kind: Service
metadata:
name: myweb
spec:
type: NodePort
ports:
- port: 80
nodePort: 30008
selector:
app: myweb
Access the application at http://10.0.0.11:30008/