System Architecture
The underlying infrastructure follows a layered design typical of distributed systems, separating registration, authentication, and gateway responsibilities to ensure modularity.
CI/CD Workflow Implementation
Automated delivery relies on a Jenkins-based pipeline that handles source control integration, build artifacts, containerization, and environment-specific deployments.
pipeline {
agent {
label 'jenkins-worker-1'
}
tools {
maven 'maven-standard-version'
jdk 'openjdk-17'
}
options {
ansiColor('256')
timestamps()
disableConcurrentBuilds(true)
logRotator(numToKeepStr: 5, artifactDaysToKeepStr: 4)
}
parameters {
string(name: 'BUILD_SOURCE_BRANCH', defaultValue: 'main', description: 'Target branch from version control')
choice(choices: ['development', 'production'], description: 'Select deployment target', name: 'DEPLOY_ENV')
choice(choices: ['dev-ns', 'prod-ns'], description: 'Kubernetes Namespace selection', name: 'TARGET_NS')
}
environment {
PROJECT_ID = 'pig-core-platform'
CONTAINER_HOST = 'registry.tke.com.cn'
PROJECT_REPOSITORY = 'microservices-demo'
CREDENTIAL_NAME = 'docker-secret-01'
}
stages {
stage('Source Retrieval') {
steps {
git branch: "${params.BUILD_SOURCE_BRANCH}", credentialsId: 'git-admin', url: 'https://gitlab.internal.org/platform/pig.git'
}
}
stage('Compilation & Packaging') {
steps {
sh 'mvn clean deploy -DskipTests=true -P production'
}
post {
success {
archiveArtifacts allowEmptyArchive: true, artifacts: '**/target/*.jar', fingerprint: true, followSymlinks: false
}
}
}
stage('Container Image Generation') {
steps {
echo 'Preparing docker image...'
sh '''
cd $WORKSPACE
docker build --no-cache -t ${CONTAINER_HOST}/${PROJECT_REPOSITORY}/core:${BUILD_NUMBER} ./core-module/
docker push ${CONTAINER_HOST}/${PROJECT_REPOSITORY}/core:${BUILD_NUMBER}
docker rmi ${CONTAINER_HOST}/${PROJECT_REPOSITORY}/core:${BUILD_NUMBER}
'''
}
}
stage('Infrastructure Update') {
when {
expression { currentBuild.result == null || currentBuild.result == 'SUCCESS' }
}
steps {
script {
if ("${params.DEPLOY_ENV}" == "production") {
echo 'Executing production update...'
sh """
cd $WORKSPACE
sed -i.bak 's/__NAMESPACE__/${params.TARGET_NS}/g; s/__REGISTRY__/${env.CONTAINER_HOST}/g' k8s-manifests/prod/*.yaml
/usr/local/bin/kubectl apply --kubeconfig=/etc/tke/production.conf -f k8s-manifests/prod/
"""
} else {
echo 'Executing development update...'
sh """
cd $WORKSPACE
sed -i.bak 's/__NAMESPACE__/${params.TARGET_NS}/g; s/__REGISTRY__/${env.CONTAINER_HOST}/g' k8s-manifests/dev/*.yaml
/usr/local/bin/kubectl apply --kubeconfig=/etc/tke/staging.conf -f k8s-manifests/dev/
"""
}
}
}
}
}
}
Network Topology Configuration
To enable hybrid cloud access, route rules are configured to direct traffic between public cloud VPCs and private IDC segments. Cloud subnets point routing tables toward the N2N endpoint servers, while internal network devices forward requests back through their respective gateways.
Cluster Initialization Strategy
- Resource Allocation: Define worker node CPU and memory specifications based on anticipated load curves before provisioning.
- Cluster Type Selection: Use Managed Kubernetes (TKE) with GlobalRouter mode for enhanced pod networking capabilities.
- Dependencies: Provision necessary auxiliary services including MySQL, Redis, and Message Queues within the same security zone as the compute nodes.
Service Deployment Configurations
Microservices must be launched in dependency order. Core identity services should precede API gateways and UI layers.
apiVersion: apps/v1
kind: Deployment
metadata:
name: pig-register-service
spec:
replicas: 2
selector:
matchLabels:
component: register-layer
template:
metadata:
labels:
component: register-layer
spec:
containers:
- name: core-container
image: registry.internal.io/demo/core-app:v1.2
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8848
protocol: TCP
env:
- name: DB_USERNAME
value: "admin_user"
- name: DB_PASSWORD
value: "secure_password_123"
livenessProbe:
httpGet:
path: /health
port: 8848
initialDelaySeconds: 45
periodSeconds: 15
readinessProbe:
httpGet:
path: /ready
port: 8848
initialDelaySeconds: 30
periodSeconds: 10
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "2000m"
memory: "2Gi"
imagePullSecrets:
- name: harbor-creds-prod
---
apiVersion: v1
kind: Service
metadata:
name: pig-reg-svc
spec:
type: LoadBalancer
selector:
component: register-layer
ports:
- port: 8848
targetPort: 8848
protocol: TCP
# Generic Template for Backend Modules
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{SERVICE_NAME}}-module
spec:
replicas: 1
selector:
matchLabels:
app: {{SERVICE_NAME}}
template:
metadata:
labels:
app: {{SERVICE_NAME}}
spec:
containers:
- name: backend-exec
image: {{IMAGE_REF}}:{{TAG_VERSION}}
imagePullPolicy: Always
ports:
- containerPort: 9000
resources:
limits:
cpu: "4000m"
memory: "8Gi"
volumeMounts:
- name: config-volume
mountPath: /etc/config
volumes:
- name: config-volume
configMap:
name: service-configmap
---
apiVersion: v1
kind: Service
metadata:
name: {{SERVICE_NAME}}-api
spec:
selector:
app: {{SERVICE_NAME}}
ports:
- port: 9000
targetPort: 9000
Web Interface Component
The frontend application requires specific timezone configuration to align logs with local times.
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-dashboard
spec:
template:
spec:
containers:
- name: dashboard-frontend
image: nginx:alpine
ports:
- containerPort: 80
livenessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 20
volumeMounts:
- name: time-sync
mountPath: /etc/timezone
volumes:
- name: time-sync
hostPath:
path: /usr/share/zoneinfo/UTC
---
apiVersion: v1
kind: Service
metadata:
name: ui-gateway
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 31000
Auto-scaling Implementation
Horizontal Scaling
Adjust replica counts dynamically based on resource consumption thresholds.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: scalable-backend
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: scalable-backend
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: AverageValue
averageValue: 512Mi
Vertical Scaling
Configure the controller to optimize individual Pod resource requests over time.
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: resource-advisor
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: scalable-backend
updatePolicy:
updateMode: "Off"
Monitoring Integration
For enhanced cluster observability, connect external management dashboards such as Kuboard directly to the TKE API server endpoints.