Preparing the Calico Configuration

To begin the migration, download the Calico manifest that utilizes etcd as the datastore. This is common in custom-provisioned clusters where etcd is managed independently.

curl https://docs.projectcalico.org/manifests/calico-etcd.yaml -o calico-config.yaml

Configuring etcd Secrets and Endpoints

Calico needs to communicate with your etcd cluster securely. Encode your etcd CA, certificate, and private key into base64 format without line breaks:

cat /etc/kubernetes/pki/etcd/ca.pem | base64 -w 0
cat /etc/kubernetes/pki/etcd/server.pem | base64 -w 0
cat /etc/kubernetes/pki/etcd/server-key.pem | base64 -w 0

In the calico-config.yaml file, locate the Secret named calico-etcd-secrets and populate the fields. Ensure you uncomment the relevant lines:

data:
  etcd-key: <BASE64_ENCODED_KEY>
  etcd-cert: <BASE64_ENCODED_CERT>
  etcd-ca: <BASE64_ENCODED_CA>

Next, update the ConfigMap to specify where the certificates will be mounted inside the container and provide the etcd endpoints:

etcd_endpoints: "https://10.0.0.10:2379,https://10.0.0.11:2379,https://10.0.0.12:2379"
etcd_ca: "/calico-secrets/etcd-ca"
etcd_cert: "/calico-secrets/etcd-cert"
etcd_key: "/calico-secrets/etcd-key"

Aligning the Pod CIDR and Network Mode

Verify your cluster's Pod CIDR by checking the kube-controller-manager configuration. If your cluster uses 10.244.0.0/16, udpate the Calico environment variables accordingly. Additionally, determine if you want to use BGP or IPIP. To use pure BGP (performance-optimized), set CALICO_IPV4POOL_IPIP to Never.

- name: CALICO_IPV4POOL_CIDR
  value: "10.244.0.0/16"
- name: CALICO_IPV4POOL_IPIP
  value: "Never"

Decommissioning Flannel

Before deploying Calico, you must remove the Flannel resources and clean up the network stack on every node. Failure to do so will result in routing conflicts.

1. Delete Flannel Manifests:

kubectl delete -f kube-flannel.yaml

2. Clean Up Local Interfaces and Routes: Run these commands on all master and worker nodes to remove the bridge and tunnel interfaces created by Flannel:

# Remove Flannel and CNI bridges
ip link delete cni0
ip link delete flannel.1

# Remove stale routes (example CIDR)
ip route delete 10.244.0.0/24

Deploying Calico

Apply the modified configuration to the cluster:

kubectl apply -f calico-config.yaml

Monitor the deployment status:

kubectl get pods -n kube-system -l k8s-app=calico-node -o wide

Note: Existing Pods will not automatically transition to the Calico network. You must restart or recreate all application Pods to ensure they receive IP addresses from the Calico IPAM and follow the new routing logic.

Post-Migration Management with calicoctl

Install the calicoctl binary to manage advanced networking features:

curl -L https://github.com/projectcalico/calicoctl/releases/download/v3.20.0/calicoctl-linux-amd64 -o calicoctl
chmod +x calicoctl
mv calicoctl /usr/local/bin/

Create a configuration file at /etc/calico/calicoctl.cfg to point to your etcd cluster:

apiVersion: projectcalico.org/v3
kind: CalicoAPIConfig
metadata:
spec:
  datastoreType: "etcdv3"
  etcdEndpoints: "https://10.0.0.10:2379,https://10.0.0.11:2379,https://10.0.0.12:2379"
  etcdKeyFile: "/etc/kubernetes/pki/etcd/server-key.pem"
  etcdCertFile: "/etc/kubernetes/pki/etcd/server.pem"
  etcdCACertFile: "/etc/kubernetes/pki/etcd/ca.pem"

Verify node peering status:

calicoctl node status

Configuring BGP Route Reflectors (RR)

In large clusters (typically over 100 nodes), the default full-mesh BGP configuration becomes inefficient. You can switch to a Route Reflector model.

1. Disable Full Mesh: Create a BGPConfiguration resource:

apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  name: default
spec:
  nodeToNodeMeshEnabled: false
  asNumber: 64512

calicoctl apply -f bgp-config.yaml

2. Designate a Node as a Route Reflector: Label the target node and update its configuration to include a RouteReflectorClusterID:

kubectl label node node-01 infrastructure=route-reflector

Modify the node resource using calicoctl:

spec:
  bgp:
    ipv4Address: 10.0.0.11/24
    routeReflectorClusterID: 1.1.1.1

3. Configure Peerings: Define a BGPPeer resource to tell other nodes to connect to the reflector:

apiVersion: projectcalico.org/v3
kind: BGPPeer
metadata:
  name: peer-to-reflector
spec:
  nodeSelector: all()
  peerSelector: infrastructure == 'route-reflector'

Utilizing IPIP Mode

If your underlying network environment (e.g., certain VPCs) does not support BGP traffic between hosts, use IPIP encapsulation. This creates a tunl0 interface on each node.

To enable IPIP on an existing pool:

calicoctl get ippool -o yaml > pool.yaml
# Set ipipMode: Always in pool.yaml
calicoctl apply -f pool.yaml

Technical Comparison: Flannel vs. Calico

• Policy Enforcement: Calico supports Kubernetes NetworkPolicy and its own extended GlobalNetworkPolicy, whereas Flannel provides no native ACL or security policy support.
• Performance: Both Flannel (host-gw) and Calico (BGP) offer high performance by avoiding encapsulation when nodes are on the same Layer 2 segment. Calico is generally more scalable in Layer 3 environments.
• Encapsulation: Flannel primarily uses VXLAN. Calico offers IPIP or no encapsulation (BGP), providing flexibility based on cloud provider limitations.
• Complexity: Flannel is simpler to deploy but limited in features. Calico requires more management (e.g., BGP peering, etcd maintenance) but is suitable for complex, enterprise-grade multi-tenant environments.