Modern enterprises increasingly require powerful search and analytics capabilities to handle growing data volumes. Easysearch emerges as an enterprise-grade search engine that enables organizations to build high-performance, scalable data retrieval systems. In today's cloud computing landscape, containerization has become the standard approach for deploying and managing such solutions, with Amazon Elastic Kubernetes Service (EKS) providing a robust and user-friendly platform for running containerized applications.
This guide walks through the complete process of deploying Easysearch on AWS EKS, from initial cluster configuration to production-ready service deployment. Readers will learn how to leverage cloud resources effectively, utilizing the elasticity and scalability that cloud infrastructure provides.
Prerequisites
Before beginning the deployment, ensure you have the following preparations in place:
- An active AWS Global account with access to the Tokyo region (ap-northeast-1).
- An EKS cluster running version 1.30, along with the following command-line tools installed in you're Linux environment: AWS CLI, Helm, eksctl, and kubectl. This guide uses eksctl, the official AWS command-line utility for efficient EKS cluster management.
- The EBS-CSI-Driver configured as the storage provider for Easysearch persistent volumes.
- The AWS LoadBalancer Controller deployed to expose the Easysearch Console service via an AWS Network Load Balancer.
Installing Command-Line Tools
Begin by installing the necessary command-line tools on your Linux environment. These tools will be used throughout the deployment process to interact with AWS services and manage the Kubernetes cluster.
Installing AWS CLI:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install -i /usr/local/aws-cli -b /usr/local/bin
aws --version
Installing Helm:
curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 > get_helm.sh
chmod 700 get_helm.sh
./get_helm.sh
Installing eksctl:
# Set ARCH for your system architecture: arm64, armv6, or armv7 for ARM systems
ARCH=amd64
PLATFORM=$(uname -s)_$ARCH
curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
# Optional: Verify checksums
curl -sL "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_checksums.txt" | grep $PLATFORM | sha256sum --check
tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
sudo mv /tmp/eksctl /usr/local/bin
Installing kubectl:
curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/1.30.2/2024-07-12/bin/linux/amd64/kubectl
chmod +x ./kubectl
mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$HOME/bin:$PATH
Configuring the EKS Cluster Environment
This section demonstrates how to create and configure an EKS cluster using eksctl with a YAML configuration file. The configuration defines the cluster's VPC networking settings and worker node groups according to eksctl best practices.
Save the following configuration as my-cluster.yaml:
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: production-cluster
region: ap-northeast-1
vpc:
subnets:
private:
ap-northeast-1a: { id: subnet-private-1a-id }
ap-northeast-1c: { id: subnet-private-1c-id }
ap-northeast-1d: { id: subnet-private-1d-id }
nodeGroups:
- name: primary-workers
labels: { role: workers }
instanceType: m5.2xlarge
minSize: 2
maxSize: 4
desiredCapacity: 3
privateNetworking: true
volumeSize: 30
Create the cluster using the following command:
eksctl create cluster -f my-cluster.yaml
Once cluster creation completes, verify the cluster is ready by updating your kubeconfig and checking node status:
aws eks update-kubeconfig --name production-cluster --region ap-northeast-1
kubectl get nodes
You should see output similar to:
NAME STATUS ROLES AGE VERSION
ip-10-0-100-132.ap-northeast-1.compute.internal Ready <none> 16m v1.30.2-eks-1552ad0
ip-10-0-101-148.ap-northeast-1.compute.internal Ready <none> 16m v1.30.2-eks-1552ad0
The next step involves installing the EBS-CSI-Driver plugin, which enables dynamic volume provisioning using AWS EBS block storage for Easysearch workloads:
eksctl utils associate-iam-oidc-provider --region=ap-northeast-1 --cluster=production-cluster --approve
eksctl create iamserviceaccount \
--name ebs-csi-controller-sa \
--namespace kube-system \
--cluster production-cluster \
--region ap-northeast-1 \
--role-name AmazonEKS_EBS_CSI_DriverRole \
--role-only \
--attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
--approve
eksctl create addon --cluster production-cluster --name aws-ebs-csi-driver --version latest --region ap-northeast-1 \
--service-account-role-arn arn:aws:iam::112233445566:role/AmazonEKS_EBS_CSI_DriverRole --force
Verify the EBS-CSI-Driver pods are running correctly:
kubectl get pods -n kube-system | grep -i ebs
Expected output shows the driver components running across cluster nodes:
ebs-csi-controller-868598b64f-pwmxq 6/6 Running 0 11m
ebs-csi-controller-868598b64f-qn2lz 6/6 Running 0 11m
ebs-csi-node-fplxg 3/3 Running 0 11m
ebs-csi-node-v6qwj 3/3 Running 0 11m
Now install the AWS LoadBalancer Controller to manage Network Load Balancers for exposing services:
eksctl create iamserviceaccount \
--cluster=production-cluster \
--region ap-northeast-1 \
--namespace=kube-system \
--name=aws-load-balancer-controller \
--role-name AmazonEKSLoadBalancerControllerRole \
--attach-policy-arn=arn:aws:iam::112233445566:policy/AWSLoadBalancerControllerIAMPolicy \
--approve
helm repo add eks https://aws.github.io/eks-charts
helm repo update eks
wget https://raw.githubusercontent.com/aws/eks-charts/master/stable/aws-load-balancer-controller/crds/crds.yaml
kubectl apply -f crds.yaml
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=production-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller \
--set region=ap-northeast-1
Confirm the controller deployment is ready:
kubectl get deployment -n kube-system aws-load-balancer-controller
NAME READY UP-TO-DATE AVAILABLE AGE
aws-load-balancer-controller 2/2 2 2 39s
The EKS cluster environment is now fully configured and ready for Easysearch deployment.
Deploying Easysearch Services
This section covers deploying the Easysearch Console and cluster components. The Console provides a web-based management interface for your search infrastructure.
First, download and extract the Console Helm chart:
helm pull infinilabs/console
tar -zxvf console-0.2.0.tgz
cd console
The chart directory contains the following structure:
[user@host console]$ tree
.
├── Chart.yaml
├── templates
│ ├── NOTES.txt
│ ├── _helpers.tpl
│ ├── hpa.yaml
│ ├── ingress.yaml
│ ├── service.yaml
│ ├── serviceaccount.yaml
│ └── statefulset.yaml
└── values.yaml
Modify the service configuration to expose the Console via an internet-facing Network Load Balancer. Update service.yaml with the following annotations:
# service.yaml
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
service.beta.kubernetes.io/aws-load-balancer-subnets: subnet-private-1a-id, subnet-private-1c-id, subnet-private-1d-id
Update values.yaml to configure the service type and storage class:
# values.yaml
service:
type: LoadBalancer
storageClassName: gp2
Deploy the Console service using Helm:
kubectl create namespace easysearch
helm upgrade --install console . -f values.yaml -n easysearch
Verify the service was created and retrieve the load balancer DNS name:
kubectl get svc -n easysearch
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
console LoadBalancer 172.20.237.237 k8s-xxxx.elb.ap-northeast-1.amazonaws.com 9000:32190/TCP 6h49m
Next, deploy the Easysearch cluster with a single node configuration. Create a values file specifying the storage class and apply the necessary certificates:
cd ~
echo 'storageClassName: gp2' > custom-values.yaml
cat << EOF | kubectl apply -n easysearch -f -
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: easysearch-ca-issuer
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: easysearch-ca-certificate
spec:
commonName: easysearch-ca-certificate
duration: 87600h0m0s
isCA: true
issuerRef:
kind: Issuer
name: easysearch-ca-issuer
privateKey:
algorithm: ECDSA
size: 256
renewBefore: 2160h0m0s
secretName: easysearch-ca-secret
EOF
helm install easysearch infinilabs/Easysearch -n easysearch -f custom-values.yaml
The Easysearch deployment on AWS EKS is now complete. Access the Console using the load balancer DNS name, then configure a connecsion to the internal Easysearch service at easysearch.easysearch.svc.cluster.local:9200.
Verify connectivity by executing a curl command from within the Easysearch pod:
kubectl exec -n easysearch easysearch-0 -it -- curl -ku 'admin:admin' https://easysearch.easysearch.svc.cluster.local:9200
Successful verification returns cluster information:
{
"name" : "easysearch-0",
"cluster_name" : "infinilabs",
"cluster_uuid" : "fq3r_ZaHSFuZDjDtKyJY_w",
"version" : {
"distribution" : "Easysearch",
"number" : "1.6.0",
"distributor" : "INFINI Labs",
"build_hash" : "e5d1ff9067b3dd696d52c61fbca1f8daed931fb7",
"build_date" : "2023-09-22T00:55:32.292580Z",
"build_snapshot" : false,
"lucene_version" : "8.11.2",
"minimum_wire_lucene_version" : "7.7.0",
"minimum_lucene_index_compatibility_version" : "7.7.0"
},
"tagline" : "You Know, For Easy Search!"
}