Easysearch includes native support for S3 storage, enabling users to create and restore snapshots directly to Amazon S3-compatible services without installing external plugins. This integration streamlines the backup and migration workflow for data managed within the cluster.

Prerequisite: Object Storage Setup

A compatible S3 service is required. For this example, MinIO, an open-source object storage system with S3-compatible APIs, is used.

Configure S3 Storage:

1. In your S3/MinIO management console, create a new bucket (e.g., es-backup-bucket).
2. Generate and note down the Access Key and Secret Key for programmatic access.

Configuring Easysearch for S3

To enable S3 snapshot functionality, update the Easysearch configuration files.

Update easysearch.yml: Add the endpoint configuration for your S3 service to the main configuration file.

s3.client.primary.endpoint: minio.local:9000
s3.client.primary.protocol: http

Important: After modifying easysearch.yml, a cluster restart is required for changes to take effect.

Store Credentials Securely: Use the keystore utility to securely add S3 access credentials instead of plain text in configuration files.

# Add the Access Key and Secret Key to the keystore
./bin/easysearch-keystore add s3.client.primary.access_key
./bin/easysearch-keystore add s3.client.primary.secret_key
# List stored keys to verify
./bin/easysearch-keystore list

Registering the S3 Snapshot Repository

Before creating snapshots, you must rgeister the S3 bucket as a snapshot repository within Easysearch. Execute the following command via the Console's Dev Tools or a REST client.

PUT /_snapshot/backup_repository
{
  "type": "s3",
  "settings": {
    "bucket": "es-backup-bucket",
    "compress": true
  }
}

Creatnig a Snapshot

To back up specific indices, execute a snapshot creation command. The following command creates a snapshot named daily_backup_1 for the logs-* and .infini_metrics-* index patterns.

PUT /_snapshot/backup_repository/daily_backup_1?wait_for_completion=true
{
  "indices": "logs-*, .infini_metrics-*",
  "ignore_unavailable": true
}

The wait_for_completion=true parameter makes the call synchronous, returning only after the backup is complete. You can verify the backup files have been created by checking the contents of your S3 bucket.

Restoring Data from a Snapshot

This section demonstrates restoring a deleted index from the S3 backup.

1. Verify and Delete a Target Index: First, check the index status and document count, then delete it.

# Check index info
GET /_cat/indices/.infini_metrics-0001?v

# Delete the index
DELETE /.infini_metrics-0001

2. Execute Snapshot Restoration: Restore the deleted index from the previously created snapshot.

POST /_snapshot/backup_repository/daily_backup_1/_restore?wait_for_completion=true
{
  "indices": ".infini_metrics-0001",
  "rename_pattern": "(.+)",
  "rename_replacement": "restored_$1"
}

This command restores the infini_metrics-0001 index and renames it to restored_.infini_metrics-0001.

3. Verify the Restoration: Confirm the index has been restored with the correct document count.

GET /_cat/indices/restored_.infini_metrics-0001?v