As data analysis requirements continue to grow, efficient querying and analysis of large datasets has become increasingly important. Easysearch, a powerful domestic search and analytics engine, serves as a native alternative to Elasticsearch. It supports native DSL query syntax and SQL queries, ensuring seamless migration of existing business code without modifications. Easysearch is compatible with ES 7.x existing SDKs and index storage formats, supporting hot-cold architectures and index lifecycle management, providing users with comprehensive data processing solutions. This article will detail how to interact with Easysearch using the ES 7.x Python SDK, covering installation, connection, data operations, and querying.

1. Installing the Elasticsearch Python Client

---

To use the Elasticsearch Python client, you first need to install it via pip. Open your terminal or command prompt and run:

pip install elasticsearch==7.13.1

If you install the default version, it will install 8.x dependencies, which might cause the error elasticsearch.UnsupportedProductError: The client noticed that the server is not Elasticsearch and we do not support this unknown product.

Since Elasticsearch 7.10.2 changed its licensing model, introducing Server Side Public License (SSPL) and Elastic License, many search engines derived from Elasticsearch 7.10.2 require using 7.x version SDKs and agents, including the entire Beats family.

Here's a demo for retrieving cluster information, using es.cluster.health() to call Elasticsearch's cluster health check API, returning the cluster's health status.

Since we're using self-signed certificates, we add the verify_certs=False parameter during initialization, and use warnings.filterwarnings("ignore") to set Python's warning system to ignore all warnings. This is generally not recommended in production code as it hides potential issues, but in development or testing environments, it might be temporarily used if warning messages interfere with debugging.

import urllib3
import elasticsearch
from elasticsearch import Elasticsearch
import warnings
from pprint import pprint

# Disable all warnings
warnings.filterwarnings("ignore")

print(elasticsearch.VERSION)
# Disable warnings
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

server_url = "https://ip:9200/"
credentials = ('user', 'passwd')

# Establish connection
es_client = Elasticsearch(
    [server_url],
    http_auth=credentials,
    verify_certs=False,
)

# Check cluster health status
health_status = es_client.cluster.health()
pprint(health_status)

2. Preparing Sample Data

---

Before querying, we need to create some sample data in Easysearch.

# Define sample data
sample_records = [
    {"index": {"_index": "sample_index"}},
    {"field": "value1", "another_field": 10},
    {"index": {"_index": "sample_index"}},
    {"field": "value2", "another_field": 20},
    {"index": {"_index": "sample_index"}},
    {"field": "value3", "another_field": 30},
    {"index": {"_index": "sample_index"}},
    {"field": "bulk_value1", "another_field": 100},
    {"index": {"_index": "sample_index"}},
    {"field": "bulk_value2", "another_field": 200},
    {"index": {"_index": "sample_index"}},
    {"field": "bulk_value3", "another_field": 300}
]

# Bulk insert sample data
insert_response = es_client.bulk(body=sample_records)
print(insert_response)

3. Querying via REST API

---

REST API is a common way to communicate with Easysearch. Through REST API, developers can send HTTP requests to perform various operations, including indexing documents and searching data. The following example shows how to execute REST queries in Python.

Since it's a REST API, we can first test with Postman.

We can see the HTTP endpoint returns normally, then we can access it programmatically:

import requests
from requests.auth import HTTPBasicAuth
from pprint import pprint

server_url = "https://ip:9200/"
credentials = ('user', 'passwd')

# Build query parameters
search_query = {
    "query": {
        "match": {
            "field": "value1"
        }
    }
}

dsl_endpoint = f"{server_url}/sample_index/_search"

response = requests.get(dsl_endpoint, json=search_query, auth=HTTPBasicAuth(*credentials), verify=False)
pprint(response.json())

# Process query results
if response.status_code == 200:
    results = response.json()
    for hit in results['hits']['hits']:
        print(hit)
else:
    print(f"Error: {response.status_code}")

4. CRUD Operations on Index Data via DSL

---

DSL (Domain-Specific Language) is Easysearch's native query language, allowing users to build complex queries. Here are some examples:

# Build DSL query
dsl_search = {
    "query": {
        "match": {
            "field": "value1"
        }
    }
}

# Execute DSL query
response = es_client.search(index="sample_index", body=dsl_search)

results = response.get("hits")
# Process query results
if results:
    for hit in results['hits']:
        print(hit)
else:
    print(f"Error: {response.status_code}")

Inserting Data

If you don't specify a document ID, one will be randomly generated and written.

document = {"field": "value4", "another_field": 9999}
response = es_client.index(index="sample_index", body=document)
print(response)

Updating Data

Manually update the index by specifying ID as 1:

document = {"field": "value4", "another_field": 9999}
response = es_client.index(index="sample_index", body=document, id=1)
print(response)

Updating a Single Document

# Update a single document
update_payload = {"doc": {"another_field": 50}}
response = es_client.update(index="sample_index", id="1", body=update_payload)
pprint(response)

Deleting Data

# Delete a single document
response = es_client.delete(index="sample_index", id="1")
pprint(response)

5. SQL Queries on Index Data

---

After creatnig the client instance, we can use the sql method to execute SQL queries. The following example shows how to execute a simple SELECT query.

# Execute SQL query
sql_query = {
    "query": "SELECT * FROM sample_index"
}

result = es_client.sql.query(body=sql_query)
pprint(result)

6. Bulk Operations on Index Data

---

The Bulk API allows users to perform create, update, or delete operations on multiple documents at once, significantly improving operational efficiency. Here are some examples:

Bulk Insert Data

# Define bulk insert data
bulk_records = [
    {"index": {"_index": "sample_index"}},
    {"field": "bulk_value1", "another_field": 100},
    {"index": {"_index": "sample_index"}},
    {"field": "bulk_value2", "another_field": 200},
    {"index": {"_index": "sample_index"}},
    {"field": "bulk_value3", "another_field": 300}
]

# Execute bulk insert operation
response = es_client.bulk(body=bulk_records)
pprint(response)

Bulk Update Data

# Define bulk update data
bulk_update_records = [
    {"update": {"_id": "1", "_index": "sample_index"}},
    {"doc": {"another_field": 110}},
    {"update": {"_id": "2", "_index": "sample_index"}},
    {"doc": {"another_field": 220}}
]

# Execute bulk update operation
response = es_client.bulk(body=bulk_update_records)
pprint(response)

Bulk Delete Data

# Define bulk delete data
bulk_delete_records = [
    {"delete": {"_id": "1", "_index": "sample_index"}},
    {"delete": {"_id": "2", "_index": "sample_index"}}
]

# Execute bulk delete operation
response = es_client.bulk(body=bulk_delete_records)
print(response)

7. Index-Level Operations

---

Next, we'll cover index creation, deletion, and checking if an index exists. Here are some examples:

Creating an Index

# Create index
index_config = {
    "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 0
    },
    "mappings": {
        "properties": {
            "field": {"type": "text"},
            "another_field": {"type": "integer"}
        }
    }
}
response = es_client.indices.create(index="new_index", body=index_config)
pprint(response)

Deleting an Index

# Delete index
response = es_client.indices.delete(index="new_index")
pprint(response)

Checking if Index Exists

# Check if index exists
response = es_client.indices.exists(index="new_index")
pprint(response)

8. Summary

---

Although Easysearch doesn't have a dedicated Python SDK, it's fully compatible with the ES 7.x Python SDK client, which provides great convenience for developers. By using the ES 7.x Python SDK, developers can easily use DSL and SQL syntax to query and manipulate data in Easysearch. Easysearch's main advantages include:

1. Strong compatibility: No need to modify existing code to migrate from ES to Easysearch.
2. Comprehensive functionality: Supports advanced features like DSL queries, SQL queries, and bulk operations.
3. Easy to use: Provides clear and concise APIs, reducing learning curve.
4. High efficiency: Bulk operation APIs significantly improve data processing efficiency.

Easysearch combined with the powerful features of the ES 7.x Python SDK provides developers with an efficient and flexible big data processing platform. Whether executing simple SQL queries or building complex DSL queries, it can meet various data analysis needs. If you're looking for a powerful search and analytics solution, Easysearch is definitely worth trying. It not only helps you process and analyze large datasets more efficiently but also provides strong support for data-driven decision making.