Introduction to ElasticSearch

Data base Query Challenges

1. Performance Issues: Wildcard searches (e.g., leading %) prevent index usage, leading to full table scans.
2. Limited Functionality: Unable to query compound terms like "Huawei phone" effectively.

Inverted Index

An inverted index maps terms to document IDs. For example:

• Forward index: Document → "Moonlight before my bed" → "before"
• Inverted index: "before" → "Moonlight before my bed" → Document

Text is tokenized into terms, enabling term-based document retrieval.

ElasticSearch Storage and Query Principles

• Index: Analogous to a database.
• Mapping: Defines field types and analyzers (similar to table schema).
• Document: Basic data unit stored as JSON (equivalent to a table row).

Core Concepts

• ElasticSearch: Distributed, scalable search engine based on Lucene.
• Features: Real-time search, analytics, RESTful API.
• Use Cases: Large-scale data search, log analysis, real-time analytics.

Installation and Setup

ElasticSearch Installation

Refer to installation guide. Verify with:

ps -ef | grep elastic

Kibana Installation

Refer to installation guide. Start in background:

nohup ../bin/kibana &

Core Components

• Index: Container for data.
• Mapping: Schema defining field properties.
• Document: JSON data record.
• Type: Deprecated; default is _doc in ES 7.x.

RESTful Operations

Index Management

• Create Index:

PUT /index_name

• Query Index:

GET /index_name
GET /index1,index2
GET /_all

• Delete Index:

DELETE /index_name

• Open/Close Index:

POST /index_name/_close
POST /index_name/_open

Data Types

• Primitive Types:
  • Strings: text (tokenized), keyword (non-tokenized).
  • Numeric, boolean, binary, range, date.
• Complex Types:
  • Arrays: nested.
  • Objects: object.

Mapping Operations

Define Mapping:

PUT /index_name
{
  "mappings": {
    "properties": {
      "name": { "type": "text" },
      "age": { "type": "integer" }
    }
  }
}

Add Field:

PUT /index_name/_mapping
{
  "properties": {
    "new_field": { "type": "keyword" }
  }
}

Document Operations

• Create Document (ID Specified):

POST /index_name/_doc/id
{ "field": "value" }

• Create Document (Auto ID):

POST /index_name/_doc/
{ "field": "value" }

• Query Documents:

GET /index_name/_search

• Delete Document:

DELETE /index_name/_doc/id

IK Analyzer

Installation

Install from GitHub. Configure Maven mirror if needed.

Usage

• ik_max_word: Fine-grained tokenization.

  GET /_analyze
  {
    "analyzer": "ik_max_word",
    "text": "Table tennis next year champion"
  }
  

• ik_smart: Coarse-grained tokenization.

  GET /_analyze
  {
    "analyzer": "ik_smart",
    "text": "Table tennis next year champion"
  }
  

Query with IK Analyzer

1. Create Index with Custom Analyzer:

PUT /index_name
{
  "mappings": {
    "properties": {
      "address": {
        "type": "text",
        "analyzer": "ik_max_word"
      }
    }
  }
}

2. Insert Documents:

POST /index_name/_doc
{ "address": "Beijing Haidian District" }

3. Term Query:

GET /index_name/_search
{
  "query": {
    "term": {
      "address": "Beijing"
    }
  }
}

4. Match Query (Full-Text):

GET /index_name/_search
{
  "query": {
    "match": {
      "address": "Beijing Changping"
    }
  }
}

Java API Integration

Spring Boot Setup

Dependencies:

<dependency>
  <groupId>org.elasticsearch.client</groupId>
  <artifactId>elasticsearch-rest-high-level-client</artifactId>
  <version>7.4.0</version>
</dependency>

Configuration:

@Configuration
@ConfigurationProperties(prefix = "elasticsearch")
public class ElasticSearchConfig {
  private String host;
  private int port;

  @Bean
  public RestHighLevelClient client() {
    return new RestHighLevelClient(
      RestClient.builder(new HttpHost(host, port, "http"))
    );
  }
}

Index Operations

Create Index:

CreateIndexRequest request = new CreateIndexRequest("index_name");
CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);

Create Index with Mapping:

CreateIndexRequest request = new CreateIndexRequest("index_name");
request.mapping("{ \"properties\": { ... } }", XContentType.JSON);

Query Index:

GetIndexRequest request = new GetIndexRequest("index_name");
GetIndexResponse response = client.indices().get(request, RequestOptions.DEFAULT);

Delete Index:

DeleteIndexRequest request = new DeleteIndexRequest("index_name");
AcknowledgedResponse response = client.indices().delete(request, RequestOptions.DEFAULT);

Index Existence Check:

GetIndexRequest request = new GetIndexRequest("index_name");
boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);

Document Operations

Insert Document (Map):

Map<String, Object> data = new HashMap<>();
data.put("name", "John");
IndexRequest request = new IndexRequest("index_name").id("1").source(data);
IndexResponse response = client.index(request, RequestOptions.DEFAULT);

Insert Document (POJO):

Person person = new Person("1", "John", 30);
String json = JSON.toJSONString(person);
IndexRequest request = new IndexRequest("index_name").id("1").source(json, XContentType.JSON);

Update Document:

Person updated = new Person("1", "John Updated", 31);
String json = JSON.toJSONString(updated);
IndexRequest request = new IndexRequest("index_name").id("1").source(json, XContentType.JSON);

Query Document by ID:

GetRequest request = new GetRequest("index_name", "1");
GetResponse response = client.get(request, RequestOptions.DEFAULT);
String source = response.getSourceAsString();

Delete Document:

DeleteRequest request = new DeleteRequest("index_name", "1");
DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);