Overview

This guide walks through setting up a centralized logging system using the ELK stack (Elasticsearch, Logstash, Kibana) with Docker Compose. The configuration collects appliaction logs, processes them via Logstash, stores them in Elasticsearch, and visualizes them in Kibana.

1. Define the docker-compose.yaml File

Create a docker-compose.yaml file that defines three services: Elasticsearch, Logstash, and Kibana.

version: '3'
services:
  elasticsearch:
    image: elasticsearch:7.6.2
    container_name: elasticsearch
    privileged: true
    user: root
    volumes:
      # Mount the ES security configuration file
      - ./es/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    environment:
      # Set the cluster name
      - cluster.name=elasticsearch
      # Start in single-node mode
      - discovery.type=single-node
      # JVM memory settings
      - ES_JAVA_OPTS=-Xms2048m -Xmx2048m
      # Elasticsearch password
      - ELASTIC_PASSWORD=d2d76300bba2ffe0088019a20e902c82
    ports:
      - 9200:9200
      - 9300:9300

  logstash:
    image: logstash:7.6.2
    container_name: logstash
    ports:
      - 4560:4560
    privileged: true
    environment:
      - TZ=Asia/Shanghai
    volumes:
      # Mount Logstash configuration files
      - ./logstash/logstash.conf:/usr/share/logstash/pipeline/logstash.conf
      - ./logstash/logstash.yml:/usr/share/logstash/config/logstash.yml
      # Mount the application log file (adjust path as needed)
      - /data/zhangc/podpartner/log/log.log:/usr/share/logstash/log/podpartner/log.log
    depends_on:
      - elasticsearch
    links:
      # Allow Logstash to reach Elasticsearch via the hostname 'es'
      - elasticsearch:es

  kibana:
    image: kibana:7.6.2
    container_name: kibana
    ports:
      - 5601:5601
    privileged: true
    volumes:
      # Mount Kibana configuration file
      - ./kibana/kibana.yml:/usr/share/kibana/config/kibana.yml
    links:
      # Allow Kibana to reach Elasticsearch via the hostname 'es'
      - elasticsearch:es
    depends_on:
      - elasticsearch
    environment:
      # Set the locale to Chinese (optional)
      - i18n.locale=zh-CN
      # Elasticsearch host for Kibana
      - elasticsearch.hosts=http://es:9200

2. Define the logstash.conf File

Create a logstash.conf file that specifies the input source and output destination.

input {
    file {
        path => "/log.log"
        type => "systemlog"
        start_position => "beginning"
        stat_interval => "5"
    }
}

output {
    elasticsearch {
        hosts => ["es:9200"]
        index => "logstash-%{+YYYY.MM.dd}"
    }
}

• path: The path to the log file to be collected.
• type: A label for the log type.
• start_position: Where to start reading the file (e.g., beginning).
• stat_interval: How often (in seconds) to check the file for new data.
• output.elasticsearch.hosts: The Elasticsearch host and port.
• output.elasticsearch.index: The index pattern for storing logs (daily rolling indices).

3. Start the Services

Run the following command to build and start all containers in detached mode:

docker-compose up -d

4. Configure Kibana (Language Settings)

To set Kibana's language to Chinese (optional), add the following line to /config/kibana.yml:

i18n.locale: "zh-CN"

After making the change, restart the Kibana container:

docker restart kibana

5. Password Setup (Reference)

If you need to set up password protection for Elasticsearch, refer to these resources (add you're own links as needed):

• Setting passwords for Docker Elasticsearch containers
• Elasticsearch, Logstash, Kibana password configuration guide