Blackbox Exporter enables black-box probing of targets via HTTP, HTTPS, DNS, TCP, ICMP, and gRPC.

1. Deploying Blackbox Exporter

• Deploy using the binary directly or via a container.
• When using a container, its recommended to mount the configuration file (/etc/blackbox/blackbox.yml) and expose port 9115.
• See the end of this article for a detailed configuration file explanation.

2. Adding Monitoring Targets in Prometheus

Add the following configuration to your prometheus.yml file:

  - job_name: 'blackbox'
    metrics_path: /probe
    params:
      # Module defined in blackbox_exporter
      module: [http_2xx]  # Look for a HTTP 200 response.
    static_configs:
      # Targets to monitor
      - targets:
        - http://prometheus.io    # Target to probe with http.
          labels:
            app: cmoon
            dev: prod
        - https://prometheus.io   # Target to probe with https.
          labels:
            app: cmoon-test
        - http://example.com:8080 # Target to probe with http on port 8080.
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        # Address of the blackbox_exporter service
        replacement: 192.168.2.23:9115  # The blackbox exporter's real hostname:port.

3. Common Metrics

4. Alert Rules Reference

Alert rules reference (to be defined based on your needs).

5. Blackbox Exporter Configuration File Explained

# Definition of each monitoring module
modules:
  # HTTP module, GET request
  http_2xx:
    prober: http
  # HTTP module, GET request with response validation
  http:
    valid_status_codes: []
    method: GET
    # If the response body contains "fail", the probe fails (probe_success = 0)
    fail_if_body_matches_regexp:
      - "#fail#"
    # If the response body does NOT contain "success", the probe fails (probe_success = 0)
    fail_if_body_not_matches_regexp:
      - "#SUCCESS#"    
  # HTTP module, POST request
  http_post_2xx:
    prober: http
    http:
      method: POST
  # HTTP module, POST request with request body
  http_post_2xx_request:
    prober: http
    timeout: 5s
    http:
      method: POST
      headers:
        Content-Type: application/json
      body: '{"app": "c-moon"}'
  tcp_connect:
    prober: tcp
  pop3s_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^+OK"
      tls: true
      tls_config:
        insecure_skip_verify: false
  ssh_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^SSH-2.0-"
  irc_banner:
    prober: tcp
    tcp:
      query_response:
      - send: "NICK prober"
      - send: "USER prober prober prober :prober"
      - expect: "PING :([^ ]+)"
        send: "PONG ${1}"
      - expect: "^:[^ ]+ 001"
  icmp:
    prober: icmp
    timeout: 5s
    icmp:
      preferred_ip_protocol: "ip4"