This document provides a comprehensive guide to configuring Promtail, a log collection agent for Loki. Proper configuration is essential for efficiently gathering, processing, and forwarding logs to your Loki instance.

Promtail Server and Client Settings

The server section defines the ports Promtail listens on for its own API and health checks. The positions section specifies where Promtail stores its state, crucial for resuming log collection after restarts. The clients array lists the Loki instances where logs will be pushed.

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

Syslog Log Collection

The scrape_configs section is where you define how Promtail discovers and collects logs. This example demonstrates configuring Promtail to scrape logs from a syslog source.

scrape_configs:
  - job_name: syslog
    syslog:
      listen_address: 0.0.0.0:1514
      labels:
        job: "syslog"
    relabel_configs:
      - source_labels: ['__syslog_connection_ip_address']
        target_label: 'instance'
      - source_labels: ['__syslog_message_app_name']
        target_label: 'app'
      - source_labels: ['__syslog_message_severity']
        target_label: 'severity'

Pipeline Stages for Log Processing

Promtail's pipeline stages allow for powerful log processing, including parsing, labeling, and metric generation. Here, we use match stages to apply different processing based on log content and labels.

    pipeline_stages:
      - match:
          selector: '{app="dockerd"}'
          stages:
            - regex:
                expression: "Health check for container (?P\\w+) (?P\\S+:).*"
            - labels:
                containerid:
                msglevel:
      - match:
          selector: '{severity="warning"}'
          stages:
            - metrics:
                warning_total:
                  type: Counter
                  description: "total count of warnings"
                  prefix: homepc_logs_
                  config:
                    match_all: true
                    action: inc

Advanced Relabeling and Metrics

Relabeling rules are essential for shaping your log data and adding meaningful labels. The metrics stage allows you to create Prometheus-compatible metrics from your logs, enabling advanced monitoring and alerting.

For more information on Promtail configuration, refer to the official Promtail documentation and the Loki configuration guide.