Blackbox Alerts
Blackbox Alerts is a tool to monitor the health and availability of your endpoints. It uses the Blackbox exporter to probe endpoints and trigger alerts in Prometheus when issues are detected. This ensures you are immediately notified of any downtime or unresponsiveness, allowing for quick resolution and minimizing impact on your users.
Alert Configurations
The following YAML configuration defines the rules for triggering alerts based on the Blackbox exporter's data.
Blackbox Exporter Down
This alert triggers if the Blackbox exporter itself is down or not being scraped, indicating a critical monitoring failure.
Probe Failing
This alert triggers if a probe fails, indicating an unresponsive endpoint. This is a key indicator of potential service disruptions.
# references:
# https://medium.com/@yitaek/practical-monitoring-with-prometheus-grafana-part-ii-5020be20ebf6
groups:
- name: blackbox_alerts.yml
rules:
- alert: blackbox_exporter_down
expr: up{job="blackbox-exporter"} == 0
for: 5m
labels:
severity: warning
annotations:
summary: "Blackbox exporter is down"
description: "Blackbox exporter is down or not being scraped correctly"
- alert: probe_failing
expr: probe_success{job="blackbox-exporter"} == 0
for: 5m
labels:
severity: page
priority: P1
annotations:
summary: "Endpoints are down"
description: "Endpoint {{ $labels.instance }} is unresponsive for more than 5m"
Further Reading
For more information on Prometheus, Grafana, and the Blackbox exporter, refer to the following resources: