добавление ссылок в blackbox без изменения конфига prometheus или victoriametrics

Чтобы не править руками конфиг в prometheus, или victoria metrics, при добавлении урла в проверки blackbox-exporter, можно передавать эти урлы через consul.

Структура каталогов:

.
├── blackbox
│   └── blackbox.yml
├── consul
│   └── config.json
├── docker-compose.yml
└── victoriametrics
    └── prometheus.yml

docker-compose.yml:

---
services:
  victoriametrics:
    container_name: victoriametrics
    image: victoriametrics/victoria-metrics:v1.103.0
    ports:
      - 8428:8428
    volumes:
      - vmdata:/storage
      - ./victoriametrics/prometheus.yml:/etc/prometheus/prometheus.yml
    command:
      - "--storageDataPath=/storage"
      - "--graphiteListenAddr=:2003"
      - "--opentsdbListenAddr=:4242"
      - "--httpListenAddr=:8428"
      - "--influxListenAddr=:8089"
      - "--promscrape.config=/etc/prometheus/prometheus.yml"
    restart: always

  blackbox:
    image: prom/blackbox-exporter:v0.25.0
    container_name: blackbox
    ports:
    - 9115:9115
    command:
    - --config.file=/etc/blackbox/blackbox.yml
    volumes:
    - ./blackbox/blackbox.yml:/etc/blackbox/blackbox.yml

  consul:
    image: consul:1.15.4
    container_name: consul
    ports:
      - 8500:8500
    command:
      - agent
      - -config-dir=/etc/consul
      - -ui
    volumes:
      - ./consul:/etc/consul


volumes:
  vmagentdata: {}
  vmdata: {}

./victoriametrics/prometheus.yml:

global:
  scrape_interval: 10s

scrape_configs:
  - job_name: 'vmagent'
    static_configs:
      - targets:
          - victoriametrics:8428
        labels:
          team: team

  - job_name: blackbox_all
    metrics_path: /probe
    params:
      module: [ http_2xx ]
    consul_sd_configs:
      - server: 'consul:8500'
        services:
          - web-server
        tags:
          - env:prod
    relabel_configs:
      - source_labels: [__meta_consul_service_metadata_url]
        target_label: __param_target
      - source_labels: [__meta_consul_service_metadata_url]
        target_label: instance
      - target_label: __address__
        replacement: blackbox:9115

./blackbox/blackbox.yml:

modules:
  http_2xx:
    prober: http
    http:
      preferred_ip_protocol: "ip4"

consul/config.json:

{
  "server": true,
  "client_addr": "0.0.0.0",
  "ports": {
    "dns": -1
  },
  "bind_addr": "0.0.0.0",
  "datacenter": "dc-test",
  "bootstrap_expect": 1,
  "advertise_addr": "127.0.0.1",
  "retry_join": ["127.0.0.1"],
  "disable_host_node_id": false,
  "telemetry": {
    "disable_hostname": true,
    "prometheus_retention_time": "1h"
  },
  "limits": {
    "http_max_conns_per_client": 1000
  },
  "disable_update_check": true,
  "log_json": true
}

Пример использования. Допустим хотим чтобы блекбокс мониторил два сайта: https: https://1.1.1.1 и https://google.com/test

Регистрируем ссылки в консуле:

curl --request PUT http://127.0.0.1:8500/v1/agent/service/register -d '{"Name": "web-server", "id" : "test-1.1.1.1", "Tags": ["env:prod"], "Meta": {"url": "https://1.1.1.1"}}'
curl --request PUT http://127.0.0.1:8500/v1/agent/service/register -d '{"Name": "web-server", "id" : "test-google", "Tags": ["env:prod"], "Meta": {"url": "https://google.com/test"}}'
curl --request PUT http://127.0.0.1:8500/v1/agent/service/register -d '{"Name": "web-server", "id" : "test-ya.ru", "Tags": ["env:test"], "Meta": {"url": "https://ya.ru/123"}}'

В примере видно, что опрашиваются ссылки у которых есть лейбл env:prod.

Убираем ссылки из консула и мониторинга:

curl --request PUT http://127.0.0.1:8500/v1/agent/service/deregister/test-1.1.1.1
curl --request PUT http://127.0.0.1:8500/v1/agent/service/deregister/test-google
curl --request PUT http://127.0.0.1:8500/v1/agent/service/deregister/test-ya.ru