Voir :

• https://linuxfr.org/news/decouverte-de-l-outil-de-supervision-prometheus
• https://www.scaleway.com/en/docs/configure-prometheus-monitoring-with-grafana/
• PromQL
• https://enix.io/fr/blog/comment-creer-exporter-prometheus/

Voir aussi :

• Prometheus VictoriaMetrics victoriadb
• Alertmanager (as part of Prometheus)
  • https://developers.redhat.com/articles/2023/10/04/5-steps-build-self-healing-server-alertmanager

Voir Identity correlation / CorrelationId

• https://en.wikipedia.org/wiki/Identity_correlation
• https://opentelemetry.io/
• https://grafana.com/oss/tempo/

#docker run -p 9090:9090 -v ~/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus
 
mkdir -p prometheus/nodes
sudo docker run -p 9090:9090 -v ~/prometheus:/etc/prometheus prom/prometheus

Config

prometheus.yml

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).
 
# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093
 
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"
 
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'
 
    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.
 
    # static_configs:
    # - targets: ['localhost:9090']
    
  - job_name: 'node'
    file_sd_configs:
        - files: [ "/etc/prometheus/nodes/*.yml" ]

prometheus/nodes/vm1.yml

- targets: [ "172.17.0.1:9100" ]
  labels:
      # Vous pouvez ajouter ce que vous voulez pour taguer la machine
      host: "vm1"

Créer un compte node_exporter

/etc/systemd/system/node_exporter.service

[Unit]
Description=node_exporter
Wants=network-online.target
After=network-online.target
 
[Service]
User=node_exporter
Group=node_exporter
Restart=on-failure
Type=simple
ExecStart=/usr/local/bin/node_exporter --collector.systemd --collector.ntp --collector.processes --collector.tcpstat
 
[Install]
WantedBy=multi-user.target

Prometheus

Voir

docker run -d --name=grafana -p 3000:3000 grafana/grafana

docker-compose.yml

version: '3.7'
services:
  grafana:
    image: grafana/grafana
    ports:
      - 3000:3000
	volume:
	  - "$PWD/data:/var/lib/grafana"

Run Grafana container with persistent storage (recommended)

Create a persistent volume for your data in /var/lib/grafana (database and plugins)

docker volume create grafana-storage

Start grafana

docker run -d -p 3000:3000 --name=grafana -v grafana-storage:/var/lib/grafana grafana/grafana

Run Grafana container using bind mounts

You may want to run Grafana in Docker but use folders on your host for the database or configuration. When doing so, it becomes important to start the container with a user that is able to access and write to the folder you map into the container.

mkdir data # creates a folder for your data
ID=$(id -u) # saves your user id in the ID variable
 
# starts grafana with your user id and using the data folder
docker run -d --user $ID --volume "$PWD/data:/var/lib/grafana" -p 3000:3000 grafana/grafana:7.2.1