Centrum zbierani logów będzie kontener z aplikacją Loki. Będzie ona źródłem logów dla Grafana.
Aplikacja Loki
version: '3.8'
networks:
grafana-net:
name: grafana-net
driver: bridge
services:
grafana:
container_name: grafana
image: grafana/grafana:latest
networks: [grafana-net]
ports: ['3000:3000']
depends_on: [prometheus, loki]
environment: {}
volumes:
- grafana_storage:/var/lib/grafana
loki:
container_name: loki
image: grafana/loki:2.4.1
networks: [grafana-net]
ports: [3100:3100]
command: -config.file=/etc/loki/loki-config.yaml
volumes:
- all_config:/etc/loki
prometheus:
container_name: prometheus
image: prom/prometheus
networks: [grafana-net]
ports: [9090:9090]
command:
- '--config.file=/config/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/etc/prometheus/console_libraries'
- '--web.console.templates=/etc/prometheus/consoles'
#- '--storage.tsdb.retention=200h'
- '--web.enable-lifecycle'
volumes:
- all_config:/config
- prometheus_storage:/prometheus
volumes:
grafana_storage:
driver: local
name: grafana_storage
prometheus_storage:
driver: local
name: prometheus_storage
all_config:
driver: local
name: all_config
Konfiguracja Loki loki-config.yaml
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
common:
path_prefix: /tmp/loki
storage:
filesystem:
chunks_directory: /tmp/loki/chunks
rules_directory: /tmp/loki/rules
replication_factor: 1
ring:
instance_addr: 127.0.0.1
kvstore:
store: inmemory
schema_config:
configs:
- from: 2020-10-24
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
ruler:
alertmanager_url: http://localhost:9093
Przekierowanie logów z docker
Konfiguracja Docker PC
sudo docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
docker plugin ls
sudo nano /etc/docker/daemon.json
sudo systemctl restart docker
Zawartość pliku /etc/docker/daemon.json.
{
"log-driver": "loki",
"log-opts": {
"loki-url": "http://portainer-pc.lan:3100/loki/api/v1/push",
"loki-batch-size": "400"
}
}
Promtail
Jeśli chcemy monitorować systemowe pliki logów lub logi dokera to możemy uruchomić agenta, który monitoruje wybrany katalog i przesyła logi do Loki.
promtail:
image: grafana/promtail:2.4.0
container_name: promtail
ports: [1514:1514, 9080:9080]
command: -config.file=/etc/promtail/promtail-config.yml
volumes:
- /var/log:/var/log
- /var/lib/docker:/var/lib/docker
- all_config:/etc/promtail
Konfiguracja Promtail promtail-config.yml
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://portainer-pc.lan:3100/loki/api/v1/push
scrape_configs:
- job_name: local
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log
- job_name: docker
pipeline_stages:
- docker: {}
static_configs:
- labels:
job: docker
__path__: /var/lib/docker/containers/*/*-json.log
# syslog target
- job_name: syslog
syslog:
listen_address: 0.0.0.0:1514 # make sure you also expose this port on the container
idle_timeout: 60s
label_structured_data: yes
labels:
job: "syslog"
relabel_configs:
- source_labels: ['__syslog_message_hostname']
target_label: 'host'
Sprawdzenie
Sprawdzenie, czy Loki i Promtail działają poprawnie:
http://portainer-pc.lan:3100/metrics
http://portainer-pc.lan:3100/ready
http://portainer-pc.lan:9080/targets
Linki
- Techno Tim, Youtube (opens new window)
- Techno Tim, Blog (opens new window)
- computingforgeeks (opens new window)
============================================================