engine_daemon_container_actions_seconds_bucket {action = "changes", le = "0.01"} 1
engine_daemon_container_actions_seconds_bucket {action = "changes", le = "0.025"} 1
engine_daemon_container_actions_seconds_bucket {action = "changes", le = "0.05"} 1
engine_daemon_container_actions_seconds_bucket {action = "changes", le = "0.1"} 1
In order for the docker daemon to apply the parameters, it must be restarted, which will lead to the fall of all containers, and when the daemon starts, the containers will be raised in accordance with their policy:
essh @ kubernetes-master: ~ $ sudo chmod a + w /etc/docker/daemon.json
essh @ kubernetes-master: ~ $ echo '{"metrics-addr": "127.0.0.1:9323", "experimental": true}' | jq -M -f / dev / null> /etc/docker/daemon.json
essh @ kubernetes-master: ~ $ cat /etc/docker/daemon.json
{
"metrics-addr": "127.0.0.1:9323",
"experimental": true
}
essh @ kubernetes-master: ~ $ systemctl restart docker
Prometheus will only respond to metrics on the same server from different sources. In order for us to collect metrics from different nodes and see the aggregated result, we need to put an agent collecting metrics on each node:
essh @ kubernetes-master: ~ $ docker run -d \
–v "/ proc: / host / proc" \
–v "/ sys: / host / sys" \
–v "/: / rootfs" \
–-net = "host" \
–-name = explorer \
quay.io/prometheus/node-exporter:v0.13.0 \
–collector.procfs / host / proc \
–collector.sysfs / host / sys \
–collector.filesystem.ignored-mount-points "^ / (sys | proc | dev | host | etc) ($ | /)"
1faf800c878447e6110f26aa3c61718f5e7276f93023ab4ed5bc1e782bf39d56
and register to listen to the address of the node, but for now everything is local, localhost: 9100. Now let's tell Prometheus to listen to agent and docker:
essh @ kubernetes-master: ~ $ mkdir prometheus && cd $ _
essh @ kubernetes-master: ~ / prometheus $ cat << EOF> ./prometheus.yml
global:
scrape_interval: 1s
evaluation_interval: 1s
scrape_configs:
– job_name: 'prometheus'
static_configs:
– targets: ['127.0.0.1:9090', '127.0.0.1:9100', '127.0.0.1:9323']
labels:
group: 'prometheus'
EOF
essh @ kubernetes-master: ~ / prometheus $ docker rm -f prometheus
prometheus
essh @ kubernetes-master: ~ / prometheus $ docker run \
–d \
–-net = host \
–-restart always \
–-name prometheus \
–v $ (pwd) /prometheus.yml:/etc/prometheus/prometheus.yml
prom / prometheus
7dd991397d43597ded6be388f73583386dab3d527f5278b7e16403e7ea633eef
essh @ kubernetes-master: ~ / prometheus $ docker ps \
–f name = prometheus
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7dd991397d43 prom / prometheus "/ bin / prometheus –c…" 53 seconds ago Up 53 seconds prometheus
1702 host metrics are now available:
essh @ kubernetes-master: ~ / prometheus $ curl http: // localhost: 9100 / metrics | grep -v '#' | wc -l
1702
out of all the variety, it is difficult to find the ones you need for everyday tasks, for example, the amount of memory used by node_memory_Active. There are metrics aggregators for this:
http: // localhost: 9090 / consoles / node.html
http: // localhost: 9090 / consoles / node-cpu.html
But it's better to use Grafana. Let's install it too, you can see an example:
essh @ kubernetes-master: ~ / prometheus $ docker run \
–d \
–-name = grafana \
–-net = host
grafana / grafana
Unable to find image 'grafana / grafana: latest' locally
latest: Pulling from grafana / grafana
9d48c3bd43c5: Already exists
df58635243b1: Pull complete
09b2e1de003c: Pull complete
f21b6d64aaf0: Pull complete
719d3f6b4656: Pull complete
d18fca935678: Pull complete
7c7f1ccbce63: Pull complete
Digest: sha256: a10521576058f40427306fcb5be48138c77ea7c55ede24327381211e653f478a