NOTE: This article assumes you are somewhat familiar with Prometheus and the node_exporter, and already have a working setup. Visit this tutorial if you don’t.
I’ve used Prometheus + Grafana to keep an eye on my servers and devices (a lot of things support node_exporter, like OpenWRT!) for quite a while now, but they’ve always been on my local network, or at least connected to my VPN subnet.
Recently, I’ve wanted to do the same for my VPS instances, but I didn’t like the idea of having Prometheus metrics on a public facing server for anyone to see. So it was with great pleasure that I found some documentation for node_exporter recently to add TLS and Basic Authentication.
However, doing this on multiple hosts manually can be tedious, as we need to install node_exporter, copy certificates to the host, and configure the systemd unit to use a config file for the Basic Authentication. Once we’ve set up one host, we’ll run through the Ansible playbook.
basic_auth_users: prom: $2b$12$MzUQjmLxPRM9WW6OI4ZzwuZHB7ubHiiSnngJxIufgZms27nw.5ZAq tls_server_config: cert_file: node_exporter.crt key_file: node_exporter.key
openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 \ -nodes -out node_exporter.crt -keyout node_exporter.key
Let’s put all of these files in the default node_exporter configuration directory -
Finally, to get node_exporter to read this configuration file, we need to tell systemd to add a command line argument when it starts node_exporter. We can do this in
/etc/default/prometheus-node-exporter, by making sure that the line that begins with
ARGS= is set to (depending on node_exporter version):
# node_exporter version < 1.5.0: ARGS="--web.config /etc/prometheus-node-exporter/web-config.yml" # node_exporter version >= 1.5.0: ARGS="--web.config.file /etc/prometheus-node-exporter/web-config.yml"
Now, if we restart our systemd unit for
systemctl restart prometheus-node-exporter) and visit
https://ip-address:9100/metrics, we should get the self-signed warning from our browser, which we can bypass, and then a Basic Auth prompt. I’ve used the username
prom and the password
node_exporter in this example.
To configure our Prometheus server, we need to add a new scrape_config job. This is how I define mine:
- job_name: 'prometheus_auth' scheme: https tls_config: insecure_skip_verify: true static_configs: - targets: ['22.214.171.124:9100'] - targets: ['example.com:9100'] basic_auth: username: prom password: node_exporter
As promised, here’s the Ansible playbook, along with a README with some instructions to get you started: