Grafana on Azure – Enabling SSL with LetsEncrypt

This is part of a series of posts about running Grafana on Azure. Checkout the others

What is LetsEncrypt?

LetsEncrypt.org is an initiative to promote sites using SSL. Regardless of whether there is a data you feel is critical, SSL should still be enabled. LetsEncrypt provides completely FREE SSL certificates that you can use on any domain. The certificates are limited to 30 days, which means that you need to regnerate the key regularly. However, there is a utility called “certbot” which will automate this for us.

There are multiple ways you can generate the certificates, but for this example, we’ll be use what is referred to as the HTTP Challenge method. It’s generally a little bit more secure to run the DNS Challenge method as this doesn’t rely on you opening up additional ports, however, we won’t be covering that method here as it specific to the DNS provider you’re using.

Step 1 – Setup your domain name

In order to get a valid certificate, we’ll need to have a domain name, this won’t work on an IP address.

Setup your domain name to point to the IP address of your grafana instance. To make sure that you’re not going to hit issues, I’d recommend checking using dig or nslookup to ensure that it’s correctly pointed. Also, try hitting the grafana using that domain on port 3000 (i.e. http://{domain}:3000/).

Step 2 – Open the required ports

You’ll need to open up ports in your Network Security group so that both LetsEncrypt can communicate on Port 80 (http) for the HTTP Challenge, and your users can communicate on Port 443 (https).

Navigate to your Virtual Machine in the Azure Portal, Click “Networking” then “Add Inbound Port Rule”.

For “Destination Port Ranges” enter both 80 and 443 separated by a comma e.g. 80,443, and click Save.

Step 3- Let Grafana Access the certificates on the filesystem

The grafana process will used the certificates as it has it’s own inbuilt HTTP server. Therefore, the process will need access to the location where the certificates are generated.

First, we’ll create a group that will provide access to our ssl certificates:

sudo groupadd sslcerts

Then we’ll create the directories for the certificates, and change the ownership to our newly created group, and modify some permissions

sudo mkdir /etc/letsencrypt
sudo mkdir /etc/letsencrypt/archive
sudo mkdir /etc/letsencrypt/live
sudo chown -R root:sslcerts /etc/letsencrypt/
sudo chmod 755 /etc/letsencrypt/archive
sudo chmod 755 /etc/letsencrypt/live

Finally, we’ll add grafana’s process user to our newly created group.

sudo usermod -G sslcerts -a grafana

Step 4 – Install and run LetsEncrypt Certbot

Next we’ll need to install the certbot which is the application that will communicate with LetsEncrypt. There is a package in APT, so that’s pretty easy.

sudo apt install -y certbot

Then we can run the tool in standalone mode.

sudo certbot certonly --standalone

You’ll need to give LetsEncrypt an email address, and agree to their terms, then add the domain you want to generate the certificate for. This will be the one you setup in Step 1, without https, etc.

Certbot will then setup a temporary http server running on port 80 that will allow LetsEncrypt’s servers to verify that you actually own the domain by literally just hitting the url.

Once that’s done, you’ll find that there is now a certificate file in the folders we created in step 3.

azureuser@grafana-azure:~$ sudo ls -hal /etc/letsencrypt/live
total 16K
drwx------ 3 root root 4.0K Mar 13 21:08 .
drwxr-xr-x 9 root sslcerts 4.0K Mar 13 21:08 ..
-rw-r--r-- 1 root root 740 Mar 13 21:08 README
drwxr-xr-x 2 root root 4.0K Mar 13 21:08 grafanablog.martinjt.me

You’ll notice that the certbot hasn’t honoured our changing of the directories groups, so we’ll need to rectify that. If we change it on the files, certbot will honor them on renewal.

Step 5 – Configure Grafana to use SSL

Open the grafana config for editing

sudo nano /etc/grafana/grafana.ini

Edit the following settings:

protocol = https
domain = <your-domain>
enforce_domain = true
root_url = https://<your-domain>
cert_file = /etc/letsencrypt/live/<your-domain>/fullchain.pem
cert_key = /etc/letsencrypt/live/<your-domain>/privkey.pem

Now restart the grafana service:

sudo systemctl restart grafana-server.service

You should now be able to access your service using:

https://<your-domain>:3000/
Using port 443 for https

by default, Grafana won’t be able to listen on port 443 due to restrictions in Linux. You’ll need to enable this using the following command:

sudo setcap 'cap_net_bind_service=+ep' /usr/sbin/grafana-server

Now we can edit our grafana.ini again:

http_port = 443

Then restart our service

sudo systemctl restart grafana-server.service

Now you should be able to access your grafana instance without the port:

https://<your-domain>

Additional Links

Setup Grafana on Ubuntu 18.04 with LetsEncrypt

29

Tags:

AzureObservabilityGrafanaLetsencrypt