Provisioning Grafana Datasources and Dashboards Automagically

Grafana released v5.0 at GrafanaCon last week. It is a significant update packed with mouth-watering features. I’ve been testing the beta for quite some time and was very impressed with the maturity already in Beta.

Most people will notice the New Dashboard Engine, UX, and themes as prominent visual updates. However, down the list of features which to be honest, I missed the first few times reading the release notes is an essential feature: Provisioning from Configuration. This small bullet point has a significant impact on DevOps teams which deploy Grafana with GitOps.

Previous to v5.0, we had to use the Grafana API to provision data-sources and dashboards. It was always a bit tricky as we had to ensure Grafana is first running, pass credentials to the service, then load the data-source and dashboards.

Configure Grafana Provisioning

Now, the new Grafana v5.0 provisioning system utilizes YAML config files. New in v5.0 is the /etc/grafana/provisioning directory. Here is where we can place datasource config files /etc/grafana/provisioning/datasource and dashboards /etc/grafana/provisioning/dashboards in their respective sub-folders.

I’ve updated the Docker Prometheus monitoring stack to take advantage of the new provisioning tool so you can see it in practice. Let’s start with looking at the new volume in the docker-compose.yml file:

Line 9 of the docker-compose file is mounting the local /grafana/provisioning directory containing both the datasource and dashboard directories into the Grafana container’s /etc/grafana/provisioning directory.

Next, we can configure a datasource. I have selected to create a Prometheus datasource as it is what is in use with the Docker monitoring project. Here we can define the name of the datasource, proxy settings, datasource URL, and basic auth settings. Line 50 is essential as if it is false, we can only edit the datasource via the config files and no longer via the GUI.

Here is the Prometheus datasource.yml file:

Once the datasource is available, we can then provision dashboards. Create a dashboard.yml file and drop it in the /etc/grafana/provisioning/dashboards directory. In this config file we point to the path to the dashboards which is used to import. The dashboards should be in JSON format.

Once we start the Grafana container, it will read the /etc/grafana/provisioning directory and make updates to either the datasource of dashboards based on the changes. Drop in additional datasource or dashboard config, or dashboard JSON files and restart. Yes. that is all that is needed to update or add new datasources or dashboards.

Summary

Now that we understand the mechanics on how to provision datasources and dashboards it requires just a bit more effort to integrate this into your GitOps workflows. For the Monitoring projects, I maintain for getting started with container monitoring it significantly reduces the amount of energy required to start a monitoring stack. I also have some production installations where this will be much easier to maintain new versions and updates.

Find out more about 56K.Cloud

We love Cloud, Containers, DevOps, and Infrastructure as Code. If you are interested in chatting connect with us on Twitter or drop us an email: info@56K.Cloud We hope you found this article helpful. If there is anything you would like to contribute or you have questions, please let us know!