2019-04-12 : HAProxy Exposes a Prometheus Metrics Endpoint

See also

- https://www.haproxy.com/blog/haproxy-exposes-a-prometheus-metrics-endpoint/ - HAProxy Exposes a Prometheus Metrics Endpoint - :ref:prometheus`

Introduction

See also

HAProxy has added native support for Prometheus, allowing you to export metrics directly.

Metrics are a key aspect of observability, along with logging and tracing. They contain the data that inform you about the state of your systems, which in turn allows you to see patterns and make course corrections as needed. Metrics give you essential feedback about how well, or unwell, things are going:

  • Are customers using the new features ?

  • Did traffic rates drop after that last deployment ?

  • If there’s an error, how long has it been happening and how many customers have likely been affected ?

HAProxy currently provides exceptional visibility through its Stats page, which displays more than 100 metrics. It can be consumed as a CSV-formatted feed—although you can also use the Runtime API to export the data as JSON.

CSV is perhaps one of the easiest formats to parse and, as an effect, many monitoring tools utilize the Stats page to get near real-time statistics from HAProxy. In particular, many users have benefited from the HAProxy Exporter for Prometheus, which consumes the HAProxy Stats page and converts the data to Prometheus time series.

Starting in the 2.0-dev branch, you can compile HAProxy with native Prometheus support and expose a built-in Prometheus endpoint.

The new module can be found in the HAProxy source code, under the contrib directory .

HAProxy Enterprise users can begin using this feature today as it has been backported to version 1.9r1.

Having Prometheus support builtin means that you don’t need to run an extra exporter process. Enable the service in your HAProxy configuration file and you’ll be all set.

In this blog post, we’ll explain:

  • how to set up the metrics endpoint,

  • how to configure Prometheus to scrape it,

  • and offer some guidance on graphing the data and alerting on it.

The Prometheus Metrics Page

With traffic flowing through HAProxy, it becomes a goldmine of information regarding everything from request rates and response times to cache hit ratios and server errors. Graphing these statistics will help you identify when and where issues are happening and alerting will ensure that you’re notified when issues arise. Having this information on hand might even help you to quickly resolve that complaint at 2:27 a.m.!

Prometheus is especially helpful because it collects the metrics, stores them in its time-series database, and allows you to select and aggregate the data using its PromQL query language. It also integrates nicely with graphing tools like Grafana and the alerting tool Alertmanager.

The new HAProxy Prometheus exporter exposes all of the counters and gauges available from the Stats page. In fact, it exposes more than 150 unique metrics. However, the data is extracted directly from the running HAProxy process

Compiling HAProxy for Prometheus

See also

First, you’ll need to compile HAProxy with the Prometheus Exporter. You’ll find instructions in the README. The commands look like this on an Ubuntu server: sudo apt update sudo apt install -y git ca-certificates gcc libc6-dev liblua5.3-dev libpcre3-dev libssl-dev libsystemd-dev make wget zlib1g-dev git clone https://github.com/haproxy/haproxy.git cd haproxy make TARGET=linux2628 USE_LUA=1 USE_OPENSSL=1 USE_PCRE=1 USE_ZLIB=1 USE_SYSTEMD=1 EXTRA_OBJS=”contrib/prometheus-exporter/service-prometheus.o” sudo make install-bin

Note that when cloning from the git repository, you’re using the latest, cutting-edge code. You may not want to launch this into production until the official 2.0 release arrives this summer. If you previously installed HAProxy using your system’s package manager, execute these commands as well: sudo systemctl stop haproxy sudo cp /usr/local/sbin/haproxy /usr/sbin/haproxy sudo systemctl start haproxy

Afterwards, verify that the Prometheus Exporter has been included in the compiled executable: haproxy -vv