Consul KV Store ACLs as an authorization engine for Nginx

HashiCorp’s Consul is a well known “Swiss Army knife” in the world of service-oriented infrastructure. Among its many features, it also provides KV store capabilities along with an ACL system that allows for fine-grained access control.

Regardless of the environment where Consul is run, it’s always a good idea to have the ACL system enabled. Besides its apparent uses, it can be used for performance optimizations and easy data filtering.

Nginx web server has ngx_http_auth_request_module that implements client authorization based on the result of a sub-request. In a nutshell, the upstream used for a sub-request has to return a 2XX status code in case of success and 401 or 403 in case of a failure. Any other code considered an error.

Starting from Consul v1.0.0 the KV API semantics changed so that it would return 403 status code for a request to retrieve a key’s value in case when key exists, but the request does not provide an ACL token that grants access to it. In other words, you get 404 when the key does not exist and 403 when it exists, but you do not have access to it.

At this point, it should be obvious that those two behaviors can be nicely combined:

server {
  ...

  location /protected/path {
    auth_request            /auth;
    proxy_pass              http://protected-upstream;
  }

  location /auth {
    proxy_pass              http://localhost:8500/v1/kv/my-api-access/foo;
    proxy_method            GET;
    proxy_set_header        Content-Length "";
    proxy_set_header        X-Consul-Token $http_my_token; # optional
    proxy_pass_request_body off;
  }

}

For simplicity, we omit irrelevant configuration options and assume that Consul is accessed via plain HTTP over the loopback interface.

What this config does though is that when a request to /protected/path comes in, it makes Nginx to send a GET request to Consul KV Store API while forwarding all the headers and discarding its body. This way, it can work with any methods, regardless of whether the original request has any data payload or no. When talking to a Consul API, the standard way to provide an access token is via X-Consul-Token header. If the original request has it set and the given token is allowed to access my-api-access/foo in Consul’s KV store API, then the original request would be authorized and proxied to http://protected-upstream.

We can hide away the fact that Consul is used for this and instead get the token for an arbitrary header like My-Token with the following addition:

proxy_set_header        X-Consul-Token $http_my_token; # optional

In fact, we can derive the token itself along with the path in KV store from any part of the original request to the extent that is supported by Nginx.

To make this more efficient we even can apply proxy_cache along with proxy_cache_key and proxy_cache_valid to cache Consul responses and avoid re-authorization on every request (starting from Nginx v1.7.3).

This approach is rather a “hack” or an “abuse” of APIs that were not designed to work in the given way but can be a practical solution in some cases as it allows you to avoid the need to build or set up yet another authorization engine. Assuming that Consul is already in place and is available, we extend the use of its ACL system by mapping protected HTTP endpoints to entries in the KV store. We can reuse all existing mechanisms for setting up policies and generating and distributing tokens.

An extra takeaway that might not be immediately obvious is that Consul’s ACL system is “prefix-based” and therefore a carefully designed KV store structure can be used to grant access to a set of API endpoints that share a common “prefix” or a “dir” in KV store, rather than on one by one basis.