Opinions are mine

HashiCorp's Consul is a well known "Swiss Army knife" in the world of service oriented infrastructure and among its many features it also provides KV store capabilities along with an ACL system that allows for fine grained access control.

Regardless of the environment where Consul is ran, it's always a good idea to have the ACL system enabled as besides its obvious uses, it can be used for performance optimizations and easy data filtering. But if there is a need to protect an HTTP endpoint in an environment, it's highly likely that ACLs are already enabled in Consul anyway.

Nginx web server has ngx_http_auth_request_module that implements client authorization based on the result of a sub-request. The main requirement for the upstream used for a sub-request is that it has to return 2XX status code in case of a success and 401 or 403 in case of a failure (any other code considered an error).

Starting from Consul v1.0.0 the KV API semantics changed so that it would return 403 status code for a request to retrieve a key's value in case when key exists but the request does not provide an ACL token that grants access to it. In other words, you get 404 when key does not really exist and 403 when it exists but you do not have access to it.

At this point it should be obvious that this 2 behaviors can be nicely combined together:

server {

  location /protected/path {
    auth_request            /auth;
    proxy_pass              http://protected-upstream;

  location /auth {
    proxy_pass              http://localhost:8500/v1/kv/my-api-access/foo;
    proxy_method            GET;
    proxy_set_header        Content-Length "";
    proxy_set_header        X-Consul-Token $http_my_token; # optional
    proxy_pass_request_body off;


For simplicity we omit irrelevant configuration options and assume that Consul is accessed via plain HTTP over the loopback interface.

What this config does though is that when request to /protected/path comes it makes Nginx to make a GET request to Consul KV Store API while forwarding all the headers and discarding the body (so that it works with any methods, regardless of whether the original request has any data payload in the body or no). When talking to Consul API the common way to provide the access token via X-Consul-Token header. If the original request has it set and the given token is allowed to access my-api-access/foo in Consul's KV store API then the original request will be authorized and proxied to http://protected-upstream

We can hide away the fact that Consul is used for this and instead get the token for an arbitrary header like My-Token with

proxy_set_header        X-Consul-Token $http_my_token; # optional

In fact, we can derive the token itself along with the path in KV store from any part of the original request to the extent that is supported by Nginx.

In order to make this more efficient we even can apply proxy_cache along with proxy_cache_key and proxy_cache_valid to cache Consul responses and avoid re-authorization on every request (starting from Nginx v1.7.3).

Obviously, this approach is rather a "hack" or an "abuse" of APIs that were not designed to work in the given way but can be a practical solution in some cases as it allows to avoid the need to build or setup yet another authorization engine. Assuming that Consul is already in place and is available we just extend the use of its ACL system to by mapping protected HTTP endpoints to entries in KV store. This way we can reuse all existing mechanisms for setting up policies as well as generating and distributing tokens.

An extra nice takeaway that might not be immediately obvious is that Consul's ACL system is "prefix-based" (and from v1.4.0 there it's also possible to use concrete matches) and therefore given a carefully designed KV store structure can be used to grant access to entire set of APIs that share the common "prefix" or "dir" in KV store rather than on 1 by 1 basis.