Baked-in data security with ACLs

Security should be at the heart of the design of any software project, and Redpanda is no exception. Here we'll dive into authorization, one of the baked-in mechanisms that helps you make your Redpanda cluster more secure. If you want to know about authentication with mTLS and SASL, check out our docs on security. But if you want a demonstration of ACLs, follow along.

Note: System security requires attention to many domains, such as network configuration and organization-wide roles management. Here we only address the security of client access to data. Make sure to review all of your other security requirements on a regular basis.

Access control lists (ACLs) make sure your data is protected by granular, least-privilege policies. By following this guide, you will get to know how to set up Redpanda with ACLs, both on a conceptual level and on a technical level, through hands-on examples.

While authentication is about making sure that whatever client connects to your cluster is verified as a trusted client, authorization is about making sure that each client has access to exactly the data it should. ACLs are the main mechanism supported by Redpanda to manage permissions. Our implementation of ACLs is Kafka-compatible, too!

Access Control Lists (ACLs)

At a high level, ACLs specify what authenticated users can or cannot do.

ACLs are all about who from where can or cannot do what action to what object.

In ACL terminology, the entities accessing the resources are called principals. Principals are users (User:) that are allowed or denied access to resources. You can also specify from which hosts the principals are allowed or denied.

For more granular ACLs, you can allow or deny principals access to specific resources, such as specific topics, and for specific operations, such as "read", "write", or "describe". These ACLs really lock down access to your data so you can rest assured that no one is getting unauthorized access.

The fields and values you can use are:

In addition to other Kafka-compatible tools, ACLs are managed with the Redpanda rpk utility with the commands:

• Creating ACLs - rpk acl create
• Listing ACLs - rpk acl list
• Deleting ACLs - rpk acl delete

Securing your data with ACLs

Now we are going to create some ACLs so you get the feel of how it works. Make sure you have the latest version of Redpanda, or check out our Getting Started guides to set up an environment.

Creating granular ACLs with a single command

• Create an ACL allowing a user to perform all operations from all hosts to a topic named "pings":

rpk acl create \
  --allow-principal 'User:Charlie' \
  --allow-host '*' \
  --operation all \
  --topic pings 

rpk confirms that you created the ACL:

PRINCIPAL     HOST  RESOURCE-TYPE  RESOURCE-NAME  RESOURCE-PATTERN-TYPE  OPERATION  PERMISSION  ERROR
User:Charlie  *     TOPIC          pings          LITERAL                ALL        ALLOW             

You can also use a single command to create multiple ACLs, including ACLs that deny certain principals access to a resource and allow other principals access to the same resource:

rpk acl create \
--cluster \
--allow-host 168.72.98.52 \
--deny-host 169.78.31.9 \
--deny-principal 'User:david' \
--allow-principal 'User:Alex' \
--allow-principal 'User:Ben' \
--operation 'all' 

rpk confirms that you created multiple ACLs:

PRINCIPAL   HOST          RESOURCE-TYPE  RESOURCE-NAME  RESOURCE-PATTERN-TYPE  OPERATION  PERMISSION  ERROR
User:Ben    168.72.98.52  CLUSTER        kafka-cluster  LITERAL                ALL        ALLOW       
User:david  168.72.98.52  CLUSTER        kafka-cluster  LITERAL                ALL        ALLOW       
User:david  169.78.31.9   CLUSTER        kafka-cluster  LITERAL                ALL        DENY 

What ACLs do I have now?

Now let's see what the ACLs look like for a specific user:

rpk acl list --allow-principal 'User:david' --deny-principal 'User:david'

This list of ACLs is shown with all of their parameters:

PRINCIPAL   HOST          RESOURCE-TYPE  RESOURCE-NAME  RESOURCE-PATTERN-TYPE  OPERATION  PERMISSION  ERROR
User:david  169.78.31.9   CLUSTER        kafka-cluster  LITERAL                ALL        DENY
User:david  168.72.98.52  CLUSTER        kafka-cluster  LITERAL                ALL        ALLOW

Notice that you can choose which flags to use in specific situations.

Of course, you can list ACLs with other filters, like host, permission and resource.

You can check a list of possible flags with rpk acl list -h

If you want to see them all execute rpk acl list.

Cleaning up ACLs

rpk acl delete allows you to delete ACLs. It's important to note, however, that wildcard values such as any for operations and permissions, as well as * for hosts and resources may result in the deletion filters matching more than one ACL.

• Delete all ACLs for a specific user targeting a specific resource:

  rpk acl delete --allow-principal 'User:david' --deny-principal 'User:david' --cluster

  Redpanda will ask you for confirmation

  ? Confirm deletion of the above matching ACLs? (Y/n)

  All of the related ACLs are listed as deleted:

  DELETIONS
  =========
  PRINCIPAL   HOST         RESOURCE-TYPE  RESOURCE-NAME  RESOURCE-PATTERN-TYPE  OPERATION  PERMISSION  ERROR
  User:david  169.78.31.9  CLUSTER        kafka-cluster  LITERAL                ALL        DENY 

• Delete all ACLs granting a principal permissions to all operations from a host:

  rpk acl delete \
    --allow-principal 'User:Ben' \
    --allow-host 168.72.98.52 \
    --operation all

This is just a small example of what we can do with ACLs.

If you want to dive deeper, check our in-depth guide about ACLs.

We are moving fast to keep up with the demand for Redpanda, the intelligent data API, and security with ACLs is only one of the many features that we're working on.

Join our Slack community to stay up on the latest developments.