Kafka Security 101
Master the fundamentals of securing Apache Kafka. Learn authentication, authorization, encryption, and operational security practices to protect your streaming data infrastructure.
The Three Pillars of Kafka Security
Secure Kafka operations are built on three fundamental pillars: authentication (who are you?), authorization (what can you do?), and encryption (protecting data in transit and at rest). A comprehensive security strategy addresses all three.
Authentication
Verify the identity of clients and brokers using SASL or SSL certificates.
Authorization
Control what authenticated users can do with ACLs and RBAC.
Encryption
Protect data with TLS encryption in transit and at-rest encryption.
Authentication Methods
SASL/PLAIN
Simple username/password authentication. Easy to set up but credentials are sent in clear text—always use with TLS encryption.
# Client configuration
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="user" password="password";SASL/SCRAM
Salted Challenge Response Authentication Mechanism. More secure than PLAIN as passwords are never sent over the wire, even without TLS.
# Create SCRAM credentials
kafka-configs.sh --bootstrap-server localhost:9092 \
--alter --add-config 'SCRAM-SHA-256=[password=secret]' \
--entity-type users --entity-name aliceSASL/GSSAPI (Kerberos)
Enterprise-grade authentication using Kerberos. Ideal for organizations with existing Active Directory or MIT Kerberos infrastructure.
mTLS (Mutual TLS)
Certificate-based authentication where both client and server present certificates. No username/password management required.
# Client configuration
security.protocol=SSL
ssl.keystore.location=/path/to/client.keystore.jks
ssl.keystore.password=keystorepass
ssl.key.password=keypassSASL/OAUTHBEARER
OAuth 2.0 token-based authentication. Integrates with identity providers like Okta, Auth0, or Keycloak for modern authentication workflows.
Authorization with ACLs
Kafka uses Access Control Lists (ACLs) to determine what operations authenticated users can perform on resources like topics, consumer groups, and clusters.
ACL Resource Types
| Resource | Operations | Description |
|---|---|---|
Topic | Read, Write, Create, Delete, Alter, Describe | Control access to individual topics |
Group | Read, Delete, Describe | Control consumer group membership |
Cluster | Create, Alter, Describe, ClusterAction | Cluster-wide operations |
TransactionalId | Write, Describe | Control transactional producer access |
Creating ACLs
# Allow user 'producer-app' to write to topics starting with 'orders-'
kafka-acls.sh --bootstrap-server localhost:9092 \
--add --allow-principal User:producer-app \
--operation Write --topic 'orders-' --resource-pattern-type prefixed
# Allow user 'consumer-app' to read from topic and use consumer group
kafka-acls.sh --bootstrap-server localhost:9092 \
--add --allow-principal User:consumer-app \
--operation Read --topic orders-events \
--operation Read --group order-processing-groupBest Practice: Principle of Least Privilege
Grant only the minimum permissions required for each application. A consumer doesn't need Write permissions, and a producer doesn't need Read permissions on the same topics.
Encryption: Protecting Data
Encryption in Transit (TLS)
TLS encrypts all communication between clients and brokers, and between brokers themselves (inter-broker communication).
# Broker configuration for TLS
listeners=SSL://:9093
ssl.keystore.location=/path/to/broker.keystore.jks
ssl.keystore.password=keystorepass
ssl.key.password=keypass
ssl.truststore.location=/path/to/truststore.jks
ssl.truststore.password=truststorepass
ssl.client.auth=required # For mTLSEncryption at Rest
Kafka doesn't provide built-in at-rest encryption, but you have several options:
Secure Kafka Operations Checklist
Enable authentication on all listeners
Never run production Kafka without authentication enabled.
Use TLS for all client and inter-broker communication
Encrypt all network traffic to prevent eavesdropping and tampering.
Implement ACLs with least privilege
Grant only necessary permissions to each application and user.
Rotate credentials and certificates regularly
Establish a rotation schedule for all authentication credentials.
Enable audit logging
Track authentication attempts and authorization decisions for compliance.
Network segmentation
Isolate Kafka brokers in private networks with controlled access.
Monitor for security anomalies
Alert on failed authentication attempts and unusual access patterns.
Common Security Pitfalls
Using SASL/PLAIN without TLS
PLAIN sends credentials in base64 encoding (not encryption). Without TLS, anyone sniffing network traffic can capture usernames and passwords.
Overly permissive ACLs
Granting ALL operations or using wildcard *patterns excessively defeats the purpose of authorization.
Not enabling authorization after authentication
Authentication without authorization means any authenticated user has full access. Always configure authorizer.class.name on brokers.
Expired or weak certificates
Using self-signed certificates in production, letting certificates expire, or using weak key lengths (less than 2048 bits for RSA) compromises security.
Secure Kafka Monitoring with KLogic
KLogic helps you maintain secure Kafka operations with comprehensive security monitoring, audit logging, and compliance reporting built into the platform.