Kafka RBAC & Multi-Tenancy Patterns
Running a shared Kafka cluster across multiple teams is a cost-efficiency win that becomes a security nightmare without proper isolation. This guide covers RBAC design, workspace isolation, SSO integration, and namespace patterns that scale.
Multi-Tenancy Models
There is no single right approach to Kafka multi-tenancy. The correct model depends on your isolation requirements, team autonomy needs, and operational complexity tolerance.
All teams share brokers, ZooKeeper/KRaft, and Schema Registry. Isolation enforced purely through ACLs and topic naming conventions.
- • Lowest infrastructure cost
- • Centralized operations
- • Noisy-neighbor risk
- • Limited blast radius isolation
Shared brokers but enforced topic prefix namespacing, per-team service accounts, and quotas. The sweet spot for most organizations.
- • Good cost/isolation balance
- • Team autonomy via self-service
- • Shared broker failure domain
- • More complex ACL management
Each team or business unit gets dedicated brokers. Maximum blast radius isolation. Appropriate for regulated workloads with hard compliance requirements.
- • Full fault isolation
- • Independent scaling
- • Highest cost
- • Operational complexity
Topic Namespace Design
A consistent topic naming convention is the foundation of namespace-based isolation. Once established, ACLs can be written as prefix patterns rather than per-topic rules, reducing management overhead by 10×.
Recommended Naming Convention
# Format: {team}.{domain}.{entity}.{version}
#
# Examples:
payments.transactions.orders.v1
payments.transactions.refunds.v1
logistics.shipping.tracking-events.v2
marketing.campaigns.email-clicks.v1
platform.audit.user-actions.v1
# Internal topics (team-private, not for cross-team consumption)
payments._internal.payment-state.v1
# Dead letter queues
payments.dlq.orders-processor.v1Kafka ACL Role Design
Role Taxonomy
| Role | Scope | Permissions |
|---|---|---|
| namespace-owner | Team namespace prefix | Create/delete/alter topics, read/write, manage consumer groups |
| producer | Specific topic(s) | Write, describe topic |
| consumer | Topic + consumer group | Read topic, read/write/describe consumer group |
| readonly | All topics (for monitoring) | Describe topics, describe consumer groups (no data access) |
| cluster-admin | Entire cluster | All operations. Granted only to automation service accounts. |
Applying Prefix-Based ACLs
# Grant payments team full access to their namespace kafka-acls.sh --bootstrap-server broker:9092 \ --add \ --allow-principal User:payments-namespace-owner \ --operation All \ --topic payments. \ --resource-pattern-type PREFIXED # Grant consumer group access for a specific service kafka-acls.sh --bootstrap-server broker:9092 \ --add \ --allow-principal User:logistics-tracker-service \ --operation Read \ --topic payments.transactions.orders.v1 kafka-acls.sh --bootstrap-server broker:9092 \ --add \ --allow-principal User:logistics-tracker-service \ --operation Read \ --group logistics.tracker \ --resource-pattern-type PREFIXED # List all ACLs for a namespace kafka-acls.sh --bootstrap-server broker:9092 \ --list \ --topic payments. \ --resource-pattern-type PREFIXED
SSO Integration with OAUTHBEARER
Kafka 2.6+ supports OAUTHBEARER authentication, enabling integration with enterprise SSO providers (Okta, Azure AD, Keycloak) so that human operators and CI/CD pipelines authenticate with short-lived tokens rather than long-lived certificates.
Keycloak / Azure AD Integration
# server.properties — broker configuration listeners=SASL_SSL://0.0.0.0:9093 security.inter.broker.protocol=SASL_SSL sasl.mechanism.inter.broker.protocol=OAUTHBEARER sasl.enabled.mechanisms=OAUTHBEARER # Point to your token endpoint listener.name.sasl_ssl.oauthbearer.sasl.jaas.config=\ org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \ oauth.client.id="kafka-broker" \ oauth.client.secret="${BROKER_CLIENT_SECRET}" \ oauth.token.endpoint.uri="https://your-idp.com/oauth/token" \ oauth.valid.issuer.uri="https://your-idp.com"; # Client configuration sasl.mechanism=OAUTHBEARER sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \ oauth.client.id="payments-service" \ oauth.client.secret="${PAYMENTS_CLIENT_SECRET}" \ oauth.token.endpoint.uri="https://your-idp.com/oauth/token";
Benefits of Token-Based Authentication
- Tokens expire automatically — no manual certificate rotation.
- Revoke a compromised service account at the IdP level, effective immediately.
- Group memberships from LDAP/AD map directly to Kafka ACL principals.
Tenant Quotas for Noisy-Neighbor Prevention
Kafka client quotas cap the network bandwidth and request rate available to each tenant, preventing one team from consuming all broker resources during a traffic spike.
# Set per-user produce quota (bytes/second) kafka-configs.sh --bootstrap-server broker:9092 \ --alter \ --add-config 'producer_byte_rate=104857600' \ # 100 MB/s --entity-type users \ --entity-name payments-producer-service # Set per-user consumer quota kafka-configs.sh --bootstrap-server broker:9092 \ --alter \ --add-config 'consumer_byte_rate=209715200' \ # 200 MB/s --entity-type users \ --entity-name payments-consumer-service # Set request rate quota (broker CPU protection) kafka-configs.sh --bootstrap-server broker:9092 \ --alter \ --add-config 'request_percentage=25' \ --entity-type users \ --entity-name analytics-etl-service
Key Takeaways
Manage Multi-Tenant Kafka with KLogic
KLogic's workspace model maps directly to Kafka namespaces. Each team gets a scoped dashboard, alert rules, and API access restricted to their prefix — with SSO integration via Okta, Azure AD, or any SAML 2.0 provider.
Request a Demo