Strimzi is the most widely used operator for running Apache Kafka on Kubernetes. You describe a cluster with a Kafka resource and node pools, and the operator handles brokers, controllers, certificates, rolling updates and configuration changes. What it does not do by default is expose metrics. This guide covers the three layers you need: Kafka metrics, consumer lag and Kubernetes signals.
Layer 1: broker and controller metrics
Strimzi can attach the Prometheus JMX exporter to every Kafka node. You enable it in the Kafka resource and point it at a ConfigMap that holds the exporter rules:
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: prod
spec:
kafka:
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: kafka-metrics-config.ymlThe Strimzi repository ships example exporter rules and Grafana dashboards in its examples/metrics directory. Start from those rules rather than writing your own, then trim them: the defaults export many per-topic and per-partition series, which can overwhelm Prometheus on large clusters. Keep cluster-wide metrics at full resolution and drop per-partition series you never look at.
Scraping with the Prometheus Operator
If you run the Prometheus Operator, a PodMonitor selecting Strimzi pods by label is the simplest way to scrape every broker, controller and operator pod. Strimzi's examples include one. Make sure the Prometheus instance is allowed to watch the namespace your Kafka cluster lives in, a common reason dashboards stay empty.
Layer 2: consumer lag with Kafka Exporter
JMX metrics come from brokers, so they do not tell you how far behind your consumer groups are. Strimzi can deploy Kafka Exporter next to the cluster to publish consumer group lag and topic offsets:
spec:
kafkaExporter:
topicRegex: ".*"
groupRegex: ".*"Narrow both regexes on clusters with many topics or groups, and exclude internal topics you do not care about. Lag in offsets is a start; for alerting, convert it to time using the consumption rate. Our consumer lag guide explains why.
Layer 3: the Kubernetes signals Kafka adds
Running on Kubernetes creates failure modes that do not exist on dedicated hosts. Watch these alongside Kafka metrics:
- Persistent volume usage. Each broker writes to a PVC. The kubelet exposes
kubelet_volume_stats_used_bytesandkubelet_volume_stats_capacity_bytes; alert well before the volume fills, because a full log directory goes offline. Strimzi supports resizing volumes when the storage class allows expansion. - Pod restarts and OOM kills. A broker container killed for memory looks like a broker crash to Kafka. Set memory limits with room for the JVM heap plus the off-heap memory Kafka uses, and remember the page cache.
- CPU throttling. CPU limits that are too tight cause throttling that shows up as latency spikes and ISR shrinks rather than obvious errors. Compare container throttling metrics with request latency.
- Node pressure and evictions. Kafka pods should not be evicted for disk or memory pressure. Use dedicated node pools or taints, and a pod disruption budget so maintenance drains one broker at a time.
- Zone placement. Configure rack awareness so replicas of each partition land in different zones. Strimzi uses a node label (usually the zone topology label) to set each broker's rack.
Rolling updates are a monitoring event. Strimzi restarts brokers one by one for configuration and version changes. Expect under-replicated partitions to rise and fall with each pod, and suppress or delay alerts during planned rolls so they do not page anyone.
Watch the operator too
The Strimzi cluster operator reconciles your resources. If it is unhealthy, configuration changes and certificate renewals silently stop. Monitor the operator pod, its reconciliation metrics and the status conditions on your Kafka resource, which report whether the cluster is ready and why not.
Trimming the exporter rules
The example rules in the Strimzi repository are a good start, but they export per-topic and per-partition series that add up quickly. On a cluster with a few thousand partitions, edit the ConfigMap so that:
- Broker-wide metrics (replication, request handling, controller state) are kept at full resolution.
- Per-topic throughput is kept only for topics that match an allow-list.
- Per-partition log size is either dropped or scraped less frequently.
Strimzi picks up ConfigMap changes during the next reconciliation and may roll the brokers to apply them, so make these changes during a maintenance window. The same principles are covered in our guide to the Kafka JMX metrics that matter.
Dashboards worth building
- Cluster overview: brokers ready, under-replicated and offline partitions, bytes in and out, request latency, controller status.
- Per-broker detail: request handler idle, CPU usage against the pod limit, CPU throttling, heap usage, persistent volume usage.
- Consumers: lag per group from Kafka Exporter, converted to time for the groups with freshness targets.
- Operator and resources: reconciliation results and the Ready condition of each Kafka, KafkaTopic and KafkaUser resource.
Putting Kubernetes signals and Kafka signals on the same dashboard is what makes Kubernetes-specific problems obvious: a latency spike that lines up with CPU throttling, or ISR shrinks that line up with a node drain.
A worked example: the throttled broker
In an (illustrative) cluster, produce latency at p99 doubles every afternoon, and a few partitions briefly drop out of the ISR. Broker metrics look fine on average. The per-broker dashboard shows the cause: during the afternoon peak, one broker's container is being CPU-throttled because its CPU limit was set close to its normal usage. The broker is not overloaded in Kafka terms; Kubernetes is pausing it. Raising the CPU limit (or removing it while keeping the request) removes the throttling, and both the latency spikes and the ISR shrinks disappear.
Alerts to start with
- Under-min-ISR or offline partitions greater than zero.
- Any broker or controller pod not ready for more than five minutes outside a planned roll.
- Persistent volume usage above 80%, and a forecast to full within seven days.
- Consumer lag in time above each group's freshness target.
- The
Kafkaresource not in a Ready condition.
Where KLogic fits
Strimzi clusters are standard Apache Kafka, so KLogic connects to them like any other cluster through the bootstrap service. You get broker health, partition and replication status, consumer lag per partition, Connect and Schema Registry views and built-in alert rules without maintaining exporter rules and dashboards. Many teams keep the Strimzi metrics for Kubernetes-level signals and use KLogic for the Kafka-level picture.
Cruise Control for balancing
Uneven partition placement causes many of the per-broker problems that show up on these dashboards: one broker with more leaders, more disk use or more network traffic than the rest. Strimzi integrates Cruise Control, which analyses broker load and proposes partition moves. You enable it in the Kafka resource and request optimisations with a KafkaRebalance resource, reviewing the proposal before approving it.
Rebalances move data, so schedule them outside peak hours and watch replication throttling and under-replicated partitions while they run. Afterwards, per-broker disk and leader counts should look much more even; our guide to Kafka disk usage shows how to track that.
Securing metrics endpoints
Exporter endpoints reveal topic names, consumer group names and traffic levels. Keep them on the cluster network, restrict which namespaces can reach them with network policies, and scrape them only from your Prometheus. If you send metrics to an external service, check what labels leave the cluster. Our Kafka security best practices cover the broader picture.
Common questions
Does Strimzi include monitoring out of the box?
Strimzi provides the integration points (JMX exporter configuration, Kafka Exporter and example dashboards), but metrics are not enabled until you configure them in the Kafka resource and deploy Prometheus and Grafana.
Should Kafka pods have CPU limits on Kubernetes?
Many teams set CPU requests but avoid tight CPU limits for brokers, because throttling causes latency spikes and ISR shrinks. Memory limits are still important; size them for heap plus off-heap usage.
How do I monitor consumer lag on Strimzi?
Enable Kafka Exporter in the Kafka resource, or use a tool that reads lag through the Kafka admin API. See our consumer lag guide for turning offsets into time.
What breaks during a Strimzi rolling update?
Each broker restart briefly makes its partitions under-replicated and moves leadership. Clients see short reconnects. Suppress replication alerts during planned rolls and confirm the cluster returns to zero under-replicated partitions afterwards.
Can I monitor Strimzi clusters with KLogic?
Yes. Strimzi clusters are standard Apache Kafka, so KLogic connects through the bootstrap service like any other cluster.