I have a Kafka cluster that is running on K8S. I am using the confluent kafka image as and I have an EXTERNAL listeners that is working. How can I add SSL encryption? Should I use an ingress? Where can I find good documentation? Thank you
You have a manual way in this gist, which does not use the confluent image.
But for Confluent and its Helm chart (see "Confluent Operator: Getting Started with Apache Kafka and Kubernetes" from Rohit Bakhshi), you can follow:
"Encryption, authentication and external access for Confluent Kafka on Kubernetes" from Ryan Morris
Out of the box, the helm chart doesn’t support SSL configurations for encryption and authentication, or exposing the platform for access from outside the Kubernetes cluster.
To implement these requirements, there are a few modifications to the installation needed.
In summary, they are:
- Generate some private keys/certificates for brokers and clients
- Create Kubernetes Secrets to provide them within your cluster
- Update the broker StatefulSet with your Secrets and SSL configuration
- Expose each broker pod via an external service
I recommend using Strimzi kafka operator to deploy Kafka to Kubernetes. I'm using it in production for a year now. It supports SSL, external load balancers, kafka exporter, etc Strimzi Kafka Operator