Encryption, authentication and external access for Confluent Kafka on Kubernetes

5/22/2019

Am trying to configure Encryption, authentication and external access for Confluent Kafka on Kubernetes.

helm chart https://github.com/confluentinc/cp-helm-charts.

Document following : https://medium.com/weareservian/encryption-authentication-and-external-access-for-confluent-kafka-on-kubernetes-69c723a612fc

As per document I have configured

  1. secrets.yaml https://raw.githubusercontent.com/bakuppus/kafka-demo/master/secrets.yaml

  2. Statefulset.yaml https://github.com/bakuppus/kafka-demo/blob/master/cp-helm-charts/charts/cp-kafka/templates/statefulset.yaml

While set SSL as per export KAFKA_ADVERTISED_LISTENERS=SSL://${POD_IP}:9092,EXTERNAL://${HOST_IP}:$((31090 + ${KAFKA_BROKER_ID})) && \

Pods are getting CrashLoopBackOff

my-confluent-oss-cp-kafka-0 2/3 CrashLoopBackOff 3 2m40s my-confluent-oss-cp-kafka-1 2/3 CrashLoopBackOff 3 2m13s my-confluent-oss-cp-kafka-2 2/3 Error 3 111s

kubectl logs error

SSL is enabled. 'ascii' codec can't decode byte 0xb5 in position 0: ordinal not in range(128) Command [/usr/local/bin/dub template /etc/confluent/docker/kafka.properties.template /etc/kafka/kafka.properties] FAILED !

-- fiverrbala
apache-kafka
kubernetes
kubernetes-helm

1 Answer

5/23/2019

You have plaintext passwords in the secrets.yaml file: they need to be base64 encoded like the keystore files.

Also, it looks like you're missing a protocol mapping for the named EXTERNAL listener, KAFKA_LISTENER_SECURITY_PROTOCOL_MAP environment variable should be:

        - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
          value: SSL:SSL,PLAINTEXT:PLAINTEXT,EXTERNAL:SSL
-- Ryan M
Source: StackOverflow