Exposing a pod to local enviroment for easy developing purposes?

1/13/2020

I am curently working with kubernetes, and trying to make a development process making it capable for developer to access services within a local kubernetes cluster? I would like it to keep it simple, and have for now tried kubectl port-forward kafka 10000:9092 but this didn't seem to expose the pod to localhost:10000..

I've tried converting the kafka service to a nodeport, still with no luck - only way I could access it was by creating my application as dockerized application, and run the application in a docker container - meaning that running the exe file would not connect to it, but executing it using docker would make it work.

I've tried Kubectl proxy - which doesn't work either - I am not able to ping the clusterIp.

I have not tried with ingress or loadbalancer, as I find it a bit too elaborate, considering that this is only for developing purposes, and not something that should be production "secure"..

How do i easily expose the kafka service such that a console application on my laptop running kubernetes cluster locally can access it?

apiVersion: v1
kind: Service
metadata:
  name: kafka-headless
spec:
  clusterIP: None
  ports:
  - name: broker
    port: 9092
    protocol: TCP
    targetPort: 9092
  selector:
    app: kafka
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: kafka
spec:
  ports:
  - name: broker
    port: 9092
    protocol: TCP
    targetPort: 9092
  selector:
    app: kafka
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: kafka
  name: kafka
spec:
  podManagementPolicy: OrderedReady
  replicas: 1
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      app: kafka
  serviceName: kafka-headless
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
      - command:
        - sh
        - -exc
        - |
          unset KAFKA_PORT && \
          export KAFKA_BROKER_ID=${HOSTNAME##*-} && \
          export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://${POD_IP}:9092 && \
          exec /etc/confluent/docker/run
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        - name: KAFKA_HEAP_OPTS
          value: -Xmx1G -Xms1G
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: leader-zookeeper:2181
        - name: KAFKA_LOG_DIRS
          value: /opt/kafka/data/logs
        - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
          value: "1"
        image: confluentinc/cp-kafka:latest
        imagePullPolicy: IfNotPresent
        livenessProbe:
          exec:
            command:
            - sh
            - -ec
            - /usr/bin/jps | /bin/grep -q SupportedKafka
          failureThreshold: 3
          initialDelaySeconds: 30
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5
        name: kafka-broker
        readinessProbe:
          failureThreshold: 3
          initialDelaySeconds: 30
          periodSeconds: 10
          successThreshold: 1
          tcpSocket:
            port: kafka
          timeoutSeconds: 5
        ports:
        - containerPort: 9092
          name: kafka
          protocol: TCP
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /opt/kafka/data
          name: datadir
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 60
  updateStrategy:
    type: OnDelete
  volumeClaimTemplates:
  - metadata:
      name: datadir
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 1Gi
---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  labels:
    app: kafka-pdb
  name: kafka-pdb
spec:
  maxUnavailable: 0
  selector:
    matchLabels:
      app: kafka
---
-- kafka
docker-desktop
kubernetes
visual-studio

2 Answers

1/13/2020

To port forward to a service, you need to use svc/ infront of the name. So your command would be either kubectl port-forward svc/kafka 10000:9092 or kubectl port-forward kafka-0 10000:9092

On Windows, make sure windows firewall is not blocking kubectl.

Reference: https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/#forward-a-local-port-to-a-port-on-the-pod


You can also use telepresence to debug a existing service on cluster by swapping it local development version.

Install telepresence and use telepresence --swap-deployment $DEPLOYMENT_NAME

Reference:

https://kubernetes.io/docs/tasks/debug-application-cluster/local-debugging/#developing-or-debugging-an-existing-service

https://www.telepresence.io/reference/install

-- Shashank V
Source: StackOverflow

1/15/2020

If I understand you correctly I have some additional options for you to check:

  1. This answer uses an idea of externalTrafficPolicy: Local alongside other possible solutions.

  2. I see from the comments that you are using Docker Desktop for Windows. You can try to use type: LoadBalancer service instead of ClusterIP or NodePort. I know it may sound wiered but I have seen few examples like this one showing that it actually works.

I am posting this as a community answer because the proposed solutions were not originally my ideas.

Please let me know if that helped.

-- OhHiMark
Source: StackOverflow