Discover+Request a Kafka Streams Instance on Kubernetes when ReplicaSet=2

1/24/2019

How can I send a RPC request to another instance of the same Kafka Streams app instance on the same Kubernetes Service.

NOTE: This worked previously on Docker when exposing the current instances ip:port with the application.server property

e.g. ReplicaSet=2

StreamApp Instance 1 endpoint = 1.2.3.4:7079

StreamApp Instance 2 endpoint = 1.2.3.5:7079

I would like to send a rest request from Instance 1 accessing the remote Interactive Queries on Instance 2

what I have tried already

I sent a CURL request from instance 1 -> instance 2 : but got a 404 error

curl -X GET "http://1.2.3.5:7079/user/1" -H "accept: application/json"

but if I send the CURL request from K8 Host -> Instance 2 :I get a 200 ok

curl -X GET "http://1.2.3.5:7079/user/1" -H "accept: application/json"



 #values.yml
replicaCount: 1

 image:
  repository: "docker.hosted/steam-app"
  tag: "0.1.0"
  pullPolicy: Always
  pullSecret: "a_secret"

service:
 name: http
 type: NodePort
 externalPort: 7079
 internalPort: 7079

kafka:
 host: "kafka.default"
 port: "9092"

ingress:
 enabled: false

deployment.yml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: {{ template "stream-app.fullname" . }}
  labels:
    app: {{ template "stream-app.name" . }}
    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
spec:
  replicas: {{ .Values.replicaCount }}
  template:
    metadata:
      labels:
        app: {{ template "stream-app.name" . }}
        release: {{ .Release.Name }}
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          env:
            - name: KAFKA_HOST
              value: "{{ tpl .Values.kafka.host . }}"
            - name: KAFKA_PORT
              value: "{{ .Values.kafka.port }}"
            - name: MY_POD_IP
              valueFrom:
               fieldRef:
                fieldPath: status.podIP
            - name: MY_POD_PORT
              value: "{{ .Values.service.internalPort }}"
          ports:
            - containerPort: {{ .Values.service.internalPort }}
          livenessProbe:
            httpGet:
              path: /actuator/alive
              port: {{ .Values.service.internalPort }}
            initialDelaySeconds: 60
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /actuator/ready
              port: {{ .Values.service.internalPort }}
          initialDelaySeconds: 60
          periodSeconds: 10
          timeoutSeconds: 1
          successThreshold: 1
          failureThreshold: 3
          resources:
{{ toYaml .Values.resources | indent 12 }}
    {{- if .Values.nodeSelector }}
      nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
    {{- end }}
      imagePullSecrets:
        - name: {{ .Values.image.pullSecret }

service.yml

kind: Service
metadata:
  name: {{ template "stream-app.fullname" . }}
  labels:
    app: {{ template "stream-app.name" . }}
    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
spec:
  type: {{ .Values.service.type }}
  ports:
    - port: {{ .Values.service.externalPort }}
      targetPort: {{ .Values.service.internalPort }}
      protocol: TCP
      name: {{ .Values.service.name }}
  selector:
    app: {{ template "stream-app.name" . }}
    release: {{ .Release.Name }}
-- M_K
apache-kafka-streams
kubernetes

1 Answer

2/20/2019

I disabled Istio injection before installing the service and then re enabled it after installing the service and now its all working fine, so the commands that worked for me were:

enter image description here

-- M_K
Source: StackOverflow