Kubernetes pods refusing connections to each other

8/22/2019

I'm trying to implement an ElasticStack in Kubernetes via Minikube. I've barely started, as I'm writing basically everything from scratch to get a better understand of K8s and because the provided yml's from Elastic don't offer any explanation as to what is done why, so I'm doing my own thing.

The problem I've ran into is that my Kibana-pod cannot communicate with my ElasticSearch-pod, although I've set up the necessary services and ports on my pods.

Where it gets weird is that

kubectl port-forward services/elastic-http 9200

works flawlessly and lets me get information from my ElasticSearch pod. However, when I enter a pod via

kubectl exec -it <pod-name> -- /bin/bash

and try to use curl to get the same information my browser just showed me, the connection is being refused and my pods won't talk to one another.

My configs look as follows.

Kibana.yml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: my-kb
  namespace: default
spec:
  selector:
    matchLabels:
      app: kibana
  template:
    metadata:
      name: kibana
      labels:
        app: kibana
    spec:
      containers:
      - name: kibana
        image: docker.elastic.co/kibana/kibana:7.3.0
        ports:
        - containerPort: 5601
          name: kibana-web
        volumeMounts:
        - name: kb-conf
          mountPath: /usr/share/kibana/config/kibana.yml
          subPath: kibana.yml
      volumes:
        - name: kb-conf
          configMap:
            name: kibana-config
            items:
            - key: kibana.yml
              path: kibana.yml
---
kind: Service
apiVersion: v1
metadata:
  name: kibana-http
  namespace: default
spec:
  selector:
    app: kibana
  ports:
    - protocol: TCP
      port: 5601
      name: kibana-web
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kibana-config
  namespace: default
data:
  kibana.yml: |
    elasticsearch.hosts: ["http://elastic-http.default.svc:9200"]

ElasticSearch.yml

kind: PersistentVolume
apiVersion: v1
metadata:
  name: elastic-pv
  namespace: default
spec:
  capacity:
    storage: 15Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /data
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: elastic-pv-claim
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 15Gi
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: elastic-deploy
  namespace: default
spec:
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      name: elasticsearch
      labels:
        app: elasticsearch
    spec:
      containers:
        - name: elasticsearch
          image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
          ports:
          - containerPort: 9200
            name: elastic-http
            protocol: TCP
          - containerPort: 9300
            name: node-sniffer
            protocol: TCP
          #readinessProbe:
          #  httpGet:
          #    port: 9200
          #  periodSeconds: 5
          volumeMounts:
          - name: elastic-conf
            mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
            subPath: elasticsearch.yml
          - name: elastic-data
            mountPath: /var/data
          securityContext:
            privileged: true
      initContainers:
      - name: sysctl-adj
        image: busybox
        command: ['sysctl', '-w', 'vm.max_map_count=262144']
        securityContext:
          privileged: true
      volumes:
        - name: elastic-data
          persistentVolumeClaim:
            claimName: elastic-pv-claim
        - name: elastic-conf
          configMap:
            name: elastic-config
            items:
            - key: elasticsearch.yml
              path: elasticsearch.yml
---
kind: Service
apiVersion: v1
metadata:
  name: elastic-http
  namespace: default
spec:
  selector:
    app: elasticsearch
  ports:
    - port: 9200
      targetPort: elastic-http
      name: elastic-http
    - port: 9300
      targetPort: node-sniffer
      name: node-finder
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: elastic-config
  namespace: default
data:
  elasticsearch.yml: |
    xpack.security.enabled: false
    node.master: true
    path.data: /var/data
    http.port: 9200
-- ErikHolgersson
elastic-stack
kubernetes
minikube

2 Answers

8/22/2019

I'm not sure about this part in service:

targetPort: elastic-http
targetPort: node-sniffer

could you try to remove them and try again

-- Oles Rid
Source: StackOverflow

8/22/2019

I think you are having clusterIP service type and if you want to see it in browser one of the option is to have service type as NodePort.

You can see more details here

-- Bimal
Source: StackOverflow