Can't connect elasticsearch and kibana on kubernetes single mode

3/2/2019

I try to run elasticsearch and kibana on kubernetes. I ran:

kubectl run elasticsearch --image=elasticsearch:6.6.1 --env="discovery.type=single-node" --port=9200 --port=9300
kubectl run kibana --image=kibana:6.6.1 --port=5601

then I ran $kubectl proxy,

http://localhost:$IP_FROM_KUBECTL_PROXY(usually 8081)/api/v1/namespaces/default/pods/$POD_NAME/proxy/

When I entered to elasticsearch pod, everything looks fine, but when I entered to kibana, the app didn't work (I see "Kibana server is not ready yet" for infinity).

The logs of kibana are the following:

{"type":"log","@timestamp":"2019-03-02T10:38:47Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
{"type":"log","@timestamp":"2019-03-02T10:38:49Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}

This is kibana.yml on kibana pod:

Default Kibana configuration from kibana-docker.

server.name: kibana
server.host: "0"
elasticsearch.url: http://elasticsearch:9200
xpack.monitoring.ui.container.elasticsearch.enabled: true

I'm pretty new with Kubernetes, and I can't figure out why they can't talk one to another.

-- Yagel
elasticsearch
kibana
kubernetes

2 Answers

3/2/2019

Kibana explains you what a problem:

{"type":"log","@timestamp":"2019-03-02T10:38:49Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}

Name you pod elasticsearch is not enough for kubernetes.

You should make service as Amitio explain you. This is if kibana and elasticsearch running in the same namespace.

If kibana and elasticsearch running in different namespaces you'll write a full DNS name for service: elasticsearch.my-namespace.svc.cluster.local

If you'll run elasticsearch and kibana in the same pod. Then localhost:9200 will be able to query.

And for your situation. When elasticsearch is running you can use as ELASTIVSEARCH_URL pod DNS name: 1-2-3-4.default.pod.cluster.local when 1-2-3-4 is IP address of pod with dots replaced by dashes.

If you'll use hostname when you create elasticsearch:

apiVersion: v1
kind: Pod
metadata:
  name: elasticsearch
  labels:
    name: elasticsearch-single
spec:
  hostname: elasticsearch
  subdomain: for-kibana
  containers:
  - image: elasticsearch:6.6.1
    name: elasticsearch

You'll be able to send ELASTICSEARCH_URL pod DNS name: elasticsearch.for-kibana.default.svc.cluster.local service.

All information you can find here

-- ozlevka
Source: StackOverflow

3/2/2019

In kubernetes pods communicate with Services. You will need to define a service that selects your pod (with selector).

for example:

kind: Service
apiVersion: v1
metadata:
  name: elasticsearch
spec:
  selector:
    app: elasticsearch
  ports:
  - protocol: TCP
    port: 9200
    targetPort: 9200

Read more about services here and about labels here

We usually define the pods as yml files and add the labels there, but if you want to use kubectl run you can add labels with -l

  -l, --labels='': Comma separated labels to apply to the pod(s). Will override previous values.
-- Amityo
Source: StackOverflow