K
Q

Kubernetes pod wget -qO- fails for some service, but not all

6/15/2018

As I was going over my recent experiments, I went over my notes to recreate a relatively simple setup using Kubernetes for a back-end and front-end service setup. In my scenario both of these services need to be exposed, and for now I'm doing that using NodePort.

This all worked quite nicely a week or so ago, but I think I managed to mess things up and this has me going nuts. The result is that I cannot seem to get access to my back-end pods via the service. I've followed along the Debug Service document (https://kubernetes.io/docs/tasks/debug-application-cluster/debug-service) and things are going haywire pretty quickly.

So this is my current yaml file:

apiVersion: v1
kind: Service
metadata:
  name: test
spec:
  type: NodePort
  ports:
  - name: default
    protocol: TCP
    port: 80
    targetPort: 8080
  selector:
    app: test
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
spec:
  selector:
    matchLabels:
      app: test
  replicas: 1
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
      - name: test
        image: jan/test:v1.0.0
        ports:
        - containerPort: 8080
          protocol: TCP

The application starts fine - it reports in the log it is ready for requests. (It is a Java/Grizzly application). Now here is a list of what I tried.

  1. check kubectl services: it is there (for this example it is 172.17.0.4)
  2. exec into the pod (alpine)
    • ifconfig - 172.17.0.4, 127.0.0.1
    • nslookup test 10.96.0.10 - works (note without nameservice this will return can't resolve '(null)' : Name does not resolve
    • ping 127.0.0.1 - works
    • wget http://127.0.0.1:8080 - responds fine
    • ping 172.17.0.4 - works
    • wget http://172.17.0.4:8080 - fails immediately, connection refused
    • wget -qO- test - fails after a while, operation times out
  3. exec into another (busybox) pod
    • ifconfig - 172.17.0.8, 127.0.0.1
    • nslookup test - works
    • ping to pod 172.17.0.4 - works
    • wget http://172.17.0.8:8080 - fails immediately, connection refused
    • wget -qO- test - fails immediately, connection refused

Most importantly - I think that the wget -qO- {service} need to start reporting its pod, which currently it does not. Again - I went through the scenario of the Debug Service document and that completes without issues.

So what (else) could be wrong for that wget -qO- to fail?

-- Jan
iptables
kubernetes
networking

Similar Questions

Targetting external REST API using openshift external service
Kubernetes - how to map host directory inside pod?
Link Gitlab with Kubernetes
kubernetes python watch stream
Use Servicemonitor to check services via external hostname
Dedicated IP address to LoadBalancer mapping
How to update a container image of a running pod?

2 Answers

6/16/2018

I removed an important property that was fed into the application. So actually the problem was not at all at the level of K8S. Essentially I was rendering my deployed application 'invisible'.

-- Jan
Source: StackOverflow
6/16/2018

So, let's see...You are in a busybox pod.

ifconfig - 172.17.0.8, 127.0.0.1

wget http://172.17.0.8:8080 - fails immediately, connection refused

What are you doing here? This is like to do localhost:8080. Of course you are getting connection refused. There is nothing serving on port 8080 of busybox.

wget -qO- test - fails immediately, connection refused

Same here. Now you are doing the request on port 80 of busybox, that again has nothing serving.

There is absolutely no way this configuration has ever worked. All you are doing is to do requests to yourself from within a busybox.

You need to do the request to a service that points to your app or directly to the pod that contains your app.

-- suren
Source: StackOverflow