I'm trying a simple microservices app on a cloud Kubernetes cluster. This is the Ingress yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-nginx-nginx-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
spec:
defaultBackend:
service:
name: auth-svc
port:
number: 5000
rules:
- host: "somehostname.xyz"
http:
paths:
- path: "/"
pathType: Prefix
backend:
service:
name: auth-svc
port:
number: 5000The problem:
When I use this URL, I'm able to access the auth service: http://somehostname.xyz:31840. However, if I use http://somehostname.xyz, I get a "This site can’t be reached somehostname.xyz refused to connect." error.
The auth service sends GET requests to other services too, and I'm able to see the response from those services if I use:http://somehostname.xyz:31840/go or http://somehostname.xyz:31840/express. But again, these work only if the nodeport 31840 is used.
My questions:
What typically causes such a problem, where I can access the service using the hostname and nodeport, but it won't work without supplying the nodeport?
Is it a problem with the Ingress or Auth namespace? Is it a problem with the hostname in Flask? Is it a problem with the Ingress controller? How do I debug this?
These are the results of kubectl get all and other commands.
NAME READY STATUS RESTARTS
pod/auth-flask-58ccd5c94c-g257t 1/1 Running 0
pod/ingress-nginx-nginx-ingress-6677d54459-gtr42 1/1 Running 0
NAME TYPE EXTERNAL-IP PORT(S)
service/auth-svc ClusterIP <none> 5000/TCP
service/ingress-nginx-nginx-ingress LoadBalancer 172.xxx.xx.130 80:31840/TCP,443:30550/TCP
NAME READY UP-TO-DATE AVAILABLE
deployment.apps/auth-flask 1/1 1 1
deployment.apps/ingress-nginx-nginx-ingress 1/1 1 1
NAME DESIRED CURRENT READY
replicaset.apps/auth-flask-58ccd5c94c 1 1 1
replicaset.apps/ingress-nginx-nginx-ingress-6677d54459 1 1 1
NAME CLASS HOSTS ADDRESS PORTS
ingress-nginx-nginx-ingress <none> somehostname.xyz 172.xxx.xx.130 80 Describing ingress also seems normal.
kubectl describe ingress ingress-nginx-nginx-ingress
Name: ingress-nginx-nginx-ingress
Namespace: default
Address: 172.xxx.xx.130
Default backend: auth-svc:5000 (10.x.xx.xxx:5000)
Rules:
Host Path Backends
---- ---- --------
somehostname.xyz
/ auth-svc:5000 (10.x.xx.xxx:5000)
Annotations: kubernetes.io/ingress.class: nginx
This is the code of Auth.
import requests
from flask import Flask
app = Flask(__name__)
@app.route('/')
def indexPage():
return ' <!DOCTYPE html><html><head><meta charset="UTF-8" />\
<title>Microservice</title></head> \
<body><div style="text-align: center;">Welcome to the Auth page</div></body></html>'
@app.route('/go')
def getGoJson():
return requests.get('http://analytics-svc:8082/info').content
@app.route('/express')
def getNodeResponse():
return requests.get('http://node-svc:8085/express').content
if __name__ == '__main__':
app.run(debug=True, host="0.0.0.0")and Auth's Dockerfile:
FROM python:3.8-slim-buster
WORKDIR /usr/src/app
ENV FLASK_APP=app.py
ENV FLASK_RUN_HOST=0.0.0.0
ENV FLASK_ENV=development
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY . .
CMD ["flask", "run"] The part of docker-compose yaml for auth:
version: "3.3"
services:
auth:
build: ./auth/
image: nav9/auth-flask:v1
ports:
- "5000:5000"Auth's Kubernetes manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-flask
spec:
selector:
matchLabels:
any-name: auth-flask
template:
metadata:
labels:
any-name: auth-flask
spec:
containers:
- name: auth-name
image: nav9/auth-flask:v1
imagePullPolicy: Always
ports:
- containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
name: auth-svc
spec:
# type: ClusterIP
ports:
- targetPort: 5000
port: 5000
selector:
any-name: auth-flaskWhat typically causes such a problem, where I can access the service using the hostname and nodeport, but it won't work without supplying the nodeport?
If the URL works when using the nodeport and not without the nodeport, then this means that the ingress is not configured properly for what you want to do.
Is there a method to test this in a different way to figure out where the problem is?
Steps for troubleshooting are:
In your case, the error This site can’t be reached somehostname.xyz refused to connect, sounds like the Ingress found the service to map to and used port 5000 to connect to it, and the connection was refused or nothing was listening on port 5000 for that service.
My guess is that the auth service is listening on port 31840 but your ingress says to connect to port 5000 based on the configuration.
You might try adding a port mapping from 80 to 31840 as a hack/test to see if you get a different error.
Something like:
spec:
rules:
- host: "somehostname.xyz"
http:
paths:
- path: "/"
backend:
service:
port:
number: 31840I've only included the part needed to show the indentation properly. So the other way to test this out is to create additional URLs that map to different ports, so for example:
/try1 => auth-svc:5000
/try2 => auth-svc:31840
/try3 => auth-svc:443The other part that I haven't played with that might be an issue is that you are using http and I don't know of any auth service that would use http, so simply trying to connect using http to an app that wants https will get a connection either refused or a strange error, so that might be related to the problem/error you are seeing.
Hope this gives you some ideas to try.
The solution has three parts:
1. Use kubectl get all to find out the running ingress service:
NAME TYPE EXTERNAL-IP PORT(S)
service/ingress-nginx-nginx-ingress LoadBalancer 172.xxx.xx.130 80:31840/TCP,443:30550/TCP
Copy the EXTERNAL-IP of the service (in this case 172.xxx.xx.130).
Add a DNS A record named *.somehostname.xyz for the cloud cluster, and use the IP address 172.xxx.xx.130.
When accessing the hostname via the browser, make sure that http is used instead of https.