I'm currently on OSX (Catalina) with Docker Desktop (19.03.5) running k8s (v.14.8)
Following the ambassador getting started article, I've done the following:
Created a file called ambassador-ingress.yaml
---
apiVersion: v1
kind: Service
metadata:
labels:
service: ambassador
name: ambassador
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 80
selector:
service: ambassador
---
apiVersion: v1
kind: Service
metadata:
name: google
annotations:
getambassador.io/config: |
---
apiVersion: ambassador/v0
kind: Mapping
name: google_mapping
prefix: /google/
service: https://google.com:443
host_rewrite: www.google.com
spec:
type: ClusterIP
clusterIP: None
And I've run the following
$ kubectl apply -f https://www.getambassador.io/yaml/ambassador/ambassador-rbac.yaml
$ kubectl apply -f ambassador-ingress.yaml
I can now look at kubectl get pods
and kubectl get service
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
ambassador-74fb8f5668-2b85z 1/1 Running 0 20m
ambassador-74fb8f5668-r6jrf 1/1 Running 0 20m
ambassador-74fb8f5668-vrmjg 1/1 Running 0 20m
$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ambassador LoadBalancer 10.97.166.229 localhost 80:31327/TCP 18m
ambassador-admin NodePort 10.96.1.56 <none> 8877:30080/TCP 18m
google ClusterIP None <none> <none> 18m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 13d
Everything looks like it is setup correctly, however, whenever I attempt to curl k8s I can't get anything but an empty server response even though I can hit google directly:
$ curl localhost/google/
> curl: (52) Empty reply from server
$ curl www.google.com
> <!doctype html> ............
The question I have is, where do I begin troubleshooting? I don't know where the failure lies or how to begin digging to find what has gone wrong. What is the right direction?
Based on "The Kubernetes network model" [1] there are 2 important rules:
- pods on a node can communicate with all pods on all nodes without NAT
- agents on a node (e.g. system daemons, kubelet) can communicate with all pods on that node
So basically it says that since your K8s cluster is located on your machine, you can directly communicate to the service ip "10.97.166.229" and POD IP.
So, regarding how to begin your troubleshooting steps, since your PODs are up and running, most likely this is a network error. You can try this:
a) Try to connect to your pod directly. You can get your IP by executing the command:
kubectl get pod -o wide
b) Get the LOGs of your POD and search for any error:
kubectl logs ambassador-74fb8f5668-2b85z
c) Go inside your POD and check if you can test connectivity inside your POD. [2]
kubectl exec -it ambassador-74fb8f5668-2b85z -- /bin/bash
[1] https://kubernetes.io/docs/concepts/cluster-administration/networking/
[2] https://kubernetes.io/docs/tasks/debug-application-cluster/get-shell-running-container/