I am installing the official jenkins
helm chart on GKE.
I am enabling Ingress
therefore the corresponding template should be applied and the resource created.
According to the official GKE documentation:
When you create the Ingress, the GKE ingress controller creates and configures an HTTP(S) load balancer according to the information in the Ingress and the associated Services. Also, the load balancer is given a stable IP address that you can associate with a domain name.
However, this does not happen in my case; the ingress does not get an external IP associated with it:
▶ k get ingress --all-namespaces
NAMESPACE NAME HOSTS ADDRESS PORTS AGE
jenkins jenkins-inception * 80 82s
Here is the actual Ingress
resource:
▶ k get ingress --all-namespaces -o yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
creationTimestamp: "2020-01-24T21:20:49Z"
generation: 1
labels:
app.kubernetes.io/component: jenkins-master
app.kubernetes.io/instance: jenkins-inception
app.kubernetes.io/managed-by: Tiller
app.kubernetes.io/name: jenkins
helm.sh/chart: jenkins-1.9.16
name: jenkins-inception
namespace: jenkins
resourceVersion: "15741661"
selfLink: /apis/extensions/v1beta1/namespaces/jenkins/ingresses/jenkins-inception
uid: 6461793e-3eef-11ea-a0a5-42010a790807
spec:
rules:
- http:
paths:
- backend:
serviceName: jenkins-inception
servicePort: 8080
path: /jenkins
status:
loadBalancer: {}
kind: List
metadata:
resourceVersion: ""
selfLink: ""
Why is that?
Have tried both nginx
and gce
in kubernetes.io/ingress.class
annotation values.
edit_1: it seems that the HTTP Load Balancing
Add On is enabled.
The weirdest part however is the following:
▶ k describe ingress jenkins-inception -n jenkins
Error from server (NotFound): the server could not find the requested resource
~
▶ k get ingress jenkins-inception -n jenkins
NAME HOSTS ADDRESS PORTS AGE
jenkins-inception * 80 11h
edit_2:
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.11-gke.14", GitCommit:"56d89863d1033f9668ddd6e1c1aea81cd846ef88", GitTreeState:"clean", BuildDate:"2019-11-07T19:12:22Z", GoVersion:"go1.12.11b4", Compiler:"gc", Platform:"linux/amd64"}
edit_3: After addressing the kubectl
compatibility issue, it turns out no events are generated by the ingress
▶ k describe ingress jenkins-inception -n jenkins
Name: jenkins-inception
Namespace: jenkins
Address:
Default backend: default-http-backend:80 (10.8.32.33:8080)
Rules:
Host Path Backends
---- ---- --------
*
/jenkins jenkins-inception:8080 (10.8.33.87:8080)
Annotations:
kubernetes.io/ingress.class: nginx
Events: <none>
Is there a command line to list enabled add-ons on a cluster?
edit_4: About the add ons..
▶ gcloud container clusters describe inception-cluster --zone us-east4-b | grep -i add -A 6
addonsConfig:
horizontalPodAutoscaling: {}
httpLoadBalancing: {}
kubernetesDashboard:
disabled: true
networkPolicyConfig:
disabled: true
edit_5: After enabling the actual GCE ingress with the correct annotation as pointed out in comments by @Arghya Sadhu, I see the following error in the ingress description
Warning Translate 2s (x11 over 7s) loadbalancer-controller error while evaluating the ingress spec: service "jenkins/jenkins-inception" is type "ClusterIP", expected "NodePort" or "LoadBalancer"
However this contradicts with the following recommendation from the official jenkins
helm charts:
# For minikube, set this to NodePort, elsewhere use LoadBalancer
# Use ClusterIP if your setup includes ingress controller
edit_6: The ingress managed to get a public IP:
spec:
rules:
- http:
paths:
- backend:
serviceName: jenkins-inception
servicePort: 8080
path: /jenkins
status:
loadBalancer:
ingress:
- ip: 12.234.543.111
However I am unable to access the following paths:
https://12.234.543.111/jenkins
http://12.234.543.111/jenkins
https://12.234.543.111
http://12.234.543.111
edit_7: The yaml
of the jenkins-inception
service
▶ k get svc jenkins-inception -o yaml
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2020-01-25T12:40:14Z"
labels:
app.kubernetes.io/component: jenkins-master
app.kubernetes.io/instance: jenkins-inception
app.kubernetes.io/managed-by: Tiller
app.kubernetes.io/name: jenkins
helm.sh/chart: jenkins-1.9.16
name: jenkins-inception
namespace: jenkins
resourceVersion: "16000964"
selfLink: /api/v1/namespaces/jenkins/services/jenkins-inception
uid: d5bfd760-3f6f-11ea-a0a5-42010a790807
spec:
clusterIP: 10.8.59.184
externalTrafficPolicy: Cluster
ports:
- name: http
nodePort: 30361
port: 8080
protocol: TCP
targetPort: 8080
selector:
app.kubernetes.io/component: jenkins-master
app.kubernetes.io/instance: jenkins-inception
sessionAffinity: None
type: NodePort
status:
loadBalancer: {}
Run describe on the Ingress. If you see create/add events, you have an Ingress controller running in the cluster, otherwise, you probably have the HttpLoadBalancing(GKE Ingress Controller) add-on disabled on your GKE cluster
Edit1:
You have version incompatibility between kubernetes server and kubectl. You can check both client and server version by running below command. Check this issue for details.
kubectl version
Edit2:
You either should not have the annotation kubernetes.io/ingress.class
or if you have it needs be gce kubernetes.io/ingress.class: gce
Edit3:
As per the google cloud doc the service type for jenkins-inception service needs to be of type NodePort