I'm trying to create a multi-cluster ingress on Google Kubernetes Engine using kubemci, however when running the following command the program waits indefinitely for the ingress service to get the ingress.gcp.kubernetes.io/instance-groups
annotation (as illustrated in the output below).
What is preventing this annotation from being set?
Input
./kubemci create app-mci \
--ingress=ingress.yaml \
--gcp-project=app-prod \
--kubeconfig=mcikubeconfig
Output
% ./kubemci create app-mci --ingress=ingress.yaml --gcp-project=app-prod --kubeconfig=clusters.yaml
Created Ingress in cluster: gke_app-prod_europe-west4-a_app-europe-west4
Created Ingress in cluster: gke_app-prod_us-east4-a_app-us-east4
Ensuring health checks
Pod app-deployment-c99578769-xdmql matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-xgq2m matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-qms7r matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-tsrsw matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Path for healthcheck is /
Ensuring health check for port: {SvcName:default/app-service SvcPort:{Type:0 IntVal:80 StrVal:} NodePort:30061 Protocol:HTTP SvcTargetPort: NEGEnabled:false}
Health check mci1-hc-30061--app-mci exists already. Checking if it matches our desired health check
Desired health check exists already
Determining instance groups for cluster gke_app-prod_europe-west4-a_app-europe-west4
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
⋮
As you can see my configuration is identical (aside from resource names) to that in the multi-cluster ingress guide:
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-deployment
spec:
selector:
matchLabels:
app: app
replicas: 2
template:
metadata:
labels:
app: app
spec:
containers:
- name: app
image: gcr.io/app-prod/app:tag
ports:
- containerPort: 8080
service.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: app
name: app-service
spec:
ports:
- port: 80
protocol: TCP
targetPort: 8080
name: http
nodePort: 30061
selector:
app: app
type: NodePort
ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: app-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: app-ip
kubernetes.io/ingress.class: gce-multi-cluster
spec:
backend:
serviceName: app-service
servicePort: 80
Enable the HTTP load balancing add-on to allow the load balancer controller to set the ingress.gcp.kubernetes.io/instance-groups
annotation.
% gcloud container clusters update [CLUSTER_NAME] --update-addons HttpLoadBalancing=ENABLED
Updating ***...done.
Updated [https://container.googleapis.com/v1/projects/***/zones/us-east4-a/clusters/***].
To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-east4-a/***?project=***
View the cluster configuration:
% gcloud container clusters describe [CLUSTER_NAME]
# ENABLED
addonsConfig:
httpLoadBalancing: {}
# DISABLED
addonsConfig:
httpLoadBalancing:
disabled: true
Ensure that the backend Services used in the multi-cluster ingress are configured correctly.
Services must:
- Have the same name in all of the clusters.
- Be in the same namespace in all of the clusters.
- Be of type NodePort.
Use the same port number for all of the clusters.
— Setting up a multi-cluster Ingress, Google