Creating a Kubernetes LoadBalancer returns immediatly (ex: kubectl create -f ...
or kubectl expose svc NAME --name=load-balancer --port=80 --type=LoadBalancer
).
I know a manual way to wait in shell:
external_ip=""
while [ -z $external_ip ]; do
sleep 10
external_ip=$(kubectl get svc load-balancer --template="{{range .status.loadBalancer.ingress}}{{.ip}}{{end}}")
done
This is however not ideal:
--wait
or --wait-once
but using those the command never returns.Is there a better way to wait until a service external IP (aka LoadBalancer Ingress IP) is set or failed to set?
Really just a clean-up of @Dan Garfield's working example; My OCD wouldn't let this slide. In this case:
apiVersion: v1
kind: Service
metadata:
name: yo
annotations:
cloud.google.com/load-balancer-type: "Internal"
# external-dns.alpha.kubernetes.io/hostname: vault.stage.domain.tld.
...
NOTE: I've only been able to get external-dns to associate names to public IP addresses.
This has been scripted to accept a few arguments, now it's a library; example:
myServiceLB=$1
while true; do
successCond="$(kubectl get svc "$myServiceLB" \
--template="{{range .status.loadBalancer.ingress}}{{.ip}}{{end}}")"
if [[ -z "$successCond" ]]; then
echo "Waiting for endpoint readiness..."
sleep 10
else
sleep 2
export lbIngAdd="$successCond"
pMsg """
The Internal LoadBalancer is up!
"""
break
fi
done
Later, $lbIngAdd
can be used to set records. Seems like -o jsonpath="{.status.loadBalancer.ingress[*].ip}"
would work as well; whatever works.
Thanks for getting us started Dan :-)
Just to add to the answers here, the best option right now is to use a bash script. For convenience, I've put it into a single line that includes exporting an environmental variable.
Command to wait and find Kubernetes service endpoint
bash -c 'external_ip=""; while [ -z $external_ip ]; do echo "Waiting for end point..."; external_ip=$(kubectl get svc NAME_OF_YOUR_SERVICE --template="{{range .status.loadBalancer.ingress}}{{.ip}}{{end}}"); [ -z "$external_ip" ] && sleep 10; done; echo "End point ready-" && echo $external_ip; export endpoint=$external_ip'
I've also modified your script so it only executes a wait if the ip isn't available. The last bit will export an environment variable called "endpoint"
Bash Script to Check a Given Service
Save this as check-endpoint.sh
and then you can execute $sh check-endpoint.sh SERVICE_NAME
#!/bin/bash
# Pass the name of a service to check ie: sh check-endpoint.sh staging-voting-app-vote
# Will run forever...
external_ip=""
while [ -z $external_ip ]; do
echo "Waiting for end point..."
external_ip=$(kubectl get svc $1 --template="{{range .status.loadBalancer.ingress}}{{.ip}}{{end}}")
[ -z "$external_ip" ] && sleep 10
done
echo 'End point ready:' && echo $external_ip
Using this in a Codefresh Step
I'm using this for a Codefresh pipeline and it passes a variable $endpoint when it's done.
GrabEndPoint:
title: Waiting for endpoint to be ready
image: codefresh/plugin-helm:2.8.0
commands:
- bash -c 'external_ip=""; while [ -z $external_ip ]; do echo "Waiting for end point..."; external_ip=$(kubectl get svc staging-voting-app-vote --template="{{range .status.loadBalancer.ingress}}{{.ip}}{{end}}"); [ -z "$external_ip" ] && sleep 10; done; echo "End point ready-" && echo $external_ip; cf_export endpoint=$external_ip'
Here's a generic bash function to watch with timeout, for any regexp in the output of a given command:
function watch_for() {
CMD="$1" # Command to watch. Variables should be escaped \$
REGEX="$2" # Pattern to search
ATTEMPTS=${3:-10} # Timeout. Default is 10 attempts (interval of second)
COUNT=0;
echo -e "# Watching for /$REGEX/ during $ATTEMPTS seconds, on the output of command:\n# $CMD"
until eval "$CMD" | grep -m 1 "$REGEX" || [[ $COUNT -eq $ATTEMPTS ]]; do
echo -e "$(( COUNT++ ))... \c"
sleep 1
done
if [[ $COUNT -eq $ATTEMPTS ]]; then
echo "# Limit of $ATTEMPTS attempts has exceeded."
return 1
fi
return 0
}
And here's how I used it to wait until a worker node gets an external IP (which took more than a minute):
$ watch_for "kubectl get nodes -l node-role.kubernetes.io/worker -o wide | awk '{print \$7}'" \
"[0-9]" 100
0... 1... 2... 3... .... 63... 64...
3.22.37.41
There's not really a "failed to set" condition because we will retry it forever. A failure might have been a transient error in the cloud provider or a quota issue that gets resolved over the course of hours or days, or any number of things. The only failure comes from "how long are you willing to wait?" - which only you can know.
We don't have a general "wait for expression" command because it ends up being arbitrarily complex and you're better off just coding that in a real language. Ergo the bash loop above. We could do better about having a 'watch' command, but it's still a timeout in the end.