How to create ordered Kubernetes deployment with non-headless service?

3/16/2017

I need to have pod1 and pod2 be created before pod3. This means that I need order. I also want pod3 to be exposed outside of the cluster (externally) at port 80. At first I was thinking that I want to organize these creations in a StatefulSet.

However, it looks like Stateful Sets require a Headless Service.

StatefulSets currently require a Headless Service to be responsible for the network identity of the Pods. You are responsible for creating this Service.

This means

Sometimes you don’t need or want load-balancing and a single service IP. In this case, you can create “headless” services by specifying "None" for the cluster IP (spec.clusterIP).

I need pod3 to be exposed (through a service) outside the cluster and headless services does not allow me to do this. What is the proper way to do this in Kubernetes?

-- writofmandamus
kubernetes

1 Answer

6/1/2017

FYI, a headless service can coexist with other types of services, such as loadBalancer. Simply create an additional service with a different name and the same selector.

The more idiomatic way to do this is to change your pod3 application to handle outages of your DB and API servers. (Applications should not crash or become unusable just because the DB or other is temporarily unreachable.) I realize this is not so easy in some applications, (eh-hem, Ruby on Rails, eh-hem,) in those cases you can write an entrypoint (E.G. bash script) for the pod3 container, and that entrypoint simply loops until it can reach the other two services, then uses exec to replace the process and initialize the pod3 app. I have implemented this and it works very well. Here's a simplified example from memory:

#!/bin/bash

while ! $(mysqladmin ping); do
  sleep 2
done

exec node app.js
# Or exec whatever arguments were passed to the container init.
# exec $@

Whether you can improve your app or or have to use a custom entrypoint, the same code that would check the reachability of pod2 and pod1 can be added as a k8s readinessProbe on the pod3 container. - This will ensure pod3 is removed/not added as an endpoint to your service until the readinessProbe passes. Obviously this only helps if your app waits to receive requests before reaching out to it's dependencies.

Note: all this holds true if you were to swap the pod abstraction for the container abstraction, I.E. 3 containers in one pod.

statefulSets create near replicas of the same pod definition, so this is not the abstraction you want. Instead I would use a distinct statefulSet for your DB service, (to ensure it gets shutdown down before being started elsewhere,) and 2 distinct deployments for your API server and app, and configure them in such a way that init order does not matter, as suggested above.

-- damick
Source: StackOverflow