I need to have pod1 and pod2 be created before pod3. This means that I need order. I also want pod3 to be exposed outside of the cluster (externally) at port 80. At first I was thinking that I want to organize these creations in a StatefulSet.
However, it looks like Stateful Sets require a Headless Service.
StatefulSets currently require a Headless Service to be responsible for the network identity of the Pods. You are responsible for creating this Service.
This means
Sometimes you don’t need or want load-balancing and a single service IP. In this case, you can create “headless” services by specifying "None" for the cluster IP (spec.clusterIP).
I need pod3 to be exposed (through a service) outside the cluster and headless services does not allow me to do this. What is the proper way to do this in Kubernetes?
FYI, a headless service
can coexist with other types of service
s, such as loadBalancer
. Simply create an additional service
with a different name and the same selector
.
The more idiomatic way to do this is to change your pod3 application to handle outages of your DB and API servers. (Applications should not crash or become unusable just because the DB or other is temporarily unreachable.) I realize this is not so easy in some applications, (eh-hem, Ruby on Rails, eh-hem,) in those cases you can write an entrypoint (E.G. bash script) for the pod3 container
, and that entrypoint simply loops until it can reach the other two services, then uses exec
to replace the process and initialize the pod3 app. I have implemented this and it works very well. Here's a simplified example from memory:
#!/bin/bash
while ! $(mysqladmin ping); do
sleep 2
done
exec node app.js
# Or exec whatever arguments were passed to the container init.
# exec $@
Whether you can improve your app or or have to use a custom entrypoint, the same code that would check the reachability of pod2 and pod1 can be added as a k8s readinessProbe
on the pod3 container
. - This will ensure pod3 is removed/not added as an endpoint
to your service
until the readinessProbe
passes. Obviously this only helps if your app waits to receive requests before reaching out to it's dependencies.
Note: all this holds true if you were to swap the pod
abstraction for the container
abstraction, I.E. 3 container
s in one pod
.
statefulSet
s create near replicas of the same pod
definition, so this is not the abstraction you want. Instead I would use a distinct statefulSet
for your DB service, (to ensure it gets shutdown down before being started elsewhere,) and 2 distinct deployment
s for your API server and app, and configure them in such a way that init order does not matter, as suggested above.