Deploying an ephemeral volume-on-demand and sharing that among pods

8/4/2015

I would like to achieve the following functionality:

  • when a given pod (let's call it application pod) is deployed on a node, another pod providing an ephemeral volume is deployed before that, if such "volume pod" has not existed on the target node yet
  • the number of application pods can be scaled up and down, and all application pods on the same node share the single volume pod

The first requirement assumes a kind of dependency definition among pods (just like it can be done among Marathon apps in case of Marathon).

The second requirement assumes that an ephemeral volume created in a container in a pod can be attached to other container(s) in other pod(s).

It is important that the volume is ephemeral (i.e. there is no host directory or attached storage, that could be mapped to the application). Also, it is important that it is not on GCE.

Please advise how such a setup can be achieved with Kubernetes.

I think such dynamic, dependency-based deployment would be welcomed by everyone. Also, sharing ephemeral volumes (e.g. files stored on a tmpfs volume, once such volume is supported by Docker) may be interesting for others, too.

-- janosi
kubernetes

1 Answer

9/18/2015

We do not support dependencies at the moment. In the future will support a "daemon" scheduler which can run a pod on every node, but not only if some other pod is being scheduled to a node. In the future we might support existence dependencies, but that's more for creation: create Pod P iff Service S exists.

We also do not support refcounted local storage, which seems to be what you're proposing. There are a lot of very unclear semantics in your quick sketch. In order to consider something as complex as this, we would have to really think through the corner cases. If this is something you want us to consider, you're welcome to file a proposal on GitHub with use cases and details.

In the mean time, it sounds like you want a workflow manager and hostPath volumes.

-- Tim Hockin
Source: StackOverflow