I'm fairly new to Elixir and container orchestration technologies like Kubernetes. I know that Elixir runs on the BEAM, which I've heard makes it possible to easily run programs on a network of computers. From what I understand of Kubernetes, Kubernetes manages a swarm of containers in a virtual network.
My question: would it be a good idea to use Kubernetes to manage a scalable swarm of containers running an Elixir program? If so, are there any pre-configured Kubernetes setups that make this easy?
We just moved to kubernetes for container management at my startup. I have a couple opinions on this:
If the goal is simply to distribute an elixir application over a "swarm" of nodes and / or machines, Kubernetes adds quite a bit of complexity and plumbing towards that goal. You end up having not just the learning curve of BEAM distribution, but of the kubernetes ecosystem itself.
However, kubernetes and elixir applications can work really well together, especially if your application deployment is more complex than pure elixir projects. For example we have custom services run from javascript, python, elixir, and golang in our application deployment (not too mention a dozen random tools).
Kubernetes helps extend the ability to distribute and manage nodes for all of these services with one strategy.
To answer your question: it can be a good idea, but I'd recommend first learning to manage the elixir program distributed by itself. Its not quite as simple as everyone makes it out to be, although its better than most programming technologies for other languages IMO. Here's a good guide to getting started with distributed elixir/erlang: http://engineering.pivotal.io/post/how-to-set-up-an-elixir-cluster-on-amazon-ec2/.
You can isolate that learning curve, and once you feel comfortable with it move on to kubernetes. Here's a really good resource on getting started with swarming via kubernetes: http://bitwalker.org/posts/2016-08-04-clustering-in-kubernetes/
Oops, one final comment: the advantages of tackling both learning curves are that you end up with the power of distributed elixir, the rest of your application deployment can keep up, and the kubernetes echosystem enables a lot of very cool tech like auto-scaling, helping unlock you out of a cloud provider, etc.
Our team had used elixir builded binary and docker/consul for three years.
When the servers per service are less than three, just used containers to manage are fine.
Otherwise, it is painful to scale your services(servers are expensive and you can not figure out the pressure of your services).
I had experienced this and tried using Swarm. It works like a charm! You can easily scale your services and manage servers.
So, when you meeting many servers, it is a good idea to use Kubernetes to manage a scalable swarm of containers running an Elixir program.