nixos etcd.pem (kubernetes)

12/13/2019

While trying to install Kubernetes on nixos, using the following stanza:

services.kubernetes.masterAddress = "XXXXXX";
users.users.XXXXXX.extraGroups = [ "kubernetes" ];

services.kubernetes = {
  roles = ["master" "node"];
};

I hit the following issue:

open /var/lib/kubernetes/secrets/etcd.pem: no such file or directory

I recognize this as a TLS/SSL certificate, but how should I go about generating that file?

-- user3416536
etcd
kubernetes
nixos
ssl

1 Answer

12/27/2019

The article you used is really old. It was published 2017-07-21 so almost 2,5 years ago. You can be pretty sure it's outdated in one way or another however major NixOS approach to setting up kubernetes cluster from end user perspective may have not changed a lot during this time.

So, after familiarizing with it a bit more... I see that this is actually yet another approach to installing kubernetes cluster and it has nothing to do with "the hard way" I mentioned in my previous comment. On the contrary, it's the easiest kubernetes cluster setup I've ever seen. Actually you don't have to do anything but add a single entry in your configuration.nix and then run nixos-rebuild switch and you can expect everything to be up and running. But there is really a lot, not just a few things that NixOS takes care about "under the hood". Generating proper certificates is just one of many steps involved in kubernetes cluster setup. Keep in mind that Kubernetes installation from scratch is pretty complex task. Take a brief look at this article and you'll see what I mean. This is really amazing thing for educational purposes as there is probably no better way to understand something in-deep, than to build it from scratch, in the possibly most manual way.

On the other hand, if you just need to set up relatively quickly a working kubernetes cluster, Kubernetes the Hard Way won't be your choice. Fortunatelly there are a few solutions that give you possibility to set up your kubernetes cluster relatively quickly and simply.

One of them is Minikube. The other one which gives you possibility to set-up multi-node kubernetes cluster is kubeadm.

Going back to NixOS, I'm really impressed by how simple it is to set up your kubernetes cluster on this system, provided everything works as expected. But what if it doesn't ( and this is mainly what your question was about ) ? You may try to debug it on your own and try to look for a workaround of your issue or simply create an issue on NixOS project github page like this one. As you can see someone already reported exactly the same problem as yours. They say that on the 18.09 release it works properly so probably you're using newer version like 19.03. You can further read that there were some major changes like moving to mandatory pki in 19.03.

Take a closer look at this issue if you're particularly interested in running kubernetes on NixOS as there are a few advices and workarounds described there:

https://github.com/NixOS/nixpkgs/issues/59364#issuecomment-485122860 https://github.com/NixOS/nixpkgs/issues/59364#issuecomment-485249797

First of all make sure that your masterAddress is set properly i.e. as hostname, not ip address. As you put there only "XXXXXX" I can't guess what is currently set there. It's quite likely that when you set it e.g. to localhost, appropriate certificate would be generated properly:

services.kubernetes = {
   roles = ["master"];
   masterAddress = "localhost";
  };

You may also want to familiarize with this info in NixOS docs related with Kubernetes.

Let me know if it helped.

-- mario
Source: StackOverflow