I'm currently dealing with this situation.

Whenever I create multi-node cluster using Minikube, when I stop it and restart it again. It will lose track of the "middle" nodes, e.g. I create 4 nodes: m1, m2, m3, m4; by some reason Minikube loses track of m2 and m3.

Scenario:

Let's say I want to create a Kubernetes cluster with Vault, so then I create one a profile named "vault-cluster" with 4 nodes (1 control plane and 3 worker nodes):

$ minikube start --nodes 4 -p vault-cluster

Then when I stop them using:

minikube stop -p vault-cluster

Expected behaviour:

Output:

✋  Stopping node "vault-cluster"  ...
✋  Stopping node "vault-cluster-m02"  ...
✋  Stopping node "vault-cluster-m03"  ...
✋  Stopping node "vault-cluster-m04"  ...
🛑  4 nodes stopped.

So when when I started again:

Output:

$ minikube start -p vault-cluster
😄  [vault-cluster] minikube v1.20.0 on Microsoft Windows 10 Pro 10.0.19042 Build 19042
✨  Using the virtualbox driver based on existing profile
🎉  minikube 1.21.0 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.21.0
💡  To disable this notice, run: 'minikube config set WantUpdateNotification false'

👍  Starting control plane node vault-cluster in cluster vault-cluster
🔄  Restarting existing virtualbox VM for "vault-cluster" ...
🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.6 ...
🔗  Configuring CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image kubernetesui/dashboard:v2.1.0
    ▪ Using image kubernetesui/metrics-scraper:v1.0.4
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
👍  Starting node vault-cluster-m02 in cluster vault-cluster
🔄  Restarting existing virtualbox VM for "vault-cluster-m02" ...
🌐  Found network options:
    ▪ NO_PROXY=192.168.99.120
    ▪ no_proxy=192.168.99.120
🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.6 ...
    ▪ env NO_PROXY=192.168.99.120
🔎  Verifying Kubernetes components...
👍  Starting node vault-cluster-m03 in cluster vault-cluster
🔄  Restarting existing virtualbox VM for "vault-cluster-m03" ...
🌐  Found network options:
    ▪ NO_PROXY=192.168.99.120,192.168.99.121
    ▪ no_proxy=192.168.99.120,192.168.99.121
🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.6 ...
    ▪ env NO_PROXY=192.168.99.120
    ▪ env NO_PROXY=192.168.99.120,192.168.99.121
🔎  Verifying Kubernetes components...
👍  Starting node vault-cluster-m04 in cluster vault-cluster
🔄  Restarting existing virtualbox VM for "vault-cluster-m04" ...
🌐  Found network options:
    ▪ NO_PROXY=192.168.99.120,192.168.99.121,192.168.99.122
    ▪ no_proxy=192.168.99.120,192.168.99.121,192.168.99.122
🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.6 ...
    ▪ env NO_PROXY=192.168.99.120
    ▪ env NO_PROXY=192.168.99.120,192.168.99.121
    ▪ env NO_PROXY=192.168.99.120,192.168.99.121,192.168.99.122
🔎  Verifying Kubernetes components...
🏄  Done! kubectl is now configured to use "vault-cluster" cluster and "default" namespace by default

ACTUAL BEHAVIOUR:

$ minikube stop -p vault-cluster
✋  Stopping node "vault-cluster"  ...
✋  Stopping node "vault-cluster-m04"  ...
✋  Stopping node "vault-cluster-m04"  ...
✋  Stopping node "vault-cluster-m04"  ...

So when this is what happens when I try to start cluster again:

$ minikube start -p vault-cluster
😄  [vault-cluster] minikube v1.20.0 on Microsoft Windows 10 Pro 10.0.19042 Build 19042
✨  Using the virtualbox driver based on existing profile
👍  Starting control plane node vault-cluster in cluster vault-cluster
🔄  Restarting existing virtualbox VM for "vault-cluster" ...
🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.6 ...
🔗  Configuring CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
    ▪ Using image kubernetesui/metrics-scraper:v1.0.4
    ▪ Using image kubernetesui/dashboard:v2.1.0
🌟  Enabled addons: default-storageclass, dashboard
👍  Starting node vault-cluster-m04 in cluster vault-cluster
🔄  Restarting existing virtualbox VM for "vault-cluster-m04" ...
🌐  Found network options:
    ▪ NO_PROXY=192.168.99.120
    ▪ no_proxy=192.168.99.120
🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.6 ...
    ▪ env NO_PROXY=192.168.99.120
🔎  Verifying Kubernetes components...
👍  Starting node vault-cluster-m04 in cluster vault-cluster
🏃  Updating the running virtualbox "vault-cluster-m04" VM ...
🌐  Found network options:
    ▪ NO_PROXY=192.168.99.120,192.168.99.123
    ▪ no_proxy=192.168.99.120,192.168.99.123
🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.6 ...
    ▪ env NO_PROXY=192.168.99.120
    ▪ env NO_PROXY=192.168.99.120,192.168.99.123
🔎  Verifying Kubernetes components...
👍  Starting node vault-cluster-m04 in cluster vault-cluster
🏃  Updating the running virtualbox "vault-cluster-m04" VM ...
🌐  Found network options:
    ▪ NO_PROXY=192.168.99.120,192.168.99.123
    ▪ no_proxy=192.168.99.120,192.168.99.123
🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.6 ...
    ▪ env NO_PROXY=192.168.99.120
    ▪ env NO_PROXY=192.168.99.120,192.168.99.123
🔎  Verifying Kubernetes components...
🏄  Done! kubectl is now configured to use "vault-cluster" cluster and "default" namespace by default

This is the output when I least the nodes:

$ minikube node list -p vault-cluster
vault-cluster   192.168.99.120
vault-cluster-m04       192.168.99.123
vault-cluster-m04       192.168.99.123
vault-cluster-m04       192.168.99.123

Any ideas what could be wrong?

Environment:

• Windows 10 Pro

• Virtual Box 6.1

$ minikube version
minikube version: v1.20.0
commit: c61663e942ec43b20e8e70839dcca52e44cd85ae

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:20:00Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}