What is the equivalent command for minikube delete
in docker-for-desktop on OSX
As I understand, minikube creates a VM to host its kubernetes cluster but I do not understand how docker-for-desktop is managing this on OSX.
Tear down Kubernetes in Docker for OS X is quite an easy task.
Go to Preferences
, open Reset
tab, and click Reset Kubernetes cluster
.
All object that have been created with Kubectl before that will be deleted.
You can also reset docker VM image (Reset disk image
) and all settings (Reset to factory defaults
) or even uninstall Docker.
In recent Docker Edge versions for Mac ( 2.1.7 ) Preferences
design has been changed. Now you can reset Kubernetes cluster and other docker aspects by switching to the bug plane in the top right of Preferences
window:
Note: You are able to reset Kubernetes cluster only if it's enabled. If you uncheck "Enable Kubernetes" checkbox, "Reset Kubernetes cluster" button becomes inactive.
For convenience "Reset Kubernetes cluster" is also present on the Kubernetes tab in the main Preferences plane:
It's really under the hood in the code. Docker for Mac uses these components: Hyperkit, VPNkit and DataKit
Kubernetes runs in the same Hyperkit VM created for docker and the kube-apiserver is exposed.
You can connect to the VM with this:
screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
Then you can see all the Kubernetes processes in the VM:
linuxkit-025000000001:~# ps -Af | grep kube
1251 root 0:00 /usr/bin/logwrite -n kubelet /usr/bin/kubelet.sh
1288 root 0:51 kubelet --kubeconfig=/etc/kubernetes/kubelet.conf --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true --cluster-dns=10.96.0.10 --cluster-domain=cluster.local --cgroups-per-qos=false --enforce-node-allocatable= --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --cadvisor-port=0 --kube-reserved-cgroup=podruntime --system-reserved-cgroup=systemreserved --cgroup-root=kubepods --hostname-override=docker-for-desktop --fail-swap-on=false
3564 root 0:26 kube-scheduler --address=127.0.0.1 --leader-elect=true --kubeconfig=/etc/kubernetes/scheduler.conf
3616 root 1:45 kube-controller-manager --cluster-signing-key-file=/run/config/pki/ca.key --address=127.0.0.1 --root-ca-file=/run/config/pki/ca.crt --service-account-private-key-file=/run/config/pki/sa.key --kubeconfig=/etc/kubernetes/controller-manager.conf --cluster-signing-cert-file=/run/config/pki/ca.crt --leader-elect=true --use-service-account-credentials=true --controllers=*,bootstrapsigner,tokencleaner
3644 root 1:59 kube-apiserver --admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --requestheader-username-headers=X-Remote-User --requestheader-group-headers=X-Remote-Group --service-account-key-file=/run/config/pki/sa.pub --secure-port=6443 --insecure-port=8080 --insecure-bind-address=0.0.0.0 --requestheader-client-ca-file=/run/config/pki/front-proxy-ca.crt --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --requestheader-extra-headers-prefix=X-Remote-Extra- --advertise-address=192.168.65.3 --service-cluster-ip-range=10.96.0.0/12 --tls-private-key-file=/run/config/pki/apiserver.key --enable-bootstrap-token-auth=true --requestheader-allowed-names=front-proxy-client --tls-cert-file=/run/config/pki/apiserver.crt --proxy-client-key-file=/run/config/pki/front-proxy-client.key --proxy-client-cert-file=/run/config/pki/front-proxy-client.crt --allow-privileged=true --client-ca-file=/run/config/pki/ca.crt --kubelet-client-certificate=/run/config/pki/apiserver-kubelet-client.crt --kubelet-client-key=/run/config/pki/apiserver-kubelet-client.key --authorization-mode=Node,RBAC --etcd-servers=https://127.0.0.1:2379 --etcd-cafile=/run/config/pki/etcd/ca.crt --etcd-certfile=/run/config/pki/apiserver-etcd-client.crt --etcd-keyfile=/run/config/pki/apiserver-etcd-client.key
3966 root 0:01 /kube-dns --domain=cluster.local. --dns-port=10053 --config-dir=/kube-dns-config --v=2
4190 root 0:05 /usr/local/bin/kube-proxy --config=/var/lib/kube-proxy/config.conf
4216 65534 0:03 /sidecar --v=2 --logtostderr --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,SRV --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,SRV
4606 root 0:00 /compose-controller --kubeconfig --reconciliation-interval 30s
4905 root 0:01 /api-server --kubeconfig --authentication-kubeconfig --authorization-kubeconfig --etcd-servers=https://127.0.0.1:2379 --etcd-cafile=/etc/docker-compose/etcd/ca.crt --etcd-certfile=/etc/docker-compose/etcd/client.crt --etcd-keyfile=/etc/docker-compose/etcd/client.key --secure-port=9443 --tls-ca-file=/etc/docker-compose/tls/ca.crt --tls-cert-file=/etc/docker-compose/tls/server.crt --tls-private-key-file=/etc/docker-compose/tls/server.key
So if you uncheck the following box (unclear from the docs what command it uses):
You can see that the processes are removed:
linuxkit-025000000001:~# [ 6616.856404] cni0: port 2(veth5f6c8b28) entered disabled state
[ 6616.860520] device veth5f6c8b28 left promiscuous mode
[ 6616.861125] cni0: port 2(veth5f6c8b28) entered disabled state
linuxkit-025000000001:~#
linuxkit-025000000001:~# [ 6626.816763] cni0: port 1(veth87e77142) entered disabled state
[ 6626.822748] device veth87e77142 left promiscuous mode
[ 6626.823329] cni0: port 1(veth87e77142) entered disabled state
linuxkit-025000000001:~# ps -Af | grep kube
linuxkit-025000000001:~#
To reset Docker-desktop Kubernetes cluster using command line, put the following content to a file (dd-reset.sh
) and mark it executable ( chmod a+x dd-reset.sh
)
#!/bin/bash
dr='docker run -it --rm --privileged --pid=host debian nsenter -t 1 -m -u -n -i'
${dr} sh -c 'export PATH=$PATH:/containers/services/docker/rootfs/usr/bin:/containers/services/docker/rootfs/usr/local/bin:/var/lib/kube-binary-cache/ && \
if [ ! -e /var/run/docker.sock ] ; then ln -s /containers/services/docker/rootfs/var/run/docker.sock /var/run/docker.sock ; fi && \
kube-reset.sh'
sleep 3
echo "cluster resetted. restarting docker-desktop..."
osascript -e 'quit app "Docker"'
open --background -a Docker
echo "docker-desktop started. Wait 3-5 mins for kubernetes to start."
Explanation:
This method uses internal scripts from Docker-desktop VM. To make it work, some preparation of user environment is required.
I wasn't able to start Kubernetes cluster using kube-start.sh
script from inside the VM, so I've used MacOS commands to restart Docker application instead.
This method works even if your Kubernetes cluster is not enabled in Docker preferences at the moment, but it's required to enable Kubernetes at least once to use the script.
It was tested on Docker Edge for MacOS v2.2.2.0 (43066)
There is no guarantee that it will be compatible with earlier or later versions.
This version of Docker uses kubeadm
to initialize Kubernetes cluster. Scripts are located in the folder /containers/services/docker/rootfs/usr/bin
:
kube-pull.sh
(brings kubernetes binaries to VM)kube-reset.sh
(runs kube-stop.sh
and do kubeadm reset
+ some rm
stuff)kube-restart.sh
(runs kube-stop.sh
and kube-start.sh
)kube-start.sh
(runs kube-pull.sh
and kubelet.sh
)kube-stop.sh
(kills kubelet
and kube-apiserver
processes, and all k8s
containers)kubeadm-init.sh
(initializes Kubernetes cluster)kubelet.sh
(runs kubeadm-init.sh
and starts kubelet
binary)Cluster configuration is located in the file /containers/services/docker/lower/etc/kubeadm/kubeadm.yaml
Resources used: