I have several docker images that I want to use with minikube
. I don't want to first have to upload and then download the same image instead of just using the local image directly. How do I do this?
Stuff I tried :
1. I tried running these commands (separately, deleting the instances of minikube both times and starting fresh)
kubectl run hdfs --image=fluxcapacitor/hdfs:latest --port=8989
kubectl run hdfs --image=fluxcapacitor/hdfs:latest --port=8989 imagePullPolicy=Never
Output :
NAME READY STATUS RESTARTS AGE
hdfs-2425930030-q0sdl 0/1 ContainerCreating 0 10m
It just gets stuck on some status but never reaches the ready state.
2. I tried creating a registry and then putting images into it but that didn't work either. I might've done that incorrectly but I can't find proper instructions to do this task.
Please provide instructions to use local docker images in local kubernetes instance.
OS: ubuntu 16.04
Docker : Docker version 1.13.1, build 092cba3
Kubernetes :
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.3", GitCommit:"029c3a408176b55c30846f0faedf56aae5992e9b", GitTreeState:"clean", BuildDate:"2017-02-15T06:40:50Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"linux/amd64"}
If someone could help me get a solution that uses docker-compose to do this, that'd be awesome. Thanks.
Edit :
Images loaded in eval $(minikube docker-env
:
REPOSITORY TAG IMAGE ID CREATED SIZE
fluxcapacitor/jupyterhub latest e5175fb26522 4 weeks ago 9.59 GB
fluxcapacitor/zeppelin latest fe4bc823e57d 4 weeks ago 4.12 GB
fluxcapacitor/prediction-pmml latest cae5b2d9835b 4 weeks ago 973 MB
fluxcapacitor/scheduler-airflow latest 95adfd56f656 4 weeks ago 8.89 GB
fluxcapacitor/loadtest latest 6a777ab6167c 5 weeks ago 899 MB
fluxcapacitor/hdfs latest 00fa0ed0064b 6 weeks ago 1.16 GB
fluxcapacitor/sql-mysql latest 804137671a8c 7 weeks ago 679 MB
fluxcapacitor/metastore-1.2.1 latest ea7ce8c5048f 7 weeks ago 1.35 GB
fluxcapacitor/cassandra latest 3cb5ff117283 7 weeks ago 953 MB
fluxcapacitor/apachespark-worker-2.0.1 latest 14ee3e4e337c 7 weeks ago 3.74 GB
fluxcapacitor/apachespark-master-2.0.1 latest fe60b42d54e5 7 weeks ago 3.72 GB
fluxcapacitor/package-java-openjdk-1.8 latest 1db08965289d 7 weeks ago 841 MB
gcr.io/google_containers/kubernetes-dashboard-amd64 v1.5.1 1180413103fd 7 weeks ago 104 MB
fluxcapacitor/stream-kafka-0.10 latest f67750239f4d 2 months ago 1.14 GB
fluxcapacitor/pipeline latest f6afd6c5745b 2 months ago 11.2 GB
gcr.io/google-containers/kube-addon-manager v6.1 59e1315aa5ff 3 months ago 59.4 MB
gcr.io/google_containers/kubedns-amd64 1.9 26cf1ed9b144 3 months ago 47 MB
gcr.io/google_containers/kube-dnsmasq-amd64 1.4 3ec65756a89b 5 months ago 5.13 MB
gcr.io/google_containers/exechealthz-amd64 1.2 93a43bfb39bf 5 months ago 8.37 MB
gcr.io/google_containers/pause-amd64
actually what happens here , your Minikube can't recognise your docker daemon as it is independent service.You have to first set your minikube-docker environment use below command to check
"eval $(minikube docker-env)"
If you run below command it will show where your minikube looks for docker.
~$ minikube docker-env
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.37.192:2376"
export DOCKER_CERT_PATH="/home/ubuntu/.minikube/certs"
export MINIKUBE_ACTIVE_DOCKERD="minikube"
**# To point your shell to minikube's docker-daemon, run:**
# eval $(minikube -p minikube docker-env)
You have to again build images once you setup minikube docker-env else it will fail.
you can either reuse the docker shell, with eval $(minikube docker-env)
, alternatively, you can leverage on docker save | docker load
across the shells.
As the README describes, you can reuse the Docker daemon from Minikube with eval $(minikube docker-env)
.
So to use an image without uploading it, you can follow these steps:
eval $(minikube docker-env)
docker build -t my-image .
)my-image
)imagePullPolicy
to Never
, otherwise Kubernetes will try to download the image.Important note: You have to run eval $(minikube docker-env)
on each terminal you want to use, since it only sets the environment variables for the current shell session.
What worked for me, based on the solution by @svenwltr:
# Start minikube
minikube start
# Set docker env
eval $(minikube docker-env)
# Build image
docker build -t foo:0.0.1 .
# Run in minikube
kubectl run hello-foo --image=foo:0.0.1 --image-pull-policy=Never
# Check that it's running
kubectl get pods
This Answer isnt limited to minikube!
Use a local registry:
docker run -d -p 5000:5000 --restart=always --name registry registry:2
Now tag your image properly:
docker tag ubuntu localhost:5000/ubuntu
Note that localhost should be changed to dns name of the machine running registry container.
Now push your image to local registry:
docker push localhost:5000/ubuntu
You should be able to pull it back:
docker pull localhost:5000/ubuntu
Now change your yaml file to use local registry.
Think about mounting volume at appropriate location to persist the images on registry.
update:
as Eli stated, you'll need to add the local registry as insecure in order to use http (may not apply when using localhost but does apply if using the local hostname)
Don't use http in production, make the effort for securing things up.
Adding to to @Farhad 's answer based on this answer,
This are the steps to setup a local registry.
Setup in local machine
Setup hostname in local machine: edit /etc/hosts
to add this line
docker.local 127.0.0.1
Now start a local registry (remove -d to run non-daemon mode) :
docker run -d -p 5000:5000 --restart=always --name registry registry:2
Now tag your image properly:
docker tag ubuntu docker.local:5000/ubuntu
Now push your image to local registry:
docker push docker.local:5000/ubuntu
Verify that image is pushed:
curl -X GET http://docker.local:5000/v2/ubuntu/tags/list
Setup in minikube
ssh into minikube with: minukube ssh
edit /etc/hosts
to add this line
docker.local <your host machine's ip>
Verify access:
curl -X GET http://docker.local:5000/v2/ubuntu/tags/list
Now if you try to pull, yo might get an http access error.
Enable insecure access:
If you are always planning to use minkube with this local setup then create a minikube to use insecure registry by default (wont work on existing cluster).
minikube start --insecure-registry="docker.local:5000"
else follow below steps:
systemctl stop docker
edit the docker serice file: get path from systemctl status docker
it might be :
/etc/systemd/system/docker.service.d/10-machine.conf or /usr/lib/systemd/system/docker.service
append this text (replace 192.168.1.4 with your ip)
--insecure-registry docker.local:5000 --insecure-registry 192.168.1.4:5000
to this line
ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox --insecure-registry 10.0.0.0/24
systemctl daemon-reload
systemctl start docker
try pulling:
docker pull docker.local:5000/ubuntu
Now change your yaml file to use local registry.
containers: - name: ampl-django image: dockerhub/ubuntu
to
containers:
- name: ampl-django
image: docker.local:5000/nymbleup
Don't use http in production, make the effort for securing things up.
In addition to the accepted answer, you can also achieve what you originally wanted (creating a deployment using the run
command) with the following command:
kubectl run hdfs --image=fluxcapacitor/hdfs:latest --port=8989 --generator=run-pod/v1
I found the information about the generator on the Kubernetes-dev forum:
If you're using
kubectl run
, it generates a manifest for you that happens to haveimagePullPolicy
set toAlways
by default. You can use this command to get animagePullPolicy
ofIfNotPresent
, which will work forminikube
:
kubectl run --image=<container> --generator=run-pod/v1
Dan Lorenc
https://groups.google.com/forum/#!topic/kubernetes-dev/YfvWuFr_XOM
One approach is to build the image locally and then do:
docker save imageNameGoesHere | pv | (eval $(minikube docker-env) && docker load)
minikube docker-env
might not return the correct info running under a different user / sudo. Instead you can run sudo -u yourUsername minikube docker-env
.
It should return something like:
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/home/chris/.minikube/certs"
export DOCKER_API_VERSION="1.23"
# Run this command to configure your shell:
# eval $(minikube docker-env)
If anyone is looking to come back to the local environment after setting the minikube env, use following command.
eval $(docker-machine env -u)
From the kubernetes docs:
https://kubernetes.io/docs/concepts/containers/images/#updating-images
The default pull policy is IfNotPresent which causes the Kubelet to skip pulling an image if it already exists. If you would like to always force a pull, you can do one of the following:
- set the imagePullPolicy of the container to Always;
- use :latest as the tag for the image to use;
- enable the AlwaysPullImages admission controller.
Or read the other way: Using the :latest tag forces images to always be pulled. If you use the eval $(minikube docker-env)
as mentioned above, then either don't use any tag, or assign a tag to your local image you can avoid Kubernetes trying to forcibly pull it.
There is now a Minikube Registry addon, this is probably the easiest way. Here is how to use it: https://minikube.sigs.k8s.io/docs/tasks/registry/insecure/
Note that I had DNS issues, might be a bug.
A simpler method that answers the original question "How to use local docker images with Minikube?", is to save the image to a tar file and load it into minikube:
# export the docker image to a tar file
docker save --output my-image.tar the.full.path.to/the/docker/image:the-tag
# set local environment variables so that docker commands go to the docker in minikube
eval $(minikube docker-env)
# or if on windows: @FOR /f "tokens=*" %i IN ('minikube docker-env') DO @%i
# import the docker image from the tar file into minikube
docker load --input my-image.tar
# cleanup - put docker back to normal
eval $(minikube docker-env -u)
# or if on windows: @FOR /f "tokens=*" %i IN ('minikube docker-env -u') DO @%i
Then running the image involves a command like the following. Make sure to include the "--image-pull-policy=Never" parameter.
kubectl run my-image --image=the.full.path.to/the/docker/image:the-tag --image-pull-policy=Never --port=80
Other answers suppose you use minikube with VM, so your local images are not accessible from minikube VM.
In case if you use minikube with --vm-driver=none
, you can easily reuse local images by setting image_pull_policy
to Never:
kubectl run hello-foo --image=foo --image-pull-policy=Never
or setting imagePullPolicy
field for cotainers in corresponding .yaml
manifests.
To add to the previous answers, if you have a tarball image, you can simply load it to you local docker set of images docker image load -i /path/image.tar
.Please remember to run it after eval $(minikube docker-env)
, since minikube does not share images with the locally installed docker engine.
what if you could just run k8s within docker's vm? there's native support for this with the more recent versions of docker desktop... you just need to enable that support.
https://www.docker.com/blog/kubernetes-is-now-available-in-docker-desktop-stable-channel/ https://www.docker.com/blog/docker-windows-desktop-now-kubernetes/
how i found this out:
while reading the docs for helm, they give you a brief tutorial how to install minikube. that tutorial installs minikube in a vm that's different/separate from docker.
so when it came time to install my helm charts, i couldn't get helm/k8s to pull the images i had built using docker. that's how i arrived here at this question.
so... if you can live with whatever version of k8s comes with docker desktop, and you can live with it running in whatever vm docker has, then maybe this solution is a bit easier than some of the others.
disclaimer: not sure how switching between windows/linux containers would impact anything.
One idea would be to save the docker image locally and later load it into minikube as follows:
Let say, for example, you already have puckel/docker-airflow image.
Save that image to local disk -
docker save puckel/docker-airflow > puckel_docker_airflow.tar
Now enter into minikube docker env -
eval $(minikube docker-env)
Load that locally saved image -
docker load < puckel_docker_airflow.tar
It is that simple and it works like a charm.