Is it possible to setup a local Kubernetes development environment (Minikube, etc.) from an AWS EC2 instance?

8/18/2018

Background : I am trying to learn and experiment a bit on docker and kubernetes in a "development/localhost" environment, that I could later replicate "for real" on some Cloud. But I'm running low on everything (disk capacity, memory, etc.) on my laptop. So I figured out "why not develop from the cloud ?"

I know AWS has some Kubernetes service, but if my understanding is correct, this is mostly to deploy already well configured stacks, and it is not very suited for the development of the stack configuration itself.

After searching a bit, I found out about Minikube, that helps us experiment our configs by running kubernetes deployments on a single machine. I'd like to setup a kubernetes + Minikube (or equivalent) development environment from an EC2 instance (ideally running Amazon Linux 2 OS).

I'm having a hard time figuring out

  • Is it actually possible to setup Minikube on EC2 ?
  • (If yes), how do I do it ? I tried following this answer but I'm getting stuck at registering the Virtualbox Repo and downloading Virtualbox command line tools
-- Cyril Duchon-Doris
amazon-ec2
amazon-linux
amazon-web-services
kubernetes
minikube

4 Answers

6/24/2019

Consider tools providing lightweight Kubernetes cluster for development/testing purposes like:

-- oleh.kovaliuk
Source: StackOverflow

8/19/2018

Go here: https://github.com/scholzj/aws-minikube

I'm running it on t2.small but it's difficult, you have to remove resource requests from the infrastructure deployments. That runs me about 14$ a month with a reserved instance. It may make better sense to go with GCP since they graciously pay for the master.

-- Lev Kuznetsov
Source: StackOverflow

8/18/2018

I am not sure if EC2 allows nested virtualization. If not, you can always use minikube --vm-driver=none. This is how I do on premises where there are constraints in running virtualboxes. But i have to admit, there's not a lot of good documentation around using minikube without vm. This i have not personally tried, check out canonical's lxd.
https://kubernetes.io/docs/getting-started-guides/ubuntu/local/

-- Bal Chua
Source: StackOverflow

8/18/2018

Heres how to do it

Start an ec2 instance with 8gb of ram and a public ip, ensure you can ssh to this box in the normal ways. Ensure its an unbuntu instance (I'm using 16.04).

once ssh'd into the instance run the following to update and install docker

sudo -i
apt-get update -y && apt-get install docker.io

Install minikube

curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/

Install kube cli

curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/v1.8.0/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/

now verify the version just to make sure you can see it

/usr/local/bin/minikube version

Add autocompletion to the current shell with

source <(kubectl completion bash)

Start the cluster with this (note the no vm driver line)

/usr/local/bin/minikube start --vm-driver=none

Check its up and running with this:

/usr/local/bin/minikube status

right that should have you a basic cluster running with no extra nodes :)

If you want a nice dashboard do the following (I am using windows here making use of wsl on windows 10, you can do this on mac or linux if you like but the steps are slightly different but as long as you can follow basic steps like setting variables you will be cool)

In order to see the gui on your local box you are going to need to run a dashboard and to do other useful stuff run kubectl locally

Please follow this to install kubectl locally

On windows you can use chocolatey like so:

choco install kubernetes-cli

Now download your admin.conf file from the ec2 instance using scp this is located in /etc/kubernetes.

Now set a local variable called KUBECONFIG and point to the file you just downloaded.

Go on to the ec2 instance and use this to install a dashboard.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/alternative/kubernetes-dashboard-arm.yaml

This dashboard is a dev dashboard do not use this in production :)

run the following command to find out what ip address the dashboard is running on

/usr/local/bin/kubectl get svc --namespace kube-system

output should look a bit like this:

root@ip-172-31-39-236:~# /usr/local/bin/kubectl get svc --namespace kube-system
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
kube-dns               ClusterIP   10.96.0.10      <none>        53/UDP,53/TCP   49m
kubernetes-dashboard   NodePort    10.109.248.81   <none>        80:30000/TCP    49m

now run this on your localbox to tunnel to the dashboard from the local machine

ssh -i ~/.ssh/keyfile.pem -L 8080:10.109.248.81:80 ubuntu@ec2-i-changed-this-for-this-post.eu-west-1.compute.amazonaws.com

now open a web browser at:

http://localhost:8080 

and you should now be able to see the dashboard. Which looks like this:

enter image description here

Sorry the post is so long but its pretty involved. also please note this is really only a dev machine if you are going to need a prod instance you need to do this with better security and probably not run stuff as root :)

One other thing, you may note kubectl locally isn't being used in this guide you can use it to hit the remote api if you use (locally)

kubectl proxy

There is a guide on this on kubernetes homepage here Also note the admin.conf probably has localhost as the server address, it needs to be the address of the ec2 instance and you'll need to make sure the port is accessible from your ip in your security group for the ec2 instance.

If you curl or browser to http://localhost:8001/api you should see this or something like it :)

{
  "kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "192.168.4.60:6443"
    }
  ]
}
-- krystan honour
Source: StackOverflow