Background : I am trying to learn and experiment a bit on docker and kubernetes in a "development/localhost" environment, that I could later replicate "for real" on some Cloud. But I'm running low on everything (disk capacity, memory, etc.) on my laptop. So I figured out "why not develop from the cloud ?"
I know AWS has some Kubernetes service, but if my understanding is correct, this is mostly to deploy already well configured stacks, and it is not very suited for the development of the stack configuration itself.
After searching a bit, I found out about Minikube, that helps us experiment our configs by running kubernetes deployments on a single machine. I'd like to setup a kubernetes + Minikube (or equivalent) development environment from an EC2 instance (ideally running Amazon Linux 2 OS).
I'm having a hard time figuring out
Consider tools providing lightweight Kubernetes cluster for development/testing purposes like:
Go here: https://github.com/scholzj/aws-minikube
I'm running it on t2.small but it's difficult, you have to remove resource requests from the infrastructure deployments. That runs me about 14$ a month with a reserved instance. It may make better sense to go with GCP since they graciously pay for the master.
I am not sure if EC2 allows nested virtualization. If not, you can always use minikube --vm-driver=none. This is how I do on premises where there are constraints in running virtualboxes. But i have to admit, there's not a lot of good documentation around using minikube without vm. This i have not personally tried, check out canonical's lxd.
https://kubernetes.io/docs/getting-started-guides/ubuntu/local/
Heres how to do it
Start an ec2 instance with 8gb of ram and a public ip, ensure you can ssh to this box in the normal ways. Ensure its an unbuntu instance (I'm using 16.04).
once ssh'd into the instance run the following to update and install docker
sudo -i
apt-get update -y && apt-get install docker.io
Install minikube
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
Install kube cli
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/v1.8.0/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
now verify the version just to make sure you can see it
/usr/local/bin/minikube version
Add autocompletion to the current shell with
source <(kubectl completion bash)
Start the cluster with this (note the no vm driver line)
/usr/local/bin/minikube start --vm-driver=none
Check its up and running with this:
/usr/local/bin/minikube status
right that should have you a basic cluster running with no extra nodes :)
If you want a nice dashboard do the following (I am using windows here making use of wsl on windows 10, you can do this on mac or linux if you like but the steps are slightly different but as long as you can follow basic steps like setting variables you will be cool)
In order to see the gui on your local box you are going to need to run a dashboard and to do other useful stuff run kubectl locally
Please follow this to install kubectl locally
On windows you can use chocolatey like so:
choco install kubernetes-cli
Now download your admin.conf file from the ec2 instance using scp this is located in /etc/kubernetes.
Now set a local variable called KUBECONFIG
and point to the file you just downloaded.
Go on to the ec2 instance and use this to install a dashboard.
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/alternative/kubernetes-dashboard-arm.yaml
This dashboard is a dev dashboard do not use this in production :)
run the following command to find out what ip address the dashboard is running on
/usr/local/bin/kubectl get svc --namespace kube-system
output should look a bit like this:
root@ip-172-31-39-236:~# /usr/local/bin/kubectl get svc --namespace kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP 49m
kubernetes-dashboard NodePort 10.109.248.81 <none> 80:30000/TCP 49m
now run this on your localbox to tunnel to the dashboard from the local machine
ssh -i ~/.ssh/keyfile.pem -L 8080:10.109.248.81:80 ubuntu@ec2-i-changed-this-for-this-post.eu-west-1.compute.amazonaws.com
now open a web browser at:
http://localhost:8080
and you should now be able to see the dashboard. Which looks like this:
Sorry the post is so long but its pretty involved. also please note this is really only a dev machine if you are going to need a prod instance you need to do this with better security and probably not run stuff as root :)
One other thing, you may note kubectl locally isn't being used in this guide you can use it to hit the remote api if you use (locally)
kubectl proxy
There is a guide on this on kubernetes homepage here Also note the admin.conf probably has localhost as the server address, it needs to be the address of the ec2 instance and you'll need to make sure the port is accessible from your ip in your security group for the ec2 instance.
If you curl or browser to http://localhost:8001/api
you should see this or something like it :)
{
"kind": "APIVersions",
"versions": [
"v1"
],
"serverAddressByClientCIDRs": [
{
"clientCIDR": "0.0.0.0/0",
"serverAddress": "192.168.4.60:6443"
}
]
}