We have created a single-master three-node worker cluster on AWS using Terraform, user-data YAML files, and CoreOS AMIs. The cluster works as expected but we are now in need to scale the master's up from one to three for redundancy purposes. My question is: other than using etcd clustering and/or the information provided on http://kubernetes.io/docs/admin/high-availability/, do we have any options to deploy a new or scale-up the existing cluster with multi-master nodes? Let me know if more details are required to answer this question.
The kops project can set up a high-availability master for you when creating a cluster.
Pass the following when you create the cluster (replacing the zones with whatever is relevant to you):
--master-zones=us-east-1b,us-east-1c,us-east-1d
Additionally, it can export Terraform files if you want to continue to use Terraform.