How to Setup HA for Kubernetes on CentOS, Runing 1 Master 2 Worker already using kubeadm

11/15/2017

I have successfully setup Normal Cluster, Now when I am trying for HA setup following doc https://kubernetes.io/docs/admin/high-availability/

Here after copying etcd.yaml file in /etc/kubernetes/manifest I see 3 etcd container inside my cluster

default etcd-server-kuber-poc-app1 1/1 Running 1 2d default etcd-server-kuber-poc-app2 1/1 Running 72 20h kube-system etcd-kuber-poc-app1 1/1 Running 4 13d

But when I check logs for any etcd pods I see error like

2017-11-15 08:53:25.398815 E | discovery: error #0: x509: failed to load system roots and no roots provided

2017-11-15 08:53:25.398907 I | discovery: cluster status check: error connecting to https://discovery.etcd.io, retrying in 18h12m16s

Seems like missing certs for them

But I am not sure which certs to create and where to place

Yaml Content

apiVersion: v1
kind: Pod
metadata:
  name: etcd-server
spec:
  hostNetwork: true
  containers:
  - image: gcr.io/google_containers/etcd:3.0.17
    name: etcd-container
    command:
    - /usr/local/bin/etcd
    - --name
    - NODE-1
    - --initial-advertise-peer-urls
    - http://10.127.38.18:2380
    - --listen-peer-urls
    - http://10.127.38.18:2380
    - --advertise-client-urls
    - http://10.127.38.18:4001
    - --listen-client-urls
    - http://127.0.0.1:4001
    - --data-dir
    - /var/etcd/data
    - --discovery
    - https://discovery.etcd.io/9458bcd46077d558fd26ced5cb9f2a6a
    ports:
    - containerPort: 2380
      hostPort: 2380
      name: serverport
    - containerPort: 4001
      hostPort: 4001
      name: clientport
    volumeMounts:
    - mountPath: /var/etcd
      name: varetcd
    - mountPath: /etc/ssl
      name: etcssl
      readOnly: true
    - mountPath: /usr/share/ssl
      name: usrsharessl
      readOnly: true
    - mountPath: /var/ssl
      name: varssl
      readOnly: true
    - mountPath: /usr/ssl
      name: usrssl
      readOnly: true
    - mountPath: /usr/lib/ssl
      name: usrlibssl
      readOnly: true
    - mountPath: /usr/local/openssl
      name: usrlocalopenssl
      readOnly: true
    - mountPath: /etc/openssl
      name: etcopenssl
      readOnly: true
    - mountPath: /etc/pki/tls
      name: etcpkitls
      readOnly: true
  volumes:
  - hostPath:
      path: /var/etcd/data
    name: varetcd
  - hostPath:
      path: /etc/ssl
    name: etcssl
  - hostPath:
      path: /usr/share/ssl
    name: usrsharessl
  - hostPath:
      path: /var/ssl
    name: varssl
  - hostPath:
      path: /usr/ssl
    name: usrssl
  - hostPath:
      path: /usr/lib/ssl
    name: usrlibssl
  - hostPath:
      path: /usr/local/openssl
    name: usrlocalopenssl
  - hostPath:
      path: /etc/openssl
    name: etcopenssl
  - hostPath:
      path: /etc/pki/tls
    name: etcpkitls

So 2 Issue

1) How to Create Certs?

2) Where to Keep them?

-- Ganesh Rathore
etcd
kubernetes

1 Answer

11/15/2017

I don't think we can make kubeadm cluster as HA. your option is to recreate the cluster with kubespray https://github.com/kubespray/kubespray-cli tool, this will create the certificate with all the nodes.

for step by step instruction follow Kubernetes The Hard Way https://github.com/kelseyhightower/kubernetes-the-hard-way

-- sfgroups
Source: StackOverflow