Done according to this article.
I installed Kubernetes. Then installed etcd cluster that works via HTTPS and listens to the localhost interface only (reachable from inside any Docker container). Now I need persistent volume to install DB cluster. Chose Portworx. It generated daemonset YAML-config. Here is the description of installed daemonset:
# kubectl describe daemonset portworx --namespace=kube-system
Name: portworx
Selector: name=portworx
Node-Selector: <none>
Labels: name=portworx
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"extensions/v1beta1","kind":"DaemonSet","metadata":{"annotations":{"portworx.com/install-source":"http://install.portworx.com/?c=bp_clust...
portworx.com/install-source=http://install.portworx.com/?c=bp_cluster&k=etcd:https://127.0.0.1:2379&kbver=1.11.0&s=/dev/xvda1&d=ens3&m=ens3&stork=false&ca=/etc/kubernetes/pki/etcd/ca.crt%%20&cert=/etc...
Desired Number of Nodes Scheduled: 2
Current Number of Nodes Scheduled: 2
Number of Nodes Scheduled with Up-to-date Pods: 2
Number of Nodes Scheduled with Available Pods: 0
Number of Nodes Misscheduled: 0
Pods Status: 2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: name=portworx
Service Account: px-account
Containers:
portworx:
Image: portworx/oci-monitor:1.3.4
Port: <none>
Host Port: <none>
Args:
-k
etcd:https://127.0.0.1:2379
-c
bp_cluster
-d
ens3
-m
ens3
-s
/dev/xvda1
-ca
/etc/kubernetes/pki/etcd/ca.crt
-cert
/etc/kubernetes/pki/etcd/server.crt
-key
/etc/kubernetes/pki/etcd/server.key
-x
kubernetes
Liveness: http-get http://127.0.0.1:9001/status delay=840s timeout=1s period=30s #success=1 #failure=3
Readiness: http-get http://127.0.0.1:9015/health delay=0s timeout=1s period=10s #success=1 #failure=3
Environment:
PX_TEMPLATE_VERSION: v3
Mounts:
/etc/pwx from etcpwx (rw)
/etc/systemd/system from sysdmount (rw)
/host_proc/1/ns from proc1nsmount (rw)
/opt/pwx from optpwx (rw)
/var/run/dbus from dbusmount (rw)
/var/run/docker.sock from dockersock (rw)
Volumes:
dockersock:
Type: HostPath (bare host directory volume)
Path: /var/run/docker.sock
HostPathType:
etcpwx:
Type: HostPath (bare host directory volume)
Path: /etc/pwx
HostPathType:
optpwx:
Type: HostPath (bare host directory volume)
Path: /opt/pwx
HostPathType:
proc1nsmount:
Type: HostPath (bare host directory volume)
Path: /proc/1/ns
HostPathType:
sysdmount:
Type: HostPath (bare host directory volume)
Path: /etc/systemd/system
HostPathType:
dbusmount:
Type: HostPath (bare host directory volume)
Path: /var/run/dbus
HostPathType:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 22m daemonset-controller Created pod: portworx-67w7m
Normal SuccessfulCreate 22m daemonset-controller Created pod: portworx-mxtr8
But in the log of portworx I see that it is trying to connect to etcd via plain HTTP and obviously get error because cannot interpreter the response wrapped to SSL:
# kubectl logs -f pod/portworx-67w7m --namespace=kube-system
<some logs are erased du to lack of relevance>
Jul 02 13:19:25 ip-172-31-18-91 px-runc[25417]: time="2018-07-02T13:19:25Z" level=error msg="Could not load config file /etc/pwx/config.json due to: Error in obtaining etcd version: Get http://127.0.0.1:2379/version: net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x15\\x03\\x01\\x00\\x02\\x02\". Please visit http://docs.portworx.com for more information."
Jul 02 13:19:25 ip-172-31-18-91 px-runc[25417]: PXPROCS: px daemon exited with code: 1
Jul 02 13:19:25 ip-172-31-18-91 px-runc[25417]: 2107
Jul 02 13:19:25 ip-172-31-18-91 px-runc[25417]: 2018-07-02 13:19:25,474 INFO exited: pxdaemon (exit status 1; not expected)
What I am doing wrong?
I have no idea why they didn't surface the "cannot read the -cert
file" error, but you specified /etc/kubernetes/pki/etcd/server.crt
in the options but did not volume mount /etc/kubernetes/pki
into the container. For obvious reasons, kubernetes will not automatically volume mount its pki directory, thus, you must specify it.
If that DaemonSet
was generated for you (as it appears based on the annotation), then what happened is that they are expecting the certs to live in /etc/pwx/etcdcerts
(it's in their manual provisioning docs also), so when you provided a non-/etc
path, the two worlds collided.