kubectl container not able to connect with Kubernetes

10/8/2017

I am using RancherOS as host and trying to setup kubectl container. I have modified the image and just changed the kubectl version to the latest (1.8.0) and added proxy settings to the Dockerfile because without it, docker build was failing to run the apk command. Also, Kubernetes is being managed by Rancher server. I downloaded the kubectl CLI config from the Rancher UI. It's shown below:

apiVersion: v1
kind: Config
clusters:
- cluster:
    api-version: v1
    server: "https://rancher.dev.abc.net/r/projects/1a6842/kubernetes:6443"
  name: "test"
contexts:
- context:
    cluster: "test"
    user: "test"
  name: "test"
current-context: "test"
users:
- name: "test"
  user:
    token: "QmFzaWMgTnpV9UZ3hPVVV4TXpaRFJrSTFSRFpDTkNOa2hSUTNscGNsSXpjMXAxVUdacVZUWk9NWFZaYVVGd1NqUk5UazVDUkZSM1lWZFhUZz09"

Dockerfile:

FROM docker.artifactory.abc.net/alpine:3.6

# Required for apk to install openssl
ENV http_proxy='http://proxy.abc.net:8080'  \
    https_proxy='http://proxy.abc.net:8080' \
    no_proxy='localhost,abc.net'

ADD https://storage.googleapis.com/kubernetes-release/release/v1.8.0/bin/linux/amd64/kubectl /usr/local/bin/kubectl

ENV HOME=/config

RUN set -x && \
    apk add --no-cache curl ca-certificates && \
    chmod +x /usr/local/bin/kubectl && \
    \
    # Create non-root user (with a randomly chosen UID/GUI).
    adduser kubectl -Du 2342 -h /config && \
    \
    # Basic check it works.
    kubectl version --client

USER kubectl

ENTRYPOINT ["/usr/local/bin/kubectl"]

Also tried adding the following to the Dockerfile but to no avail.

COPY .kube/chain.pem /config/.kube/ca.crt
RUN cat /config/.kube/ca.crt

Now when i run the command,

$ docker run --rm --user $UID -v ~rancher/kubectl/.kube:/config/.kube kubectl:v1.8.0 version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"6e937839ac04a38cac63e6a7a306c5d035fe7b0a", GitTreeState:"clean", BuildDate:"2017-09-28T22:57:57Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: x509: certificate signed by unknown authority

As seen above, client version is showing fine but while connecting to server, it fails. I copied the ca.crt file inside the ~rancher/kubectl/.kube dir. Also tried renaming the file to ca.pem but it's not working. Not sure what parameter has to be provided so kubectl can get the crt file.

-- Technext
kubectl
kubernetes
rancher

1 Answer

10/9/2017

So i finally got it working. No change in Dockerfile. In the .kube/config file shown above, i just had to add the following entry:

certificate-authority: /config/.kube/ca.crt

So the .kube/config file now looks as shown below:

apiVersion: v1
kind: Config
clusters:
- cluster:
    api-version: v1
    certificate-authority: /config/.kube/ca.crt
    server: "https://rancher.dev.abc.net/r/projects/1a6842/kubernetes:6443"
  name: "test"
contexts:
- context:
    cluster: "test"
    user: "test"
  name: "test"
current-context: "test"
users:
- name: "test"
  user:
    token: "QmFzaWMgTnpV9UZ3hPVVV4TXpaRFJrSTFSRFpDTkNOa2hSUTNscGNsSXpjMXAxVUdacVZUWk9NWFZaYVVGd1NqUk5UazVDUkZSM1lWZFhUZz09"

Finally, i could see the server version. Phew...

$ docker run --rm --user $UID -v ~rancher/kubectl/.kube:/config/.kube kubectl:v1.8.0 version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"6e937839ac04a38cac63e6a7a306c5d035fe7b0a", GitTreeState:"clean", BuildDate:"2017-09-28T22:57:57Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"7+", GitVersion:"v1.7.2-rancher1", GitCommit:"eda266858c448156b6d6fee372ff43ffb458a70c", GitTreeState:"clean", BuildDate:"2017-08-03T17:22:27Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
-- Technext
Source: StackOverflow