K
Q

SSH kubernetes pod that is not successfully initialized

9/23/2020

I am trying to debug jenkins kubernetes pod which is not initializing. The pod is in initialized state.

Once I run kubectl get pods --all-namespaces | grep jenkins

I am getting

infrastructure jenkins-fdfc9cf6c-2tvvm 0/1 Init:0/1 0 4m

I am able to extract the logs using

kubectl logs jenkins-fdfc9cf6c-2tvvm -c copy-default-config -n infrastructure

Is there any way that I can actually SSH to this pod?

I tried kubectl -n infrastructure exec -it jenkins-fdfc9cf6c-2tvvm /bin/bash, but I am getting error error: unable to upgrade connection: container not found ("jenkins").

This is how my container status looks like

  containerStatuses:
  - image: jenkins/jenkins:lts
    imageID: ""
    lastState: {}
    name: jenkins
    ready: false
    restartCount: 0
    state:
      waiting:
        reason: PodInitializing

Any advice how to do this?

-- Bob
kubernetes

Similar Questions

Pods failed to start after switch cni plugin from flannel to calico and then flannel
Running command in loop for n times inside kubernetes pods container
Kubernetes ExternalName, exposing 10.0.2.2
kubernetes rbac authorization layer seems to give incorrect result
How Scale increases by adding new container in host?

2 Answers

9/23/2020

I don't think you can run the bash of a pod that is not initializing. But you can start the pod with sleep command first and then run kubectl exec command to run the bash of that pod.

apiVersion: v1
kind: Pod
...
spec:
  containers:
  - name: ***
    image: ***
    command: ["sleep 3600"]

By doing this, the pod won't really run the docker's command, and will initialize correctly. Then you can run kubectl exec -it -n infrastructure jenkins-fdfc9cf6c-2tvvm /bin/bash to access the bash of that pod, then run the original command manually and see what will happen.

-- Rui
Source: StackOverflow
9/23/2020

I can't put this as a comment, so I'll post this as an answer and will be editing it. 

NAMESPACE        NAME                      READY   STATUS      RESTARTS   AGE
infrastructure   jenkins-fdfc9cf6c-2tvvm   0/1     Init:0/1    0          4m

Try doing kubectl -n infrastructure get pod jenkins-fdfc9cf6c-2tvvm -o yaml and checking the Pod's Phase and ContainerStatus in status: section of the YAML. Below is an example from my own test jnkns install. 

status:
  phase: Running
  containerStatuses:
  - containerID: xxxxx
    image: jenkins/jenkins:lts
    name: jenkins
    ready: true
    restartCount: 0
    state:
      running:
        startedAt: "2020-09-14T10:46:15Z"

I tried kubectl -n infrastructure exec -it jenkins-fdfc9cf6c-2tvvm /bin/bash, am getting error error: unable to upgrade connection: container not found ("jenkins").

The exec merely executes a command in a first container of pod (unless -c is specified). So if your container is stuck in some "weird" state, you might be not able to connect to it.

Additionally, one must separate the command from the pod name by -- (dash dash) in order to help the kubectl parser know that you didn't intend that /bin/bash for it .

$ kubectl exec --help
Usage:
  kubectl exec (POD | TYPE/NAME) [-c CONTAINER] [flags] -- COMMAND [args...] [options]

So the command should be: kubectl -n infrastructure exec -it jenkins-fdfc9cf6c-2tvvm -- /bin/bash

Official doc says that

Pod conditions

A Pod has a PodStatus, which has an array of PodConditions through which the Pod has or has not passed:

That is why I think i'd be interesting to check your deployment file. In my case I was able running Jenkins with no Init containers. You can find my YAML here.

--- UPD#1 I have played with the yaml you have provided and I'm able to achieve the Pod stuck in a different states :)

NAME                           READY   STATUS     RESTARTS   AGE
jenkins-7d9459df65-f4sv8       0/1     Init:0/1   0          1s

what is the output of kubectl describe pod <your jenkins pod>? If your pod is stuck in Init:0/1 then the problem is not with the Jenkins container, but the InitContainer

-- Nick
Source: StackOverflow