I spun up an Openshift 4.6 cluster on AWS, 3 masters, 2 Workers for learning/play. Since it was just for learning, I shut down all nodes at once using the AWS Web Console. When I brought them back up a few days later, the console will not spin up.
So I followed this doc on restarting the cluster. Didn't work, so I started looking at how the pods were doing for the console itself. I ran oc get pods -n openshift-console -o wide -w
and got:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
console-59f557f67d-ks5kq 0/1 Pending 0 13m <none> <none> <none> <none>
console-59f557f67d-q6zrc 0/1 UnexpectedAdmissionError 0 4d6h <none> ip-10-0-139-41.us-west-2.compute.internal <none> <none>
console-59f557f67d-w4q7l 0/1 UnexpectedAdmissionError 0 56m <none> ip-10-0-131-234.us-west-2.compute.internal <none> <none>
console-59f557f67d-zvxzn 0/1 UnexpectedAdmissionError 0 59m <none> ip-10-0-131-234.us-west-2.compute.internal <none> <none>
downloads-55f4ff79-lqdj7 1/1 Running 0 59m 10.131.0.4 ip-10-0-208-19.us-west-2.compute.internal <none> <none>
downloads-55f4ff79-mrfzn 1/1 Running 0 59m 10.131.0.13 ip-10-0-208-19.us-west-2.compute.internal <none> <none>
So seeing that there were a few messed up pods, I wanted to look at their logs, but I don't know how to target just a specific pod by the NAME
column above. I tried each of these:
oc get pods/console-59f557f67d-q6zrc
oc get podtemplates/console-59f557f67d-q6zrc
I consistently get Error from server (NotFound): pods "console-59f557f67d-q6zrc" not found
.
I then found the command oc get pods -n openshift-console -o name
which reveals:
pod/console-59f557f67d-ks5kq
pod/console-59f557f67d-q6zrc
pod/console-59f557f67d-w4q7l
pod/console-59f557f67d-zvxzn
pod/downloads-55f4ff79-lqdj7
pod/downloads-55f4ff79-mrfzn
So I was right, it's a "pod", but then if I try to run anything like oc logs <name>
it returns the same error that it can't be found. Is this a bug? Does Openshift think there are Pods around that no longer exist and is routing to those Pods despite not existing?
If not, what resource type is the thing under the NAME
column? How do I target it with say an oc logs
or oc delete
command?
I discovered the correct syntax in this Redhat Bugzilla Issue.
The correct syntax is to place the name as another argument after the namespace declaration.
Examples:
oc describe pod -n openshift-console console-59f557f67d-zvxzn
oc logs pod -n openshift-console console-59f557f67d-zvxzn
oc delete pod -n openshift-console console-59f557f67d-zvxzn
I'll follow up and update when I find this reference in the official docs or command line help.