I have a pod and two services. When I ping one of the service with its name from another pod, I am able to get the response. but not for another service.
$ kubectl get pod --show-labels
NAME READY STATUS RESTARTS AGE LABELS
cd-jenkins-6dbdb5c8f9-xcf44 1/1 Running 10 16d app.kubernetes.io/component=jenkins-master,app.kubernetes.io/instance=cd,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=jenkins,helm.sh/chart=jenkins-1.9.17,pod-template-hash=6dbdb5c8f9$ kubectl get svc --show-labels
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE LABELS
cd-jenkins ClusterIP 10.96.192.55 <none> 8080/TCP 16d app.kubernetes.io/component=jenkins-master,app.kubernetes.io/instance=cd,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=jenkins,helm.sh/chart=jenkins-1.9.17
cd-jenkins-agent ClusterIP 10.96.98.10 <none> 50000/TCP 16d app.kubernetes.io/component=jenkins-master,app.kubernetes.io/instance=cd,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=jenkins,helm.sh/chart=jenkins-1.9.17If I try pinging cd-jenkins-agent service from an another pod, it works fine.
But when I try pining cd-jenkins service from the same pod, it is not working.
I am wondering which pod is rensible for cd-jenkins-agent service?
$ kubectl exec -it nginx-86c57db685-pnkxx -- sh
# host cd-jenkins
cd-jenkins.default.svc.cluster.local has address 10.96.192.55
# host cd-jenkins-agent
cd-jenkins-agent.default.svc.cluster.local has address 10.96.98.10
# ping -c 2 cd-jenkins
PING cd-jenkins.default.svc.cluster.local (10.96.192.55) 56(84) bytes of data.
^C
--- cd-jenkins.default.svc.cluster.local ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 6ms
# ping -c 2 cd-jenkins-agent
PING cd-jenkins-agent.default.svc.cluster.local (10.96.98.10) 56(84) bytes of data.
64 bytes from cd-jenkins-agent.default.svc.cluster.local (10.96.98.10): icmp_seq=1 ttl=61 time=388 ms
64 bytes from cd-jenkins-agent.default.svc.cluster.local (10.96.98.10): icmp_seq=2 ttl=61 time=470 ms
--- cd-jenkins-agent.default.svc.cluster.local ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 2ms
rtt min/avg/max/mdev = 387.755/428.769/469.783/41.014 ms
UPDATE
Adding more information
$ kubectl describe service cd-jenkins-agent
Name: cd-jenkins-agent
Namespace: default
Labels: app.kubernetes.io/component=jenkins-master
app.kubernetes.io/instance=cd
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=jenkins
helm.sh/chart=jenkins-1.9.17
Annotations: <none>
Selector: app.kubernetes.io/component=jenkins-master,app.kubernetes.io/instance=cd
Type: ClusterIP
IP: 10.96.98.10
Port: slavelistener 50000/TCP
TargetPort: 50000/TCP
Endpoints:
Session Affinity: None
Events: <none>
$ kubectl describe pod cd-jenkins-6dbdb5c8f9-xcf44 |grep -i ip:
IP: 172.17.0.5
$ kubectl describe service cd-jenkins
Name: cd-jenkins
Namespace: default
Labels: app.kubernetes.io/component=jenkins-master
app.kubernetes.io/instance=cd
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=jenkins
helm.sh/chart=jenkins-1.9.17
Annotations: <none>
Selector: app.kubernetes.io/component=jenkins-master,app.kubernetes.io/instance=cd
Type: ClusterIP
IP: 10.96.192.55
Port: http 8080/TCP
TargetPort: 8080/TCP
Endpoints: 172.17.0.5:8080
Session Affinity: None
Events: <none>
Manifest content for all the pods:
$ kubectl get pod -oyaml
apiVersion: v1
items:
- apiVersion: v1
kind: Pod
metadata:
annotations:
checksum/config: df1933f788ea761f62ce06ba7b344e4245c42dc7d187cb3055cbfc10921ace86
creationTimestamp: 2020-03-07T14:16:09Z
generateName: cd-jenkins-6dbdb5c8f9-
labels:
app.kubernetes.io/component: jenkins-master
app.kubernetes.io/instance: cd
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: jenkins
helm.sh/chart: jenkins-1.9.17
pod-template-hash: 6dbdb5c8f9
name: cd-jenkins-6dbdb5c8f9-xcf44
namespace: default
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: cd-jenkins-6dbdb5c8f9
uid: c275c986-d0f2-4985-b367-b628149bd89e
resourceVersion: "113287"
selfLink: /api/v1/namespaces/default/pods/cd-jenkins-6dbdb5c8f9-xcf44
uid: c402430a-7ded-4aff-b316-e1ca08bac22a
spec:
containers:
- args:
- --argumentsRealm.passwd.$(ADMIN_USER)=$(ADMIN_PASSWORD)
- --argumentsRealm.roles.$(ADMIN_USER)=admin
- --httpPort=8080
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: JAVA_OPTS
- name: JENKINS_OPTS
- name: JENKINS_SLAVE_AGENT_PORT
value: "50000"
- name: ADMIN_PASSWORD
valueFrom:
secretKeyRef:
key: jenkins-admin-password
name: cd-jenkins
- name: ADMIN_USER
valueFrom:
secretKeyRef:
key: jenkins-admin-user
name: cd-jenkins
image: jenkins/jenkins:lts
imagePullPolicy: Always
livenessProbe:
failureThreshold: 5
httpGet:
path: /login
port: http
scheme: HTTP
initialDelaySeconds: 90
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: jenkins
ports:
- containerPort: 8080
name: http
protocol: TCP
- containerPort: 50000
name: slavelistener
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /login
port: http
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: "2"
memory: 4Gi
requests:
cpu: 50m
memory: 256Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /tmp
name: tmp
- mountPath: /var/jenkins_home
name: jenkins-home
- mountPath: /var/jenkins_config
name: jenkins-config
readOnly: true
- mountPath: /usr/share/jenkins/ref/secrets/
name: secrets-dir
- mountPath: /usr/share/jenkins/ref/plugins/
name: plugin-dir
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: cd-jenkins-token-w6xtk
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
initContainers:
- command:
- sh
- /var/jenkins_config/apply_config.sh
env:
- name: ADMIN_PASSWORD
valueFrom:
secretKeyRef:
key: jenkins-admin-password
name: cd-jenkins
- name: ADMIN_USER
valueFrom:
secretKeyRef:
key: jenkins-admin-user
name: cd-jenkins
image: jenkins/jenkins:lts
imagePullPolicy: Always
name: copy-default-config
resources:
limits:
cpu: "2"
memory: 4Gi
requests:
cpu: 50m
memory: 256Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /tmp
name: tmp
- mountPath: /var/jenkins_home
name: jenkins-home
- mountPath: /var/jenkins_config
name: jenkins-config
- mountPath: /usr/share/jenkins/ref/secrets/
name: secrets-dir
- mountPath: /usr/share/jenkins/ref/plugins
name: plugins
- mountPath: /var/jenkins_plugins
name: plugin-dir
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: cd-jenkins-token-w6xtk
readOnly: true
nodeName: minikube
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
runAsUser: 0
serviceAccount: cd-jenkins
serviceAccountName: cd-jenkins
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- emptyDir: {}
name: plugins
- emptyDir: {}
name: tmp
- configMap:
defaultMode: 420
name: cd-jenkins
name: jenkins-config
- emptyDir: {}
name: secrets-dir
- emptyDir: {}
name: plugin-dir
- name: jenkins-home
persistentVolumeClaim:
claimName: cd-jenkins
- name: cd-jenkins-token-w6xtk
secret:
defaultMode: 420
secretName: cd-jenkins-token-w6xtk
status:
conditions:
- lastProbeTime: null
lastTransitionTime: 2020-03-24T14:01:22Z
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: 2020-03-24T14:02:41Z
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: 2020-03-24T14:02:41Z
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: 2020-03-07T14:16:09Z
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://b68b1dc92b9d3ddbc2276b95a14029d6b14d6e19c616b833b8878f2ff43f54f1
image: jenkins/jenkins:lts
imageID: docker-pullable://jenkins/jenkins@sha256:7ea1d29c621a10d1e231013f62e32f0eb726dde15a4c219e5010564a6766daa8
lastState:
terminated:
containerID: docker://5b422b334d50502002959b5ab7478eb7aa8d69767fdcca9b39504314e120a347
exitCode: 143
finishedAt: 2020-03-24T13:57:24Z
reason: Error
startedAt: 2020-03-24T11:10:17Z
name: jenkins
ready: true
restartCount: 11
started: true
state:
running:
startedAt: 2020-03-24T14:01:26Z
hostIP: 192.168.99.107
initContainerStatuses:
- containerID: docker://63702bb2f84abafffcc3daea86fab5aa8df8989407a03878ff47e235b33a4369
image: jenkins/jenkins:lts
imageID: docker-pullable://jenkins/jenkins@sha256:7ea1d29c621a10d1e231013f62e32f0eb726dde15a4c219e5010564a6766daa8
lastState: {}
name: copy-default-config
ready: true
restartCount: 7
state:
terminated:
containerID: docker://63702bb2f84abafffcc3daea86fab5aa8df8989407a03878ff47e235b33a4369
exitCode: 0
finishedAt: 2020-03-24T14:01:21Z
reason: Completed
startedAt: 2020-03-24T14:01:11Z
phase: Running
podIP: 172.17.0.5
podIPs:
- ip: 172.17.0.5
qosClass: Burstable
startTime: 2020-03-07T14:16:09Z
- apiVersion: v1
kind: Pod
metadata:
creationTimestamp: 2020-03-07T14:52:54Z
labels:
jenkins/cd-jenkins-slave: "true"
jenkins/label: cd-jenkins-slavex
name: default-896lw
namespace: default
resourceVersion: "18328"
selfLink: /api/v1/namespaces/default/pods/default-896lw
uid: 47d45a2a-1cc6-4b13-8a1b-b5ec8fb10efd
spec:
containers:
- args:
- e169f2448e514120084b5e486bcf780a6e7c1bf0eee013c57704b10ebbb1270e
- default-896lw
env:
- name: JENKINS_SECRET
value: e169f2448e514120084b5e486bcf780a6e7c1bf0eee013c57704b10ebbb1270e
- name: JENKINS_TUNNEL
value: cd-jenkins-agent:50000
- name: JENKINS_AGENT_NAME
value: default-896lw
- name: JENKINS_NAME
value: default-896lw
- name: JENKINS_AGENT_WORKDIR
value: /home/jenkins
- name: JENKINS_URL
value: http://cd-jenkins.default.svc.cluster.local:8080
image: jenkins/jnlp-slave:3.27-1
imagePullPolicy: IfNotPresent
name: jnlp
resources:
limits:
cpu: 512m
memory: 512Mi
requests:
cpu: 512m
memory: 512Mi
securityContext:
privileged: false
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /home/jenkins
name: workspace-volume
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-pbs7v
readOnly: true
workingDir: /home/jenkins
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: minikube
nodeSelector:
beta.kubernetes.io/os: linux
priority: 0
restartPolicy: Never
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- emptyDir: {}
name: workspace-volume
- name: default-token-pbs7v
secret:
defaultMode: 420
secretName: default-token-pbs7v
status:
conditions:
- lastProbeTime: null
lastTransitionTime: 2020-03-07T14:52:54Z
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: 2020-03-07T14:56:35Z
message: 'containers with unready status: [jnlp]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: 2020-03-07T14:56:35Z
message: 'containers with unready status: [jnlp]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: 2020-03-07T14:52:54Z
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://675283a8493b685d7632276fbe7b5bec8957bc665d05f6bdcca0e647146a3ffd
image: jenkins/jnlp-slave:3.27-1
imageID: docker-pullable://jenkins/jnlp-slave@sha256:a5f42dab13d74c079a653408e45cda3ca2bc3483200bbaa44b88fae486ad7e05
lastState: {}
name: jnlp
ready: false
restartCount: 0
started: false
state:
terminated:
containerID: docker://675283a8493b685d7632276fbe7b5bec8957bc665d05f6bdcca0e647146a3ffd
exitCode: 143
finishedAt: 2020-03-07T14:53:38Z
reason: Error
startedAt: 2020-03-07T14:52:55Z
hostIP: 192.168.99.107
phase: Failed
qosClass: Guaranteed
startTime: 2020-03-07T14:52:54Z
- apiVersion: v1
kind: Pod
metadata:
creationTimestamp: 2020-03-24T11:12:58Z
generateName: nginx-86c57db685-
labels:
app: nginx
pod-template-hash: 86c57db685
name: nginx-86c57db685-pnkxx
namespace: default
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: nginx-86c57db685
uid: e1dfdf35-06a8-4277-948b-9c8538a44cd5
resourceVersion: "113057"
selfLink: /api/v1/namespaces/default/pods/nginx-86c57db685-pnkxx
uid: e1e05570-4aba-4c74-8865-6fd2a6c19bff
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-pbs7v
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: minikube
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: default-token-pbs7v
secret:
defaultMode: 420
secretName: default-token-pbs7v
status:
conditions:
- lastProbeTime: null
lastTransitionTime: 2020-03-24T11:12:58Z
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: 2020-03-24T14:01:16Z
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: 2020-03-24T14:01:16Z
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: 2020-03-24T11:12:58Z
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://a4e00ee2a882cf4e0f8521f97bdb9e9e46b9eee674a83e8b4cd2190b01ecaff1
image: nginx:latest
imageID: docker-pullable://nginx@sha256:2539d4344dd18e1df02be842ffc435f8e1f699cfc55516e2cf2cb16b7a9aea0b
lastState:
terminated:
containerID: docker://d53426252c397a50e33cea7c6d9e70ec1967ac3e230e5f790eade209259cf46f
exitCode: 0
finishedAt: 2020-03-24T13:57:22Z
reason: Completed
startedAt: 2020-03-24T11:13:06Z
name: nginx
ready: true
restartCount: 1
started: true
state:
running:
startedAt: 2020-03-24T14:01:16Z
hostIP: 192.168.99.107
phase: Running
podIP: 172.17.0.9
podIPs:
- ip: 172.17.0.9
qosClass: BestEffort
startTime: 2020-03-24T11:12:58Z
kind: List
metadata:
resourceVersion: ""
selfLink: ""Manifest for all the services:
$ kubectl get svc -oyaml
apiVersion: v1
items:
- apiVersion: v1
kind: Service
metadata:
creationTimestamp: 2020-03-07T14:16:09Z
labels:
app.kubernetes.io/component: jenkins-master
app.kubernetes.io/instance: cd
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: jenkins
helm.sh/chart: jenkins-1.9.17
name: cd-jenkins
namespace: default
resourceVersion: "13047"
selfLink: /api/v1/namespaces/default/services/cd-jenkins
uid: aca40b6c-33d0-439d-8398-bde7739ed2f6
spec:
clusterIP: 10.96.192.55
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
selector:
app.kubernetes.io/component: jenkins-master
app.kubernetes.io/instance: cd
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
- apiVersion: v1
kind: Service
metadata:
creationTimestamp: 2020-03-07T14:16:09Z
labels:
app.kubernetes.io/component: jenkins-master
app.kubernetes.io/instance: cd
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: jenkins
helm.sh/chart: jenkins-1.9.17
name: cd-jenkins-agent
namespace: default
resourceVersion: "13044"
selfLink: /api/v1/namespaces/default/services/cd-jenkins-agent
uid: 4d8cdb2a-7db8-4e61-8d63-474ad2983ed7
spec:
clusterIP: 10.96.98.10
ports:
- name: slavelistener
port: 50000
protocol: TCP
targetPort: 50000
selector:
app.kubernetes.io/component: jenkins-master
app.kubernetes.io/instance: cd
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
- apiVersion: v1
kind: Service
metadata:
creationTimestamp: 2020-03-07T12:41:08Z
labels:
component: apiserver
provider: kubernetes
name: kubernetes
namespace: default
resourceVersion: "148"
selfLink: /api/v1/namespaces/default/services/kubernetes
uid: 4881b523-a8e2-4ea5-a826-aa28060eb7a3
spec:
clusterIP: 10.96.0.1
ports:
- name: https
port: 443
protocol: TCP
targetPort: 8443
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
kind: List
metadata:
resourceVersion: ""
selfLink: ""If run kubectl describe svc servicename it should have Endpoints which will have IPs of the pods behind the service.
kubectl describe svc kubernetes
Name: kubernetes
Namespace: default
Labels: component=apiserver
provider=kubernetes
Annotations: <none>
Selector: <none>
Type: ClusterIP
IP: 10.96.0.1
Port: https 443/TCP
TargetPort: 6443/TCP
Endpoints: 172.17.0.2:6443
Session Affinity: None
Events: <none>If the Endpoints is empty that means the label selector in the service did not match the label on pods.