Jenkins app is not accessible outside Kubernetes cluster

12/19/2018

On CentOS 7.4, I have set up a Kubernetes master node, pulled down jenkins image and deployed it to the cluster defining the jenkins service on a NodePort as below.

I can curl the jenkins app from the worker or master nodes using the IP defined by the service. But, I can not access the Jenkins app (dashboard) from my browser (outside cluster) using the public IP of the master node.

[administrator@abcdefgh ~]$ kubectl get nodes
NAME           STATUS   ROLES    AGE   VERSION
abcdefgh   Ready    master   19h   v1.13.1
hgfedcba   Ready    <none>   19h   v1.13.1

[administrator@abcdefgh ~]$ sudo docker pull jenkinsci/jenkins:2.154-alpine

[administrator@abcdefgh ~]$ sudo docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy                v1.13.1             fdb321fd30a0        5 days ago          80.2MB
k8s.gcr.io/kube-controller-manager   v1.13.1             26e6f1db2a52        5 days ago          146MB
k8s.gcr.io/kube-apiserver            v1.13.1             40a63db91ef8        5 days ago          181MB
k8s.gcr.io/kube-scheduler            v1.13.1             ab81d7360408        5 days ago          79.6MB
jenkinsci/jenkins                    2.154-alpine        aa25058d8320        2 weeks ago         222MB
k8s.gcr.io/coredns                   1.2.6               f59dcacceff4        6 weeks ago         40MB
k8s.gcr.io/etcd                      3.2.24              3cab8e1b9802        2 months ago        220MB
quay.io/coreos/flannel               v0.10.0-amd64       f0fad859c909        10 months ago       44.6MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        12 months ago       742kB

[administrator@abcdefgh ~]$ ls -l
total 8
-rw------- 1 administrator administrator 678 Dec 18 06:12 jenkins-deployment.yaml
-rw------- 1 administrator administrator 410 Dec 18 06:11 jenkins-service.yaml

[administrator@abcdefgh ~]$ cat jenkins-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: jenkins-ui
spec:
  type: NodePort
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      name: ui
  selector:
    app: jenkins-master
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins-discovery
spec:
  selector:
    app: jenkins-master
  ports:
    - protocol: TCP
      port: 50000
      targetPort: 50000
      name: jenkins-slaves

[administrator@abcdefgh ~]$ cat jenkins-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins-master
    spec:
      containers:
        - image: jenkins/jenkins:2.154-alpine
          name: jenkins
          ports:
            - containerPort: 8080
              name: http-port
            - containerPort: 50000
              name: jnlp-port
          env:
            - name: JAVA_OPTS
              value: -Djenkins.install.runSetupWizard=false
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
      volumes:
        - name: jenkins-home
          emptyDir: {}

[administrator@abcdefgh ~]$ kubectl create -f jenkins-service.yaml
service/jenkins-ui created
service/jenkins-discovery created

[administrator@abcdefgh ~]$ kubectl get services
NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
jenkins-discovery   ClusterIP   10.98.--.--     <none>        50000/TCP        19h
jenkins-ui          NodePort    10.97.--.--     <none>        8080:31587/TCP   19h
kubernetes          ClusterIP   10.96.--.--     <none>        443/TCP          20h

[administrator@abcdefgh ~]$ kubectl create -f jenkins-deployment.yaml
deployment.extensions/jenkins created

[administrator@abcdefgh ~]$ kubectl get deployments
NAME      READY   UP-TO-DATE   AVAILABLE   AGE
jenkins   1/1     1            1           19h

[administrator@abcdefgh ~]$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE
default       jenkins-6497cf9dd4-f9r5b               1/1     Running   0          19h
kube-system   coredns-86c58d9df4-jfq5b               1/1     Running   0          20h
kube-system   coredns-86c58d9df4-s4k6d               1/1     Running   0          20h
kube-system   etcd-abcdefgh                          1/1     Running   1          20h
kube-system   kube-apiserver-abcdefgh                1/1     Running   1          20h
kube-system   kube-controller-manager-abcdefgh       1/1     Running   5          20h
kube-system   kube-flannel-ds-amd64-2w68w            1/1     Running   1          20h
kube-system   kube-flannel-ds-amd64-6zl4g            1/1     Running   1          20h
kube-system   kube-proxy-9r4xt                       1/1     Running   1          20h
kube-system   kube-proxy-s7fj2                       1/1     Running   1          20h
kube-system   kube-scheduler-abcdefgh                1/1     Running   8          20h

[administrator@abcdefgh ~]$ kubectl describe pod jenkins-6497cf9dd4-f9r5b
Name:               jenkins-6497cf9dd4-f9r5b
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               hgfedcba/10.41.--.--
Start Time:         Tue, 18 Dec 2018 06:32:50 -0800
Labels:             app=jenkins-master
                    pod-template-hash=6497cf9dd4
Annotations:        <none>
Status:             Running
IP:                 10.244.--.--
Controlled By:      ReplicaSet/jenkins-6497cf9dd4
Containers:
  jenkins:
    Container ID:   docker://55912512a7aa1f782784690b558d74001157f242a164288577a85901ecb5d152
    Image:          jenkins/jenkins:2.154-alpine
    Image ID:       docker-pullable://jenkins/jenkins@sha256:b222875a2b788f474db08f5f23f63369b0f94ed7754b8b32ac54b8b4d01a5847
    Ports:          8080/TCP, 50000/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Running
      Started:      Tue, 18 Dec 2018 07:16:32 -0800
    Ready:          True
    Restart Count:  0
    Environment:
      JAVA_OPTS:  -Djenkins.install.runSetupWizard=false
    Mounts:
      /var/jenkins_home from jenkins-home (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-wqph5 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  jenkins-home:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  default-token-wqph5:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-wqph5
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

[administrator@abcdefgh ~]$ kubectl describe svc jenkins-ui
Name:                     jenkins-ui
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 app=jenkins-master
Type:                     NodePort
IP:                       10.97.--.--
Port:                     ui  8080/TCP
TargetPort:               8080/TCP
NodePort:                 ui  31587/TCP
Endpoints:                10.244.--.--:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

# Check if NodePort along with Kubernetes ports are open
[administrator@abcdefgh ~]$ sudo su root
[root@abcdefgh administrator]# systemctl start firewalld
[root@abcdefgh administrator]# firewall-cmd --permanent --add-port=6443/tcp        # Kubernetes API Server
Warning: ALREADY_ENABLED: 6443:tcp
success
[root@abcdefgh administrator]# firewall-cmd --permanent --add-port=2379-2380/tcp   # etcd server client API
Warning: ALREADY_ENABLED: 2379-2380:tcp
success
[root@abcdefgh administrator]# firewall-cmd --permanent --add-port=10250/tcp       # Kubelet API
Warning: ALREADY_ENABLED: 10250:tcp
success
[root@abcdefgh administrator]# firewall-cmd --permanent --add-port=10251/tcp       # kube-scheduler
Warning: ALREADY_ENABLED: 10251:tcp
success
[root@abcdefgh administrator]# firewall-cmd --permanent --add-port=10252/tcp       # kube-controller-manager
Warning: ALREADY_ENABLED: 10252:tcp
success
[root@abcdefgh administrator]# firewall-cmd --permanent --add-port=10255/tcp       # Read-Only Kubelet API
Warning: ALREADY_ENABLED: 10255:tcp
success
[root@abcdefgh administrator]# firewall-cmd --permanent --add-port=31587/tcp       # NodePort of jenkins-ui service
Warning: ALREADY_ENABLED: 31587:tcp
success
[root@abcdefgh administrator]# firewall-cmd --reload
success

[administrator@abcdefgh ~]$ kubectl cluster-info
Kubernetes master is running at https://10.41.--.--:6443
KubeDNS is running at https://10.41.--.--:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

[administrator@hgfedcba ~]$ curl 10.41.--.--:8080
curl: (7) Failed connect to 10.41.--.--:8080; Connection refused

# Successfully curl jenkins app using its service IP from the worker node
[administrator@hgfedcba ~]$ curl 10.97.--.--:8080

  <!DOCTYPE html><html><head resURL="/static/5882d14a" data-rooturl="" data-resurl="/static/5882d14a">
    <title>Dashboard [Jenkins]</title><link rel="stylesheet" ...
    ...

Would you know how to do that? Happy to provide additional logs. Also, I have installed jenkins from yum on another similar machine without any docker or kubernetes and it's possible to access it through 10.20.30.40:8080 in my browser so there is no provider firewall preventing me from doing that.

-- Robin
kubernetes

1 Answer

12/19/2018

Your Jenkins Service is of type NodePort. That means that a specific port number, on any node within your cluster, will deliver your Jenkins UI.

When you described your Service, you can see that the port assigned was 31587.

You should be able to browse to http://SOME_IP:31587

-- Rawkode
Source: StackOverflow