Implementation Goal

Expose Zookeeper instance, running on kubernetes, to the internet.

(configuration & version information provided at the bottom)

Implementation Attempt

I currently have a minikube cluster running on ubuntu 14.04, backed by docker containers. I'm running a bare metal k8s cluster, and I'm trrying to expose a zookeeper service to the internet. Seeing as my cluster is not running on a cloud provider, I set up metallb, in order to provide a network-loadbalancer implementation for my zookeeper service.

On startup everything looks good, an external IP is assigned and I can access it from the same host via a curl command.

$ kubectl get pods -n metallb-system
NAME                          READY   STATUS    RESTARTS   AGE
controller-5c9894b5cd-9gh8m   1/1     Running   0          5h59m
speaker-j2z8q                 1/1     Running   0          5h59m


$ kubectl get svc
NAME         TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                         AGE
kubernetes   ClusterIP      10.xxx.xxx.xxx  <none>          443/TCP                         6d19h
zk-cs        LoadBalancer   10.xxx.xxx.xxx  172.1.1.x       2181:30035/TCP                  56m
zk-hs        LoadBalancer   10.xxx.xxx.xxx  172.1.1.x       2888:30664/TCP,3888:31113/TCP   6m15s

When I curl the above mentioned external IP's, I get a valid response

$ curl -D- "http://172.1.1.x:2181"
curl: (52) Empty reply from server

So far it all looks good, I can access the LB from outside the cluster with no issues, but this is where my lack of Kubernetes/Networking knowledge gets me.I'm finding it impossible to expose this LB to the internet. I've tried running minikube tunnel which I had high hopes for, only to be deeply disappointed.

Running a curl command from another node, whilst minikube tunnel is running will just see the request time out.

$ curl -D- "http://172.1.1.x:2181"
curl: (28) Failed to connect to 172.1.1.x port 2181: Timed out

At this point, as I mentioned before, I'm stuck. Is there any way that I can get this service exposed to the internet without giving my soul to AWS or GCP?

Any help will be greatly appreciated.

Service Configuration

apiVersion: v1
kind: Service
metadata:
  name: zk-hs
  labels:
    app: zk
spec:
  selector:
    app: zk
  ports:
    - port: 2888
      targetPort: 2888
      name: server
      protocol: TCP
    - port: 3888
      targetPort: 3888
      name: leader-election
      protocol: TCP
  clusterIP: ""
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  name: zk-cs
  labels:
    app: zk
spec:
  selector:
    app: zk
  ports:
    - name: client
      protocol: TCP
      port: 2181
      targetPort: 2181
  type: LoadBalancer
---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: zk-pdb
spec:
  selector:
    matchLabels:
      app: zk
  maxUnavailable: 1
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zk
spec:
  selector:
    matchLabels:
      app: zk
  serviceName: zk-hs
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  podManagementPolicy: OrderedReady
  template:
    metadata:
      labels:
        app: zk
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - zk
              topologyKey: "kubernetes.io/hostname"
      containers:
        - name: zookeeper
          imagePullPolicy: Always
          image: "library/zookeeper:3.6"
          resources:
            requests:
              memory: "1Gi"
              cpu: "0.5"
          ports:
            - containerPort: 2181
              name: client
            - containerPort: 2888
              name: server
            - containerPort: 3888
              name: leader-election
          volumeMounts:
            - name: datadir
              mountPath: /var/lib/zookeeper
            - name: zoo-config
              mountPath: /conf
      volumes:
        - name: zoo-config
          configMap:
            name: zoo-config
      securityContext:
        fsGroup: 2000
        runAsUser: 1000
        runAsNonRoot: true
  volumeClaimTemplates:
    - metadata:
        name: datadir
      spec:
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 10Gi
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: zoo-config
  namespace: default
data:
  zoo.cfg: |
    tickTime=10000
    dataDir=/var/lib/zookeeper
    clientPort=2181
    initLimit=10
    syncLimit=4

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 172.1.1.1-172.1.1.10

minikube: v1.13.1
docker: 18.06.3-ce