Exposing Neo4j Bolt using Kubernetes Ingress

6/16/2019

I'm trying to build a Neo4j Learning Tool for some of our Trainings. I want to use Kubernetes to spin up a Neo4j Pod for each participant to use. Currently I struggle exposing the bolt endpoint using an Ingress and I don't know why. Here are my deployment configs:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: neo4j
  namespace: learn
  labels:
    app: neo-manager
    type: database
spec:
  replicas: 1
  selector:
    matchLabels:
      app: neo-manager
      type: database
  template:
    metadata:
      labels:
        app: neo-manager
        type: database
    spec:
      containers:
        - name: neo4j
          imagePullPolicy: IfNotPresent
          image: neo4j:3.5.6
          ports:
            - containerPort: 7474
            - containerPort: 7687
              protocol: TCP
---
kind: Service
apiVersion: v1
metadata:
  name: neo4j-service
  namespace: learn
  labels:
    app: neo-manager
    type: database
spec:
  selector:
    app: neo-manager
    type: database
  ports:
    - port: 7687
      targetPort: 7687
      name: bolt
      protocol: TCP
    - port: 7474
      targetPort: 7474
      name: client
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: neo4j-ingress
  namespace: learn
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - host: learn.neo4j.com
      http:
        paths:
          - path: /
            backend:
              serviceName: neo4j-service
              servicePort: 7474
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: learn
data:
  7687: "learn/neo4j-service:7687"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: learn
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-nginx
  template:
    metadata:
      labels:
        app: ingress-nginx
    spec:
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0-beta.16
          args:
            - /nginx-ingress-controller
            - --tcp-services-configmap=${POD_NAMESPACE}/tcp-services
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace

The client gets exposed nicely and it reachable under learn.neo4j.com but I don't know where to point it to to connect to the DB using bolt. Whatever I try, it fails saying ServiceUnavailable: Websocket Connection failure (WebSocket network error: The operation couldn’t be completed. Connection refused in the console). What am I missing?

-- Urr4
bolt
kubernetes
neo4j
nginx-ingress

3 Answers

6/16/2019

I haven't used kubernetes ingress in this context before, but I think that when you use HTTP or HTTPS to connect to Neo4J, you still require external availability to connect to the the bolt port (7687). Does your setup allow for that?

-- terryf82
Source: StackOverflow

6/17/2019

The nginx-ingress-controller by default creates http(s) proxies only.

In your case you're trying to use a different protocol (bolt) so you need to configure your ingress controller in order for it to make a TCP proxy.

In order to do so, you need to create a configmap (in the nginx-ingress-controller namespace) similar to the following:

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
data:
  7687: "<your neo4j namespace>/neo4j-service:7687"

Then, make sure your ingress controller has the following flag in its command:

--tcp-services-configmap tcp-services

This will make your nginx-ingress controller listen to port 7687 with a TCP proxy.

You can delete the neo4j-bolt-ingress Ingress, that's not going to be used.

Of course you have to ensure that the ingress controller correctly exposes the 7687 port the same way it does with ports 80 and 443, and possibly you'll have to adjust the settings of any firewall and load balancer you might have.

Source: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/

-- whites11
Source: StackOverflow

6/17/2019

It automatically tries to connect to port 7687 by default - if you enter the connection url http://learn.neo4j.bolt.com:80 (or https), it works.

-- Lars
Source: StackOverflow