How can I fix MongoError: no mongos proxy available on GKE

6/3/2019

I am trying to deploy and Express api on GKE, with a Mongo StatefulSet.

googlecloud_ssd.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: fast
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd

mongo-statefulset.yaml

apiVersion: v1
kind: Service
metadata:
  name: mongo
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  clusterIP: None
  selector:
    role: mongo
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongo
spec:
  serviceName: "mongo"
  replicas: 2
  template:
    metadata:
      labels:
        role: mongo
        environment: test
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongo
          image: mongo
          command:
            - mongod
            - "--replSet"
            - rs0
            - "--smallfiles"
            - "--noprealloc"
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-persistent-storage
              mountPath: /data/db
        - name: mongo-sidecar
          image: cvallance/mongo-k8s-sidecar
          env:
            - name: MONGO_SIDECAR_POD_LABELS
              value: "role=mongo,environment=test"
  volumeClaimTemplates:
  - metadata:
      name: mongo-persistent-storage
      annotations:
        volume.beta.kubernetes.io/storage-class: "fast"
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 10Gi

I deployed my Express app and it works perfect, I then deployed Mongo using the above yaml config.

Having set the connection string in express as: "mongodb://mongo-0.mongo,mongo-1.mongo:27017/"

I can see the updated pod(s) not starting.

Looking at the logs for that container I see

{
 insertId:  "a9tu83g211w2a6"  
 labels: {…}  
 logName:  "projects/<my-project-id>/logs/express"  
 receiveTimestamp:  "2019-06-03T14:19:14.142238836Z"  
 resource: {…}  
 severity:  "ERROR"  
 textPayload:  "[ ERROR ] MongoError: no mongos proxy available
"  
 timestamp:  "2019-06-03T14:18:56.132989616Z"  
}

enter image description here

I am unsure how to debug / fix MongoError: no mongos proxy available

Edit So I scaled down my replicas to 1 on each and it's now working.

I'm confused as to why this won't work more than 1 replica.

enter image description here

-- Tim J
google-kubernetes-engine
kubernetes
mongodb

1 Answer

6/10/2019

The connection to your Mongodb database doesn't work for two reasons:

  1. You cannot connect to high-available MongoDB deployment running inside your Kubernetes cluster using Pods DNS names. These unique POD names: mongo-0.mongo, mongo-1.mongo, with corresponding FQDNs as mongo-0.mongo.default.svc.cluster.local, mongo-1.mongo.default.svc.cluster.local, can be only reached within the K8S cluster. You have an Express web application that runs on client side (Web browser), and needs to connect to your mongodb from outside of cluster.

  2. Connection string: you should connect to primary node via Kubernetes service name, that abstracts access to the Pods behind the replica sets.

Solution:

Create a separate Kubernetes Service of LoadBalancer or NodePort type for your Primary ReplicaSet, and use <ExternalIP_of_LoadBalancer>:27017 in your connection string.

I would encourage you to take a look at official mongodb helm chart, to see what kind of manifest files are required to satisfy your case.

Hint: use '--set service.type=LoadBalancer' with this helm chart

-- Nepomucen
Source: StackOverflow