I'm running a Kubernetes cluster on AWS and need to configure a replicated MongoDB 4.2 Database. I'm using StatefulSets in order for other Pods (e.g., REST API NodeJS Pod) to easily connect to the mongo instances (example dsn: "mongodb://mongo-0.mongo,mongo-1.mongo,mongo-2.mongo:27017/app").

mongo-configmap.yaml (provides a shell script to perform the replication initialization upon mongo container creation):

apiVersion: v1
kind: ConfigMap
metadata:
  name: mongo-init
data:
  init.sh: |
    #!/bin/bash
    # wait for the readiness health check to pass 
    until ping -c 1 ${HOSTNAME}.mongo; do
      echo "waiting for DNS (${HOSTNAME}.mongo)..."
      sleep 2
    done
    until /usr/bin/mongo --eval 'printjson(db.serverStatus())'; do
      echo "connecting to local mongo..."
      sleep 2
    done
    echo "connected to local."
    HOST=mongo-0.mongo:27017
    until /usr/bin/mongo --host=${HOST} --eval 'printjson(db.serverStatus())'; do
      echo "connecting to remote mongo..."
      sleep 2
    done
    echo "connected to remote."
    if [[ "${HOSTNAME}" != 'mongo-0' ]]; then
      until /usr/bin/mongo --host=${HOST} --eval="printjson(rs.status())" \
            | grep -v "no replset config has been received"; do
        echo "waiting for replication set initialization"
        sleep 2
      done
      echo "adding self to mongo-0"
      /usr/bin/mongo --host=${HOST} --eval="printjson(rs.add('${HOSTNAME}.mongo'))"
    fi
    if [[ "${HOSTNAME}" == 'mongo-0' ]]; then
      echo "initializing replica set"
      /usr/bin/mongo --eval="printjson(rs.initiate(\
          {'_id': 'rs0', 'members': [{'_id': 0, \
           'host': 'mongo-0.mongo:27017'}]}))"
    fi
    echo "initialized"
    while true; do
      sleep 3600
    done

mongo-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: mongo
  labels:
    app: mongo
spec:
  clusterIP: None
  ports:
  - port: 27017
  selector:
    app: mongo

mongo-statefulset.yaml (2 containers inside one Pod, 1 for the actual DB, the other for initialization of the replication):

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongo
  labels:
    app: mongo
spec:
  selector:
    matchLabels:
      app: mongo
  serviceName: "mongo"
  replicas: 3
  template:
    metadata:
      labels:
        app: mongo
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: mongodb
        image: mongo:4.2
        command:
        - mongod
        args:
        - --replSet
        - rs0
        - "--bind_ip_all"
        ports:
        - containerPort: 27017
          name: web
        volumeMounts:
        - name: database
          mountPath: /data/db
        livenessProbe:
          exec:
            command:
            - /usr/bin/mongo
            - --eval
            - db.serverStatus()
          initialDelaySeconds: 10
          timeoutSeconds: 10
      - name: init-mongo
        image: mongo:4.2
        command:
        - bash
        - /config/init.sh
        volumeMounts:
        - name: config
          mountPath: /config
      volumes:
      - name: config
        configMap:
          name: "mongo-init"
  volumeClaimTemplates:
  - metadata:
      name: database
      annotations:
        volume.beta.kubernetes.io/storage-class: mongodb-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5Gi

After applying these configurations, the 3 mongo pods start running (mongo-0, mongo-1, mongo-2). However, other pods can't connect to these mongo pods. Further look into the mongo-0 pods (should be primary instance) reveals that the replication did not work.

kubectl exec -it mongo-0 -- /bin/bash

Then running 'mongo' to start the mongo shell, and entering 'rs.status()' into the mongo shell results in the following output:

{
    "info" : "run rs.initiate(...) if not yet done for the set",
    "ok" : 0,
    "errmsg" : "no replset config has been received",
    "code" : 94,
    "codeName" : "NotYetInitialized"
}