My goal is to create a StatefulSet
in the production
namespace and the staging
namespace. I am able to create the production StatefulSet however when deploying one to the staging namespace, I receive the error:
failed to connect to server [127.0.0.1:27017] on first connect [MongoError: connect ECONNREFUSED 127.0.0.1:27017]
The YAML I am using for the staging setup is as so:
staging-service.yml
apiVersion: v1
kind: Service
metadata:
name: mongodb-staging
namespace: staging
labels:
app: ethereumdb
environment: staging
spec:
ports:
- name: http
protocol: TCP
port: 27017
targetPort: 27017
clusterIP: None
selector:
role: mongodb
environment: staging
staging-statefulset.yml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: mongodb-staging
namespace: staging
labels:
app: ethereumdb
environment: staging
annotations:
prometheus.io.scrape: "true"
spec:
serviceName: "mongodb-staging"
replicas: 1
template:
metadata:
labels:
role: mongodb
environment: staging
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: role
operator: In
values:
- mongo
- key: environment
operator: In
values:
- staging
topologyKey: "kubernetes.io/hostname"
terminationGracePeriodSeconds: 10
containers:
- name: mongo
image: mongo
command:
- mongod
- "--replSet"
- rs0
- "--smallfiles"
- "--noprealloc"
- "--bind_ip_all"
- "--wiredTigerCacheSizeGB=0.5"
ports:
- containerPort: 27017
volumeMounts:
- name: mongo-persistent-storage
mountPath: /data/db
- name: mongo-sidecar
image: cvallance/mongo-k8s-sidecar
env:
- name: MONGO_SIDECAR_POD_LABELS
value: "role=mongodb,environment=staging"
- name: KUBERNETES_MONGO_SERVICE_NAME
value: "mongodb-staging"
volumeClaimTemplates:
- metadata:
name: mongo-persistent-storage
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: fast-storage
resources:
requests:
storage: 1Gi
The production
namespace deployment differs only in:
--replSet
value (rs0
instead of rs1
)Everything else remains identical in both deployments.
The only thing I can imagine is that it is not possible to run both these deployments on the port 27017
despite being in separate namespaces.
I am stuck as to what is causing the failed to connect to server
error described above.
Full log of the error
Error in workloop { MongoError: failed to connect to server [127.0.0.1:27017] on first connect [MongoError: connect ECONNREFUSED 127.0.0.1:27017]
at Pool.<anonymous> (/opt/cvallance/mongo-k8s-sidecar/node_modules/mongodb-core/lib/topologies/server.js:336:35)
at Pool.emit (events.js:182:13)
at Connection.<anonymous> (/opt/cvallance/mongo-k8s-sidecar/node_modules/mongodb-core/lib/connection/pool.js:280:12)
at Object.onceWrapper (events.js:273:13)
at Connection.emit (events.js:182:13)
at Socket.<anonymous> (/opt/cvallance/mongo-k8s-sidecar/node_modules/mongodb-core/lib/connection/connection.js:189:49)
at Object.onceWrapper (events.js:273:13)
at Socket.emit (events.js:182:13)
at emitErrorNT (internal/streams/destroy.js:82:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
name: 'MongoError',
message:
'failed to connect to server [127.0.0.1:27017] on first connect [MongoError: connect ECONNREFUSED 127.0.0.1:27017]' }
It seems that the problem is similar to: mongodb-error but still you have two database listen on the same port.
In the context of two mongoDB databases listening on the same port:
The answer differs depending on what OS is being considered. In general though:
But since Linux Kernel 3.9 and later, support for multiple applications listening to the same port was added using the SO_REUSEPORT option. More information is available at this lwn.net article.
But there is workaround.
Run containers on different port and set up Apache or Nginx. As Apache/Nginx works on port 80, you'll not lose any traffic, as 80 is common port.
I recommend Nginx - I find it much easier to set up reverse proxy with Nginx and it's lighter on resource compared to Apache. For nginx, you need to set up it and learn more about Server Blocks: How To Install Nginx on Ubuntu 16.04. How To Set Up Nginx Server Blocks (Virtual Hosts) on Ubuntu 16.04. In Server Blocks, you need to use proxy_pass, for which you can learn more about on nginx site.
It seems like the error you are getting is from the mongo-sidecar container in the pod. As for why the mongo container is failing, can you obtain more detailed information? It could be something like a failed PVC.