K
Q

Statefulset for mongodb with replica set

11/19/2019

I want to create a Statefulset from a mongo image using a replica set (mongo).

For this, I have my statefulset:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongo-test
spec:
  selector:
    matchLabels:
      app: mongo-test
  serviceName: "mongo-test"
  replicas: 1
  template:
    metadata:
      labels:
        app: mongo-test
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: mongodb
        image: mongo:4.2.1
        ports:
        - containerPort: 27017
          name: web
        command: ["/bin/sh","-c"]
        args: ["mongod --oplogSize 128 --replSet rs0 --bind_ip_all && mongo --eval \"rs.initiate()\""]
        volumeMounts:
        - name: mongo-persistent-storage
          mountPath: /data/db
  volumeClaimTemplates:
  - metadata:
      name: mongo-persistent-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 2Gi

Statefulset and pod created are ok, but the replica set is not initialized. I'm sending the command: mongo --eval "rs.initiate()" in order to create the replica but for some reason the replica is not created.

I also used the command: sleep 30 before the rs.initiate() command just like a delay but no luck.

Is there something wrong in my commands?

Pd. I'm not detailing the rest of configuration (services, pvc, pv) because everything is ok.

-- UPDATE

Adding logs:

2019-11-19T15:10:05.257+0000 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten] MongoDB starting : pid=8 port=27017 dbpath=/data/db 64-bit host=mongo-test-0
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten] db version v4.2.1
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten] git version: edf6d45851c0b9ee15548f0f847df141764a317e
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1  11 Sep 2018
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten] allocator: tcmalloc
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten] modules: none
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten] build environment:
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten]     distmod: ubuntu1804
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten]     distarch: x86_64
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten]     target_arch: x86_64
2019-11-19T15:10:05.259+0000 I  CONTROL  [initandlisten] options: { net: { bindIp: "*" }, replication: { oplogSizeMB: 128, replSet: "rs0" } }
2019-11-19T15:10:05.259+0000 I  STORAGE  [initandlisten] 
2019-11-19T15:10:05.259+0000 I  STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-11-19T15:10:05.259+0000 I  STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-11-19T15:10:05.259+0000 I  STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=5442M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
2019-11-19T15:10:06.307+0000 I  STORAGE  [initandlisten] WiredTiger message [1574176206:307928][8:0x7fa43d4aeb00], txn-recover: Set global recovery timestamp: (0,0)
2019-11-19T15:10:06.721+0000 I  RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0)
2019-11-19T15:10:07.142+0000 I  STORAGE  [initandlisten] Timestamp monitor starting
2019-11-19T15:10:07.272+0000 I  CONTROL  [initandlisten] 
2019-11-19T15:10:07.272+0000 I  CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-11-19T15:10:07.272+0000 I  CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2019-11-19T15:10:07.272+0000 I  CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2019-11-19T15:10:07.272+0000 I  CONTROL  [initandlisten] 
2019-11-19T15:10:07.273+0000 I  SHARDING [initandlisten] Marking collection local.system.replset as collection version: <unsharded>
2019-11-19T15:10:07.273+0000 I  STORAGE  [initandlisten] Flow Control is enabled on this deployment.
2019-11-19T15:10:07.273+0000 I  SHARDING [initandlisten] Marking collection admin.system.roles as collection version: <unsharded>
2019-11-19T15:10:07.273+0000 I  SHARDING [initandlisten] Marking collection admin.system.version as collection version: <unsharded>
2019-11-19T15:10:07.273+0000 I  STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: acb2882f-a531-456a-8150-9d4eb0273ef1 and options: { capped: true, size: 10485760 }
2019-11-19T15:10:07.532+0000 I  INDEX    [initandlisten] index build: done building index _id_ on ns local.startup_log
2019-11-19T15:10:07.532+0000 I  SHARDING [initandlisten] Marking collection local.startup_log as collection version: <unsharded>
2019-11-19T15:10:07.532+0000 I  FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
2019-11-19T15:10:07.533+0000 I  STORAGE  [initandlisten] createCollection: local.replset.oplogTruncateAfterPoint with generated UUID: 6c30e1dc-7da5-4698-a96e-d6736e450999 and options: {}
2019-11-19T15:10:07.773+0000 I  INDEX    [initandlisten] index build: done building index _id_ on ns local.replset.oplogTruncateAfterPoint
2019-11-19T15:10:07.774+0000 I  STORAGE  [initandlisten] createCollection: local.replset.minvalid with generated UUID: 703fef2a-479a-4ae9-9052-6c04cf942af4 and options: {}
2019-11-19T15:10:07.997+0000 I  INDEX    [initandlisten] index build: done building index _id_ on ns local.replset.minvalid
2019-11-19T15:10:07.997+0000 I  SHARDING [initandlisten] Marking collection local.replset.minvalid as collection version: <unsharded>
2019-11-19T15:10:07.998+0000 I  STORAGE  [initandlisten] createCollection: local.replset.election with generated UUID: 950b0919-e330-4983-a1ff-537fb0e28d72 and options: {}
2019-11-19T15:10:08.153+0000 I  INDEX    [initandlisten] index build: done building index _id_ on ns local.replset.election
2019-11-19T15:10:08.153+0000 I  SHARDING [ftdc] Marking collection local.oplog.rs as collection version: <unsharded>
2019-11-19T15:10:08.153+0000 W  REPL     [ftdc] Rollback ID is not initialized yet.
2019-11-19T15:10:08.154+0000 I  SHARDING [initandlisten] Marking collection local.replset.election as collection version: <unsharded>
2019-11-19T15:10:08.155+0000 I  REPL     [initandlisten] Did not find local initialized voted for document at startup.
2019-11-19T15:10:08.155+0000 I  REPL     [initandlisten] Did not find local Rollback ID document at startup. Creating one.
2019-11-19T15:10:08.156+0000 I  STORAGE  [initandlisten] createCollection: local.system.rollback.id with generated UUID: 9e2939db-087f-40a9-9710-585b4b26c57b and options: {}
2019-11-19T15:10:08.330+0000 I  INDEX    [initandlisten] index build: done building index _id_ on ns local.system.rollback.id
2019-11-19T15:10:08.330+0000 I  SHARDING [initandlisten] Marking collection local.system.rollback.id as collection version: <unsharded>
2019-11-19T15:10:08.331+0000 I  REPL     [initandlisten] Initialized the rollback ID to 1
2019-11-19T15:10:08.331+0000 I  REPL     [initandlisten] Did not find local replica set configuration document at startup;  NoMatchingDocument: Did not find replica set configuration document in local.system.replset
2019-11-19T15:10:08.332+0000 I  CONTROL  [LogicalSessionCacheRefresh] Sessions collection is not set up; waiting until next sessions refresh interval: Replication has not yet been configured
2019-11-19T15:10:08.333+0000 I  NETWORK  [initandlisten] Listening on /tmp/mongodb-27017.sock
2019-11-19T15:10:08.333+0000 I  NETWORK  [initandlisten] Listening on 0.0.0.0
2019-11-19T15:10:08.333+0000 I  NETWORK  [initandlisten] waiting for connections on port 27017
2019-11-19T15:10:08.339+0000 I  SHARDING [LogicalSessionCacheReap] Marking collection config.system.sessions as collection version: <unsharded>
2019-11-19T15:10:08.339+0000 I  CONTROL  [LogicalSessionCacheReap] Sessions collection is not set up; waiting until next sessions reap interval: config.system.sessions does not exist
-- georgeos
kubernetes
minikube
mongodb
replicaset

Similar Questions

Is there a way to use Helm 3's new library charts without needing to involve `helm dependency update`?
Kubernetes Ingress Whitelist IP for path
Profiling Kubernetes Deployment Process
What's the most elegant/right way to stop a spark job running on a Kubernetes cluster?
what is the meaning of &quot;pilot_duplicate_envoy_clusters&quot; in istio pilot logs
Can I install minikube on ubuntu without virtualBox?

2 Answers

11/19/2019

You can try this

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongo
  app: mongodb
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongo
          image: mongo
          command:
            - mongod
            - "--replSet"
            - rs0
            - "--smallfiles"
            - "--noprealloc"
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-persistent-volume
              mountPath: /data/db
  volumeClaimTemplates:
  - metadata:
      name: mongo-persistent-volume
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5Gi
-- Harsh Manvar
Source: StackOverflow
11/24/2019

I am not expert in this matter but, as per your logs your options were passed during startup:

options: { net: { bindIp: "*" }, replication: { oplogSizeMB: 128, replSet: "rs0" } }

But mongodb cant find repl settings during the startup:

Did not find local replica set configuration document at startup;
  NoMatchingDocument: Did not find replica set configuration document in local.system.replset

The second issue with additional parameters:

Starting in version 4.2, MongoDB removes the deprecated MMAPv1 storage engine and the MMAPv1-specific configuration options:

Removed Command-line Option:

  • mongod --smallfiles
  • mongod --noprealloc

What I did: I have applied below args:

args:
  - mongod
  - "--replSet"
  - rs0
  - "--bind_ip_all"

For hostnames, stable network identifiers please refer to the statefulset basics:

For a StatefulSet with N replicas, each Pod in the StatefulSet will be assigned an integer ordinal, from 0 up through N-1, that is unique over the Set A StatefulSet can use a Headless Service to control the domain of its Pods. The domain managed by this Service takes the form: $(service name).$(namespace).svc.cluster.local, where “cluster.local” is the cluster domain. As each Pod is created, it gets a matching DNS subdomain, taking the form: $(podname).$(governing service domain), where the governing service is defined by the serviceName field on the StatefulSet.

As mentioned in the limitations section, you are responsible for creating the Headless Service responsible for the network identity of the pods.

According to those rules in my example I have also created headless service and provided those information during mongodb initialization:

host: "mongo-test-0.mongo-test"
host: "mongo-test-1.mongo-test"

From the running pod I have initialized the cluster with this command:

rs.initiate( {_id: "rs0", version:1, members: [
      { _id: 0, host: "mongo-test-0.mongo-test:27017" },
      { _id: 1, host: "mongo-test-1.mongo-test:27017" },
   ]
})

{
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1574614871, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1574614871, 1)
}

Verification:

rs0:PRIMARY> rs.config()
{
    "_id" : "rs0",
    "version" : 1,
    "protocolVersion" : NumberLong(1),
    "writeConcernMajorityJournalDefault" : true,
    "members" : [
        {
            "_id" : 0,
            "host" : "mongo-test-0.mongo-test:27017",
            "arbiterOnly" : false,
            "buildIndexes" : true,
            "hidden" : false,
            "priority" : 1,
            "tags" : {

            },
            "slaveDelay" : NumberLong(0),
            "votes" : 1
        },
        {
            "_id" : 1,
            "host" : "mongo-test-1.mongo-test:27017",
            "arbiterOnly" : false,
            "buildIndexes" : true,
            "hidden" : false,
            "priority" : 1,
            "tags" : {

            },

rs0:PRIMARY> rs.isMaster()
{
    "hosts" : [
        "mongo-test-0.mongo-test:27017",
        "mongo-test-1.mongo-test:27017"
    ],
    "setName" : "rs0",
    "setVersion" : 1,
    "ismaster" : true,

For more information about mongodb Replication plaese refer to the official mongodb docs.

In my opinion you should prepare env variables and additional scripts/entrypoint applied into your deployment in order to manage f.e. proper hostnames, stable network identifiers, replication and initialization.

As an example I would like suggest to use helm charts for mongodb.
Hope this help.

-- Hanx
Source: StackOverflow