stable/mongodb-replicaset helm chart authentication error for non admin user

5/5/2019


I am trying to use mongodb-replicaset helm chart as subchart for one of our micro service. we are using mongodb version 4.0 . I customized this helm chart to create new db and non admin user.
to perform this i perform following steps.

A) added following to mongodb-statefulset.yaml

{{- if .Values.initdb.enabled }}
            - name: INITDB
              value: "true"
            - name: INITDB_DATABASE_NAME
              valueFrom:
                configMapKeyRef:
                  name: {{ .Values.initdb.dbConfigName }}
                  key: {{ .Values.initdb.initDbConfigKey }}
            - name: NON_ADMIN_USER
              valueFrom:
                configMapKeyRef:
                  name: {{ .Values.initdb.dbConfigName }}
                  key: {{ .Values.initdb.dbNonAdminUserKey }}
            - name: NON_ADMIN_USER_ROLE
              valueFrom:
                configMapKeyRef:
                  name: {{ .Values.initdb.dbConfigName }}
                  key: {{ .Values.initdb.dbNonAdminRoleKey }}
            - name: NON_ADMIN_USER_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: {{ .Values.initdb.dbSecretName }}
                  key: {{ .Values.initdb.dbNonAdminPasswordKey }}
          {{- end }}  

B) created non admin user with the help of on-start.sh present in init folder of this chart

# init db and non admin user
if [[ -n "${primary}" && "$AUTH" == "true" && "$INITDB" == "true" ]]; then
    log "Creating DB ${dbName} and non admin user"
    log "checking if non admin user present by count"
    non_admin_user_count=$(mongo admin --host "${primary}" "${admin_creds[@]}" "${ssl_args[@]}" --eval "db.system.users.find({user: '${non_admin_user}'}).count()" --quiet)
    log "non_admin_user_count is ${non_admin_user_count} "
    if [[ "${non_admin_user_count}" == "0" ]]; then
        log "since no non admin user present creating new "
        mongo admin --host "${primary}" "${admin_creds[@]}" "${ssl_args[@]}" --eval "newDb=db.getSiblingDB('${dbName}'); newDb.createUser({user: '${non_admin_user}', pwd: '${non_admin_password}', roles: [{role: '${non_admin_role}', db: '${dbName}'}]})"
        log "Done with  DB ${dbName} and non admin user"
    fi
fi

C) added these extra values in values.yaml. please note mongo-replicaset acts as subchart

initdb:
  enabled: false
  dbConfigName: "-"
  initDbConfigKey: "-"
  dbNonAdminUserKey: "-"
  dbNonAdminRoleKey: "-"
  dbSecretName: "-"
  dbNonAdminPasswordKey: "-"

D)override these values in values.yaml of main chart

   mongodb-replicaset:
    # total number of replica for mongodb
      #change it based on env. for dxx,dev use replicas 1 for other use 3
      replicas: 3
      image:
        tag: 4
      persistentVolume:
        #possible values default, managed-premium for AKS
        #change it based on env. for dxx,dev use default for other use managed-premium
        #storageClass: "managed-premium"
        storageClass: "managed-premium"
        # change it based on env. for dxx,dev use based on 
        # application skeleton use 30 Gi, for others use 5 GB
        size: 30
      auth:
        # it will enable authentication for mongodb   
        enabled: true
        #use credentials for authentication from secret named skeleton
        #it will check for keys user,password(admin user name and password) and key.txt (kind of token) for internal communication. 
        # more info can be available on https://github.com/helm/charts/tree/master/stable/mongodb-replicaset
        existingKeySecret: skeleton
        existingAdminSecret: skeleton
      initdb:
        # flag to create db or not
        enabled: true
        #configmap name to pick db name and non root user name and its role.
        dbConfigName: skeleton
        # config map key represents db name
        initDbConfigKey: mongo_db
        # key indicates non root user name
        dbNonAdminUserKey: mongo_username
        # key indicates non root user role
        dbNonAdminRoleKey: mongo_userrole
        #secret used to get non admin user password
        dbSecretName: skeleton
        # key present in secret for creating non admin user
        dbNonAdminPasswordKey: mongodb_user-password

config map for this

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: skeleton
    data:
      mongo_port: "27017"
      #redundent using mongo url instead of this
      #mongo_host: skeleton-mongodb-replicaset-client
      mongo_db: skeleton
      mongo_username: skeleton
      mongo_root-username: madmin
      mongo_userrole: readWrite

secret

    apiVersion: v1
    kind: Secret
    metadata:
      name: skeleton
    type: Opaque
    data:
      ######################################
      #---for-mongo-db-ha
      # mongo db admin user
      user: <64-bit-encode-value>
      #mongo db admin password: 
      password: <64-bit-encode-value>
      # Key for internal authentication    
      key.txt: <64-bit-encode-value>
      # password for skeleton user 
      mongodb_user-password: <64-bit-value>

observations while checking logs for bootstrap container of mongodb-replicaset i can see it is creating database and user

[2019-05-05T05:31:24,217264945+00:00] [on-start.sh] ✓ Replica reached PRIMARY state.
[2019-05-05T05:31:24,219243724+00:00] [on-start.sh] Creating admin user...
MongoDB shell version v4.0.9
connecting to: mongodb://127.0.0.1:27017/admin?gssapiServiceName=mongodb
2019-05-05T05:31:24.276+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:44422 #6 (1 connection now open)
2019-05-05T05:31:24.277+0000 I NETWORK  [conn6] received client metadata from 127.0.0.1:44422 conn6: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "4.0.9" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "16.04" } }
Implicit session: session { "id" : UUID("9df70da8-5237-408d-963e-aafee4cf1b75") }
MongoDB server version: 4.0.9
2019-05-05T05:31:24.318+0000 I STORAGE  [conn6] createCollection: admin.system.users with generated UUID: 4b27c904-4d02-4605-a85a-22fddd1bbfad
2019-05-05T05:31:24.393+0000 I COMMAND  [conn6] command admin.system.version appName: "MongoDB Shell" command: createUser { createUser: "madmin", pwd: "xxx", roles: [ { role: "root", db: "admin" } ], digestPassword: true, writeConcern: { w: "majority", wtimeout: 600000.0 }, lsid: { id: UUID("9df70da8-5237-408d-963e-aafee4cf1b75") }, $clusterTime: { clusterTime: Timestamp(1557034282, 5), signature: { hash: BinData(0, 2AA442F705DDC61830428E585F5E848633943718), keyId: 6687411319940841474 } }, $db: "admin" } numYields:0 reslen:163 locks:{ Global: { acquireCount: { r: 7, w: 4 } }, Database: { acquireCount: { r: 3, W: 4 } }, Collection: { acquireCount: { r: 3, w: 3 } } } storage:{} protocol:op_msg 114ms
Successfully added user: {
    "user" : "madmin",
    "roles" : [
        {
            "role" : "root",
            "db" : "admin"
        }
    ]
}
2019-05-05T05:31:24.398+0000 I NETWORK  [conn6] end connection 127.0.0.1:44422 (0 connections now open)
[2019-05-05T05:31:24,399482206+00:00] [on-start.sh] Creating DB skeleton and non admin user
[2019-05-05T05:31:24,401917503+00:00] [on-start.sh] checking if non admin user present by count
2019-05-05T05:31:24.476+0000 I NETWORK  [listener] connection accepted from 10.104.0.28:43544 #7 (1 connection now open)
2019-05-05T05:31:24.477+0000 I NETWORK  [conn7] received client metadata from 10.104.0.28:43544 conn7: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "4.0.9" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "16.04" } }
2019-05-05T05:31:24.506+0000 I ACCESS   [conn7] Successfully authenticated as principal madmin on admin
2019-05-05T05:31:24.509+0000 I NETWORK  [conn7] end connection 10.104.0.28:43544 (0 connections now open)
[2019-05-05T05:31:24,512715318+00:00] [on-start.sh] non_admin_user_count is 0 
[2019-05-05T05:31:24,514729998+00:00] [on-start.sh] since no non admin user present creating new 
MongoDB shell version v4.0.9
connecting to: mongodb://skeleton-mongodb-replicaset-0.skeleton-mongodb-replicaset.default.svc.cluster.local:27017/admin?gssapiServiceName=mongodb
2019-05-05T05:31:24.578+0000 I NETWORK  [listener] connection accepted from 10.104.0.28:43546 #8 (1 connection now open)
2019-05-05T05:31:24.579+0000 I NETWORK  [conn8] received client metadata from 10.104.0.28:43546 conn8: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "4.0.9" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "16.04" } }
2019-05-05T05:31:24.607+0000 I ACCESS   [conn8] Successfully authenticated as principal madmin on admin
Implicit session: session { "id" : UUID("8f1ff9f1-23bb-497e-97ef-33ef52846726") }
MongoDB server version: 4.0.9
Successfully added user: {
    "user" : "skeleton",
    "roles" : [
        {
            "role" : "readWrite",
            "db" : "skeleton"
        }
    ]
}
2019-05-05T05:31:24.653+0000 I NETWORK  [conn8] end connection 10.104.0.28:43546 (0 connections now open)
[2019-05-05T05:31:24,654571671+00:00] [on-start.sh] Done with  DB skeleton and non admin user
[2019-05-05T05:31:24,656743957+00:00] [on-start.sh] MongoDB bootstrap complete

but when we try to connect this db either using seedlist or client service url we are getting authentication error

[1;30m2019-05-05 06:04:16,968[0;39m => [34mINFO [0;39m [[32mmain[0;39m] [33morg.springframework.web.reactive.result.method.annotation.ControllerMethodResolver[0;39m: Looking for @ControllerAdvice: org.springframework.boot.web.reactive.context.AnnotationConfigReactiveWebServerApplicationContext@d7b1517: startup date [Sun May 05 06:04:03 GMT 2019]; root of context hierarchy
[1;30m2019-05-05 06:04:21,722[0;39m => [34mINFO [0;39m [[32mmain[0;39m] [33morg.springframework.cache.ehcache.EhCacheManagerFactoryBean[0;39m: Initializing EhCache CacheManager
[1;30m2019-05-05 06:04:21,881[0;39m => [34mINFO [0;39m [[32mmain[0;39m] [33mde.deutschefintechsolutions.mongobee.MongoBeeConfig[0;39m: MongoBee connection: mongodb://skeleton:complexpassword@skeleton-mongodb-replicaset-client:27017/skeleton?authSource=admin&replicaSet=rs0
[1;30m2019-05-05 06:04:21,899[0;39m => [34mINFO [0;39m [[32mmain[0;39m] [33mcom.mongodb.diagnostics.logging.SLF4JLogger[0;39m: Cluster created with settings {hosts=[skeleton-mongodb-replicaset-client:27017], mode=MULTIPLE, requiredClusterType=REPLICA_SET, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500, requiredReplicaSetName='rs0'}
[1;30m2019-05-05 06:04:21,899[0;39m => [34mINFO [0;39m [[32mmain[0;39m] [33mcom.mongodb.diagnostics.logging.SLF4JLogger[0;39m: Adding discovered server skeleton-mongodb-replicaset-client:27017 to client view of cluster
[1;30m2019-05-05 06:04:21,983[0;39m => [34mINFO [0;39m [[32mmain[0;39m] [33mcom.mongodb.diagnostics.logging.SLF4JLogger[0;39m: No server chosen by com.mongodb.Mongo$4@31133b6e from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=skeleton-mongodb-replicaset-client:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
[1;30m2019-05-05 06:04:23,260[0;39m => [34mINFO [0;39m [[32mcluster-ClusterId{value='5cce7ce57ad067000150d58b', description='null'}-skeleton-mongodb-replicaset-client:27017[0;39m] [33mcom.mongodb.diagnostics.logging.SLF4JLogger[0;39m: Exception in monitor thread while connecting to server skeleton-mongodb-replicaset-client:27017
com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=null, userName='skeleton', source='admin', password=<hidden>, mechanismProperties={}}
    at com.mongodb.connection.SaslAuthenticator.wrapException(SaslAuthenticator.java:162)
    at com.mongodb.connection.SaslAuthenticator.access$200(SaslAuthenticator.java:39)
    at com.mongodb.connection.SaslAuthenticator$1.run(SaslAuthenticator.java:68)
    at com.mongodb.connection.SaslAuthenticator$1.run(SaslAuthenticator.java:46)
    at com.mongodb.connection.SaslAuthenticator.doAsSubject(SaslAuthenticator.java:168)
    at com.mongodb.connection.SaslAuthenticator.authenticate(SaslAuthenticator.java:46)
    at com.mongodb.connection.DefaultAuthenticator.authenticate(DefaultAuthenticator.java:32)
    at com.mongodb.connection.InternalStreamConnectionInitializer.authenticateAll(InternalStreamConnectionInitializer.java:122)
    at com.mongodb.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:52)
    at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:127)
    at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:114)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.mongodb.MongoCommandException: Command failed with error 18: 'Authentication failed.' on server skeleton-mongodb-replicaset-client:27017. The full response is { "operationTime" : { "$timestamp" : { "t" : 1557036263, "i" : 1 } }, "ok" : 0.0, "errmsg" : "Authentication failed.", "code" : 18, "codeName" : "AuthenticationFailed", "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1557036263, "i" : 1 } }, "signature" : { "hash" : { "$binary" : "TGrXuvE8ZTBFxjvF1PU5dK0e2cA=", "$type" : "00" }, "keyId" : { "$numberLong" : "6687411319940841474" } } } }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:164)

url used for this mongodb://skeleton:complexpassword@skeleton-mongodb-replicaset-client:27017/skeleton?authSource=admin&replicaSet=rs0

-- Ganesh Pol
azure-aks
kubernetes
mongodb
mongodb-replica-set

1 Answer

5/12/2019

I observed that mongodb-replicaset not tested with mongodb version 4 again it does not provide support for non root user.

Another chart from helm stable repository provide support for mongodb version 4 and non root users due to these reasons I prefer to use mongodb helm chart instead of mongodb-replicaset

-- Ganesh Pol
Source: StackOverflow