I just created a kubernetes cluster in AWS. What I am trying to do is create multiple deployments of Rocket.Chat for each customer (each customer needs their own server). I'm using helm to deploy Rocket.Chat (using the official chart). When I make my first deployment, I'm able to port-forward it to localhost and see the Rocket.Chat server just fine. When I make a second deployment, however, the pods go into a crashloopbackoff state. Even after deleting all the deployments and trying to start from scratch, I still get the same errors. The errors I get are:
/app/bundle/programs/server/node_modules/fibers/future.js:280
throw(ex);
^
MongoParseError: Unescaped slash in userinfo section
at parseConnectionString (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/uri_parser.js:538:21)
at connect (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/mongo_client_ops.js:195:3)
at connectOp (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/mongo_client_ops.js:284:3)
at executeOperation (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:416:24)
at MongoClient.connect (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongo_client.js:175:10)
at Function.MongoClient.connect (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongo_client.js:341:22)
at new MongoConnection (packages/mongo/mongo_driver.js:177:11)
at new MongoInternals.RemoteCollectionDriver (packages/mongo/remote_collection_driver.js:4:16)
at Object.<anonymous> (packages/mongo/remote_collection_driver.js:38:10)
at Object.defaultRemoteCollectionDriver (packages/underscore.js:784:19)
at new Collection (packages/mongo/collection.js:97:40)
at new AccountsCommon (packages/accounts-base/accounts_common.js:23:18)
at new AccountsServer (packages/accounts-base/accounts_server.js:23:5)
at packages/accounts-base/server_main.js:7:12
at server_main.js (packages/accounts-base/server_main.js:19:1)
at fileEvaluate (packages/modules-runtime.js:336:7)
and
/app/bundle/programs/server/node_modules/fibers/future.js:313
throw(ex);
^
MongoNetworkError: failed to connect to server [fc-mongodb:27017] on first connect [MongoError: Authentication failed.]
at Pool.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/server.js:431:11)
at emitOne (events.js:116:13)
at Pool.emit (events.js:211:7)
at connect (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:557:14)
at callback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connect.js:109:5)
at provider.auth.err (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connect.js:352:21)
at _authenticateSingleConnection (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/auth/auth_provider.js:66:11)
at sendAuthCommand (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/auth/scram.js:215:18)
at Connection.messageHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connect.js:334:5)
at emitTwo (events.js:126:13)
at Connection.emit (events.js:214:7)
at processMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connection.js:364:10)
at Socket.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connection.js:533:15)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
I do notice, however, that when I run helm delete fc
that the persistent volumes stay in place and it seems to take forever to delete them (I have always cancelled deleting so I'm not sure if they would ever be able to). I'm guessing that the fact that the persistent volumes are still in place is the reason for why "starting from scratch" doesn't work after I've already deployed. Anyway, has anyone seen these errors before in a similar context? What did you do to fix it?
The issue is with the suggested way to generate passwords on rocket chat's helm chart. They tell you to run:
helm install stable/rocketchat --set mongodb.mongodbPassword=$(echo -n $(openssl rand -base64 32)),mongodb.mongodbRootPassword=$(echo -n $(openssl rand -base64 32))
I was able to fix this error when I switched from using openssl
like the documentation recommends to pwgen
. I'm the way the documentation recommended to create passwords added some illegal characters. This is what I ran instead:
helm install <installation-name> stable/rocketchat --set mongodb.mongodbPassword="$(pwgen 20 1)",mongodb.mongodbRootPassword="$(pwgen 20 1)" -f values.yaml
And it worked!