K
Q

ArangoDB init container fails on minikube

12/9/2019

I'm working on a NodeJS service which uses ArangoDB as datastore, and deployed on minikube. I use an initContainer directive in the kubernetes deployment manifest to ensure that the database is ready to receive connections before the application attempts to connect. The relevant portion of the kubernetes YAML is shown below:

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: carservice
  template:
    spec:
      initContainers:
        - name: init-carservice
          image: arangodb/arangodb:3.5.1
          command: ['sh', 'c', 'arangosh --server.endpoint="https://${CARSERVICE_CARSERVICEDB_SERVICE_HOST}:${CARSERVICE_CARSERVICEDB_SERVICE_PORT}" --server.password=""; do echo waiting for database to be up; sleep 2; done;']
      containers:
        - name: carservice
          image: carservice
          imagePullPolicy: IfNotPresent

The challenge has been that sometimes the initContainer is able to wait for the database connection to be established successfully. Most of the other times, it randomly fails with the error:

ERROR caught exception: invalid endpoint spec: https://

Out of desperation, I changed the scheme to http, and it fails with a corresponding error:

ERROR caught exception: invalid endpoint spec: http://

My understanding of these errors is that the database is not able to recognize https and http in these instances, which is strange. The few times the initContainer bit worked successfully, I used https in the related command in the kubernetes spec.

I must add that the actual database (https://${CARSERVICE_CARSERVICEDB_SERVICE_HOST}:${CARSERVICE_CARSERVICEDB_SERVICE_PORT}) has been successfully deployed to minikube using kube-arangodb, and can be accessed through the web UI, so that bit is sorted.

What I'd like to know:

  1. Is this the recommended way to wait for ArangoDB to connect using the initContainer directive, or do I have to use an entirely different approach?

  2. What could be causing the error I'm getting? Am I missing something fundamental here?

Would be glad for any help.

-- ceblay
arangodb
kubernetes
minikube

Similar Questions

Can we store the TLS certificates in Azure Key vault as a secret and use the same in Ingress in Azure Kubernetes service
why not create multiple containers instead of using kubernetes
Providing DNS for Kubernetes services with bare metal load balancer
Service deployed on EKS does not answer on its EXTERNAL-IP
Restart pod when spring vault renew token does not work
Use Application Load Balancers | Helm chart | AWS
How to enable Alert manager pick alert manager configs from a new kubernetes namespace?

1 Answer

12/10/2019

The issue was that for those times the init container failed to connect to ArangoDB, the env variables were not correctly set. Therefore, I added another init container before that (since init containers are executed in sequence), that'd wait for the corresponding kubernetes "service" resource of the ArangoDB deployment to come up. That way, by the time the second init container would run, the env variables would be available.

The corresponding portion of kubernetes deployment YAML is shown as:

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: carservice
  template:
    spec:
      initContainers:
        - name:init-db-service
          image: busybox:1.28
          command: ['sh', '-c', 'until nslookup carservice-carservicedb; do echo waiting for kubernetes service resource for db; sleep 2; done;']

        - name: init-carservice
          image: arangodb/arangodb:3.5.1
          command: ['sh', 'c', 'arangosh --server.endpoint="https://${CARSERVICE_CARSERVICEDB_SERVICE_HOST}:${CARSERVICE_CARSERVICEDB_SERVICE_PORT}" --server.password=""; do echo waiting for database to be up; sleep 2; done;']
      containers:
        - name: carservice
          image: carservice
          imagePullPolicy: IfNotPresent
-- ceblay
Source: StackOverflow