jdbc driver does not work in kubernetes, fails with timeout

6/21/2019

I have a java 11 app with jdbc driver running together with mysql 8.0, the app is able to connect to mysql and execute one sql, but it looks like it never gets a response back?

-- Dmitry
jdbc
kubernetes
mysql

1 Answer

6/28/2019

It looks like a connectivity issue. At first it'd be good to look at the Java program output.

First simple checks are at the Kubernetes level to ensure that key components are alive:

$ kubectl get deployments
$ kubectl get services
$ kubectl get pods

Additional checks could be done from within the container where your Java app is running. A possible approach is below.

List deployments of your app and their labels:

$ kubectl get deployments --show-labels
NAME         READY   UP-TO-DATE   AVAILABLE   AGE   LABELS
hello-node   2/2     2            2           1h   app=hello-node

Having got the label, you can list the relevant pods and their containers:

$ LABEL=hello-node; kubectl get pods -l app=$LABEL -o custom-columns=POD:metadata.name,CONTAINER:spec.containers[*].name
POD                           CONTAINER
hello-node-55b49fb9f8-7tbh4   hello-node
hello-node-55b49fb9f8-p7wt6   hello-node

Now it's possible to run basic diagnostic commands from within the Java app container. Ping might not achieve the target but is available almost always in container and does primitive check of DNS resolution. Services from the same namespace should be available via short DNS name. Services from other namespaces inside of the same Kubernetes cluster should be available via internal FQDN.

$ kubectl exec hello-node-55b49fb9f8-p7wt6 -c hello-node -- ping -c1 hello-node
$ kubectl exec hello-node-55b49fb9f8-p7wt6 -c hello-node -- ping -c1 hello-node.default.svc.cluster.local
$ kubectl exec hello-node-55b49fb9f8-p7wt6 -c hello-node -- mysql -u [username] -p [dbname] -e [query]

From here on the connectivity diagnostics is pretty similar to the bare-metal server except the fact that you are limited by the tools available inside of container. You might install missing packages into the container as needed.

As soon as you obtain more diagnostic information, you'll get a clue what to check next.

-- mebius99
Source: StackOverflow