K
Q

Contacting service in kubernetes

1/27/2022

I thought I understood how kubernetes services work, and have always seen them like a way to "group" several pods, in order to make it able to contact the service instead of the single pods. However, it seems like I am wrong. I created a mysql deployment (with only one pod) and a service in order to reach out to the service if I want to use the mysql connection from other pods(other microservices). This is the service I made:

apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    run: mysql
spec:
  ports:
  - port: 3306
    targetPort: 3306
    protocol: TCP
  selector:
    run: mysql

I hoped this would have allowed me to connect to the mysql pod by reaching the <clusterIp>:<targetPort>, but the connection is refused whenever I try to connect. I tried reading online and initially thought nodeport service type was a good idea, but the kubernetes website tells that the service is than reachable by <NodeIP>:<NodePort> so this got me confused. MySQL should be reachable only inside the cluster by other nodes. How can I make this happen?

NOTES:
  • I am working on minikube.
  • Here is how I try connecting from one pod to the mysql service on python:
service = api.read_namespaced_service(name="mysql", namespace="default")

mydb = mysql.connector.connect(host=service.spec.cluster_ip, user="root",
                                 password="password", database="db_name",
                                 auth_plugin='mysql_native_password')

This is the error I get:

Traceback (most recent call last):
  File "/init/db_init.py", line 10, in <module>
    mydb = mysql.connector.connect(host=service.spec.cluster_ip, user="root",
  File "/usr/local/lib/python3.9/site-packages/mysql/connector/__init__.py", line 272, in connect
    return CMySQLConnection(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/mysql/connector/connection_cext.py", line 85, in __init__
    self.connect(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/mysql/connector/abstracts.py", line 1028, in connect
    self._open_connection()
  File "/usr/local/lib/python3.9/site-packages/mysql/connector/connection_cext.py", line 241, in _open_connection
    raise errors.get_mysql_exception(msg=exc.msg, errno=exc.errno,
mysql.connector.errors.DatabaseError: 2003 (HY000): Can't connect to MySQL server on '10.107.203.112:3306' (111)

UPDATE

As requested, here is the whole log of the mysql pod:

2022-01-27 17:57:14+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.28-1debian10 started.
2022-01-27 17:57:15+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2022-01-27 17:57:15+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.28-1debian10 started.
2022-01-27 17:57:15+00:00 [Note] [Entrypoint]: Initializing database files
2022-01-27T17:57:15.090697Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.28) initializing of server in progress as process 43
2022-01-27T17:57:15.105399Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-01-27T17:57:16.522380Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-01-27T17:57:20.805814Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2022-01-27 17:57:29+00:00 [Note] [Entrypoint]: Database files initialized
2022-01-27 17:57:29+00:00 [Note] [Entrypoint]: Starting temporary server
2022-01-27T17:57:29.868217Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.28) starting as process 92
2022-01-27T17:57:29.892649Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-01-27T17:57:30.100941Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-01-27T17:57:30.398700Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2022-01-27T17:57:30.398743Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2022-01-27T17:57:30.419293Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2022-01-27T17:57:30.430833Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
2022-01-27T17:57:30.430879Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.28'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server - GPL.
2022-01-27 17:57:30+00:00 [Note] [Entrypoint]: Temporary server started.
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
2022-01-27 17:57:32+00:00 [Note] [Entrypoint]: Creating database football

2022-01-27 17:57:32+00:00 [Note] [Entrypoint]: /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/SQL.sql


2022-01-27 17:57:33+00:00 [Note] [Entrypoint]: Stopping temporary server
2022-01-27T17:57:33.143178Z 12 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.28).
2022-01-27T17:57:36.222404Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.28)  MySQL Community Server - GPL.
2022-01-27 17:57:37+00:00 [Note] [Entrypoint]: Temporary server stopped

2022-01-27 17:57:37+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.

2022-01-27T17:57:37.329690Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.28) starting as process 1
2022-01-27T17:57:37.336444Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-01-27T17:57:37.525143Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-01-27T17:57:37.738175Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2022-01-27T17:57:37.738216Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2022-01-27T17:57:37.745722Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2022-01-27T17:57:37.757638Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2022-01-27T17:57:37.757679Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.28'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.

Also, here is the deployment I used:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: repo/football-mysql
        name: mysql
        env:
          # Use secret in real usage
        - name: MYSQL_ROOT_PASSWORD
          value: password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim
-- Placido Pellegriti
kubernetes
kubernetes-pod
mysql
python

Similar Questions

Is having two envoy proxies in the same K8S pod a good practice?
Azure Kubernetes token based authentication to ML webservice
Upgrade helm using ansible playbook in kubernetes namespace
How to implement absolute path forwarding request using ingress-nginx
How to run `args` as `command` in kubernetes

1 Answer

1/27/2022

There is a label mismatch between the Service and the Pod: 

# service
spec:
  selector:
    run: mysql  # <- this

vs

# deployment
spec:
  template:
    metadata:
      labels:
        app: mysql  # <- and this

Because of it, the Service targets some other Pod, not the one created by the Deployment. In other words, the Service looks for Pods with run label value equal to mysql, while your MySQL Pod has app label with mysql in it.

To make it work, at least one pair of labels has to be completely equal on both sides for the Service and the Pod. In this case, replacing run for app should be enough:

apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    run: mysql  # changing here isn't mandatory, up to you
spec:
  ports:
  - port: 3306
    targetPort: 3306
    protocol: TCP
  selector:
    app: mysql  # here is the required change
-- anemyte
Source: StackOverflow