Mysql 5.7 image on kubernetes terminates after every 2 to 3 weeks

12/28/2017

I noticed that mysql 5.7 images on google container engine terminates itself after every 2 to 3 weeks of running in my cluster . i configured a small cluster as a test environment . I have 3 nodes with one for database , one for api and the other for my node js front end . This all works well after my configuration i am able to create my database and its accompanying tables , stored procedures and our usual db objects . My back ends all connects to the db and also my front ends are all up and running . Then suddenly after a period i can estimate about 3 weeks my back ends can no longer connect to my databases any more . it just points out that it cant connect to mysql server . I dash to my cmd and check if the mysql pod is running . it actually is running . But i cant connect access my db . I had to redeploy the mysql image luckily because of my persistent volumes could still recover the db files . The second time it occurred it kept saying no root user , i was surprised because i normally do all my db design and all using this user . The third time it just couldn't locate my db any more . I'm also thinking it might be my deployment script i attached it here as well for nay suggestions :

apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: mysql
labels:
app: mysql
spec:
replicas: 1
selector:
matchLabels:
  app: mysql
strategy:
type: Recreate
template:
metadata:
  labels:
    app: mysql
spec:
  containers:
  - image: mysql:5.7
    args:
      - "--ignore-db-dir=lost+found"
    name: mysql
    env:
      - name: MYSQL_ROOT_PASSWORD
        valueFrom:
          secretKeyRef:
            name: mysql
            key: password
    ports:
    - containerPort: 3306
      name: mysql
    volumeMounts:
    - name: mysql-persistent-storage
      mountPath: /var/lib/mysql
  volumes:
  - name: mysql-persistent-storage
    persistentVolumeClaim:
      claimName: mysql-pv-claim

This is what i get in the logs

W1231 11:59:23.713916   14792 cmd.go:392] log is DEPRECATED and will be 
removed in a future version. Use logs instead.
Initializing database
2017-12-31T10:57:23.236067Z 0 [Warning] TIMESTAMP with implicit DEFAULT 
value is deprecated. Please use --explicit_defaults_for_timestamp server 
option (see documentation for more details).
2017-12-31T10:57:23.237652Z 0 [ERROR] --initialize specified but the data 
directory has files in it. Aborting.
2017-12-31T10:57:23.237792Z 0 [ERROR] Aborting
-- I.Tyger
kubernetes

0 Answers