K
Q

Question

Kubernetes Pod Object which encapsulates PostgreSQL docker image crashes in infinite loop

5/28/2018

I am trying to run my Django + PostgreSQL application on Kubernetes Google Cloud cluster. I've successfully deployed the following files:

Django App Deployment
Django App Service
Kubernetes Secret object for DB credentials
PersistentVolumeClaim

But I am having trouble when deploying my PostgreSQL DB to the cluster. Here is the definition of my .yml file:

apiVersion: v1
kind: Service
metadata:
  name: postgres-service
spec:
  selector:
    app: postgres-container
    tier: backend
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
  type: ClusterIP
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
  labels:
    type: local
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres-container
      tier: backend
  template:
    metadata:
      labels:
        app: postgres-container
        tier: backend
    spec:
      containers:
        - name: postgres-container
          image: postgres:9.6.6
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: postgres-credentials
                  key: user

            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-credentials
                  key: password

            - name: POSTGRES_DB
              value: agent_technologies_db
          ports:
            - containerPort: 5432
          volumeMounts:
            - name: postgres-volume-mount
              mountPath: /var/lib/postgresql/data

      volumes:
        - name: postgres-volume-mount
          persistentVolumeClaim:
            claimName: postgres-pvc
        - name: postgres-credentials
          secret:
            secretName: postgres-credentials

And here is the error I get when I run kubectl logs postgres-85c56dfb9b-95c74 command:

initdb: directory "/var/lib/postgresql/data" exists but is not empty
It contains a lost+found directory, perhaps due to it being a mount point.
Using a mount point directly as the data directory is not recommended.
Create a subdirectory under the mount point.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Could someone please explain this error to me. Thanks in advance!

*****UPDATE******

When I run kubectl logs $pod I am getting the following error (even though container is RUNNING on cluster):

Host: 10.52.1.5
Production - Using "POSTGRESQL" Database
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Host: 10.52.1.5
Production - Using "POSTGRESQL" Database
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Performing system checks...

System check identified no issues (0 silenced).
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x7f62f4c948c8>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 168, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/utils/autoreload.py", line 225, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 124, in inner_run
    self.check_migrations()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 427, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 49, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 206, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 61, in applied_migrations
    if self.has_table():
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 44, in has_table
    return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 255, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 232, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 168, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

This is from my settings.py file:

import socket

print("Host: "+socket.gethostbyname(socket.gethostname()))


if(os.getenv('POSTGRES_DB_HOST')==None):
    print('Development - Using "SQLITE3" Database')
    DATABASES = {
        'default':{
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': os.path.join(BASE_DIR,'db.sqlite3'),
        }
    }
else:
    print('Production - Using "POSTGRESQL" Database')
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.postgresql_psycopg2',
            'NAME': 'agent_technologies_db',
            'USER': 'stefan_radonjic',
            'PASSWORD': 'cepajecar995',
            'HOST': os.getenv('POSTGRES_HOST'),
            'PORT': os.getenv('POSTGRES_PORT'),
            }
    }

Could the error I am getting when i run kubectl logs $pod command be from the fact that the PostgreSQL contianer is not running so it cannot find it or? If anyone is interested in other files of my project here is github link :https://github.com/StefanCepa/agent-technologies-bachelor

-- Stefan Radonjic

django

docker

kubernetes

postgresql

1 Answer

5/28/2018

It does not look like an error to me, more like a warning. PostgreSQL is complaining that you should use a totally empty directory for Postgres data folder. What the application is telling you to do is to create an empty directory inside your volume and the mount that specific subdirectory. That would be done by doing the following:

Create a subdirectory in your volume, you can use for example an initContainer that mounts the volume and creates the directory.
Now that you have a volume that created the directory, then you can modify the postgres-volume-mount in Postgres and add a subPath to your newly created directory.
```
    - name: postgres-volume-mount
      mountPath: /var/lib/postgresql/data
      subPath: NAME_OF_YOUR_SUBDIRECTORY
```

-- Javier Salmeron

Source: StackOverflow

KQ

Kubernetes Pod Object which encapsulates PostgreSQL docker image crashes in infinite loop

Similar Questions

1 Answer

K
Q