Kubernetes Pod Object crashing in infinite loop when trying to deploy Dockerized Django application with PostgreSQL DB

5/23/2018

I am working on my Django project and I am trying to deploy it on Kubernetes cluster ( Google Cloud Provider ). I've managed to create all the files I need for configuring the cluster:

  • Django App Deployment + Service .yml files
  • Postgres DB Deployment + Service + PersistentVolumeClaim ( which dynamicly creates PersistentVolume object and uses PD storage on cloud nodes by default )+ Secret .yml files
  • Docker File + Docker Compose ( I dont have much experience with docker so there might be something wrong with them, maybe that is the issue but I cannot figure it out )

For some reason when i try to apply them my PODs, specifically Django Application PODs, are Crashing in an infinite loop. Also my Postgres Pod is not crashing but it is not running eather.

Can anyone help me figure out what am I doing wrong? Here is my github repo: https://github.com/…/mast…/agents/config/kubernetes/postgres

My best guess is that I did not set DATABASES configuration in settings.py file correct, and that my Django Application cannot find DATABASE HOST specified in settings.py. If anyone has any suggestions please leave a comment.

Here is a picture of my Kubernetes Pod list:

enter image description here Here is a picture where I've described 1st Pod object:

enter image description here And here is a picture of one of my nodes described:

enter image description here

* UPDATE *

Here is what my ERROR message looks like when i try to run python src/manage.py runserver:

(web_development) cepa995@cepa995-VirtualBox:~/Agentske_Tehnologije/agents$ python src/manage.py runserver
/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Performing system checks...

System check identified no issues (0 silenced).
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x7f6b37e9a9d8>
Traceback (most recent call last):
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 168, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "postgres-service" to address: Name or service not known


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/utils/autoreload.py", line 225, in wrapper
    fn(*args, **kwargs)
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 124, in inner_run
    self.check_migrations()
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/core/management/base.py", line 427, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/migrations/loader.py", line 49, in __init__
    self.build_graph()
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/migrations/loader.py", line 206, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 61, in applied_migrations
    if self.has_table():
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 44, in has_table
    return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/base/base.py", line 255, in cursor
    return self._cursor()
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/base/base.py", line 232, in _cursor
    self.ensure_connection()
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 168, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/home/cepa995/anaconda3/envs/web_development/lib/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: could not translate host name "postgres-service" to address: Name or service not known
-- Stefan Radonjic
django
docker
google-kubernetes-engine
kubernetes
postgresql

1 Answer

5/25/2018

Your application cannot connect to the database, because your Postgres server cannot start (you have 0/1 pods in Ready state).

You did not publish its Pod events, but I guess the main reason is its volume.

In your Postgres specs, I see you set the volume type to ReadWriteMany, but Google Cloud does not provide volumes with that type (documentation, path "Access Modes") and, actually, I don't see the reason why you need it. Change the type to ReadWriteOnce and redeploy the database, it should help.

P.S. To get a log of application in a cluster, you can call kubectl logs $podname, it’s much more helpful P.P.S. When you publish text information, use a text format, please.

-- Anton Kostenko
Source: StackOverflow