postgres with citus extension on openshift

10/15/2019

I want to set up a Postgres cluster with Citus Extention on Openshift platform. Postgres works with Postgres user with having user id 26 but for my project, Openshift allows only some fixed range of user id's so whenever I try to deploy my image on Openshift, Postgres user id is overriding with the default id. I am sharing my Docker file and error log here.

FROM centos/s2i-core-centos7

# PostgreSQL image for OpenShift.
# Volumes:
#  * /var/lib/psql/data   - Database cluster for PostgreSQL
# Environment:
#  * $POSTGRESQL_USER     - Database user name
#  * $POSTGRESQL_PASSWORD - User's password
#  * $POSTGRESQL_DATABASE - Name of the database to create
#  * $POSTGRESQL_ADMIN_PASSWORD (Optional) - Password for the 'postgres'
#                           PostgreSQL administrative account

ENV POSTGRESQL_VERSION=10 \
    POSTGRESQL_PREV_VERSION=9.6 \
    HOME=/var/lib/pgsql \
    PGUSER=postgres \
    APP_DATA=/opt/app-root

ENV SUMMARY="PostgreSQL is an advanced Object-Relational database management system" \
    DESCRIPTION="PostgreSQL is an advanced Object-Relational database management system (DBMS). \
The image contains the client and server programs that you'll need to \
create, run, maintain and access a PostgreSQL DBMS server."

LABEL summary="$SUMMARY" \
      description="$DESCRIPTION" \
      io.k8s.description="$DESCRIPTION" \
      io.k8s.display-name="PostgreSQL 10" \
      io.openshift.expose-services="5432:postgresql" \
      io.openshift.tags="database,postgresql,postgresql10,rh-postgresql10" \
      io.openshift.s2i.assemble-user="26" \
      name="centos/postgresql-10-centos7" \
      com.redhat.component="rh-postgresql10-container" \
      version="1" \
      usage="podman run -d --name postgresql_database -e POSTGRESQL_USER=user -e POSTGRESQL_PASSWORD=pass -e POSTGRESQL_DATABASE=db -p 5432:5432 centos/postgresql-10-centos7" \
      maintainer="SoftwareCollections.org <sclorg@redhat.com>"

EXPOSE 9700
EXPOSE 9701
EXPOSE 9702

COPY root/usr/libexec/fix-permissions /usr/libexec/fix-permissions

# This image must forever use UID 26 for postgres user so our volumes are
# safe in the future. This should *never* change, the last test is there
# to make sure of that.
RUN curl https://install.citusdata.com/community/rpm.sh | bash && \
    yum install -y citus81_11
RUN yum install -y centos-release-scl-rh && \
    yum-config-manager --add-repo https://cbs.centos.org/repos/sclo7-rh-postgresql10-rh-candidate/x86_64/os/ && \
    echo gpgcheck=0 >> /etc/yum.repos.d/cbs.centos.org_repos_sclo7-rh-postgresql10-rh-candidate_x86_64_os_.repo && \
    INSTALL_PKGS="rsync tar gettext bind-utils nss_wrapper" && \
    yum -y --setopt=tsflags=nodocs install $INSTALL_PKGS && \
    rpm -V $INSTALL_PKGS && \
    yum -y clean all --enablerepo='*' && \
    localedef -f UTF-8 -i en_US en_US.UTF-8 && \
    test "$(id postgres)" = "uid=26(postgres) gid=26(postgres) groups=26(postgres)" && \
    mkdir -p /var/lib/pgsql/data && \
    /usr/libexec/fix-permissions /var/lib/pgsql /var/run/postgresql

# Get prefix path and path to scripts rather than hard-code them in scripts
ENV CONTAINER_SCRIPTS_PATH=/usr/share/container-scripts/postgresql \
    ENABLED_COLLECTIONS=rh-postgresql10

COPY root /
COPY ./s2i/bin/ $STI_SCRIPTS_PATH

# When bash is started non-interactively, to run a shell script, for example it
# looks for this variable and source the content of this file. This will enable
# the SCL for all scripts without need to do 'scl enable'.
ENV BASH_ENV=${CONTAINER_SCRIPTS_PATH}/scl_enable \
    ENV=${CONTAINER_SCRIPTS_PATH}/scl_enable \
    PROMPT_COMMAND=". ${CONTAINER_SCRIPTS_PATH}/scl_enable"

# Not using VOLUME statement since it's not working in OpenShift Online:
# https://github.com/sclorg/httpd-container/issues/30
# VOLUME ["/var/lib/pgsql/data"]

# S2I permission fixes
# --------------------
# 1. unless specified otherwise (or - equivalently - we are in OpenShift), s2i
#    build process would be executed as 'uid=26(postgres) gid=26(postgres)'.
#    Such process wouldn't be able to execute the default 'assemble' script
#    correctly (it transitively executes 'fix-permissions' script).  So let's
#    add the 'postgres' user into 'root' group here
#
# 2. we call fix-permissions on $APP_DATA here directly (UID=0 during build
#    anyways) to assure that s2i process is actually able to _read_ the
#    user-specified scripting.
RUN usermod -a -G root postgres && \
    /usr/libexec/fix-permissions --read-only "$APP_DATA"

USER 26

ENTRYPOINT ["container-entrypoint"]
CMD ["run-citus"]

I just modified the postgresql-10 dockerfile which is used on openshift public repository for postgres image.

file: container-entrypoint

#!/bin/bash

exec "$@"

file: run-citus

#!/bin/bash

export PATH=$PATH:/usr/pgsql-11/bin
export ENABLE_REPLICATION=${ENABLE_REPLICATION:-false}

set -eu
export_vars=$(cgroup-limits) ; export $export_vars

cd ~
mkdir -p citus/coordinator citus/worker1 citus/worker2

initdb -D citus/coordinator
initdb -D citus/worker1
initdb -D citus/worker2

echo "shared_preload_libraries = 'citus'" >> citus/coordinator/postgresql.conf
echo "shared_preload_libraries = 'citus'" >> citus/worker1/postgresql.conf
echo "shared_preload_libraries = 'citus'" >> citus/worker2/postgresql.conf


pg_ctl -D citus/coordinator -o "-p 9700" -l coordinator_logfile start
pg_ctl -D citus/worker1 -o "-p 9701" -l worker1_logfile start
pg_ctl -D citus/worker2 -o "-p 9702" -l worker2_logfile start



psql -p 9700 -c "CREATE EXTENSION citus;"
psql -p 9701 -c "CREATE EXTENSION citus;"
psql -p 9702 -c "CREATE EXTENSION citus;"



psql -p 9700 -c "SELECT * from master_add_node('localhost', 9701);"
psql -p 9700 -c "SELECT * from master_add_node('localhost', 9702);"


psql -p 9700 -c "select * from master_get_active_worker_nodes();"

echo "Starting server..."
exec postgres "$@"

my error log in openshift

Can't read /etc/scl/conf/rh-postgresql10, rh-postgresql10 is probably not installed.
initdb: could not look up effective user ID 1000400000: user does not exist
initdb: could not look up effective user ID 1000400000: user does not exist
initdb: could not look up effective user ID 1000400000: user does not exist
2019-10-14 08:13:59.744 GMT [19] LOG:  skipping missing configuration file "/var/lib/pgsql/citus/coordinator/postgresql.auto.conf"
pg_ctl: directory "/var/lib/pgsql/citus/coordinator" is not a database cluster directory
2019-10-14 08:13:59.773 GMT [22] LOG:  skipping missing configuration file "/var/lib/pgsql/citus/worker1/postgresql.auto.conf"
pg_ctl: directory "/var/lib/pgsql/citus/worker1" is not a database cluster directory
2019-10-14 08:13:59.793 GMT [25] LOG:  skipping missing configuration file "/var/lib/pgsql/citus/worker2/postgresql.auto.conf"
pg_ctl: directory "/var/lib/pgsql/citus/worker2" is not a database cluster directory
psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.9700"?
psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.9701"?
psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.9702"?
psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.9700"?
psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.9700"?
psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.9700"?
Starting server...
postgres does not know where to find the server configuration file.
You must specify the --config-file or -D invocation option or set the PGDATA environment variable.
-- Ravi Theja
docker
kubernetes
openshift

0 Answers