Python(pip) not install ElasticSearch library on kubernetes

9/8/2019

I'm writing a python application to push some results onto elasticsearch.

I've written a Dockerfile to build it & am deploying it over Kubernetes.

Things seems to be working without any problem on my local machine, when I execute docker run.

The application is running and it is pushing data onto ElasticSearch.

But when I run it on K8S, I'm getting below error:

Traceback (most recent call last):
  File "application.py", line 2, in <module>
    from elasticsearch import Elasticsearch
ModuleNotFoundError: No module named 'elasticsearch'

I'm installing elasticsearch, using pip.

Dockerfile:

FROM python:3.7.3-alpine

RUN apk update && apk upgrade && apk add gcc libc-dev g++ libffi-dev libxml2 unixodbc-dev mariadb-dev postgresql-dev \
python-dev vim

RUN addgroup -S -g 1000 docker \
    && adduser -D -S -h /var/cache/docker -s /sbin/nologin -G docker -u 1000 docker \
    && chown docker:docker -R /usr/local/lib/python3.7/site-packages/

WORKDIR /app/
COPY application.py /app/
COPY lib.txt /app/
RUN chown docker:docker -R /app/

USER docker

# Install the dependencies
RUN ["pip", "install", "-r", "lib.txt", "--user"]

ENV PYTHONPATH=/usr/local/lib/python2.7/site-packages

RUN echo $PYTHONPATH

CMD [ "python",  "application.py"]

lib.txt

Flask==1.0.2
prometheus_client>=0.6.0
requests>=2.21.0
six>=1.12.0
# Elasticsearch 7.x
elasticsearch>=7.0.0,<8.0.0
pyodbc

As suggested in one answer, I'm also setting PYTHONPATH in Dockerfile.

Any suggestions, what am I missing?

Example code here.

Thanks

-- reiley
docker
elasticsearch
kubernetes
pip
python

1 Answer

9/8/2019

Try with this Dockerfile:

FROM python:3.7.3-alpine

RUN apk update && apk upgrade && apk add gcc libc-dev g++ libffi-dev libxml2 unixodbc-dev mariadb-dev postgresql-dev \
python-dev vim

# Install the dependencies
RUN pip install --upgrade pip
RUN mkdir /app
COPY lib.txt /app/lib.txt
RUN pip install -r lib.txt

RUN addgroup -S -g 1000 docker \
    && adduser -D -S -h /var/cache/docker -s /sbin/nologin -G docker -u 1000 docker

WORKDIR /app
COPY application.py /app/
RUN chown docker:docker -R /app/

USER docker

CMD [ "python",  "application.py"]

Changes:

  • Updated pip before install dependencies. This remove some warnings into my containers, and keep pip package with the last version when building the image.
  • Installed the pip packages as part of the system, when still root user is executing.
  • Removed PYTHONPATH, which seems pointing to wrong place.
  • Removed unnecessary owner changing.
-- Alejandro Visiedo GarcĂ­a
Source: StackOverflow