I am using Strapi within a digital-ocean Kubernetes cluster. The public folder images are handled by a persistent volume claim(PVC). After a redeploy, the images are visible within Strapi and also from http://api.mywebsite.com/uploads/blabla.jpg. An Imaginary image processor located within the same cluster returns a 404 Error when trying to get the same images from Strapi.

What might be the cause of this?

I have tried to build an initContainer like written here https://medium.com/faun/digitalocean-kubernetes-and-volume-permissions-820f46598965 but it did not help.

initContainers:
  - name: data-permissions-fix
    image: busybox
    command: ["/bin/chmod","-R","777", "/backend/public/uploads"]
    volumeMounts:
      - name: backend-images
        mountPath: /backend/public/uploads

The flow is like this: frontend -> ingress -> image-processor (Fastify server) -> imaginary -> backend

Backend:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.18.0 ()
  creationTimestamp: null
  labels:
    io.kompose.service: backend
  name: backend
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        io.kompose.service: backend
    spec:
      containers:
        image: backend
        name: backend
        ports:
        - containerPort: 1337
        resources: {}
        volumeMounts:
        - mountPath: /backend/public/uploads
          name: backend-images
          readOnly: false
      initContainers:
      - name: data-permissions-fix
        image: busybox
        command: ["/bin/chmod","-R","777", "/backend/public/uploads"]
        volumeMounts:
        - name: backend-images
          mountPath: /backend/public/uploads
      volumes:
      - name: backend-images
        persistentVolumeClaim:
          claimName: backend-images
      initContainers:
      - name: init-db
        image: busybox
        command: ['sh', '-c', 'until nc -z db:5432; do echo waiting for db; sleep 2; done;']
      restartPolicy: Always
status: {}

Backend PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  creationTimestamp: null
  labels:
    io.kompose.service: backend-images
  name: backend-images
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: do-block-storage
  # persistentVolumeReclaimPolicy: Recycle
  resources:
    requests:
      storage: 1Gi
status: {}

Describe backend pod:

Name:               backend-5f-vhx48
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               pool-1-xveq/10.135.181.55
Start Time:         Thu, 27 Jun 2019 19:07:31 +0200
Labels:             io.kompose.service=backend
                    pod-template-hash=5f9fb4fbb6
Annotations:        <none>
Status:             Running
IP:                 10.244.1.92
Controlled By:      ReplicaSet/backend-5f9fbb6
Init Containers:
  init-db:
    Container ID:  docker://e4728305d970fb2d76f1f203271d3ce902a5ef56
    Image:         busybox
    Image ID:      docker-pullable://busybox@sha256:7a4d4ed96e15da96906910d57fc4a13210160
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      until nc -z db:5432; do echo waiting for db; sleep 2; done;
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Thu, 27 Jun 2019 19:07:39 +0200
      Finished:     Thu, 27 Jun 2019 19:07:39 +0200
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-fl98h (ro)
Containers:
  backend:
    Container ID:   docker://b42bea24655d3d40e59985f8fff96bce
    Image:          backend
    Image ID:       docker-pullable://backend@sha25663765ef8841b45e4717f047b71446c1058d2
    Port:           1337/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 27 Jun 2019 19:07:41 +0200
    Ready:          True
    Restart Count:  0
    Environment:
    Mounts:
      /usr/src/backend/public/uploads from backend-images-teuberkohlhoff (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-fl98h (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  backend-images-teuberkohlhoff:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  backend-images
    ReadOnly:   false
  default-token-fl98h:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-flh72
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

Describe PVC:

Name:          backend-images
Namespace:     default
StorageClass:  do-block-storage
Status:        Bound
Volume:        pvc-de757a78-8b8a-364b3aed3
Labels:        io.kompose.service=backend-images
Annotations:   kubectl.kubernetes.io/last-applied-configuration:
                 {"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"creationTimestamp":null,"labels":{"io.kompose.service":"ba...
               pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: dobs.csi.digitalocean.com
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      1Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Events:        <none>
Mounted By:    backend-5f-vhx48

Image-processor:

const imaginary = require('imaginary');
const fastify = require('fastify')({ logger: true });

const imageServer = 'http://imaginary:9000/';

fastify.get('*', async (request, reply) => {
  const {
    filename, type: format, width: imageWidth, url: imageUrl,
  } = request.query;

  const imageStream = imaginary()
    .server(imageServer)
    .resize({ width: imageWidth, url: imageUrl, type: format })
    .on('error', (err) => {
      console.error('Cannot resize the image:', err);
    });

  reply
    .header('Content-Disposition', `attachment; filename="${filename}.${format}"`)
    .header('Content-Type', `image/${format}`)
    .send(imageStream);
});

const start = async () => {
  try {
    await fastify.listen(9009, '0.0.0.0');
    fastify.log.info(`server listening on ${fastify.server.address().port}`);
  } catch (err) {
    fastify.log.error('ERROR', err);
    process.exit(1);
  }
};
start();

The frontend img-url is

http://imagehandling.domain.com/b2b36f31caa9d8f6/320/title.webp?type=webp&width=320&url=http://backend:1337/uploads/b2b36f31caa9d8f6.jpg&filename=title