I am using this ingress controller and would like to setup a s3 proxy to some bucket. If I call in a browser the url
https://my-kube-server.org/img/dog.jpg I expect to see/download the image at
https://s3.eu-central-1.amazonaws.com/mybucket123/pictures/dog.jpgI can setup a rewrite rule and point to an external service as explained in this example:
kind: Service
apiVersion: v1
metadata:
name: s3-proxy
spec:
type: ExternalName
externalName: s3.eu-central-1.amazonaws.com
headers:
- host: s3.eu-central-1.amazonaws.com
But I get errors from aws because it's required to have "Host:s3.eu-central-1.amazonaws.com" in the header. I cannot set this header neither in the s3-proxy service definition nor in the ingress rule (configuration-snippet doesn't work because it will add another Host header after it's set already in the nginx.conf pod.
My solution is to take the whole location block for this ingress rule and to include it as a server-snippet, which is pretty brute force.
Another option is to have an nginx pod+service behind ingress that takes care of setting the right headers. So the flow would be request -> ingress-controller -> nginx -> s3.
Has anybody an idea how to proxy s3?
One of the possible solutions is to start the pods on each cluster node using DaemonSet that connect the S3 storage to the local directory using s3fs.
S3FS-FUSE:This is a free, open-source FUSE plugin and an easy-to-use utility which supports major Linux distributions & MacOS. S3FS also takes care of caching files locally to improve performance. This plugin simply shows the Amazon S3 bucket as a drive on your system.
Here is a good article that gives you step-by-step instructions on how to do it.
Then you can use this directory as a Volume in your Pods, for example, as a directory with a static content for your proxy server.
{ Or you can create a custom proxy server image with the s3fs tool inside and mount your S3 bucket directly into the Pod. Check out this and this articles for the details.
UPD: (This solution doesn't work yet because of limited support of FUSE in Kubernetes - FUSE volumes #7890) There is a workaround that require to run a privileged container }
There are two alternatives to s3fs available:
Alternatively, you could try Traefik ingress controller: