We recently upgraded from nginx ingress controller from 0.8.2 to 0.11.0, and started getting 502 bad gateway error on large file uploads around 10 MB or higher, we have set the client_max_body_size to 500m through proxy-body-size in the configmap and verified its set. The smaller files around 5-6 MB works fine.
There are no errors in the logs, just these messages.
redacted - [redacted] - - [25/Mar/2018:02:08:49 +0000] "POST /redacted/upload HTTP/1.1" 000 0 "https://redacted/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 3371263 10.850 [uploader-443] ----
and
[warn] 30684#30684: *42090 a client request body is buffered to a temporary file /var/lib/nginx/body/0000000482, client: redacted, server: redacted, request: "POST /redacted/upload HTTP/1.1", host: "redacted", referrer: "https://redacted/"
The proxied server is tomcat and requests do not make it to tomcat. We have tried increasing:
but nothing worked.
Going back to 0.8.2 version resolves the issue.
update 1: nginx.conf snippet
location /redacted/ {
port_in_redirect off;
set $proxy_upstream_name "redacted-443";
....
....
client_max_body_size "500m";
It looks like you may need to set the bigger file size globally for all Ingress rules.
This can be achieved by using proxy-body-size parameter i.e.:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/proxy-body-size: 20mb
....
Afterwards it should work for files bigger than 10MB.
I'd recommend using a configmap setting:
body-size: "1024m"
proxy-body-size: "1024m"Here is a full example: https://github.com/mateothegreat/k8-byexamples-ingress-controller/blob/master/manifests/configmap.yaml
To apply a configmap to the ingress-controller deployment you'll want to pass it as an argument to the container such as: https://github.com/mateothegreat/k8-byexamples-ingress-controller/blob/master/manifests/controller-deployment.yaml#L59
Did you verify the generated nginx.conf to see if the value is actually set? I used the in the ingress config itself and it works (version 0.11 as well)