I am trying to duplicate an incoming request containing a file in it. This request will need to be processed by 2 backends, one being responsible for storing an image and another for processing it. The official documentation describes a method by using the mirror feature in nginx. Upon trying this, I get an error saying that the subrequest is too large.

My relevant ingress annotations (parts of uri replaced):

nginx.ingress.kubernetes.io/use-reges: "true"
nginx.ingress.kubernetes.io/rewrite-target: /backend/process
nginx.ingress.kubernetes.io/proxy-body-size: 50m
nginx.ingress.kubernetes.io/mirror-target: https://us-west2-project-name.cloudfunctions.net/backend-store

The error I get when using these annotations (parts of log replaced):

2021/09/30 04:47:06 [warn] 108#108: *3214706 a client request body is buffered to a temporary file /tmp/client-body/0000000024, client: 123.123.123.123, server: staging.some-service.com, request: "POST /rewrite-path HTTP/1.1", host: "staging.some-service.com"
2021/09/30 04:47:06 [error] 108#108: *3214706 client intended to send too large body: 1479070 bytes, client: 182.171.248.99, server: staging.some-service.com, request: "POST /rewrite-path HTTP/1.1", subrequest: "/_mirror-9901c7c6-0757-4cec-96bf-e4317981a8f7", host: "staging.some-service.com"

My image processing backend seems to work fine, the storing backend is never invoked. Can someone point out the issue here

Some Extra information relevant to the generated nginx.conf file inside my controller.

bash-5.1$ cat nginx.conf | grep body_size
			client_max_body_size                    50m;
			client_max_body_size                    50m;
			client_max_body_size                    50m;
			client_max_body_size                    50m;
			client_max_body_size                    50m;
			client_max_body_size                    50m;
			client_max_body_size                    50m;
			client_max_body_size                    21M;

Relevant location and server blocks (sensitive info replaced):

server {
		server_name staging.some-service.com ;
		
		listen 80  ;
		listen 443  ssl http2 ;
		
		set $proxy_upstream_name "-";
		
		ssl_certificate_by_lua_block {
			certificate.call()
		}
		
		location = /_mirror-9901c7c6-0757-4cec-96bf-e4317981a8f7 {
			internal;
			proxy_pass https://us-west2-secret-project.cloudfunctions.net/secret-endpoint;
		}

        # ... extra config
		location ~* "^/backend/secret/process" {
			
			set $namespace      "default";
			set $ingress_name   "ingress-torchserve-service-stg";
			set $service_name   "torchserve-service-stg";
			set $service_port   "infer-stg";
			set $location_path  "/super-secret";
			set $global_rate_limit_exceeding n;
			
			mirror /_mirror-9901c7c6-0757-4cec-96bf-e4317981a8f7;
			mirror_request_body on;
            # ...
            client_max_body_size                    50m;