Using Location Blocks In Kubernetes Ingress Nginx server-snippet Causes 404

10/30/2018

I'm hoping someone can help me here because I'm stuck.

I'm moving over our nginx config from a traditional nginx/node server config whereby both nginx and node server are on the same machine.

In Kubernetes, the ingress controller (nginx) obviously lives in another container.

Where I'm getting stuck is reimplementing our rules that disable access logging for images and assets using location blocks.

Our configuration looks like

location ~* \.(?:jpg|jpeg|gif|png|ico|cur|mp4|ogg|ogv|webm|htc)$ {
  access_log off;
  expires 2M;
  add_header Cache-Control "public, max-age=5184000"; # 5184000 is 60 days
}

When I implement this same block in a server-snippet it matches, but all the assets throw a 404.

I did some Googling and found an answer that might explain why here https://stackoverflow.com/a/52711388/573616

but the suggested answer hints to use an if block instead of a location block because the location interferes with the proxy upstream, however, disabling access logs is not possible from inside the if block, only from a location context.

The rest of my ingress looks like (everything else is default)

real_ip_header X-Forwarded-For;
real_ip_recursive on;        
underscores_in_headers on;
gzip_types text/css application/x-javascript application/javascript application/json image/svg+xml;
client_max_body_size 5M;

proxy_buffers 8 16k;      
proxy_set_header X-Request-Start "t=${msec}";   
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_redirect off;

The images live at /images/ on the upstream server path.

So I'm back to trying to figure out how to get these location blocks working so I can actually disable the access logs for these images from a server-snippet

So can anyone tell me how to get the above location block to not throw 404's for assets in an ingress controller?

-- Cheyne
kubernetes
kubernetes-ingress
nginx

1 Answer

10/31/2018

I'm assuming that your backend is serving your assets, so I think the problem is that your location {} block doesn't have an upstream like the regular paths defined in the nginx ingress.

There's a lot of lua code in the nginx.conf of your nginx-ingress-controller so it might take time to understand, but you can copy your nginx.conf locally:

$ kubectl cp nginx-ingress-controller-xxxxxxxxx-xxxxx:nginx.conf .

Check the location {} blocks that are defined for your current services and copy them in the bottom of your server-snippet location {} block like this:

I believe a server-snippet like this would do:

location ~* \.(?:jpg|jpeg|gif|png|ico|cur|mp4|ogg|ogv|webm|htc)$ {
  access_log off;
  expires 2M;
  add_header Cache-Control "public, max-age=5184000"; # 5184000 is 60 days
  <== add what you copied here
  set $namespace      "k8s-namespace";
  set $ingress_name   "ingress-name";
  set $service_name   "service-name";
  set $service_port   "80";
  set $location_path  "/images";
  ...
  ...
  ...
  proxy_http_version                      1.1;

  proxy_cookie_domain                     off;
  proxy_cookie_path                       off;

  # In case of errors try the next upstream server before returning an error
  proxy_next_upstream                     error timeout;
  proxy_next_upstream_tries               3;

  proxy_pass http://upstream_balancer;

  proxy_redirect                          off;
}
-- Rico
Source: StackOverflow