I have a GKE cluster running multiple load balancers and dozens of sites each running in their own deployment as clusterip's.

What I would like to do is inject an include path to include various site configs per load balancer (nginx ingress controller). I'm not using the kubernetes ingress, rather the various load balancers. I have shared data path that is mounted that contains all the server {} nginx configs.

All I cannot figure out is how to get the load balancer to see them. I'm pretty sure it involves a configmap entry but I believe my syntax is wrong.

Just to confirm I know that if I can get this then my solution will work as if I login to the load balancer and reload nginx with a different config it works.

I just need this to happen on its own based on a different path per nginx ingress controller config. I am using different classes in each of the nginx-ingress-controllers like this

metadata:
  annotations:
    kubernetes.io/ingress.class: nginx-1

and in the respective config I'm trying to figure out the syntax for get it to see those various configs

nginx.ingress.kubernetes.io/configuration-snippet: '{"http": "{include /data/nginx/sites-enabled/*;}"}'

I even tried to to include a higher level config file for nginx that inside had the http include lines..

The install I used was helm from

https://kubernetes.github.io/ingress-nginx/deploy/

Here is an example of a config I've tried

apiVersion: v1
data:
  enable-vts-status: "false"
kind: ConfigMap
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx-1
    nginx.ingress.kubernetes.io/server-snippet: |
      http{
        include /data/nginx/sites-enabled/*;
        }
  creationTimestamp: 2018-12-13T21:00:53Z
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.0.2
    component: controller
    heritage: Tiller
    release: nginx-controller-1
  name: nginx-controller-1-nginx-ingress-controller
  namespace: default
  resourceVersion: "312253"
  selfLink: /api/v1/namespaces/default/configmaps/nginx-controller-1-nginx-ingress-controller

I have also tried putting it in the data section.

data:
  enable-vts-status: "false"
  nginx.conf: |
    http{
        include /data/nginx/sites-enabled/*;
        }

Thank you for any help

Update:

This isn't an answer but allows me to continue and use these multiple nginx ingresses on a single kubernetes cluster. Hope this helps anyone else in my shoes..

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: mysite-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx-2"
    custom.nginx.org/rate-limiting: "on"
    custom.nginx.org/rate-limiting-rate: "5r/s"
    custom.nginx.org/rate-limiting-burst: "1"
spec:
  rules:
  - host: "mysite.com"
    http:
      paths:
      - path: /
        backend:
          serviceName: mysiteservicename
          servicePort: 8080

Doing the above does modify the nginx.conf and better yet it auto refreshes the config I believe without taking down the controller.

So it appears the ingress class annotation does work for multiple nginx-ingress-controllers. In the end I needed

• nginx-ingress-controller workload (which also creates a default backend workload and the load balancer service)
• the asp.net core deployment workload (or whatever your service is)
• the Cluster IP service (the exposure of your internal deployment)
• the Ingress service that associates nginx.conf entries to the respective ingress-controller