K
Q

Running an Ingress controller with hostNetwork: true

6/30/2020

In working through a solution to one Stack Overflow post that I wrote, I ran into a separate point of confusion. I'm not sure how to search for the answer to this question as there are too many potential pieces that I don't understand.

Situation

I want to run an Ingress controller on hostNetwork (for dev purposes). What I've learned so far is that I can create an Ingress controller with a config file like this:

apiVersion: networking.k8s.io/v1beta1 # for versions before 1.14 use extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1 
spec:
  rules:
  - http:
      paths:
      - path: /
        backend:
          serviceName: web 
          servicePort: 8080

It doesn't matter too much what web is other than it's a service running in my cluster.

In the question I linked to above, I'm instructed to "Install nginx ingress controller and use hostNetwork: true in the deployment.yaml".

I haven't created a deployment.yaml for this project, but my understanding is that it's used for defining Pods, and how I want those pods to scale with replica sets.

Question

Are deployments used just for defining Pods (as is my current understanding) or can I also include my service/ingress in there? In either case, where would I define the hostNetwork: true directive? (NOTE: The purpose of all of this is to open up port 80 and point it towards my Ingress controller, so it doesn't seem correct that I'd just want to set hostNetwork: true on the Pods themselves.)

-- wheresmycookie
kubernetes
kubernetes-ingress

Similar Questions

GKE VPC Master Authorized Network
What version of Kubernetes Dashboard (1.6 or 1.7) is this Yaml file?
OpenShift/Kubernetes SDN: Meaning of NetId 0
How do I run a cron inside a kubernetes pod/container which has a running spring-boot application?
IBM Blockchain 2.0 (beta) does not detect cluster

2 Answers

11/5/2021

This is not obvious. The Ingress Resource Controller is deployed with a .yaml (e.g. https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.4/deploy/static/provider/baremetal/deploy.yaml)

This yaml has lots of stuff in it - one resource is the deployment. If, like me, you're deploying into a bare-metal private LAN environment, you are okay with setting the hostNetwork to true. So - wget it (wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.4/deploy/static/provider/baremetal/deploy.yaml), and modify it to set the spec of the template of the spec of the Deployment, so that hostNetwork: true - something like (scroll down to the end):

# Source: ingress-nginx/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.6
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.0.4
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/component: controller
  revisionHistoryLimit: 10
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    spec:
      dnsPolicy: ClusterFirst
      containers:
        - name: controller
          image: k8s.gcr.io/ingress-nginx/controller:v1.0.4@sha256:545cff00370f28363dad31e3b59a94ba377854d3a11f18988f5f9e56841ef9ef
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --election-id=ingress-controller-leader
            - --controller-class=k8s.io/ingress-nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          livenessProbe:
            failureThreshold: 5
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP
          volumeMounts:
            - name: webhook-cert
              mountPath: /usr/local/certificates/
              readOnly: true
          resources:
            requests:
              cpu: 100m
              memory: 90Mi
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      hostNetwork: true
      volumes:
        - name: webhook-cert
          secret:
            secretName: ingress-nginx-admission

Deploy that, then setup your ingress resource to find your controller.

Something like:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-api-ingress
  annotations:
    kubernetes.io/ingress.class: nginx

Your key tests are:

  • Does my ingress have a dedicated node in my bare-metal implementation?
  • if I hit port 80 on the ingress node, do I get the same thing as hitting the NodePort? (let's say NodePort is 31207 - do I get the same thing hitting port 80 on the ingress node as hitting port 31207 on any node?)

Final note: Ingress has changed a lot over the past few years, and tutorials are often providing examples that don't pass validation - please feel free to comment on this answer if it has become out of date.

Also, some references will casually say "just update the deployment!" - the deployment they're referring to is the Ingress Controller's deployment, not your pod deployment or anything like that.

-- James_SO
Source: StackOverflow
7/1/2020

After you have installed nginx ingress controller there should be a deployment ingress-nginx-controller in namespace ingress-nginx. You need to modify the deployment. You can edit it using 

kubectl edit deploy ingress-nginx-controller -n ingress-nginx

and add hostNetwork: true in the pod spec section

...
spec:
  hostNetwork: true
  containers:
    - name: controller
      image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0
...
-- Arghya Sadhu
Source: StackOverflow