Kubernetes nginx webserver configmap stops nginx working

5/22/2018

I'm trying to apply my configurations to nginx webserver via configmap but I'm getting connection refused from nginx

if I comment out the volume it works fine

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx
  namespace: namespace
data:
  nginx.conf: |
    user  nginx;
    worker_processes  1;

    error_log  /var/log/nginx/error.log warn;
    pid        /var/run/nginx.pid;


    events {
        worker_connections  1024;
    }


    http {
        include       /etc/nginx/mime.types;
        default_type  application/octet-stream;

        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                          '$status $body_bytes_sent "$http_referer" '
                          '"$http_user_agent" "$http_x_forwarded_for"';

        access_log  /var/log/nginx/access.log  main;

        sendfile        on;
        #tcp_nopush     on;

        keepalive_timeout  65;

        #gzip  on;

        # client_body_buffer_size 32k;
        # client_header_buffer_size 1k;
        # large_client_header_buffers 4 8k;
        # # php max upload limit cannot be larger than this
        # client_max_body_size 33m;
        #
        # proxy_buffers 16 32k;
        # proxy_buffer_size 64k;
        #
        # fastcgi_buffers 16 128k;
        # fastcgi_buffer_size 256k;
        #
        # upstream php {
        #     #server unix:/tmp/php-cgi.socket;
        #     server 127.0.0.1:9000;
        # }
        #
        # fastcgi_cache_path /var/run/nginx-cache levels=1:2 keys_zone=WORDPRESS:100m inactive=60m;
        # fastcgi_cache_key "$scheme$request_method$host$request_uri";
        # fastcgi_cache_use_stale error timeout invalid_header http_500;
        # fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
        #
        # real_ip_header X-Forwarded-For;
        # set_real_ip_from 0.0.0.0/0;
        #
        # limit_req_zone $binary_remote_addr zone=one:10m rate=16r/s;
        # limit_req_zone $binary_remote_addr zone=two:10m rate=30r/m;
        # limit_conn_zone $binary_remote_addr zone=addr:10m;
        # limit_conn addr 10;
        # limit_conn_status 444;

        include /etc/nginx/conf.d/*.conf;
    }

  fastcgi_params: |
    fastcgi_param  QUERY_STRING       $query_string;
    fastcgi_param  REQUEST_METHOD     $request_method;
    fastcgi_param  CONTENT_TYPE       $content_type;
    fastcgi_param  CONTENT_LENGTH     $content_length;

    fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
    fastcgi_param  REQUEST_URI        $request_uri;
    fastcgi_param  DOCUMENT_URI       $document_uri;
    fastcgi_param  DOCUMENT_ROOT      $document_root;
    fastcgi_param  SERVER_PROTOCOL    $server_protocol;
    fastcgi_param  REQUEST_SCHEME     $scheme;
    fastcgi_param  HTTPS              $https if_not_empty;

    fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
    fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;

    fastcgi_param  REMOTE_ADDR        $remote_addr;
    fastcgi_param  REMOTE_PORT        $remote_port;
    fastcgi_param  SERVER_ADDR        $server_addr;
    fastcgi_param  SERVER_PORT        $server_port;
    fastcgi_param  SERVER_NAME        $server_name;

    # PHP only, required if PHP was built with --enable-force-cgi-redirect
    fastcgi_param  REDIRECT_STATUS    200;

  scgi_params: |
    scgi_param  REQUEST_METHOD     $request_method;
    scgi_param  REQUEST_URI        $request_uri;
    scgi_param  QUERY_STRING       $query_string;
    scgi_param  CONTENT_TYPE       $content_type;

    scgi_param  DOCUMENT_URI       $document_uri;
    scgi_param  DOCUMENT_ROOT      $document_root;
    scgi_param  SCGI               1;
    scgi_param  SERVER_PROTOCOL    $server_protocol;
    scgi_param  REQUEST_SCHEME     $scheme;
    scgi_param  HTTPS              $https if_not_empty;

    scgi_param  REMOTE_ADDR        $remote_addr;
    scgi_param  REMOTE_PORT        $remote_port;
    scgi_param  SERVER_PORT        $server_port;
    scgi_param  SERVER_NAME        $server_name;

  uwsgi_params: |
    uwsgi_param  QUERY_STRING       $query_string;
    uwsgi_param  REQUEST_METHOD     $request_method;
    uwsgi_param  CONTENT_TYPE       $content_type;
    uwsgi_param  CONTENT_LENGTH     $content_length;

    uwsgi_param  REQUEST_URI        $request_uri;
    uwsgi_param  PATH_INFO          $document_uri;
    uwsgi_param  DOCUMENT_ROOT      $document_root;
    uwsgi_param  SERVER_PROTOCOL    $server_protocol;
    uwsgi_param  REQUEST_SCHEME     $scheme;
    uwsgi_param  HTTPS              $https if_not_empty;

    uwsgi_param  REMOTE_ADDR        $remote_addr;
    uwsgi_param  REMOTE_PORT        $remote_port;
    uwsgi_param  SERVER_PORT        $server_port;
    uwsgi_param  SERVER_NAME        $server_name;

  mime.types: |
    types {
        text/html                                        html htm shtml;
        text/css                                         css;
        text/xml                                         xml;
        image/gif                                        gif;
        image/jpeg                                       jpeg jpg;
        application/javascript                           js;
        application/atom+xml                             atom;
        application/rss+xml                              rss;

        text/mathml                                      mml;
        text/plain                                       txt;
        text/vnd.sun.j2me.app-descriptor                 jad;
        text/vnd.wap.wml                                 wml;
        text/x-component                                 htc;

        image/png                                        png;
        image/svg+xml                                    svg svgz;
        image/tiff                                       tif tiff;
        image/vnd.wap.wbmp                               wbmp;
        image/webp                                       webp;
        image/x-icon                                     ico;
        image/x-jng                                      jng;
        image/x-ms-bmp                                   bmp;

        application/font-woff                            woff;
        application/java-archive                         jar war ear;
        application/json                                 json;
        application/mac-binhex40                         hqx;
        application/msword                               doc;
        application/pdf                                  pdf;
        application/postscript                           ps eps ai;
        application/rtf                                  rtf;
        application/vnd.apple.mpegurl                    m3u8;
        application/vnd.google-earth.kml+xml             kml;
        application/vnd.google-earth.kmz                 kmz;
        application/vnd.ms-excel                         xls;
        application/vnd.ms-fontobject                    eot;
        application/vnd.ms-powerpoint                    ppt;
        application/vnd.oasis.opendocument.graphics      odg;
        application/vnd.oasis.opendocument.presentation  odp;
        application/vnd.oasis.opendocument.spreadsheet   ods;
        application/vnd.oasis.opendocument.text          odt;
        application/vnd.openxmlformats-officedocument.presentationml.presentation
                                                         pptx;
        application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
                                                         xlsx;
        application/vnd.openxmlformats-officedocument.wordprocessingml.document
                                                         docx;
        application/vnd.wap.wmlc                         wmlc;
        application/x-7z-compressed                      7z;
        application/x-cocoa                              cco;
        application/x-java-archive-diff                  jardiff;
        application/x-java-jnlp-file                     jnlp;
        application/x-makeself                           run;
        application/x-perl                               pl pm;
        application/x-pilot                              prc pdb;
        application/x-rar-compressed                     rar;
        application/x-redhat-package-manager             rpm;
        application/x-sea                                sea;
        application/x-shockwave-flash                    swf;
        application/x-stuffit                            sit;
        application/x-tcl                                tcl tk;
        application/x-x509-ca-cert                       der pem crt;
        application/x-xpinstall                          xpi;
        application/xhtml+xml                            xhtml;
        application/xspf+xml                             xspf;
        application/zip                                  zip;

        application/octet-stream                         bin exe dll;
        application/octet-stream                         deb;
        application/octet-stream                         dmg;
        application/octet-stream                         iso img;
        application/octet-stream                         msi msp msm;

        audio/midi                                       mid midi kar;
        audio/mpeg                                       mp3;
        audio/ogg                                        ogg;
        audio/x-m4a                                      m4a;
        audio/x-realaudio                                ra;

        video/3gpp                                       3gpp 3gp;
        video/mp2t                                       ts;
        video/mp4                                        mp4;
        video/mpeg                                       mpeg mpg;
        video/quicktime                                  mov;
        video/webm                                       webm;
        video/x-flv                                      flv;
        video/x-m4v                                      m4v;
        video/x-mng                                      mng;
        video/x-ms-asf                                   asx asf;
        video/x-ms-wmv                                   wmv;
        video/x-msvideo                                  avi;
      }

  win-utf: |
    # This map is not a full windows-1251 <> utf8 map: it does not
    # contain Serbian and Macedonian letters.  If you need a full map,
    # use contrib/unicode2nginx/win-utf map instead.

    charset_map  windows-1251  utf-8 {

        82  E2809A ; # single low-9 quotation mark

        84  E2809E ; # double low-9 quotation mark
        85  E280A6 ; # ellipsis
        86  E280A0 ; # dagger
        87  E280A1 ; # double dagger
        88  E282AC ; # euro
        89  E280B0 ; # per mille

        91  E28098 ; # left single quotation mark
        92  E28099 ; # right single quotation mark
        93  E2809C ; # left double quotation mark
        94  E2809D ; # right double quotation mark
        95  E280A2 ; # bullet
        96  E28093 ; # en dash
        97  E28094 ; # em dash

        99  E284A2 ; # trade mark sign

        A0  C2A0 ;   #  
        A1  D18E ;   # capital Byelorussian short U
        A2  D19E ;   # small Byelorussian short u

        A4  C2A4 ;   # currency sign
        A5  D290 ;   # capital Ukrainian soft G
        A6  C2A6 ;   # borken bar
        A7  C2A7 ;   # section sign
        A8  D081 ;   # capital YO
        A9  C2A9 ;   # (C)
        AA  D084 ;   # capital Ukrainian YE
        AB  C2AB ;   # left-pointing double angle quotation mark
        AC  C2AC ;   # not sign
        AD  C2AD ;   # soft hypen
        AE  C2AE ;   # (R)
        AF  D087 ;   # capital Ukrainian YI

        B0  C2B0 ;   # °
        B1  C2B1 ;   # plus-minus sign
        B2  D086 ;   # capital Ukrainian I
        B3  D196 ;   # small Ukrainian i
        B4  D291 ;   # small Ukrainian soft g
        B5  C2B5 ;   # micro sign
        B6  C2B6 ;   # pilcrow sign
        B7  C2B7 ;   # ·
        B8  D191 ;   # small yo
        B9  E28496 ; # numero sign
        BA  D194 ;   # small Ukrainian ye
        BB  C2BB ;   # right-pointing double angle quotation mark

        BF  D197 ;   # small Ukrainian yi

        C0  D090 ;   # capital A
        C1  D091 ;   # capital B
        C2  D092 ;   # capital V
        C3  D093 ;   # capital G
        C4  D094 ;   # capital D
        C5  D095 ;   # capital YE
        C6  D096 ;   # capital ZH
        C7  D097 ;   # capital Z
        C8  D098 ;   # capital I
        C9  D099 ;   # capital J
        CA  D09A ;   # capital K
        CB  D09B ;   # capital L
        CC  D09C ;   # capital M
        CD  D09D ;   # capital N
        CE  D09E ;   # capital O
        CF  D09F ;   # capital P

        D0  D0A0 ;   # capital R
        D1  D0A1 ;   # capital S
        D2  D0A2 ;   # capital T
        D3  D0A3 ;   # capital U
        D4  D0A4 ;   # capital F
        D5  D0A5 ;   # capital KH
        D6  D0A6 ;   # capital TS
        D7  D0A7 ;   # capital CH
        D8  D0A8 ;   # capital SH
        D9  D0A9 ;   # capital SHCH
        DA  D0AA ;   # capital hard sign
        DB  D0AB ;   # capital Y
        DC  D0AC ;   # capital soft sign
        DD  D0AD ;   # capital E
        DE  D0AE ;   # capital YU
        DF  D0AF ;   # capital YA

        E0  D0B0 ;   # small a
        E1  D0B1 ;   # small b
        E2  D0B2 ;   # small v
        E3  D0B3 ;   # small g
        E4  D0B4 ;   # small d
        E5  D0B5 ;   # small ye
        E6  D0B6 ;   # small zh
        E7  D0B7 ;   # small z
        E8  D0B8 ;   # small i
        E9  D0B9 ;   # small j
        EA  D0BA ;   # small k
        EB  D0BB ;   # small l
        EC  D0BC ;   # small m
        ED  D0BD ;   # small n
        EE  D0BE ;   # small o
        EF  D0BF ;   # small p

        F0  D180 ;   # small r
        F1  D181 ;   # small s
        F2  D182 ;   # small t
        F3  D183 ;   # small u
        F4  D184 ;   # small f
        F5  D185 ;   # small kh
        F6  D186 ;   # small ts
        F7  D187 ;   # small ch
        F8  D188 ;   # small sh
        F9  D189 ;   # small shch
        FA  D18A ;   # small hard sign
        FB  D18B ;   # small y
        FC  D18C ;   # small soft sign
        FD  D18D ;   # small e
        FE  D18E ;   # small yu
        FF  D18F ;   # small ya
    }

  koi-utf: |
    # This map is not a full windows-1251 <> utf8 map: it does not
    # contain Serbian and Macedonian letters.  If you need a full map,
    # use contrib/unicode2nginx/win-utf map instead.

    charset_map  windows-1251  utf-8 {

    82  E2809A ; # single low-9 quotation mark

    84  E2809E ; # double low-9 quotation mark
    85  E280A6 ; # ellipsis
    86  E280A0 ; # dagger
    87  E280A1 ; # double dagger
    88  E282AC ; # euro
    89  E280B0 ; # per mille

    91  E28098 ; # left single quotation mark
    92  E28099 ; # right single quotation mark
    93  E2809C ; # left double quotation mark
    94  E2809D ; # right double quotation mark
    95  E280A2 ; # bullet
    96  E28093 ; # en dash
    97  E28094 ; # em dash

    99  E284A2 ; # trade mark sign

    A0  C2A0 ;   #  
    A1  D18E ;   # capital Byelorussian short U
    A2  D19E ;   # small Byelorussian short u

    A4  C2A4 ;   # currency sign
    A5  D290 ;   # capital Ukrainian soft G
    A6  C2A6 ;   # borken bar
    A7  C2A7 ;   # section sign
    A8  D081 ;   # capital YO
    A9  C2A9 ;   # (C)
    AA  D084 ;   # capital Ukrainian YE
    AB  C2AB ;   # left-pointing double angle quotation mark
    AC  C2AC ;   # not sign
    AD  C2AD ;   # soft hypen
    AE  C2AE ;   # (R)
    AF  D087 ;   # capital Ukrainian YI

    B0  C2B0 ;   # °
    B1  C2B1 ;   # plus-minus sign
    B2  D086 ;   # capital Ukrainian I
    B3  D196 ;   # small Ukrainian i
    B4  D291 ;   # small Ukrainian soft g
    B5  C2B5 ;   # micro sign
    B6  C2B6 ;   # pilcrow sign
    B7  C2B7 ;   # ·
    B8  D191 ;   # small yo
    B9  E28496 ; # numero sign
    BA  D194 ;   # small Ukrainian ye
    BB  C2BB ;   # right-pointing double angle quotation mark

    BF  D197 ;   # small Ukrainian yi

    C0  D090 ;   # capital A
    C1  D091 ;   # capital B
    C2  D092 ;   # capital V
    C3  D093 ;   # capital G
    C4  D094 ;   # capital D
    C5  D095 ;   # capital YE
    C6  D096 ;   # capital ZH
    C7  D097 ;   # capital Z
    C8  D098 ;   # capital I
    C9  D099 ;   # capital J
    CA  D09A ;   # capital K
    CB  D09B ;   # capital L
    CC  D09C ;   # capital M
    CD  D09D ;   # capital N
    CE  D09E ;   # capital O
    CF  D09F ;   # capital P

    D0  D0A0 ;   # capital R
    D1  D0A1 ;   # capital S
    D2  D0A2 ;   # capital T
    D3  D0A3 ;   # capital U
    D4  D0A4 ;   # capital F
    D5  D0A5 ;   # capital KH
    D6  D0A6 ;   # capital TS
    D7  D0A7 ;   # capital CH
    D8  D0A8 ;   # capital SH
    D9  D0A9 ;   # capital SHCH
    DA  D0AA ;   # capital hard sign
    DB  D0AB ;   # capital Y
    DC  D0AC ;   # capital soft sign
    DD  D0AD ;   # capital E
    DE  D0AE ;   # capital YU
    DF  D0AF ;   # capital YA

    E0  D0B0 ;   # small a
    E1  D0B1 ;   # small b
    E2  D0B2 ;   # small v
    E3  D0B3 ;   # small g
    E4  D0B4 ;   # small d
    E5  D0B5 ;   # small ye
    E6  D0B6 ;   # small zh
    E7  D0B7 ;   # small z
    E8  D0B8 ;   # small i
    E9  D0B9 ;   # small j
    EA  D0BA ;   # small k
    EB  D0BB ;   # small l
    EC  D0BC ;   # small m
    ED  D0BD ;   # small n
    EE  D0BE ;   # small o
    EF  D0BF ;   # small p

    F0  D180 ;   # small r
    F1  D181 ;   # small s
    F2  D182 ;   # small t
    F3  D183 ;   # small u
    F4  D184 ;   # small f
    F5  D185 ;   # small kh
    F6  D186 ;   # small ts
    F7  D187 ;   # small ch
    F8  D188 ;   # small sh
    F9  D189 ;   # small shch
    FA  D18A ;   # small hard sign
    FB  D18B ;   # small y
    FC  D18C ;   # small soft sign
    FD  D18D ;   # small e
    FE  D18E ;   # small yu
    FF  D18F ;   # small ya
    }

  koi-win: |
    charset_map  koi8-r  windows-1251 {

        80  88 ; # euro

        95  95 ; # bullet

        9A  A0 ; #  

        9E  B7 ; # ·

        A3  B8 ; # small yo
        A4  BA ; # small Ukrainian ye

        A6  B3 ; # small Ukrainian i
        A7  BF ; # small Ukrainian yi

        AD  B4 ; # small Ukrainian soft g
        AE  A2 ; # small Byelorussian short u

        B0  B0 ; # °

        B3  A8 ; # capital YO
        B4  AA ; # capital Ukrainian YE

        B6  B2 ; # capital Ukrainian I
        B7  AF ; # capital Ukrainian YI

        B9  B9 ; # numero sign

        BD  A5 ; # capital Ukrainian soft G
        BE  A1 ; # capital Byelorussian short U

        BF  A9 ; # (C)

        C0  FE ; # small yu
        C1  E0 ; # small a
        C2  E1 ; # small b
        C3  F6 ; # small ts
        C4  E4 ; # small d
        C5  E5 ; # small ye
        C6  F4 ; # small f
        C7  E3 ; # small g
        C8  F5 ; # small kh
        C9  E8 ; # small i
        CA  E9 ; # small j
        CB  EA ; # small k
        CC  EB ; # small l
        CD  EC ; # small m
        CE  ED ; # small n
        CF  EE ; # small o

        D0  EF ; # small p
        D1  FF ; # small ya
        D2  F0 ; # small r
        D3  F1 ; # small s
        D4  F2 ; # small t
        D5  F3 ; # small u
        D6  E6 ; # small zh
        D7  E2 ; # small v
        D8  FC ; # small soft sign
        D9  FB ; # small y
        DA  E7 ; # small z
        DB  F8 ; # small sh
        DC  FD ; # small e
        DD  F9 ; # small shch
        DE  F7 ; # small ch
        DF  FA ; # small hard sign

        E0  DE ; # capital YU
        E1  C0 ; # capital A
        E2  C1 ; # capital B
        E3  D6 ; # capital TS
        E4  C4 ; # capital D
        E5  C5 ; # capital YE
        E6  D4 ; # capital F
        E7  C3 ; # capital G
        E8  D5 ; # capital KH
        E9  C8 ; # capital I
        EA  C9 ; # capital J
        EB  CA ; # capital K
        EC  CB ; # capital L
        ED  CC ; # capital M
        EE  CD ; # capital N
        EF  CE ; # capital O

        F0  CF ; # capital P
        F1  DF ; # capital YA
        F2  D0 ; # capital R
        F3  D1 ; # capital S
        F4  D2 ; # capital T
        F5  D3 ; # capital U
        F6  C6 ; # capital ZH
        F7  C2 ; # capital V
        F8  DC ; # capital soft sign
        F9  DB ; # capital Y
        FA  C7 ; # capital Z
        FB  D8 ; # capital SH
        FC  DD ; # capital E
        FD  D9 ; # capital SHCH
        FE  D7 ; # capital CH
        FF  DA ; # capital hard sign
    }

  modules:

Container nginx:

  - name: nginx
    image: nginx:1.14
    ports:
    - containerPort: 80
    env:
    volumeMounts:
    - name: persistent-storage
      mountPath: /var/www/html
    # Commenting out this makes it work:
    - name: nginx
      mountPath: /etc/nginx/
    #
    - name: nginx-conf-d
      mountPath: /etc/nginx-conf/conf.d
    - name: nginx-global
      mountPath: /etc/nginx-conf/global
    - name: nginx-html
      mountPath: /etc/nginx-conf/html
    - name: cache
      mountPath: /var/run/nginx-cache
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: pvc
  - name: cache
    emptyDir:
      medium: Memory
  - name: nginx
    configMap:
      name: nginx
  - name: nginx-conf-d
    configMap:
      name: nginx-conf-d
  - name: nginx-global
    configMap:
      name: nginx-global
  - name: nginx-html
    configMap:
      name: nginx-html

The configmap is now identical when commented out to the default nginx container but it doesn't work through configmaps

I just keep getting connection refused from nginx

Edit: nginx conf.d files:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-conf-d
  namespace: thehub
data:
  default.conf: |
    server {
          ## Your website name goes here.
          server_name _;
          ## Your only path reference.
          ## This should be in your http block and if it is, it's not needed here.
          #index index.php;

          root /var/www/html;
          include /etc/nginx-conf/global/*.conf;

          location / {
                  # This is cool because no php is touched for static content.
                  # include the "?$args" part so non-default permalinks doesn't break when using query string
                  try_files $uri $uri/ /index.php$is_args$args;
          }

          location ~ \.php$ {
                  #NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
                  limit_req zone=one burst=12 nodelay;
                  include fastcgi_params;
                  fastcgi_pass php;
          }
    }
-- Nathan Ogden
docker
kubernetes
nginx

1 Answer

5/22/2018

When nginx configmap is mounted to the pod, it's supposed to occupy the whole directory, that means, when you mount nginx cm into /etc/nginx/ directory, the directory /etc/nginx has only one file: nginx.conf, of couse the pod will fail.

You can try to mount nginx into another directory, like /etc/ngx, then start the pod with command: [ "/bin/sh", "-c", "nginx -c /etc/ngx/nginx.conf -g 'daemon off;'" ]

-- Kun Li
Source: StackOverflow