K
Q

Question

Elasticsearch pod is not running restarting

5/1/2019

I am using this helm chart : https://github.com/imubit/graylog-helm-chart

Whic is having elasticsearch, mongodb and graylog.i want to run single pod of every services not repica pods.

when i am running single pod of elastic search it is restarting in log node initialization come but not starting pod.

i have updated version of gryalog in this helm chart to 3.0.1 and elasticsearch 6.5.0.

spec:
  serviceName: {{ template "elasticsearch.fullname" . }}
  replicas: 1
  podManagementPolicy: Parallel
  template:
    metadata:
      labels:
        app: {{ template "graylog.name" . }}
        component: elasticsearch
        release: {{ .Release.Name }}
    spec:
      terminationGracePeriodSeconds: 10
      initContainers:
      - name: set-dir-owner
        image: busybox:1.29.2
        securityContext:
          privileged: true
        command: ['sh', '-c' ,'chown -R 1000:1000 /usr/share/elasticsearch/data','sysctl -w vm.max_map_count=262144', 'chmod 777 /usr/share/elasticsearch/data','chomod 777 /usr/share/elasticsearch/data/node', 'chmod g+rwx /usr/share/elasticsearch/data', 'chgrp 1000 /usr/share/elasticsearch/data']
        volumeMounts:
        - name: elasticsearch-persistent-storage
          mountPath: /usr/share/elasticsearch/data
      containers:
        - name: elasticsearch
          image: elasticsearch:6.5.0
          securityContext:
            privileged: true
            runAsUser: 1000
          command:
          - elasticsearch
          - "-Eenforce.bootstrap.checks=true"
          - "-Ediscovery.zen.ping.unicast.hosts={{ $elasticsearchServiceName }}-0.{{ $elasticsearchServiceName }}"
          - "-Ediscovery.zen.minimum_master_nodes=1"
          - "-Ediscovery.zen.ping.unicast.hosts.resolve_timeout=90s"
          - "-Ediscovery.zen.ping_timeout=90s"
          - "-Ecluster.name=graylog"
          env:
          - name: discovery.zen.ping.unicast.hosts
            value: {{ $elasticsearchServiceName }}-0.{{ $elasticsearchServiceName }}
          - name: cluster.name
            value: "graylog"
          - name: ES_JAVA_OPTS
            value: "-Xms512m -Xmx512m"
          - name: bootstrap.memory_lock
            value: "true"
          ports:
          - containerPort: 9200
            name: http
          - containerPort: 9300
            name: transport
          volumeMounts:
          - name: elasticsearch-persistent-storage
            mountPath: /usr/share/elasticsearch/data

pod are running for other service but elastic search restarting

NAME                                READY   STATUS    RESTARTS   AGE
test-logs-graylog-elasticsearch-0   1/1     Running   4          11m
test-logs-graylog-master-0          1/1     Running   1          64m
test-logs-graylog-slave-0           1/1     Running   1          64m
test-logs-mongodb-replicaset-0      1/1     Running   0          70m

here sharing a logs :

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-05-01T05:22:35,571][WARN ][o.e.c.l.LogConfigurator  ] [unknown] Some logging configurations have %marker but don't have %node_name. We will automatically add %node_name to the pattern to ease the migration for users who customize log4j2.properties but will stop this behavior in 7.0. You should manually replace `%node_name` with `[%node_name]%marker ` in these locations:
  /usr/share/elasticsearch/config/log4j2.properties
[2019-05-01T05:22:41,066][INFO ][o.e.e.NodeEnvironment    ] [cwstj4Y] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/disk/by-id/scsi-0DO_Volume_pvc-3886ce76-6926-11e9-8fbf-7e8b62b9a87c)]], net usable_space [9.2gb], net total_space [9.7gb], types [ext4]
[2019-05-01T05:22:41,067][INFO ][o.e.e.NodeEnvironment    ] [cwstj4Y] heap size [503.6mb], compressed ordinary object pointers [true]
[2019-05-01T05:22:41,070][INFO ][o.e.n.Node               ] [cwstj4Y] node name derived from node ID [cwstj4Y8Q5-C6mdodTZqAA]; set [node.name] to override
[2019-05-01T05:22:41,071][INFO ][o.e.n.Node               ] [cwstj4Y] version[6.5.0], pid[1], build[default/tar/816e6f6/2018-11-09T18:58:36.352602Z], OS[Linux/4.19.0-0.bpo.2-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/11.0.1/11.0.1+13]
[2019-05-01T05:22:41,071][INFO ][o.e.n.Node               ] [cwstj4Y] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.unseH83l, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Xms512m, -Xmx512m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2019-05-01T05:23:01,683][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [aggs-matrix-stats]
[2019-05-01T05:23:01,683][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [analysis-common]
[2019-05-01T05:23:01,683][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [ingest-common]
[2019-05-01T05:23:01,684][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [lang-expression]
[2019-05-01T05:23:01,684][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [lang-mustache]
[2019-05-01T05:23:01,684][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [lang-painless]
[2019-05-01T05:23:01,684][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [mapper-extras]
[2019-05-01T05:23:01,684][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [parent-join]
[2019-05-01T05:23:01,684][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [percolator]
[2019-05-01T05:23:01,685][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [rank-eval]
[2019-05-01T05:23:01,685][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [reindex]
[2019-05-01T05:23:01,685][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [repository-url]
[2019-05-01T05:23:01,685][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [transport-netty4]
[2019-05-01T05:23:01,685][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [tribe]
[2019-05-01T05:23:01,685][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-ccr]
[2019-05-01T05:23:01,685][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-core]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-deprecation]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-graph]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-logstash]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-ml]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-monitoring]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-rollup]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-security]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-sql]
[2019-05-01T05:23:01,686][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-upgrade]
[2019-05-01T05:23:01,687][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded module [x-pack-watcher]
[2019-05-01T05:23:01,687][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded plugin [ingest-geoip]
[2019-05-01T05:23:01,688][INFO ][o.e.p.PluginsService     ] [cwstj4Y] loaded plugin [ingest-user-agent]
[2019-05-01T05:23:45,482][INFO ][o.e.x.s.a.s.FileRolesStore] [cwstj4Y] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
[2019-05-01T05:23:52,079][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [cwstj4Y] [controller/58] [Main.cc@109] controller (64 bit): Version 6.5.0 (Build 71882a589e5556) Copyright (c) 2018 Elasticsearch BV
[2019-05-01T05:23:59,157][INFO ][o.e.d.DiscoveryModule    ] [cwstj4Y] using discovery type [zen] and host providers [settings]
[2019-05-01T05:24:09,858][INFO ][o.e.n.Node               ] [cwstj4Y] initialized
[2019-05-01T05:24:09,858][INFO ][o.e.n.Node               ] [cwstj4Y] starting ...
[2019-05-01T05:24:11,674][INFO ][o.e.t.TransportService   ] [cwstj4Y] publish_address {10.244.0.139:9300}, bound_addresses {[::]:9300}
[2019-05-01T05:24:12,064][INFO ][o.e.b.BootstrapChecks    ] [cwstj4Y] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-05-01T05:24:12,447][INFO ][o.e.n.Node               ] [cwstj4Y] stopping ...
[2019-05-01T05:24:12,763][INFO ][o.e.n.Node               ] [cwstj4Y] stopped
[2019-05-01T05:24:12,764][INFO ][o.e.n.Node               ] [cwstj4Y] closing ...
[2019-05-01T05:24:12,954][INFO ][o.e.n.Node               ] [cwstj4Y] closed
[2019-05-01T05:24:12,965][INFO ][o.e.x.m.j.p.NativeController] [cwstj4Y] Native controller process has stopped - no new native processes can be started

-- Harsh Manvar

docker

elasticsearch

kubernetes

1 Answer

5/1/2019

I have to add the

env:
          - name: discovery.type
            value: single-node
          - name: cluster.name
            value: "graylog"

environment variables which will make a single node as a master node in elasticsearch.while the container command will be like

command:
          - elasticsearch
          - "-Ecluster.name=graylog"

-- Harsh Manvar

Source: StackOverflow

KQ

Elasticsearch pod is not running restarting

Similar Questions

1 Answer

K
Q