I had created the a stateful set file for Elasticsearch 7.16.1 but on upgrading the ELK stack to 8.0.0, I get this error in the logs of the elastic pod:- "java.lang.IllegalStateException: failed to obtain node locks, tried [/usr/share/elasticsearch/data];
maybe these locations are not writable or multiple nodes were started on the same data path?
Likely root cause:
java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/node.lock"
Kibana pod gives this error:-
"curl: (7) Failed to connect to elastic-cluster port 9200: Connection refused"
I didn't get this error with the 7.16.1 version.
Should I make some changes in the statefulset files or any other files? Please help me solve this.
most likely the folder /usr/share/elasticsearch/data doesnt have required permissions.
include init container in your deployment manifest and provide appropriate permissions needed by the elastic process to write the index data.
Adding the following initContainers in my statefulset.yaml fixed the issue for me:
initContainers:
- name: fix-permissions
image: busybox
command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
securityContext:
privileged: true
volumeMounts:
- name: local-storage
mountPath: sharedata