Automating snapshot repository creation on elasticsearch - kubernetes

9/19/2018

We got Kubernetes on AWS (with EKS if that matters). Elasticsearch is deployed in pod. We do not use ES service on AWS. I've configured ES with a snapshot repository that is located on S3 bucket. Everything works fine.

However I created the snapshot repository through a REST api call - MANUALLY. We will use Curator to run snapshot jobs however, it seems curator expects a snapshot repository in place. And it seems there is no action to help with snapshot repository creation

I need to automate the snapshot repository creation. I was looking to find if there is a file where this configuration is located but it seems to be on the elasticsearch data rather. Recreating the cluster and mounting the data volumes brings the snapshot repository back. So probably is one time configuration.

We use yaml manifests created by helm charts.

curl -X PUT "localhost:9200/_snapshot/s3-snapshot-repo" -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "my_bucket"
  }
}
'

What would be the best way to achieve this? I've been thinking about a lifecycle postStart command but not sure how stable this will be, in terms of running when API is up etc.

-- Greg Hill
elasticsearch
elasticsearch-curator
kubernetes
repository
snapshot

1 Answer

9/20/2018

I created job that runs a bash filter that in turns makes REST calls with curl to elasticsearch service and configures it. The script waits for ES service to be up for a certain amount of time.

-- Greg Hill
Source: StackOverflow