K
Q

Elasticsearch on Kubernetes - 'Elastic Cloud (ECK)' vs 'Helm charts'

12/3/2019

For the purpose of log file aggregation, I'm looking to setup a production Elasticsearch instance on an on-premise (vanilla) Kubernetes cluster.

There seems to be two main options for deployment:

  1. Elastic Cloud (ECK) - https://github.com/elastic/cloud-on-k8s
  2. Helm Charts - https://github.com/elastic/helm-charts

I've used the old (soon to be deprecated) helm charts successfully but just discovered ECK.

What are the benefits and disadvantages of both of these options? Any constraints or limitations that could impact long-term use?

-- Wayne Shelley
elastic-stack
elasticsearch
kubernetes
kubernetes-helm

Similar Questions

Nginx Status Code 499 using Google Cloud Kubernetes TCP Load Balancer
Clients get a RemoteDisconnected error at 60s from my flask/gunicorn/nginx-ingress stack. Why? Timeouts set...nothing
Error: `selector` does not match template `labels` after adding selector
k8s ambassador ingress sample not working

2 Answers

12/4/2019

1. Elastic Cloud (ECK):

ADVANTAGES

  • document oriented (JSON)
  • multilingual - the ICU plugin is used to index and tokenize multilingual content which is an elasticsearch plugin based on the lucene implementation of the unicode text segmentation standard
  • managing and monitoring multiple clusters
  • upgrading to new stack versions with ease
  • scaling cluster capacity up and down
  • changing cluster configuration
  • dynamically scaling local storage (includes Elastic Local Volume, a local storage driver)
  • scheduling backups
  • secure by default - have encryption enabled and are protected with a strong default password right at creation time
  • free features - Canvas, Maps, Uptime
  • hot-warm-cold and custom topologies
  • official GKE support
  • free tier

DISADVANTAGES

  • it is not as good at being a data store as some other options like MongoDB, Hadoop, etc. For smaller use cases, it will perform fine. If you are streaming TB’s of data every day, you will find that it either chokes or loses data
  • it’s learning curve is much steeper
  • when you can’t or won’t create a production-worthy setup because of economics. For test and dev, a single node will work fine. When you move to production, you should have no less than a 3-node/2-replica

More information you can find here: ECK.

2. Elastic Stack Kubernetes Helm Charts:

ADVANTAGES

  • huge community
  • easy to deploy and use in Kubernetes
  • each component in the stack takes care of a different step in the logging pipeline, and together, they all provide a comprehensive and powerful logging solution for Kubernetes
  • rich analysis capabilities

DISADVANTAGES

  • difficult to maintain at scale

More information you can find here: open-source-monitoring-tools-for-kubernetes.

-- MaggieO
Source: StackOverflow
12/6/2019

The main difference is that the Helm Charts are pretty unopinionated while the Operator is opinionated — it has a lot of best practices built in like a hard requirement on using security. Also the Operator Framework is built on the reconcilliation loop and will continuously check if your cluster is in the desired state or not. Helm Charts are more like a package manager where you run specific commands (install a cluster in version X with Y nodes, now add 2 more nodes, now upgrade to version Z,...).

If ECK is Cloud-on-Kubernetes, you can think of the Helm charts as Stack-on-Kubernetes. They're a way of defining exact specifications running our Docker images in a Kubernetes environment.

Another difference is that the Helm Charts are open source while the Operator is free, but uses the Elastic License (you can't use it to run a paid Elasticsearch service is the main limitation).

-- xeraa
Source: StackOverflow