Storage plugin configuration on Zookeeper for Apache Drill + Zookeeper in Kubernetes cluster

9/19/2018

I am running Apache Drill and Zookeeper on a Kubernetes cluster.

Drill is connecting to zookeeper through a zookeeper-service running on port 2181. I am trying the persist storage plugin configuration on zookeeper. On the Apache Drill docs (https://drill.apache.org/docs/persistent-configuration-storage/), it is given that sys.store.provider.zk.blobroot key needs to be added to drill-override.conf property. But I am not able to figure out a value for this key if I want to connect it to Zookeeper service in Kubernetes.

-- Jyotsana Walia
apache-drill
apache-zookeeper
kubernetes

2 Answers

9/20/2018

This is an optional config. You can specify it to modify where the ZooKeeper PStore provider offloads query profile data [1] or you can remove this property from your drill-override.conf and restart drillbits.

[1] http://doc.mapr.com/display/MapR/Persistent+Configuration+Storage

-- Vitalii Diravka
Source: StackOverflow

9/19/2018

The value should be:

<name-of-your-zk-service>.<namespace-where-zk-is-running>.svc.cluster.local:2181

That's how services get resolved internally in Kubernetes. You can always test it by creating a Pod, connecting to is using kubectl exec -it <pod-name> sh, and running:

ping <name-of-your-zk-service>.<namespace-where-zk-is-running>.svc.cluster.local

Hope it helps!

-- Rico
Source: StackOverflow