Using below command to deploy spark app (default from spark operator).
kubectl apply -f spark/template/spark-application.yaml
used this to create spark-application
Added below values along with memory and core to run as user 1000 but sill driver and executor runs in root user.added security context from here
spec:
driver:
securityContext:
runAsUser: 1000
executor:
securityContext:
runAsUser: 1000
web-hook service is running in spark-operator
namespace.
kubectl get service -n spark-operator
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
spark-driver-metrics ClusterIP 172.20.134.198 <none> 8090/TCP 5h50m
spark-executor-metrics ClusterIP 172.20.91.165 <none> 8090/TCP 5h50m
spark-stream-1579262191507-driver-svc ClusterIP None <none> 7078/TCP,7079/TCP 20m
spark-stream-ui-svc NodePort 172.20.57.54 <none> 4040:31337/TCP 20m
spark-webhook ClusterIP 172.20.123.24 <none> 443/TCP 5h50m
secrets details:
kubectl get secrets -n spark-operator
NAME TYPE DATA AGE
default-token-2fgv7 kubernetes.io/service-account-token 3 6h13m
spark-token-74b8n kubernetes.io/service-account-token 3 6h13m
spark-webhook-certs Opaque
4 6h11m
any help appreciated.