Can't submit new job via gui on standalone kubernetes flink deployment (session mode)

9/1/2021

After deploy a flink in standalone kubernetes mode (session cluster) i can't upload any new job using flink GUI. After click +Add New button and choosing jar file, the progress strap ends and nothing happens. There is no information/error on Job Manager logs about this. When I try to upload any file (eg. text file) I get an error, and there is an info at the log:

"Exception occured in REST handler: Only Jar files are allowed."

I've also try to upload fake jar (an empty file called .jar) and it works - I can upload this kind of file.

I have a brand new, clean Apache Flink cluster running on Kubernetes cluster. I have used docker hub image and I've try two different versions:

*1.13.2-scala_2.12-java8, and
1.13-scala_2.11-java8*

But the result was the same on both versions.

My deployment are based on this howto:

https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/standalone/kubernetes/

and I've used yaml files provided in Appendix # Common cluster resource definitions # to this article:

flink-configuration-configmap.yaml
jobmanager-service.yaml
taskmanager-session-deployment.yaml
jobmanager-session-deployment-non-ha.yaml

I'e also used ingress controller to publish GUI running on 8081 on jobmanager.

I have tree pods (1 jobmanager, 2 task managers) and can't see any errors from flink logs.

Any suggestions what I'm missing, or when to find any errors ?

-- danthelos
apache-flink
deployment
kubernetes

1 Answer

9/2/2021

Problem solved. Problem was caused by nginx upload limit (default is 1024kb). Flink GUI are published outside Kubernetes using ingress controller and nginx. When we try to upload job files bigger than 1MB (1024kb), nginx limit prevented to do it. Jobs with size below this limit (for example fake jar with 0 kb size) was uploaded successfully

-- danthelos
Source: StackOverflow