I have a Java SE application which is nothing but a Kafka consumer. It is not exposing any REST endpoint but it polls data from Kafka topic and pushes to Elastic Search. This SE application is docker containerized and we need to run this to Kubernetes.
Some recommendations based on own experience, at my company all of our stuff is java and kafka based.
Java
1. You don't need a kubernetes service (obviously since you don't expose a server).
2. Set Heap Size
limits by making sure the container can read an environment variable for XMS
and XMS
.
3. Set spec.resources.limits.[]
and spec.resources.requests.[]
to values above XMX and XMS. Your container might need some extra resources for other operations.Kafka
4. Make sure to have a static consumer group
. So if you scale your application, it does not produce rebalances.
5. Create a variable to configure number of consumers per consumergroup. You can tune throughput
with it.