I have added HPA support for my containers and they are scaling out and in as expected. But, I'm not sure about the internal state of the docker containers during the scaling.
Let's say I have an ongoing process going and number of replicas is 1. If the cpu usage goes above threshold, replicas scale out to 2 or 3. I understand the new replicas are ready to serve new requests, but what happens to the ongoing process ? Also, how would it be impacted in case of stateless and stateful process ?
Your ongoing process shouldn't be affected by the scaling operation, existing requests will keep being processed in the existing container, while new requests be routed to the new containers the cluster provisioned.
In the case of stateless processes the scaling shouldn't affect the response as the service doesn't hold any state in any of it's containers.
The case for stateful services is a lot more complicated and kinda goes beyond the scope of an SO question. You can examine the k8s statefulset object to see how they tackle this issue, and consider it for stateful processes running in a cluster. Check out the documentation for more info.