Lets say my company has 50 logs captured from the main server and distributed to many containers which has its own config files associated to, my question is how we can use Kubernetes Master & worker node together to maintain all these containers which should scale up if a new log come and creates a new container, and at the same time all the worker nodes should run only the specific container within and shouldn't replicate itself if one of the node goes down.
Sounds like distributed processing of a queue. Kubernetes API doesn't solve this. Use Kafka https://kafka.apache.org/intro. Depending on whether you need to spread the logs between all containers (consumers), or pass every log record thru every container, put consumers in the same or unique consumer groups correspondingly.
Horizontal Pod Autoscaler can be used to scale the number of containers depending on the length of the queue. Has been answered here: Kubernetes message consumer scalability