You can see more use cases here

http://alesnosek.com/blog/2017/02/14/accessing-kubernetes-pods-from-outside-of-the-cluster/

Two containers using the same hostPort cannot be scheduled on the same node and the usage of the hostPort is considered a privileged. Hence it has some limitations like

1) no. of replicas should not be more than no. of nodes which will exhaust hostports :)

2) all hosts must be in healthy state so that schedular can schedule pods on each of them.

Hope it help you.....

You can do that by using Deployment/ReplicaSet in Kubernetes with hostPort.

Assuming you have 4 nodes in Kubernetes cluster, you can create a deployment or replicaset with hostPort and replicas equal to number of nodes in cluster.

For example you want to run nginx pod on every node with clustersize equal to 4 then you have mention hostport to container port in deployment/replicaset definition. The kubernetes scheduler will be unable to schedule more than 1 pod on same host and in this way all nodes have at least one pod scheduled.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-hello
  labels:  
    tier: frontend
    app: nginx-hello
spec:
  replicas: 4
  template:
    metadata:
      labels:
        tier: frontend
        app: nginx-hello
    spec:
      containers:
      - name: nginx-hello
        image: nginxdemos/hello
        ports:
        - containerPort: 80
          hostPort: 8088