Horizontal Autoscaling

• Pros

  • Likely to have more capacity since you are expanding VMs or/and servers. You are essentially expanding your cluster.
  • In theory, more redundancy since you are spreading your workloads across different physical servers.

• Cons

  • In theory, it's slower. Meaning it's slower to provision servers and VMs than pods/containers in the same machine (for vertical autoscaling)
  • Also, you need to provision both servers/VMs and containers/pods when you scale up.
  • Doesn't work that well with plain bare-metal infrastructure/servers.

Vertical Autoscaling

• Pros

  • In theory, it should be faster to autoscale if you have large servers provisioned. (Also, faster response)
  • If you have data-intensive apps you might benefit from workloads running on the same machines.
  • Great if you have a few extra unused bare-metal servers.

• Cons

  • If you have large servers provisioned you may waste a lot of resources.
  • You need to calculate the capacity of your workloads more precisely (this could be a pro or a con depending on how you see it)
  • If you have a fixed set of physical servers, you will run into eventual limitations of CPUs, Storage, Memory, etc.

Generally, you'd want to have a combination of both Horizontal and Vertical autoscaling.

In general I would recommend larger nodes because it's easier to place containers on them.

If you have a pod that resources: {requests: {cpu: 2.5}}, you can only place one of them on a 4-core node, and two on 2x 4-core nodes, but you can put 3 on a single 8-core node.

+----+----+----+----+    +----+----+----+----+
|-WORKLOAD--|       |    |-WORKLOAD--|       |
+----+----+----+----+    +----+----+----+----+

+----+----+----+----+----+----+----+----+
|-WORKLOAD--|--WORKLOAD--|-WORKLOAD--|  |
+----+----+----+----+----+----+----+----+

If you have 16 cores total and 8 cores allocated, it's possible that no single node has more than 2 cores free with 4x 4-CPU nodes, but you're guaranteed to be able to fit that pod with 2x 8-CPU nodes.

+----+----+----+----+    +----+----+----+----+
|-- USED -|         |    |-- USED -|         |
+----+----+----+----+    +----+----+----+----+
+----+----+----+----+    +----+----+----+----+
|-- USED -|         |    |-- USED -|         |
+----+----+----+----+    +----+----+----+----+

Where   |-WORKLOAD--| goes?

+----+----+----+----+----+----+----+----+
|------- USED ------|                   |
+----+----+----+----+----+----+----+----+
+----+----+----+----+----+----+----+----+
|------- USED ------|                   |
+----+----+----+----+----+----+----+----+

At the specific scale you're talking about, though, I'd be a little worried about running a 2-node cluster: if a single node dies you've lost half your cluster capacity. Unless I knew that I was running multiple pods that needed 2.0 CPU or more I might lean towards the 4-node setup here so that it will be more resilient in the event of node failure (and that does happen in reality).