I have a private gke cluster with multiple node pools, and reading the documentation it seems if I create a Cloud NAT its only possible to do for the whole cluster. Is there a way to configure only a subset of the cluster to have access to send traffic through the NAT? I poked around and it doesn't seem to be possible to assign a network, IP range or subnetwork to a node pool, so I am not sure if this is possible.
First we have to outline of what you will need to setup.
Components:
1.) NAT Gateway Instance
2.) Network Tags and VPC Routes
3.) Node Pools with Network Tags.
For 1 and 2, you'd need to configure another GCE VM instance as a NAT gateway in a similar fashion as shown in our GCE NAT tutorial. This should lead to the creation of a NAT instance and a VPC route that directs network traffic from tagged resources towards the NAT.
Make sure to use tags specific to the node-pools that will be using the instance as a NAT.
For 3. In order to point the node pools to the NAT instance, you must use the correct network tags on the nodes such that traffic from these nodes are routed correctly.
Node/Nodepool with tags -> VPC Route -> NAT Instance -> Internet
In order to apply this to an entire Node Pool, you'd need to apply tags at the Node Pool level so it gets cascaded onto the underlying nodes you can see how to do it here. Unfortunately this isn't possible for existing Node Pools so you will have to delete and recreate your existing Node Pools with the corresponding tags such as below:
gcloud container node-pools create Proxy --cluster=prod-cluster --tags=NatRouteA
You could check old nat gw solution from Google, VM instance-based instead of Cloud NAT, as an example. Possible way to do what you need is following:
Referenced nat gw solution uses similar technique, but it applies route to all node pools using whole cluster tag instead of separate node pool tag.