I am trying to upgrade the nodes in my Kubernetes cluster. When I go to do that, I get a notification saying:
PDB istio-ingressgateway in namespace istio-system allows 0 pod disruptions
PDB is Pod Disruption Budget. Basically, istio is saying that it can't loose that pod and keep things working right.
There is a really long discussion about this over on the Istio GitHub issues. This issue has been on going for over 2 years. Most of the discussions center around saying that the defaults are wrong. There are few workaround suggestions. But most of them are pre 1.4 (and the introduction of Istiod). The closest workaround I could find that might be compatible with current version is to add some additional replicas to the IstioOperator.
I tried that with a patch operation (run in PowerShell):
kubectl patch IstioOperator installed-state --patch $(Get-Content istio-ha-patch.yaml -Raw) --type=merge -n istio-system
Where istio-ha-patch.yaml
is:
spec:
components:
egressGateways:
- enabled: true
k8s:
hpaSpec:
minReplicas: 2
name: istio-egressgateway
ingressGateways:
- enabled: true
k8s:
hpaSpec:
minReplicas: 2
name: istio-ingressgateway
pilot:
enabled: true
k8s:
hpaSpec:
minReplicas: 2
I applied that, and checked the yaml of the IstioOperator, and it did apply to the resource's yaml. But the replica count for the ingress pod did not go up. (It stayed at 1 of 1.)
At this point, my only option is to uninstall Istio, apply my update then re-install Istio. (Yuck)
Is there anyway to get the replica count of Istio's ingress gateway up such that I can keep it running as I do a rolling node upgrade?
Turns out that if you did not install Istio using the Istio Kubernetes Operator, you cannot use the option I tried.
Once I uninstalled Istio and reinstalled it using the Operator, then I was able to get it to work.
Though I did not use the Patch operation, I just did a kubectl apply -f istio-operator-spec.yaml
where istio-operator-spec.yaml
is:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
name: istio-controlplane
namespace: istio-system
spec:
components:
ingressGateways:
- enabled: true
k8s:
hpaSpec:
minReplicas: 2
name: istio-ingressgateway
pilot:
enabled: true
k8s:
hpaSpec:
minReplicas: 2
profile: default