Locality LoadBalacing not working on Istio

11/4/2019

We have a kubernetes cluster with ~100 nodes with istio and want to enable the Locality LoadBalancing feature. This will save us up to 70k USD/year because our interzone data traffic is too high.

I've followed the docs and setup the istio configmap like this:

 ...
  meshNetworks: {}


  localityLbSetting:
    enabled: true
    distribute:
    - from: us-east-1/us-east-1a/*
      to:
        "us-east-1/us-east-1a/*": 100
    - from: us-east-1/us-east-1b/*
      to:
        "us-east-1/us-east-1b/*": 100
...

And then deployed 2 apps, one of them just responds with the zone where the node is deployed (we are using a VirtualService) and the another one just do the requests.

The requests that are coming from node in us-east-1a should only be replied by the nodes in the same zone, right?

But it's not happening.

We also tried to set this variable inside pilot pods:

PILOT_ENABLE_LOCALITY_LOAD_BALANCING

When I get logs from one pod that is deployed in zone "us-east-1a" it shows replies from both zones.

Istio Version: 1.2.8

Kubernetes Version: 1.14

Any help is appreciated! Thank you!

-- panicked
istio
kubernetes

1 Answer

12/13/2019

I'm afraid your configuration is invalid in case of 'Locality' weights between regions/zones in context of 'Locality Load Balancing' feature in 'distribute' mode.

The logs of your istio-pilot should give you a clue about it, in the form of warning similar to this one:

<timestamp> warn    failed to read mesh configuration, using default: 1 error occurred:
    * locality weight must not be in range [1, 100]

I don't think you can find it documented anywhere in Istio documentation, but the logic behind the weights' validation can be found here.

-- Nepomucen
Source: StackOverflow