Unable to connect to application on EKS via Istio with deny-all GlobalNetworkPolicy

3/13/2020

My setup: Running EKS 1.14 with Calico and Istio 1.5.

Trying to get the sample bookinfo to run with specific NetworkPolicies.

I've applied a GlobalNetworkPolicy that denies all traffic:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: default-deny
spec:
  selector: all()
  types:
  - Ingress
  - Egress

I also added a GlobalNetworkPolicy for Istio to namespace and intra-namespace traffic:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: allow-istio-system-to-ns
spec:
  selector: all()
  namespaceSelector: istio-injection == 'enabled'
  types:
  - Ingress
  - Egress
  ingress:
  - action: Allow
    source:
      selector: all()
      namespaceSelector: projectcalico.org/namespace == 'istio-system'
  - action: Allow
    source:
      selector: all()
      namespaceSelector: istio-injection == 'enabled'
  egress:
  - action: Allow

and a NetworkPolicy allowing all ingress and egress on istio-system

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-istio-system-all
  namespace: istio-system
spec:
  selector: all()
  types:
  - Ingress
  - Egress
  ingress:
  - action: Allow 
  egress:
  - action: Allow

And a NetworkPolicy to allow traffic to the ingress-gateway on ports 80 and 443. I know this one is redundant, but I was hoping to cut down the previous one to only necessary ingress.

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-istio-ingress-on-80-443
  namespace: istio-system
spec:
  selector: app == 'istio-ingressgateway'
  ingress:
    - action: Allow
      protocol: TCP
      destination:
        ports:
          - 80
          - 443

Also some other, unrelated NetworkPolicies (access to kube-dns, metrics-server). Deploying bookinfo works with and without the policies and same with deploying the Gateway.

The connection between the components works (I can exec into one pod and connect to the others). But when I try to access the productpage via the gateway with the policies I get nothing back, instead of the HTML I get without the policies.

Am I missing some traffic that should be allowed? Should I have policies for the master nodes or for the loadbalancer somewhere else?

Edit: If I allow all ingress into istio-system and into the namespace, it works. So I think I'm just missing some other ingress from the namespace, is there a way to limit it to just the loadbalancer?

-- char
aws-eks
istio
kubernetes-networkpolicy

2 Answers

3/13/2020

First of all there is a typo in Your allow-istio-system-to-ns yaml:

      namespaceSelector: projectcalico.org/namespace == 'istio-system

There should be another ' at the end of the line.


Secondly this could be caused by the changes to policy and mixer in istio version 1.5.

According to Istio documentation:

The mixer policy is deprecated in Istio 1.5 and not recommended for production usage.

There is a guide in istio documentation which allows to turn back on depreciated features:

For an existing Istio mesh

  1. Check the status of policy enforcement for your mesh.
$ kubectl -n istio-system get cm istio -o jsonpath="{@.data.mesh}" | grep disablePolicyChecks
disablePolicyChecks: true

If policy enforcement is enabled (disablePolicyChecks is false), no further action is needed.

  1. Update the istio configuration to enable policy checks.

Execute the following command from the root Istio directory:

$ istioctl manifest apply --set values.global.disablePolicyChecks=false --set values.pilot.policy.enabled=true
configuration "istio" replaced
  1. Validate that policy enforcement is now enabled.
$ kubectl -n istio-system get cm istio -o jsonpath="{@.data.mesh}" | grep disablePolicyChecks
disablePolicyChecks: false

Note that on calico documentation it says it requires the following istio versions:

Istio v1.0, v1.1, v1.2, or v1.3

Hope it helps.

-- Piotr Malec
Source: StackOverflow

3/16/2020

For some reason it seems to work when I replace projectcalico.org/namespace == 'istio-system' with a label from istio-system (e.g. istio-injection=disabled).

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: allow-istio-system-to-ns
spec:
  selector: all()
  namespaceSelector: istio-injection == 'enabled'
  types:
  - Ingress
  - Egress
  ingress:
  - action: Allow
    source:
      selector: all()
      namespaceSelector: istio-injection == 'disabled'  
  - action: Allow
    source:
      selector: all()
      namespaceSelector: istio-injection == 'enabled'
  egress:
  - action: Allow
-- char
Source: StackOverflow