I am trying to write a network policy on Kubernetes that works under AWS EKS. What I want to achieve is to allow traffic to pod/pods from the same Namespace and allow external traffic that is forwarded from AWS ALB Ingress.
AWS ALB Ingress is created under the same NameSpace so I was thinking that only using DENY all traffic from other namespaces would suffice but when I use that traffic from ALB Ingress Load Balancer (whose internal IP addresses are at at the same nameSpace with the pod/pods) are not allowed. Then if I add ALLOW traffic from external clients it allows to Ingress but ALSO allows other namespaces too.
So my example is like: (this does not work as expected)
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-from-other-namespaces
namespace: os
spec:
podSelector:
matchLabels:
ingress:
- from:
- podSelector: {}
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: allow-external
namespace: os
spec:
podSelector:
matchLabels:
app: nginx
tier: prod
customer: os
ingress:
- ports:
- port: 80
from: []
When using first policy ALB Ingress is blocked, with adding second one other namespaces are also allowed too which i dont want. I can allow only internal IP address of AWS ALB Ingress but it can change over time and it is created dynamically.
By design (of Kubernetes NetworkPolicy API), if an endpoint accessible externally, it does not make sense to block it for other namespaces. (After all it can be accessed via the public LB from other namespaces, too, so it doesn't make sense to have an internal firewall for an endpoint that's already publicly accessible.) Back in the day when this API was being designed, this is what I was told.
However you might find that certain CNI plugins (Calico, Cilium etc) provide non-standard CRD APIs that have explicit “deny” operations that supersede “allow”s. They can solve your problem.
And finally, the answer depends on the CNI plugin implementation, how AWS does ALBs in terms of Kubernetes networking and how that CNI plugin deals with that. There’s no easy answer short of asking the CNI provider (or their docs).