I am using the NetworkPolicy below to allow egress on HTTP and HTTPS ports, but running wget https://google.com
doesn't work when the network policy is applied. The domain name is resolved (DNS egress rule works) but connecting to the external host times out.
I've tried on minikube with cilium and on Azure with azure-npm in case it was some quirk with the network policy controller, but it behaves the same on both. I'm confused since I use the same method for DNS egress (which works) but this fails for other ports.
What's preventing egress on HTTP/HTTPS ports?
Kubernetes version 1.11.5
apiVersion: extensions/v1beta1
kind: NetworkPolicy
metadata:
name: my-netpolicy
spec:
egress:
- ports:
- port: 53
protocol: UDP
- port: 53
protocol: TCP
- ports:
- port: 443
protocol: UDP
- port: 443
protocol: TCP
- port: 80
protocol: UDP
- port: 80
protocol: TCP
podSelector:
matchLabels:
my-label: my-app
(Yes, the UDP rules are probably unnecessary, but trying everything here)
(I've also tried wget
on a private server in case Google/etc. block Azure IPs, same result)
(I've also tried matching ingress rules because "why not", same result)
kubectl describe
on the network policy:
Name: my-netpolicy
Namespace: default
Created on: 2019-01-21 19:00:04 +0000 UTC
Labels: ...
Annotations: <none>
Spec:
PodSelector: ...
Allowing ingress traffic:
To Port: 8080/TCP
From: <any> (traffic not restricted by source)
----------
To Port: https/UDP
To Port: https/TCP
To Port: http/TCP
To Port: http/UDP
From: <any> (traffic not restricted by source)
Allowing egress traffic:
To Port: 53/UDP
To Port: 53/TCP
To: <any> (traffic not restricted by source)
----------
To Port: https/UDP
To Port: https/TCP
To Port: http/UDP
To Port: http/TCP
To: <any> (traffic not restricted by source)
Policy Types: Ingress, Egress
Minimal reproducible example:
apiVersion: v1
kind: Pod
metadata:
name: netpolicy-poc-pod
labels:
name: netpolicy-poc-pod
spec:
containers:
- name: poc
image: ubuntu:18.04
command: ["bash", "-c", "while true; do sleep 1000; done"]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: netpolicy-poc
spec:
podSelector:
matchLabels:
name: netpolicy-poc-pod
egress:
- ports:
- port: 80
protocol: UDP
- port: 80
protocol: TCP
- port: 443
protocol: UDP
- port: 443
protocol: TCP
- port: 53
protocol: UDP
- port: 53
protocol: TCP
ingress: []
Then:
kubectl exec -it netpolicy-poc /bin/bash
apt update
apt install wget -y
wget https://google.com
Turns out the policy I gave works fine, it's just that the controllers implementing the policy had some bugs. On Minikube+Cilium it just didn't work for IPv6 but worked fine for IPv4, and on AKS the feature is still generally in beta and there are other options that we could try. I haven't found anything on my specific issue when using the azure-npm implementation but since it works fine in Minikube on IPv4 I'll assume that it would work fine in Azure as well once a "working" controller is set up.
Some resources I found for the Azure issue: