K
Q

How to set information in Kubernetes Network Policy dynamically?

5/31/2019

I have a few questions regarding Kubernetes: How to secure Kubernetes cluster?

My plan is to develop an application that secures Kubernetes cluster by default. I have written and tested a few Network Policy successfully. As a second step I want to set these information dynamically in my application based on cloud provider and so one.

1.) I want to block access the host network as well as the meta data services (my cluster runs on AWS):

  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.250.0.0/16     # host network
        - 169.254.169.254/32    # metadata service

Does anyone know how I can access the host network dynamically? I found an issue that says that you must use the Meta Data Service: https://github.com/kubernetes/kubernetes/issues/24657

Does anyone know how I can find out on which cloud provider I am currently running? Based on that information, I want to set the meta data service IP.

2.) I want to block access to the "kube-system" namespace:

  egress:
  - to:
    - podSelector:
        matchExpressions:
        - key: namespace
          operator: NotIn
          values:
          - kube-system

Does anyone know how I can enforce the actual denied access? As far as I understood, the key labled "namespace" is just a name that I choosed. How does Kubernetes know that I actually mean the namespace and nothing else?

3.) I want to block Internet access:

spec:
  podSelector: {}
  policyTypes:
  - Egress

Does anyone know, if something like the DNS server in the DMZ zone is still reachable?

4.) I want to block communication to pods with a different namespace:

  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          project: default

Here, I developed a controller that set the namespace dynamically.

-- Tobias1501
kubernetes
kubernetes-networkpolicy
security

Similar Questions

Can Ambassador handle CORS requests?
Kubectl get status of pod as yaml
What is the difference between |+ and |- when creating a configmap from a file in kubernetes yaml definitions?
Service account fails to delete resources, even though it should have rights to do so

2 Answers

6/5/2019

Your ideas are good in terms of a least-privilege policy but the implementation is problematic due to the following reasons.

  1. The logic you are trying to achieve it beyond the capabilities of Kubernetes network policies. It is very difficult to combine multiple block and allow policies in k8s without them conflicting with each other. For example, your first snippet allows access to any IP outside of the cluster and then your 3rd question is about blocking access to the internet - these two policies can't work simultaneously.

  2. You shouldn't block access to the kube-system namespace because that's where the k8s DNS service is deployed and blocking access to it will prevent all communications in the cluster.

  3. To answer your 1st question specifically:

    • How I can access the host network dynamically?
      The cluster subnet is defined when you deploy it on AWS - you should store it during creation and inject it to your policies. Alternatively, you may be able to get it by calling an AWS API. You can also get the cluster node IPs from Kubernetes: kubectl get nodes -o wide
    • How I can find out on which cloud provider I am currently running?
      Kubernetes doesn't know which platform it is running on, but you can guess it based on the node name prefix, for example: aks-nodepool1-18370774-0 or gke-...

  4. Your 4th point about blocking access between namespaces is good but it would be better to do it with an ingress policy like this:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Ingress

For more details, I recommend this blog post that explains the complexities of k8s network policies: https://medium.com/@reuvenharrison/an-introduction-to-kubernetes-network-policies-for-security-people-ba92dd4c809d

-- Mark
Source: StackOverflow
6/11/2019

As Mark pointed out, NP may not be able to address all your use cases. You might want to check out the Open Policy Agent project and there specifically the Gatekeeper tool, which could be utilized, to at least in part cover some of your needs.

-- Michael Hausenblas
Source: StackOverflow