Https certificates and Kubernetes (EKS)

11/6/2018

I would like to secure my web application running on Kubernetes (EKS). All the nodes attached to the cluster are running on private subnets.

I have one front-end service and a dozen back-end services.

The front-end service is a pod running a container which is running on port 80. It is configured to be attached to an ELB which is only accepting traffic from 443 with an https certificate.

apiVersion: v1
kind: Service
metadata:
  name: service_name
  labels:
    app: service_name
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxxxxxxxxx
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
spec:
  ports:
    - port: 443 # Exposed port
      targetPort: 80 # Container port
  selector:
     app: service_name
  type: LoadBalancer

The back-end services are pods running containers also running on port 80. None of them have been configured to be accessible from outside the cluster. Back-end services talk to each other by pointing to http://service_name (NOT https) as I configured them with this template:

apiVersion: v1
kind: Service
metadata:
  name: service_name
spec:
  ports:
    - port: 80 # Exposed port
      targetPort: 80 # Container port
  selector:
     app: service_name

It all works but is it sufficient?

Should the front-end/back-end containers use certificate/443 too with a wildcard https certificate? Should this configuration be done inside the container or on the services' configurations?

-- Newalp
https
kubernetes
security

1 Answer

11/25/2018

I have done quite a bit of investigation now and here is what I came down to.

All my EKS EC2 instances are running on the private subnets which means they are not accessible from outside. Yes, by default Kubernetes does not encrypt traffic between pods which means that a hacker who gained access to my VPC (could be a rogue AWS engineer, someone who manages to physically access AWS data centers, someone who managed to access my AWS account...) will be able to sniff the network traffic. At the same time, I feel that in that instance the hacker will have access to much more! If he has access to my AWS account, he can download the https certificate himself for instance. If he manages to walk into an (high security) AWS data center and finds my server - it's good to compare the risk he has to take against the value of your data. If your data includes credit card/payments or sensitive personal data (date of birth, health details...), SSL encryption is a must. Anyway, to secure pods traffic, there are 2 options.

  1. Update all the pod source code and add the certificate there. It requires a lot of maintenance if you are running many pods and certificates expire every other year..
  2. Add an extra 'network layer' like https://istio.io/. This will add complexity to your cluster and in the case of EKS, support from AWS will be 'best effort'. Ideally, you would pay for Istio support.

For the load balancer, I decided to add an ingress to the cluster (Ngnix, Traefik...) and set it up with https. That's critical as the ELB sits on the public subnets.

-- Newalp
Source: StackOverflow