I'm currently exploring running an Istio / Kubernetes cluster on AWS using EKS. I would like to be able to assign a different IAM role to each service running in the cluster to limit the AWS privileges of each service.
In non-Istio Kubernetes clusters this facility is provided by projects such as kube2iam but this doesn't seem ideal in the Istio world as kube2iam relies on iptables rules and Istio is already using iptables rules to divert all outbound traffic to the Envoy sidecar.
The Istio security documentation says that identity model caters for different underlying implementations and on AWS that implementation is IAM:
In the Istio identity model, Istio uses the first-class service identity to determine the identity of a service. This gives great flexibility and granularity to represent a human user, an individual service, or a group of services. On platforms that do not have such identity available, Istio can use other identities that can group service instances, such as service names.
Istio service identities on different platforms:
Kubernetes: Kubernetes service account
GKE/GCE: may use GCP service account
GCP: GCP service account
AWS: AWS IAM user/role account
But I haven't come across any additional documentation about how to assign IAM roles to Istio ServiceRoles.
Has anyone found a solution to this?
Istio configuration allows to exclude some IP ranges from the proxy https://istio.io/docs/tasks/traffic-management/egress/egress-control/#direct-access-to-external-services
So you if you add global.proxy.excludeIPRanges: "169.254.169.254/32" to the istio configuration, all requests to AWS metadata IP will not be handled by istio but directly sent to this IP.
It will allow the kube2iam iptables rule to be applied.
cf. https://github.com/istio/istio/issues/9297#issuecomment-516353921
I'm also struggling with this and have found little help. I did have success with this persons suggestion https://groups.google.com/forum/m/#!topic/istio-users/3-fp2JPb2dQ
I was having no luck getting kube2iam working until I added that serviceentry (see below or follow link)
Basically you add this
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: apipa
spec:
hosts:
- 169.254.169.254
ports:
- number: 80
name: http
protocol: HTTP
resolution: DNS
location: MESH_EXTERNALFrom looking at the istio-proxy sidecar before applying the serviceentry you could lots of 404 errors in the log with paths all looking like aws api calls. After the service entry those turned to 200's.
UPDATE.... Later I found out that this is expected requirement when using istio for any external-mesh communication. See https://istio.io/docs/concepts/traffic-management/#service-entries