I have an EKS cluster to which I've added support to work in hybrid mode (in other words, I've added Fargate profile to it). My intention is to run only specific workload on the AWS Fargate while keeping the EKS worker nodes for other kind of workload.

To test this out, my Fargate profile is defined to be:

• Restricted to specific namespace (Let's say: mynamespace)
• Has specific label so that pods need to match it in order to be scheduled on Fargate (Label is: fargate: myvalue)

For testing k8s resources, I'm trying to deploy simple nginx deployment which looks like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: mynamespace
  labels:
    fargate: myvalue
spec:
  selector:
    matchLabels:
      app: nginx
      version: 1.7.9
      fargate: myvalue
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
        version: 1.7.9
        fargate: myvalue
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

When I try to apply this resource, I get following:

$ kubectl get pods -n mynamespace -o wide
NAME                                                        READY   STATUS      RESTARTS   AGE     IP            NODE                          NOMINATED NODE                                READINESS GATES
nginx-deployment-596c594988-x9s6n                           0/1     Pending     0          10m     <none>        <none>                        07c651ad2b-7cf85d41b2424e529247def8bda7bf38   <none>

Pod stays in the Pending state and it is never scheduled to the AWS Fargate instances.

This is a pod describe output:

$ kubectl describe pod nginx-deployment-596c594988-x9s6n -n mynamespace
Name:               nginx-deployment-596c594988-x9s6n
Namespace:          mynamespace
Priority:           2000001000
PriorityClassName:  system-node-critical
Node:               <none>
Labels:             app=nginx
                    eks.amazonaws.com/fargate-profile=myprofile
                    fargate=myvalue
                    pod-template-hash=596c594988
                    version=1.7.9
Annotations:        kubernetes.io/psp: eks.privileged
Status:             Pending
IP:
Controlled By:      ReplicaSet/nginx-deployment-596c594988
NominatedNodeName:  9e418415bf-8259a43075714eb3ab77b08049d950a8
Containers:
  nginx:
    Image:        nginx:1.7.9
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-784d2 (ro)
Volumes:
  default-token-784d2:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-784d2
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

One thing that I can conclude from this output is that correct Fargate profile was chosen:

eks.amazonaws.com/fargate-profile=myprofile

Also, I see that some value is added to NOMINATED NODE field but not sure what it represents.

Any ideas or usual problems that happen and that might be worth troubleshooting in this case? Thanks