I use kubeadm to launch cluster on AWS. I can successfully create a load balancer on AWS by using kubectl, but the load balancer is not registered with any EC2 instances. That causes problem that the service cannot be accessed from public.
From the observation, when the ELB is created, it cannot find any healthy instances under all subnets. I am pretty sure I tag all my instances correctly.
Updated: I am reading the log from k8s-controller-manager, it shows my node does not have ProviderID set. And according to Github comment, ELB will ignore nodes where instance ID cannot be determined from provider. Could this cause the issue? How Should I set the providerID?
apiVersion: v1
kind: Service
metadata:
name: load-balancer
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "elb"
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
- name: https
port: 443
protocol: TCP
targetPort: 443
selector:
app: replica
type: LoadBalancer
apiVersion: apps/v1
kind: Deployment
metadata:
name: replica-deployment
labels:
app: replica
spec:
replicas: 1
selector:
matchLabels:
app: replica
template:
metadata:
labels:
app: replica
spec:
containers:
- name: web
image: web
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
- containerPort: 443
command: ["/bin/bash"]
args: ["-c", "script_to_start_server.sh"]
status
sectionstatus:
addresses:
- address: 172.31.35.209
type: InternalIP
- address: k8s
type: Hostname
allocatable:
cpu: "4"
ephemeral-storage: "119850776788"
hugepages-1Gi: "0"
hugepages-2Mi: "0"
memory: 16328856Ki
pods: "110"
capacity:
cpu: "4"
ephemeral-storage: 130046416Ki
hugepages-1Gi: "0"
hugepages-2Mi: "0"
memory: 16431256Ki
pods: "110"
conditions:
- lastHeartbeatTime: 2018-07-12T04:01:54Z
lastTransitionTime: 2018-07-11T15:45:06Z
message: kubelet has sufficient disk space available
reason: KubeletHasSufficientDisk
status: "False"
type: OutOfDisk
- lastHeartbeatTime: 2018-07-12T04:01:54Z
lastTransitionTime: 2018-07-11T15:45:06Z
message: kubelet has sufficient memory available
reason: KubeletHasSufficientMemory
status: "False"
type: MemoryPressure
- lastHeartbeatTime: 2018-07-12T04:01:54Z
lastTransitionTime: 2018-07-11T15:45:06Z
message: kubelet has no disk pressure
reason: KubeletHasNoDiskPressure
status: "False"
type: DiskPressure
- lastHeartbeatTime: 2018-07-12T04:01:54Z
lastTransitionTime: 2018-07-11T15:45:06Z
message: kubelet has sufficient PID available
reason: KubeletHasSufficientPID
status: "False"
type: PIDPressure
- lastHeartbeatTime: 2018-07-12T04:01:54Z
lastTransitionTime: 2018-07-11T15:45:06Z
message: kubelet is posting ready status. AppArmor enabled
reason: KubeletReady
status: "True"
type: Ready
How can I fix the issue?
Thanks!
In My case the issue was with the worker nodes not getting the providerId assigned properly.
I managed to patch the node like - kubectl patch node ip-xxxxx.ap-southeast-2.compute.internal -p '{"spec":{"providerID":"aws:///ap-southeast-2a/i-0xxxxx"}}'
to add the ProviderID. And then when i deployed the service . The ELB got created. the node group got added and end to end it worked. This is not a straight forward answer . But until i find a better solution let remain here
In my case - the problem was missing the option --cloud-provider=aws
After I placed the following in /etc/default/kubelet (via terraform in my case ) and redeployed my nodes, everything worked
/etc/default/kubelet
KUBELET_EXTRA_ARGS='--cloud-provider=aws'