How to get k8s master logs on EKS?

6/26/2018

I am looking for these logs:

/var/log/kube-apiserver.log
/var/log/kube-scheduler.log
/var/log/kube-controller-manager.log

In EKS user does not have access to the control plane and can't see these files directly.

I am aware of CloudTrail integration announced by AWS. But it shows events not from k8s API, but AWS EKS API like CreateCluster event. Also the open question how to get scheduler and controller manager logs.

There is no pods for api and controller in pods list.

$ kubectl get po --all-namespaces
NAMESPACE     NAME                             READY     STATUS             RESTARTS   AGE
kube-system   aws-node-9f4lm                   1/1       Running            0          2h
kube-system   aws-node-wj2cg                   1/1       Running            0          2h
kube-system   kube-dns-64b69465b4-4gw6n        3/3       Running            0          2h
kube-system   kube-proxy-7mt7l                 1/1       Running            0          2h
kube-system   kube-proxy-vflzv                 1/1       Running            0          2h

There is no master nodes in the node list

$ kubectl get nodes
NAME                        STATUS    ROLES     AGE       VERSION
ip-10-0-0-92.ec2.internal   Ready     <none>    9m        v1.10.3
ip-10-0-1-63.ec2.internal   Ready     <none>    9m        v1.10.3
-- lexsys
amazon-eks
amazon-web-services
kubernetes

2 Answers

6/27/2018

Things like kube-api server logs, the kube-scheduler logs, the kube-controller manager logs, etc. will be available in CloudWatch logs. While (as you have stated) EKS API calls will be logged to cloudtrail.

**I take that back, I guess AWS EKS has not gotten around to that yet. You will need to use an EFK stack to get the logs.

Someone has already put together a quick how-to: https://github.com/aws-samples/aws-workshop-for-kubernetes/tree/master/02-path-working-with-clusters/204-cluster-logging-with-EFK

-- DojoHighCat
Source: StackOverflow

11/5/2019

Logs can be send to CloudWatch (not free of charge). The following logs can be individually selected to be send to CloudWatch:

  • API server
  • Audit
  • Authenticator
  • Controller Manager
  • Scheduler

Logs can be enabled via UI or AWS CLI. See Amazon EKS Control Plane Logging

-- Frank Legler
Source: StackOverflow