I set up datadog and kubernetes to test to out monitoring, although in datadog i can see some logs and metrics, in the agent in kubernetes I have the following errors:
TRACE ] trace-agent exited with code 0, disabling
[ AGENT ] 2018-10-17 08:18:24 UTC | WARN | (datadog_agent.go:149 in LogMessage) | (base.py:212) | DEPRECATION NOTICE: device_name is deprecated, please use a device: tag in the tags list instead
[ AGENT ] 2018-10-17 08:18:26 UTC | ERROR | (kubeutil.go:50 in GetKubeletConnectionInfo) | connection to kubelet failed: temporary failure in kubeutil, will retry later: try delay not elapsed yet
[ AGENT ] 2018-10-17 08:18:26 UTC | ERROR | (runner.go:289 in work) | Error running check kubelet: [{"message": "Unable to detect the kubelet URL automatically.", "traceback": "Traceback (most recent call last):\n File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/datadog_checks/checks/base.py", line 352, in run\n self.check(copy.deepcopy(self.instances[0]))\n File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/datadog_checks/kubelet/kubelet.py", line 107, in check\n raise CheckException("Unable to detect the kubelet URL automatically.")\nCheckException: Unable to detect the kubelet URL automatically.\n"}]
[ AGENT ] 2018-10-17 08:18:28 UTC | ERROR | (autoconfig.go:604 in collect) | Unable to collect configurations from provider Kubernetes: temporary failure in kubeutil, will retry later: try delay not elapsed yet
image:
repository: datadog/agent
tag: 6.4.2
As the logs state the agent cannot connect to Kubectl, has anyone come across this?
This might be a problem that other people are running into too. kubelet is no longer listening on the ReadOnlyPort in newer Kubernetes versions, and the port is being deprecated. Samuel Cormier-Iijima reports that the issue can be solved by adding adding KUBELET_EXTRA_ARGS=--read-only-port=10255
in /etc/default/kubelet
on the node host.