I have a deamonset which defines
ports:
- containerPort: 24231
in spec.
After deploying a set I can curl /metrics
endpoint on localhost(in container) and can curl the endpoint of one container from another container of the same set.
Then I define a service like this:
apiVersion: v1
kind: Service
metadata:
name: fluentd
labels:
app: fluentd
spec:
ports:
- name: prometheus
port: 80
targetPort: 24231
selector:
app: fluentd
I expect that this will allow me to: kubectl proxy --port=8001
And after following the link: http://127.0.0.1:8001/api/v1/namespaces/default/services/fluentd:prometheus/proxy/metrics
What I see is this:
Error: 'dial tcp 10.136.4.30:24231: i/o timeout'
Trying to reach: 'http://10.136.4.30:24231/metrics'
But http://127.0.0.1:8001/api/v1/namespaces/default/services/fluentd
returns me service information as expected which includes:
"spec": {
"ports": [
{
"name": "prometheus",
"protocol": "TCP",
"port": 80,
"targetPort": 24231
What am I doing wrong. This service has worked a week ago, but stopped working after redeployment.
There are few things you should check:
kubectl port-forward
instead of proxy - if the points above did not help. More about it here.I have listed my suggestion from the most probable one to the least. Please let me know if that helped.
This was a GKE issue that I had to resolve with their support.