I am running k8s v1.17.4 and 1.14.2 with integration with cinder using in-tree openstack cloud provider.
I received a report from openstack team that they are receiving a huge number of GET cinder volume info requests (like 18 request per volume per minute). (volumes are linked to pv/pvc)
I checked out my controller manager and kubelet in the hole cluster but i did not find any abnormal requests outgoing to the openstack API.
I would like to know what this kind of requests and if it is an expected behavior or not. if yes, is there any manner to reduce this number of requests ?
This is a very critical issue for me, so any feedback will be more than welcome
please find below some of the reported logs from cloud team:
[18/May/2020:08:50:07 +0300] "GET /v3/e740134000244afea2710df1f624b098/volumes/c495eef3-671d-4ece-b460-bc616ed43d44 HTTP/1.1" 200 1505 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/38671dda-72d2-471b-8f68-3e563c68d3ab HTTP/1.1" 200 1506 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/6a3426c0-2ec6-4180-9e6b-828a9a4f0b68 HTTP/1.1" 200 1499 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/867991d7-5176-4686-9999-d8405dc91911 HTTP/1.1" 200 1503 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/2a209f46-9ae1-4604-8b69-edf541f8f00b HTTP/1.1" 200 1506 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/ed28da8e-ccf4-4d9a-94d6-96ac9c3b9939 HTTP/1.1" 200 1516 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/68733711-79f0-4799-88ec-1419702cae31 HTTP/1.1" 200 1504 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/48cbd610-ba05-4fbe-9ccd-2de1a5cfcdbc HTTP/1.1" 200 1504 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/3f77bbaa-713a-40fc-80b4-36e32aafb252 HTTP/1.1" 200 1497 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/297918ee-273b-4ff7-9a70-40afd00a2cde HTTP/1.1" 200 1498 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/53f844c2-0c2b-4f97-92bf-4396669a246a HTTP/1.1" 200 1496 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/26299e1f-dddf-4293-bfc5-2db21fbf5344 HTTP/1.1" 200 1507 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]
"GET /v3/e740134000244afea2710df1f624b098/volumes/acc08122-ff8e-4ea5-b7b9-9fb357718e00 HTTP/1.1" 200 1491 "-" "gophercloud/2.0.0" [18/May/2020:08:50:07 +0300]