Kubernetes object size limitations

2/29/2020

I am dealing with CRDs and creating Custom resources. I need to keep lots of information about my application in the Custom resource. As per the official doc, etcd works with request up to 1.5MB. I am hitting errors something like

"error": "Request entity too large: limit is 3145728"

I believe the specified limit in the error is 3MB. Any thoughts around this? Any way out for this problem?

-- Yudi
etcd
google-kubernetes-engine
kubernetes

1 Answer

3/2/2020
  • The "error": "Request entity too large: limit is 3145728" is probably the default response from kubernetes handler for objects larger than 3MB, as you can see here at L305 of the source code:
expectedMsgFor1MB := `etcdserver: request is too large`
expectedMsgFor2MB := `rpc error: code = ResourceExhausted desc = trying to send message larger than max`
expectedMsgFor3MB := `Request entity too large: limit is 3145728`
expectedMsgForLargeAnnotation := `metadata.annotations: Too long: must have at most 262144 bytes`
  • The ETCD has indeed a 1.5MB limit for processing a file and you will find on ETCD Documentation a suggestion to try the--max-request-bytes flag but it would have no effect on a GKE cluster because you don't have such permission on master node.

  • But even if you did, it would not be ideal because usually this error means that you are consuming the objects instead of referencing them which would degrade your performance.

I highly recommend that you consider instead these options:

  • Determine whether your object includes references that aren't used;
  • Break up your resource;
  • Consider a volume mount instead;

There's a request for a new API Resource: File (orBinaryData) that could apply to your case. It's very fresh but it's good to keep an eye on.

If you still need help let me know.

-- willrof
Source: StackOverflow