Kubernetes CRD versioning

12/12/2018

Kuberentes has a mechanism for supporting versioning of CRDs. See https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definition-versioning/. What is not clear to me is how you actually support an evolution of CRD v1 to CRD v2 when you cannot always convert from v1 <-> v2. Suppose we introduce a new field in v2 that can not be populated by a web hook conversion, then perhaps all we can do is leave the field null? Furthermore when you request api version N you always get back an object as version N even if it was not written as version N so how can you controller know how to treat the object?

-- Tony Clarke
kubernetes

2 Answers

12/21/2018

As you can read in Writing, reading, and updating versioned CustomResourceDefinition objects

If you update an existing object, it is rewritten at the version that is currently the storage version. This is the only way that objects can change from one version to another.

Kubernetes returns the object to you at the version you requested, but the persisted object is neither changed on disk, nor converted in any way (other than changing the apiVersion string) while serving the request.

If you update an existing object, it is rewritten at the version that is currently the storage version. This is the only way that objects can change from one version to another.

You read your object at version v1beta1, then you read the object again at version v1. Both returned objects are identical except for the apiVersion field, Upgrade existing objects to a new stored version

The API server also supports webhook conversions that call an external service in case a conversion is required. The webhook handles the ConversionReview requests sent by the API servers, and sends back conversion results wrapped in ConversionResponse. You can read about Webbooks here.

Webhook conversion was introduced in Kubernetes v1.13 as an alpha feature. When the webhook server is deployed into the Kubernetes cluster as a service, it has to be exposed via a service on port 443. When deprecating versions and dropping support, devise a storage upgrade procedure.

-- Crou
Source: StackOverflow

7/12/2019

Let say you have CRD specification like below.

// API v6.
type Frobber struct {
  Height int    `json:"height"`
  Param  string `json:"param"`
}

Then you added some new spec called Width.

// Still API v6.
type Frobber struct {
  Height int    `json:"height"`
  Width  int    `json:"width"`
  Param  string `json:"param"`
}

So as long as you can handle this change using controller logic you do not need any version change. Which means this addition would be backward compatible.

But let say now you are changing the previous spec as below.

// Internal, soon to be v7beta1.
type Frobber struct {
  Height int
  Width  int
  Params []string
}

Here you change an existing spec which is not backward compatible. Also, you might not handle this change using controller logic. In these kinds of changes, it is better to use a version upgrade.

For more details about kubernetes API version changes, refer the following links.

[1] https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md

[2] https://groups.google.com/forum/#!topic/operator-framework/jswZUe1rlho

-- Buddhi
Source: StackOverflow