Parse YAML manifests to Kubernetes []client.Object

9/21/2021

I have a YAML file defining multiple Kubernetes resources of various types (separated with --- according to the YAML spec):

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  # ...
spec:
  # ...
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # ...
rules:
  # ...
---
# etc

Now, I want to parse this list into a slice of client.Object instances, so I can apply some filtering and transforms, and eventually send them to the cluster using

myClient.Patch( # myClient is a client.Client instance
  ctx,
  object,       # object needs to be a client.Object
  client.Apply,
  client.ForceOwnership,
  client.FieldOwner("my.operator.acme.inc"),
)

However, I can't for the life of me figure out how to get from the YAML doc to []client.Object. The following gets me almost there:

results := make([]client.Object, 0)

scheme := runtime.NewScheme()
clientgoscheme.AddToScheme(scheme)
apiextensionsv1beta1.AddToScheme(scheme)
apiextensionsv1.AddToScheme(scheme)

decode := serializer.NewCodecFactory(scheme).UniversalDeserializer().Decode
data, err := ioutil.ReadAll(reader)
if err != nil {
	return nil, err
}
for _, doc := range strings.Split(string(data), "---") {
	object, gvk, err := decode([]byte(doc), nil, nil)
	if err != nil {
		return nil, err
	}

    // object is now a runtime.Object, and gvk is a schema.GroupVersionKind
    // taken together, they have all the information I need to expose a
    // client.Object (I think) but I have no idea how to actually construct a
    // type that implements that interface

    result = append(result, ?????)

}

return result, nil

I am totally open to other parser implementations, of course, but I haven't found anything that gets me any further. But this seems like it must be a solved problem in the Kubernetes world... so how do I do it?

-- Tomas Aschan
go
kubernetes

1 Answer

9/22/2021

I was finally able to make it work! Here's how:

import (
	"k8s.io/client-go/kubernetes/scheme"
	"sigs.k8s.io/controller-runtime/pkg/client"

	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
	apiextensionsv1beta1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1"
)

func deserialize(data []byte) (*client.Object, error) {
   	apiextensionsv1.AddToScheme(scheme.Scheme)
	apiextensionsv1beta1.AddToScheme(scheme.Scheme)
	decoder := scheme.Codecs.UniversalDeserializer()

	runtimeObject, groupVersionKind, err := decoder.Decode(data, nil, nil)
	if err != nil {
		return nil, err
	}

	return runtime
}

A couple of things that seem key (but I'm not sure my understanding is 100% correct here):

  • while the declared return type of decoder.Decode is (runtime.Object, *scheme.GroupVersionKind, error), the returned first item of that tuple is actually a client.Object and can be cast as such without problems.
  • By using scheme.Scheme as the baseline before adding the apiextensions.k8s.io groups, I get all the "standard" resources registered for free.
  • If I use scheme.Codecs.UniversalDecoder(), I get errors about no kind "CustomResourceDefinition" is registered for the internal version of group "apiextensions.k8s.io" in scheme "pkg/runtime/scheme.go:100", and the returned groupVersionKind instance shows __internal for version. No idea why this happens, or why it doesn't happen when I use the UniversalDeserializer() instead.
-- Tomas Aschan
Source: StackOverflow