How to read in YAMLs for object creation in a Kubernetes Go Operator, instead of hardcoding?

8/6/2019

I'm working on a custom Kubernetes Operator in Golang, as a shift away from an Operator generated based on existing Helm Charts. The way you can create a Pod (or Job in the example I will share) is like this, as I understand it:

func returnJob(cr *myApi.MyCRObject) *batchv1.Job {
    return &batchv1.Job{
        ObjectMeta: metav1.ObjectMeta{
            Name: cr.Name,
            Namespace: cr.Namespace,
        },
        Spec: batchv1.JobSpec{
            BackOffLimit: 0,
            Template: apiv1.PodTemplateSpec{
                ObjectMeta: metav1.ObjectMeta{
                    Name: cr.Name
                },
                Spec: apiv1.PodSpec{
                    RestartPolicy: "Never",
                    ServiceAccountName: "",
                    SecurityContext: apiv1.SecurityContext{
                        RunAsUser: "",
                        FsGroup: ""
                    },
                    ImagePullSecrets: apiv1.ImagePullSecrets{
                        Name: ""
                    },
                    Volumes: []apiv1.Volumes{
                        {
                            Name: "config",
                            ConfigMap: apiv1.ConfigMap{
                                Name: cr.Name + "-config"
                            }
                        }
                    },
                    Containers: []apiv1.Container{
                        {
                            Name:  "container",
                            Image: "prefix/image:tag",
                            ImagePullPolicy: "Always",
                            WorkingDir: "installdir",
                            SecurityContext: apiv1.SecurityContext{
                                RunAsUser: "",
                                FsGroup: ""
                            },
                            Env: []apiv1.Env{
                                {
                                    Name: "KEY",
                                    Value: "VAL"
                                },
                                {
                                    Name: "NAME",
                                    Value: cr.Name
                                }
                            }
                            Command: []string{
                                "mycommand",
                                "-x"
                            },
                            Resources: apiv1.Resources{
                                Requests: apiv1.Requests{
                                    Cpu: 1,
                                    Memory: "1G"
                                },
                                Limits: apiv1.Limits{
                                    Cpu: 1,
                                    Memory: "1G"
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Then elsewhere in the Reconcile call, the job is actually created on Kubernetes:

    job := returnJob(instance)

    // Set instance as the owner and controller
    if err := controllerutil.SetControllerReference(instance, job, r.scheme); err != nil {
        return reconcile.Result{}, err
    }

    // Check if this Job already exists
    found := &batchv1.Job{}
    err = r.client.Get(context.TODO(), types.NamespacedName{Name: job.Name, Namespace: job.Namespace}, found)
    if err != nil && errors.IsNotFound(err) {
        reqLogger.Info("Creating a new Job", "Job.Namespace", job.Namespace, "Job.Name", job.Name)
        err = r.client.Create(context.TODO(), job)
        if err != nil {
            return reconcile.Result{}, err
        }

        // Job created successfully - don't requeue
        return reconcile.Result{}, nil
    } else if err != nil {
        return reconcile.Result{}, err
    }

This is... very clearly not optimal. I can't see a great way to fill in things such as the "Image" or to allow overrides for "Resources" etc., and that's all second to it being a massive hardcoded blob (I removed identifying info and about 120 lines from the specification above, things like additional volumes and other containers).

I'd like to just be able to pull from an existing YAML file in the manifest (what was previously used with Helm Charts and Helm Operators) and return that in the format Go needs from a method such as this.

I've been scouring the internet for something like this, but there just aren't a lot of Go or k8s Operators resources out there yet, so I figured I'd ask here.

Any ideas or solutions you guys have found?

-- Michael Conard
go
kubernetes
kubernetes-helm
kubernetes-operator
operator-sdk

0 Answers