I am working on a custom kubernetes controller, which creates kubernetes secrets based off some inputs. As part of this controller it is required to copy these secrets to multiple namespaces. Which it successfully does.
For e.g.
CustomSecretGenerator
metadata:
names : mypersonalsecret
spec:
secret:
name: mysecret
namespace: default
target:
namespaces:
- dev
- personal
Above is a rough CRD of the controller input. I am able to copy secrets from one namespace to another with owner reference set to my Controller name.
My PROBLEM:
Here is watch code block
err = c.Watch(&source.Kind{Type: &corev1.Secret{}}, &handler.EnqueueRequestForOwner{
IsController: true,
OwnerType: &appv1alpha1.MyCustomController{},
})
and this is the metadata section for all the secrets that get generated with the controller
name: mysecret
namespace: dev
ownerReferences:
- apiVersion: com.company.app/v1alpha1
blockOwnerDeletion: true
controller: true
kind: MyCustomController
name: example-customer-controller
uid: ed2fa8c5-6855-11e9-94c6-0050569d445e
Also my controller is running in a default namespace and has role with all access to secrets.
EDIT:
This is the section of the code which creates the Secret.
mySecret := &corev1.Secret{
TypeMeta: meta_v1.TypeMeta{
APIVersion: "v1",
Kind: "Secret",
},
ObjectMeta: meta_v1.ObjectMeta{
Name: secretName,
Namespace: secretNamespace,
Labels: labelsforRegistryToken(crt.Name),
},
Data: map[string][]byte{
v1.DockerConfigJsonKey: jsonValue,
},
Type: corev1.SecretTypeDockerConfigJson,
}
controllerutil.SetControllerReference(crt, mySecret, r.scheme)
NOTE: crt
is the custom controller object which controls the creation of the object.
I tried copying the ownerrefence from the default/mysecret to dev/mysecret, the controller is still not able to detect the deletion of the dev/mysecret.
you need to add Owner References to the generated secrets like this.
func setOwnerRef(secret *corev1.Secret)error{
ownerRef := generateOwneRef(secret)
secret.SetOwnerRefrences(ownerRef)
return nil
}
func generateOwnerRef(secret *corev1.Secret) []metav1.OwnerReference {
return []metav1.OwnerReference{
*metav1.NewControllerRef(secret, schema.GroupVersionKind{
Group: appv1alpha1.SchemeGroupVersion.Group,
Version: appv1alpha1.SchemeGroupVersion.Version,
Kind: constants.ObjectKind,
}),
}
}
and then you check for owner reference type.
Since the controller wasn't able to detect the changes from the object in other namespace, I tried to scan all the namespaces in the cluster and had the reconcile loop end with
return reconcile.Result{RequeueAfter: time.Duration(15) * time.Minute}, nil
to allow reconciliation every 15 minutes. The controller does respond to the events to the secrets created in the same namespace as the controller is running.