How can I delete the Argo events launched in the Argo namespace?

12/10/2021

I am trying to delete (and recreate) the Argo namespace, but it won't fully delete because I tried launching an eventsource and eventbus there. Now these will not delete.

I have tried to delete them via yaml and individually - no success yet.

The frustrating result is that I cannot re-launch argo

customresourcedefinition.apiextensions.k8s.io/clusterworkflowtemplates.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/cronworkflows.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workfloweventbindings.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflows.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowtasksets.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowtemplates.argoproj.io unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-admin unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-edit unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-view unchanged
clusterrole.rbac.authorization.k8s.io/argo-cluster-role unchanged
clusterrole.rbac.authorization.k8s.io/argo-server-cluster-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/argo-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/argo-server-binding unchanged
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": serviceaccounts "argo" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": serviceaccounts "argo-server" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": roles.rbac.authorization.k8s.io "argo-role" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": rolebindings.rbac.authorization.k8s.io "argo-binding" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": configmaps "workflow-controller-configmap" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": services "argo-server" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": services "workflow-controller-metrics" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": deployments.apps "argo-server" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": deployments.apps "workflow-controller" is forbidden: unable to create new content in namespace argo because it is being terminated

Here is what is going on in the argo namespace itself

{
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
        "creationTimestamp": "2021-10-21T14:52:51Z",
        "deletionTimestamp": "2021-12-10T18:48:17Z",
        "labels": {
            "kubernetes.io/metadata.name": "argo"
        },
        "name": "argo",
        "resourceVersion": "9222845",
        "uid": "2ce61352-74a5-40d1-b35e-fe1efa39c3af"
    },
    "spec": {
        "finalizers": [
            "kubernetes"
        ]
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2021-12-10T18:48:23Z",
                "message": "All resources successfully discovered",
                "reason": "ResourcesDiscovered",
                "status": "False",
                "type": "NamespaceDeletionDiscoveryFailure"
            },
            {
                "lastTransitionTime": "2021-12-10T18:48:23Z",
                "message": "All legacy kube types successfully parsed",
                "reason": "ParsedGroupVersions",
                "status": "False",
                "type": "NamespaceDeletionGroupVersionParsingFailure"
            },
            {
                "lastTransitionTime": "2021-12-10T18:48:23Z",
                "message": "All content successfully deleted, may be waiting on finalization",
                "reason": "ContentDeleted",
                "status": "False",
                "type": "NamespaceDeletionContentFailure"
            },
            {
                "lastTransitionTime": "2021-12-10T18:48:23Z",
                "message": "Some resources are remaining: eventbus.argoproj.io has 1 resource instances, eventsources.argoproj.io has 1 resource instances",
                "reason": "SomeResourcesRemain",
                "status": "True",
                "type": "NamespaceContentRemaining"
            },
            {
                "lastTransitionTime": "2021-12-10T18:48:23Z",
                "message": "Some content in the namespace has finalizers remaining: eventbus-controller in 1 resource instances, eventsource-controller in 1 resource instances",
                "reason": "SomeFinalizersRemain",
                "status": "True",
                "type": "NamespaceFinalizersRemaining"
            }
        ],
        "phase": "Terminating"
    }
}

UPDATE: Even though one test of Kubernetes shows that the eventsource persists, another test shows that it does not

This is the response to kubectl get eventsources argo

Error from server (NotFound): eventsources.argoproj.io "argo" not found
-- user3877654
argo
events
kubernetes
namespaces

1 Answer

12/12/2021

For anyone who stumbles onto this question, it is a permissions issue. Make certain your service account has permissions to work in both namespaces (argo and argo-events).

-- user3877654
Source: StackOverflow