K
Q

Question

argo workflow-controller can't connect to Kubernetes APIServer

8/15/2018

I have installed argo in my own namespace in a central kubernetes cluster in my organization.

After installation when argo "workflow-controller" tries to fetch the configmaps using the API server, I get timeout error.

time="2018-08-15T01:24:40Z" level=fatal msg="Get https://192.168.0.1:443/api/v1/namespaces/2304613691/configmaps/workflow-controller-configmap: dial tcp 192.168.0.1:443: i/o timeout\ngithub.com/argoproj/argo/errors.Wrap\n\t/root/go/src/github.com/argoproj/argo/errors/errors.go:87\ngithub.com/argoproj/argo/errors.InternalWrapError\n\t/root/go/src/github.com/argoproj/argo/errors/errors.go:70\ngithub.com/argoproj/argo/workflow/controller.(*WorkflowController).ResyncConfig\n\t/root/go/src/github.com/argoproj/argo/workflow/controller/controller.go:295\nmain.Run\n\t/root/go/src/github.com/argoproj/argo/cmd/workflow-controller/main.go:96\ngithub.com/argoproj/argo/vendor/github.com/spf13/cobra.(*Command).execute\n\t/root/go/src/github.com/argoproj/argo/vendor/github.com/spf13/cobra/command.go:750\ngithub.com/argoproj/argo/vendor/github.com/spf13/cobra.(*Command).ExecuteC\n\t/root/go/src/github.com/argoproj/argo/vendor/github.com/spf13/cobra/command.go:831\ngithub.com/argoproj/argo/vendor/github.com/spf13/cobra.(*Command).Execute\n\t/root/go/src/github.com/argoproj/argo/vendor/github.com/spf13/cobra/command.go:784\nmain.main\n\t/root/go/src/github.com/argoproj/argo/cmd/workflow-controller/main.go:68\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:195\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2337"

It is trying to access the follwing url: https://192.168.0.1:443/api/v1/namespaces/2304613691/configmaps/workflow-controller-configmap from within the pod container.

I have also modified the kubernetes host config to reflect kubernetes.default and added an open all ingress and egress network policy. But still the exception is there.

time="2018-08-16T18:23:55Z" level=fatal msg="Get https://kubernetes.default:443/api/v1/namespaces/2304613691/configmaps/workflow-controller-configmap: dial tcp: i/o timeout\ngithub.com/argoproj/argo/errors.Wrap\n\t/root/go/src/github.com/argoproj/argo/errors/errors.go:87\ngithub.com/argoproj/argo/errors.InternalWrapError\n\t/root/go/src/github.com/argoproj/argo/errors/errors.go:70\ngithub.com/argoproj/argo/workflow/controller.(*WorkflowController).ResyncConfig\n\t/root/go/src/github.com/argoproj/argo/workflow/controller/controller.go:295\nmain.Run\n\t/root/go/src/github.com/argoproj/argo/cmd/workflow-controller/main.go:96\ngithub.com/argoproj/argo/vendor/github.com/spf13/cobra.(*Command).execute\n\t/root/go/src/github.com/argoproj/argo/vendor/github.com/spf13/cobra/command.go:750\ngithub.com/argoproj/argo/vendor/github.com/spf13/cobra.(*Command).ExecuteC\n\t/root/go/src/github.com/argoproj/argo/vendor/github.com/spf13/cobra/command.go:831\ngithub.com/argoproj/argo/vendor/github.com/spf13/cobra.(*Command).Execute\n\t/root/go/src/github.com/argoproj/argo/vendor/github.com/spf13/cobra/command.go:784\nmain.main\n\t/root/go/src/github.com/argoproj/argo/cmd/workflow-controller/main.go:68\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:195\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2337"

apiVersion: v1
items:
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: argo
    namespace: 2304613691
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: argo-ui
    namespace: 2304613691
kind: List    
---
apiVersion: v1
items:
- apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    name: argo-role
    namespace: 2304613691
  rules:
  - apiGroups:
    - ""
    resources:
    - pods
    - pods/exec
    verbs:
    - create
    - get
    - list
    - watch
    - update
    - patch
  - apiGroups:
    - ""
    resources:
    - configmaps
    verbs:
    - get
    - watch
    - list
  - apiGroups:
    - ""
    resources:
    - persistentvolumeclaims
    verbs:
    - create
    - delete
  - apiGroups:
    - argoproj.io
    resources:
    - workflows
    verbs:
    - get
    - list
    - watch
    - update
    - patch
- apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    name: argo-ui-role
    namespace: 2304613691
  rules:
  - apiGroups:
    - ""
    resources:
    - pods
    - pods/exec
    - pods/log
    verbs:
    - get
    - list
    - watch
  - apiGroups:
    - ""
    resources:
    - secrets
    verbs:
    - get
  - apiGroups:
    - argoproj.io
    resources:
    - workflows
    verbs:
    - get
    - list
    - watch
kind: List    
---
apiVersion: v1
items:
- apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    name: argo-binding
    namespace: "2304613691"
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: argo-role
  subjects:
  - kind: ServiceAccount
    name: argo
    namespace: "2304613691"
- apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    name: argo-ui-binding
    namespace: "2304613691"
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: argo-ui-role
  subjects:
  - kind: ServiceAccount
    name: argo-ui
    namespace: "2304613691"
kind: List    
---
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    annotations:
      deployment.kubernetes.io/revision: "1"
    generation: 1
    name: workflow-controller
    namespace: 2304613691
  spec:
    progressDeadlineSeconds: 600
    replicas: 1
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app: workflow-controller
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: workflow-controller
      spec:
        containers:
        - args:
          - --configmap
          - workflow-controller-configmap
          command:
          - workflow-controller
          env:
          - name: ARGO_NAMESPACE
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.namespace
          image: <our repo>/sample-agupta34/workflow-controller:v2.1.1
          imagePullPolicy: IfNotPresent
          name: workflow-controller
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        serviceAccount: argo
        serviceAccountName: argo
        terminationGracePeriodSeconds: 30
- apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    annotations:
      deployment.kubernetes.io/revision: "1"
    generation: 1
    name: argo-ui
    namespace: 2304613691
  spec:
    progressDeadlineSeconds: 600
    replicas: 1
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app: argo-ui
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: argo-ui
      spec:
        containers:
        - env:
          - name: ARGO_NAMESPACE
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.namespace
          - name: IN_CLUSTER
            value: "true"
          - name: ENABLE_WEB_CONSOLE
            value: "false"
          - name: BASE_HREF
            value: /
          image: <our repo>/sample-agupta34/argoui:v2.1.1
          imagePullPolicy: IfNotPresent
          name: argo-ui
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        serviceAccount: argo-ui
        serviceAccountName: argo-ui
        terminationGracePeriodSeconds: 30
kind: List        
---
apiVersion: v1
data:
  config: |
    artifactRepository: {}
    executorImage: <our repo>/sample-agupta34/argoexec:v2.1.1
kind: ConfigMap
metadata:
  name: workflow-controller-configmap
  namespace: 2304613691
---
apiVersion: v1
kind: Service
metadata:
  name: argo-ui
  namespace: 2304613691
  labels:
    app: argo-ui
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8001
  selector:
    app: argo-ui
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name:  argo-ui
  namespace: 2304613691
  annotations:
    kubernetes.io/ingress.class: "netscaler.v2"
    netscaler.applecloud.io/insecure-backend: "true"
spec:
  backend:
    serviceName: argo-ui
    servicePort: 80
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: argo-and-argo-ui-netpol
spec:
  podSelector:
    matchLabels:
      app: workflow-controller
      app: argo-ui
  ingress:
  - {}
  egress:
  - {}

-- Ashu

argo-workflows

argoproj

kubernetes

kubernetes-apiserver

K
Q

argo workflow-controller can't connect to Kubernetes APIServer

Similar Questions

0 Answers

KQ

argo workflow-controller can't connect to Kubernetes APIServer

Similar Questions

0 Answers

K
Q