helm upgrade: failed to create patch on google container builder

4/28/2018

What I am trying to do

I am trying to deploy an upgrade to an helm chart on my k8s cluster (GKE) using a CI/CD pipeline (google container builder) but I get an error.

What I am using

Google Cloud SDK 195.0.0

GKE Cluster version: v1.9.6-gke.0

kubectl v1.8.6

I am using this builder: https://github.com/GoogleCloudPlatform/cloud-builders-community/tree/master/helm

I am triggering the build like this:

gcloud container builds submit . --config=cloudbuild.yaml --substitutions=TAG_NAME=v0.1.8

Here's the section of the cloudbuild.yaml that is causing this error:

- name: 'gcr.io/$PROJECT_ID/helm'
  args:
  - upgrade
  - mmh-user
  - ./mmh-users-0.1.7.tgz
  - --reuse-values
  - --set
  - image.tag=${TAG_NAME}
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=northamerica-northeast1-a'
  - 'CLOUDSDK_CONTAINER_CLUSTER=mycluster'

The error I am getting on that step:

Starting Step #17
Step #17: Already have image (with digest): gcr.io/myproject-2018/helm
Step #17: Running: helm init --client-only
Step #17: $HELM_HOME has been configured at /builder/home/.helm.
Step #17: Not installing Tiller due to 'client-only' flag having been set
Step #17: Happy Helming!
Step #17: Running: helm repo update
Step #17: Hang tight while we grab the latest from your chart repositories...
Step #17: ...Skip local chart repository
Step #17: ...Successfully got an update from the "stable" chart repository
Step #17: Update Complete. ⎈ Happy Helming!⎈
Step #17: Running: helm upgrade mmh-user ./mmh-users-0.1.7.tgz --reuse-values --set image.tag=v0.1.8
Step #17: Error: UPGRADE FAILED: failed to create patch: failed to get versionedObject: unable to convert unstructured object to extensions/v1beta1, Kind=Ingress: unrecognized type: string
Finished Step #17

What I tried to do to find a cause/solution:

  • When I run this command (helm upgrade mmh-user ./mmh-users-0.1.7.tgz --reuse-values --set image.tag=v0.1.8) from my laptop, no issues.
  • I tried reducing the cloudbuild.yaml to only contains the steps necessary for the helm upgrade, same error.
  • The ci/cd can run helm lint and package steps just fine. Only the upgrade step fails.
  • I checked the Helm version 2.8.2 on both my laptop and the ci/cd.
  • When I use --debug --dry-run it does not report errors and shows what appears to be valid yaml, even near the ingress

Any clues as to what else I could do to find a cause/solution?

-- roychri
google-cloud-platform
google-kubernetes-engine
kubernetes
kubernetes-helm

1 Answer

5/1/2018

Thanks to juanchimienti from the Kubernetes #helm-users slack channel, I was able to solve this problem. Juanchimienti suggested that it might have something to do with the tls-acme annotation's value that was showing as:

kubernetes.io/tls-acme: true

but should have been shown as "true".

I changed my build step to include the annotation like this:

  - --set
  - 'ingress.annotations.kubernetes\.io/tls-acme="true"'

and now its working.

I presume that this has something to do with the version of kubectl that is in the helm builder (https://github.com/GoogleCloudPlatform/cloud-builders-community/tree/master/helm) because running the same command without the annotation from my laptop was working just fine. My laptop had a more recent version of kubectl. I didn't confirm this.

-- roychri
Source: StackOverflow