I am trying to deploy a sample app using circleci to gke using google cloudrun. I created a cluster in google cloud and wanted to build and deploy an image to the container. It works perfectly if I do it manually. But I wanted an automated CI/CD pipeline to be built and hence using CircleCI to do it.
Skipping the testing and code coverage parts for now, I want to build a pipeline for gke deploymment
Here is the config.yaml file for circleci. I am trying to use the certified orbs already available, since creating one from scratch takes a longer time
version: 2.1
orbs:
gcp-gcr: circleci/gcp-gcr@0.6.1
cloudrun: circleci/gcp-cloud-run@1.0.2
executors:
node-executor:
docker:
- image: node:12.8.1-stretch
gcloud-executor:
docker:
- image: google/cloud-sdk
machine-executor:
machine: true
jobs:
build:
description: initial build
executor: machine-executor
steps:
- checkout
build_push_image_cloud_run_mangaged:
executor: node-executor
steps:
- checkout
- setup_remote_docker:
docker_layer_caching: false
- run:
name: Prepare env vars
command: |
echo 'export PATH=~$PATH:~/.local/bin' >> $BASH_ENV
echo 'export GOOGLE_PROJECT_ID=$GCLOUD_PROJECT' >> $BASH_ENV
echo 'export GOOGLE_COMPUTE_ZONE=us-east1-b' >> BASH_ENV
echo ${GCP_PROJECT_KEY} > ${HOME}/gcloud-service-key.json
echo 'export GOOGLE_CLOUD_KEYS=$(cat $HOME/gcloud-service-key.json)' >> $BASH_ENV
echo 'export TAG=${CIRCLE_SHA1}' >> $BASH_ENV
echo 'export IMAGE_NAME=$CIRCLE_PROJECT_REPONAME' >> $BASH_ENV && source $BASH_ENV
- gcp-gcr/gcr-auth:
gcloud-service-key: GOOGLE_CLOUD_KEYS # this is throwing error
google-project-id: GOOGLE_PROJECT_ID
google-compute-zone: GOOGLE_COMPUTE_ZONE
- gcp-gcr/build-image:
dockerfile: Dockerfile
google-project-id: GOOGLE_PROJECT_ID
image: $IMAGE_NAME
registry-url: "gcr.io"
tag: $CIRCLE_SHA1
- gcp-gcr/push-image:
google-project-id: GOOGLE_PROJECT_ID
image: $IMAGE_NAME
registry-url: "gcr.io"
tag: $CIRCLE_SHA1
- cloudrun/init:
gcloud-service-key: GCLOUD_SERVICE_KEY
google-project-id: GOOGLE_PROJECT_ID
google-compute-zone: GOOGLE_COMPUTE_ZONE
- cloudrun/deploy:
cluster: "new-cluster"
cluster-location: "us-east1-b"
platform: "gke"
image: "gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME"
service-name: "orb-gcp-cloud-run"
workflows:
build_gcloud_deploy:
jobs:
- build
- build_push_image_cloud_run_mangaged:
requires:
- build
I have the environment vars set in project settings with the GCLOUD_SERVICE_KEY, & GCP_PROJECT_KEY both having the encoded versions of my service account json file. I also have the GOOGLE_PROJECT_ID & GOOGLE_COMPUTE_ZONE env values set respectively. Now when I trigger the build for a checking (configured the webhook to execute for a sucessful checkin - will modify later for successful merges) , it always errors in the step: initialize gcloud
#!/bin/bash -eo pipefail
# Store service account
echo $GOOGLE_CLOUD_KEYS > ${HOME}/gcloud-service-key.json
# Initialize gcloud CLI
gcloud auth activate-service-account --key-file=${HOME}/gcloud-service-key.json
gcloud --quiet config set project $GOOGLE_PROJECT_ID
gcloud --quiet config set compute/zone $GOOGLE_COMPUTE_ZONE
ERROR: (gcloud.auth.activate-service-account) Could not read json file /root/gcloud-service-key.json: No JSON object could be decoded
Exited with code exit status 1
CircleCI received exit code 1
I tried using the GOOGLE_CLOUD_KEYS env var that I set in the gcloud-service-key.json variable in the circleci steps , but that too results in the same error. I also tried to specify an env variable that has the actual value of the json file (not decoded) but that too results in the same error. As you can see I use the orb: gcp-gcr: circleci/gcp-gcr@0.6.1 . Could you let me know what is causing the error and how to rectify it?
EDIT:
As Ahmet correctly pointed out , it was an issue with the file containing no data. I made changes such that I created an env variable for the project as GCLOUD_SERVICE_KEY and access it directly without encoding ( this is not recommned approach as it is best to encode it and then store the key).
As @AhmetB-Google pointed out , the issue was with the service key not being properly loaded into tne env vars. So I made changes like this. It is always advisable to encode and add it to the env vars. So in the project settings I have a key named -GCLOUD_SERVICE_KEY and here is my circleci configuration
version: 2.1
orbs:
gcp-gcr: circleci/gcp-gcr@0.6.1
cloudrun: circleci/gcp-cloud-run@1.0.1
gcp-gke: circleci/gcp-gke@0.2.0
executors:
gcloud-executor:
docker:
- image: google/cloud-sdk
machine-executor:
machine: true
jobs:
build:
description: initial build - Can make use of test coverage and tests
executor: machine-executor
steps:
- checkout
- run:
name: Test the source
command: |
echo "test"
- run:
name: Coverage report
command: |
echo "npm coverage"
build_push_image_gcr:
description: Build docker image and push to gcr registry
executor: machine-executor
steps:
- checkout
- run:
name: Prepare env vars
command: |
echo $GCLOUD_SERVICE_KEY > base64 --decode --ignore-garbage > ${HOME}/gcloud-service-key.json
echo $GCP_PROJECT_KEY > ./gcloud-service-key.json
cat ./gcloud-service-key.json
cat ${HOME}/gcloud-service-key.json
export $GCP_SERVICE_KEY=cat(${HOME}/gcloud-service-key.json)
pwd
- gcp-gcr/gcr-auth:
gcloud-service-key: GCLOUD_SERVICE_KEY
google-project-id: GOOGLE_PROJECT_ID
google-compute-zone: GOOGLE_COMPUTE_ZONE
- gcp-gcr/build-image:
dockerfile: Dockerfile
google-project-id: GOOGLE_PROJECT_ID
image: $IMAGE_NAME
registry-url: "gcr.io"
tag: $CIRCLE_SHA1
- gcp-gcr/push-image:
google-project-id: GOOGLE_PROJECT_ID
image: $IMAGE_NAME
registry-url: "gcr.io"
tag: $CIRCLE_SHA1
gcp_cloudrun_deploy:
description: Deploy using cloud run
executor: machine-executor
steps:
- cloudrun/init
- cloudrun/deploy:
cluster: 'new-cluster'
cluster-location: us-east1-b
platform: 'gke'
image: 'gcr.io/$GOOGLE_PROJECT_ID/$IMAGE_NAME:$CIRCLE_SHA1'
service-name: 'feedback-ui-service'
workflows:
build_gcloud_deploy:
jobs:
- build
- build_push_image_gcr:
requires:
- build
- gcp_cloudrun_deploy:
requires:
- build_push_image_gcr