K
Q

Clarification on where testing falls in the pipeline

2/25/2021

I was reading something that made me question if I'm implementing testing in my pipeline correctly.

I currently have it setup like the following:

  • Local feature branch for a microservice pushed to remote
  • PR request to merge feature request into testing branch for microservice
  • Accepted and merged
  • Triggers CI/CD pipeline:
    • First, a Linux VM is created
    • The microservice is deployed to the VM
    • The tests are run for the microservice
    • If all testing passes, it builds the Docker image and pushes it to the container registry
    • The deployment stage then pulls the image and deploys to the Kubernetes cluster

My questions are the following:

  • The dev should be testing locally, but should testing automatically happen when the PR is submitted?
  • Should I be building the Docker image first, creating a running container of the image and testing that (that is after all what is getting deployed)?

In other words, is the flow:

  1. Build app -> test -> if passes, build image -> deploy
  2. Build image -> test -> if passes, deploy
  3. Build app -> run certain tests -> if passes, build image -> run more tests -> deploy

Thanks for clarifying.

-- cjones
azure-pipelines
docker
kubernetes

Similar Questions

where is bin/elasticsearch on kuberenetes
What does the following deployment strategy imply in kubernetes?
get PreStop lifecycle hook execution results
Kubernetes ingress service NodePort is only listen to tcp6/ipv6 not tcp/ipv4
what are the Kubernetes modules directly communicating with etcd

1 Answer

2/25/2021

There are a couple of things I'd rearrange here.

As @DanielMann notes in a comment, your service should have a substantial set of unit tests. These don't require running in a container and don't have dependencies on any external resources; you might be able to use mock implementations of service clients, or alternate implementations like SQLite as an in-process database. Once your CI system sees a commit, before it builds an image, it should run these unit tests.

You should then run as much of the test sequence as you can against the pull request. (If the integration tests fail after you've merged to the main branch, your build is broken, and that can be problematic if you need an emergency fix.) Especially you can build a Docker image from the PR branch. Depending on how your system is set up you may be able to test that directly; either with an alternate plain-Docker environment, or in a dedicated Kubernetes namespace, or using a local Kubernetes environment like minikube or kind.

Absolutely test the same Docker image you will be running in production.

In summary, I'd rearrange your workflow as:

  1. Local feature branch for a microservice pushed to remote
  2. PR request to merge feature request into testing branch for microservice
    1. Triggers CI pipeline:
    2. Run unit tests (without Docker)
    3. Builds a Docker image (and optionally pushes to a registry)
    4. Deploy to some test setup and run integration tests (with Docker)
  3. Accepted and merged
    1. Triggers CI/CD pipeline:
    2. Runs unit tests (without Docker)
    3. Builds a Docker image and pushes it
    4. Deploy to the production Kubernetes environment
-- David Maze
Source: StackOverflow