skaffold using external docker image without build

10/19/2021

We are using skaffold for our kubernetes deployment and don't need the feature of building a docker container for every microservice, since we are building a base docker image from our microservice monorepo.

Right now, we have a dockerfile for each service which looks like this:

FROM microservice-base-image:latest
WORKDIR <service>
CMD ["node", "."]

This brings the problem, that for every change in our base image skaffold builds and pushes n times docker images, which are the size of the base image.

Is there any way to tell skaffold to just use our base docker image without building a separate one for each microservice, so that we can adjust our base image in a way to start the specific index file via an ENTRYPOINT?

We already tried the following, but skaffold complains about that the image can not be pulled, even though we are logged in into the private docker registry.

skaffold.yaml

  apiVersion: skaffold/v2alpha3
  kind: Config
  
  profiles:
    - name: localhost
      deploy:
        kubectl:
          manifests:
            - ./infra/k8s/tmp/*
    - name: staging
      deploy:
        kubectl:
          manifests:
            - ./infra/k8s/tmp/*
  
  deploy:
    kubectl:
      manifests:
        - ./infra/k8s/tmp/*

service-1-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: $CI_ENVIRONMENT_PREFIX-service-1-depl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: $CI_ENVIRONMENT_PREFIX-service-1
  template:
    metadata:
      labels:
        app: $CI_ENVIRONMENT_PREFIX-service-1
    spec:
      imagePullSecrets:
        - name: <our-private-registry-secret>
      containers:
        - name: $CI_ENVIRONMENT_PREFIX-service-1
          image: <accountid>.dkr.ecr.eu-west-1.amazonaws.com/microservice-base-image:latest

Skaffold log of both skaffold dev and skaffold run --default-repo <accountid>.dkr.ecr.eu-west-1.amazonaws.com -p $CI_ENVIRONMENT_PREFIX

Waiting for deployments to stabilize...
 - deployment/staging-service-1-depl: container staging-service-1 is waiting to start: <accountid>.dkr.ecr.eu-west-1.amazonaws.com/microservice-base-image:latest can't be pulled
    - pod/staging-service-1-6bf9b46c68-jpmnt: container staging-service-1 is waiting to start: <accountid>.dkr.ecr.eu-west-1.amazonaws.com/microservice-base-image:latest can't be pulled
 - deployment/staging-service-1-depl failed. Error: container staging-service-1 is waiting to start: <accountid>.dkr.ecr.eu-west-1.amazonaws.com/microservice-base-image:latest can't be pulled.
Cleaning up...
-- Vetterjack
kubernetes
microservices
skaffold

1 Answer

10/26/2021

Skaffold only does image replacements for images that it built. You have no images configured in your skaffold.yaml and so no image replacement will occur.

Skaffold does provide for expressing dependencies between images, so that you can say that one image requires another image as a base image.

Instead of building N microservice images that just set the WORKDIR, you could just use your base image as the only image and set the container.workingDir in your Kubernetes manifests to decide which microservice should be used.

That said, your current images setup has a disadvantage that a change in a single microservice will require redeploying all of your microservices. Having each image include only its direct requirements means that only changed images will be redeployed, which results in faster redeploys.

-- Brian de Alwis
Source: StackOverflow