How Do I Get Skaffold And Helm Charts To Work With A Local Image Repository?

4/3/2019

We're trying to set up a local development environment with several microservices app under Skaffold. We managed to do it with base Skaffold, using a (slightly outdated) tutorial at https://github.com/ahmetb/skaffold-from-laptop-to-cloud. And to get Skaffold to push images to a local repository without Helm, all I had to do was set up the imageName to use something like localhost:5000/image_name.

But with Helm, well.... I set up a very crude Helm install (DISCLAIMER: I am not much familiar with Helm yet), just changing the skaffold YAML to use Helm and dumping all the .YAML deployment and service files into the Helm chart's /templates directory, and that bombed.

Skaffold then successfully creates any pods that rely on a stock external image (like redis), but then whenever anything uses an image that would be generated from a local Dockerfile, it gets stuck and throws this error:

Failed to pull image "localhost:5000/k8s-skaffold/php-test": rpc error: code = Unknown desc = Error response from daemon: Get http://localhost:5000/v2/: dial tcp [::1]:5000: connect: connection refused

As far as I can tell, that's the error that comes when we haven't initialized a local Docker image repository - but with the non-Helm version, we don't need to start up a local image repository, Skaffold just makes that magic happen. Which is part of the appeal OF Skaffold.

So how do we automagically get Skaffold to create Helm charts that create and pull from a local repository? (As noted, this may be my unfamiliarity with Helm. If so, I apologize.)

The Skaffold YAML is this:

apiVersion: skaffold/v1beta7
kind: Config
build:
  tagPolicy:
    sha256: {}
  artifacts:
  - image: localhost:5000/k8s-skaffold/php-test
    context: voting-app/php-test  
deploy:
  helm:
    releases:
    - name: php-help-test
      chartPath: helm
      #wait: true
      #valuesFiles:
      #- helm-skaffold-values.yaml
      values:
        image: localhost:5000/k8s-skaffold/php-test
      #recreatePods will pass --recreate-pods to helm upgrade
      #recreatePods: true
      #overrides builds an override values.yaml file to run with the helm deploy
      #overrides:
      # some:
      #   key: someValue
      #setValues get appended to the helm deploy with --set.  
      #setValues:
        #some.key: someValue

And the Helm Chart values.yaml is the default provided by a generated chart. I can also provide the Dockerfile if needed, but it's just pulling from that image.

-- Ferrett Steinmetz
kubernetes
kubernetes-helm
skaffold

1 Answer

4/3/2019

You can't use localhost in your image definition. For the sake of testing you can try to use the ip of the host where your private registry is running, say if the host has address 222.0.0.2, then use image: 222.0.0.2:5000/k8s-skaffold/php-test.

It is of course undesirable to hard-code an address so a better way is to omit the "host" part entirely;

  image: k8s-skaffold/php-test:v0.1

In this case your CRI (Container Runtime Interface) plugin will try a sequence of servers, for instance docker.io. The servers are configurable but unfortunately I don't know how to configure it for "docker" since I use cri-o myself.

-- lgekman
Source: StackOverflow