gitlab-ci cache on kubernetes with minio-service not working

8/22/2018

I'm running gitlab with current gitlab-runner 10.3.0 as a kubernetes deployment with a minio-server for caching. Everything is deployed using helm. The gitlab runner's helm is customized using this values.yml:

cache:
  cacheType: s3
  s3ServerAddress: http://wizened-tortoise-minio:9000
  s3BucketName: runners
  s3CacheInsecure: false
  cacheShared: true
  secretName: s3access
  # s3CachePath: gitlab_runner

The s3access is defined as cluster secret, the runners bucket exists on minio. Problem is that the cache is not being populated although the build log doesn't show any issues:

Checking cache for onekey-6
Successfully extracted cache
...
Creating cache onekey-6...
.m2/repository/: found 5909 matching files         
Created cache

Looking into the minio bucket it is empty. I'm confident that the gitlab runner s3ServerAddress is correct as changing it shows as errors in the build process (here e.g. when using https):

Checking cache for onekey-6...
WARNING: Retrying...                               
WARNING: Retrying...                               
Failed to extract cache

Creating cache onekey-6...
.m2/repository/: found 5909 matching files         
Uploading cache.zip to https://wizened-tortoise-minio/runners/gitlab_runner/runner/b87d7697/project/1644/onekey-6 
WARNING: Retrying...                               
Uploading cache.zip to https://wizened-tortoise-minio/runners/gitlab_runner/runner/b87d7697/project/1644/onekey-6 
WARNING: Retrying...                               
Failed to create cache
Uploading cache.zip to https://wizened-tortoise-minio/runners/gitlab_runner/runner/b87d7697/project/1644/onekey-6 
FATAL: Put https://wizened-tortoise-minio

I've also added echo $S3_SERVER_ADDRESS to the build and it's empty.

So: how do I need to configure gitlab-runner to use minio for caching?

Note: I'm aware of gitlab-ci cache on kubernetes with minio-service not working anymore

-- andig
gitlab-ci
gitlab-ci-runner
kubernetes
minio

1 Answer

10/12/2018

For sake of completeness the problem is with:

s3ServerAddress: http://wizened-tortoise-minio:9000

While gitlab apparently does some "presence" check where it accepts the http:// it doesn't when actually cloning the cache. Unfortunately it seems to silently swallow the error. Working version needs:

s3ServerAddress: wizened-tortoise-minio:9000

Opened gitlab issue at https://gitlab.com/gitlab-org/gitlab-runner/issues/3539#note_103371588

-- andig
Source: StackOverflow