Issue configuring insecure registry access with kops

4/17/2018

I'm using kops v1.8.1 and Kubernetes v1.10.0 on AWS to build my infrastructure.

I created an external plain HTTP docker registry following the procedure in this link: https://docs.docker.com/registry/deploying/#run-a-local-registry

I then created a cluster using kops and added the following lines in the cluster spec manifest:

docker: 
  insecureRegistry: <hostnameofregistry> 
  logDriver: ""

Cluster is created but when trying to docker push to the insecure registry, it fails

I was expecting to be able to push or pull images from insecure registry without having to add any configuration on the master or nodes

I also tried using hooks to give access to the insecure registry to the master and nodes (based on the following procedure https://docs.docker.com/registry/insecure/#deploy-a-plain-http-registry).

I added the following hook to the cluster spec manifest:

hooks:
  - execContainer:
  command:
    - sh
    - -c
    - echo 35.155.193.162 <hostnameofregistry> >> /rootfs/etc/hosts &&
      echo { '"insecure-registries":["<hostnameofregistry>:5000"] } >> /rootfs/etc/docker/daemon.json &&
      /usr/bin/systemctl daemon-reload &&
      /usr/bin/systemctl restart docker.service
  image: busybox

That hook does modify /etc/hosts and creates /etc/docker/daemon.json on the master and nodes but I still need to manually do the daemon reload and restart docker for the master and nodes to have access to the insecure registry.

I'm relatively new to Docker and Kubernetes in general so not sure what I'm missing or doing wrong. Any help or comment would help.

Thanks

-- julienla
docker
docker-registry
kops
kubernetes

1 Answer

4/17/2018

First of all, regarding your local registry, probably you may have a problem in firewall when you create the registry on one AWS node and try to access it from other nodes, but you may have not opened the port on registry node to your network, this is may be the reason it fails.

In general, Amazon Elastic Container Registry would be a good solution for you. Your infrastructure already is on AWS, so the good way is to use the components of AWS instead of some strange hacks.

In case you are using AWS ECR, you just need to add IAM role to your Kubernetes nodes to get access to the registry.

-- Nick Rak
Source: StackOverflow