K
Q

Jenkins on Kubernetes in Helm - how to attach ssh keys to the agent

4/21/2020

I'm using Jenkins from stable/jenkins helm chart - all is fine I attached secrets with id_rsa/.pub key as volume to the jenkins master (in /opt/sshkeys/ let's say, why there? because the created files are a soft-link with 777 permissions so cannot go directly to $HOME/.ssh)

root@agent-pod:/opt/config# ls -alh
total 12K
drwxrwxrwx 3 root root 4.0K Apr 20 13:00 .
drwxr-xr-x 1 root root 4.0K Apr 20 13:02 ..
drwxr-xr-x 2 root root 4.0K Apr 20 13:00 ..2020_04_20_13_00_29.601158101
lrwxrwxrwx 1 root root   31 Apr 20 13:00 ..data -> ..2020_04_20_13_00_29.601158101
lrwxrwxrwx 1 root root   13 Apr 20 13:00 id_rsa -> ..data/id_rsa

and then run few scripts (cp, chmod) to move them as proper files to /home/jenkins/.ssh/id_rsa - issues with the jenkins agent is - I can't simply attach volume as described above for the 777 reason, and I cannot use any command on the agent as the main ENTRYPOINT command is the one that makes it THE agent

command:
args: "${computer.jnlpmac} ${computer.name}"
  • trying to do any combination of mkdir /home/jenkins/.ssh && chmod ... && <above args> or with ; makes the pod go into 0/1 Completed state rather than 1/1 Running

Does anyone have any idea how to put the two priv and pub keys into jenkins agent ? That's the main question. All else I just said is my way of solving it.

Part of the Kubernetes plugin for Jenkins (which is required when running Jenkins on Kubernetes) shows this:

Constraints
Multiple containers can be defined in a pod. One of them is automatically created with name
jnlp, and runs the Jenkins JNLP agent service, with args ${computer.jnlpmac} ${computer.name},
and will be the container acting as Jenkins agent.

Other containers must run a long running process, so the container does not exit. If the default 
entrypoint or command just runs something and exit then it should be overridden with something
like cat with ttyEnabled: true.

WARNING If you want to provide your own Docker image for the JNLP slave, you must name the
container jnlp so it overrides the default one. Failing to do so will result in two slaves 
trying to concurrently connect to the master.

PS: I see the official docs describing how to attach keys to pod here https://kubernetes.io/docs/concepts/configuration/secret/#use-cases (ctrl+f Use-Case: Pod with ssh keys) but I met the issue as above with 777

PS: To see how the deployment and values files looks like and test you can install helm3 and helm3 show values stable/jenkins > jenkins.yaml

-- potatopotato
docker
jenkins
kubernetes

Similar Questions

connection pool configuration in kubernetes
Connection pooling with HikariCP, springboot and kubernetes
Minikube keeps failing on creation : StartHost failed, but will try again: creating host: create host timed out in 120.000000 seconds
Triggering auto-expansion of openshift persistent volumes

1 Answer

4/23/2020

As @potatopotato mentioned in comments, he solved this issue with creating a nfs server and attaching it as a volume to agent, with manually putting keys there.

I would recommend to check this medium article as it describe well how it should be done, all resources mentioned in this article can be found in this git repository

This git repository contains files needed to run through the demo for deploying

  • a Jenkins master
  • a Jenkins agent with a persistent volume

More precisely check this part.

Communication between master and agent

There are a number of ways a Jenkins master and agent can connect to each other. We will initiate this connection from the Jenkins master over SSH, using SSH credentials to secure the connection. This approach requires that we install the ssh-slaves plugin to our master, mount ssh credentials into the master and agent, and base our agent off of an ssh-slave image.

Additionally, the Jenkins master needs to connect to each agent via unique static hostnames, which we get by deploying our agents using a StatefulSet.

Initialization of agent(s)

Jenkins master will not connect to an ssh-based agent unless it is configured to do so. We can bootstrap this configuration by giving each agent pod an Init Container in charge of configuring Jenkins master. By the time the agent starts up, Jenkins master will already be trying to connect.

The Init Container will need to complete the following:

  • Download the Jenkins CLI from master
  • Use the CLI to check if master is already configured to use this agent
  • If needed, use the CLI to configure master to use this agent

This will require some configuration within our agent pod:

  • fsGroup set to the “jenkins” user group. This is needed because persistent volumes mount as the root user by default, making them unusable for the “jenkins” user.
  • Environment variables within the Init Container: JENKINS_URL: http address of Jenkins master JENKINS_LABEL: label for our agent
  • Environment variables within the main container: JENKINS_SLAVE_SSH_PUBKEY: public key to authenticate connection attempts
-- jt97
Source: StackOverflow