K
Q

K8s plugin for Jenkins: Always running two separate containers inside a pod

5/31/2018

I have created a Dockerfile (for a Node JNLP slave which can be used with the Kubernetes Plugin of Jenkins ). I am extending from from the official image jenkinsci/jnlp-slave

FROM jenkinsci/jnlp-slave

USER root


MAINTAINER Aryak Sengupta <aryak.sengupta@hyland.com>
LABEL Description="Image for NodeJS slave"

COPY cert.crt /usr/local/share/ca-certificates
RUN update-ca-certificates

RUN curl -sL https://deb.nodesource.com/setup_8.x | bash \
    && apt-get install -y nodejs

ENTRYPOINT ["jenkins-slave"]

I have this image saved inside my Pod template (in K8s plugin configuration). Now, when I'm trying to run a build on this slave, I find that two containers are getting spawned up inside the Pod (A screenshot to prove the same.).

enter image description here

My Pod template looks like this:

enter image description here

And my Kubernetes configuration looks like this: enter image description here

Now if I do a simple docker ps, I find that there are two containers which started up (Why?):

enter image description here

Now, inside the Jenkins Job configuration of Jenkins, whatever I add in the build step, the steps get executed in the first container .

Even if I use the official Node container inside my PodTemplate, the result is still the same:

enter image description here

I have tried to print the Node version inside my Jenkins Job, and the output is "Node not found" . Also, to verify my haunch, I have done a docker exec into my second container and tried to print the Node version. In this case, it works absolutely fine.

This is what my build step looks like:

enter image description here

So, to boil it down, I have two major questions:

  1. Why does two separate (one for JNLP and one with all custom changes) containers start up whenever I fire up the Jenkins Job?
  2. Why is my job running on the first container where Node isn't installed? How do I achieve the desired behaviour of building my project with Node using this configuration?

What am I missing?

P.S. - Please do let me know if the question turns out to be unclear in some parts.

Edit: I understand that this can be done using the Pipeline Jenkins plugin where I can explicitly mention the container name, but I need to do this from the Jenkins UI. Is there any way to specify the container name along with the slave name which I am already doing like this:

enter image description here

-- Aryak Sengupta
docker
jenkins
jenkins-plugins
kubernetes

Similar Questions

how to recreate all the resources from a Weave Net manifest?
kubectl get pods fails after authentication
How kube-proxy runs iptables commands on the host node, while running inside a process isolated container?
Istio(0.7.1) : Circuit Breaker Doesn't work for httpConsecutiveErrors

3 Answers

6/27/2018

To set Container Template -> Name as jnlp. https://issues.jenkins-ci.org/browse/JENKINS-40847

-- mhang li
Source: StackOverflow
5/31/2018

The Jenkins kubernetes plugin will always create a JNLP slave container inside the pod that is created to perform the build. The podTemplate is where you define the other containers you need in order to perform your build.

In this case it seems you would want to add a Node container to your podTemplate. In your build you would then have the build happen inside the named Node container.

You shouldn't really care where the Pod runs. All you need to do is make sure you add a container that has the resources you need (like Node in this case). You can add as many containers as you want to a podTemplate. I have some with 10 or more containers for steps like PMD, Maven, curl, etc.

I use a Jenkinsfile with pipelines.

podTemplate(cloud: 'k8s-houston', label: 'api-hire-build', 
  containers: [
    containerTemplate(name: 'maven', image: 'maven:3-jdk-8-alpine', ttyEnabled: true, command: 'cat'),
    containerTemplate(name: 'pmd', image: 'stash.company.com:8443/pmd:pmd-bin-5.5.4', alwaysPullImage: false, ttyEnabled: true, command: 'cat')
  ],
  volumes: [
    persistentVolumeClaim(claimName: 'jenkins-pv-claim', mountPath: '/mvn/.m2nrepo')
  ]
)
{
  node('api-hire-build') {
    stage('Maven compile') {
      container('maven') {
        sh "mvn -Dmaven.repo.local=/mvn/.m2nrepo/repository clean compile"
      }
    }
    stage('PMD SCA (docker)') {
      container('pmd') {
        sh 'run.sh pmd -d "$PWD"/src -f xml -reportfile "$PWD"/target/pmd.xml -failOnViolation false -rulesets java-basic,java-design,java-unusedcode -language java'
        sh 'run.sh pmd -d "$PWD"/src -f html -reportfile "$PWD"/target/pmdreport.html -failOnViolation false -rulesets java-basic,java-design,java-unusedcode -language java'
        sh 'run.sh cpd --files "$PWD"/src --minimum-tokens 100 --failOnViolation false --language java --format xml > "$PWD"/target/duplicate-code.xml'
      }
      archive 'target/duplicate-code.xml'
      step([$class: 'PmdPublisher', pattern: 'target/pmd.xml'])
    }
  }
}
-- Daniel Watrous
Source: StackOverflow
11/12/2018

Alright so I've figured out the solution. mhang li's answer was the clue but he didn't explain it one bit.

Basically, you need to modify the official Jenkins Slave image found here and modify it to include the changes for your slave as well. Essentially, you are clubbing the JNLP and Slave containers into one and building a combined image.

The modification format will just look like this (picking up from the Dockerfile linked)

FROM jenkins/slave:3.27-1
MAINTAINER Oleg Nenashev <o.v.nenashev@gmail.com>
LABEL Description="This is a base image, which allows connecting Jenkins agents via JNLP protocols" Vendor="Jenkins project" Version="3.27"

COPY jenkins-slave /usr/local/bin/jenkins-slave

**INCLUDE CODE FOR YOUR SLAVE. Eg install node, java, whatever**

ENTRYPOINT ["jenkins-slave"] # Make sure you include this file as well

Now, name the slave container jnlp (Reason - bug). So now, you will have one container that spawns which will be your JNLP + Slave. All in all, your Kubernetes Plugin Pod Template will look something like this. Notice the custom url to the docker image I have put in. Also, make sure you don't include a Command To Run unless you need one.

enter image description here

Done! Your builds should now run within this container and should function exactly like you programmed the Dockerfile!

-- bholagabbar
Source: StackOverflow