I'm trying to run some end-to-end tests in parallel and on DIFFERENT kubernetes pods on a declarative jenkins pipeline, jenkins however seems to attempt running the parallel stages on the SAME kubernetes pod. This leads to database deadlocks since both processes try inserting/truncating/updating/querying the same tables. Is there a way I can spin up a different pod for each of the parallel stages?
The kubernetes-plugin config:
agent {
kubernetes {
label 'my-label'
defaultContainer 'jnlp'
yaml """
apiVersion: v1
kind: Pod
metadata:
name: dind
spec:
containers:
- name: < default container >
image: < image >
securityContext:
privileged: true
fsGroup: 1000
command:
- cat
tty: true
volumeMounts:
- name: jenkins-bundle-gems
mountPath: /usr/local/bundle
- name: <tests-container-name>
image: < image >
securityContext:
privileged: true
fsGroup: 1000
volumeMounts:
- name: jenkins-bundle-gems
mountPath: /usr/local/bundle
command:
- cat
tty: true
"""
}
}
The parallel stage:
stage('Test'){
parallel {
stage("Branch 1") {
steps {
container('<tests-container-name>') {
sh "jenkins/scripts/initdb.sh"
sh 'bundle exec rspec --exclude-pattern "spec/features/*_spec.rb" spec'
}
}
}
stage("Branch 2") {
steps {
container('<tests-container-name>') {
sh "jenkins/scripts/initdb.sh"
sh "bundle exec rspec `jenkins/scripts/split_features.sh 0`"
}
}
}
}
}
EXPECTATION: I would like for jenkins to spin up two different pods for each of the parallel stages. This would allow me to use different databases for each of the tests.
ACTUAL RESULT: Jenkins runs both stages concurrently on the same pod.
Try something like this
stage("Run additional parallel tests") {
parallel(
"parallel stage 1": {
[INSERT YOUR CODE HERE]
},
"parallel stage 2": {
[INSERT YOUR CODE HERE]
}
)
}
}
You can set agent {} for each parallel stage to launch a pod per stage.
You have__________: stage > parallel > stage > steps.
You need to have also: stage > parallel > stage > agent.
Do not repeat pod definition twice, it is recommended to put the pod definition in a separated file and refer to it using yamlFile instead of yaml :
stage('Test'){
parallel {
stage("Branch 1") {
agent {
kubernetes {
defaultContainer 'jnlp'
yamlFile 'Jenkins.pod.yaml'
}
}
steps {
container('<tests-container-name>') {
sh "jenkins/scripts/initdb.sh"
sh 'bundle exec rspec --exclude-pattern "spec/features/*_spec.rb" spec'
}
}
}
stage("Branch 2") {
agent {
kubernetes {
defaultContainer 'jnlp'
yamlFile 'jenkins.pod.yaml'
}
}
steps {
container('<tests-container-name>') {
sh "jenkins/scripts/initdb.sh"
sh "bundle exec rspec `jenkins/scripts/split_features.sh 0`"
}
}
}
}
}
Hint
if blueocean is one of your plugins, it will help you to draw your pipeline under http://HOST/blue/organizations/jenkins/pipeline-editor/, then you can copy the Jenkinsfile code by typing [Cmd + s]