Automation testcases failing in kubernetes but jenkins pipeline passing

5/17/2021

Hello i am using testcafe for automation and used docker jenkins and kubernetes for deploying it. However if testcases are failing in Kubernetes POD the jenkins Pipeline is passing! How can i make it fail? How can i make Jenkins Pipeline fail if testcases are failing in Kubernetes POD?

Below is my Jenkins file:

#!/usr/bin/env groovy

def gitUrl = 'https://github…’
def emailUser = 'user@abc.com'

String dockerOrg='abc'
String appName='abc-testcafe-poc' 
String dockerTag='feature'
String dockerCredential='abc_docker'  

if(env.BRANCH_NAME.contains('feature/')){
    environment='dev'
    dockerTag='feature'
}
if(env.BRANCH_NAME == 'develop'){
    environment ='test'
    dockerTag ='develop'
}


pipeline {
    agent {
        label 'docker-slave' //build -server
    }
    environment {
        TECH_USER = ''
        GIT_CREDENTIALS_ID = ''
        NAMESPACE_DEV = ''
        NAMESPACE_TST = ''
        NAMESPACE_STG = ''
        NAMESPACE_PRD = ''
        CLUSTER_DEV = ''
        CLUSTER_TST = ''
        CLUSTER_STG = ''
        CLUSTER_PRD = ''
        K8S_CRED_ID_DEV = ''
        K8S_CRED_ID_TST = ''
        K8S_CRED_ID_STG = ''
        K8S_CRED_ID_PRD = ''
        KUBECTL_VERSION = '1.14.2'
        NPM_AUTH_KEY  = " "
        NPM_EMAIL     = " "
    }
    stages {
            stage ('Checkout') {
                steps {
                    deleteDir()
                    echo "Jenkins is checking out latest code from git repo..."
		            echo "Branch name : ${env.BRANCH_NAME}"          
                    glGitCheckoutWin(credentialsId: 'abc’, tag: env.BRANCH_NAME, url: "$gitUrl")
                }
            }   
            stage ('Build & Push Docker Image') {
                steps {
                   script {
                   dockerPath = "docker.com/$dockerOrg/$appName:$dockerTag"
                   echo dockerPath
                   glDockerImageBuildPush( image: "$dockerPath", credentialsId: "$dockerCredential", containerRegistry: "docker.repo.com") 
                }
              }
            }

            stage ('deploy using yml template') {
                
                steps {
                    script {
                    sh "sed -i 's/{dockerTag}/${dockerTag}/g' Testcafe_Deploy.yaml"    
                    String timestamp = sh(returnStdout: true, script: "date -u +'%Y-%m-%dT%H:%M:%SZ'").trim()
                    sh "sed -i 's/THIS_STRING_IS_REPLACED_DURING_BUILD/${timestamp}/g' automationdeploy.yaml"
    
                    if (env.BRANCH_NAME.contains('feature/')) 
                    {
                        glKubernetesApplyBasic credentials: "", cluster: "", namespace: "", yaml: "automationdeploy.yaml", env: "Dev", isProduction: false
                        
                        
			              
                    } else if (env.BRANCH_NAME=='develop') {
                        glKubernetesApplyBasic credentials: "$env.", cluster: "$env.", namespace: "$env", yaml: "Testcafe_Deploy.yaml", env: "Test", isProduction: false
                    }  
                    }
                }    
            } 
                  
        }
    post {
        always {
            echo 'This will always run'
		    emailext body:  "Build URL: ${BUILD_URL}",
			subject: "$currentBuild.currentResult-$JOB_NAME",
			to: "$emailUser"	
        }
        success {
            echo 'This will run only if successful'
        }
        failure {
            echo 'This will run only if failed'
        }
        unstable {
            echo 'This will run only if the run was marked as unstable'
        }
        changed {
            echo 'This will run only if the state of the Pipeline has changed'
            echo 'For example, if the Pipeline was previously failing but is now successful'
        }
    }
}
-- TS0306
automated-tests
jenkins
kubernetes
testcafe
testing

1 Answer

5/19/2021

TestCafe finishes test execution when the exit code equals the number of failed tests. E.g., if all tests are passed, the exit code will be 0; if three tests fail, the exit code will be 3.

So, you need to abort the Jenkins pipeline using any available way: Executor.interrupt, error-signal, etc.

-- mlosev
Source: StackOverflow