Open in app

Sign in

Write

Sign in

Sameera De Silva
6 min readOct 25, 2021

How to rerun failed test cases only in Jenkins Free Style project with TestNG without any additional plugins.

Running failed test cases in Jenkins free style project is a challenge.
This is how I overcame it .

I used maven and My pom.xml is as per below.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion><groupId>org.example</groupId>
    <artifactId>MultiWebdriverDemo</artifactId>
    <version>1.0-SNAPSHOT</version><properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <aspectj.version>1.9.6</aspectj.version>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M1</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>mytestng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                </configuration>
            </plugin>
        </plugins>
        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <filtering>true</filtering>
            </resource>
        </resources>
    </build>
    <dependencies>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.3.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-firefox-driver -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-firefox-driver</artifactId>
            <version>3.141.59</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>4.3.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/commons-io/commons-io -->
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.8.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.github.fge/json-schema-validator -->
        <dependency>
            <groupId>com.github.fge</groupId>
            <artifactId>json-schema-validator</artifactId>
            <version>2.2.6</version>
        </dependency>
        <dependency>
            <groupId>com.github.fge</groupId>
            <artifactId>json-schema-core</artifactId>
            <version>1.2.5</version>
        </dependency><!-- https://mvnrepository.com/artifact/io.rest-assured/rest-assured -->
        <dependency>
            <groupId>io.rest-assured</groupId>
            <artifactId>rest-assured</artifactId>
            <version>4.3.3</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.json/json -->
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20190722</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.aventstack/extentreports -->
        <dependency>
            <groupId>com.aventstack</groupId>
            <artifactId>extentreports</artifactId>
            <version>5.0.8</version>
        </dependency>
        <!-- Parent class pages dependency -->
        <dependency>
            <groupId>CommonPages</groupId>
            <artifactId>CommonPagesStorage</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.sonarsource.scanner.maven</groupId>
            <artifactId>sonar-maven-plugin</artifactId>
            <version>3.9.0.2155</version>
        </dependency>
        <dependency>
            <groupId>io.opentelemetry</groupId>
            <artifactId>opentelemetry-exporter-jaeger</artifactId>
            <version>0.14.0</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-netty</artifactId>
            <version>1.34.1</version>
        </dependency>
    </dependencies>
</project>

First I created a Jenkins free free style project.Then I parameterized as per below and configured the job.

Here is to select the test plan name.

Here is to select my browser.

Here is to select number of reruns.

JDK version selection

GITHUB url and credentials selection

BrowserStack credentials

Now let’s look at the building the project part.

Add Execute shell and enter the below code .In the shell.

export M2_HOME=/opt/software/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.8.1
export PATH=$PATH:$M2_HOME/bin
export http_proxy="http://proxy-dev.dem.skynet.com:8088"
export https_proxy="http://proxy-dev.dem.skynet.com:8088"
export NO_PROXY=160.254.135.154,.dem.skynet.com,localhost,127.0.0.1
mvn --version
cd CommonPagesStorage
mvn clean install -DskipTests
cd ..
cd MultiWebdriverDemo
if [ -n "$browser_version" ]; then
    echo " ${browser_version} not empty so running with this on ${Browser} "
    mvn clean test -Dmaven.test.failure.ignore=true -Dbrowser="${Browser}" -Dbrowser_version="${browser_version}" -Dhttp.proxyHost=proxy-dev.dem.skynet.com -Dhttp.proxyPort=8088 -Dhttps.proxyHost=proxy-dev.dem.skynet.com -Dhttps.proxyPort=8088  -Dsurefire.suiteXmlFiles=TestSuites/${TestPlanName}else
    echo " ${browser_version} is empty"
    mvn clean test -Dmaven.test.failure.ignore=true -Dhttp.proxyHost=proxy-dev.dem.skynet.com -Dhttp.proxyPort=8088 -Dhttps.proxyHost=proxy-dev.dem.skynet.com -Dhttps.proxyPort=8088  -Dsurefire.suiteXmlFiles=TestSuites/${TestPlanName}fiecho "${TestPlanName}is ran for the first time."

What it does is ,
First, export the M2_HOME
Then to connect to browserStack proxy is used.
After that, build the project called CommonPagesStorage, since it’s only has my page objects , there is nothing to execute so I used, DskipTests to build only but skip the Test execution.


mvn clean install -DskipTests

In next step, in If user picked browser and browser version as Jenkins parameter used it and build the project.
Else in else statement, build the plan without it.
In both cases, I used below command , so even the script is failed build won’t mark as a failure, what it does is it’ marks the build as unstable and continue.

mvn clean test -Dmaven.test.failure.ignore=true

In the next Execute shell add the below code, It is for the reruninng the failed test suite.

export M2_HOME=/opt/software/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.8.1
export PATH=$PATH:$M2_HOME/bin
export http_proxy="http://proxy-dev.dem.skynet.com:8088"
export https_proxy="http://proxy-dev.dem.skynet.com:8088"
export NO_PROXY=160.254.135.154,.dem.skynet.com,localhost,127.0.0.1
mvn --version
cd MultiWebdriverDemo
echo "the second Script executed from: ${PWD}"i=1
echo "the NoOfReruns: ${NoOfReruns}"while [ $i -le ${NoOfReruns} ]
do
  echo Number: $i
  i=$((i+1))
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
    echo "testng-failed.xml file found some test-cases were failed."
    #Move  to the TestSuites folder
    mv target/surefire-reports/testng-failed.xml  ./TestSuites
    #Delete the target folder
    rm -rf target
 echo "testng-failed.xml rerun is started for the $i time."
if [ -n "$browser_version" ]; then
    echo " ${browser_version} not empty so rerun is started for the $i with this on ${Browser} "
    mvn clean test -Dmaven.test.failure.ignore=true -Dbrowser="${Browser}" -Dbrowser_version="${browser_version}" -Dhttp.proxyHost=proxy-dev.dem.skynet.com -Dhttp.proxyPort=8088 -Dhttps.proxyHost=proxy-dev.dem.skynet.com -Dhttps.proxyPort=8088  -Dsurefire.suiteXmlFiles=TestSuites/testng-failed.xmlelse
    echo " ${browser_version} is empty so rerun is started for the $i"
    mvn clean test -Dmaven.test.failure.ignore=true -Dhttp.proxyHost=proxy-dev.dem.skynet.com -Dhttp.proxyPort=8088 -Dhttps.proxyHost=proxy-dev.dem.skynet.com -Dhttps.proxyPort=8088  -Dsurefire.suiteXmlFiles=TestSuites/testng-failed.xmlfi
else
    echo "testng-failed.xml file not found, all tests were passed in first attempt ..."
    #Mark the Jenkins build as pass.
    exit 0
    break
fi
doneif [ -e ./target/surefire-reports/testng-failed.xml ]
then
    echo "Even ${NoOfReruns} in re run some scripts are failed so marking the build as failed."
    exit 1
else
    echo "testng-failed.xml file not found, after number of reruns ${NoOfReruns}  is passed marking the build as passed ..."
     exit 0
     break
fi

Here, we can ignore the building the CommonPagesStorage, since it’s already built.

Then I defined a variable to use a while loop and variable i is less than number of re runs variable run it and increment i by one.

i=1

Noe that, if the test failed, ./target/surefire-reports/testng-failed.xml this file is created and if it’s passed it’s not there. Another thing to mention is that do not add any dependOn methods , or else even the test is failed this file is not generated.

So don’t use methods like below in your scripts.

@Test()
    public void studentFail(ITestContext Test){
        driver.goToUrl("https://blazedemo.com/",driver);
        Assert.assertEquals("BlazeDemo",driver.getTitle());

    }
    @Test(dependsOnMethods = { "studentFail" })
    private void webPageTitleVerification(){
        driver.get("https://blazedemo.com/register");
}

So if the file is not found, no need of further executions , So below command will mark the build as pass and job is completed.

else
    echo "testng-failed.xml file not found, all tests were passed in first attempt ..."
    #Mark the Jenkins build as pass.
    exit 0
    break
fi
done

However, in first attempt if the testng-failed.xml file is found ,I cut and moved to another location and then deleted the target folder.(Because if it’s failed again it should be created freshly, normally it does but this to be precise.)

Then rerun it till mentioned no of reruns.

After completing number of reruns, check that the testng-failed.xml file is existing , if so exit 1 command will mark the build as failed.

If the file not found means the scripts were passed in rerun, so with exit 0, I could mark the Jenkins job as passed.

if [ -e ./target/surefire-reports/testng-failed.xml ]
then
    echo "Even ${NoOfReruns} in re run some scripts are failed so marking the build as failed."
    exit 1
else
    echo "testng-failed.xml file not found, after number of reruns ${NoOfReruns}  is passed marking the build as passed ..."
     exit 0
     break
fi

Next step is archiving the results. In post build steps I can publish my results to Extent report and TestNG report respectively.

Jenkins
Testng
Test Automation
Webdriver
Maven
Sameera De Silva
Sameera De Silva

Written by Sameera De Silva

38 Followers
24 Following

https://www.linkedin.com/in/sameera-de-silva-11247922/

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams