How can I write a Jenkins email-ext template to display test results like the standard test report

Question

I have tweaked the standard jelly template to display the current test results in a table, however I really want to be able to display diffs as seen in Jenkins own test results page.

For example:

JUnit Tests: 0 failures (±0) , 1 skipped (+1)

Package               Duration   Fail  (diff)  Skip  (diff)  Total  (diff)
foo.bar.baz              89 ms      0      0     1       +1     5       +2

Hinge

Answer 1

Write a Groovy template for Email Ext plugin instead of Jelly template. In Groovy template you'll have access to Build object for your build. You can then call getTestResultAction on it to obtain the AbstractTestResultAction for the build which you can then query for everything you need.

Here is a link to Jenkins Main Module API. A sample Groovy template for Ext Email plugin could be found in $JENKINS_HOME/plugins/email-ext/WEB-INF/classes/hudson/plugins/emailext/templates/groovy-html.template. More info on Groovy template/script usage can be found in Email Ext plugin documentation.

Dr

@Jon, was this answer helpful? – Dr

I've managed to pull out the failed tests using build.testResultAction.failedTests, but I am not sure how to access all tests – Hinge

Looks like accessing the PackageResult's depends on on the type returned by getTestResultAction - AggregatedTestResultAction & TestResultAction require different handling. – Hinge

Here's a link to the groovy-html.template. – Octarchy

This is now "easier" with Jenkins pipelines, you can write the email template in a @NonCPS annotated method in the Jenkinsfile and access the test results with def testResult = currentBuild.rawBuild.getAction(hudson.tasks.junit.TestResultAction.class) – Hinge

Answer 2

If you are struggle on how to access it via internal API (difficult to know and it exists limitation always), there is another more flexible way to do it.

Using FILE token instead of groovy template

1. using script to access your testing data via Jenkins API, for your case, it is like http://jenkins.server/job/yourjob/lastCompletedBuild/testReport/api/xml and generate your own html file like email.html under the workspace
2. In Default Content form in the email-ext configuration, using FILE token to send the email directly ${FILE, path="email.html"}

In step 1 above, you can also use more flexible way for your own template, I use python script and simple string Template.

It works perfect for me.

Aeromechanic

Would you care to share the script you described as step 1? – Kiker

If you use HTML format, you'll need to use ${SCRIPT, template="groovy-html.template"} instead of ${FILE}. – Octarchy

Can you please be detail on this one. I think I need to do similar thing. I wanted to display complete git commit message from git in the email. But it displays just first line. I think I need to write some script.. – Settera

Answer 3

To expand on this answer: Write a Groovy template for Email Ext plugin instead of Jelly template. In Editable Email Notification content

• set content type to "HTML" or "Both HTML and Plain Text"

• and include the groovy script like this:

  ${SCRIPT, template="test.groovy"}

• put the groovy script in email-templates home e.g. /var/lib/jenkins/email-templates. see below test.groovy.

In the example below every test is iterated by getting each of these objects: '''junitResult.getChildren()'''. If one desired to iterate only failed tests then junitResult.getFailedTests() could be used. See the hudson.tasks.junit.TestResult API: http://hudson-ci.org/javadoc/hudson/tasks/junit/PackageResult.html also see http://hudson-ci.org/javadoc/hudson/model/Build.html

Collection<ClassResult> getChildren()
List<CaseResult>    getFailedTests()

Example/template from email-ext-plugin can be seen here: https://github.com/jenkinsci/email-ext-plugin/blob/master/src/main/resources/hudson/plugins/emailext/templates/groovy-html.template

This example shows summary test result and table for results for each test suite and individual test. test.groovy:

<html>
<body>
<%

    import hudson.model.*

    def build = Thread.currentThread().executable
    def buildNumber = build.number
    def buildNumHash = build.getDisplayName()

    def testCount = "0"
    def testPassed = "0"
    def testFailed = "0"
    def testSkipped = "0"
    def buildDuration = "0"
    if(build.testResultAction) {
        def testResult = build.testResultAction
        testCount = String.format("%d",(testResult.totalCount))
        testPassed = String.format("%d",(testResult.result.passCount))
        testFailed = String.format("%d",(testResult.result.failCount))
        testSkipped = String.format("%d",(testResult.result.skipCount))
        testDuration = String.format("%.2f",(testResult.result.duration ))
    }

    def workspace = build.getEnvVars()["WORKSPACE"]
    def buildName = build.getEnvVars()["JOB_NAME"]
    def BUILD_STATUS = build.getEnvVars()["BUILD_STATUS"]
    def BUILD_URL = build.getEnvVars()["BUILD_URL"]

    def testResult = hudson.tasks.junit.TestResult

    def testResult2 = build.getAction(hudson.tasks.junit.TestResultAction.class)

%>

start test.groovy <br><br>
<b>TEST RESULT:</b> $testCount total, <b>$testPassed pass</b>, <b>$testFailed fail</b>, $testSkipped skip.<br>
Workspace : $workspace<br>
Project Name : $buildName $buildNumHash<br><br>

<!-- GENERAL INFO -->

<TABLE>
  <TR><TD align="right">
    <j:choose>
      <j:when test="${build.result=='SUCCESS'}">
        <IMG SRC="${rooturl}static/e59dfe28/images/32x32/blue.gif" />
      </j:when>
          <j:when test="${build.result=='FAILURE'}">
        <IMG SRC="${rooturl}static/e59dfe28/images/32x32/red.gif" />
      </j:when>
      <j:otherwise>
        <IMG SRC="${rooturl}static/e59dfe28/images/32x32/yellow.gif" />
      </j:otherwise>
    </j:choose>
  </TD><TD valign="center"><B style="font-size: 200%;">BUILD ${build.result}</B></TD></TR>
  <TR><TD>Build URL</TD><TD><A href="${rooturl}${build.url}">${rooturl}${build.url}</A></TD></TR>
  <TR><TD>Project:</TD><TD>${project.name}</TD></TR>
  <TR><TD>Date of build:</TD><TD>${it.timestampString}</TD></TR>
  <TR><TD>Build duration:</TD><TD>${build.durationString}</TD></TR>
  <TR><TD>Test duration:</TD><TD>${testDuration}</TD></TR>
</TABLE>
<BR/>

<!-- JUnit TEMPLATE  hudson.tasks.junit.TestResult   -->

<% def junitResultList = it.JUnitTestResult
try {
 def cucumberTestResultAction = it.getAction("org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultAction")
 junitResultList.add(cucumberTestResultAction.getResult())
} catch(e) {
        //cucumberTestResultAction not exist in this build
}
// API: http://hudson-ci.org/javadoc/hudson/tasks/junit/PackageResult.html
%>

<!-- JUnit TEMPLATE: all tests PASS FAIL SKIP >
<% 
if (junitResultList.size() > 0) { %>
 <TABLE width="100%">
 <TR><TD class="bg1" colspan="2"><B>${junitResultList.first().displayName}</B></TD></TR>
 <% junitResultList.each{
  junitResult -> %>
     <% junitResult.getChildren().each { packageResult -> %>
        <TR><TD class="bg2" colspan="2"> <B>TEST SUITE: ${packageResult.getName()} Failed: ${packageResult.getFailCount()} test(s), Passed: ${packageResult.getPassCount()} test(s)</B>, Skipped: ${packageResult.getSkipCount()} test(s), Total: ${packageResult.getPassCount()+packageResult.getFailCount()+packageResult.getSkipCount()} test(s)</TD></TR>
        <% packageResult.getChildren().each{ suite -> 
               suite.getChildren().each{ test ->
           def colour = "lightgreen"
           def highlight1=""
           def highlight2=""
           RESULT = test.getStatus() // FAILED or PASSED or SKIPPED
           if (RESULT == hudson.tasks.junit.CaseResult.Status.FAILED || RESULT == hudson.tasks.junit.CaseResult.Status.REGRESSION) {
               colour = "#ffcccc" 
               highlight1="<B>"
               highlight2="</B>"
           }
           if (RESULT == hudson.tasks.junit.CaseResult.Status.SKIPPED) { colour = "#ffffb3" }
         %>
          <TR bgcolor="${colour}"><TD class="test" colspan="2">${highlight1}<li>${RESULT}: ${test.getFullName()} </li>${highlight2}</TD></TR>
        <% } }
      }
 } %>
 </TABLE>
 <BR/>
<%
} %>

end of test.groovy

</body>
</html>

e.g. output (text only no colours/formatting)

start test.groovy 

TEST RESULT: 18 total, 18 pass, 0 fail, 0 skip. 
Workspace : /var/lib/jenkins/jobs/jobname-1/workspace 
Project Name : jobname-1 #20

BUILD SUCCESS 

Build URL   http://jenkinsurl:port/job/jobname-1/20/
Project:    jobname-1 
Date of build:  Mon, 23 Jan 2017 09:29:00 +0000 
Build duration: 10 min 
Test duration:  267.12

Test Results 
TEST SUITE: suitename1 Failed: 0 test(s), Passed: 3 test(s), Skipped: 0 test(s), Total: 3 test(s) 
 * PASSED: suitename1.testclass.testname1
 * PASSED: suitename1.testclass.testname2
 * PASSED: suitename1.testclass.testname3
TEST SUITE: suitename2 Failed: 2 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 3 test(s) 
 * PASSED: suitename2.testclass.testname1
 * FAILED: suitename2.testclass.testname2
 * REGRESSION: suitename2.testclass.testname3

end of test.groovy

Blown

Answer 4

My solution in Jelly basing on default static-analysys.jelly script

  <!-- JUnit TEMPLATE -->
  <j:set var="junitResultList" value="${it.JUnitTestResult}" />
  <j:if test="${junitResultList.isEmpty()!=true}">
    <div class="content">
      <a href="${rooturl}${build.url}/testReport">
        <h1>JUnit Tests</h1>
      </a>
      <table class="border">
        <tr>
          <th class="border">Package</th>
          <th class="border">Failed</th>
          <th class="border">Failed (diff)</th>
          <th class="border">Passed</th>
          <th class="border">Passed (diff)</th>
          <th class="border">Skipped</th>
          <th class="border">Skipped (diff)</th>
          <th class="border">Total</th>
          <th class="border">Total (diff)</th>
        </tr>
        <j:forEach var="junitResult" items="${it.JUnitTestResult}">
          <j:forEach var="packageResult" items="${junitResult.getChildren()}">
            <tr>
              <td class="border">
                <tt>${packageResult.getName()}</tt>
              </td>
              <td class="border test_failed">${packageResult.getFailCount()}</td>
              <td class="border test_failed">${packageResult.getFailCount()-packageResult.previousResult.getFailCount()}</td>
              <td class="border test_passed">${packageResult.getPassCount()}</td>
              <td class="border test_passed">${packageResult.getPassCount()-packageResult.previousResult.getPassCount()}</td>
              <td class="border test_skipped">${packageResult.getSkipCount()}</td>
              <td class="border test_skipped">${packageResult.getSkipCount()-packageResult.previousResult.getSkipCount()}</td>
              <td class="border">
                <b>${packageResult.getPassCount()+packageResult.getFailCount()+packageResult.getSkipCount()}
                </b>
              </td>
              <td class="border">
                <b>${packageResult.getPassCount()+packageResult.getFailCount()+packageResult.getSkipCount()-packageResult.previousResult.getPassCount()-packageResult.previousResult.getFailCount()-packageResult.previousResult.getSkipCount()}
                </b>
              </td>
            </tr>
            <j:forEach var="failed_test"
              items="${packageResult.getFailedTests()}">
              <tr>
                <td class="test_failed" colspan="5">
                  <tt>${failed_test.getFullName()}</tt>
                </td>
              </tr>
            </j:forEach>
          </j:forEach>
        </j:forEach>
      </table>
      <br />
    </div>
  </j:if>

Acapulco

Answer 5

Made one with Allure and JUnit

https://gist.github.com/unickq/036224c766a76bdd9eb5de379a187af5

Rabbi

@LeeWay here you go gist.github.com/unickq/036224c766a76bdd9eb5de379a187af5 – Rabbi

do we need to create the template and put it into a jenkins folder or something? can you please provide full instructions? – Beslobber