How to generate JUnit test reports on android for jenkins

Question

I'am trying to make use of the "Publish JUnit test result report" in Jenkins, but can't get it to work for my android test project. The setup of my android test project in Jenkins is based on this guide: https://wiki.jenkins-ci.org/display/JENKINS/Building+an+Android+app+and+test+project

I hope someone could post an easy step by step guide on how to get the JUnit test result reports out of the test run to being able to use "Publish JUnit test result report". Would like to use this feature because the standard console output of the Junit tests in Jenkins are not quite convenient.

I found some guides (not so many) on the Internet, but non of them worked for me. This is what I tried so far (with no success):

• Just adding the "Publish JUnit test result report" post-build-action with no arguments
• Followed this guide, which suggests, that the reports can be downloaded from the device/emulator ( http://blackriver.to/2012/08/android-continuous-integration-with-ant-and-jenkins-part-2-2/ )
• Then this guide ( http ://blog.cloudbees.com/2012/11/unit-test-results-code-coverage-and.html ) which is similar to the previous guide, but adds a custom instrumentation library
• And then I have found this API for ant on junit reports ( http:// ant.apache.org/manual/Tasks/junitreport.html ) but I have absolutly no idea how to add this to my android test projected generated by the android sdk

(Some links are not displayed correctly because cant post more then two links with my reputition.)

Help is very appreciated :)

Jaxartes

Answer 1

I just had to solve this problem for myself, this is how I got it working:

1. Install the performance Plugin
2. In the project settings, add a new Post build action "Publish JUnit-testresults"
3. As path, add "/app/build/androidTest-results/connected/*.xml" (Path may be slightly different on your machine)
4. Now, after the project ran the first time, you have a new Link "Last testresult" and on the right side a graphical representation.

Regards

Emilia

Answer 2

Include Spoon library into your project from jenkins add build step to build apk's and one more step to run the command

java -jar spoon-runner-1.1.8-jar-with-dependencies.jar \
--apk example-app.apk \
--test-apk example-tests.apk

to run instrumentation tests and see the magic.

Stake

Answer 3

Generating report for instrumentation test cases for some test suite "TestExecutionOrder" and project "com.example"

adb shell am instrument -w -r -e log true -e class 
'com.example.TestExecutionOrder' 
com.example.test/android.support.test.runner.AndroidJUnitRunner > 
/home/user/Downloads/raw-tests.txt

This makes a raw-tests file that contains the success and fail test cases. Success test cases dont have stack field but failed test cases have.

Parsing this text file using simple java parser.

import java.io.*;

public class ReportScript {
    public static void main(String[] args) {
    File file = new File("/home/user/Downloads/raw-tests.txt");
    FileInputStream fis = null;
    try {
        fis = new FileInputStream(file);
        byte[] data = new byte[(int) file.length()];
        fis.read(data);
        fis.close();

        String inputStream = new String(data, "UTF-8");
        String[] tests = inputStream.split("INSTRUMENTATION_RESULT: ");
        String[] testCases = tests[0].split("NSTRUMENTATION_STATUS_CODE: -2|INSTRUMENTATION_STATUS_CODE: -1|INSTRUMENTATION_STATUS_CODE: 0|INSTRUMENTATION_STATUS_CODE: 1|INSTRUMENTATION_STATUS_CODE: 2");
        //Don't take last one since blank string
        String cases="";
        int failed = 0;
        for(int i = 1 ; i < testCases.length-1 ; i+=2){
            String[] result = testCases[i].split("INSTRUMENTATION_STATUS: ");
            String test  = " ", classname  = " ", time = " " , stack = " ";
            if(result.length == 7){
                for(int j =1 ; j<=6 ; j++){
                    String[] map = result[j].split("=");
                    String key  = map[0];
                    String value = map[1];
                    if("test".equalsIgnoreCase(key)){
                        test = value;
                    }else if ("class".equalsIgnoreCase(key)){
                        classname = value;
                    }else if ("time".equalsIgnoreCase(key)){
                        time = value;
                    }
                }
                cases += makePassXml(test,classname,time);
            }else{
                for(int j =1 ; j<=6 ; j++){
                    String[] map = result[j].split("=");
                    String key  = map[0];
                    String value = map[1];
                    if("test".equalsIgnoreCase(key)){
                        test = value;
                    }else if ("class".equalsIgnoreCase(key)){
                        classname = value;
                    }else if ("time".equalsIgnoreCase(key)){
                        time = value;
                    }else if ("stack".equalsIgnoreCase(key)){
                        stack = value;
                    }
                }
                cases += makeFailXml(test,classname,stack,time);
                failed++;
            }
        }
        String xml = makeTestSuiteXml(cases,"TestExecutorOrder",(testCases.length-1)/2 + "" ,failed + "");
        FileWriter writer = new FileWriter("/home/user/Downloads/junit_report.xml");
        writer.write(xml);
        writer.close();
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (UnsupportedEncodingException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

    }

    public static String makePassXml(String test, String classname, String time){
    return "<testcase name=\""+ test +"\" classname=\""+classname+"\" time=\"0.0\"/>\n";
    }

    public static String makeFailXml(String test, String classname, String stack,String time){
    return "<testcase name=\""+ test +"\" classname=\""+ classname +"\" time=\"0.0\">\n" +
            "\t\t<failure message=\"java.lang.AssertionError: test failed \" type=\"java.lang.AssertionError\">" +  stack.replaceAll("\\<","") +
            "</failure>\n" +
            "\t</testcase>\n";
    }

    public static String makeTestSuiteXml(String cases, String testSuite, String total , String failed){
    String top  = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
            "<testsuite name=\""+testSuite+"\" tests=\""+total+"\" skipped=\"0\" failures=\""+failed+"\" errors=\"0\" timestamp=\"2019-01-08T18:36:58\" hostname=\"jenkins-android-testing\" time=\"0.000\">\n" +
            "\t<properties/>\n";

    String bottom = "<system-out>\n" +
            "\t\t<![CDATA[]]>\n" +
            "\t</system-out>\n" +
            "\t<system-err>\n" +
            "\t\t<![CDATA[]]>\n" +
            "\t</system-err>\n" +
            "</testsuite>";

    return top + cases + bottom;
    }

}

Output :

junit_report.xml will be a JUnit report which can be visualized by Jenkins Junit report plugin.

Hagi