Execution time of tests in unit test class via maven surefire-report in a single file in summarized format

Question

Can anyone let me know how can I get the time taken by each of the unit tests in a unit test class in a single file via maven-surefire? I have seen my target/surefire-report it has files for each test. Basically I am looking for a single file with all the execution times summarized. If possible also sort the result by execution time of each test.

I am using maven 3.5 & surefire-plugin 2.4.2 on MacOSX 10.12.6.

Shelve

It has already been an answer how-to-identify-slow-unit-tests-when-using-maven-surefire-plugin – Oversell

Possible duplicate of How to identify slow unit tests when using maven-surefire-plugin in parallel mode? – Romeoromeon

I have seen that question. As mentioned in the question I am looking for a single file with the execution time of each test in a unit test class sorted by execution time. – Shelve

Really using such an old version of maven-surefire-plugin? – Maternal

@Maternal - Is there anything related to the question that has been discussed here added in the newest version of surefire-plugin? – Shelve

Answer 1

The maven-surefire-plugin currently doesn't let you do this. It writes all the results in separate files. You could create a feature-request in its issue tracker, if you feel like this is a missing feature.

However you can use some Linux commands to convert the output to what you need. Here are some commands that turn the separate XML files into a single file that looks like what you want:

grep testcase target/surefire-reports/TEST-*.xml |
  sed 's/.* name="\(.*\)" classname="\(.*\)" time="\(.*\)".*/\2#\1() - \3ms/g' |
  sort -n -k 3 > output.txt

---

Update: Numeric sorting has problems with varying number of fraction digits. Use awk version below to solve this.

---

The same thing could be done with awk a bit shorter and less cryptic:

grep -h testcase target/surefire-reports/TEST-*.xml |
  awk -F '"' '{printf("%s#%s() - %.3fms\n", $4, $2, $6); }' |
  sort -n -k 3 > output.txt

You have to execute these commands from the toplevel directory of your maven project after the surefire-reports were generated.

If you have multi-module project, use this instead:

find . -name TEST-*.xml -exec grep -h testcase {} \; |
  awk -F '"' '{printf("%s#%s() - %.3fms\n", $4, $2, $6); }' |
  sort -n -k 3 > output.txt

The resulting file is output.txt and contains lines of the following format:

<classname>#<methodname>() - <time>ms

The result is sorted by consumed time.

Ani

On executing the command find -name TEST-*.xml -exec grep -h testcase {} \; | awk -F '"' '{print $4 "#" $2 "() - " $6 "ms" }' | sort -n -k 3 > output.txt. It is saying find: illegal option -- n. I am trying in MacOSX 10.12.6 – Shelve

I was testing this on Linux with GNU find. Don't know the find version on MacOS or its syntax. – Chariness

I have updated the answer to add Mac OS X version of find command. – Shelve

For mac the last command will be find . -name TEST-*.xml -exec grep -h testcase {} \; | awk -F '"' '{print $4 "#" $2 "() - " $6 "ms" }' | sort -n -k 3 > output.txt – Shelve

Well, this also works on GNU find, so I updated the answer. – Chariness

Answer 2

I wrote a small bash script with Martin Höller's snippet that might help someone:

#!/bin/bash

# This script goes through tests results in `target/surefire-reports` and sorts the results by the
# amount of time they took. This is helpful to identify slow tests.

set -e

# Make sure the surefire folder exists
if [ ! -d "target/surefire-reports" ]; then
    echo "The folder 'target/surefire-reports' doesn't exists. Please run tests before using this script."
    exit 1
fi

# Go through the surefire test reports and sort tests by time taken. Source for this snippet:
# https://mcmap.net/q/1078376/-execution-time-of-tests-in-unit-test-class-via-maven-surefire-report-in-a-single-file-in-summarized-format/45859700#45859700
grep -h testcase target/surefire-reports/TEST-*.xml |
    awk -F '"' '{print $4 "#" $2 "() - " $6 "ms" }' |
      sort -rn -k 3

Aspa

Answer 3

You can use surefire-report-plugin for aggregation the results (but it still won't been sorted)

mvn surefire-report:report -DlinkXRef=false -Daggregate=true

or

mvn surefire-report:report-only -DlinkXRef=false -Daggregate=true

if you've already build the project with surefire plugin.

This will generate surefire-report.html in your root target directory where you can find time stats for each module and each test suite.

Casting