Since Xcode 9.3 Apple provides xccov, so here is a solution using this, that does not rely on third party dependencies (and does not care if you use xcpretty, -quiet etc.)
Overview of xccovreport and xccov
When your Scheme has test coverage enabled (or when you pass -enableCodeCoverage YES to xcodebuild), an .xccovreport file is created. It contains the coverage percentages that you can see in Xcode UI.
The file is located in:
(for Xcode 9)
/Users/somename/Library/Developer/Xcode/DerivedData/MyApp-airjvkmhmywlmehdusimolqklzri/Logs/Test/E387E6E7-0AE8-4424-AFBA-EF9FX71A7E46.xccovreport
(for Xcode 10)
/Users/somename/Library/Developer/Xcode/DerivedData/MyApp-airjvkmhmywlmehdusimolqklzri/Logs/Test/Test-MyApp-2018.10.12_20-13-43-+0100.xcresult/action.xccovreport
(unless you specify a different folder in xcodebuild via -derivedDataPath)
Note: In Xcode 10, the .xccovreport file location is printed out in console after test finishes, but Xcode 9 does not do this. In any case, relying on this is probably not a good idea, as it might be silenced (e.g. by xcpretty)
The file isn't a plain text and to view it you have to call:
xcrun xccov view <path_to_xccovreport_file>
Which will output the report.
(You can pass --json for JSON report)
What we need to do
We want to be able to parse the file and print out the total percentage (so then GitLab can pick this up and use it in the dashboard).
There are a couple of challenges:
- Firstly we need to find out the file path of the xccovreport which contains random strings (in two places)
- Then we need to parse the file (using some regular expressions) and extract the total percentage.
The script
Here's what I am using (any improvement suggestions welcome, as I am not a bash expert):
#!/bin/bash
#1
# Xcode 10
TEST_LOGS_DIR=`xcodebuild -project MyApp.xcodeproj -showBuildSettings | grep BUILD_DIR | head -1 | perl -pe 's/\s+BUILD_DIR = //' | perl -pe 's/\/Build\/Products/\/Logs\/Test/'`
TEST_RESULTS_DIR=`ls -t $TEST_LOGS_DIR | grep "xcresult" | head -1`
TEST_COV_REPORT_FILENAME=`ls "$TEST_LOGS_DIR/$TEST_RESULTS_DIR/1_Test" | grep "xccovreport"`
TEST_COV_REPORT_FULL_PATH="$TEST_LOGS_DIR/$TEST_RESULTS_DIR/1_Test/$TEST_COV_REPORT_FILENAME"
# Xcode 9
# TEST_LOGS_DIR=`xcodebuild -project MyApp.xcodeproj -showBuildSettings | grep BUILD_DIR | head -1 | perl -pe 's/\s+BUILD_DIR = //' | perl -pe 's/\/Build\/Products/\/Logs\/Test/'`
# TEST_COV_REPORT_FILENAME=`ls $TEST_LOGS_DIR | grep "xccovreport"`
# TEST_COV_REPORT_FULL_PATH="$TEST_LOGS_DIR/$TEST_COV_REPORT_FILENAME"
# More general recursive search. Perhaps less likely to fail on new Xcode versions. Relies on filepaths containing timestamps that sort alphabetically correctly in time
# TEST_LOGS_DIR=`xcodebuild -project MyApp.xcodeproj -showBuildSettings | grep BUILD_DIR | head -1 | perl -pe 's/\s+BUILD_DIR = //' | perl -pe 's/\/Build\/Products/\/Logs\/Test/'`
# TEST_COV_REPORT_FULL_PATH=`find $TEST_LOGS_DIR -name '*.xccovreport' | sort -r | head -1`
#2
TOTAL_XCTEST_COVERAGE=`xcrun xccov view $TEST_COV_REPORT_FULL_PATH | grep '.app' | head -1 | perl -pe 's/.+?(\d+\.\d+%).+/\1/'`
#3
echo "TOTAL_XCTEST_COVERAGE=$TOTAL_XCTEST_COVERAGE"
What it does
#1 - gets the BUILD_DIR and then manipulates the path to get to the xccovreport file. Comment / uncomment the block for your version of Xcode.
#2 - We start with the full report as text. grep '.app' takes only the lines that contain .app. This is guaranteed to exist, because there is a line that reports the total coverage and contains MyApp.app. There will be multiple matches, but the first match will always be the overall total codecov score. So we use head -1 to take that first line of the grep result.
Now we have a line that looks like this:
MyApp.app 12.34% (8/65)
We use a perl regex to take only the “12.34%” part.
#3 - We simply print out the result (together with the variable name to make it easier to locate later in GitLab CI)
How to use
- Replace
MyApp.xcodeproj with your correct value
- Make sure the correct logic is applied in step
#1 (Xcode 9 / Xcode 10 / Generalized recursive search)
- Save to a
printCodeCov.sh file in the root of your project.
- Make the file executable
chmod +x printCodeCov.sh
- In your
.gitlab-ci.yml file, add a line to the script that says - ./printCodeCov.sh
- In your GitLab Pipeline Settings, set the Test coverage parsing to
TOTAL_XCTEST_COVERAGE=(.+)%
Notes
- This does not use the
--json format of xccov. For that version, see below.
- This solution might be fragile, because of multiple assumptions about folder locations and report format
- I use
perl instead of sed because the latter was too difficult (BSD/GNU differences, regex limitations etc).
JSON version
If you'd rather use the JSON report (from xccov), then in the script you need something like this:
# Xcode 10
COV_NUMBER_FRACTION=`xcrun xccov view --json $TEST_COV_REPORT_FULL_PATH | perl -pe 's/.+?\.app.+?"lineCoverage":([^,]+).+/\1/'`
# Xcode 9
# COV_NUMBER_FRACTION=`xcrun xccov view --json $TEST_COV_REPORT_FULL_PATH | perl -pe 's/.+"targets"[^l]+lineCoverage":([^,]+),.+/\1/'`
COV_NUMBER_PERCENTAGE=`bc <<< $COV_NUMBER_FRACTION*100`
TOTAL_XCTEST_COVERAGE=`printf "%0.2f%%\n" $COV_NUMBER_PERCENTAGE`
