Can I integrate Jenkins and XCTest

Question

Is it possible to use Jenkins with XCTest?

There's a ruby script (OCUnit2JUnit -- https://github.com/ciryon/OCUnit2JUnit) that parses OCUnit output and creates XML files that Jenkins can parse.

Is there something equivalent to handle the new XCTest output?

Right now, using Apple's continuous integration is not a possibility.

Extrinsic

XCTool has XCTest support, and an excellent history in regards to Jenkins integration. – Dastard

Just in case someone is reviewing this in 2018, you should consider using fastlane with the multi_scan action from the test_center plugin to automate your tests and get junit reports for the Jenkins test publisher. – Bareback

Answer 1

Two options:

1) pipe your xcodebuild output into xcpretty and use their Junit formatter. Then Jenkins can publish that.

B) use xctool instead of xcodebuild. Xctool has built in Junit reporter you can specify for tests.

We've done both and they both work fine on Jenkins.

Ibson

@piiluke i am not able to launch the simulator using jenkins. How did you do that ? – Lope

@HarishGupta command line xcodebuild has everything you need to run tests. If you just want to launch simulator itself (without any context) for some reason just run the app either using a shell script or AppleScript. – Ibson

Thanks for the reply !! My test cases are UI test case which required simulator to be running. So when jenkins execute the test case, it needs to start simulator first. – Lope

@HarishGupta Xcode starts the simulator for you with xcodebuild test. You do not need to start the simulator. If you are not using the xcodebuild test and you want to run your own custom process then you can use xcrun simctl to control the simulator but that's a whole different question. Also, if you're using xcodebuild test you may need to connect to jenkins slave using JNLP instead of SSH as earlier version of Xcode had problems launching UI tests when agent was over SSH. Also a different question entirely. – Ibson

Answer 2

You need to add a shell script build phase.

First add the ocunit2junit gem to the build machine:

sudo gem install ocunit2junit

Then add the build phase shell script and make sure you're in the project directory when you execute the script:

xcodebuild -workspace yourWorkSpace.xcworkspace -scheme YourTestsScheme -configuration Debug clean test 2>&1 | ocunit2junit

Then make sure to add the publish JUnit test results post build action to the Jenkins job configured to find the output for ocunit2junit: **/test-reports/*.xml

Vaudevillian

Answer 3

for me it doesn't work. also on the Github Welcome Page of the Projekt XCTest is not mentioned to be supported. So it appears that this is not supported

Sandblast

Answer 4

I 've written a tool that parses the TestSummaries plist file from Logs/Test folder and generates the JUnit report xml file: (https://github.com/nacuteodor/ProcessTestSummaries). The generated report should be more accurate than xcpretty.

Wilds

Answer 5

This is what I run on my Jenkins project after installing ocunit2junit on my build machine:

xcodebuild test -scheme <my_scheme> -configuration Debug -sdk iphonesimulator7.0 -destination OS=7.0,name="iPhone Retina (4-inch)" | ocunit2junit

I then added a Publish JUnit test result report as a post build action. Was a bit flakey at first (only creating the xml files when there was no post-build action), but after initiating the build directly on my build machine, it works.

Constitution

Hmm, apparently ocunit2junit supports XCTest as well now? https://mcmap.net/q/1272195/-xctest-output-to-a-file – Constitution