OCUnit Application Test with Simulator in terminal

Question

Is it possible to start an application test that runs in the simulator with a terminal command(s)?

Thanks

Bumpkin

Answer 1

Yes, I got it to work. My solution is somehow rough and might not be suitable in every case.

Disclaimer: This solution requires to edit system files. It works for me, but may mess up XCode's unit testing stack, especially if you do not understand what you are doing.

In /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Tools/RunPlatformUnitTests replace

if [ "${TEST_HOST}" != "" ]; then

    Warning ${LINENO} "Skipping tests; the iPhoneSimulator platform does not currently support application-hosted tests (TEST_HOST set)."

else

with

if [ "${TEST_HOST}" != "" ]; then

    mkdir -p "${BUILT_PRODUCTS_DIR}/Documents"
    mkdir -p "${BUILT_PRODUCTS_DIR}/Library/Caches"
    mkdir -p "${BUILT_PRODUCTS_DIR}/Library/Preferences"
    mkdir -p "${BUILT_PRODUCTS_DIR}/tmp"

    export CFFIXED_USER_HOME="${BUILT_PRODUCTS_DIR}/"

    RunTestsForApplication "${TEST_HOST}" "${TEST_BUNDLE_PATH}"
else

You may move the fixed user home to a different location, but I think you would need to move the .app and .octest bundles along.

Add -RegisterForSystemEvents to the OTHER_TEST_FLAGS build setting of your test bundle.

Make sure your test bundle contains a run script build phase with the contents

# Run the unit tests in this test bundle.
"${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"

Create a new scheme for your tests.

You should be able to run the tests from the command line using the standard xcodebuild:

xcodebuild -workspace $(WORKSPACE_NAME).xcworkspace -scheme $(TEST_SCHEME) -configuration debug -sdk iphonesimulator

The simulator must not be running, at the time you what to run the tests.

I hope this information is complete, if something doesn't work as expected please ask.

Nervous

I'll try it out. Will let you know. – Bumpkin

I tried it out. I get the following Result: 'xcodebuild: error: Failed to build workspace project with scheme TestScheme. Reason: Scheme "TestScheme" is not configured for launching.'. Did you encounter this problem? In Xcode itself i can only execute the tests with 'cmd-u' and not with 'cmd-r' (The run command). – Bumpkin

Already got it. Edited the wrong 'RunPlatformUnitTests' file. Can run tests with UI elements and they pass. I do not see the simulator, is the simulator starting in your 'version'? Or are the application tests just running without the simulator? – Bumpkin

They are running in the simulator, but the simulator UI and App-Icon is not shown. – Nervous

Hee, one more thing, is there a way to run single test suite with this option? – Bumpkin

You may add the -SenTest {someTest} argument to $(OTHER_TEST_FLAGS) to run only some tests/suites. Check SenTesting docs for syntax. – Nervous

Answer 2

You can ensure that the Simulator isn't running with this:

osascript -e 'tell app "iPhone Simulator" to quit'

You can determine if the Simulator is active with this:

sh -c 'ps -xaco command | grep "iPhone Simulator"'

Blus

Answer 3

Worked perfectly, thanks! Automated testing is back in action on our Jenkins CI-server! Just had to fix my TEST_HOST=${BUNDLE_LOADER}. Do this if you get errors about "no such file" when running the tests.

Pinelli

Change into what? I leave my TEST_HOST empty, that works for me. – Bumpkin

Answer 4

It seems that with Xcode 4.5GM, running application tests in the simulator is now supported.

Dion

From within Xcode, but not using xcodebuild because the RunPlatformUnitTests script has not been updated. – Scipio

Still doesn't change the fact that application tests, out of the box, do not work when using 'xcodebuild'. They work fine from within Xcode. SDK is irrelevant. – Scipio

You are right. The weird thing is though, that building app test targets do not throw any error or warning message in contrast to earlier SDKs... – Dion