Running SenTestingKit unit tests in Instruments

Question

I'm developing a database-access library and I'm trying to check for memory leaks using already-written unit tests.

These are logic tests based on SenTestingKit, set up the normal way in Xcode 4.2. I can run them fine using Cmd-U but don't see a way to start them from Instruments, or to invoke Instruments to examine them.

How can I make this work? Do I need to write new cases and build them into an application?

(This question is about leak-checking unit tests but I think is pre-SenTestingKit.)

Selfexpression

Answer 1

This answer begins with SenTest for OS X. If you are concerned with using SenTest for iOS under instruments, skip to the next section.

OS X Unit Tests in Instruments

I realize this is a bit late, but I needed to run instruments on a logic test today and got it working for my environment, so here's what I did:

1. With Instruments running, create a new document.
2. Choose Target from the Target drop down in the document
3. Check the "Traverse Packages" option in the lower-right to allow you to dig inside of the Xcode package
4. Navigate into /Applications/Xcode and find the otest binary inside of /Applications/Xcode.app/Contents/Developer/Tools/ and select it
5. Add OBCJ_DISABLE_GC with a value of YES as an environment variable (don't do this if you're running GC)
6. Add DYLD_FRAMEWORK_PATH and point at the directory containing of your .octest package
7. Add DYLD_LIBRARY_PATH with the same value as DYLD_FRAMEWORK_PATH
8. Set your Working Directory to be the directory where your .xcodeproj lives
9. For arguments, use -SenTest Self and the full path to your .octest package

Some of the environment variables and your working directory may not be essential depending on whether you need additional frameworks to load and whether you have test files in your code directory that require reading. So, you may omit those steps if they don't apply to you.

Once you're done, you'll be able to Save the document and use it to run the tests.

iOS Unit Tests in Instruments

After the OP clarified the requirements, I went about verifying the basic instructions for this with an iOS-simulator-based project.

For iOS, you need a somewhat different configuration, owing to the fact that you're debugging a different executable, and you will need different libraries. So, the DYLD_FRAMEWORK_PATH will include both the path to your .octest package's directory, and that of the Simulator's Frameworks. The latter will be something like this: /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator6.0.sdk/Developer/Library/Frameworks depending on which version of the simulator you are using and we'll refer to it below as SDKPATH

1. With Instruments running, create a new document.
2. Choose Target from the Target drop down in the document
3. Check the "Traverse Packages" option in the lower-right to allow you to dig inside of the Xcode package
4. Navigate into /Applications/Xcode and find the otest binary inside of /Applications/Xcode/Developer/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator6.0.sdk/Developer/usr/bin/ and select it
5. Add DYLD_LIBRARY_PATH and point at the directory containing of your .octest package
6. Add DYLD_ROOT_PATH pointed just at the SDKPATH (above)
7. Add IPHONE_SIMULATOR_ROOT with SDKPATH (above)
8. Add DYLD_FRAMEWORK_PATH and point at the directory containing of your .octest package and the SDKPATH (above), separating them by a colon (:) in the path list
9. Add CFFIXED_USER_HOME with your iPhone simulator home (~/Library/Application Support/iPhone Simulator although I've expanded it before putting it here, as I'm not sure the ~ gets interpreted)
10. Set your Working Directory to be the directory where your .xcodeproj lives
11. For arguments, use -SenTest Self and the full path to your .octest package

Undersurface

I would love to get this working like you have but nothing is happening when I push record. I have selected the otest binary and set the DYLD flags to /Users/compName/Library/Developer/Xcode/DerivedData/projectName/Build/Products/Debug-iphonesimulator (The .octest package lives in that dir), the argument to -SenTest Self /Users/everythingInTheAboveLink, and the Working Directory to the dir containing the .xcodeproj. But... nothing. Suggestions? I'm probably missing something here.. not an instruments expert. Thanks in advance. – Cryptomeria

Edited to include how-to for iOS SenTestKit projects as well. The original post didn't say anything specific about iOS, so I assumed OS X. I have, however, gotten it to work, and here's the revised set of instructions. – Undersurface

Thanks to @noa for pointing out that one of the paths was wrong. Fixed now. – Undersurface

Thank you for putting together these instructions! I'm finally tracking down some memory leaks by following these steps! – Selfexpression

@Undersurface Where is the "Traverse Packages" option? Using Xcode 4.6.3 and try to do that on iOS... – Magellan

@Magellan Traverse Packages is in instruments in the Choose Target window when you are working on OS X apps. Make sure that you have do not have the device selected when doing this, you need to use the target on your Mac or else it won't work. Note the phrase "iOS-simulator-based project" at the top. – Undersurface

Answer 2

Heres another way to do this thats far simpler and allows greater control over what you profile:

Put a breakpoint at the start of setUp and run the test you want to run. When Xcode hits the breakpoint, open Instruments and search for octest under targets. Then you can simply attach to that process. Switch to Xcode and continue running after your breakpoint and instruments should profile all the tests you wanted to run.

Hope this helps.

Periphery

It says "Current Instrumentation Disallows Attach". Trying to use "Leaks" & "Allocations" – Magellan

can you attach normally if you use the "Profile" run setting in Xcode? – Periphery