What is your iPhone app testing strategy?

Question

Before submitting to the App Store, it is a good idea to test the App once again precisely. I tend to install my App on a device and give it a friend for a while. Then I take the feedback and start changing my app accordingly.

I'd like to know what your testing strategies are.

Dulcle

Good question. Looking forward to read all the answers! – Wiredraw

I am most concerned about memory testing, which no one seems to have a good handle on. Ideas? – Capsulate

Answer 1

Write a test plan. If you don't have experience with doing that, start with a list of every feature and UI control in the application.

Write down a simple set of steps that could be followed to determine whether or not each feature is working correctly.

Twobyfour

Answer 2

Two major points:

1. Use unit testing. You can use Google Toolbox for Mac for that or just roll your own.

2. User testing, well, it's user testing. A colleague of mine designed a 50-point walkthrough/questionnaire of the app and had some 10-20 people do it -- and then repeated certain parts when we made changes to certain sections.

Eld

Answer 3

You are talking about two different things:

Defect testing and usability testing.

Or I think you may be. The other answers are about defect testing, your approach sounds like usability testing - or a mix of both.

Defect testing is about finding errors in your code. Other people have responded about this:

1. Have unit tests but don't rely on them
2. User testing - firstly by you. Think about your code and what might break it. Throck on controls, paste a zillion lines of text into your editos
3. Have other people who are not familiar with the code use the app
4. use tools like ObjectAlloc and clang to find non-functional defects

In my mind testing is not about tools but the attitude. How hard you look for defects and how honest you are about reporting your own defects.

You should also have a good defect tracking system to keep a handle on them.

Usability testing is more difficult. People do not understand their own thought process when interacting with software.

A good (cheap) approach is to give the softwar eto your friend and then ask him to speak outloud what he is thinking. Then you get statments like "I see this screen but I don't know what to press (you need to add help or cues). I'm not sure if deleted this worked (you need to add feedback). Etc.

You can buy very sophisticated tools to help with user testing but this approach gets a long way there.

Sum

A mix of both. For example, a fried played with my prototype and then said: "Hey, why doesn't this button work?" and then I realized that I had removed the action method for that button, since it was just for debugging purposes. And I forgot to remove the button that sometimes appears. – Dulcle

ObjectAlloc and clang sound interesting. I'll check them out! – Dulcle

Answer 4

At first, I do a functional testing to check if the function of every features work fine. Then, I execute a system testing to check the interaction between functions and perform exploratory testing.

At the end, I make a focus group, which represent the users, to get feedback on its usability. Actually, a focus group will be great if it is held at the beginning of development and the end of development. The first event aims to get feedback on the user interface design and the second one is to get feedback on the real application.

Herrington

Answer 5

For a serious professional app that you plan on making money with -- first you do in-house "white box" alpha testing with Instruments, etc., then you hire a professional quality assurance testing company to do "black box" functional beta testing, and then you hire a professional usability testing company to do user testing on live guinea pigs with video surveillance.

Matrices

Answer 6

In terms of unit testing, I have found that GHUnit and OCMock are two very good tools. Especially GHUnit because it comes with it's own test runner which will run on the device or simulator.

Sideward

Answer 7

I would first install Crashlytics. So that anyone you give the app that has issues, you can see exactly what is going on. Then another thing you could do is install Hockeykit, so you can push new updates just to the beta users. Those are my suggestions.

https://www.crashlytics.com/

https://github.com/TheRealKerni/HockeyKit

Entryway