Hockey vs. Testflight

Question

We're trying to decide between these two services for an upcoming beta. The new TestFlight looks much improved but we are still concerned about 3 things:

1. The user experience for testers (iTunes reviews of the TestFlight app imply this can be confusing)
2. The possibility of rejection or delay by apple when reviewing our beta releases.
3. Limited to IOS8

Hockey, on the other hand, seems to have a better tester UX. It supports multiple OS versions. And of course, no review is required.

The downside with Hockey seems to be a limit of 100 devices: http://support.hockeyapp.net/kb/client-integration-ios-mac-os-x/adding-new-devices-to-your-provisioning-profile

Here's our comparison grid: https://docs.google.com/spreadsheets/d/1CuYlsLsZPW-79hEre7jLppfwQpG4WmW3fDvHvIJ86wY/edit#gid=0

Would appreciate any feedback.

Polychaete

Answer 1

We've been beta testing our App via both HockeyApp and Apple's new Testflight within the last few weeks as well. I would recommend using both in parallel and seeing the pros and cons for yourself and from there you can choose one over the other. Here are our insights from the last few weeks:

HockeyApp Pros:

• no need for an approval process
• quick upload of new versions while maintaining access to old versions

HockeyApp Cons:

• requires a bit more work to initially set up each beta tester (need to have their device registered with HockeyApp so that you can register their UDID with a provisioning profile and then a new archived build needs to be uploaded to HockeyApp containing that updated provisioning profile)
• Only 100 tester slots (although unless you're really close to app store submission, you probably don't even need 100 spots)

TestFlight Pros:

• Don't have to deal with UDID or provisioning profiles, strictly emails only
• 1000 10000 slots
• Now includes sharing of public links, so that other testers outside your organisation can use to download your beta builds.
• Beta Approval process is pretty short (1.5 days for us)

TestFlight Cons:

• Need iOS 8 to install TestFlight
• Only 25 internal tester slots
• Only can have one active build at a time Now supports having multiple active builds

Chromatin

Thanks very helpful. One thing I don't understand about Hockey: Does it automatically collect and upload UDIDs from testers or do they have to manually pull UDIDs via iTunes? – Polychaete

It helps you collect UDIDs and shows the registered devices of your testers and the UDIDs present in your provisioning profiles. You have to add the UDIDs to the provisioning profiles yourself on Apples developer portal since Apple doesn't allow developers to hand over their passwords to others and without the developers password this cannot be automated. – Parchment

Does it take 1.5 days each time you release a new build? Or just the very first time you want to register for using TestFlight? – Ekg